Friday, November 21, 2014

Blunder of experts and numeracy of our bosses



I had for some time getting a "brainwave" to write a post like this. Ramanujan, the strange mathematical genius, said he got his ideas revealed by 'Lord Krishna" in his dreams. No one ever thought of questioning the source of his genius, because the pudding has been eaten and only after many generations later that his discoveries have been properly understood and rigorously proved. Still some remains and this is not surprising because when Ramanujan sent his theorems to England to Hardy, and others in the spring of 1913, Hardy, a well-known mathematician of his time, could only say: I had never seen anything in the least like them before. A single look at them is enough to show that they could only be written by a mathematician of the highest class. They must be true because, if they were not true, no one would have the imagination to invent them. However, one mathematician, M. J. M. Hill of University College London, commented that Ramanujan's papers were riddled with holes and said that although Ramanujan had a taste for mathematics, and some ability, he lacked the educational background and foundation needed to be accepted by mathematicians.  A monumental blunder indeed.

Anyway, my brainwave arose from more humble sources: tea-shops, gossips, passer-bys, books (old days), the Web (today), and what meager experience I have got.

There was an expatriate who came as the expert to supervise a survey project for one UN agency in Myanmar that has been contracted out to my friend. When fieldwork had been completed and the work reached the analysis stage, this expert said that using weights in analysis is unnecessary. However, my friend didn't take the expert's advice and did the analysis using the appropriate weights. Whereby, the expert did a few tabulations of the data herself, compared the results and remarked they were not really different.

My friend didn't take the expert's advice because the sample for the survey had been designed as a two-staged PPS (probability proportional to size) design and it is standard practice to use the design-weight and non-response weight in analysis. In my opinion the expert was either (i) being ignorant of survey data analysis, or (ii) knew so much of it to choose model based inference over the customary design based inference. If it be the latter case, I don't know how it could be done in that given situation, except perhaps for some selected aspects.

Some years before that, I had the opportunity to do a survey for the same agency for the same type of survey. The sample design was essentially the same as that of my friend's. I had no problem with the two experts assigned to the task of looking after the project. The fieldwork went well. At the editing stage, the preliminary analysis showed that: a significant number of persons still in school and no longer in school who has completed primary school were in the category not literate. It was implausible, so I was about to change them into the literate category when a senior field staff chanced to visit me and explained me the reasons that they were in fact not literate as reported. So I was saved from making a blunder. Sometimes, an "expert" needs to be helped by not so experts to set things right.

When I had submitted the draft report, at least four or five experts ranging from local to the regional office and their headquarters sent back their multipage comments. They were mostly on the wording in the report and on implications in interpretation. In one instant they mistakenly and hastily assume that I was talking about an interval estimate (where data point as well as confidence limits of an indicator is given) when I was in realty giving only a point estimate. I had to explain at great length that I was talking about the range of individual point estimates for different cross-classes, and cite similar way of expressing the idea by Macro International the prestigious firm that runs the DHS (Demographic and Health Survey). 

I've heard of misgivings due to pigeonholing before. When I was in the Pacific, a regional adviser told me that the SIAP (Statistical Institute for Asia and Pacific) ran a course somewhere with the same stereotyped content as everywhere, suitable only to an audience with much less sophisticated knowledge of statistics than those of the participants in that particular country and particular course.

A real surprising case I knew was the case of a big survey where the sample design was stratified multistage with the first stage units drawn with PPS with townships as domains. However at analysis, the first stage weight used was uniformly 3 which mean that the sampling at first stage was assumed to be SRS (simple random sampling). This clearly is not justified by any consideration. Additionally, their data entry design was poor and after data entry process has been completed, the experts could not consolidate their data for analysis. So, what they did was took the data back to their country, worked there, and somehow delivered the results. Lesson: our people need numeracy. More specifically, they need statistical literacy or some basic capacity at least to sense if the experts and advisers are doing things right. For example they should be aware of and honor the principle that confidentiality of data is important for national interest as well as for individual respondents and should not have allowed the micro-data to be taken out of the country. On the other hand, I was surprised with the apparent lack of control of the international agency that provided the technical assistance and was supposed to oversee the technical process.

In my opinion, before we expect our people or our bosses to act responsibly, we need to remove a stumbling block and it is systemic. I don't know if this still operates, though—in our days the public service had been governed by three unwritten laws: (i) thou shalt not take initiative, (ii) thou shalt not meddle, then (iii) thou shalt be rewarded with immunity; better known as ma-loke, ma-shoke, ma-pyote in our language.



No comments:

Post a Comment