I had for some time getting a "brainwave" to write
a post like this. Ramanujan, the strange mathematical genius, said he got his
ideas revealed by 'Lord Krishna" in his dreams. No one ever thought of
questioning the source of his genius, because the pudding has been eaten and only
after many generations later that his discoveries have been properly understood
and rigorously proved. Still some remains and this is not surprising because
when Ramanujan sent his theorems to England to Hardy, and others in the spring
of 1913, Hardy, a well-known mathematician of his time, could only say: I had never
seen anything in the least like them before. A single look at them is enough to
show that they could only be written by a mathematician of the highest class.
They must be true because, if they were not true, no one would have the
imagination to invent them. However, one
mathematician, M. J. M. Hill of University College London, commented
that Ramanujan's papers were riddled with holes and said that although
Ramanujan had a taste for mathematics,
and some ability, he lacked the educational background and foundation
needed to be accepted by mathematicians.
A monumental blunder indeed.
Anyway, my brainwave arose from more humble sources: tea-shops,
gossips, passer-bys, books (old days), the Web (today), and what meager
experience I have got.
There was an expatriate who came as the expert to supervise a survey project for one UN agency in Myanmar
that has been contracted out to my friend. When fieldwork had been completed
and the work reached the analysis stage, this expert said that using weights in
analysis is unnecessary. However, my friend didn't take the expert's advice and
did the analysis using the appropriate weights. Whereby, the expert did a few
tabulations of the data herself, compared the results and remarked they were
not really different.
My friend didn't take the expert's advice because the sample
for the survey had been designed as a two-staged PPS (probability proportional
to size) design and it is standard practice to use the design-weight and
non-response weight in analysis. In my opinion the expert was either (i) being
ignorant of survey data analysis, or (ii) knew so much of it to choose model based inference over the customary
design based inference. If it be the
latter case, I don't know how it could be done in that given situation, except
perhaps for some selected aspects.
Some years before that, I had the opportunity to do a survey
for the same agency for the same type of survey. The sample design was
essentially the same as that of my friend's. I had no problem with the two
experts assigned to the task of looking after the project. The fieldwork went
well. At the editing stage, the preliminary analysis showed that: a significant
number of persons still in school and no longer in school who has completed
primary school were in the category not literate.
It was implausible, so I was about to change them into the literate category when a senior field staff chanced to visit me and
explained me the reasons that they were in fact not literate as reported.
So I was saved from making a blunder. Sometimes, an "expert" needs to
be helped by not so experts to set things right.
When I had submitted the draft report, at least four or five
experts ranging from local to the regional office and their headquarters sent
back their multipage comments. They were mostly on the wording in the report
and on implications in interpretation. In one instant they mistakenly and hastily
assume that I was talking about an interval estimate (where data point as well
as confidence limits of an indicator is given) when I was in realty giving only
a point estimate. I had to explain at great length that I was talking about the
range of individual point estimates for different cross-classes, and cite
similar way of expressing the idea by Macro International the prestigious firm
that runs the DHS (Demographic and Health Survey).
I've heard of misgivings due to pigeonholing before. When I
was in the Pacific, a regional adviser told me that the SIAP (Statistical
Institute for Asia and Pacific) ran a course somewhere with the same
stereotyped content as everywhere, suitable only to an audience with much less
sophisticated knowledge of statistics than those of the participants in that
particular country and particular course.
A real surprising case I knew was the case of a big survey
where the sample design was stratified multistage with the first stage units
drawn with PPS with townships as domains. However at analysis, the first stage
weight used was uniformly 3 which mean that the sampling at first stage was
assumed to be SRS (simple random sampling). This clearly is not justified by
any consideration. Additionally, their data entry design was poor and after
data entry process has been completed, the experts could not consolidate their
data for analysis. So, what they did was took the data back to their country,
worked there, and somehow delivered the results. Lesson: our people need
numeracy. More specifically, they need statistical literacy or some basic
capacity at least to sense if the experts and advisers are doing things right. For
example they should be aware of and honor the principle that confidentiality of
data is important for national interest as well as for individual respondents
and should not have allowed the micro-data to be taken out of the country. On
the other hand, I was surprised with the apparent lack of control of the
international agency that provided the technical assistance and was supposed to
oversee the technical process.
In my opinion, before we expect our people or our bosses to
act responsibly, we need to remove a stumbling block and it is systemic. I
don't know if this still operates, though—in our days the public service had
been governed by three unwritten laws: (i) thou shalt not take initiative, (ii)
thou shalt not meddle, then (iii) thou shalt be rewarded with immunity; better
known as ma-loke, ma-shoke, ma-pyote
in our language.
No comments:
Post a Comment