Python for development sector data workflows: cleaning, exploration, modelling, - impact evaluation, and design-based estimates from complex surveys. Part of - OpenStacks.
+The Python you reach for on a development research project: getting messy data into + shape, asking whether a programme worked, and getting the confidence intervals right + when the survey is clustered. Part of OpenStacks.
Weights are the easy half. A national household survey is stratified and clustered, so an
- independence standard error is too small, often by a factor of two, and every interval
- built on it errs in the direction that flatters the result. This implements Taylor
- linearisation, the estimator behind Stata's svy: and R's survey.
Weighting is the half everyone remembers. A national household survey is also clustered,
+ so an interval that ignores that is too narrow, often by half, and it errs in the
+ direction that flatters your result. This does it properly, the same way Stata's
+ svy: and R's survey do.
Read the module →
Rebuilds India's published stunting table by wealth quintile from the raw DHS children's - recode: 35.5 percent nationally, 46.1 in the poorest quintile falling to 22.9 in the - richest. Reproducing a published table is the only cheap proof a pipeline is right end - to end.
+Rebuilds India's published stunting table from the raw NFHS-5 files: 35.5 percent + nationally, 46.1 in the poorest fifth falling to 22.9 in the richest. If your numbers + match the published ones, your pipeline is sound. If they do not, you know before you + publish rather than after.
Read the script →
Checked against R's survey package on the same data, agreeing to
- twelve significant figures. Three behaviours that look like details and are not: subgroups
- are domains rather than subsets, proportions get a logit interval so a small one cannot go
- negative, and intervals use t on clusters minus strata and say which distribution produced
- them.
Checked against R's survey package on the same data, to twelve significant
+ figures. Three things it gets right that a hand-rolled version usually does not: a subgroup
+ keeps the clusters it does not appear in, a small percentage never gets a negative lower
+ bound, and the interval widens correctly when you have few clusters.
These open on GitHub. They are source code rather than pages: Python modules - meant to be read, cloned and run, not browsed like a website.
+These open on GitHub, where the code lives.