We conducted a retrospective quasi-experimental study using previously collected cross-sectional data from the 2005 and 2010 Rwanda Demographic and Health Surveys (DHS), 2010 DHS oversampled data, monthly rainfall data collected from meteorological stations over the same period, and modelled output of long-term rainfall averages, soil moisture, and rain water run-off. Difference-in-difference models were used.
Rainfall factors confounded the PIH intervention impact evaluation. When we adjusted our estimates of programme effect by controlling for a variety of rainfall variables, several effectiveness estimates changed by 10% or more. The analyses that did not adjust for rainfall-related variables underestimated the intervention effect on the prevalence of ARI by 14.3%, fever by 52.4% and stunting by 10.2%. Conversely, the unadjusted analysis overestimated the intervention\'s effect on diarrhoea by 56.5% and wasting by 80%.
Rainfall-related patterns have a quantifiable effect on programme evaluation results and highlighted the importance and complexity of controlling for contextual factors in quasi-experimental design evaluations.