By Sebastien Gachot, Gines Suarez, Bruno Jaquet and Carmine Paolo de Salvo
Artibonite river, Haiti. Image: IDB.
Water is the most essential element to life on earth, yet too much water all at once can have devastating effects. One country very familiar with the destructive force of water is Haiti, which has a particularly high exposure to natural disasters, but very low capacity for natural disaster risk management.
In 2008, for example, within less than a month, Haiti was hit by three hurricanes and a tropical storm. The damage caused by the resulting floods and erosion was massive: Hurricanes Gustav and Hanna alone caused an estimated US$900 million in damage that year, according to the Haitian government.
In 2009, the IDB financed the construction of key infrastructure as part of the Natural Disaster Mitigation Program I (PMDN I) with the goal of reducing the environmental and socioeconomic vulnerability to natural disasters of the population living in the Artibonite and Cavaillon watersheds.
Once the project was completed the team faced some serious challenge: How can you reliably analyze the impact of natural disaster mitigation measures when data are scarce? How can you know that the benefits are higher than the costs of the project? And how can the Bank use the findings to improve future operations?
What Did Natural Disaster Mitigation Program (PMDN I) Deliver?
PMDN I financed the construction of four downstream riverbank protection works along the Artibonite and Cavaillon Rivers. The powerful current of the Artibonite River caused erosion along the canals built alongside the river to feed the country’s largest irrigation system.
Without these canals, rice and vegetable producers working an area that spans 15,862 hectares would have to rely on rainwater to irrigate their fields – a much less reliable source of water, and a haphazard system that would eventually lead to agricultural losses.
To prevent the collapse of these canals due to erosion, PMDN I financed the construction of three concrete retaining walls that can reach up to nine meters in height, commonly known as “Berlin Walls”.
PMDN I also financed the construction of gabion walls (metal cages filled with rocks) along the downstream river-bank of the Cavaillon watershed to protect the town of Maniche.
The town had experienced significant economic losses, particularly in the agricultural sector, as a result of frequent exposure to flooding.
How Can We Know Whether the Project Was Effective?
After investing US$13.9 million in the infrastructure component of the project, the Haitian government and the IDB wanted to measure the effectiveness of the infrastructure built under PMDN I before contemplating a second phase of the project.
A rigorous evaluation would fulfill two purposes: first, to estimate the benefits of what had been done, and second, to facilitate decision-making regarding the second project phase. However, evaluating infrastructure projects poses three big challenges for evaluators.
First, it is very difficult to find a comparable population exposed to a similar event: that is, to construct a counterfactual.
Second, the infrastructure financed by PMDN I was built to mitigate the effects of natural disasters—random events that cannot be planned or replicated.
Finally, the lack of data intensifies the first two challenges and makes it difficult to establish baselines and follow-ups.
Given these challenges, the PMDN I evaluation team decided to assess the benefits of the infrastructure measures through a cost-benefit analysis in order to evaluate the cost-effectiveness of each infrastructure alternative.
What Methodologies Were Used?
To assess the works to prevent erosion of the canals in the Artibonite watershed, the evaluation team conducted a technical diagnosis of the state of the canals before the project, to establish a baseline before anything new was built. The results showed that the canals would have collapsed within an average duration of five years.
To evaluate the agricultural losses that would result from the destruction of the canals, the team incorporated data in the cost-benefit analysis from a household survey conducted in 2012 for another IDB operation in the same area.
This was crucial for establishing a counterfactual scenario that demonstrated that all three Berlin Walls built by the project helped prevent agricultural losses and generated a positive net present value ranging from US$22 million to US$32 million, where the investment costs on the Berlin Walls ranged from US$1.9 million to US$3.8 million.
The cost-benefit analysis used to evaluate the impact of the gabion walls built in the area of Maniche was more challenging. The team aimed to compare expected annual economic losses caused by floods before the project (ex-ante) with projected losses after the project (ex-post).
A direct comparison of the effects of two similar climate events before and after the project was not possible because – fortunately – no significant climate event occurred in the direct aftermath of the construction work.
Instead, the team created loss curves specifically built for the Cavaillon watershed by establishing the relationship between expected annual economic losses caused by floods and the annual probability of flooding for both pre- and post-project scenarios.
The post-project scenario was created under the assumption that the new infrastructure would provide protection against flood events having a return period (or probability of occurrence) ranging from less than once every year (for small floods which, on average, occur every year or less) to once every ten years (for larger floods which, on average, occur once every ten years).
The main challenge here was the scarcity of concrete data on the effects of climate events on households. To overcome this problem, the project team conducted a survey inquiring about households’ assets, as well as the height of water levels that those surveyed had witnessed in their homes during specific historical flood events. With this information, the team was able to create a map visualizing the reported water levels and the exposed economic assets.
To estimate how often extreme rains occur in Haiti, the team used data collected by the National Water Resource Service (Service National des Ressources d’Eau – SNRE) and the National Center for Meteorology (Centre National de Météorologie – CNM).
Despite avoiding annual economic losses in the Cavaillon watershed, which were estimated to be around US$500,000, the analysis found that the flood mitigation infrastructure yielded a negative net present value of US$2.2 million.
This means that the costs of this portion of the project exceeded the benefits. The construction of the Berlin and gabion walls yielded a positive net present value.
What Did We Learn from the Evaluation?
The two methodologies that were developed and used to measure ex-post the impact of the downstream riverbank protection works built as part of PMDN I were then used ex-ante to identify and objectively prioritize the best opportunities to mitigate flooding and erosion in Haiti in the project’s second phase (PMDN II).
This marks a significant improvement in the design of natural disaster and risk management programs in Haiti, where cost-benefit analyses in the past were typically conducted only after the infrastructure in which to invest had already been selected.
The evaluation of PMDN I thus contributed to the IDB’s goal to improve policymaking in natural disaster risk management in Haiti.
This story is one of the project stories included in the Development Effectiveness Overview, an annual publication that highlights the lessons learned from IDB projects and evaluations.
About the Authors:
Sebastien Gachot is an impact evaluation consultant in the Environment, Rural Development and Disaster Risk Management Division in the IDB office in Port au Prince, Haiti.
Gines Suárez is a disaster risk specialist in the Environment, Rural Development and Disaster Risk Management Division in the IDB office in El Salvador.
Bruno Jacquet is a rural development specialist in the Environment, Rural Development and Disaster Risk Management Division in the IDB office in Port au Prince, Haiti.
Carmine Paolo De Salvo is a rural development specialist in the Environment, Rural Development and Disaster Risk Management Division in the IDB headquarters in Washington, DC.