Impact evaluation (IE) is a cross-cutting theme that is relevant in all countries, sectors, and regions. Conducting impact evaluation for strategically selected interventions will help improve relevance and effectiveness of country and regional partnership strategies. Properly implemented impact evaluations will help in the proper selection of a strategic mix of effective interventions. By implementing impact evaluations of development interventions in the Asia-Pacific region, the proposed cluster TA will contribute to the creation of knowledge on what works in development, how it works, and why.
The proposed cluster TA will comprise three sub-projects with each sub-project covering 4 to 5 impact evaluation studies, or a total of 12 to 15 studies distributed over the five regional departments. By the end of the proposed TA, impact evaluation will have been mainstreamed in ADB operations, i.e., it is envisioned that impact evaluation is incorporated in the design of ADB's investment projects, especially in project preparatory technical assistance (PPTA) projects.
Each subproject will include 4 to 5 impact evaluation studies and will comprise two main components, namely, (a) the conduct of IE studies applying various methodologies and survey instruments, and (b) the development of impact evaluation methodologies, approaches, and capacities in both ADB and the DMCs.
(a) Conduct of IE studies
The IE studies component will cover support to conduct of a number of IE studies while ensuring that at least one study will be supported in each of the five regions (Central and West Asia, East Asia, the Pacific, South Asia, and Southeast Asia). Activities under the individual studies may include design of the study methodology and survey instruments; conduct of baseline, mid-line and end-line surveys; and production of the study reports. The proposed TA will endeavor to support the conduct of full IE studies from beginning to end within the timeframe of the proposed TA. However, as learned from the experience of TA 7680, the TA may support key phases of individual studies; for example, as to be approved by the IE Committee, only the design and conduct of the baseline survey may be implemented under the proposed TA, while the mid-line and end-line surveys can be financed through separate initiatives. Through this component, the TA will provide direct technical support by individual and firm consultants, and ERD specialists to individual studies by advising on study design, methodology, and application in all phases of the study. This technical support will also include partnerships with renowned IE research institutes (e.g., International Initiative for Impact Evaluation or the Jameel Lateef Poverty Action Lab) to provide expertise, as well as ground-level support for DMCs in the design of IE frameworks and methodologies (e.g., visits to project sites, pre-testing of survey instruments, and other fieldwork activities).
(b) Development of impact evaluation methodologies, approaches, and capacities
The second component on development of IE-methodologies, approaches and capacities will entail a number of activities including (i) review of IE methodologies and adoption of available techniques and tools for sector and theme specific studies; (ii) conduct of pilot studies to test sector- or theme-specific techniques and tools; (iii) IE awareness raising and capacity building for DMC officials and project staff; (iv) establishment of a central IE data source containing information on project baseline, mid-line and end-line surveys (e.g., questionnaires, sampling methodology, interviewer manuals, field data, etc.); and (v) conduct of knowledge-sharing activities (i.e., regional conferences, where IE studies conducted will be presented; workshops and seminars, where experts will be invited to share methods, practices and lessons on IE) and dissemination of IE studies (publishing IE studies for distribution to ADB and government counterparts, and uploading studies in the ADB website).
|Project Rationale and Linkage to Country/Regional Strategy
Bilateral and multilateral development agencies spend more than $30 billion each year on social development programs, with developing countries spending another $300 billion annually. However, the development effectiveness of this spending is not always clear, and evidence on their impact, if they are available, is scant. In particular, it is not clearly known how much of Asia's recent improvements in poverty reduction and human capital outcomes have been due to development spending and how much are trickle-down effects of the region's rapid economic growth. This lack of information limits the ability of policy makers and development partners to further improve effectiveness of the development spending through proper design of programs and projects.
ADB has supported its developing member countries (DMCs) in designing, financing, and implementing project and policy interventions aimed at fostering economic development and reducing poverty in the region. ADB's economic sector work, cost benefit analysis and past effort at benefit monitoring and evaluation, as well as design and monitoring frameworks and social safeguard requirements, collect information that provide a basis for identifying project performance ex-ante and ex-post. While these are useful exercises in designing projects and monitoring their performance, they do not necessarily identify the actual impacts of ADB's development support. Although it may not be possible to identify specific impacts of development spending at the macro level, it is possible to do so at the project level through the use of rigorous impact evaluation (IE) studies.
Thus, mainstreaming IE in ADB operations is highly beneficial not only for the Bank but also, and more importantly, for DMCs. Gaining accuracy in the evaluation of interventions will permit policymakers in DMCs and ADB to anticipate the expected consequences of these interventions. In this way, IE will allow DMCs and ADB to allocate limited resources more efficiently, guide the design of more effective interventions, and enhance evidence-based policy dialogue.