|1. Project Data:
ICR Review Date Posted:
|Pakistan: Punjab Education Sector Project
Project Costs(US $M)
Loan/Credit (US $M)
Cofinancing (US $M)
Board Approval Date
|Primary education (39%), Secondary education (39%), Public administration- Education (22%)|
|Education for all (60% - P)
financial management and procurement (25% - S)
Gender (15% - S)|
||ICR Review Coordinator:
|Susan Ann Caceres
||Judyth L. Twigg
||Christopher D. Gerrard
|2. Project Objectives and Components:|
According to the Financing Agreement (p. 5), the objectives of the project were to "improve access and equity, and the quality and relevance of education in Punjab." The same objectives were noted in the Project Appraisal Document (pp. vi and 6).
b. Were the project objectives/key associated outcome targets revised during implementation?
There were two components (actual amounts noted reflect total project costs and are not broken down in the ICR by IDA contribution). Disbursements were made against a set of Eligible Expenditure Programs, which were conditioned on the achievement of Disbursement Linked Indicators (DLIs), when 80% of these indicators were achieved in each fiscal year. Eligible Expenditure Programs were teacher salaries, School Council capacity building, District capacity building, School Council recurrent budget, and monitoring (PAD, p. 55). DLIs were: preparation of Medium Term Fiscal Framework; improved quality of teaching; decrease in number of schools with missing facilities; increased transition from primary to middle school, middle to high, and high to higher secondary; continued delivery of free textbooks to public schools; timely provision of stipends to eligible girls enrolled in grades 6-10 in public schools; increase in public financial support for students in low cost private schools; enhanced accountability of teachers; increased capacity of School Councils for effective school management; reduced teacher absenteeism; teacher recruitment on a merit and need basis; transparent examination and assessment system; and good quality information on student learning (PAD, pp. 41-42).
1. Financing of the Punjab Education Sector Reform Program (PESRP) (appraisal, US$ 3,340.0 million of which IDA was to finance US$340.0 million; actual, US$4,361.7 million) contained four subcomponents:
- Enhanced Education Sector Expenditures and Improved Fiduciary Environment was to focus on education sector expenditures and fiduciary reforms to promote improvements in the composition, quality, and efficiency of sector expenditures. The component was to support strengthening of financial management practices to instill a culture of remedial action on audits and build capacity to keep spending in line with sector objectives and reduce internal audits and audit observations. It also was to work to improve financial management at the provincial level so that timely expenditure reports were completed, and implement new rules and practices in procurement of goods and works (e.g. civil works, textbooks, and furniture). There were no Eligible Expenditure Programs for this subcomponent.
- Improved Quality and Relevance of Education was to strengthen the system for measuring student learning. It was intended to improve quality by enhancing teacher professional development and teachers' instructional practices and use of learning materials. It was to design and pilot a provincial teacher accreditation system for teacher certification and licensing. Performance-based incentives were to be provided to well-performing schools, along with additional support to low-performing schools. The Eligible Expenditure Program for this component was the performance-linked incentives for schools.
- Improved Access and Equity was to focus on: (1) decreasing the number of schools with missing facilities; (2) increasing the transition from primary to middle and secondary level; (3) providing demand-side interventions such as timely delivery of free textbooks in Grade 1-10, open competition in textbook publishing and printing, and stipends to eligible girls in Grades 6-10 in fifteen low-literacy districts; (4) increasing public financial support for the private sector to establish low-cost schools in areas with low enrollment through the Punjab Education Foundation; and (5) offering supply-side improvements such as filling of teacher vacancies, reducing missing facilities in schools, and adding grades to schools (i.e. middle and secondary) in rural areas. The Eligible Expenditure Programs for this subcomponent were the girls' stipend program and public-private foundation.
- School Management, Governance & Monitoring was to create an enabling environment for schools to function more effectively by enhancing accountability of teachers to parents and stakeholders at the local level, and by increasing capacity of School Councils for effective school management to reduce teacher absenteeism. It also was to use merit criteria to recruit and place teachers and establish a transparent examination and assessment system to provide quality information on student learning. A streamlined mechanism was to be put in place to transfer recurrent non-salary budgets to all School Councils. Greater capacity for planning and evaluation at the provincial level was to be created by strengthening field and district education monitoring. The autonomous functions of the Punjab Examination Commission were to be strengthened to ensure better quality provincial assessment of students in key subjects. The Eligible Expenditure Programs for this subcomponent were teacher and staff salaries, School Council capacity building, School Council recurrent budget, and District-level capacity building and monitoring.
2. Technical Assistance (appraisal, US$ 10 million of which IDA was to finance US$10 million; actual, US$2.41 million) was to finance generic cost elements of the project through a system of Special Drawing Accounts. These accounts were to be held with the Program Monitoring and Implementation Unit for consultancy services in the areas of expenditure tracking, third party validation and monitoring, training and other capacity-building activities, education strategy development, and budget planning and management. This component was to support the medium-term education sector framework of the Punjab Education Sector Reform Program to strengthen financial management and procurement systems and processes in the sector. It was to fund the use of third-party validation for programs such as delivery of textbooks, teacher recruitment, and environmental enhancements. It was also to provide technical assistance to improve the quality of education, enhance access, and strengthen governance and monitoring of the performance of ongoing programs.
An Additional Financing Agreement was approved by the Board in February 2011. After the 2010 floods in the province, the Government requested support to maintain an uninterrupted flow of funds to the core education program, since there was a financing gap after the Government funded relief and reconstruction work. Two project covenants were dropped (establishment of a teacher accreditation and licensing body) because of lack of clarity in the role of federal accreditation versus the role of the provincial government.
d. Comments on Project Cost, Financing, Borrower Contribution, and Dates
Project Costs: Unused funds (SDR 5.1 million equivalent) were reallocated from the second to the first component, equally distributed across the 10 disbursement-linked indicators, which increased each indicator from US$11.33 million to US$11.59 million. One half of the amount allocated for the disbursement-linked indicator for upgrading of schools was cancelled (US$8.1 million), since the target related to upgrading schools from primary to middle level was not achieved. Project costs were 130% higher than estimated. The ICR does not explain this escalation in project costs.
Financing: The IDA credit for the Sector Wide Approach (SWAp) was estimated to be US$350 million, with US$65.70 million in parallel financing from the UK Department for International Development (DFID). In March 2011 US$50 million Additional Financing was approved of which US$ 48.5 million disbursed and the remaining amount was cancelled. The actual amount financed by IDA was US$ 401.82 million, with an estimated US$40.43 in parallel financing from DFID. The Canadian International Development Agency (CIDA) provided US$17.56 million in joint financing. DFID committed GBP 50 million for 5 years at appraisal. The period of support was shortened to 3.5 years, but the agency increased the total amount of the grant to GBP 80 million, linked to additional targets established by DFID related to missing facilities in five districts (the ICR does not provide the US dollar equivalents for these contribution amounts). In addition to its technical, financial, and advisory support, DFID also provided high-level advisory assistance to the provincial and district leadership, including a special advisor to the Chief Minister of Punjab.
Borrower Contribution: The Borrower provided US$3.9 million, 133% higher than what was estimated. The ICR does not provide an explanation for this higher-than-estimated contribution.
Dates: The project closed as planned without an extension. On June 6, 2012 there was a level two restructuring, which reallocated resources as noted above.
|3. Relevance of Objectives & Design:|
a. Relevance of Objectives:
Even though Punjab is the most prosperous province in the country, school enrollment levels are low. At the time of preparation, only 62% of children were enrolled in primary school, with levels even lower for rural girls and children from poor households. Transition rates from primary to secondary were low, with Net Enrollment Rates respectively of 20% and 11%. Issues in the education sector at appraisal included: (1) insufficient resources allocated to education; (2) inadequate district-level capacity for planning, budgeting, and managing and implementing reforms; (3) limited parent and community oversight and participation in schools; (4) weak support systems for teachers with inadequate provision of materials and aids; (5) patronage-based public school teacher hiring; (6) lack of teacher standards and quality assurance in teacher education and professional development; and (7) inequitable provision of school facilities. All of these factors affected the public education system and caused poor outcomes in terms of access and learning, with independent assessments finding students performing significantly below grade level.
The Bank's current Country Partnership Strategy (FY2010-13 extended to FY14 with the Progress Report) emphasizes improving equitable access to quality education. It also notes that improving the efficiency of public expenditures is critical to achieving development objectives, since modest progress in human development indicators is a function of both inadequate investment and ineffective use of available resources (p. 14). The project's objectives were substantially relevant throughout the life of the project to the education sector's needs and to the Bank's country strategy.
b. Relevance of Design:
To address challenges in the education sector, the Government of Punjab initiated a medium term Education Sector Reform Program in 2003. This Sector Wide Approach was designed to support this Program, specifically to improve governance and accountability in education service delivery and combine education reforms with fiscal and public finance management interventions to improve effectiveness and efficiency of public expenditures. Strengthening of monitoring and evaluation systems was intended to focus the Government's efforts during program implementation. Changes in governance and budgetary/financial management underpinned project design by revising structures, incentives, and practices through a results-based approach, which was also intended to promote teacher accountability. The interventions and DLIs were designed to improve service delivery performance, particularly at the school level. The project promoted regular administration of student assessments, with technical support provided by Development Partners. In response to the draft ICR review, the project team indicated that it views the objectives of improving education quality and improving relevance as the same, and therefore the strategies used to improve education quality (e.g. testing, mentoring of teachers, provision of textbooks, and incentives to low-performing schools) were applicable to improving the relevance of education. Disbursements were to be linked to the achievement of targets (at least 80% had to be met each fiscal year) and were made against a set of selected Eligible Expenditure Programs: teacher and sector staff salaries, girls' stipends, financing of low-cost private sector schools, capacity building for school councils and districts, and monitoring.
The ICR (p. 7) states that a Quality Enhancement Review endorsed design.
However, there were shortcomings. Most importantly, while the DLIs provided a strong results orientation and were linked to the objectives, some were poorly defined or not focused on the needed actions. For example, in the case of student assessment, the DLI did not relate to actions to demonstrate improvements in test design, administration, and analysis. Finally, the ICR reports that the lack of detail in the Technical Assistance Component was a design weakness.
|4. Achievement of Objectives (Efficacy) :|
Punjab was affected by large-scale floods and heavy rains in 2010 that impacted the functioning and performance of the education sector. The floods caused extensive damage to school infrastructure, loss of school and student resources, disruption to school operations, economic losses due to the physical displacement of households, and the diversion of Government attention, efforts, and scarce funds to disaster relief, recovery, and reconstruction activities. The floods most likely adversely impacted the achievement of the project's development objectives.
Improve access: Modest
The project supported low-cost private schools where enrollment increased from approximately 500,000 students to more than 800,000 by the end of the project. This program was expanded from 18 to 29 districts. The ICR (p. 20) also reports that the funding of the program was based on students attaining a minimum level of learning to create incentives for the private schools to be responsive to the needs of students and ensure quality education, but no additional information is provided to be able to gauge the effect of the program. The ICR does not describe whether this program crowded out existing private or public schools.
972 primary schools were upgraded to middle schools, which did not meet the target (1400 schools). 578 middle schools were upgraded to high schools, which exceeded the target (400 schools). School Departments prepared an implementation plan and developed a priority list of 1,400 primary schools and 400 middle schools that were identified for upgrading, which gave priority to girls and rural areas (ICR p. 39). Third party validation showed non-compliance with the priority list of schools, as only 44 of 682 schools were on the list (ICR, p. v).
The primary net enrollment slightly increased from 61.7% in 2007 and 63.6% in 2012, not meeting the downwardly revised target (65.0%). 2011/2012 data were not available at the time of the ICR. See Table 1 for trend data. DFID implemented a household survey in 2011 showing that enrollment had improved, but the extent of this improvement is not provided in the ICR. The ICR provides several explanations for the stagnation in primary enrollments: (1) the floods could have dampened demand for school; (2) the decelerating economy and decreasing commodity prices affected the Government's ability to expand investments in education in keeping with a growing school-age population; (3) delays in implementation of provision of missing school facilities. The combination of the floods and the economic slowdown could have made it harder for poor households to send their children to school; the estimated out-of-pocket monthly expenditure to send a child to public school in Sindh province was PKR 600 (approximately US$6.11), and most families have 3-4 school-age children.
The middle-level net enrollment rate slightly increased from 20.1% in 2007 to 25.4% in 2012, meeting the upwardly revised target (24%). See Table 1 for trend data. Progress could be related to the project interventions to enhance participation at the middle school level (stipends for secondary school girls, school upgrades, and provision of missing school facilities).
The primary completion rate slightly increased from 50.5% in 2007 to 53.5% in 2012, not meeting the target (55%). Household survey data were used to calculate this rate, and therefore it includes both government and private schools. 2011/2012 household survey data were not available at the time of the ICR. The ICR reports that its interventions to improve quality (merit placement of teachers, capacity building of school councils) promoted greater retention. The ICR (p. 17) also reports that the children who entered school during project implementation have yet to complete the primary school cycle, and so the full effect of the intervention is yet to be realized.
|Primary net enrollment rate||61.7||60.5||61.2||60.7||63.6||65.0|
|Middle level net enrollment rate||20.1||19.0||21.8||22.6||25.4||24.0|
|Primary completion rate||50.5||51.7||52.7||53.6||53.5||55|
PSLM data for 2011-12 were provided by the region after reading IEG's draft review, which were not available at the time of the writing of the ICR.
Improve equity: Modest
The public-private partnership program provided low-cost schools in some hard-to-reach disadvantaged communities. 200 schools were supported under the New School Program. The ICR (p. vii) notes that there were delays in transfer of funds to the Punjab Education Foundation during the project, which created delays in implementation.
The project supported female students by recruiting more female teachers (no numbers provided) and giving female teachers better access to training (no numbers or impact provided).
A Girls Stipend program was implemented, and in FY2011-2012 the program covered 385,000 girls in 16 districts. This program targeted the worst-off and underserved areas of the province (ICR, p. 11). Based on third party validation, 93% of eligible girls received the stipends each year. The delivery of the stipends took more than five months, while it was planned to take less than 3 months (ICR, p. 29).
The female primary net enrollment rate increased from 59% at baseline to 62.2% in FY12. See Table 2 for trend data. The ICR does not provide middle level net enrollment data disaggregated by gender.
During the life of the project, the ratio of female-male net enrollment for both primary and middle education in rural Punjab increased respectively from 89% in 2006/2007 to 93% in 2011/2012 (which met the target of 92%) and from 74% in 2006/2007 to 78% in 2011/2012 (which did not meet the target of 86%). See Table 2 for trend data. However, the ICR indicates that ratio-based indicators should be avoided, since the estimator is sensitive to slight changes in the numerator and the denominator. Changes in the indicators between 2007-2008 were related to changes in male enrollment, as the ICR suggests that boys withdrew because of the higher forgone earnings relative to girls.
Over the project period, the growth rate in female middle-level net enrollment (rural) was almost twice that for males (4% and 2%, respectively). The female primary growth rate is not provided, and the ICR does not provide middle-level data disaggregated by gender to be able to compare the trends for both boys and girls to determine if the growth rate for girls was sufficient, given their lower enrollment.
|Male primary net enrollment rate||63.6||62.3||63.4||62.4||65.0||65% (overall)|
|Female primary net enrollment rate||59.4||58.8||58.7||59.0||62.2||65% (overall)|
|Female-male ratio net primary enrollment, rural||89.7||91.2||89.3||92.5||93.5||92.0|
|Female-male ratio net middle enrollment rate, rural||74.5||91.6||82.4||82.7||78.2||86.0|
PSLM data for 2011-12 were provided by the region after reading IEG's draft review, which were not available at the time of the writing of the ICR.
Findings from the evaluation of the girls' stipend program in Punjab showed a positive impact of the program on female enrollment. Beneficiary adolescent girls were more likely to progress through and complete middle school; girls eligible for benefits increased their rates of matriculation and completion of high school; there was suggestive evidence that participating girls delayed marriage and had fewer births by the time they reached 19 years of age; and positive trends in enrollment were also observed for boys in these districts. The delivery of the stipends took more than five months, while it was planned to take less than 3 months (ICR, p. 29). The ICR (p. 17) reports that girls' secondary enrollments may have been protected by the stipend program, as secondary male enrollment slightly declined.
In response to the draft review, the region provided enrollment data disaggregated by rural and lowest income quintile. These data indicate modest changes in enrollment between 2007 and 2011.
|Indicator||2007-08 (%)||2011-12 (%)|
|NER, primary level (age 5-9 years) Rural Girls||53.8||58.4|
|NER, primary level (age 5-9 years) Rural||56.4||60.7|
|NER, primary level (age 5-9 years) Lowest income quintile||48.0||51.7|
|NER, middle level (age 10-12 years) Rural Girls||15.0||19.3|
|NER, middle level (age 10-12 years), Rural||15.7||22.3|
|NER, middle level (age 10-12 years), Lowest income quintile||11.9||13.3|
Improve the quality and relevance of education: Substantial
The project implemented interventions to reduce the teacher absenteeism rate, since instructional time is an important aspect of student learning. A monthly monitoring system for teachers was established.
Capacity building for school councils was implemented to enhance accountability of teachers to parents, but the ICR provides no evidence to document that this resulted in increased voice for parents or accountability for teachers. Guidelines related to school councils were distributed to all schools. 95% of public primary and middle school councils in 36 districts were provided with capacity development.
54,000 school councils have received annual grants and capacity building support since 2009 (with 97% of councils getting the grants in a timely fashion), but there was a disruption in training due to the floods in 2010, and lack of sustained effort could have decreased positive benefits (ICR, pp. 20- 21). The ICR notes that an impact evaluation of the capacity building program is currently underway.
The project implemented two teacher incentive programs. The "High Achievers Program" rewarded the best-performing schools. This program was discontinued in FY12, as third-party validation showed non-compliance with agreed criteria, and there were fiduciary concerns with the funds transfers. As a result, the "Improvers' Program" was introduced in FY11. In FY12, 1,028 teachers received incentive payments under the Improvers' Program, which rewarded most-improving schools. This program was piloted in 600 schools in three districts (ICR, p. 28). Score cards were distributed in all 600 schools in the 3 pilot districts (ICR, p. 38).
The project supported student diagnostic assessments of learning against grade competencies in Grade 4 and Grade 8 in 2010 and 2011, and examination in Grade 5 and Grade 8. More than 1.4 million Grade 5 and Grade 8 public school children were tested for all three years of the project. In FY11 assessments were also administered in math, social studies, and Urdu in Grade 4, and the analysis was prepared in FY12. The project strengthened the Agency responsible for assessment and improvements were noted in the administration of the examination system; however, there were limited improvements in test design, analysis, and dissemination.
4,754 schools were rehabilitated (water supplies, electricity, boundary walls, and toilets), which exceeded the target (3000 schools). DFID provided financing for rehabilitating an additional 1500 schools. While the disbursement-linked targets were met, there were delays. Third-party validation found concerns with quality of civil works and adherence to the whole school approach (provision of all missing facilities under a single scheme/contract).
1,768 Foundation-Assisted Schools (low-cost private schools) were supported.
Textbooks were delivered to 93% of schools within the first month of the school year, but there are still operational issues and delays at each stage of the process. Monthly monitoring of schools in 2008, 2009, and 2010 showed that between 95% -97% of students had a free textbook (ICR, p. 39). The ICR does not state the number of textbooks provided per grade/subject. There was inadequate use of technical assistance funds to enhance the Punjab Textbook Board to align all textbooks with the new curriculum within stipulated timelines (ICR, p. 29). Six new textbooks were developed. Teacher guides were distributed to all primary schools in the province.
33,299 teachers were recruited based on merit and placed by FY12 in 36 Districts. This entailed the preparation of a recruitment policy, advertisement of postings, display of pre, post, and final merit lists in the Districts, issuance of offer letters, and collection of data (ICR, p. 42). The ICR does not provide evidence demonstrating improved teaching practices or effectiveness.
36 districts provided training and on-site support by the Directorate of Staff Development under the Continuous Professional Development framework, which met the upwardly revised target (36 districts). No information was provided about the effectiveness and impact of the training or number of teachers who received the support.
The teacher accreditation and licensing body was dropped.
Outcomes related to quality:
Teacher absenteeism decreased from 15.1% in 2007 to 10% in 2012, meeting the target (10%). The ICR (p. 20) also indicates that, given the size of the Schools Education Department (over 500,000 employees), mechanisms to establish teacher accountability through the administrative system will remain weak in the medium-term.
An evaluation of the Foundation-Assisted Schools (low-cost private schools) showed that the program increased student achievement (a gain of 0.3-0.5 standard deviations on an unspecified set of indicators) within two years.
The ICR does not report the results from any of the student assessments conducted during the project.
The Project Appraisal Document estimated that the Internal Rate of Return was 9%, based on the expected benefits in terms of lifetime earnings of beneficiaries. The ICR states that this approach did not take into account the impact of institutional strengthening. The ICR calculates the present discount value of expected economic costs, comprising direct and indirect costs (indirect costs: foregone labor earnings of children age 10 or above), to attain school completion. Setting the discount rate equal to 10%, the net Present Discount Value (PDV) is estimated to be PKR 1,964 per child. According to the ICR (p. 18), a positive net PDV is necessary and sufficient for a successful investment operation. However, the calculations in the ICR do not include the repercussions of the floods for school enrollment. The region reported in its reply to IEG's draft review that its calculation of age-earning profiles were based on four population groups (males in urban areas, males in rural areas, females in urban areas, and females in rural areas), which is important given the large gender gap in wages presented in the PAD. It also reported that the calculation were actual changes in the probability distribution of completing different school levels using PSLM 2010/2011.
In the region's reply to the draft review, it reported that there were efficiency gains with the competitive bidding process for textbooks. Price ceilings with the increased number of bidders have resulted in cost savings of US$ 4.5 million per year.
While third party validation found concerns with the quality of civil works, the works largely met standards and were noted to be an improvement over the past.
There was a low utilization of the resources devoted to the technical assistance component (24% of planned), which were needed to build capacity. The underutilization of these resources impacted implementation.
Efficiency is rated Substantial.
a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:
* Refers to percent of total project cost for which ERR/FRR was calculated