An important outcome of any library project is determining the
impact on service to library customers. Evaluation reports provide
vital information to other communities contemplating replication of the
project and can serve as a springboard for additional ideas and
The evaluation should focus on:
- the difference in service made by an LSTA funded project
- the impact of the project on the community
- the effect on people rather than the statistical process itself
evaluation of the project is required of every grant recipient within
one month of project completion. Any costs associated with the
evaluation should be included in the grant application budget.
Evaluation Plan: How will you know if it is working? Check the appropriate user group that benefits from the project. Check the evaluation method used.
All applicants must provide the following in the evaluation section of the application:
how you will know objectives or expected results have been met.
Describe what is to be measured. Detail how results will be measured.
Describe target benchmarks used to determine success.
- Describe evaluation methods used - ways used to measure output and outcomes. Describe how the method was determined.
- Describe data collection methodology, i.e., interviews, number gathering, journals.
- Describe the anticipated outcome or results of the project.
- Explain anticipated impact to customer service, on community. Who will benefit?
- Describe plan to share evaluation results, i.e., brochures, professional organizations, journal articles, web page.
all evaluation methods directly to the stated objectives and action
plan. Detail the evaluation plan for the project, including a
description of the questions to be asked, the information needed to
answer the question, the source of the information, and instruments
used to gather that information. Describe person/group responsible for
the design of any instruments and procedures, the time schedule, the
approach to the analysis of the information gathered, and a description
of the reports that will result. Describe criteria to measure success
or failure. Make sure there are adequate funds to carry out the
Present how the public will be informed about what was accomplished and how experiences will be shared with others.
mark an evaluation method. The methods listed are categories listed on
the federal report. The State Library must report the number of grants
in each category.
Specific methods (types of evaluation) listed on LSTA federal report:
study: Relies on testimonials, stories, anecdotes, life changing
impacts. Focuses on the value of the project’s effects and the ends to
which the project’s means were directed.
- Outcome-based: Records
the extent to which a project caused changes in a desired direction in
the target population and in the cooperating organizations. A sound
outcome evaluation demonstrates that changes or progress can be
directly linked to program services or intervention. This type of
evaluation tries to answer the question of what factors exist in the
project’s environment or its history that may influence project conduct
- Tell It Techniques: Specifically for library
evaluations developed as part of a U.S. Department of Education Library
Programs contract to the University of Wisconsin - Madison School of
Library and Information Studies. The State Library has copies of this
planning and evaluating method. Tell It techniques include
questionnaires or surveys; interviews, individual or focus groups;
numbers gathering, counting use; use of existing data for comparison;
observation, structured or unobtrusive; anecdotal reports of library
activity; self-reporting; logs and journals.
- Other: List and
describe any other evaluation methods you use. i.e., performance
measurements; logic model; theory-driven; cost benefit or
cost-effectiveness; expert review; replication of previous projects;
relates to case studies
- Model - Provide project as model for others
When Does Evaluation Occur?
Evaluation occurs throughout the planning cycle, during the project,
and at the end of the project. Planning determines the evaluation
method before the project starts. Pre-project benchmarks (baseline
data) are necessary to measure any increase or decrease in service. The
above types of evaluation methods use the summative process or the
collection of data necessary for judging the ultimate success of the
completed project. The primary purpose is to document the extent to
which the project’s outcomes were achieved. The summative methods used
(those occurring at end of project) document results and justify the
program continuation or additional funding. Summative methods usually
occur after the project ends to measure outcomes, outputs and impacts,
but some information is often collected early in the life of a program
(baseline data, test scores) to determine the relationship of different
Formative methods (continual monitoring) measure how
well the project is being done and helps the project move toward its
objectives. Formative evaluations usually are conducted in the early
stages of a program and address questions about implementation and
ongoing planning. This type of evaluation typically examines process
rather than product. The primary value of formative evaluation is that
it allows adjustments to be made throughout the program, rather than at
CHECKLIST FOR REVIEW
- presents a plan for evaluating accomplishment of objectives; not whether the steps in the action plan are accomplished
- uses measurable terms to describe results
- clearly states criteria to measure success/failure
- clearly states anticipated outcomes or impacts/benefits
- includes formative evaluation methods; process for project monitoring
- describes any evaluation reports to be produced and the method of distributing the information
- clearly states who will be targeted for evaluation and how many persons will take part
- describes an evaluation method that is practical to utilize
What Are Evaluation Categories? Evaluation results can be categorized into three groups: inputs, outputs, and outcomes:
- Inputs - resources available to conduct a program, i.e., staff time, budget, volunteers
Reflects level of institutional support for program. Inputs do not totally reflect the results of the program.
- Outputs- Tend to be quantity measures of what the program did with its
resources (inputs) i.e., number of hits on web page, number of people
using service: used to measure end result ( i.e., increase in use), easier to collect, short-term measures
- Emphasize the consequences of participation in a program, often
called benefits, impact, life changes, long-term measures, or success
stories. Measure changes in knowledge, skills, behaviors, attitudes or
conditions of life status.
All three are needed to give a
complete picture of program efforts. A number of techniques used over
the life of the project is best. Ultimately, outcome measures are the
most important evaluation measures, since these relate directly to the
overall mission of any program. However, documenting changes or
measuring benefits is more difficult. An effective evaluation approach
considers both quantity (counting) and quality (difference or impact).
To receive a complete picture of the project, it is necessary to report
quantity but also quality.