CDMRP's Two-Tiered Review Process
The figure above depicts the CDMRP funding cycle. Each program cycle begins with the Congressional appropriation. All successful fiscal year applications are awarded from the single year appropriation and are fully obligated upfront with little to no out-year budget commitments. Each year each program's programmatic panel develops a vision and award mechanisms to address the vision. Awards must be made within 2 years of the initial appropriation.
To ensure both scientific excellence and programmatic relevance, the CDMRP administers a two-tier review process, consisting of scientific peer review and programmatic review, that was recommended by the National Academy of Science's Institute of Medicine.
CDMRP specifically tailors detailed review criteria for each award mechanism, and those criteria are clearly outlined in the widely-available funding opportunities published as program announcements. Reviewers at both tiers follow the published criteria to direct their reviews. An application must be favorably reviewed by both levels of the two-tier review system to be funded.
All review panels are composed of scientists, clinicians, members of the military as applicable, and consumers from advocacy communities. Consumers serve as full voting members and play a major role in maintaining the focus of the respective program on research that is relevant and has the potential to make a significant impact on the community affected.
For some funding opportunities, a third step is included, where a pre-application screening process is used to focus resources, including those of the applicant, the reviewers, and the program, on proposed projects that best meet the intent of the award mechanism. Investigators of favorably reviewed pre-applications are then invited to submit full applications.
The following review process evaluates each full application submitted to the CDMRP:
Tier 1: Peer Review (SRA International, Inc.)
Peer review, the first tier of review, is a criterion-based process in which applications are evaluated on their individual scientific and technical merits in a given discipline or combination of disciplines.
- Each application is assessed by discipline-specific panels for scientific merit and potential impact based on review criteria set forth in the Program Announcement.
- Applications are assigned to two or more scientist reviewers and a consumer reviewer, each of whom provides a written evaluation and preliminary scores based on the review criteria.
- Although some reviewers may participate in similar panels from year to year, standing panels are not used by the CDMRP. Instead, the CDMRP tailors the review panels to fit the specific expertise required by the research program and award mechanism. While applicants are asked to indicate primary and secondary research classification codes upon submission, which may be used when referring applications to certain peer review panels and recruiting peer reviewers, applicants may neither request assignment to a particular panel nor suggest scientific reviewers.
- Applicants do not know the composition or membership of the peer review panel to which their applications are assigned. At the end of each year's peer review cycle the names of the peer reviewers are released by research program; however, panel assignment is not provided.
- The output of the peer review process is a robust and comprehensive evaluation of strengths and weaknesses that serves as the basis for the second tier of review and provides valuable feedback to the applicant.
Tier 2: Programmatic Review (Leidos, Inc.)
Programmatic review, the second tier of review, is a comparison-based process in which applications of high scientific and technical merit from the entire array of disciplines compete in a common pool.
- This level of review is conducted by each program's external advisory panel (official names vary by program and include, but are not limited to, Integration Panel, Joint Programmatic Review Panel, and Steering Committee) whose members each have renowned expertise in their specific research areas.
- Programmatic reviewers do not automatically recommend funding for submissions that are highly scored by scientific peer review panels. The applications that have the highest potential to help achieve the vision and goals of the respective program (programmatic relevance, relative innovation and impact respective to the award mechanism, portfolio balance, adherence to the intent of the mechanism) are selected for funding.
- The consideration of programmatic intent and portfolio balances means that applications are not funded using an established "pay line."
- Following the two-tiered review, scientifically sound applications that best meet the program's interests and goals are recommended for funding. These recommendations are reviewed and approved by the Commanding General, USAMRMC and the Assistant Secretary of Defense for Health Affairs.
To maintain the integrity of its review process, the CDMRP utilizes an Inquiry Review Panel (IRP) to address questions and concerns by applicants regarding either the peer review or programmatic review of their applications. The IRP reviews each inquiry in detail, determines whether factual or procedural errors occurred, and may recommend that an application be re-reviewed by either a peer review or programmatic review panel. The number of applications for which inquiries are received each year is only a very small fraction (average of 0.5% per year) of the applications reviewed.
The CDMRP strives for transparency in its review and management processes, providing information such as advisory board members, peer review panelists, lists of funded applications with abstracts and funding amounts, and research accomplishments with the public on the CDMRP Website (http://cdmrp.army.mil) and in each year's Annual Report.
In addition, the CDMRP is committed to the on-going evaluation of the funding strategies used by the various CDMRP-managed research programs, and to the continuous assessment of the outcomes resulting from that research. Program evaluation is an essential tool that enables the CDMRP to fulfil its mission and vision, ensuring that CDMRP is an excellent steward of public research funds by maximizing the return on our national research investment and speeding progress toward meaningful improvements in health outcomes for Service Members, Veterans, and the American public.