First-rate multi-rater feedback
By Allan H. Church
Along with death and taxes, most employees at large firms can add evaluations to the short list of events they cannot avoid. Most employees must submit to having others evaluate their work through some institutionalized process. Typically, the process involves an employee and his or her supervisor.
Some organizations use multi-rater feedback systems, which involve not only a person’s supervisor but also others with whom the person works closely. Usually, organizations incorporate this type of system–also known as a 360-degree feedback system–into their management development and leadership-development programs.
A multi-rater feedback system relies on information collected from a manager’s direct reports and peers as well as his or her supervisors. Some firms widen the circle and collect feedback from subjects’ external customers as well; at my firm, we call this a 450-degree feedback system.
Some firms incorporate multi-rater feedback into their formal performance appraisals. But before a firm can use a developmental feedback system as a tool for managing and rewarding performance, it must resolve issues of confidentiality, validity, usefulness, and effectiveness.
Specifically, before firms incorporate multi-rater feedback into their formal appraisal systems, they must:
- Ensure that people willingly provide honest feedback
- Ensure that the data remain confidential
- Verify that the data are accurate
- Ensure that subjects can use the data to improve their performance
- Determine how the system will affect their organizations overall.
How does an organization persuade managers and employees to invest time and energy in using 360-degree feedback as an assessment tool?
It starts at the top. Often, high-level managers make the decision to implement a multi-rater system. To build trust and win participation among the lower ranks, senior-level executives must visibly and enthusiastically support the process. They also should be among the first to serve as the loci of the 360-degree feedback process. Another way that an organization can encourage managers to embrace the feedback process is to tie participation to a bonus system.
How can firms persuade others to embrace the process and provide feedback? Often, co-workers fear that the person they assess will see their individual comments and observations. Direct reports, in particular, might worry about confidentiality because they fear retaliation or dismissal.
Peers also tend to mistrust the process, probably because they want to preserve their working relationships with their co-workers. In extended systems, clients might refuse to participate if they fear the disruption of their relationships with the managers they are asked to assess.
Because most people report to only one person, a supervisor cannot remain anonymous in the process unless his or her feedback is incorporated into a general group such as “others.” Some organizations take this tack, but doing so runs counter to the premise of presenting multiple but distinct perspectives.
Organizations can both protect respondents’ confidentiality and create a safe atmosphere that encourages honest feedback, by taking the following steps:
- Distribute the feedback instrument in sealed packets.
- Use optical-scan codes to identify individual raters’ roles (peers, direct reports, or clients).
- Include an addressed envelope in each packet so that raters can mail their completed questionnaires directly to the person or group that will organize the data.
- Combine feedback collected from the same type of source; in other words, compile the responses of direct reports, of peers, and of clients.
- Do not include feedback from peers, direct reports, or clients unless at least three representatives of the group have responded. (Generally speaking, the greater the number of respondents, the more accurate the feedback.)
- Do not provide the manager being assessed with the individual responses.
Some organizations prefer to hire outside consultants to administer multi-rater feedback systems. The added distance between the implementer and the organization can help employees feel comfortable and confident that the system is confidential.
Choose instruments carefully
The accuracy of 360-degree feedback systems depends on the instruments used. You’ll find many different types of 360-degree feedback questionnaires on the market. If you use an off-the-shelf instrument, evaluate it carefully and ask yourself the following questions:
- Are all the questions clear?
- Could raters easily answer the questions, based on their typical interactions with the subject?
- Do the questions encourage objectivity by focusing on observed behaviors (for example, “Does the person demonstrate commitment and persistence in achieving goals?”)?
- Are the questions too vague (“Does the person consider alternatives?”) or too specific (“Does the person involve me in all decisions affecting my work?”)
Also, consider having an instrument customized for your organization. Every organization has its own culture, mission, reward systems, structure, and management practices. Similarly, an organization might need its own set of criteria for rating performance.
Making feedback meaningful
Collecting high-quality data is essential, but even the best instrument won’t work if the subject cannot understand and act on the feedback.
Many people have had so little experience with individualized feedback that receiving feedback from even one other person, let alone a group, can foster anxiety.
To help employees integrate feedback into performance-improvement plans, the process must include the following elements:
Self-evaluation. The person assesses his or her own workplace behaviors and skills and then compares his or her own assessment with feedback culled from others in the multi-rater process. This process enables an individual to integrate feedback into his or her self-image.
Integrating feedback. Organizations must provide managers with enough time and a structure for reflecting on the feedback and developing goals for improvement.
To help managers integrate the feedback, some organizations use self-study guides. Others use group processes guided by an internal facilitator. Still others prefer off-site programs staffed with specialists in organizational change, career development, counseling, and assessment. Off-site programs usually provide accommodations for managers in a campus-like setting. Such programs typically run from two to 10 days. They provide intensive and personal attention to participants, including private counseling.
Follow up. Follow-up is essential. After the feedback and assessment process has concluded, the person who has been assessed should meet with the various constituencies in separate groups. During the meeting, participants can discuss the positive and difficult aspects of the process.
What effect will such an assessment system have on an organization? The effects of a 360-degree feedback system–like most organization development and human resource development interventions–are not easy to quantify. Some research does suggest a link between managerial insights gained through feedback and improved performance at the individual and collective levels.
New systems are bound to have bugs, but if organizations use sound instruments and stick with the process, they are likely to find that multi-rater assessments enhance their performance-management systems.
Allan Church is a principal with W. Warner Burke Associates, 201 Wolfs Lane, Pelham, NY 10803; phone 914/738-0080; fax 914/738-1059.
© 1995 from Training and Development by Allan Church
Posted with permission of American Society for Training and Development
Call us! We're here to help1-888-467-8575