360-degree feedback review: 21 indispensable measures

Last week, I did a little research on experiences and criticisms of 360-degree feedback. In that context, I was also immediately offered a conversation with a consultant from England… and as I only noticed shortly before the conversation: She is the author of a book about 360-degree feedback that I read recently. Oh, it’s a small world.

In the small exchange of experiences, she - Elva Ainsworth - provided me with a nice English quote that is well suited to start this text (the rest of the text remains in German, don’t worry):

“Imagine, you are in a room with 3 mirrors around you. For the first time in your life, you can see the back of your head. And it does not look like you expected. That´s what 360º-Feedback could feel like.” — Elva Ainsworth, 360º-Feedback Consultant (elvaainsworth.co.uk)

As we already in  previous articles  have explained, we know ourselves relatively poorly. The reason for this lies among other things  in perception errors . In order to further develop people - whether in public administration, in associations or in controlling - the introduction of a 360-degree feedback tool can therefore be suitable (for a definition of 360º-feedback, see right here) if one takes the criticism of the instrument seriously. 360-degree feedback generally makes it possible to compare self-perceptions and those of others.

The core goal of 360-degree feedback is the further development (mostly) of managers, which manifests itself in a behavior or at least attitude change. 360-degree feedback criticism is often that this core goal is not achieved in many cases. Various factors can be to blame. To name just a few examples: poor communication of the instrument, inaccurate assessments, poorly constructed questionnaires or poor follow-up.

The following article explains 21 factors that - if observed - should drastically increase the probability of success of the measure. Because one thing you don’t want, to stay in Elva’s metaphor: To be frightened at the sight of the back of your head.

Before we explain the success factors for before, during and after the introduction of a 360º feedback tool, a note for those in a hurry: This text primarily addresses 360-degree feedback (or 180º or 270º feedback) for them development of employees - not for their performance evaluation.

Before performing 360º feedback

1. sense: The biggest mistake that companies make before the actual 360-degree feedback is to communicate no sense of the measure when it is introduced (Wimer & Nowack, 1998). It must be clearly defined why it is carried out, for example by linking it to the company’s mission, values or vision or by classifying it as preparation for a transformation phase.

2. Questioning the instrument: It should be reflected whether the introduction of a 360º-feedback tool is really the right way, or whether other methods do not offer greater advantages. An introduction with the wrong motives would be like treating a virus with an antibiotic - it has no effect on the underlying problems and may lead to unwanted side effects (Wimer & Nowack, 1998). For example, targeted coaching and training courses in communication are better suited to further develop individual behaviors or people.

3. Questioning the target group: It should also be clearly thought out and communicated accordingly, who exactly receives the 360-degree feedback. For example, if only one If the manager within a larger group receives the feedback, this could lead to the interpretation that this person is the problem child with whom you want to discharge the debt (Wimer, 2002). That would not be “best practice”.

4. Transparency regarding results: It has to be clearly communicated what exactly happens with the data and who can see it. Since the goal is further development (and not the performance assessment), the management level, for example, should only be able to see results across all participants (if there were at least 5 participants to ensure anonymity)(Carson, 2006). Consultation with the works council should also be held here.
   Does the manager also receive a results report? Or only the feedback recipient himself and he can decide whether to pass it on? So from the beginning, an idea should be established: The results are not used for promotions or dismissals - but cannot be used as an argument to demand a salary increase.

To put it in the words of Shute (2008, p. 175) accept: 

Good feedback is like good murder because it depends on three things: First, the feedback recipient has one motive or a specific reason to participate, second one opportunityto apply the feedback and thirdly the necessary medium or the ability and the resources for it.

Frequently justified criticism of 360-degree feedback: measurement accuracy

Imagine that you don’t have mirrors to look at the back of your head, but others have to draw it. If you want to cut your hair based on this drawing, you will think: Hopefully it is a good draftsman! It is best to get several drawings and average the exact contours in order to get the most precise picture possible.

It is equivalent with the assessment of our behavior in 360º-feedback - only that on the basis of the results not the hair, but the behavior should be shaped. Oops. So you need damn good draftsmen or assessors (in 360-degree feedback: feedback providers) to produce sufficient, so-called accuracy. The fact that this measurement accuracy is often not available is a legitimate criticism of the instrument. Here are a few tips to increase accuracy.

5. Number of feedback providers: There should be a minimum number of people. Relatively many companies report results for a group (e.g. assessment by colleagues) even if only two or fewer of them participated (3D Group, 2009). However, it would be best practice to report results only when at least 3 people per group participated (e.g. 3 colleagues at the same level, 3 at the level below and one manager). Even better, you have, for example, 5 to 15 feedback providers per group.
   The more feedback providers the better? Well, at some point the result is precise enough and the time of the feedback provider should not be used unnecessarily. So, 10 should be enough.

6. Selection of feedback providers - familiarity: How long do you have to know someone to be able to judge him well? Eichinger and Lombardo (2004) have found in a study that people who have known another person for between one and three years were able to most accurately assess the behavior. The second-most accurate were people with a familiarity of over 5 years, the least accurate people with a familiarity of less than one year. According to the researchers, it is therefore advantageous if you know people for a little longer than just from the “first impression” and at the same time not so long that you overgeneralize characteristics of the person.

7. Selection of feedback providers - position: Concerning. the ranking of accuracy, it can also be said that the self-assessment (like here in this article explained) and the customer’s assessment is the least precise, while the manager is the most precise and colleagues on the same level or below are the second most precise (Carson, 2006). Possibly. you can save yourself the assessment of customers and carry out a 270º or 180º feedback.

8. Training of feedback providers: “Best practice” also means training feedback providers when carrying out 360º feedback (Carson, 2006). Training the drawing skills in our hair cutting example would also be worthwhile! For the 360-degree feedback in such a training, for example, the halo effect and the Recency effect that can distort the perceptions of the feedback providers. Possibly. it is also worthwhile to explain the feedback software in more detail.

Does all this sound too complicated to you? You can of course do without individual steps. But with that you probably also give up a higher quality of evaluation and offer critics more points of attack. In general, the 360º-feedback tool will also meet with more acceptance on all sides - from managers to works councils - if the results are reliable.

The questionnaire

In order to increase the quality of the results, attention should also be paid to a high-quality questionnaire

9. Proximity to the company: In this way too, a clear connection to the company or corporate culture should be recognizable, e.g. by evaluating the feedback recipient on the company values itself and recognizing a clear connection to the actual everyday situations (Carson, 2006).
10. Adaptation of the language: In addition, the language of the company or the sub-area should also be used (does the works council have its own terminology?), e.g. “team meeting” vs. “weekly meeting”. If there are English and German-speaking participants, two versions (German & English) should be created if possible, even if everyone “understands” English - if the questionnaires are not understood correctly, their results are unusable (Carson, 2006).
11. Proximity to current goals: It will only be accepted if the feedback can be linked to existing knowledge. At best, it is also linked to certain daily objectives that are relevant for the feedback recipient, the feedback provider, HR and management (Carson, 2006). Are you, for example, in a transformation process and one of the KPIs or values is called “transparent communication”? Then it would be advantageous to measure how transparently communication between managers is perceived.
12. Level of measurement: However, one should measure as concretely as possible, at the behavioral level (Carson, 2006). It is more difficult to derive a measure on the basis of the item “Johanna Müller communicates transparently” than on the basis of the item “Johanna Müller gives an insight into current business developments in the monthly team meeting.”
13. Choice of the answer scale: The answer scale can also have a major impact on the quality of the results, as various studies have shown (Heidemeier & Moser, 2009). A 5- or 7-level scale often provides the most reliable results (source). It has also been found that a scale from “never” to “always” measures less precisely than a scale from “strongly disagree” to “strongly agree” (Kaiser & Kaplan, 2006).
14. Length of the questionnaire: Bad, but true: It has been shown that longer questionnaires measure more accurately than shorter ones (Healy & Rose, 2003) - of course, only if they are designed in a meaningful way in terms of content. This does not necessarily mean that the final report will also be longer: If I ask two questions about communication instead of just one - even if the two questions may sound similar - this will usually make the result more precise. The answer to the question can then be summarized, i.e. averaged, in the results report.

Results

When it comes to the results report and the results of the evaluation, there are a few things to consider in order to take criticism of 360-degree feedback to take the wind out of the sails.

 

15. Length of the result report: At the same time, the result report or report should not be too detailed (Shute, 2008, p. 159). The length should be chosen so that the results can be completely processed even when viewed for the first time. This should also make it easier to define fewer and more targeted measures based on them - which are more likely to lead to a change in behavior in the end.

So here it is equivalent to the employee survey: It is better to carry out surveys in small, digestible and therefore continuous snacks in order to then be able to derive targeted measures. That goes better with the ideal of continuous Improvement, as recommended for example by KAIZEN (Manos, 2007). 

Good feedback is not a one-off event, but a continuous process (Carson, 2006) - that should be clear to everyone involved.

If you would like to try out a continuous 360-degree feedback tool free of charge, e.g. by having your managers assessed every 3 months in a targeted survey based on one of your company values, please contact us regarding our software.

16. Adequate interpretation: First, the feedback takers should know how to interpret the results. This includes, for example, the knowledge that apparently contradicting results between groups need not be wrong. Often one behaves towards the manager differently than your own employees (Bracken & Rose, 2011).
17. Catching emotions: When contacting the results for the first time, it should be ensured that possible emotions are intercepted. The feedback recipient must be clear:

The feedback consists of data and data is neutral. Data cannot make decisions about you. You make decisions about the data (Fleenor, Taylor & Chappelow, 2008).

The follow-up

18. objective: What would a 360º feedback be without a follow-up process? Probably redundant if the goal is a behavior change (Fleenor, Taylor & Chappelow, 2008). In any case, it should be a behavior-promoting objective, for example according to the good old one SMART rule, based on the results.
19. Create accountability: Responsibility must be created as part of the follow-up process (De Villiers, 2013). The feedback takers must be aware that they will be held responsible for the process after the feedback.

Against this background, a “best practice” 360-degree feedback process could take place after the survey (correspondingly it also depicts the Echometer software; Bracken & Rose, 2011): 

1. Feedback takers get your results report and are guided to do so first hypotheses develop how to interpret the results and what could be useful measures.

2. In the second step, the feedback recipient is encouraged to have a conversation with his manager in order to first of all compare perceptions and develop more targeted measures. The involvement of the manager - at least in the definition of the measures - also increases the commitment to really change behavior.

3. Even if many companies unfortunately do not practice this - the feedback recipient should also present the results to his team (source design factor) and be as open as possible to feedback. This has several advantages: 

   1. The appreciation of the feedback providers is expressed
   2. Feedback providers can specify their feedback
   3. You can help define or specify the measures
   4. The commitment is increased to continuously support the feedback recipient in the further course
   5. The ability of the feedback providers to give meaningful and accurate feedback in 360º feedback is practiced and thus better and better over time (Fleenor, Taylor & Chappelow, 2008).

20. Meaningful involvement of coaches: If you decide against a coach and the works council grumbles: meta-analyzes show a medium positive effect of using a coach on behavior change (Smither, London & Reilly, 2005). So it probably depends on the coach whether it pays off. Coaches are particularly useful if negative feedback is to be processed so that, among other things, the results are accepted and, secondly, no harm is done to the self-esteem of the feedback recipient and the focus can be directed to a constructive follow-up (Wimer, 2002).
21. evaluation: Regardless of whether you work in public administration or in the investment industry: The question of whether the 360 ° feedback really has a positive effect on your own company should be asked in any case. At this point, evaluations or performance reviews can be extremely useful (Bracken & Rose, 2011). This can be done, for example, by another survey a few months later.

Conclusion - 360-degree feedback review

360-degree feedback - whether based on software or on paper - has obvious advantages over other methods of changing behavior if you take the criticism of the opponents seriously. Many of the success factors are not apparent at first glance. The article should provide insights into what to consider. Because, as explained at the beginning, nobody wants a crooked haircut. Even if it’s free.

Speaking. I’m just wondering if the back of my head looks acceptable. Thank God it’s not that complicated. Off to the hairdresser.

If you are further interested in 360º feedback, check out our website and feel free to contact us about it!

footnotes

1 360º-Feedback is generally primarily recommended for further development. In a performance appraisal used for the awarding of salary or promotions, it can be assumed that both the self-assessment and the external assessment will be carried out less objectively and will therefore be less precise (Carson, 2006).

References

3D Group (2009). Current practices in 360-degree feedback: A benchmark study of North American companies. 3D Group Technical Report #8326. Berkeley, CA: Data Driven Decisions, Inc.

Bracken, DW, & Rose, DS (2011). When does 360-degree feedback create behavior change? And how would we know it when it does ?. Journal of Business and Psychology, 26(2), 183.

Carson, M. (2006). Saying it like it isn’t: The pros and cons of 360-degree feedback. Business horizons, 49(5), 395-402.

De Villiers, R. (2013). 7 Principles of highly effective managerial feedback: Theory and practice in managerial development interventions. The International Journal of Management Education, 11(2), 66-74.

Eichinger, RW, & Lombardo, MM (2004). Patterns of rater accuracy in 360-degree feedback. Human Resource Planning, 27 (4), 23-25.

Fleenor, JW, Taylor, S., & Chappelow, C. (2008). Leveraging the impact of 360-degree feedback. John Wiley & Sons.

Heidemeier, H., & Moser, K. (2009). Self – other agreement in job performance ratings: A meta-analytic test of a process model. Journal of Applied Psychology, 94 (2), 353.

Healy, MC, & Rose, DS (2003). Validation of a 360-degree feedback instrument against sales: content matters. Paper presented at the 18th annual convention of the Society for Industrial and Organizational Psychology, Orlando, FL.

Kaiser, RB, & Kaplan, RE (2006, May). Are all scales created equal? Paper presented at the Society for Industrial and Organizational Psychology Annual Conference, Dallas, TX.

Manos, A. (2007). The benefits of Kaizen and Kaizen events. Quality progress, 40(2), 47.

Shute, V. J. (2008). Focus on formative feedback. Review of Educational Research, 78, 153–189. http://dx.doi.org/10.3102/0034654307313795.

Smither, JW, London, M., & Reilly, RR (2005). Does performance improve following multisource feedback? A theoretical model, meta-analysis and review of empirical findings. Personnel Psychology, 58, 33-66.

Wimer, S., & Nowack, KM (1998). 13 common mistakes using 360-degree feedback. Training and Development-Alexandria-American Society for Training and Development, 52, 69-82.

Wimer, S. (2002). The dark side of 360-degree feedback: The popular HR intervention has an ugly side. Training & Development, 56 (9), 37-44.