The Value of Qualitative Feedback

A Cambria client who has engaged us to provide coaching to senior executives recently posed a very good and interesting question: “Do you have anything (research, articles, etc.) that highlights the value of qualitative 360-degree feedback in executive coaching? We have some HR partners that need some ‘convincing’.”

Most organizations using 360-degree feedback have the person’s direct manager, peers, direct reports, and sometimes other stakeholders provide a “rounded” view of the person’s behavior based on a set of “competencies” or “success factors”. Behavior ratings are usually given on a 5-point scale, where a “5” means “outstanding” and a “1” means “needs improvement.”

That kind of assessment is appealing because the numbers give the appearance of rigor, though the ratings are somewhat subjective, and the numbers make it easy to compare people on the same set of dimensions.

Qualitative feedback, on the other hand, is usually developed by asking people for their written comments or by interviewing them with open-ended questions asking about “top strengths” and “top development needs”.

Although qualitative feedback can also be structured around the same set of competencies or success factors as quantitative feedback, qualitative feedback does not lend itself easily to statistical analysis or enable easy comparisons among individuals.

While quantitative 360-degree feedback can provide value by helping people gauge their strengths and development needs in comparison to others against a defined set of desirable leadership behaviors, our experience tells us that it doesn’t provide the information that is most useful in a coaching engagement.

Here are the top reasons why:

Where quantitative feedback provides a predefined set of behavior anchors that have been chosen to represent all the behaviors that are related to a particular competency or success factor, qualitative feedback isn’t constrained in this way and provides a more nuanced view of how a person demonstrates a competency in particular situations.
Where quantitative feedback may provide the “numbers” that show relative strengths and weaknesses, qualitative feedback makes the numbers “real” by giving specific observations that led to the individual competency ratings (i.e., why a person was rated a “4” instead of a “5”).
It is often the case that a person will show strengths and weaknesses on the same competency. Quantitative feedback will mask this in ratings at the midpoint of the scale (i.e., neither a strength nor a weakness), whereas qualitative feedback will surface and clarify these apparent contradictions with examples that illustrate them.
Someone’s manager, peers, and direct reports may give similar ratings even though they have different experiences with the person to whom they are giving feedback, based on their relationship with that person. Those experiences, and their development implications, are only available using a qualitative feedback process.
Quantitative feedback is less sensitive to differences among top performers who get high scores across the board. Qualitative feedback reveals the differences among top performers who score “5”s through describing the specific situations in which a strength was demonstrated as well as how it was demonstrated – there are different ways to get to a “5”.
Quantitative feedback is an abstraction from actual observations across different situations and circumstances, which requires the feedback provider to generalize these experiences into a single numerical rating. Qualitative feedback provides the direct observations that add up to the ratings, and focuses on what each stakeholder views as the best evidence supporting the designation of a competency as a strength or a weakness.
A major reason for gathering qualitative feedback is to find out what other people want to tell the person; it is the beginning of a two-way conversation. Quantitative feedback, on the other hand, requires a deeper conversation to surface the meaning behind the numbers before it can be as useful to the person receiving feedback.

All of these reasons point to the importance of context in gathering and reporting 360-degree feedback.

Effective coaching requires feedback in the context of situations and challenges in which competencies are needed, and the heart of this is an informed conversation supported by specific observations of the person in real business situations.

Qualitative feedback shows how a person deals with people and situations, where he/she is most effective and what he/she could be doing differently, more consistently, or better. Quantitative 360 feedback misses these important elements, whereas qualitative feedback can provide all of them.

Last and most important, qualitative feedback has much more impact on leaders. The power of people’s verbatim comments on a leader’s behavior — both what they appreciate about the person and where that person can improve — sends a message with both a direction and an emotional impact that the numbers do not provide. This, more than anything else I can think of, is better than any research studies or other articles can show in support of the value of qualitative 360-degree feedback.