Outcome-Linked Reviews, Not General Check-Ins or Twice-Yearly Appraisal: Manager Feedback and Skill Growth in Roles Where Results Come Back Slowly

Abstract

We cluster-randomized 42 managers, and the 318 employees reporting to them, in five organizations to one of three feedback practices for 12 months. The employees worked in roles where the results of a decision arrive late or unclearly: internal trainers, account coordinators, HR generalists, and grant administrators. "Outcome-linked" managers (14 teams, n=108) agreed with each report on three actions whose results could be traced within a few weeks and held a weekly 15-minute review tracing one recent decision to what followed. "Check-in" managers (14 teams, n=106) held weekly 15-minute meetings on priorities and obstacles without tracing outcomes. "Usual-practice" managers (14 teams, n=104) kept the twice-yearly appraisal. Of the 318 employees, 297 completed both work samples. On a role-specific work sample scored by two raters blind to condition, outcome-linked employees gained a mean 14.6 points on a 100-point scale over 12 months, against 11.9 for check-in employees (a 23% gap) and 11.4 for usual-practice employees (a 28% gap). Years in the role did not predict growth under usual practice: employees with more than five years gained 10.9, against 11.7 for less experienced colleagues. Under outcome-linked reviews, employees with more than five years gained 14.1. Outcome-linked employees also rated their roles as clearer (5.3/7, against 4.4 and 4.2). Outcome-linked managers spent a mean 3.4 hours a week on feedback, 26% more than check-in managers holding meetings of the same nominal length, and rated their strain at 4.3/7, against 3.6 and 3.4.

1. Introduction

Some jobs tell people quickly whether they did well. A spreadsheet formula works or returns an error; a delivery arrives or it does not. Many others give no such signal. An internal trainer rarely learns whether anyone used what was taught, and an HR generalist who handles a difficult case may hear nothing more about it for a year. In jobs like these, experience accumulates without much correction, and it is an open question how much of it turns into skill.

Managers are one possible source of the missing signal, if they can tell which of a report's actions actually led to results and say so specifically. That takes time, and the time comes out of the manager's own work. We compared a review practice built around tracing decisions to outcomes with weekly check-ins of the same length and with the twice-yearly appraisal most of these employees already had, and we also looked at whether years in the role predicted growth at all.

2. Method (Summary)

Five organizations took part, contributing 42 teams in roles where decision outcomes were slow or ambiguous. Each team had one manager. Teams were randomly assigned, stratified by organization, to one of three practices, and all 318 employees in those teams were included.

Outcome-linked managers (14 teams, n=108) attended a half-day session and then, with each report, chose three recurring actions whose results could be traced within a few weeks. For a trainer, one such action was the choice of practice exercise, traced through a short follow-up with participants' supervisors a month later on whether the procedure was in use. Each week, manager and report spent 15 minutes on one recent decision: what was done, what followed, what had worked, and what to change. Check-in managers (14 teams, n=106) attended a half-day session on supportive one-to-one meetings and held weekly 15-minute meetings on priorities and obstacles, with no outcome tracing. Usual-practice managers (14 teams, n=104) kept their organization's twice-yearly appraisal.

Skill was measured at baseline and at 12 months with a 60-minute work sample built for each role, such as designing a training session for a stated problem or resolving a written HR case. Two raters, blind to condition and to timepoint, scored each sample from 0 to 100 against a written rubric, and their scores were averaged. Time in the role was taken from HR records, and the comparison of employees with more than five years against the rest was specified before the study began. At 12 months, the 297 employees who completed both samples rated how clear it was what good work in their role looked like (1 = not at all clear, 7 = completely clear) in a survey. Managers kept a weekly diary of time spent on feedback to reports and, at 12 months, all 42 rated their work strain (1 = none, 7 = severe) in a survey.

3. Findings

Of the 318 employees, 297 completed both work samples (101 outcome-linked, 99 check-in, and 97 usual-practice), and the results below are for those employees. Baseline scores were similar across conditions, at 51.3, 51.8, and 50.9.

Over 12 months, outcome-linked employees gained a mean 14.6 points, check-in employees 11.9, and usual-practice employees 11.4. The outcome-linked gain was 23% larger than the check-in gain and 28% larger than the usual-practice gain. Check-in and usual-practice employees were close to each other, so meeting weekly did not by itself account for the difference.

Time in the role did little. Among usual-practice employees, the 38 with more than five years in their role had a baseline score of 51.6, against 50.5 for the other 59, and over 12 months they gained 10.9 points against 11.7. Among check-in employees, the 37 with more than five years gained 11.5 and the other 62 gained 12.1. Among outcome-linked employees, the 41 with more than five years gained 14.1 and the other 60 gained 14.9, so the longer-serving employees grew under outcome-linked reviews by about as much as their less experienced colleagues did, and 29% more than longer-serving usual-practice employees.

At 12 months, outcome-linked employees rated their roles' expectations at 5.3/7 for clarity, against 4.4/7 for check-in employees (a 20% gap) and 4.2/7 for usual-practice employees (a 26% gap).

Outcome-linked managers recorded a mean 3.4 hours a week on feedback, 26% more than check-in managers (2.7), whose weekly meetings alone took about 1.9 hours with an average of seven to eight reports. Usual-practice managers, with no scheduled meetings, recorded 1.1. Outcome-linked managers rated their strain at 4.3/7, against 3.6/7 for check-in managers (outcome-linked ratings 19% higher) and 3.4/7 for usual-practice managers (26% higher). By month 9, three of the 14 outcome-linked managers had moved their reviews from weekly to fortnightly, citing workload; their teams remained in the outcome-linked analysis.

4. Discussion

Fourteen managers per condition is not many, and a team's manager and its culture are hard to separate, so the gap we report should be read as an estimate with real uncertainty around it. The cost is less uncertain. Outcome-linked reviews took more of the manager's week than a check-in of the same nominal length, because tracing a decision meant finding out what had happened to it first, and the managers who did it reported more strain. Three of them cut back.

All three groups gained roughly 11 points or more, and much of that shared gain probably reflects familiarity with the work-sample format on a second sitting. The comparison of interest is the roughly three points above it that outcome-linked employees gained, along with a clearer idea of what good work in their roles meant. The finding on tenure is the part we would want readers to keep. Longer-serving employees started only about a point above their less experienced colleagues, and under the twice-yearly appraisal they grew slightly less than those colleagues did. That says less about the employees than about the jobs: years in a role that rarely shows its results do not appear to have added much skill. Under outcome-linked reviews, longer-serving employees gained 29% more than longer-serving employees under the appraisal, which suggests that long tenure did not limit how much these employees could still grow.

The check-in arm matters here. Weekly meetings without outcome tracing produced almost the same growth as the appraisal, so meeting frequency alone did not account for the difference, although we cannot separate the content of the outcome-linked reviews from the additional manager time that tracing required. The managers paid for that time themselves, and most of the return went to their reports and to the colleagues those reports serve. Whether the practice lasts probably depends on whether organizations count that time as part of the manager's job.

5. Limitations and Future Directions

Every arm gained on the second work sample, so part of each gain is a retest effect, which we assume was similar across arms. Managers and employees knew their condition, and outcome-linked employees may have prepared for the 12-month work sample more seriously. The work samples were built separately for each role, so scores are comparable within a role but only approximately across roles. Twenty-one employees did not complete both samples, and we do not know how they would have scored. Manager time came from self-kept diaries, which may undercount time spent finding out what happened to a decision. The organizations volunteered, and their managers may have been more willing than most to take on extra feedback work. We did not measure whether the managers' own work suffered, only the time they spent and the strain they reported. A companion report in this issue examines what happens when organizations credit that time in managers' workloads.

Editorial Note

This report underwent CfAHC's Rapid Review Protocol, including AI-assisted literature cross-checking and a two-reviewer human sign-off.