Case-Level Follow-Up, Not Quarterly Dashboards or No Outcome Return: Showing Repair Coordinators What Happened to Their Own Calls

Abstract

At the repair desks of six nonprofit housing associations, 274 coordinators who take tenants' repair calls and decide the priority, trade, and instructions for each job were randomly assigned to one of three ways of learning how their jobs turned out, for nine months. "Case follow-up" coordinators (n=92) received a weekly list of their own jobs closed three to five weeks earlier, showing whether each was fixed on the first visit and whether the tenant called again, and reviewed one case for 10 minutes with a senior coordinator. "Dashboard" coordinators (n=91) received a quarterly report of their own and their team's first-visit fix rates. "No-return" coordinators (n=91) received neither. In months 7 to 9, jobs taken by case-follow-up coordinators drew 14.2 repeat calls per 100 within 30 days, against 17.6 for dashboard coordinators (24% higher) and 18.3 for no-return coordinators (29% higher), from baseline rates of 19.0 to 19.4 in all three groups. On a test of 30 recorded calls with known outcomes, scored by raters blind to condition, case-follow-up coordinators scored 66.3/100, against 55.9 and 54.7. At baseline, coordinators with six or more years on the desk had about the same repeat-call rate as those with under two years (18.9 against 19.4). In month 2, case-follow-up coordinators rated themselves as more discouraged by feedback (3.9/7, against 3.2 and 3.1), a difference that had closed by month 9.

1. Introduction

The case-follow-up design asks coordinators to look at their own mistakes every week, and at month 2 it had left them more discouraged than either alternative. We report that first because it is the reason an organization might decline to adopt it.

A repair coordinator makes several decisions on each call: how urgent the job is, which trade to send, and what to tell the operative about the fault. The result arrives weeks later, on someone else's screen. If the wrong trade was sent, or the description missed the cause, the tenant calls again, and the second call is usually taken by a different coordinator. The first coordinator does not find out. Time on the desk adds calls handled, but without the outcomes it adds little information about which decisions worked.

We tested two ways of returning that information, one at the level of individual cases and one as quarterly aggregates, against the usual arrangement in which nothing came back.

2. Method (Summary)

Six housing associations took part. All 274 coordinators on their repair desks were randomly assigned, stratified by association, to one of three conditions for nine months.

Case-follow-up coordinators (n=92) received, each Friday, a list of their own jobs closed in the period three to five weeks earlier. For each job, the list showed whether the fault was fixed on the first visit, whether the tenant called again about the same fault within 30 days, and the operative's closing note. An analyst at each association compiled the lists from the repairs system, which took about six hours a week. Each coordinator then chose one job from the list and reviewed it for 10 minutes with one of the association's two senior coordinators assigned to the reviews, starting with what the record showed and ending with one thing to try on similar calls. Dashboard coordinators (n=91) received, each quarter, their own first-visit fix rate and their team's. No-return coordinators (n=91) received neither and worked as before. Senior coordinators were not among the 274, and the lists and reviews were not used in performance appraisal.

Repeat calls were taken from the repairs system's records: a repeat call was a further call from the same household about the same fault within 30 days of the original job's closure, attributed to the coordinator who took the original call. Rates were calculated per 100 jobs for the three months before the study and for months 7 to 9. At month 9, coordinators completed a test of 30 recorded calls drawn from the previous year, with outcomes known to the researchers but not to the coordinators. For each, they set the priority, trade, and job instructions. Two raters, blind to condition, scored the responses from 0 to 100 against a rubric written from the known outcomes by senior coordinators who did not lead any reviews, and their scores were averaged. Calls a coordinator had originally taken were excluded from that coordinator's test. Years on the desk were taken from HR records. In months 2 and 9, coordinators (266 and 249 respondents) rated how discouraged feedback on their work had left them (1 = not at all, 7 = very) in a survey.

3. Findings

Of the 274 coordinators, 252 were still on the desk at month 9 (85 case-follow-up, 84 dashboard, and 83 no-return), and the results below are for them.

In the three months before the study, repeat-call rates were 19.2 per 100 jobs for coordinators later assigned to case follow-up, 19.0 for dashboard, and 19.4 for no-return. In months 7 to 9, the rates were 14.2, 17.6, and 18.3. The dashboard rate was 24% higher than the case-follow-up rate and the no-return rate 29% higher. Dashboard and no-return coordinators were close to each other.

On the 30-call test, case-follow-up coordinators scored a mean 66.3/100, against 55.9 for dashboard coordinators (a 19% gap) and 54.7 for no-return coordinators (a 21% gap).

Time on the desk did not predict repeat calls before the study. At baseline, the 71 coordinators with six or more years had a repeat-call rate of 18.9 per 100 jobs, and the 88 with under two years 19.4. In months 7 to 9, the 27 case-follow-up coordinators with under two years had a rate of 14.6, against 18.1 for the 22 no-return coordinators with six or more years (24% higher). The 24 case-follow-up coordinators with six or more years had a rate of 14.3, and the 29 no-return coordinators with under two years 18.6, so case follow-up was followed by lower rates in both tenure bands.

In month 2, case-follow-up coordinators rated their discouragement at 3.9/7, against 3.2/7 for dashboard coordinators (case-follow-up ratings 22% higher) and 3.1/7 for no-return coordinators (26% higher). By month 9, the ratings were 3.1, 3.2, and 3.0.

The two senior coordinators per association who led reviews spent a mean 1.6 hours a week each on them, about 3.2 hours per association.

4. Discussion

The case-level design had a cost in each association: an analyst's six hours a week, about three hours a week of senior-coordinator time, and, at month 2, coordinators who felt worse about their work than their colleagues did. The quarterly dashboard cost less and did not produce the discouragement. It also did not reduce repeat calls by much.

Aggregate figures tell a coordinator how often things go wrong, but not which decisions went wrong. A first-visit fix rate of, say, 78% gives no indication which 22% of jobs to think about. The weekly list did, and the review turned one of those jobs into a specific change. The test results suggest that the change carried over to calls the coordinators had not seen.

The baseline tenure result fits this reading. Six years on a desk that never reported outcomes had produced about the same repeat-call rate as two. After nine months of case follow-up, coordinators with under two years on the desk had lower repeat-call rates than no-return colleagues with six years or more.

Much of the benefit would have gone to tenants. A lower repeat-call rate should mean fewer second visits and less waiting for a fault to be fixed, though we did not measure these directly. The associations paid for the analyst and senior coordinator time, and the coordinators bore the early discouragement.

5. Limitations and Future Directions

Repeat calls were attributed to the original coordinator, but some depended on operatives and contractors, whose work we did not observe. Coordinators knew their condition, and case-follow-up coordinators may have taken more care with calls they expected to see again, apart from anything they learned. Twenty-two coordinators left the desk before month 9 (7, 7, and 8 by condition); we do not know their reasons, and if discouraged case-follow-up coordinators were more likely to leave, the month-9 discouragement ratings may understate the design's lasting cost. Coordinators in all three conditions shared desks, and lessons from the reviews may have reached dashboard and no-return colleagues, which would narrow the gaps reported here. Repeat calls also miss tenants who did not call again about an unresolved fault. The 30-call test used recorded calls, which lack the time pressure of a live desk. The discouragement measure was a single item. The six associations used similar repairs systems, which made the weekly lists cheap to compile; where outcome records are harder to extract, the cost would be higher. A follow-up will test whether coordinators can review cases with each other once senior coordinators withdraw from the reviews.

Editorial Note

This report underwent CfAHC's Rapid Review Protocol, including AI-assisted literature cross-checking and a two-reviewer human sign-off.