A Game Layer That Steps Back, Not One That Stays or None at All: Phasing Out Rewards in AI-Coached Music Practice and What Happens After They End

Abstract

We randomly assigned 294 adult beginners in a 16-week AI-coached keyboard and guitar program to one of three game-layer designs and followed their logged practice to week 28, twelve weeks after the program's game features closed for everyone. In "fading" accounts (n=99), points, a weekly target meter, and milestone badges ran for six weeks and were then removed one at a time over weeks 7 to 10, while the coach began asking each learner weekly what they wanted to work on next. Fading learners who went a week without practice after the meter's removal in week 9 were offered it back for two weeks. In "persistent" accounts (n=98), the full game layer ran until week 16, and "no-game" accounts (n=97) had none. In weeks 7 to 10, persistent learners practiced 3.9 sessions a week, against 3.4 for fading learners and 3.2 for no-game learners. When the game layer ended at week 16, persistent learners' practice fell from 3.8 to 2.8 sessions a week, a 26% drop, while fading learners' fell from 3.3 to 3.0. In weeks 25 to 28, 45% of fading learners still practiced at least twice every week, against 36% of persistent learners (a 27% gap) and 38% of no-game learners (a 19% gap). At week 16, fading learners rated their interest in playing for its own sake 5.1/7, against 4.3/7 for persistent learners (a 19% gap) and 4.6/7 for no-game learners. Of the 17 fading learners who took the target meter back, 7 were still practicing at least twice a week in weeks 25 to 28.

1. Introduction

A common worry about game elements in coaching is that rewards crowd out interest: a learner paid in points for practicing comes to practice for the points, and stops when they stop. Put that broadly, the worry seems overstated, since plenty of people keep up activities they first took up for a badge or a prize. The narrower version is harder to dismiss. Most game layers in coaching apps are built to run indefinitely, and indefinitely usually means until the program, the subscription, or the app's features change. The moment a learner loses the rewards is chosen by someone else, often all at once, and usually when the learner has come to rely on them.

We asked whether an AI coach could instead withdraw the rewards on purpose, early and gradually, and put something of the learner's own in their place, and whether learners who went through that would keep practicing once no one was awarding points at all.

2. Method (Summary)

Participants were 294 adults who enrolled as beginners in a 16-week keyboard or guitar program offered through a network of community music schools. All used the same AI coach, which listened to practice through the device microphone, gave feedback on timing and accuracy, and suggested exercises. The schools' instructors held a group lesson once a month and were not told learners' designs. At enrollment, learners were randomly assigned to one of three designs, stratified by instrument.

In fading accounts (n=99), the game layer, consisting of points for each session, a meter toward a weekly practice target, and badges for milestones such as a first complete piece, ran in full for weeks 1 to 6. Points were removed in week 7, the target meter in week 9, and badges in week 10. From week 7, the coach asked each learner once a week what piece or technique they wanted to work on next and why, and recorded the answer as the learner's goal for that week. If a fading learner logged no session in a week between weeks 9 and 16, the coach offered to bring the target meter back for two weeks. In persistent accounts (n=98), the full game layer ran until week 16. No-game accounts (n=97) had the coach's feedback and suggestions with no game layer. At the end of week 16, the program closed and game features were switched off for all accounts; the coach, its feedback, and the practice log remained available to everyone through week 28.

A session was counted when the coach logged at least ten minutes of playing. The follow-up outcome was whether a learner logged at least two sessions in each week from 25 to 28; learners who logged nothing were counted as not meeting it. At week 16, learners completed a survey that included a four-item rating of interest in playing for its own sake (1 = none, 7 = very strong).

3. Findings

In weeks 1 to 6, learners with a game layer practiced more: 4.1 sessions a week in fading accounts and 4.0 in persistent accounts, against 3.3 in no-game accounts.

While the game layer was being removed, fading learners practiced less than persistent learners. In weeks 7 to 10, persistent learners logged 3.9 sessions a week, against 3.4 for fading learners and 3.2 for no-game learners. In weeks 13 to 16, the figures were 3.8, 3.3, and 3.0.

The end of the program reversed the order. In weeks 17 to 20, after game features closed for everyone, persistent learners' practice fell to 2.8 sessions a week, 26% below their weeks 13 to 16 level. Fading learners' practice fell to 3.0 and no-game learners' to 2.8.

In weeks 25 to 28, 45 of 99 fading learners (45%) still practiced at least twice every week, against 35 of 98 persistent learners (36%, a 27% gap) and 37 of 97 no-game learners (38%, a 19% gap).

At week 16, fading learners rated their interest in playing for its own sake 5.1/7, against 4.3/7 for persistent learners (a 19% gap) and 4.6/7 for no-game learners.

Of the 29 fading learners who had a week with no practice between weeks 9 and 16, 17 accepted the offer to bring the target meter back. Seven of those 17 were still practicing at least twice a week in weeks 25 to 28.

4. Discussion

The persistent design did what game layers are designed to do for as long as it existed, and in both measured periods between weeks 7 and 16 its learners practiced more than anyone else. The trouble came when it ended. Learners who had practiced with points and a meter until the last day lost all of them at once, at a time chosen by the program calendar, and their practice fell by about a quarter in the following month. Fading learners lost the same rewards earlier, one at a time, while the coach was asking them what they wanted to play next, and by week 16 there was little left for the program's closure to take away.

The interest ratings fit that account, though they cannot prove it. Fading learners had spent ten weeks naming their own next piece and their own reason for it, and at week 16 they rated their interest in playing for its own sake higher than either other group. This resembles the self-anchored progress result reported in this journal's sixth issue, in which a learner's own record sustained practice better than a ranking against others. Here, too, a reference point the learner chose did more than one the system supplied.

The fading design asked something of its learners during weeks 7 to 10, when they practiced about half a session a week less than they would have with the rewards still in place, and of the program, whose mid-course engagement figures were lower. It also did not insist that learners manage without help. Seventeen learners took the meter back for a while, and seven of them were among those still practicing months later.

5. Limitations and Future Directions

The fading design changed three things at once: it removed rewards gradually, added weekly learner-chosen goals, and offered the meter back after a missed week. This study cannot separate them, and a factorial design isolating gradual removal and learner-chosen goals is the obvious next step. Persistent learners were not told in advance that the game layer would end at week 16, which may have made its loss more abrupt for them than it would be in programs that announce it. Learners knew their design. The week-16 interest rating was taken before persistent learners lost their game layer, so it shows a difference in reported interest beforehand, not a measured cause of the later drop. The 17 learners who took the meter back chose to do so, and we cannot compare them with similar learners who did not. Ten minutes of logged playing is a coarse measure of practice and does not reflect its quality. The program was free and run by community schools, and adults who pay for lessons may respond differently when rewards are removed.

Editorial Note

This report underwent CfAHC's Rapid Review Protocol, including AI-assisted literature cross-checking and a two-reviewer human sign-off.