The Feedback Vacuum

Another interesting situation that teams often run into is what I call the “feedback vacuum”. This can happen when you start automating human input with model predictions (e.g., for email classification). You simply lose the “ground truth” data that you would measure the correct outcomes by. This can become a big problem if the data changes over time (i.e. concept or target drift), because it can then be impossible to accurately evaluate your model’s performance.

One logical solution is to introduce periodic manual reviews, but these come with their own problems. When serving as reviewers, people often act differently than in their day-to-day roles, introducing subtle personal biases. This can simply be a result of the different framing as a “reviewer” or more subtle things like a difference in the user interface in which reviews are made.

Suggested Solution

In practice, I’ve found the best solution to be a combination of two things:

Confidence Thresholds: Use a model that provides a confidence or prediction probability score, so that data points below a certain threshold aren’t classified and can be handled manually. This is a clever way to ensure that previously unseen or rare cases, as well as any big changes in the data, are handed off to humans. The result? In my experience, dramatically reduced error rates. The only disadvantage is a reduction of the automation rate, which becomes more severe the higher you set the confidence threshold. Held-Out Data: Leave a certain percentage of random data points for human handling, while only storing the model predictions in the background for validation purposes. In the email classification example mentioned above, my team and I left 5% of random emails in the inbox untouched, without the humans knowing that they could have been automated. This approach will help you to constantly evaluate model performance and maintain an unbiased ground truth for future trainings. What counts as an appropriate hold-out percentage depends on the volume and imbalance of your data, but the usual range is between 1% and 10%. In my experience, these two measures helped with both reduction in error rates and a drastic improvement of model maintainability.