Monitoring Machine learning system is a cumbersome process that involves quite a lot of skills other than constant business feedback.
Broadly there are 3 kinds of monitoring that one need to focus on:
1) Data : Data Monitoring includes drift monitoring, monitoring data variance and feature monitoring
2) Model : Monitoring of model accuracy over a period of time to make decisions for retraining and A/B testing, It also include model health check
3) System : Monitoring system health parameters such as average response time, Capacity utilization, Number of service requests, down time etc.
No comments:
Post a Comment