Open Source Summit + Embedded Linux Conference + OSPOCon 2021

As more machine learning models are developed and served on Kubernetes, it's becoming harder to track the incoming data and payloads by just reading the logs. For trusting model predictions, drift, anomaly, adversarial and bias detection need to be built in the platform. Data scientists have difficulty figuring out model behavior on Kubernetes since it's hard to access and process model payloads on Kubernetes. Therefore, it's important to record and persist the model input and output payloads with the proper schema. These payloads can be used with other tools to explain, analyze, and generate machine learning metrics such as fairness and drift detection for models running in production. This can help AI operators and data scientists visualize and find any potential issues with the model. This talk covers how to use KFServing, Kafka Connect, and AIF360 to serve ML models, persist payloads, and measure the model fairness in a production environment.