Deploying AI programs at scale will necessitate a back-end command center that monitors deployed models. This is critical to successfully scale the program.
As we mentioned in our previous post, models are learning algorithms and require data, training, and tuning to increase their performance. If left alone, they may need to be retrained. “Human-in-the-loop” back-end operations should be established to monitor model performance, detect anomalies, and give nascent models training and performance feedback. This “AI Operations” team is a function of the hub described here and oversees all running programs.
AI Ops may also monitor for changing trends inside datasets that may influence model performance and require new training. Market conditions are one such example of this. This is not just about monitoring models, but also the technology that supports them. This team should also closely monitor cloud consumption and problems that may occur at data ingest. The AI Ops team may include data scientists, machine learning engineers, and IT systems architects to ensure the program is running smoothly.
Since this is the final post in our series on building internal AI capabilities, we wanted to call out that digital transformation of IT systems has a history of disappointing leaders who underestimate the organizational changes needed to capture their value. Many executives are reliving this disappointment with artificial intelligence programs. AI does not deliver value by crunching data and delivering outputs. It delivers value when organizations change their behavior in terms of processes, policies, and practices.
Most AI solutions impact several business units and processes. This requires business process designers to re-engineer new roles and processes across the whole organization. Any shifts of roles or responsibilities due to process change necessitates the need to upskill or re-skill employees on new tasks. It may even generate completely new roles.
The future of many AI jobs will include some sort of human-in-the-loop component. Employees will need to manage AI solutions and ensure they are learning and not drifting. Business conditions and demands change constantly. As a result, the data used to create an AI algorithm becomes a less accurate reflection of reality over time. It is important that roles using AI predictions manage those systems making predictions to ensure performance gains, and interject when performance decreases. It will be important that processes and training are in place to up-skill employees who manage models to monitor these changes. IT will need to create reporting and alerting systems for model managers to track when deviations occur.
Want more insights into how you can better integrate AI Ops into your org? Download our free whitepaper here.
.png)
