In this article, you will learn on how to deploy Machine Learning in Agile way to support your data projects. Here are 5 steps to keep in mind when addressing this kind of projects.
Machine Learning Deployment Should be Managed as a Project
When we think about Machine Learning deployment, we often think just about the resulting models, reports, insights, or visualizations. Even if these end-products are usually the main deliverables, it’s easy to shift attention on trying to release them as fast as possible or make them “beautiful” and ignore the quality of the code, the model performance once in production and the relevance of data that generate them.
It is well known in IT project management that a well-conducted framing is essential to meet business goals and to avoid hidden costs in later phases of the projects.
Agile methods have proved its worth and have become the default choice in managing projects. The key intent is to mitigate the risks of delivering something that does not correspond to the business need.
Applying Agile Practices for Machine Learning Deployment
How to induce Agile practices when managing Machine Learning deployment? Does it work? The answer is not so straightforward as we would wish. Each ML project is specific and hard to simply copy/paste from one another.
The first recommendation is to start with the end and then work your way to the start.
In this article, we would like to spotlight some interesting agile practices that you can integrate when managing Machine Learning project. It might be helpful to follow the presented below routine:
- What are you trying to do for the end-users? What objective are you serving? Question what are you exposing. An API, a result, a table, a visualization? This will guide further implementation tasks.
- What are the main functional verticals? Understand what are the main functional verticals of your application.
- Divide up the work into functional increments called user stories and use visual planning tool such as story mapping. Identify and plan the functional tasks to be performed along the functional verticals.
- Iterate and deliver frequently (for example, one iteration every two weeks).
Understand What are the Main Functional Verticals of your Project
When you deal with a complex or big project, especially with the predictive system, it makes sense to divide up it into small functional pieces called “user stories”.
According the Agile Alliance, the intent of this practice is “to avoid a failure mode of incremental delivery, where a product could be released composed of features that in principle are of high business value but are unusable because they are functionally dependent on features which are of lower value and were therefore deferred to future releases.”
To illustrate practically this aspect, let’s take an example in solar energy management. Suppose you develop a smart home application for solar management. The app offers dashboard displaying short-term solar energy production, consumption history, and provides some recommendations on how to optimize energy use and storage.
You can organize a first framing workshop where you will define what are the main functional verticals:
- Vertical 1: Front design. It includes the way to connect application, to display the home screen with expected dashboards and graphics.
- Vertical 2: App content. Integrate consumption and production history and model short–term solar predictions (a machine learning algorithm).
- Vertical 3: Recommendation engine (a machine learning algorithm). The application recommends on how to optimize the storage of the produced energy by solar panels. It will include a basic strategy, whatever you should store or consume energy directly.
These functional verticals will create a skeleton of your project and will keep teams of scientists, engineers and developers focus on the same project breakdown.
Story Mapping to Identify and Plan the Functional Tasks to be Performed
Followed the first framing workshop, you can organise a Story Mapping workshop to identify the functional tasks to be performed within each vertical.
“Story mapping consists of ordering user stories along two independent dimensions. The “map” arranges user activities along the horizontal axis in rough order of priority. Down the vertical axis, it represents increasing sophistication of the implementation. (the definition from Agile Alliance www.agilealliance.org)
Here, some examples of tasks within each vertical:
- Front / Dashboard. The front-end application that will allow users to make and consult simulations. As a task, you will have, Flask installation and configuration, login & ID management, set up a template from Grafana for dashboarding.
- Monitor. The application will allow monitoring time series (production/consumption). The task might be here: connect to data sources and charge data history for visualisation.
- Short-term prediction. The application will allow predicting short-term production. The tasks are to develop a short-term solar prediction, to train and test model performance.
- Workflow. List all tasks to be carried out periodically (data integration, the transformation of formats). Integration of the data history into the target architecture.
Each task is associated with a T-Shirt size ranging from XS to XL. This corresponds to the estimated duration of the task:
< XS: < 1 day
< S: 1 to 3 days
< M: 3 to 5 days
< L: 5 to 8 days
< XL: 7 to 12 days
At the end of the framing and story mapping workshops, you count the number of iterations and plan iterations with a short and regular period to deliver value continuously.
Who Will be the Customers of These Applications/Models?
To design effective ML systems, it is essential to explicit the expected value proposition.
- What are you exposing? An API, a result, a table, a visualization?
- What is the use scenario for prediction models? And simulation models? How are your users (clients, technicians, call center operators)?
Our recommendation is again to work in an agile way directly with the customer(s) of these solutions to quickly develop value and allow flexibility in development. Don’t ignore the aesthetic aspect, and the solutions need to be desirable and look nice.
Going through these different aspects would bring structure and rhythm to your project. Feel free to experiment with other agile techniques and measure how useful they are. Have a good day!
