Agile for Intelligent Products

Adopting Agile practices for AI initiatives

If you’ve been around software-enabled businesses for any length of time, you’re probably already familiar with Agile practices. And with good reason. Agile allows teams to work quickly and collaboratively to deliver value with fewer headaches — even in uncertain, turbulent environments. Compared to a traditional waterfall model of software development, Agile allows teams to test hypotheses, get feedback and decide on scope and direction at a faster pace. As a result, Agile practices have become the de facto way to get things done for most tech-enabled ventures.

Why Agile is a great fit for AI projects 

In our opinion, the principles and practices behind Agile — as outlined in the Manifesto for Agile Software Development — make Agile a great way to work on AI and ML initiatives.

• AI projects are (by their very nature) uncertain. While waterfall approaches to project management often fall apart as circumstances shift and evolve, Agile works well in the presence of ambiguity. 
• Agile is (as its name suggests) extremely agile and adaptive. With AI projects, teams need to be able to evaluate the viability of technology choices, make frequent changes in direction based on rapid feedback loops. Agile affords flexibility.
• As in software, Agile tends to lead to better AI products. Agile involves faster iteration and involves appropriate stakeholders at each step. As more users interact with products, Agile adds checks and balances to the finished product.
• Lastly, Agile practices emphasize outcomes, not dogmatic adherence to processes that may not fit a project. A focus on outcomes encourages the creation of a finished product — helping teams focus on delivering value (not endless experimentation).

How Agile for ML and AI projects is different

For the reasons described above, Agile is a great fit for the projects that Machine Learning or Data Science teams undertake. But there’s a catch. Incorporating Agile into machine learning is not as easy as simply copy/pasting Agile DevOps processes into an MLOps environment. For a host of reasons, Agile practices that work well for software development don’t always translate into AI or ML initiatives. For example:

• Data scientists often can’t provide daily updates during experimentation, as experiments don’t always move at such a pace. The daily standups software developer know and love aren’t always valuable data scientists — so they often don’t like them.
• It’s extremely difficult for teams to timebox experimental data projects. So trying to stick to timeboxed sprint planning and end-of-sprint demos can be a fool’s errand
• User stories and acceptance criteria for AI projects are done differently than for traditional software

The mindset of data scientists is very different from that of software developers. In software, engineers have well-defined acceptance criteria and a very mature set of technologies they are working with, so targets are often well-defined. The same cannot be said for data scientists, as the nature of data science is uncertain and always shifting. Algorithms can change quickly. Infrastructures can shift mid-project as criteria for success evolve. Much of what is going in the field of data science is still taking place in research labs.

How to adapt Agile to work for ML and AI projects

As leaders in MLOps, we’ve worked with countless organizations at almost every phase of the model lifecycle — from experimentation to operationalization. In our experience, we believe that AI-enabled organizations should strive to be as Agile as possible. But companies cannot simply expect a software-based Agile methodology to work well when applied to an AI project. depending on the company and the organization and the type of AI project, processes and methodologies must be refined and adapted for best results. What follows are a few tips for making Agile work for AI:

1. Follow a Kanban model

A Kanban model works for many ML teams since the methodology is fairly lightweight (with few ceremonies) and facilitates more open-ended project planning. Since many ML projects cannot be scoped and defined within a sprint boundary, a Kanban board can help create a list of high-value items that a team can move through (in progress and close states). Kanban also permits teams to continue to scope work as it is in progress. 

But Kanban is not a one-size-fits-all solution to Agile for AI and ML initiatives, as the approach contains a few potential drawbacks. Kanban may cause teams to lose sight of the broader picture and business context surrounding a project. And teams using Kanban can sometimes unwittingly work themselves in a silo — finding it difficult to integrate with the rest of the product teams.

2. Follow flexible scrum practices

Teams should also consider customizing scrum processes to fit into data science projects. Even though some software engineers might believe otherwise, AI teams don’t need to put strict 2-week sprints on development processes. For example, opt for smaller or longer-duration sprints to accommodate your project type and rely on spikes for research-oriented work. This will allow teams to not spend time pondering overestimation or definition of done for open-ended features.

3. Use a hybrid approach

Lastly, we recommend using a hybrid approach. Splitting projects into two distinct phases: model experimentation and model release.  

Model experimentation (R&D) involves a lot of open questions and evaluations. This phase tends to be more spike-based, and those sprints often have unpredictable outcomes. We’ve experienced that teams are usually more successful using less rigid Agile frameworks at this phase (discussed in greater detail below).  

The closer a team gets to a model release, Agile practices should become more rigid. While getting a model ready for release, teams can spin up formal Scrum (with regular sprints, user stories, etc.) with additional stakeholders. At this phase, you may require integrations with consuming teams, model optimization, testing/validation, etc. to get the model to the last mile, for instance  release planning, stakeholder engagement, etc.

Lean into experimentation and adaptation

In our opinion, AI and ML teams should absolutely strive to become more Agile. But we should focus on embracing the ethos of flexibility and agility behind Agile — not shackling ourselves to processes that might not always fit. Don’t be afraid to experiment and iterate until you find a way of working that works for your team and project.