Evaluating ML Opportunities, Part 2
This is the second in a three-part series laying out questions you should ask before you start a machine learning project.
After you’ve answered ‘yes’ to the prerequisite questions we covered in Part 1, it’s time to ask the next set of questions. This time, you’ll take a harder look at your project idea.
Think of these questions as guidance for your own intuition. If enough of the answers make you feel uneasy, you might consider if another project makes more sense.
Does the Project Actually Matter?
Start to separate out what seems cool from what would really have business impact.
If your project succeeds, how will it help you decease costs or increase revenue? And how confident are you that you can actually achieve real business impact beyond a nifty “science experiment” that winds up on a shelf?
Will this project address an issue that is of major concern to you, your C-suite, or someone else that matters? Will the project solve a significant pain point, or is it simply a “nice to have”?
The expected impact of a project might naturally evolve over time. As you proceed, your expectations will become more refined. But even after you start a project, you need to keep asking if it actually matters.
Can the Project Scale?
You want whatever you do to eventually impact as many people, clients, teams, or other units as possible. If you’ve built a machine learning tool that can analyze 300,000 emails, then how about 300 million?
“Science experiments” that only work in certain conditions and have no chance to impact the broader organization are not a good use of machine learning resources. To butcher the old proverb about a tree: if a machine learning project goes into production and almost nobody knows about it, does it make an impact? Probably not.
You want to give more weight to machine learning projects that have a potential to end big. But this doesn’t mean you should start with your ML intensity set to maximum. It can be smart to start small.
Is This Low-Hanging Fruit?
Going for the quick win is especially important if this is your first machine learning project, you don’t know what you don’t know, and you’re trying to build institutional support.
This advice may seem inconsistent with the desire to pursue projects that can scale and have real impact. What I’m really talking about here is the order in which you proceed. Impact at scale should always be the overall strategic goal, but doing easier projects first can build the internal momentum that allows you ultimately to achieve that impact. Plus, some ideas may be able to scale precisely because they are simple.
Especially if there’s a potential shortage of C-level support for significant change, it’s a good idea to go after the simplest thing first and prove that it works, rather than trying to shoot for the moon and failing. Even the original moonshot — Apollo 11 — only occurred after other missions had paved the way.
You won’t always get to pick low-hanging fruit, of course, either because you already picked it or the situation calls for reaching higher up the tree.
Can You Break the Project Up into Smaller Pieces?
Machine learning can be complicated. Breaking up a large initiative into smaller projects can help make it simpler.
You could try your first ML initiative as a pilot project. This gives you the chance to conserve resources while you build support and learn lessons that will be valuable if you follow it up with a larger-scale endeavor. It could also be a way to deliberately create some of the low-hanging fruit we mentioned above.
Again, it’s good to have a project that can ultimately scale for maximum impact. But to get there tactically, it’s good to start small so that you can end big.
Do You Understand How It Works?
You don’t need a deep technical understanding to make a business call on an ML project. But if you can’t get your head around the technology at all, that’s a sign that you may need to press pause. In the machine learning field, people offer some farfetched ideas, making it hard to separate fact from fiction.
Ask how the technology is actually going to work. Could a human do this task before? If not, how will a piece of technology do it?
Are You Properly Calibrating Expectations Given the Hype?
It’s easy — and understandable — for scientists and engineers to get excited about the potential of their work. That excitement can motivate them to excel. But promises based on lab results may not always hold up in the real world.
Machine learning can help your business do incredible things. But when you hear fantastic promises, it’s your responsibility to align your expectations to the messy realities of day-to-day business, not the perfect conditions of a machine learning lab.
If all of these questions leave you ready for more, it’s time to jump into the final stage of the evaluation: process questions. Those are up next.
James Kotecki is the Director of Marketing & Communications at Infinia ML, a team of data scientists, engineers, and business experts putting machine learning to work.
