There is a new AI in town: hybridization of ML and OR

Since a few years, AI (artificial intelligence) is all over the place. The hype and promises are high yet it is common knowledge that more than 80% of AI projects fail in the industry. Not really a success story, is it?

OR, what's that?

Most of the time, when you talk about AI, people think ML (machine learning) or DL (deep learning, which is a particular sub branch of ML). But did you know that there are other domains in AI? In particular, OR (operations research). Never heard of it? Well, yes, it is not very known yet it is extremely powerful. Just a hint. A few years ago, ML was still considered as... a sub field of OR! You read that right. There is another field from which ML originated! So what is OR? Basically, you can see OR as one of the science of optimization. No, scrap that. It is the best science of optimization! In the sense that you get the best optimization results. ML basically uses OR algorithms to optimize its predictions (ML is more than that but that is the gist of it). Although you might not have heard about OR, it is extensively used in some sectors (supply chain, the army in general, transport, manufacturing, ...) with incredible successes. Whenever you want to optimize an industrial process, think OR!

Predictive vs prescriptive approaches

Now, if that is fantastic news, there is more! Most projects using ML are kind of trying to predict things. Very useful but again, did you hear about the prescriptive approach? In a predictive approach, you try to gather some data and figure out what this history of data can tell you about the future. So you construct a system and give it an input and the system returns an output with a certain probability. Here is a know example. You want to recognize cats in pictures. You train your system and it "learns" how to distinguish cats in pictures. It works pretty well on your training sets but then... it totally fails in production (remember the 80% or more of AI projects that fail?). What is the prescriptive way? It is a paradigm shift! You don't give your system an input, you give it... your desired output! So, in our case, let's say that our desired output is "I want to construct a system that recognizes cats in pictures more than 90% of the time". And this in the production settings! Not during a training session. And the system will "tell" what to do to construct such a system! And it works! (of course, there is no miracle recepe but the prescriptive approach - which contains the predictive one - is really a notch above the predictive one...). Basically, in a prescriptive approach, you construct systems that will tell you what to do to obtain your desired objectives. And you can take a bunch of cool things into account at the same time: the real settings, some unknowns (real unknowns that are not in the data), you can mix insights for domain knowledge and data knowledge, etc. In fact, you can even start a project without data! Indeed, collecting data is difficult and actually part of the project. You can ask your system what kind of data and how you should collect them! Most of the time, you use OR to implement a prescriptive approach to solve a problem.

Hybridization of ML and OR

Now, you can go even further by combining ML and OR together. They are really complementary. There are basically 4 ways to combine ML and OR:

1. The now "classical" use of ML as a pre-process and OR as a post-process steps. The next two approaches are purely technical and only of importance to ML and OR practitioners.
2. Use OR to improve ML.
3. Use ML to improve OR.
4. A complete hybridization of both ML and OR where we develop new algorithms.

ML and OR as separate blackboxes

This is the most common approach whenever companies propose to combine ML and OR. Often, ML is used to predict and OR is then used to optimize on those predictions.

As an example, let's say you are a train company and you want to repair/replace your tracks. Of course, you'll want to do this for the minimum cost. How can you do this? Decouple this problem in two steps:

1. Use ML to detect your defects
2. Post cameras under the wagons of your trains, take pictures in rapid succession. With computer vision and deep learning, find the defects on the pictures and stitch your pictures together to obtain a network/graph of defects of your tracks. You can classify the nature of defects into several classes:

• to be replaced:
• immediately
• in 3 months
• in 6 months
• ...
• to be repaired:
• immediately
• in 3 months
• in 6 months
• ...

1. Use OR to optimize your repair/replacement policy
2. Once you have your network/graph of defects, you still need to know how to replace/repair your tracks in an optimal way. If a maintenance team is on site to repair a track, it might be less costly for it to repair another close-by track that doesn't need immediate repair. OR will choose an optimal repair/replace policy for your tracks.

Use OR to improve ML

OR can be used to optimize, i.e. minimize or maximize some function in ML. But wait, isn't this what ML is about? Exactly, ML is strongly based on OR optimization when it optimizes its predictions.

Use ML to improve OR

It also goes the opposite way. The strong point of ML is to predict. Some research is conducted in order to predict on what variable and how to branch in search trees.

A complete hybridization of both ML and OR

We are developing new algorithms that are a complete and total hybridization of both ML and OR. These new algorithms don't treat ML and/or OR as separate black boxes but are really a new combination of the strengths of both fields. We are convinced that these new algorithms are the way to go to solve any problem and that in the future OR and ML will be merged into a new field. GNNs are a first example of such combination but this barely scratch the surface!