Article written by MSc Lars Zwanepol Klinkmeijer, CEO Datamaister

Automation works best when things remain the same. This holds both for ‘classic’, engineered, rule-based automation, and for Artificial Intelligence (AI)/ Machine Learning (ML) based automation. The fact that the world is never constant and always in motion is one of the reasons ML-based automation is so powerful compared to its ‘rule-based’ counterpart. This is because most models created using ML, for instance deep learning and SVM, are good at generalizing. when it ‘sees’ something new it can still match it with something similar it has seen during its training phase. But what to do when the world dramatically changes?

The case for interactive AI

When the Covid19 crisis hit and countries and consumers started to respond with lockdowns and hoarding, all automated planning tools in the supply chain, manufacturing, demand forecasting, etc., suddenly found themselves in the dark. The world changed too much, too sudden. So much so that not only the AI could not make sense of the new situation, even human experts had massive challenges understanding what to do next.

So how do we make sure we are ready for such events when they happen in the future? You could argue that we now have data from this pandemic so we can train models to properly respond the next time a pandemic like this occurs. But this means you are always only prepared for the second time something happens. All well after the fact. Also, trying to prepare for every worst-case scenario, even if that would be possible, would simply be too costly and too cumbersome. Such a strategy is likely to bankrupt you even before that worst-case scenario you so carefully prepared for finally occurs and bankrupts your unprepared competitors. The trillion-dollar question is: how do we respond as it happens? At Datamaister we believe the answer lies in Interactive AI: using people’s knowledge to rapidly update your models.

Updating models using knowledge of people

There are many different reasons why having the Human-in-the-loop is powerful and here we have one more example of such a case. With changes as big, and as rare, as a pandemic, even humans have difficulty making sense of the new situation. Yet people do know more than any AI system does. This is because the AI has only a very limited view of the world through its data.  People on the other hand can make a better sense of the new situation because of their general knowledge and access to additional data sources. To harness that knowledge you need an interactive AI system.

The danger is that an interactive AI system could lead to using massive amounts of a person’s time where they go over every data point or each assumption of the AI. That should not be the case. Interactive AI, therefore, is more than just labeling the new data and retraining. It requires a system that can quickly generalize on the feedback it receives from the person it is interacting with. It requires a system that can quickly update and deploy its new model. These are architectural design choices on the part of engineering, infrastructure, application design, and ML. From an ML perspective, it means it will work best if you can easily create new models. It’s possible you will require new data sources or to create new features from your data. Automated machine learning (AutoML) can potentially do all these things. We at Datamaister, therefore, believe that Interactive AI and AutoML go hand in hand.

The idea behind Interactive AI

‘Interactive’ in Interactive AI refers to the close cooperation between the AI and a person. In a very real sense, the person is tutoring the AI. Similar to when you are homeschooling your child. Your kid, for example will do some math exercises. Most likely you will let him or her do the exercises themselves. Initially, you could be inclined to go over every exercise and check each answer. This will be time-consuming. A faster method would be to let your kid show you the exercises they are not sure of and only check those. You now can give feedback where it counts. In the next round of exercises it will be likely that the number of cases your child confidently solved will be larger, leaving you fewer to check. A great time saver! 

Interactive AI follows a similar idea. Most AI models can give confidence values to their outputs. As a tutor, you focus on the examples the AI has low confidence in and continue until you think, “yeah, the rest of the examples, I believe, are all correct”. You have now given the AI more information to work with and in the next training run, it should create better results which means you have fewer examples to correct. As you can see, this is not about labeling data. It is about correcting the assumptions the AI is making. What is great, is that the person tutoring does not need to know anything about AI whatsoever, which is likely the case with your domain experts.

The benefits of AutoML in relation to Interactive AI

People with some understanding of ML/AI/Data Science will have realized that Interactive AI requires you to be able to retrain your model. At Datamaister we strongly believe that Automated Machine Learning (AutoML) is the best way to do this. In the case of Interactive AI, there are several benefits to AutoML. First of all, retraining your model might require more than just updating some hyperparameters. If you are using multiple different AI models in an ensemble you might want to create some entirely new models, using different algorithms, different ‘views’ on the data, etc. Similarly, you might want to add some new data sources, create some new features from the data sources, etc. These are all steps that a good AutoML system can do for you. Automatically! And that leads us to the second reason why AutoML is so useful in this situation. Because it is automatic you do not need to be a data scientist to do all those steps. This means that the person who was tutoring the AI using the Interactive AI is all you need to update your models.

The need for Interactive AI in post-pandemic situations

The usefulness of Interactive AI is more than just in pandemic situations. Even in the pre-COVID world, and as it will in a Post-COVID world we will encounter situations where the world around us has changed and our models need updating. Likely not as fast and radical as with the pandemic but changes that require general knowledge, new data sources, and features nonetheless. Here too Interactive AI together with AutoML is, therefore, a cost-effective method of preparing for any changes, big and small.

If you are interested to know more about Interactive AI, please contact us at or contact me directly.

Some suggested further reading:

An article on the benefits of having the human in the loop.

A scientific publication on Interactive AI:

T. van der Zant, L. Schomaker and K. Haak, “Handwritten-Word Spotting Using Biologically Inspired Features,” in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 30, no. 11, pp. 1945-1957, Nov. 2008, doi: 10.1109/TPAMI.2008.144.