In 2016, a computer programme called AlphaGo defeated the world champion in Go, one of the most difficult board games in the world. That was an important moment in the history of artificial intelligence. The technique used enabled the computer to develop more effective strategies on its own. This technique is now not only suitable for board games. Researchers at the University of Twente are applying this form of artificial intelligence to logistics.
What computers learn from board games
"Games are the perfect way to develop smart algorithms," says Fabian Akkerman. "You have clear rules, a goal, and every move has consequences for what happens next." That is precisely why games such as chess or Go are so suitable for training AI.
The technique the researchers use is called reinforcement learning. An algorithm makes choices, learns from the outcome, and tries to do better the next time. Just like a gamer who becomes more skilled at a game by making mistakes and repeating successes. In his doctoral research, Akkerman looked at logistics problems where decisions are not one-off, but have to be made over and over again. For example: What delivery times do you offer your customers?
"Traditonal methods try to calculate everything at once, as if at the beginning of a game you already know how each move will go. But in practice, the situation keeps changing. That's why we train systems that can adjust, learn and adapt," he explains.
Algorithms play against themselves
"But the world is not static. Customers change their minds, and how do you plan routes if you don't yet know which customers will come forward? That's why you need a system that learns from what happens and adapts all the time." Together with researchers from TU/e, the UT researchers developed DynaPlex: open-source logistics AI software capable of continuously learning and adapting to changing circumstances.
In his research, Akkerman built models that taught themselves. Just as AlphaGo played millions of games of Go against itself, these models mimicked thousands of logistics scenarios. In this way, they learned the smartest decisions step by step.
Discounts at the right time
A concrete example: dynamic delivery moments. Suppose you order something online. The webshop offers you a delivery moment, sometimes with a discount. These moments are not chosen at random. Behind the scenes, an algorithm determines which option is most favourable for the seller. This depends on how many other orders are already scheduled, where you live, delivery moments chosen by other customers and how much space is left in the van.
Akkerman developed a model that self-learns which choices influence customers and how this correlates with delivery planning. "For example, the system learns that you can encourage customers to choose a delivery time that fits better into the schedule by giving a small discount at the right time."
Decisions in terms of a suitable delivery moment or the prices of these delivery moments should be made in a fraction of a second once the customer wants to check out online. Since there is no time at that moment to calculate all sorts of things, the system must have learnt the ideal decision in advance.
Already successful
The approach quickly bore fruit. There are now more than fourteen successful practical examples, including at Albert Heijn, NXP, Vanderlande and ASML. Students often worked with these companies to apply the software in their graduation projects. One example is the application at ASML. There, the software showed how production and inventory management could be made five percent morper cente efficient. And that is based on real customer data and real improvements.
Even when companies do not implement the software, DynaPlex provides insights. "You can learn a lot by seeing where AI makes different choices than you would yourself," explains Martijn Mes, professor at the UT. "That challenges companies to rethink their fixed assumptions. Sometimes you discover alternatives that you would never have thought of yourself."
Faster response
These techniques make logistics more efficient. Fewer empty kilometres, better utilisation of stock and storage, and faster response to unexpected situations. Akkerman's approach also yielded good results in inventory management: "We developed a system that not only takes into account uncertainty in customer demand, but also errors in administration, such as lost products." The system decides when to order which products, but also which stock levels to inspect.
In all these applications, the algorithm does not look at a single decision, but at a series of decisions: if I order or offer something now, what will that mean for the next five steps? According to Akkerman, the strength of this approach is that the computer not only calculates but also learns. "We build systems that, like a good player, think ahead, adapt, and want to do better on the next move."
The future of smart logistics
What started as a way to learn how to play a game is growing into a key technique for modern logistics. The algorithms that not so long ago defeated the top human players in computer and board games are now helping to deliver parcels on time, replenish stocks and offer customers better choices.
And who knows? Perhaps such systems will soon also help decide how we travel, build or produce. Because one thing is certain, says Akkerman: "A learning algorithm continues to improve, which makes it a smart and perhaps even an essential player in logistics."
