At first glance it's an ordinary table-football table. A black frame, a green pitch, red and blue players on gleaming rods. Above the table, a single webcam is mounted on a frame, pointing straight down at the pitch. Red and blue LED strips run along that same frame. That one camera tracks the ball and every movement of the little players on the rods. The clatter of the rods echoes through the lab. Because the table is also just a toy: researchers and students genuinely play matches on it.
Students make the table smarter
Above all, the table is a never-ending student project. Time and again the lab sets it up as a new brief and asks small groups to make it smarter. That's how the goal detection and an automatic Elo ranking came about. The latest is the commentator. From the webcam footage, an AI works out what's happening and talks over it live.
Good commentary is two things at once. You have to see what's happening and put it into words at the right moment, with the right emotion. A computer can learn the first part. The second is the hard bit. Lorenzo Gatti, an assistant professor in Human Media Interaction, builds language technology that writes text on its own. For an open-source football game, he and his students had already built a commentator like this. On this table, that technology finally got a real pitch to play on.
A house full of smart devices
The table sits in the IoT Cyberlab, part of the Twente University Centre for Cybersecurity Research (TUCCR). The lab is set up like a studio flat where almost everything is smart. There's a smart fridge, a dishwasher, a robot vacuum and a coffee machine. A video doorbell, voice assistants and scales that send your weight to an app. There's even a games console, an air-quality monitor, a heated coffee mug and a sensor under the mattress that tracks your sleep. Forty-two devices in all, together sending out around 10 GB of data a day.
All that traffic is exactly what the researchers study. They try, for instance, to work out from the outside which devices someone has at home. "A table-football table like this really captures the imagination," says Max Pijnappel, the lab's manager. "And at the same time it shows what we're working on: everyday objects that are packed with sensors and send out data all day long."
Looking for the weakest link
That sounds harmless, but it gives away more than you'd think. Every device has a unique hardware number, and many devices build that number into their internet address as well. The first part of such a number belongs to the manufacturer. So anyone watching the traffic from outside a network can often already tell which brands of device are in the house. And your smart fridge, your doorbell or your robot vacuum says plenty about you.
And every extra device brings an extra risk. A smart bulb or a cheap plug doesn't always get updates, even though it's sitting on your network. Anyone who knows which devices are in a home can go straight for the weakest link. Through that one vulnerable device, they're into the rest of the network before you know it.
Can AI take over the commentary?
Not yet. The system neatly describes what's happening, but the magic of a goal lies in the right emotion at the right moment. That's exactly the bit a human does so well. And it can certainly get smarter. "Right now the system only knows who touched the ball last before it went in," says Pijnappel. "We want it to recognise how the goal was scored as well, and to act as a kind of VAR that flags illegal moves like spinning or pinning the ball." That turns a table-football table into a testing ground for a question the professional game is facing too. A computer can call the goal. The goosebumps, for now, still come from a human.
