Most of us use apps like Snapchat and TikTok without thinking twice. But what if you could build the technology behind them yourself? In the Image Processing and Computer Vision (IPCV) course from the Master Robotics programme, students do just that.
Dr Kenan Niu from the Robotics and Mechatronics group teaches students from different engineering disciplines the core technology behind today’s most popular apps. The students move from theory to practice. They create their own real-time interactive visual effects.
Student work from Timon Veurink
How can a computer understand what is sees through a camera?
In the IPCV course, students learned the various image processing and computer vision methods and algorithms, which allowed them teach the computer to “see the objects”. Their first challenge: getting the computer to recognise a face or a hand in a stream of images that changes every split second.
Using computer vision programming libraries like OpenCV (open source computer vision library) and MediaPipe (an AI model for face and hand tracking), students wrote python code that spots a face in a video frame. Once detected, the software looks for key reference points that almost every human face has and that stay roughly in the same place even when you move. Think of the corners of your eyes or the tip of your nose.
Digital dots on your face
These points are chosen on purpose. They are easy for a computer to recognise, and they move in predictable ways when you smile, blink or turn your head. By tracking these points frame by frame, the computer can calculate how your face or hand is moving in 2D or even 3D space.
Imagine your face as a simple wireframe made of dots. Each dot marks a spot that rarely changes: eye corners, nose tip, mouth edges. When you laugh or wink, those dots shift slightly. By following their movement, the computer knows what you’re doing and where to place a digital mask so it sticks perfectly.
Student work from Oscar Peters
Where engineering meets creativity
From there, the creative engineering begins. Students use image processing and computer vision algorithms to morph these features in real time. Eyes can grow bigger, your skin becomes smooth like a baby, or turning a frown upside down by using geometrical transformations in image processing.
Gestures like a raised palm or a peace sign can be programmed to trigger animations (flames at of the finger tip) or switch between effects (virtual glasses and hats). It’s interactive digital art, built line-by-line in Python.
More than a filter
What looks like playing is actually serious computer engineering. “Our students combine math, algorithms and coding with creativity and turn technologies into charming digital innovation”, says Dr Niu. The students don’t just learn how existing apps work, but learn how to invent new ones. And the next time a filter sticks perfectly to your face? You’ll know: somewhere, an engineer taught a computer how to see.
Student work from Oscar Peters
