By Alejandro Gloriani and Cezara Maria Muresan
Eye tracking is a powerful technology across industries – from safety and efficiency in manufacturing, to improving human perception and performance in transportation and defense, to gathering attention data in sports and research environments. While this technology continues to advance rapidly, accurately detecting and tracking eye features, in real-world conditions it still remains challenging
There are several methods to track eyes, but a common and effective approach involves tracking the pupil — the dark circular opening in the center of the eye. Pupil tracking is particularly important because it not only provides information about where the user is looking, but changes in pupil size are also correlated with certain underlying brain functions. In this article, we focus specifically on pupil tracking, as it is the core technique behind most mobile eye trackers.
In practical, day-to-day scenarios, pupil tracking doesn’t always go as smoothly as in the lab. To minimize this, our smart. glasses integrate a range of well-established solutions to handle what we like to call “tricky eyes” – situations where individual eye characteristics or external conditions introduce complexity for the tracking algorithms.
1. SUNLIGHT & REFLECTIONS
Problem: Infrared (IR) illumination is used by eye trackers to increase the contrast between the pupil (dark circular opening) and the iris (colored part of the eye) independently of lighting conditions. However, IR light from the sun can interfere with the IR illumination used by most camera-based eye trackers. This is a common issue in outdoor or in-vehicle settings, where the filters in front of the eye tracking sensors can’t block all the natural IR light from the sun, sometimes creating strong reflections on the surface of the eye that can confuse sensors. Drivers or field workers using eye tracking outdoors may generate false readings due to these reflections.
In addition, bright sunlight often causes users to squint, partially closing their eyelids. Squinting affects the correct visualization of the pupil, making it harder for the system to detect it accurately. For example, when looking at a shiny surface on a sunny day without sunglasses, squinting reduces pupil visibility — a challenge similar to that faced by eye trackers under these conditions.
Solution:
- Using special sun shields, placed in front of glasses to minimize direct IR reflections. Available as part of our hardware ecosystem, these shields have been particularly effective in outdoor and in-vehicle environments. These special sun shields also block the sun’s ultraviolet (UV) radiation, protecting the user’s eye, preventing squinting, and improving pupil visibility outdoors.
- Switching the IR illumination wavelength used in the eye tracker to a more robust one against interference from natural IR radiation at sea level. By following this approach, the intensity of ambient IR light that can interfere with the eye tracker’s IR illumination will be considerably reduced, and unwanted IR reflections minimized.
2. SPECIAL LENSES
2.1 HARD CONTACT LENS
Problem: Unlike soft contact lenses, hard ones can shift independently of the eye, especially if they aren’t sitting correctly. This creates misalignment between the pupil, iris, and the hard contact lens, which may confuse the detection algorithm. In the eye-camera videos, the hard contact lens can appear off-center, a floating layer that disrupts the usual eye structure. In applications demanding high accuracy, like sports or surgical training, this misalignment can lead to tracking inconsistencies.
2.2. ANTERIOR CHAMBER INTRAOCULAR LENS
These special lenses are implanted in the front (anterior) chamber of the eye, between the cornea and the iris. Similar to hard contact lenses, the edges caused by these artificial lenses may confuse the detection algorithm. Additionally, these lenses can interfere with corneal refraction compensation, further complicating accurate pupil detection.
Solution:
- Use algorithms and machine learning models that are trained to recognize and filter out the artificial edges caused by these lenses, ensuring that tracking stays locked on the real pupil, instead of the lens boundary.
3. DROOPY EYELIDS (PTOSIS)
Problem: Whether due to relaxed facial posture, genetic factors or age, some users have partially closed eyelids that can obscure part of the pupil, making traditional pupil detection challenging.
Solution:
- Use a robust pupil detection algorithm, capable of accurately fitting the pupil’s shape, even when part of it is hidden.
4. LONG EYELASHES OR MASCARA
Problem: Long or fake eyelashes, especially with mascara, can create strong shadows or false edges that confuse traditional image processing methods, often misidentifying the lashes as the pupil or eyelid edge. This can lead to incorrect interpretation of the gaze.
Solution:
- AI-based pupil detection can, unlike classic edge detection, learn to distinguish between real pupil edges and cosmetic interference.
MAKING EYE TRACKING WORK IN THE REAL-WORLD. PRACTICAL TIPS FOR ROBUS, REAL-WORLD EYE TRACKING
In a lab, under controlled conditions, pupil detection is straightforward. But in the real world where lighting conditions are not controlled, people wear makeup, eyelids can droop, corrective lenses can shift, and people blink, squint, or glance sideways, it’s anything but predictable.
Here are some practical tips for ensuring reliable tracking in the field
- Understand your environment: Be aware of unique lighting and motion conditions in your workspace, such as vehicle cabins or sports fields.
- Manage lighting: Wear appropriate sun shields when working outdoors or in bright conditions to reduce reflections and improve pupil visibility.
- Ensure proper lenses fitting: Well-fitted smart glasses and contact lenses reduce shifting or misalignment during movement, improving tracking stability.
No two eyes are alike, and no two use cases are either. In environments like vehicle cabins, production lines, or athletic performance settings, conditions can be challenging, but with the right mix of optical tuning and algorithmic intelligence, eye tracking remains reliable, even when things get tricky.
Whether in mobility, manufacturing, sports, or research, these techniques ensure that eye tracking works not just in theory, but also in practice.
