In a bizarre case of AI misjudgement, a driver in China was repeatedly flagged by his smart car’s fatigue detection system for allegedly being drowsy — all because of his naturally small eyes.

Mr Li, a resident of Zhejiang, found himself bombarded with more than 20 alerts during a single drive in a Xiaomi SU7 Max. Despite being fully awake, the system’s voice and screen warnings persistently ordered him to “please focus on driving” and “drive safely.”

The culprit? According to Mr Li, the driver monitoring AI mistook his eye shape for signs of drowsiness, causing the alert system to go haywire throughout the journey.

AI Safety Features Gone Too Far?

Trying to stop the fatigue alerts, Mr Li made multiple attempts to widen his eyes and adjust his posture. But the moment he relaxed his facial muscles, the alarms resumed. He even recorded a video showcasing the issue, which quickly drew attention on Chinese social media.

Mr Li explained that when his sister drives the same vehicle, no such issues arise. The fatigue detection feature — powered by a camera on the steering wheel — is meant to monitor driver behaviour and detect signs such as yawning, eye closure, or distraction. But in this case, the system flagged false positives due to the driver’s natural facial features.

The experience, which Mr Li described as “ridiculous and frustrating,” has reignited discussions about bias in AI technology and its unintended consequences.

Fatigue Detection Systems May Be Disabled — But Should They?

Xiaomi Auto’s customer service responded that the AI driving safety feature can be turned off via the car’s settings menu. However, they advised against disabling it, as it serves to enhance road safety and prevent accidents caused by fatigued drivers.

This incident raises larger questions about the accuracy of autonomous vehicle systems, particularly when applied across diverse user demographics. As manufacturers push the boundaries of smart driving technology, there is growing concern that these features may not be universally inclusive.

With AI continuing to shape the future of transport, developers must address issues of algorithmic bias to ensure that vehicle safety features protect rather than inconvenience all users — regardless of their physical characteristics.