Originally Posted by
MoAlpha
And whom does this surprise?
The designers have to get that ~2:1 passing ratio right. Dangerous to put vehicles that behave atypically out there.
Just to be clear, this study did not say that vehicles will show this biased behavior. It shows (IMO) that biased behavior is possible.
The question for vehicles is, will cars accurately detect all objects actually located in the field of traffic, classify all the pedestrians as such without errors, and determine the relevant aspects of the absolute and relative motion of the pedestrians to determine if they are placed at hazard by the motion of the host vehicle. I don't think this can be assessed with accuracy when the test did not involve any actual prototype or production vehicle sensors, did not involve any vehicle-based vision-processing and object detection/classification algorithms, and did not use any actual automotive object processing.
The car does not need to identify whom has been detected (Congress person versus convict). It does need to determine if it is a person, if it is in-path or when it will be in-path, and how the risk of collision changes with relative vehicle-to-object motion. That's not the problem these researchers set for themselves.
Other vision systems in cars look at the driver (for SAE automation levels 4 and below) to determine alertness, direction of gaze, and other characteristics that could assess safety of driver actions. While these look at the face, they also do not need to recognize who the driver is nor the race, gender or size. Race, gender, size, clothing, hairstyle, cleanliness, cosmetics, health, morphology variations natural or not, eyewear and possibly other points affect these determinations, but the basic requirements are that the head pose, eyelid status, possible gaze direction, and eye motion be detected. Not identity.
Sometimes the real problem is easier than what the researchers have available to them.
But, it is necessary to impose design and compliance/verification/validation testing requirements for such systems to direct the designer (including his/her company) to ensure no loss of relevant sensor performance exists, and to verify that the performance is effectively equal across the population. In the United States the only regulatory agency that is able to constrain automotive design is NHTSA, and their Congressional mandate only permits them to act reactively. The US structure of liability law values that safety-related design must follow state-of-the-art practices to ensure safety, or due diligence might not have been followed. This usually motivates companies within the US auto industry to "do the right thing," but ... it's not a regulation. Europe mandates following state-of-the-art practices and verifying that such has in fact been done, achieving firm regulation to their standards.
Ensuring equality across the population of design teams does not necessarily solve the problem (how do you make sure that all design teams have a member who has had significant facial injuries, for just one example requirement?), because all industrial technical professionals are subject to pressure from the corporate structure. An engineer from India, one from Africa, one from USA, and one from Asia are all subjected to cost and timing pressure when performing product design. And all are capable of eliminating biases from their thinking in their work, if they actually have any biases. And being human, imperfections can creep in despite best efforts of all involved.
This paper shows a potential for biased behavior and that training methodology may help to remove it, but it is not directly applicable for vehicles as it is.