The NIDILRR-funded Rehabilitation Engineering Research Centers (RERCs) conduct a wide array of research and development in high- and low-tech solutions to help people with disabilities live, learn, and work independently. We’re hanging out in RERC Row for the 2021 RESNA Virtual Conference and this week we’re highlighting some of the work of this community. This article describes a collaborative project working on wearable cameras to help blind people interact safely with their environment and the people in that environment.
Wearable cameras may be helpful for blind people to gather visual information from their environment. Recently, a variety of wearable systems have been developed that can provide blind users with information about people near them, such as how close another person is standing or walking, whether the person is looking toward or away from the user, and features such as the person’s gender, age, and facial expressions. Some of these systems can also recognize the faces of familiar people as programmed by the user. These wearable camera systems can be useful to blind users in social interactions. However, because they are always on and may not be visible to bystanders, their use may raise privacy concerns for bystanders who may be filmed without their consent or knowledge.
In a 2020 NIDILRR-funded study, also supported by Shimizu Corporation, researchers at the Inclusive Information and Communications Technology Rehabilitation Engineering Research Center at the University of Maryland, College Park, Carnegie Mellon University, New York University, and IBM Research examined attitudes toward the use of wearable cameras by blind pedestrians. Specifically, the researchers want to find out how sighted individuals felt about the technology and about having their visual information recorded and transmitted to a blind pedestrian by a wearable camera. The researchers also wanted to know how comfortable the blind pedestrians were with wearing and using a camera to interact with sighted individuals, and what information would be most useful for such interaction.
The study was conducted in two parts: an online survey and an in-person experiment. For the online survey, 206 sighted participants answered questions about their attitudes toward the use of wearable cameras by blind persons to gain visual information from their surroundings. The participants answered these questions twice, a first time after viewing a video of a blind person using the device without knowing what kind of information the wearable cameras provided to the blind person, and a second time after viewing a video showing them what information the device provided to the blind user. Half of the participants viewed videos of someone using a visible wearable camera (a GoPro strapped to the head), while the other half of the participants viewed videos of someone using a discreet camera (smart glasses). Using a seven-point scale, the participants indicated the extent to which they felt “OK” or uncomfortable with the use of wearable cameras for assistive purposes or for any purpose at all, with being recorded by a wearable camera, and with having their images saved for future use. The participants also indicated the extent to which they felt uncomfortable with having basic visual features (gender, age) or more complex features (ethnicity, hair color, facial expressions) detected by a wearable camera and reported to the user.
For the in-person experiment, a total of 10 blind individuals interacted with 40 sighted pedestrians. Before the experiment, the blind participants answered questions about their experience and comfort with technology in general, wearable technology, and social interactions. For the experiment itself, the blind participants used a wearable camera prototype called GLAccess to locate and gather information about pedestrians near them. GLAccess consisted of a pair of smart glasses with an app that collected and processed images, and a Bluetooth earbud to convey audible information to the user. GLAccess could detect the distance, position, head pose, approximate age, and gender of nearby pedestrians, and also match pedestrians to images of individuals pre-programmed into the system. During the study, each blind participant walked through a corridor while wearing the system. Four sighted participants, two known and two unknown to the user, were instructed to pass by the blind user who then used feedback from the system to locate the sighted participants and ask them to read the room number on a nearby office. The sighted participants were told beforehand that the blind participant would be wearing a camera, but were not told what information the camera would provide to the user.
After this walking activity, the blind users again answered the same questions (as they did before the experiment) about their comfort and use of wearable technology and about the usefulness and usability of GLAccess system. They reviewed six visual features that could be detected by a wearable camera: a person’s age, gender, position, distance, head pose, and identity, and ranked them by importance. They also answered open-ended questions about the system: what they liked or disliked, what was easy or challenging about it, and what information would be most helpful to hear. The sighted participants answered questions about their attitudes toward wearable camera systems for assistive use by blind users. Then, they viewed the video of the user perspective used in the online survey, revealing what information the cameras provided to blind users, and answered the attitude questions a second time. Finally, the sighted participants indicated which of their features they would not want to be detected by a wearable camera.
In both parts of the study, the researchers found that the sighted participants generally had a positive view of wearable camera systems being used by blind people for assistive purposes. They were generally OK with their visual features being captured by a wearable camera. However, in the online survey, the participants expressed less comfort with the idea of being observed by someone wearing smart glasses than with the idea of being observed by someone wearing a more visible camera system. Further, in both the online survey and in-person experiment, the participants’ comfort levels decreased after they watched the user perspective video and learned what visual features were being captured by the cameras. For example, one participant wrote at the beginning of the in-person study, “They can wear whatever they want” about wearable cameras. However, after watching the user perspective video, the participant’s attitude became less positive, and they wrote that “I feel like they’d be invading my privacy.” In particular, the sighted participants were generally OK being recognized by wearable camera users they knew, but felt less comfortable being recognized by a stranger using a wearable camera.
When asked about the specific visual features that would be most useful, the blind participants described a pedestrian’s body position and distance from the user, and head pose (looking ahead, down, away) as an indication of where they may be looking as the most important features to know. For example, one blind user described how knowing whether a pedestrian is looking at the user or looking down at their phone could aid the blind user in getting the pedestrian’s attention. The blind participants also reported that information about a pedestrian’s age and gender was less important. While the sighted participants were generally more comfortable with a wearable camera detecting their head pose or distance than with having their age or gender detected, about a third of the sighted participants reported that they would not want a wearable camera to convey their head pose (eye-gaze) or facial expression to a user.
The authors noted that conflicts of interest may sometimes arise between wearable camera users and others in the environment who may be recorded by these cameras. For example, the participants in the first study reported feeling less comfortable being recorded by a discreet camera than by a visible camera. However, blind individuals may prefer to use less-visible smart glasses because they are more discreet than a more visible camera system. Further, while blind users may benefit from receiving information about another person’s head pose or facial expression to determine if they are available for social interaction, some individuals may feel uncomfortable being watched by a wearable camera or having their visual features recorded and conveyed to a user without their consent. The authors suggest that the more controversial features of age and ethnicity were less important to the blind participants, so developers may want to omit these from future systems. Assistive technology developers may wish to carefully consider the unique perspectives of both potential users and potentially impacted bystanders when designing new assistive devices to convey visual information to blind users.
To Learn More
To see videos of the experiment, visit https://iamlabumd.github.io/chi2020_lee/
NavCog and CaBot are other examples of mobility technology under development to help blind people navigate indoor spaces. NavCog uses smartphone technology and available maps of indoor spaces such as museums or airports to provide turn-by-turn audio directions. CaBot is a suitcase-shaped robot with on-board artificial intelligence that can guide the user through open spaces toward a destination, such as an airport gate. Learn more about these technologies.
To Learn More About this Study
This study is part of the Inclusive ICT RERC and it is a collaboration between the Intelligent Assistive Machines Lab at the Trace R&D Center at University of Maryland, College Park and the Cognitive Assistance Lab at Carnegie Mellon University.
Lee, K., Sato, D., Asakawa, S., Kacorri, H., & Asakawa, C. (2020) Pedestrian detection with wearable cameras for the blind: A two-way perspective. CHI’20. This article is available from the NARIC collection and in full text from the publisher.
Collection Spotlight from the National Rehabilitation Information Center 