Document Layout and Formatting Helper for Blind Authors

This project, funded through NSF CAREER award, aims to facilitate independence for blind authors in producing documents that meet the presentation standards expected by sighted readers. This includes the development of an impact-weighted taxonomy of common document presentation errors of blind authors, an exploration of blind persons' mental models and strategies for learning and coping, and how these models and strategies contribute to the success of independent document formatting and layout activities. These will inform the design of an integrated solution to document preparation for blind authors in the form of a Microsoft Word compatible formatting and layout checker, as well as a fix to the compatibility problems between Word's grammar and spell checkers and commonly used screen readers. This tool will be complemented by an accessible content-sharing environment for blind authors with an interface for sighted help from Amazon Mechanical Turk.

Morales, Adams, Kurniawan.

LASSIE (Live-in ASSistance for Independence and Eldercare) 

This LASSIE robot is designed to be a monitoring assistive living robot, which is low cost and can be steered over the Internet by family member to remotely monitor and help assess the well-being of an older relative living alone. A commercially available iRobot Create platform was used as a starting point for the Assistive Living Robot, however the system could potentially be used on another robotic base. Our system (i) takes advantage of commercially available systems to reduce development cost and effort, (ii) acts as a video and audio communication tool between older persons and their family members or caregivers and (iii) can analyze the video feed to detect heart rate and breathing rate. The proposed system could be integrated with in-home monitoring sensors (Smart Grid, Professor Patrick Mantey), which could be the trigger for alerting family members that an out of ordinary event is occurring, therefore, our robot could be woken up and sent to be used as over the Internet watchful eye.


Blind Photography

Some current applications can help blind persons access a single photo but they are not effective for browsing a photo library or sharing photos. We ran a survey to understand blind persons’ problems with photo organizing and sharing and reviewed current applications for these activities. Based on our analysis, we designed a new interaction technique that allows blind users to easily browse through a photo library. This cellphone-based application reduces the number of steps to retrieve a photo, automatically provides non-visual contextual information about the photo, and organizes the photos.

Adams, Kurniawan. In collaboration with Harada, Sato, Asakawa, Takagi, IBM Research - Tokyo.


Speech Therapy Game for Children with Cleft Palate

We are developing a video game to aid young children undergoing speech therapy after cleft palate surgery. Children undergoing speech therapy must first unlearn their compensations and ommissions that they use to get around certain syllables they cannot do. Unlearning involves significant amounts of practice at home, and parents have extreme difficulty in motivating children to practice. Working directly with children who are undergoing speech therapy, we are designing a simple game using a novel speech recognition engine to help motivate them to perform their therapy exercises. We aim to accelerate the rate of recovery and give therapists more tools to use with children. We also intend to forward research in educational games.

Rubin, Kurniawan. In collaboration with UC Davis Medical Center.

Digital Birth


Labor and childbirth is a multidimensional experience for which it is difficult to prepare (without having done it before). It has been shown that emotional, informational, and physical support is key in shortening labor, increasing maternal happiness, and decreasing the need for unnecessary interventions during labor. This mobile (iPhone) video game training tool aims to train birth partners about the stages of labor and ways to support a first-time mom.

Holloway, Kurniawan. In collaboration with Shaw-Battista, School of Nursing, UCSF and Moodie, Anthropology Dept, UCSC. Research funded by CITRIS.