Deaf-Mute Communication – Part II: Background Research

This part concludes the project report for the project “Deaf-Mute Communication”. Previous posts here: Part I, Part III.

Background Research

Sign Language

Just like spoken languages, sign language is built upon certain rules of grammar, and can also be varied in terms of dialects. But unlike spoken languages, dialects in sign language are not as comprehensible between people using related dialects. For example, British sign language is almost unintelligible for users of American sign language. On the other hand, American sign language has a similarity of 60% to modern French sign language, which demonstrates the difference in relationship and affinity between different sign languages and the equivalent spoken languages.

Moreover, what should be emphasized in this context is that many signs are very similar to each other, which can make the distinction hard to perceive. Examples of similar signs in American sign language are:

• chocolate, church, and computer
• soft and want
• now and today

Technology

To bridge the communication gap between deaf-mute people and the people in their surroundings, there are some current methods put in use. Below is presented some of the technologies that are used to carry out these methods together with technologies with the potential of being used in future communication solutions.

Augmentative and Alternative Communication (AAC) is an umbrella term for all types of communication enhancing methods used by people (except oral communication) to express themselves to others. This includes body language and facial expressions. AAC  is further categorized in Unaided communication systems, where the user’s body is relied on for communication, and Aided communication systems, where devices ranging from pencils to computers that produce voice output are utilized in addition to the user’s body.

Different kinds of relay services are utilized to help deaf-mute people communicate with hearing people. Video Remote Interpreting (VRI) and Video Relay Service (VRS) are two similar services, where VRI is used for communication between people at the same location, and VRS is used to interpret messages between people at different locations. Both services rely on Internet and video communications technology, since the interpreters are never located at the same location as the people using the service. Text Relay Service (TRS) is similar to VRS, with the difference that the output to the person with hearing impairments is text instead of video. Keyboards or special assistive devices are used to send text messages to standard telephones via the telephone line. IP Relay Services are web-based, similar to chats, and do not rely on telephones at all. Thus, callers have to manually supply the operators with their location information during situation such as making emergency calls.

The Leap Motion is a sensor device that monitors hand and finger motion in order to use these as input to a computer, e.g. to control different kinds of interfaces. The device was initially created to overcome the cumbersome process of 3D modelling with mouse and keyboard as input devices, but is currently used within a large area of use, such as controlling computer games, web browsers and virtual musical instruments. Recent attempts have also been made to use the Leap Motion as a gesture-based sign translator for online chat applications.

The Kinect is also a motion sensing device, but unlike Leap Motion, Kinect monitors full-body motion. Usually the sensor device is positioned on top of a monitor, which displays the interface of whatever is being controlled with the sensor device. Recent efforts have been made to build a sign language to text/text to sign language translator using the Kinect for sign language input.

Machine translation is a subfield of computational linguistics that investigates how to translate text or speech from one natural language to another. Great efforts are currently being put into making results produced through machine translation more accurate. Corpus linguistics and statistical techniques are utilized to be able to recognize whole phrases of text or speech instead of the single words by themselves.

Interview

We conducted an interview with a man who is hard of hearing and who uses American sign language. The questions were focused on which means of communication that deaf-mute people utilize when communicating with hearing people or with people who do not know sign language. We learned that assistive tools such as pen and paper, computers and text relay services were used to lower the communication threshold. The conclusion was that although the mentioned tools are slow to use, they are valuable and better than nothing.

Tell me about autism

Our project for the course ends with this post. Outside the course, we will continue to gather information so that we can increase autism awareness. Although our results suggest that there is more general knowledge about the ASD than we thought and expected there is still work to do. “Universal Design and Evaluation” has served as a springboard to provide us with some basic knowledge for this task. As it is simply not possible to understand and fathom what autism is in just 10 weeks of course, we had to focus on a more concrete topic. So what were we able to do in this short period of time?

First we had to decide what to study. After considering some kinds of impairments, we concluded that autism was a good choice. After all, the course is about opening our minds: trying to understand the needs of someone who has problems understanding his/her own feelings and/or communicating in general sounded like a task that involved creativity and work. And we were willing to understand. You may read more about this stage of the project in our first post.

We suspected that autism awareness was low. To confirm (or not!) this, we conducted a survey. The results and a brief analysis of them can be found in this blog. As it was an online survey, we included questions that could be fast to answer in order to promote and increase participation. For a more qualitative study, we asked people on the street three questions related to our topic. The format of a live interview allowed us to obtain more improvised answers, as well as know that their responses were not part of a Google search. More information about these interviews is included here.

But what did we know about autism? A Swedish neurologist helped us understand what happens since one child does not reach a developmental milestone or some other aspect of his/her life that draws the attention of an adult. We explained how they are diagnosed and what happens next in this post, which also includes a brief definition of the impairment and its causes.

For our research, Arthur and Carly Fleischmann’s book was very helpful: not only does it give a personal point of view of what is like to live with autism, it also shows how the help autistic people can get to help them communicate has evolved in the last 20 years. Although we were aware that Carly’s was not the “typical” case of autism, we believe that many parts of her story apply to the children in the spectrum.

Maybe part of the aggressive movements an autistic child may do are to the frustration of not being able to communicate their needs. That hit us as a huge need, so we decided to study how to improve communication for young autistic children (as early intervention is crucial). We saw that there were already methods and tools to help children communicate, but we also thought that there is always room for improvement in this kind of field.

Children first start using the system we have already described: pictures that represent concepts (e.g car, go) have a velcro square attached in the back, so that they can be arranged in a velcro tape, showing a series of actions if needed. The pictures can be used on their own: if the child wants juice, he/she can point at the “Juice” picture. The good thing about this system is that the therapists have almost endless freedom regarding what to include as pictures, as they are easily made. However, these pictures can be easily lost and damaged.

Technology can help solve this last problem and improve the mentioned advantages as well. When the therapist believes that the kid is ready, they will move towards a high-tech solution after trying the low-tech one. There are communication devices that can be provided to families, such as the one shown here. However, as these devices are only used by these kids and have to be carried at all times, they are stigmatizing. That was why Carly asked for a computer instead, even though it was not yet as good for her needs. Nowadays tablets fulfil this in a better way, as they are more portable. And which kid doesn’t want to have one?

The apps for these devices included pictures sorted by category, “text” prediction, and are easy to use regarding motor skills: the kid (or the therapist) only needs to put a finger on the part of the screen that displays a particular picture. However, there are levels to get to a particular picture and we believe the number of steps to get there could be reduced. Some therapists suggest having words sorted by situation rather than by topic. As an example, instead of having words in the category “furniture”, they could be arranged in the category “at school” or “at home”. For us, it would be even better if each therapist could arrange the pictures as he/she desired, as each case is different and at every stage the number of words used will be changing. Also, they should be able to add their own pictures, as some children prefer seeing a customized image rather than a generic one.

Though the therapists may interact with the device in a more complex way, it is important to remember that we should design not only for their convenience but also for the children’s. As a source of motivation for this thought, here is a quote from the Fleischmann’s book:

““Why is it that they make these kids look so disabled?” she once asked me, referring to how teachers […] dressed them for the convenience of the caregivers in bibs and easy-fastening clothing.”

All these pictures take a lot of memory. As in any IT project, the ideal goal is a system that consumes the least amount of resources. Having our users, (in this case the children and their therapists), limited by resources could negatively affect the learning of the child. What would have happened if Carly’s therapists have removed the text function to have more room for pictures? We should keep in mind that we are adding new ways of communication, not substituting them.

The Swedish neurologist emphasized that it is important to give a reward to the children after they have worked with the pictures. This is done by including a picture at the end of the series so that a reward is the final element. For example, after going to the Emergency Room and having their weight, breathing, etc. checked, the last picture the child will see will be one of a cone of ice cream. Eating this ice cream is their reward for their hard work. Why not include that in the device? We propose gamifying it a little, by adding collectable stickers with rewarding messages. These messages should acknowledge the hard work. For example, “You are a hard worker!”. Also, they should come in different themes, as every kid has a taste of his/her own.

The autistic world is a man’s world. Out of every 4 autistic children, 3 are boys and only 1 is a girl. Some parents complain that this is noticeable in the aid they get. Are we taking this fact into account? For example, for a chart to share the reward stickers, we should have different kinds of themes. We can find themes based on football, others based on fairies,… We should have variety, gender-independent. And, of course, never suggest that one theme is just for one gender.

What about the voice of the device? In the video we can observe that it is an adult’s voice, not a child’s. This is highly stigmatizing and stigmatization is especially important to be avoid in children, as it could lead to bullying. Having some different kids voices to choose from should be easy to add and far less stigmatizing.

What challenges did we encounter during the project? The first one was thinking how to overcome a barrier: how could we communicate with someone who can’t communicate? And then we realized: can’t? It is not that they are not able to communicate but rather that they do not know how to. It is not that they are not able to communicate but rather, and simply, that they cannot do it in exactly the same way as “we” do. We realized, luckily in time, that our own language was already cutting out solutions and possibilities.

Autism may not come alone. It is not unusual that the child also has some level of mental retardation. This adds new challenges to their learning. Also, what happens if the child is blind too? There are possible solutions for this that sadly had to be left for future work.

These are the things we know. However, sadly there is too much we do not know yet. Carly could say, years later, that using the computer was not a very good solution because she did not like how touching the keyboard felt.

If we improve communication for autistic people, we increase the chances of them explaining their needs to us, so that our next designs cater those better. After all, design is about listening to your users.

Autism and its Mystery

According to webmd.com, a website specialized for providing health information services, Autism is defined as a “complex neurobehavioral disorder that includes impairments in social interaction and developmental language and communication skills and rigid, repetitive behaviors” (Web MD, 2013).

It is believed to be a genetic disorder that is detected in the first three years of a child’s life. No one knows the actual reason behind this disorder, but the genetic factors may be the most significant cause for the Autism Spectrum Disorder (ASD). Some environmental factors have been reported to participate in autism or worsen its symptoms. These factors include alcohol, drugs, smoking, pesticides and more.

In Sweden, the treatment involves no cost for the family, as it is included in the health system or is offered by the community. The process starts with a worried teacher, nurse or parent, who notices that a child is missing milestones in his/her development. After the diagnosis, neurologists, psychologists, occupational therapists and other professionals will team up to assist and evaluate the child.

Sweden aims for inclusion: an autistic child goes to a regular school. He/she may need an assistant there and, in some cases, also at home. These assistants receive guidance from the health care team.

Therefore, autism does not only involve the autistic child, but also his/her family, the health care team, and, indirectly, the community and even the country. Design to include those affected by autism means, thus, thinking about a whole community.

However, this community may lack awareness of the ASD. In some cases, this lack of awareness may be especially detrimental for the autistic person. One of the most important examples is the case where children go undiagnosed, therefore getting no treatment. This delay is highly harmful, as the earlier the treatment starts, the higher its success rate is. In other words: early intervention is a vital issue.

Lack of information is not the only problem. In some cases, there is misinformation even when research states the opposite belief. This is the case of the idea that vaccines and autism are linked. Despite the fact that numerous medical researches have not found any relationship between them, some parents stay skeptical.

People in the severe end of the autism spectrum are born, and are likely to remain, non-verbal. In this project we will be focusing on the early stages, due to importance of early intervention. We will be then focusing on these children as target group, to find solutions to their need of communication. Our research takes into account ways for inclusion, non-stigmatization and addition of options rather than changes of them.

References:

http://www.webmd.com/brain/autism/searching-for-answers/vaccines-autism

http://vimeo.com/8182809

webmd.com