Link

Project Report

Our group made a project about how you could develop smart homes for impaired people. This is our finished report.

The Internet of things and its uses for people who have an impairment

Introduction

The Internet of things is a great solution for a wide variety of impairments. It is considered to be the future of technology and that should make it widely used by everybody, no matter if they have impairment or not. Already this fact should minimize stigmatization, as both people without and with impairments should be able to use the technology.

What is the Internet of things?

The term is rather new and has been around for about four years. It implies equipping objects that normally surround users on a day-to-day basis with tiny devices that collect information. These devices are connected to the Internet and are capable of drawing information from the World Wide Web in order to add value to the local information collected from the user. All this information is used in order to offer a service that asks from users a minimal or no effort at all.

The ideal dimension of the Internet of things makes it possible to imagine that objects can easily communicate between themselves and work together in order to offer services that are desired by the user but which are asking for only minimum input.

Among the technology used for supporting the Internet of things there are often listed Radio-frequency identification (RFID), near field communication, barcodes and digital watermarking. This makes the technology seamless and that is exactly what a great design should be. The active input from the user is minimum as the information is drawn through other methods. The result is maximal, as the possibilities that we have today could be expanded and improved, while the user’s effort is minimised.

The technology would be useful to people who don’t have an impairment but also for does who do. It would find its place in all homes. Most advantages seem to be for people who are blind, although not only.

One of the most common examples of how the Internet of things would work is that of a fridge that can recognize which products it contains and even order new food when it is about to finish. For example, the fridge would notice by itself that milk is almost over and order a delivery for the next day. Such a scenario sounds like a luxury for people who are not impaired but could be a very good supportive tool for people who are blind.

The great aspect about the Internet of things is that the technology is not present today and the leaves spaces for imagination to run wild. On the other hand, Internet is around for only a couple of decades. The iPhone has been launched only 5 years ago and is already mirrored in most other mobile devices present on the market. The iPad is here only for 3 years and it is difficult to think of the time when it didn’t exist. Simple and intuitive devices make their way into day-to-day activities leaving the feeling that they were always there.

From this aspect it is fair to say that the prognoses for the next years, for example, can and should be as innovative as possible, even if they are not possible today. As long as their usability level is incontestable they might just make their way through as solutions that feel like they have always been there.

Possibilities

The Internet of things is not one singular solution, but a cloud of different possibilities that have similar technology as base. Because of that, it is difficult to say how this cloud could take shape; it is just as difficult to cover it.

Covering all possibilities that the Internet of things has to offer in the subject of impairment design is not the object of this article. It is interesting, though, to take some ideal situations that reflect user needs and project possible solutions.

In order to identify relevant user needs I have looked at forums and articles that are discussing experiences and problems that people with specific impairments are facing. Conducting user studies would of course help give better shape to these scenarios, but that can be part of a larger project.

The path towards living in a world where Internet of things is present has already started taking shape, as there are companies that are focusing on developing such ideas / products.

Memory loss

–       device which reminds about switching off the oven, for example, dangerous situations, explains sounds such as the fire alarm, doorbell. The device should be scalable to the type of memory loss level by an assistant so that it matches the person it should support.

–       device which answers on questions which are asked repetitively

–       finding lost objects. All objects should have an RFID tag that could be charged with various information. Finding out the keys’ position, for example, should also be just as easy as searching for them in a search bar and seeing on a map of the house where they are placed. A sound recognition system could be programmed to react on the question “Where did I put my keys?” and a couple variations on that with displaying a map of the flat with the position of the keys. Or a green light can be projected in the room on the spot where the keys are. It feels like the possibilities are endless.

–       Finding out the history of objects that are found in the house. This can be seen as a Tale of Things and help rebuild reality as often as it is needed. (http://dl.acm.org/citation.cfm?id=2434674)

Visual impairment

–       sensors on a belt which detect obstacles (how many steps until obstacle)

–       interacting with a serious of services through voice recognition

–       recognition of objects, their type, could be done through a table which has RFID recognition integrated which recognize the RFID tags of objects, together with the data that is loaded on them

–       recognition of persons

–       recognition of id cards, for example a policeman’s ID, which reduces the risk of being tricked

–       recognition of sounds

–       each building can have a map registered in an RFID tag at the entrance that an own device can connect automatically and provide sound options for finding specific destinations in the building

Scenarios

Some simple scenarios I found illustrated in the following article, where the following three illustrations are also extracted from:

http://lms.ctl.cyut.edu.tw/sysdata/22/27122/doc/571c22f5f635d883/attach/1523813.pdf

Google Glasses is already one step ahead the ideal situation envisioned in this research from 2012. What the article describes as an inserted artificial retina with a camera function can today be replaced by a pair of Google glasses that is connected to the Internet. No need to for surgery and insertion of technology into human body.

Shopping scenario

The shopping scenario is very interesting. A person who is blind can find out which fruits are displayed, or maybe if there are strawberries, where are they and how fresh they are through RFID tags. A class colleague was observing in a blog entry that it is impossible for a person who is blind to tell what cans contain and prices in general. RFID tags could contain all such information and more, such as validity date, country of provenience and maybe even ingredients, information which could be additionally provided to the user on request. More important, finding products on a shopping list could be possible by being indicated through a blind navigation system. This could use headsets indications or vibration response in the cane. And this could be used by persons who don’t have a disability as well. Who wouldn’t like to get some help when searching for their favorite type of packaged nuts or the soya sauce in a new supermarket?

Paying is also easy, as the person who is blind could check at all times which is the accumulated sum of products that are in his shopping cart. It is that sum that she/he knows should be paid in the end. The payment should then take place through mobile phone. The payment process should be as easy as possible. At the cashier the total sum is read out loud and authorized with a fingerprint recognition that the latest iPhone version already supports.

MIR:ROR

The possibilities are abundant. RFID tags can beset on any object and can be easily programmed to deliver any type of information may be needed when they are read. Some more inspiration can be found in the following link:

The MIR:ROR is not especially adapted to the needs of persons with impairments. Having to go to a website in order to program every stamp is tedious, especially as the website has not taken concern to non-exclusive design. What the MIR:ROR does is open the door for possibilities, as the technology is already present.

Indoor navigation system for people who are visually impaired

An indoor navigation system can also be developed. GPS has its limits when used in a closed indoor environment. I found a study that combined the use of Internet with RFID passive tags, Mobile phones with integrated Near Field Communication (NFC) interface, Voice-based navigation (they used Text-To-Speech (TTS) engine) and a System on Chip (SoC) for implementation of the Control & Monitoring Gateway.

More can be read about it at the following link

http://dl.acm.org/citation.cfm?id=2383322

Existing indoor navigation systems are limited and often disregard changes in the room’s layout. The system presented above, together with the possibility of connecting to a building’s data system with maps and that of identifying near objects through RFID tags, can make it possible to navigate without help in new environments as well. The system can be made as unnoticeable as it may be desired. Owning a cane is not a necessity. It can be used for signaling that the person is blind or for extra security, in case technology ever fails.

Conclusions

The Internet of things is a new concept and often looked upon as a futuristic layout of humans’ interaction with computers. Gathered information is scarce and still highly assumptive. At a closer look, though, technology already answers a high amount of that which is seen as desired scenarios. The tools are already here for making the scenarios possible. It’s just that they are not yet used at their best or that they lack a common language that would then help them communicate in order to deliver high results.

The technology is not particularly targeted to people that have a disability or to people that don’t have one. Everybody could profit from it and that is a clear advantage against stigmatization.

Project conclusion

Our theoretical product
A game controller solution for people who can’t make fast and precise moves using their hands. They might not even have hands. To allow maximum adaptability to the lowest cost, the product is just a safe puzzle of electronic components that allow the users to easily build their own controllers. When used with a PC, it should be possible to emulate a wireless Xbox 360 controller to support more games.

Safety
All components are connected in a way that can’t give electric shocks or burn components as long as you only use unmodified original components. Making own designs that are not approved in the example book is done at the user’s own risk since you can do very stupid things like a sharp kitchen knife controller.

Sweatband controller
A regular sweat band on the head, arm or foot allow precise controlling with any available limb by attaching a wireless core without any wires. This is a lot like the Wii controller but you don’t have to hold it.

Earphone controller
The earphone controller have a wire to allow more energy consuming things to be attached. It cost more but give a closer experience. This controller is mostly for playing motorcycle games where it feels natural to steer by tilting your head sideways. The 3D glasses allow tilting the head while keeping sight of the game. Rotating the head allow getting more information about where a sound is comming from in the game.

Hunting rifle controller
Hunters who have a broken hunting rifle can rebuild it to a game controller so that the product will sell to more users. Just attach the components, paint the barrel orange, make sure that it can’t be rebuilt to working condition and check if your laws require anything else before giving it to someone without a license.

Brainscaning
The brainscaning tecnology is already out there with a few cases where they have used it for the same purpose that we suggest. But this is part of a puzzle and is even if designed with dissability in mind not directly aimed at a dissability group but could be used by anyone, thus reducing stigmatization a bit. And the core idea is alot of components that work together but are not directly dependent on eachouther so you can use only the one you want or need in the way you want. We only gave a few examples of how they could be used the rest is up to the user.

Group: David Piuva, Rickard Lööf

Deaf-Mute Communication – Part II: Background Research

This part concludes the project report for the project “Deaf-Mute Communication”. Previous posts here: Part I, Part III.

Background Research

Sign Language

Just like spoken languages, sign language is built upon certain rules of grammar, and can also be varied in terms of dialects. But unlike spoken languages, dialects in sign language are not as comprehensible between people using related dialects. For example, British sign language is almost unintelligible for users of American sign language. On the other hand, American sign language has a similarity of 60% to modern French sign language, which demonstrates the difference in relationship and affinity between different sign languages and the equivalent spoken languages.

Moreover, what should be emphasized in this context is that many signs are very similar to each other, which can make the distinction hard to perceive. Examples of similar signs in American sign language are:

• chocolate, church, and computer
• soft and want
• now and today

Technology

To bridge the communication gap between deaf-mute people and the people in their surroundings, there are some current methods put in use. Below is presented some of the technologies that are used to carry out these methods together with technologies with the potential of being used in future communication solutions.

Augmentative and Alternative Communication (AAC) is an umbrella term for all types of communication enhancing methods used by people (except oral communication) to express themselves to others. This includes body language and facial expressions. AAC  is further categorized in Unaided communication systems, where the user’s body is relied on for communication, and Aided communication systems, where devices ranging from pencils to computers that produce voice output are utilized in addition to the user’s body.

Different kinds of relay services are utilized to help deaf-mute people communicate with hearing people. Video Remote Interpreting (VRI) and Video Relay Service (VRS) are two similar services, where VRI is used for communication between people at the same location, and VRS is used to interpret messages between people at different locations. Both services rely on Internet and video communications technology, since the interpreters are never located at the same location as the people using the service. Text Relay Service (TRS) is similar to VRS, with the difference that the output to the person with hearing impairments is text instead of video. Keyboards or special assistive devices are used to send text messages to standard telephones via the telephone line. IP Relay Services are web-based, similar to chats, and do not rely on telephones at all. Thus, callers have to manually supply the operators with their location information during situation such as making emergency calls.

The Leap Motion is a sensor device that monitors hand and finger motion in order to use these as input to a computer, e.g. to control different kinds of interfaces. The device was initially created to overcome the cumbersome process of 3D modelling with mouse and keyboard as input devices, but is currently used within a large area of use, such as controlling computer games, web browsers and virtual musical instruments. Recent attempts have also been made to use the Leap Motion as a gesture-based sign translator for online chat applications.

The Kinect is also a motion sensing device, but unlike Leap Motion, Kinect monitors full-body motion. Usually the sensor device is positioned on top of a monitor, which displays the interface of whatever is being controlled with the sensor device. Recent efforts have been made to build a sign language to text/text to sign language translator using the Kinect for sign language input.

Machine translation is a subfield of computational linguistics that investigates how to translate text or speech from one natural language to another. Great efforts are currently being put into making results produced through machine translation more accurate. Corpus linguistics and statistical techniques are utilized to be able to recognize whole phrases of text or speech instead of the single words by themselves.

Interview

We conducted an interview with a man who is hard of hearing and who uses American sign language. The questions were focused on which means of communication that deaf-mute people utilize when communicating with hearing people or with people who do not know sign language. We learned that assistive tools such as pen and paper, computers and text relay services were used to lower the communication threshold. The conclusion was that although the mentioned tools are slow to use, they are valuable and better than nothing.

Tell me about autism

Our project for the course ends with this post. Outside the course, we will continue to gather information so that we can increase autism awareness. Although our results suggest that there is more general knowledge about the ASD than we thought and expected there is still work to do. “Universal Design and Evaluation” has served as a springboard to provide us with some basic knowledge for this task. As it is simply not possible to understand and fathom what autism is in just 10 weeks of course, we had to focus on a more concrete topic. So what were we able to do in this short period of time?

First we had to decide what to study. After considering some kinds of impairments, we concluded that autism was a good choice. After all, the course is about opening our minds: trying to understand the needs of someone who has problems understanding his/her own feelings and/or communicating in general sounded like a task that involved creativity and work. And we were willing to understand. You may read more about this stage of the project in our first post.

We suspected that autism awareness was low. To confirm (or not!) this, we conducted a survey. The results and a brief analysis of them can be found in this blog. As it was an online survey, we included questions that could be fast to answer in order to promote and increase participation. For a more qualitative study, we asked people on the street three questions related to our topic. The format of a live interview allowed us to obtain more improvised answers, as well as know that their responses were not part of a Google search. More information about these interviews is included here.

But what did we know about autism? A Swedish neurologist helped us understand what happens since one child does not reach a developmental milestone or some other aspect of his/her life that draws the attention of an adult. We explained how they are diagnosed and what happens next in this post, which also includes a brief definition of the impairment and its causes.

For our research, Arthur and Carly Fleischmann’s book was very helpful: not only does it give a personal point of view of what is like to live with autism, it also shows how the help autistic people can get to help them communicate has evolved in the last 20 years. Although we were aware that Carly’s was not the “typical” case of autism, we believe that many parts of her story apply to the children in the spectrum.

Maybe part of the aggressive movements an autistic child may do are to the frustration of not being able to communicate their needs. That hit us as a huge need, so we decided to study how to improve communication for young autistic children (as early intervention is crucial). We saw that there were already methods and tools to help children communicate, but we also thought that there is always room for improvement in this kind of field.

Children first start using the system we have already described: pictures that represent concepts (e.g car, go) have a velcro square attached in the back, so that they can be arranged in a velcro tape, showing a series of actions if needed. The pictures can be used on their own: if the child wants juice, he/she can point at the “Juice” picture. The good thing about this system is that the therapists have almost endless freedom regarding what to include as pictures, as they are easily made. However, these pictures can be easily lost and damaged.

Technology can help solve this last problem and improve the mentioned advantages as well. When the therapist believes that the kid is ready, they will move towards a high-tech solution after trying the low-tech one. There are communication devices that can be provided to families, such as the one shown here. However, as these devices are only used by these kids and have to be carried at all times, they are stigmatizing. That was why Carly asked for a computer instead, even though it was not yet as good for her needs. Nowadays tablets fulfil this in a better way, as they are more portable. And which kid doesn’t want to have one?

The apps for these devices included pictures sorted by category, “text” prediction, and are easy to use regarding motor skills: the kid (or the therapist) only needs to put a finger on the part of the screen that displays a particular picture. However, there are levels to get to a particular picture and we believe the number of steps to get there could be reduced. Some therapists suggest having words sorted by situation rather than by topic. As an example, instead of having words in the category “furniture”, they could be arranged in the category “at school” or “at home”. For us, it would be even better if each therapist could arrange the pictures as he/she desired, as each case is different and at every stage the number of words used will be changing. Also, they should be able to add their own pictures, as some children prefer seeing a customized image rather than a generic one.

Though the therapists may interact with the device in a more complex way, it is important to remember that we should design not only for their convenience but also for the children’s. As a source of motivation for this thought, here is a quote from the Fleischmann’s book:

““Why is it that they make these kids look so disabled?” she once asked me, referring to how teachers […] dressed them for the convenience of the caregivers in bibs and easy-fastening clothing.”

All these pictures take a lot of memory. As in any IT project, the ideal goal is a system that consumes the least amount of resources. Having our users, (in this case the children and their therapists), limited by resources could negatively affect the learning of the child. What would have happened if Carly’s therapists have removed the text function to have more room for pictures? We should keep in mind that we are adding new ways of communication, not substituting them.

The Swedish neurologist emphasized that it is important to give a reward to the children after they have worked with the pictures. This is done by including a picture at the end of the series so that a reward is the final element. For example, after going to the Emergency Room and having their weight, breathing, etc. checked, the last picture the child will see will be one of a cone of ice cream. Eating this ice cream is their reward for their hard work. Why not include that in the device? We propose gamifying it a little, by adding collectable stickers with rewarding messages. These messages should acknowledge the hard work. For example, “You are a hard worker!”. Also, they should come in different themes, as every kid has a taste of his/her own.

The autistic world is a man’s world. Out of every 4 autistic children, 3 are boys and only 1 is a girl. Some parents complain that this is noticeable in the aid they get. Are we taking this fact into account? For example, for a chart to share the reward stickers, we should have different kinds of themes. We can find themes based on football, others based on fairies,… We should have variety, gender-independent. And, of course, never suggest that one theme is just for one gender.

What about the voice of the device? In the video we can observe that it is an adult’s voice, not a child’s. This is highly stigmatizing and stigmatization is especially important to be avoid in children, as it could lead to bullying. Having some different kids voices to choose from should be easy to add and far less stigmatizing.

What challenges did we encounter during the project? The first one was thinking how to overcome a barrier: how could we communicate with someone who can’t communicate? And then we realized: can’t? It is not that they are not able to communicate but rather that they do not know how to. It is not that they are not able to communicate but rather, and simply, that they cannot do it in exactly the same way as “we” do. We realized, luckily in time, that our own language was already cutting out solutions and possibilities.

Autism may not come alone. It is not unusual that the child also has some level of mental retardation. This adds new challenges to their learning. Also, what happens if the child is blind too? There are possible solutions for this that sadly had to be left for future work.

These are the things we know. However, sadly there is too much we do not know yet. Carly could say, years later, that using the computer was not a very good solution because she did not like how touching the keyboard felt.

If we improve communication for autistic people, we increase the chances of them explaining their needs to us, so that our next designs cater those better. After all, design is about listening to your users.

Thoughts on the Concluded Interviews

In order to reach a solid conclusion of what people think about Autism, we have conducted interviews with five individuals that shared their opinions about autism and answered a few questions.

This is the list of the questions that we have asked them:

● What is Autism?
● If we’re designing a tool for autistic people, whose opinions should we ask?
● If you couldn’t talk, or couldn’t use your hands to write with a pen, how can you communicate with others?

It was very interesting to realize how each individual has a divergent point of view that helped us know more about other people’s responses regarding this disability. The main reason that intrigued us in conducting the interviews was that we wanted to benefit from the spontaneity of the interview process, as the interviewees did not have the luxury of taking their time to think about their responses.

Starting with the first question that asks “What is Autism?”, most interviewees had a similar response and they all related autism to problems with communication and social capabilities to a certain extent. In addition, one of the interviewees stated that: “If a person is autistic, then he or she has special talents despite their social problems”. Difficulties expressing one’s self was another response that we received as well.

“It might be a disability, but an autistic person might develop certain abilities that make them specialized in various things, things that they can do with great energy”. This is the response of a professor at Uppsala University who helped us in the research phase regarding Autism. His answer was very optimistic and we really liked it, as he showed the positive side of autism rather than the negative one.

In our previous posts, we talked about how one of the symptoms of autism is that in some cases, people with autism are not able to tell what other people feel, or in other words, show full empathy. We were glad to know that one of the respondents knew about this issue, defining autism as: “When a person cannot tell what other people’s feelings are”.

Being autistic does not mean that the person affected should be excluded from the society. This is why, in order to be treated properly, they must be included in the most adequate way. An answer that showed how some people believe that autistic people are thought to be away from the social standards is this answer: “Maybe they could be perceived as deviants in our society”. Who sets these social standards that everyone needs to follow? Just because a person does not know how to communicate does not mean that this person has nothing to say. This may be mainly what causes all this inside trauma that autistic people might suffer. Imagine being inside a body where you cannot express your thought or what you have in mind? It wouldn’t be easy, would it?

In the second question, we were glad to see that the vast majority of the respondents told us to ask the autistic people themselves, as they know more than anyone else. The interviewees also mentioned that asking the people that surround the person with autism (such as parents or therapists) is very important. Especially in the case where the autistic person cannot communicate yet. One of the respondents advised us to contact an Autism Center in Sweden in order to know more. As you know from one of our posts, we had talked to one neurologist who works with autistic children in Sweden.

The third question can be categorized as the hardest question, as it raised the eyebrows of most of the interviewees. The thing that surprised us the most among all the answers is that none of respondents mentioned the use of a computer or an app. Some said that using facial and body expressions can be one way to communicate if you could not talk or use your hands to write with a pen, while others mentioned gestures and pictures. The most interesting answer was this one: “Blinking signals and connecting a machine to the brain in order to translate what the person wants to say”.

In conclusion, these interviews were very helpful and infighting to help us get more acquainted with the general public’s opinion about autism. We used interviewees of different ages and professions to have a wider look as well. In fact, they made us realize that people are more aware than we expected them to be; however, it is a long way to go till total acceptance.

Deaf-Mute Communication – Part III: The Solution

Existing Solutions

Obviously, we are not the only ones who had the idea of using a gesture recognition device such as leap motion to read and interpret sign language. We have found two applications which are being developed at the moment that use Leap motion to read sign language.

• Motionsavvy intends to translate sign language to text.
• chat software which can be used with sign language.

Our Solution

The main purpose of our solution is to ease communication for deaf people in situations where they have to interact with people who are not able to speak sign language. According to our background research we have identified a set of qualities that need to be fulfilled in order to provide a usable device and with a good user experience.

Qualities

• Reads sign language and transforms it into spoken language or text
• Dictation feature to store a set of often used questions / phrases to increase communication speed
• Has an aesthetic design e.g. can be worn like a necklace
• Communication between device and mobile phone to make voice calls

Sketched device

The outlined solution could be prototyped by using a device called Leap Motion. Leap Motion is a small device that can recognize hands with high accuracy allowing interactions through gestures.

We have build a demo to demonstrate the reading of American Sign Language (ASL) and the transformation into text output. The demo uses Leap Motion’s Javascript API and is able to recognize the following words and numbers:

• What?
• here
• Book
• Sorry
• You
• 1
• 2
• 3
• 4
• 5

If you have a Leap Motion and want to try it out you can download the following zip file which includes an html file as well as the jquery and leap motion javascript libraries.

Download Demo

Potential problems

As promising as our solution might sound there are also certain problems attached. In order to operate accurately the device relies on image recognition. That means that the device needs to have a direct view on the hands of the user. Some words in ASL require the positioning of hands above each other which cannot be detected by Leap Motion. Additionally, different light conditions e.g. direct / bright sunlight can influence the accuracy of the device up to making it completely unusable.

Another problem can be described as the King Midas problem. King Midas had the ability to turn everything he touches into gold. The same problem occurs with such a device as certain everyday gestures or hand movement might be interpreted as a word resulting in unwanted voice or text output.

Potential for universal usage

The main purpose of our solution is to include deaf people which does not mean that the device cannot be used by other people as well. We can think of applications that enhance the day-to-day experience of all people.

• Dictation of confidential information in public places
• Writing text messages while walking

Blind man group – NavsMart

Requirements

The solutions should satisfy these prerequisites:

• Intuitive interface.

• Easy to use by visually impaired people and by people without an impairment.

• Normalizing (non stigmatizing)

• The solution should facilitate shopping task.

• Help the user to locate and distinguish groceries in the store.

• Guide the user in an efficient way through the store.

Solution

The solution that meets these requirements and solves the problem is an application. Since the solution is an application in a small device that is easy to carry along, that makes the application non stigmatizing. In this application there will be an intuitive interface for both visually impaired and people without any visual impairment. This is acquired by offering an interface that can be communicated with via touch, and also by voice commands. For the application to be able to tell the user the location of the desired groceries and the best route to pick them up, the user has to at some point make a shopping list that the application can use. The application should be able to retrieve changes to this shopping list at any time during the shopping activity if the user decides to add or skip items. The user should be able to distinguish between the different brands of each product and also get important information about the product.

The technological requirements for this idea is that the grocery stores will have to install some WiFi-hotspots throughout the store. This will not be a big issue for the grocery stores since some of them already have free WiFi for their customers, and the idea of free WiFi is much appreciated by the general public as well. These will be used to obtain an indoor navigation system by measuring distances between different routers in the store. For obtaining even better indoor navigation the stores will need to install Bluetooth-transmitters in the floor at intersections between shelves, so that the user can get information on how far ahead an intersection will be, and therefore be certain that he or she enters the correct aisle given by the system.

There are parts of the solution to make it easier for visually impaired already out there. These products are good but not complete and there are several things that need to be improved. Now a few some examples of products that are available for visually impaired.

ShopTalk

This is a solution that helps the customer finding the product inside the supermarket. It is a kit consisting of a barcode-scanner and a navigation tool that is attached on the back of the customer. There are a few problems here and the first thing that is most noticeable is the backpack. It is the tool that guides the person around the supermarket and is something that is needed for the system to work. Does a visually impaired person want to wear a backpack to expose himself that he is different. Stigmatization is a big problem and this solution does not take that away. To identify the product, a scanner is used to scan the barcode on the right shelf. This requires that the barcode is easy to find or else it could be very frustrating and time consuming.

oMoby – Shopping app for blind

oMoby is an application that helps identifying an object just by scanning an image of a product. This is a lot better than the previous solution where a barcode was necessary to find out the current product. It does not have the need precision because a picture of the whole product is only needed. This is not a application that guides the customer around the supermarket, so the person have to know where to find everything. In the video he mentions that sometimes when he knows that a person is around he prefers to ask what kind of product he is holding because it is easier, but it can be exhausting to always rely on another person.

Ica Handla – Shopping app

Ica Handla is a app to store and sort a shopping list according to the chosen store. It is a good way to plan the route around in the store. This application, on the other hand, does not have support for the visually impaired. This also requires that the user needs to know what to shop for before arriving to the store.

The application would suit best for a smartphone since a smartphone has a lot of the functionality that is desired. Smartphones are used by many people that will be able to utilize the application and therefore visually impaired people would be less likely to feel stigmatized.

The user makes a shopping list within the application. When the user steps into a grocery store the application will automatically sort this shopping list in the order the items are placed within this particular store and calculate an optimal route for picking up all of the items. If the user wants to skip an item on the route, he or she tells the application to ‘skip’, and it will move on to the next item on the list and guide the user there instead. A similar solution is used if the user wants to add an item to the list while on the route to an item in the store. In that case the user tells the application that he or she wants to ‘add’ and item, and says or types in the name of that item. The application will then sort this item into it’s appropriate place within the shopping list and add it to the route in real-time.

When the user arrives at the right position for a product in the store, the user uses the camera function and takes a photo of the product. The application informs the user about the product, what it is. The user easily choose the get more information about the product such as price, table of contents, nutritional facts, origin, allergy information and other certifications.

Reflections

For our solution to work some cooperation from the grocery stores are needed. First and foremost they need to equip their stores with WiFi-routers and Bluetooth-transmitters. Secondly they also need to maintain a pretty static schema of where the items are placed in the store, this schema also need to be updated by the employees of the store if they get a new product, if a product is removed or just moved to another shelf. We believe that most stores already have some kind of database where information about their items are stored, but if they don’t this is something that they will need to implement for our idea to work. The stores also need to install some kind of sensor at the entrance so that the application can be notified that the user steps into a store, and also which store it is.

There might also be some issues where the user has a problem aiming the camera correctly towards the product, but we feel that it still is a lot easier than to find a barcode to scan. As a visually impaired person it could prove quite difficult to find the barcode on a product, but if you already have the product in your hand it would not be so hard to point the camera towards that product.

We would also want to be able to from the picture of the item get information about the expiration date, since this information is connected to that particular item and therefore not general enough to be stored in the database.

We also had a phone interview with a visually impaired man named Fredrik Larsson. He told us what he thought of our idea and also recommended some extra functionality that he felt would be appreciated. For example he missed having the ability to find out what kind of products the store had to offer, and also he wanted to know what special offers there currently were, three for the price of two etc. He also told us that some visually impaired people used a regular buttoned cellphone and some used smartphones. Another thing he recommended was that the application should have a function so that it could warn the user if he or she shops for more money than they have left on their card (the card that is connected, ica-card).

Of course this is going to be useful for people without an impairment as well, especially when it is a store you have not been to before.

 Group: Conny Anderson, Mikael Svärdström and Jonatan Moritz

Autism Awareness Survey

In order to gain instant, adequate and accurate information from individuals, an online survey was distributed to a significant number of people. Thanks to Google Forms, we were able to distribute the survey to forty six people who were willing to help us with this project. This survey helped us in getting qualitative and quantitative information of the opinions of the general public regarding autism. To be frank, the answers of some of the respondents were astonishing, while other answers were somewhat expected. Below, a detailed explanation and comments of each question will be mentioned. In addition to the survey, we also questioned some people from different ages about Autism. Many people were quite hesitant to answer when they were first asked: “What is Autism?” It is an enigma. And to understand it is another mystery. Moreover, it is noteworthy to mention that we focused a lot on knowing what Autism is and trying to get what people think of it as it is very challenging to design something for autism without knowing the deep roots of this genetic disability.

Regarding the audience of the survey, it was answered by many people from different countries including Spain, Sweden, Jordan, Lebanon, Ukraine, Denmark and more. In addition, the survey targeted people from different age groups, in order to get more varied results instead of being restricted to one age group only (e.g. students for instance). One of our main goals while distributing the online survey was to try to get opinions from all over the world, not only one part or one continent. Autism affects people from all around the world, not only United States or Europe. It is present everywhere, and it is important to know the global awareness issues about autism before working on a solution.

The majority of respondents were students (78%) who study a field that is not related to autism. Yet 9% were students who study a field related to autism. In addition, 52% of the people who answered were shown to have no or small knowledge about autism. This means that more than half of the population are still not aware of the actual definition of autism, its symptoms and causes. Regarding the experience of people with autism, most people got their information from the media (30% selected the “I have read / watched videos about it” in the question that asks about the experience).

One good indication that shows that people somehow understand that autism is related to communication problems is the high percentage of people who chose “cannot communicate” as one of the symptoms of autism. In addition, the research showed us that people think that the most appropriate option is to include autistic children in normal classrooms regardless of their disability. It is true that they might need personal assistance in some cases, but it is vital not to exclude them from society, as that will only make things worse. Moreover, 47% of the respondents agreed that when designing a product that will be used by autistic people, we need to ask them as they are the experts in this field. 36% of respondents agreed that we should ask the caregivers, which also sounds reasonable as they are thoroughly acquainted with this disorder. In situations where the autistic person cannot communicate, the caregiver’s opinion might be the closest one to the autistic person.

All in all, the survey significantly helped us in reaching our next point in our project. The analysis on the responses gave us a clear image of what idea the population has about autism.

Group: Virginia Grande and Faris Halteh

Smelling Trouble: Potential Solutions

As a continuation of the previous post on the difficulties faced by those with anosmia in the kitchen, this post presents some solutions we have thought of as a group.

The Ideal Solution

In an ideal world people would not have anosmia or problems with their sense of smell at all.  Since an ideal world doesn’t exist, we must consider the ideal solution.  Then the ideal solution is one that brings back the sense of smell without anyone being wise to the fact that it was gone in the first place.  In practical terms, a mass-spectrometer that analyses all the molecules that goes through the nose and then sends the appropriate signal to the brain so that it may make us “smell”.  In terms of odors there are already quantitative methods of assessing the concentration or intensity of the smell.  The challenge here would be to replicate the correct signal to be sent to the brain.  All this of course, wrapped in an “invisible” and unintrusive physical form that does not lead to stigmatisation or a view that such a supportive tool is out of the norm.  Such a solution has yet to materialise however and so we must look at other alternatives.

The Realistic Solution

The first thing we usually do when deprived of one of our senses is to use the other senses to compensate.  Those who are visually impaired used their sense of touch to read and those who are deaf use sight to communicate instead for example.  Thus it is a easy to use the other remaining senses to compensate for the problems faced.  These problems included but are no doubt not limited to:

1. Inability to recognise spoiled or rotten food
2. Inability to recognise the burning and charring of food and its consequences

These are the two most obvious and simple problems to solve without resorting to high-tech solutions.

The thing with perishable foods is that they will spoil and go rotten.  Eating such food is a particular health hazard, as well tasting awful in some cases.  It is primarily the sense of smell that allows us to be able to distinguish whether the milk or meat has gone bad.  In my experience, if there is anything a person with anosmia trusts less than their nose, it is the nose of another person!  Since odor can be quantified, we should be able to fashion some kind of visually observable and objective olfactory indicator similar to pH indicators.  A litmus test for smell essentially that detects whether a package of a certain food product that has gone bad or otherwise.  A visual strip of colour that changes from green to red should be enough of an indicator to even those without anosmia to determine if something has become rotten without having to take a sniff themselves.  There are a few drawbacks to this, but generally is because of the way we store food.  In fact, sometimes we don’t store food in nicely sealed containers that may cause a problem.  Another problem is that due to the diverse nature of food products, their process of decomposition is different and thus there must be specific detectors for such.

As for the matter of burning food and unable to sense it at a glance, there are already solutions in place for this.  Air-sampling smoke detectors could and should be more than adequate.  This is especially true in kitchens with ventilators.  The only modification to the system would be a silent and private alarm, perhaps something akin to a wireless earphone slaved to the system.  If it is at all useful remains is debatable as a general rule of thumb when cooking is to always pay attention to the things on a stove, or at least I hope it is.

The Exploitable Solution???

Whilst looking at existing solutions we came across a rather interesting app that was currently in development.  ChatPerf is an application for iPhone with a small accessories to be plugged into a port. The accessory is a self-triggering container of perfume.  The current downside of course that the smell that is transferred is reliant solely on the container provided by the makers of ChatPerf.  There is potential for it to assist those with anosmia in a situation where they require someone’s help to smell something.  If such a thing existed perhaps my mother could ask me if the milk is spoiled in Thailand while I sit in my small apartment in Sweden.  The thought is in ever equal parts reassuring and bothersome.