The Treasure Hunt system was tested with six blind users at the Institute for the Blind in Copenhagen, Denmark, in June 2007. After interaction with the system each user was interviewed. Users' replies to the interview questions as well as observation notes from their interaction with the system are shown below.

Subject #1

User details and background information
1. What is your name? xxx.
2. How old are you? 40.
3. Gender? Male.
4. How much can you see? Weak-sighted.
He closed his eyes throughout the interaction.
5. What is your education/occupation? Datamatiker (a 2-years informatics education), IT consultant.
6. How often do you use a computer? Whenever awake, all day.
7. How often do you play computer games? How do you, e.g., do that? Has played a lot periodically, using mouse and keyboard.
8. What (else) do you use a computer for? All kinds of things.
9. How are you used to interact with a computer? How are you used to input information and how are you used to receive information from the computer? Did you ever try haptic feedback? Interacts via GUI and speech synthesis. Has not tried haptic feedback.
What to measure How to measure
Quality og interaction
10. How was it to navigate in the city and landscape? Were there any problems? Which? Medium difficult. Bumped into things that he didnít know what was and wasnít told by the system: is this house 1, 2, 3, 4? "red house", "blue house", etc. would be useful. Would be good with zones like in "tampen brśnder". Left the room because he (mistakenly) believed that the sound was right, so he clicked. Missed the white cane and more detailed information.
11. Two questions concerning the haptic device:
a) How did it work to use the haptic device? Were there any problems? Which? Did you use such a device before?
b) How sufficient was the haptic feedback?
a) Surprisingly good 3D world. Wonderful and astonishing experience. Has not used such a device before.
b) The haptic device gave the feedback it should but information was seriously missing on what it was that was touched (what the feedback meant) and what to do.
12. To which extent did it work to recognise colours via sound? Were there any problems? Which? Did you try such a thing before? Worked badly. The sounds for blue and red are very close. The sound differences should be more marked, like a bell, a wailing sound, etc. [NOTE: something one can conceptualise] Has not tried colour recognition by sound before.
13. Two questions concerning spoken output:
a) How was it to understand what the system said? Were there any problems? Which?
b) How sufficient was it what was said?
a) Fairly Greek accent but there were only single words so it worked well enough. However, the spoken feedback should come faster, like immediately when you touch something.
b) Sufficient and comprehensible. But there should be more information on what it is that one bumps into.
14. How difficult or easy was each of the tasks in the game? Please describe. Difficult. There are too many details in the landscape. The point you need to hit in order to click is too small.
15. How easy or difficult was it in total to achieve the goal of the game? Medium difficult. For example, you cannot tell whether you are on your way in or out of an area or whether you have been there before. There is a lot of surfing about for no particular reason. You donít know if a house has more than one room, nor whether you first visited a "red" house, then a "blue" one, then a "red" one. You need to get the information you would get if you were able to see. It may be an idea to introduce a kind of sonar that would work some way ahead.
16. How did it work to follow the path to the treasure? To which extent was the physical resistance in the haptic device of any help? Not well/badly. Didnít know what the path looked like: a dam, a ditch, or what. Feedback like "You are on the path" would be helpful when you plunge into the path. The haptic force feedback was helpful.
17. How was the communication with your partner? How could you imagine that the communication might be extended? Itís well indicated who has the initiative. However, the deaf person must become tired of waiting. Brief descriptions of what is going on at the other end would be good.
18. To which extent did you miss other ways in which to communicate with the system or with your partner? Didnít miss other ways of communicating. Unless one knows what the partner is doing. Then one might, e.g., chat with the partner.
19. How easy or difficult was it to use the system to play the game? Did you have any problems when playing? Which? It was easy.
20. To which extent did you feel in control when playing the game? Not very much. For example, being inside the ruins felt like floating in the air. A wall around the ruins would be good.
21. Did you learn anything during the game which made you change the way in which you use the system? Yes, after navigating randomly I became more systematic in uncovering areas.
Functionality
22. Do you think the system offered you all the functions you need for playing the game, or did you miss anything? If yes, what? For instance:
- anything missing from what you could do with the haptic device?
- anything missing regarding spoken output;
- other missing information.
This is important for us to know in order to be able to improve the game.
Primarily information was missing. What is most needed is on-line help and feedback. The haptic device provides what it should.
User experience
23. To which extent did you try any similar games before or just something in the same direction? Has tried First Shooter and Half Life that border on this game quite well. Difficult to play these games due to the requirements to your eyesight.
24. How did you like the treasure hunting game? The idea is good. The game play is a bit difficult due to the lack of information. Maybe it could be graduated into "baner", the first one including only boxy things, the second including other shapes. Then you could have levels and ??
25. What do you consider advantages and disadvantages regarding the way in which the game is played? Advantages: the 3D world, the limited area, and the fantastic sense of space.
Disadvantages: Not so many has a such a haptic device. The radius is limited. This could be fixed by adding a time factor on the movement [he seems to mean so that you can decide how far and fast one can travel in a second?]. There are not many games for the robot right now.
26. If you got the opportunity, would you then play this or a similar game again? If yes: why? If no: why not? Yes. It is a fun world. It is a different world with totally different demands on the sense of space compared to the traditional ?? games.
27. Do you have any other comments? None.
Observations on this user
Task 1 - Find red closet
Has problems understanding temple ruins. Is told that he must go into a house to find the red closet. Searches for a long time. Finds houses but no doors. Doesnít stick to the houses. It turns out that he thinks that he must hear a beep before he clicks on a house. He is told-to just click to enter and that there are 4 houses one can enter and 4 one cannot enter. [In fact there are only 3 houses one can enter.] Enters the right house but gets out through the door after clicking on it and after having clicked around and not found the closet. He only searched in one side of the room (where there is no red closet). [Strangely, the system sometimes says "house" when the wall is touched from the inside, and sometimes it emits the tone for grey colour instead.] Enters a new house - this time the one with the blue closet. Now he sticks to the houses. Exits and finally enters the house with the red closet again. Finds the closet and is told he needs to click. This task took approx. 10 minutes.
Task 3 - Go to temple ruins
Goes to temple ruins and clicks (1 minute). Gets a new scene. Asks after a while what he is supposed to find. Is told that he is looking for an obstacle. Waits for something to talk to him. Searches for 5 minutes. Seems often to be searching beyond the visible part of the screen, which makes it difficult to help him navigate because one cannot tell where he is in virtual space. Gets navigation advice until he finds the "inscription".
Task 5 - Go to catacombs
Explores the landscape while he waits for the message from the partner. He has to wait again after the beep signalling a partner message because the system takes 1-2 minutes to convert the partnerís sign language sign to speech output (using Embrola synthesis). At first there is no message on what he is supposed to do because the initial beep is not accompanied by any message. After a long while (about a minute) the system says "catacombs" which he is told is a message. Finds the catacombs after some time and clicks. He clicks a number of times before he manages to enter. Again he gets some help to find a box. His clicking on, or near, it does not work to begin with.
Task 7 - Follow grooved paths
Moves the cursor around and doesnít at first discover the groove when passing over it. Is helped: try to find something you can follow and which goes in a certain direction. Seems to follow the groove but looses it again. Is helped, finds the groove and its end. Gets the forest map and is told that he must imagine that the map is hanging on a wall (i.e., this is a - mostly - 2D object). He is told not to click this time but to find the path, go to its end, maybe then go to the other end (its start). Finds the starting point but has difficulties following the path. Is told that the path has obstacles on it. Subject: this is when one thinks that the path ends. Finds it after some time and with some help and then finds the treasure. [The test with this first subject made us realise that we probably had to do a lot of coaching and explanation before and, mainly during, the game-play.]

To top of page

Subject #2

User details and background information
1. What is your name? xxx.
2. How old are you? 32.
3. Gender? Female.
4. How much can you see? Nothing.
5. What is your education/occupation? Attending the Youth Group at the Institute for the Blind. Will probably start on a call centre education after the summer holidays.
6. How often do you use a computer? Only while at the institute. Does not have a computer at home.
7. How often do you play computer games? How do you, e.g., do that? Never before. This is the first time.
8. What (else) do you use a computer for? Email, Internet, documents, etc.
9. How are you used to interact with a computer? How are you used to input information and how are you used to receive information from the computer? Did you ever try haptic feedback? Uses keyboard and speech synthesis. Has never tried haptic feedback.
What to measure How to measure
Quality og interaction
10. How was it to navigate in the city and landscape? Were there any problems? Which? Medium difficult. Easy towards the end. Difficult among the houses. Tended to make too big movements with the haptic device. Learned that one has to be patient.
11. Two questions concerning the haptic device:
a) How did it work to use the haptic device? Were there any problems? Which? Did you use such a device before?
b) How sufficient was the haptic feedback?
a) It worked reasonably WELL once you got used to it. Happy to try.
Got gradually easier. b) The sufficiency of the haptic feedback was okay.
12. To which extent did it work to recognise colours via sound? Were there any problems? Which? Did you try such a thing before? [Audio output was switched off at the time she got into the houses due to a problem with repeated spoken output: the system starts repeating its latest spoken output indefinitely. This happened several times later. Lets call it the SPOKEN OUTPUT LOOP] Has not tried colour recognition by sound before.
13. Two questions concerning spoken output:
a) How was it to understand what the system said? Were there any problems? Which?
b) How sufficient was it what was said?
a) She could understand it although she says that she is not good at English.
b) The sufficiency of the spoken output was okay.
14. How difficult or easy was each of the tasks in the game? Please describe. The first task was the most difficult one: to find the red cupboard. The last one was the easiest one: to find the end of the path.
15. How easy or difficult was it in total to achieve the goal of the game? It was difficult. I made it difficult by not clicking fast. Fun to try.
16. How did it work to follow the path to the treasure? To which extent was the physical resistance in the haptic device of any help? That went well, didnít it? The haptic force feedback was helpful.
17. How was the communication with your partner? How could you imagine that the communication might be extended? Fine. As long as you get to know what is going on at your own end, it is fine. Doesnít need to know what the partner is doing. Received enough information about her side of the game.
18. To which extent did you miss other ways in which to communicate with the system or with your partner? Didnít miss anything.
19. How easy or difficult was it to use the system to play the game? Did you have any problems when playing? Which? Everything is difficult the first time. The most difficult thing was to learn how to use the haptic device. It was not difficult to remember to click.
20. To which extent did you feel in control when playing the game? Not so much. Had to have a little help.
21. Did you learn anything during the game which made you change the way in which you use the system? Easier towards the end. Learned patience and to be more calm. Believed at the start that it was possible to be done better/faster?
Functionality
22. Do you think the system offered you all the functions you need for playing the game, or did you miss anything? If yes, what? For instance:
- anything missing from what you could do with the haptic device?
- anything missing regarding spoken output;
- other missing information.
This is important for us to know in order to be able to improve the game.
Didnít miss anything.
User experience
23. To which extent did you try any similar games before or just something in the same direction? Hadnít tried any games before.
24. How did you like the treasure hunting game? Fun to try.
25. What do you consider advantages and disadvantages regarding the way in which the game is played? Doesnít know what to say.
26. If you got the opportunity, would you then play this or a similar game again? If yes: why? If no: why not? Yes, I could become better.
27. Do you have any other comments? None.
Observations on this user
Task 1 - Find red closet
Doesnít understand much English. Is told that she is in a village with houses in the centre and other things surrounding the village. Searches around for a long time without bumping into anything. Gets help: stick to the houses and try to get into one by clicking. The sound starts looping, repeating "cemetery" over and over. The system is restarted. The same happens again, now with "temple ruins" over and over, and the system is restarted once again. Hasnít understood that she needs to click to enter a house and probably also doesnít realise that a red closet must be found inside a house, so she is told. Gets some navigation advice to find the house. Enters the house with a red closet but clicks on the door and exits. Enters the house with the blue closet. The sound has been switched off since it continued to start looping so she is told about the colour. Gets help to exit the house. Re-enters the house with the blue closet. Exits again. Gets help to find the house with the red closet. Finds it with some help. This task took about 20 minutes. The subject appears a bit tired from the effort.
Task 3 - Go to temple ruins
Finds the temple ruins pretty fast and goes there. Touches and clicks but does not respond by clicking fast enough. This must be done simultaneously. Enters the ruins and gets some "tampen brśnder" navigation help. Finds the inscription but doesnít click. After some attempts manages to refind the inscription and click.
Task 5 - Go to catacombs
Waits for quite some time while the input sign language from the deaf-mute is being processed. Gets the message "catacombs". Searches for a while and finds them with some help. She seems to remember from previous exploration that the catacombs are in - what a seeing person would call - the bottom-right corner of the screen. Again itís difficult to help when you cannot see the cursor. Enters after three clicks. The first two times she clicks too late. Finds the box fast with some "tampen brśnder" help and clicks.
Task 7 - Follow grooved paths
Gets a map. Gets an explanation to think of the map as if it hangs on a wall. Is told that there is a path to follow. Finds the path immediately but floats up in the air. Gets advice. Gets the forest map. Finds the path rather easily with a little help. Is told the end-beginning story. Goes to the end first, then to the beginning, and then to the end. Jumps obstacles easily.

To top of page

Subject #3

User details and background information
1. What is your name? xxx.
2. How old are you? 21.
3. Gender? Male.
4. How much can you see? Only light and dark.
5. What is your education/occupation? May be going into a traineeship.
6. How often do you use a computer? Every day at school (at the Institute for the Blind??)
7. How often do you play computer games? How do you, e.g., do that? Mostly when visiting somebody else but not that often. Likes to play. Uses keyboard and sound output.
8. What (else) do you use a computer for? Internet looking for film, music, etc.
9. How are you used to interact with a computer? How are you used to input information and how are you used to receive information from the computer? Did you ever try haptic feedback? Uses keyboard and sound. Has tried a mouse for three fingers and with a button. The mouse gave haptic feedback. Tried it with a car racing game where one could feel the surface differences, and a dart game.
What to measure How to measure
Quality og interaction
10. How was it to navigate in the city and landscape? Were there any problems? Which? Difficult in the beginning but one got used to it quite soon. You try patiently and find a way. Had to get used to clicking fast on the button.
11. Two questions concerning the haptic device:
a) How did it work to use the haptic device? Were there any problems? Which? Did you use such a device before?
b) How sufficient was the haptic feedback?
a) Fine, except for the houses. Believed that these were at the top of the village. Had difficulty finding the temple ruins. Hasnít tried this kind of haptic device before.
b) It was sufficient.
12. To which extent did it work to recognise colours via sound? Were there any problems? Which? Did you try such a thing before? Worked OK. But misses an introduction in English about what the game is about, so that you donít need to be helped during gameplay. Has not tried colour recognition by sound before but found it easy.
13. Two questions concerning spoken output:
a) How was it to understand what the system said? Were there any problems? Which?
b) How sufficient was it what was said?
a) Depends on your English skills. Did not understand all the words in Greek English. But it was okay easy. Noticed background noise when the system said "tomb".
b) It was okay. [NOTE: itís an error to only have a single click available in the final game step.]]
14. How difficult or easy was each of the tasks in the game? Please describe. Okay, as long as you get instructions from the experimenter. But you donít know what to find. It should be said that you have to find "the box", so that you donít click on, e.g., the tomb.
15. How easy or difficult was it in total to achieve the goal of the game? Not so difficult as one might have thought.
16. How did it work to follow the path to the treasure? To which extent was the physical resistance in the haptic device of any help? Good, except from where it stopped. You go from the end to the beginning and back again. Not clear what to do when the path ends. The force feedback was good.
17. How was the communication with your partner? How could you imagine that the communication might be extended? The information from the partner was sufficient. It might be good if the partner could expand on the message sent: what was sent, etc., and if one could send messages to the partner asking for more information.
18. To which extent did you miss other ways in which to communicate with the system or with your partner? Didnít miss anything.
19. How easy or difficult was it to use the system to play the game? Did you have any problems when playing? Which? It was easy. Strange to get outside the plane by the temple ruins.
20. To which extent did you feel in control when playing the game? The most difficult was to get past the houses and into them, and that you had to jump them to go beyond them. Had expected that the system would say "door".
21. Did you learn anything during the game which made you change the way in which you use the system? Not really. Just walked around. Built a sort of map in the head during the game.
Functionality
22. Do you think the system offered you all the functions you need for playing the game, or did you miss anything? If yes, what? For instance:
- anything missing from what you could do with the haptic device?
- anything missing regarding spoken output;
- other missing information.
This is important for us to know in order to be able to improve the game.
Didnít miss anything. The output is okay brief. Itís good it says "house" and not "the right house" because then it would be too easy.
User experience
23. To which extent did you try any similar games before or just something in the same direction? The closest would be the car racing game and the dart game.
24. How did you like the treasure hunting game? Okay, challenging.
25. What do you consider advantages and disadvantages regarding the way in which the game is played? Very annoying that clicking at the starting point of the path means that you wonít hear any congratulations when you find the treasure because you have already used your one click.
26. If you got the opportunity, would you then play this or a similar game again? If yes: why? If no: why not? Yes, to see if you find the treasure again and get congratulations
27. Do you have any other comments? Correct the errors [in particular the problem of not hearing any congratulation] and get the system out on the market.
Observations on this user
Gets forceful instruction in clicking on everything following spoken system output.
Task 1 - Find red closet
Searches around. The output "temple ruins" start to loop and the system is restarted. Searches again. Works a lot outside the visible area on the screen. This should be delimited more strongly. Gets a lot of help to navigate to the area with houses. Enters the right house. Finds quite soon the red closet and clicks. Does not seem to find it easy to follow instructions in going "up", "left", "right", or "down" on the map. [Experimenter realised why later in the test.] Identifies the instrument names himself.
Task 3 - Go to temple ruins
Looks for the temple ruins. Has probably forgotten where they were. Gets navigation help. Must click many times before it works. Is told to hold the button down when clicking. Enters the temple ruins. Gets an explanation of what he is supposed to do. Finds relatively soon the inscription.
Task 5 - Go to catacombs
After a while the message catacombs is returned. Finds them quickly and enters. Searches around. Must click very many times on or around [itís impossible to tell from looking at the screen] the right box before it opens. Is told to push towards the object while clicking. [Clicking gradually turns out to be quite complicated!]
Task 7 - Follow grooved paths
Gets a map. Finds the path immediately and follows it with strong help. Is told that he needs to click at the end of the path even though there is no sound output. Gets the forest map. Finds the path after searching a little. Clicks at the beginning which results in silence when he reaches the treasure at the end, i.e. it doesnít say "congratulations". He was not warned that, with the forest map, you only have a single click available. But he did get the beginning-end story. The game took 45 minutes.

To top of page

Subject #4

User details and background information
1. What is your name? xxx.
2. How old are you? 23.
3. Gender? Male
4. How much can you see? Nothing. Was born blind.
5. What is your education/occupation? Attends IT service education. His first time with a 3D game. Incredibly exiting and instructive.
6. How often do you use a computer? Several times every day, also at home.
7. How often do you play computer games? How do you, e.g., do that? Never - only back in the DOS days, with keyboard commands.
8. What (else) do you use a computer for? Internet, news, email, net banking, installing software, installing speech programs for people, etc. Homepage maintenance.
9. How are you used to interact with a computer? How are you used to input information and how are you used to receive information from the computer? Did you ever try haptic feedback? Keyboard + software enables mouse control from the keyboard. Sometimes needs the mouse button to enter. Output is speech synthesis. Uses braille but is not happy to because his Braille input machine is old and doesnít work well with Windows. Has never tried haptic feedback. Incredibly exiting.
What to measure How to measure
Quality og interaction
10. How was it to navigate in the city and landscape? Were there any problems? Which? A little difficult to begin with. Has never tried to see using 3D. Had to get used to it but just needed a little time to explore the arm.
11. Two questions concerning the haptic device:
a) How did it work to use the haptic device? Were there any problems? Which? Did you use such a device before?
b) How sufficient was the haptic feedback?
a) Good, positive, surprising, impressive. Would like to use the robot arm more. Has not tried to use such a haptic device before. Fascinated by its use for surgery training.
b) It was okay.
12. To which extent did it work to recognise colours via sound? Were there any problems? Which? Did you try such a thing before? It was quite fun but a repeat function would have been good for reminders about the colour codes, if you forget them, e.g. via an additional button. Has not tried colour recognition by sound before.
13. Two questions concerning spoken output:
a) How was it to understand what the system said? Were there any problems? Which?
b) How sufficient was it what was said?
a) A somewhat Greek accent. Better English pronunciation would be good. The most difficult to understand was "catacombs".
b) Fine.
14. How difficult or easy was each of the tasks in the game? Please describe. It was difficult to follow the walls to the catacombs. The others were fairly easy.
15. How easy or difficult was it in total to achieve the goal of the game? Very easy. There is only one way to go and you cannot die. Imagines a long game version: consider to make it possible to save the game so that you can return later to where you were.
16. How did it work to follow the path to the treasure? To which extent was the physical resistance in the haptic device of any help? Okay once you found out. The 3D was totally new. The force feedback was very helpful. It should not shake at times.
17. How was the communication with your partner? How could you imagine that the communication might be extended? Okay. Didnít pay so much attention to him. Communication is important, would be good to be able to communicate with the partner. It might be good to introduce speech recognition which could then be converted to sign language.
18. To which extent did you miss other ways in which to communicate with the system or with your partner? Doesnít miss anything.
19. How easy or difficult was it to use the system to play the game? Did you have any problems when playing? Which? Easy once you found out about the 3D. Difficult when you havenít tried 3D before.
20. To which extent did you feel in control when playing the game? Reasonably.
21. Did you learn anything during the game which made you change the way in which you use the system? Yes, once he found out there were walls, boxes, etc., he became more aware of what was in the surroundings.
Functionality
22. Do you think the system offered you all the functions you need for playing the game, or did you miss anything? If yes, what? For instance:
- anything missing from what you could do with the haptic device?
- anything missing regarding spoken output;
- other missing information.
This is important for us to know in order to be able to improve the game.
Didnít immediately miss anything. It had the needed possibilities.
User experience
23. To which extent did you try any similar games before or just something in the same direction? Never.
24. How did you like the treasure hunting game? Incredibly exciting and instructive with the 3D). Would like to act as subject again.
25. What do you consider advantages and disadvantages regarding the way in which the game is played? No immediate comments.
26. If you got the opportunity, would you then play this or a similar game again? If yes: why? If no: why not? Yes, it is just so exciting to experience that you could walk around in 3D and see things rather than just reading about them. 3D would make one much more eager to play. Itís much better than to navigate with the arrow keys.
27. Do you have any other comments? The SPOKEN OUTPUT LOOP must be corrected.
Observations on this user
Seems to have a strong technical interest. "Cool VR for the sense of touch". "When will it be on the market?" Asks to have repeated the colour/sound instructions. Is told that there are houses in the centre of the village, that he must push and click, and that not all houses can open.
Task 1 - Find red closet
Searches around. Doesnít seem to understand that you can jump an obstacle by lifting the robot arm and put it down again. Seems to get a bit impatient. "Temple ruins" loop. Starts at the houses but doesnít manage to enter. Searches some more and then enters the right house. Finds very soon the red closet and clicks.
Task 3 - Go to temple ruins
Goes quickly to the temple ruins. Is told about how to use the robot arm. Searches around and soon find the inscription but does not click. Searches again for the inscription. Falls off the plane. Gets navigation help and finds it again. Says that he has never used 3D haptics.
Task 5 - Go to catacombs
Gets the catacomb message after a while. Searches for the catacombs. Irritating that we cannot see the cursor for lengths of time. Finds them after a while and enters. Is told what he is supposed to look for: box not tomb. Searches around and finds the wrong box. Gets substantial navigation help to find the right box.
Task 7 - Follow grooved paths
Receives a map. However, the path is convoluted, does not lead to the goal and cannot be used. [Kostas says that it was a mouse problem when drawing the path.] This is explained to him and the experimenter clicks to continue to the forest map. Gets help to pass obstacles.

To top of page

Subject #5

User details and background information
1. What is your name? xxx.
2. How old are you? 21.
3. Gender? Female.
4. How much can you see? Nothing.
5. What is your education/occupation? Attends IT service education.
6. How often do you use a computer? Every day, also at home.
7. How often do you play computer games? How do you, e.g., do that? Does not play games.
8. What (else) do you use a computer for? Email, Internet, school things, ...
9. How are you used to interact with a computer? How are you used to input information and how are you used to receive information from the computer? Did you ever try haptic feedback? Keyboard and spoken output. Has not tried haptic feedback before.
What to measure How to measure
Quality og interaction
10. How was it to navigate in the city and landscape? Were there any problems? Which? Somewhat difficult to begin with. Otherwise reasonably. It was weird that one could walk on the walls and the ceiling. It felt strange as if the temple ruins were above the houses. [It is!]
11. Two questions concerning the haptic device:
a) How did it work to use the haptic device? Were there any problems? Which? Did you use such a device before?
b) How sufficient was the haptic feedback?
a) Worked reasonably well. The neck got a little stiff from holding the robot arm in mid-air. There was a little problem holding the arm so that the button was where you needed it. Has never used such a device before.
b) Fairly good. Difficult to feel that the path was there. It felt as being a bit up in the air.
12. To which extent did it work to recognise colours via sound? Were there any problems? Which? Did you try such a thing before? This is just a question of learning and remembering. Couldnít remember which was which. Has not tried colour recognition by sound before.
13. Two questions concerning spoken output:
a) How was it to understand what the system said? Were there any problems? Which?
b) How sufficient was it what was said?
a) Is not so good at English. Felt a little like a lego city. Misses people and some life in the city.
b) Okay.
14. How difficult or easy was each of the tasks in the game? Please describe. Not that difficult. Was introduced to how to use the game.
15. How easy or difficult was it in total to achieve the goal of the game? Medium difficult. Was surprised when the game ended - already!?
16. How did it work to follow the path to the treasure? To which extent was the physical resistance in the haptic device of any help? It was difficult. The force feedback was helpful. Also this path was a bit up in the air.
17. How was the communication with your partner? How could you imagine that the communication might be extended? Got the information needed. The communication could be extended by enabling the partners to write together if there is something they donít understand.
18. To which extent did you miss other ways in which to communicate with the system or with your partner? Didnít miss anything, except chat functionality.
19. How easy or difficult was it to use the system to play the game? Did you have any problems when playing? Which? Fairly easy when first you find out how the haptic device works.
20. To which extent did you feel in control when playing the game? Could have been better.
21. Did you learn anything during the game which made you change the way in which you use the system? Found out that she should not move so fast. Could use some more information during about what one has to do. [Response to a leading question: you want to know what the partner is doing?] yes.
Functionality
22. Do you think the system offered you all the functions you need for playing the game, or did you miss anything? If yes, what? For instance:
- anything missing from what you could do with the haptic device?
- anything missing regarding spoken output;
- other missing information.
This is important for us to know in order to be able to improve the game.
Missed nothing.
User experience
23. To which extent did you try any similar games before or just something in the same direction? Never.
24. How did you like the treasure hunting game? Quite fun and exiting. Totally new way in which to play.
25. What do you consider advantages and disadvantages regarding the way in which the game is played? You look at the haptic device because you think the space is there. It is a funny experience. There might be more effects and sounds that, e.g., indicate the room you are in.
26. If you got the opportunity, would you then play this or a similar game again? If yes: why? If no: why not? Yes, it is very exciting, a totally new way in which to play. Often the second time you try something it becomes easier and more fun.
27. Do you have any other comments? None.
Observations on this user
Due to circumstances beyond our control, we only had a total of 45 minutes with Subject 5. Due to observations made on the previous subjects, we not tell the subject that the robot arm is not a pen [i.e., something associated with a plane surface] and strongly stress spatiality and 3D in the use of the arm.
Task 1 - Find red closet
Searches around in the landscape. Gets navigation help. "Temple ruins" starts to loop and the system is restarted. Has great difficulty doing systematic search of the space, especially wrt. moving foreground-background. Gets quite a lot of navigation help and manages to enter the house with the red cupboard. Gets search help but clicks on the door and exits the house. [This happens to several subjects despite the fact that the door says "door". But if you search fast within the room there is a lot of spoken and sound output - grey sound + "house" from walls, ceiling and floor - so it is easy to miss it when the door says "door".] Gets help to enter the house again. Eventually finds the red closet and gets help to click on it.
Task 3 - Go to temple ruins
At the start of this task, the subject expects to still be in the room with the red closet but no, miraculously, she is out in the open without having moved there herself. Quickly finds the temple ruins. Also soon finds the inscription after having received an explanation of what she is supposed to find.
Task 5 - Go to catacombs
Receives the catacomb message after a while. Gets help to find the catacombs and enters. Finds fairly soon the right box.
Task 7 - Follow grooved paths
At the start of this task, it is emphasised that the subject should not click until at the very end. Gets a map. Finds the groove fairly soon. Exits the grooved path a number of times. Eventually finds the path and follows it to the end. Gets the forest map. Finds the path quite soon with a little help. [There is a deep mismatch between the landscape shown on the screen and what the subject experiences through haptics. The graphics and haptics output domains donít correspond at all. Look at this as an issue of perspective. In haptics, you are on the ground all the time unless you move upwards. One couldnít usefully show the landscape in this way in an image because all you would be able to see is the immediate foreground. So the image perspective is looking down onto the landscape at an occlusion-avoiding angle.]

To top of page

Subject #6

User details and background information
1. What is your name? xxx.
2. How old are you? 25.
3. Gender? Female.
4. How much can you see? Nothing.
5. What is your education/occupation? Under education as IT teacher.
6. How often do you use a computer? Every day, also at home.
7. How often do you play computer games? How do you, e.g., do that? Very often. Plays many games. Uses keyboard, sound, many have spoken output from screen reader, joystick, mouse (rarely, only for experimental games). Has not tried spoken input for gaming.
8. What (else) do you use a computer for? Internet, email, everything.
9. How are you used to interact with a computer? How are you used to input information and how are you used to receive information from the computer? Did you ever try haptic feedback? Uses keyboard and spoken output. Has tried a car game (Topspeed2) with force feedback (gamepad).
What to measure How to measure
Quality og interaction
10. How was it to navigate in the city and landscape? Were there any problems? Which? Actually rather easy. Should have used more structured search, like they tell you to. Needs to develop more structure [inside head]. It was clear when there was something you bumped into, including the boundaries of the space. Didnít understand what went on when it fell into nothing, e.g., in the temple ruins. This also happened with the map. Had to ask: Where am I - outside or inside the environment?.
11. Two questions concerning the haptic device:
a) How did it work to use the haptic device? Were there any problems? Which? Did you use such a device before?
b) How sufficient was the haptic feedback?
a) Fine. New sensation to be in a room/space that felt real. Loves to move around and find things, learn new things. Has never tried such a haptic device before.
b) Good, but sometimes it gave way too much.
12. To which extent did it work to recognise colours via sound? Were there any problems? Which? Did you try such a thing before? Tried colour recognition only a little because she found the cupboard quickly. Too little difference between red (oboe) and blue (flute). Use, e.g., drum and guitar instead. The closest to trying this before was a program that painted a sound picture of the colours in a painting so that you could "hear" the painting. Difficult, isnít good at this.
13. Two questions concerning spoken output:
a) How was it to understand what the system said? Were there any problems? Which?
b) How sufficient was it what was said?
a) Could understand it (has an American boyfriend).
b) Okay, no use for more. If you are looking for something for a long time it would be good to get a hint, like "try that".
14. How difficult or easy was each of the tasks in the game? Please describe. Not so difficult. Relatively simple and clear and in a not-so-large environment.
15. How easy or difficult was it in total to achieve the goal of the game? Not so difficult. [When there were obstacles on the path] one just had to jump over [them] to see if there was a path. Loves this.
16. How did it work to follow the path to the treasure? To which extent was the physical resistance in the haptic device of any help? Fairly well. Had to get used to being able to jump over things. It was clear where the boundary was. The force feedback was helpful.
17. How was the communication with your partner? How could you imagine that the communication might be extended? OK. You [the experimenter] explained what went on. An explanation of what the partner is doing would be good. Would like to have a chat window to the partner.
18. To which extent did you miss other ways in which to communicate with the system or with your partner? There might have been a chat window to the partner. There could be predefined messages about what the partner is doing and vice versa. This would not work in Counter Strike. There it would be irritating for the partner to both communicate what one is doing and play the game.
19. How easy or difficult was it to use the system to play the game? Did you have any problems when playing? Which? Reasonably easy after the introduction with the cube. Gives a clear sense of 3D space. Did not try to turn the arm (anti) clockwise - what will happen?
20. To which extent did you feel in control when playing the game? Reasonably good control. But then there was only one button. It would have been more difficult if there had been more buttons.
21. Did you learn anything during the game which made you change the way in which you use the system? Is used to 2D navigation. Eventually got used to being able to move up and down and started to explore the area a bit more in this dimension.
Functionality
22. Do you think the system offered you all the functions you need for playing the game, or did you miss anything? If yes, what? For instance:
- anything missing from what you could do with the haptic device?
- anything missing regarding spoken output;
- other missing information.
This is important for us to know in order to be able to improve the game.
Didnít miss anything apart from getting rid of the SPOKEN OUTPUT LOOP.
User experience
23. To which extent did you try any similar games before or just something in the same direction? Topspeed2 has force feedback. The genre reminds of Last Crusade with the computer as partner. Quake for the blind has up - down direction via sound but no joystick - is still mostly a 2D experience. This system is extremely good. Develop it further!
24. How did you like the treasure hunting game? A bit boring. Not enough action. It depends on what you are used to playing.
25. What do you consider advantages and disadvantages regarding the way in which the game is played? No disadvantages. It is good that two persons with different disabilities can play together. This makes the potential pool of game partners larger. Itís something the blind have been missing because they tend to only meet other blind people and not to meet people with other disabilities.
26. If you got the opportunity, would you then play this or a similar game again? If yes: why? If no: why not? Maybe not exactly this game, but others, happily. This technique is really cool. You get a totally different experience and sense of space. Really exciting to try.
27. Do you have any other comments? Is this technology being used for other purposes? [Yes, for, e.g., dental surgery.] Really cool. Hopes the technology development will continue so that this will not only be for games. Could perhaps be used to sense a graphical object and perhaps to draw with. Might be used to explain to a blind what something looks like. Or for reading pictures. There might be a menu on double clicking, e.g., with a choice to skip. You might use the cube [demonstrates before the game began] as an environment for creating things inside it. I can see images in my mind. I want one of these. The system could be really good for presenting visual information for the blind: to explain things by showing them to people.
Observations on this user
With this final subject we had a total of 75 minutes. No pen introduction was given. Only the haptic device was shown. [All other subjects had started the practical part of the introduction by trying to move an ordinary pen among obstacles on a table.] The subjects happily speaks English. She seems quite bright. Gets a careful introduction to 3D. Gets the scene/landscape described. "Cool". [The sounds of the flute and the oboe are too similar.]
Task 1 - Find red closet
"Town hall" starts to loop and the system is restarted. "Temple ruins" starts to loop and the system is restarted, and again. Searches a little around. Is helped to find a house. Enters the right house. Finds very soon the red closet.
Task 3 - Go to temple ruins
Immediately finds the temple ruins, moves to cemetery, immediately goes back, and enters. Quickly finds the inscription.
Task 5 - Go to catacombs
Receives the catacomb message after a while. Quickly finds the catacombs and enters. Is told that she is supposed to find a box. Looks around and finds the right one, fast. Has problems in making the clicking work. [like several other subjects]
Task 7 - Follow grooved paths
Gets a map. Quickly finds the path and follows it. Discovers that one can jump obstacles and does it. Find the turn in the path immediately [several other subjects ran off the path there.] Is told that she must click at the end of the path. Gets the forest map. Finds the path easily and follows it to the end, then follows it to the start and back to the end. She does this fast. "I love games".

To top of page