The Treasure Hunt system has been developed at ITI-CERTH in Greece. It is a game for two persons with special needs for access to information and communication technologies, i.e., a blind person and a deaf-mute person. The system was user tested at the Institute for the Blind in Copenhagen, Denmark, in June 2007 (Figure 1). Six blind users participated while one of the developers acted as the deaf-mute user. Notes from the user test are available here.

The system requires two ordinary PCs, one for the blind and one for the deaf-mute. The blind communicates via haptics (input and output) and also receives audio output. The haptic device used is a PHANToM TM force feedback device (Figure 1). Audio messages are provided via loudspeakers. The deaf-mute has a GUI interface for graphics output and mouse-drawing input (the keyboard is not used) and provides sign language input via a webcam as well. The system recognises five different words in the Greek sign language and displays an animated agent that provides sign language output. Audio messages are converted to sign language for the deaf-mute. Completion of the treasure hunt involves seven steps performed by the blind and the deaf-mute in alternation.

3D haptic device

Figure 1. 3D haptic device.

The virtual environment in which the treasure hunt takes place is shown in Figure 2.

Virtual environment

Figure 2. The virtual environment.

The blind user must perform the first of the seven steps which is to find a red closet (Figure 3).

Red closet

Figure 3. Red closet.

Finding the closet results in an audio message which tells the deaf-mute user to go to the town hall. The audio message is converted into sign language and communicated by an avatar (Figure 4).

Avatar performing sign

Figure 4. Avatar performing sign.

In the town hall the new message is to go to the temple ruins and find an oracle inscription (Figure 5). The blind user hears the message and it is his turn to perform this step.

Temple ruins

Figure 5. Temple ruins.

The inscription text says "the dead will save the city". Thus the deaf-mute user must go to the cemetery and look for the next key (Figure 6).


Figure 6. Cemetery.

In the cemetery the message is to go to the catacombs (Figure 7) and look for a hidden map which the blind then does.


Figure 7. Catacombs.

The deaf-mute user receives the map with a riddle that he must solve and then draw the path to the treasure (Figure 8). The sketched path is transformed into a grooved line so that the blind user can "see" it.

Path to treasure

Figure 8. Sketching the path to the treasure.

The blind user follows the sketched path and then comes to a forest (Figure 9).

In the forest

Figure 9. In the forest.

He then gets a new map with a grooved line which he must follow to the treasure (Figure 10).

The treasure

Figure 10. The treasure: design patterns for war machines.