The Sudoku Case system was developed as a student project at ZGDV in Darmstadt and user tested at NISLab in Denmark in June 2007. At NISLab we ran the prototype on an HP Deskpro workstation, 2.13GHz Intel Core2 DUO, 1GB ram, Firewire and Windows XP. The cameras mounted at the ceiling were DMK 21F04 from Imaging Source (640x480pixels, monochrome). Each camera had a Velleman CAML2IR 4mm objective with built-in IR-illumination. The speech recogniser was from Microsoft delivered for free as part of the SAPI 1.5 package. A Logitech headset microphone was used for spoken input. A 42'' flat panel screen was used for output, cf. Figure 1.

The system's output is presented on the screen as a Sudoku game board and two labelled icons in the top-left corner, new game and reset game, for choosing a new game and deleting all inserted numbers from the current game, respectively (Figure 1). When looking carefully at the game board, it is possible to see two differences between the fixed and the (user-) inserted numbers: the latter are slightly larger and shown on a slightly lighter and more greenish background than the former.

Game board

Figure 1. Game board showing a completed Sudoku game. The two options to the left of the game board are "New game" (top) and "Reset game".

When starting a new game, the user must choose its level of difficulty in a subsidiary window by pointing to one of the three displayed options easy, medium, or hard (Figure 2), which have been set to correspond to an initial filling of 30, 45, and 60 squares, respectively. Speech input is not expected (nor understood) at this point.

Game level

Figure 2. Game level.

An example of a new, medium-difficulty game is shown in Figure 3. To insert a number into a square, the user must point at the square and say, in English, the word "number" followed by an integer between 1 and 9, e.g., "Number 2". It does not matter whether or not the square is already filled as long as the user is not trying to replace a fixed number with a new one: the number spoken gets inserted into the square and, if the square already contains an inserted number, this number gets replaced by the one just uttered. A pointed-at square gets highlighted no matter if it contains a number or not.

New game

Figure 3. A new Sudoku game.

When playing the game in the test, the user must stand in front of the large (42??) screen, shoetips to a chalk line on the floor confined laterally by other chalk marks (Figure 4). The cameras had been calibrated for this position as well as for a certain, broadly defined, amount of illumination in the room. Otherwise, the room was a standard lab room in which it was only necessary to remove from the vicinity of the user certain spatial shape/coloured entities lest the cameras might identify these as human hand/index fingers and lock onto them.

Stand inside the marked area

Figure 4. The user must stand inside the area marked by a chalk line on the floor.

During the game, the system provides elementary on-line help, as follows. If the inserted number at insertion time conflicts with (a direct implication of) the basic rule of the game, so that the same number occurs twice in a row, column, or 3x3 square, that row, column, or 3x3 square turns red (Figure 5). The red error colouring remains until the error has been corrected. If the user proceeds to make another error before correcting the previous one, more of the game board may turn red. We call this basic (or surface) error correction because the algorithm simply spots two identical numbers in the wrong place. What the system does not do is to provide deep error correction by comparing the inserted number with the final solution to the game. This means that the user may insert any amount of wrong numbers without being notified, as long as these numbers do not produce a basic error. The following commands can be used to delete the contents of a (user-)filled square: "delete that", "delete this", "remove that", and "remove this". These commands are equivalent.

Game error

Figure 5. Game error.

It is possible to restart an ongoing game by pointing at the reset game button below the new game button (Figure 1). When restarting a game, the user must confirm the choice made (Figure 6). Speech is not expected (nor understood) in connection with reset and proceed yes/no.


Figure 6. Proceed?

When a game has been completed correctly, a congratulation screen is shown (Figure 7).


Figure 7. Congratulations.