UC Riverside

While traditional computer interfaces based on the mouse and keyboard

are ubiquitous, they are ill suited to many common application

domains. This is particularly true in education, where recent research

suggests that students perform better when instructional interfaces

are more similar to work practice. Thus, the goal of our work is to

create computational techniques and user interface design principles

to enable natural, pen-based tutoring systems that scaffold students

in solving problems in the same way they would ordinarily solve them

with paper and pencil. In our work, we have focused on interfaces

suitable for a ``pentop computer,'' a writing instrument that is used

with special dot-patterned paper, and that has an integrated digitizer

and embedded processor. A pentop is capable of producing dynamic output

in the form of synthesized speech and recorded sound clips.

Accurate shape recognition is an essential foundation for developing

pen-based interfaces. We created a trainable, multi-stroke recognizer

that is insensitive to orientation, non-uniform scaling, and drawing

order. Symbols are represented internally as attributed relational

graphs describing both the geometry and topology of the symbols.

Symbol recognition is accomplished by finding the definition symbol

whose attributed relational graph best matches that of the unknown

symbol. We developed five efficient approximate matching techniques

to perform the graph matching.

To explore instructional and interface design issues, we created

Newton's Pen, a pentop-based statics tutor. This system, which is

intended for undergraduate education, scaffolds students in the

construction of free body diagrams and equilibrium equations.

Newton's Pen employs a finite state machine architecture that

effectively models the student's problem-solving progress, thus

serving as a convenient means for providing context-sensitive tutorial

feedback. User studies suggest that Newton's Pen is an effective

teaching tool, and that students are satisfied with the interface.

A key issue in the design of pentop interfaces is how to provide

effective feedback to the user. To explore this issue, we developed

PaperCAD, a system that enables users to query geometric information

from printed CAD drawings. PaperCAD employs two methods of feedback:

audio feedback with an adjustable level of conciseness, and a PDA that

provides a video display of the portion of the drawing near the pen

tip. This system also employs a novel technique that uses a hidden

Markov model to correct interpretation errors in hand-written

equations. Results of a user study suggest that users are highly

satisfied with the interface and prefer it to a traditional WIMP

interface.