Automatic Text Illustration in Augmented Reality

The purpose of this study, which is part of the ARC project “A Pictorial Communication Framework for Inclusion“, is to explore the design of an application that uses pictures to offer people who cannot read a visual interpretation of what is written in an everyday environment. For example, menus at a cafĂ© or direction signs in a shopping centre could be replaced on the screen of a mobile device by emojis.

Through this study, we also seek to understand which pictures should be offered as replacement for text, for example, emojis, photographs, or a combination.

Finally, we will be observing how the pictures offered on the screen may become a tool for expressing choices, or starting a conversation, for people with intellectual disability who prefer to use images to communicate.


Laurianne Sitbon

William Ma (VRES, Bachelor of IT)

Liam Percy (Honours Engineering Student, Graduated in 2021)

Tia Chapple (Honours Engineering Student, Graduated in 2021)