The system presents an approach towards augmenting a physical book and enabling interaction through physical cards, with the purpose of providing a playful approach to learning the alphabet
The “Book of Ellie” is the augmented version of a classic schoolbook for teaching the Greek alphabet to primary school children. The book introduces the alphabet letters and possible combinations of them by increasing the difficulty level. Each letter or letter combination, provides relevant images and descriptional text. The short stories for each letter are structured around dialogues and activities of a Greek family, with the protagonist being Ellie, one of the four children. In the augmented version of the book, Ellie has become an animated character, constantly available to assist the young learner by reading phrases from the book, asking questions or providing advice.
The system recognizes the currently open page and displays on a digital screen its electronic version. The child can interact with the physical book by pointing with its finger at any letter or text passage of interest and listen to its pronunciation by a native speaker audio recording.
Initially, Ellie is waiting for the child to touch a phrase on the physical book page. When this occurs, the phrase is highlighted on the screen and Ellie reads it outloud, while pointing towards it. Each phrase is transformed into a series of lip-synched visual mouth phonemes, providing Ellie’s speaking animation with a realistic appearance. In this way, phonetic pronunciation is understood, casting the application ideal for alphabet learning.
The child is introduced to an educational card game that acts as a recapitulation of the letters that have been taught and a teaching tool for the spelling of some basic words. During the game, Ellie asks the child 24 questions while visual feedback is provided on the screen. In more details, there are two types of questions which are randomly chosen: (a) find the picture that begins with a given letter where feedback is provided by Ellie both through speech and textq; Ellie assists the learner by suggesting to place another card on the table, or to place just one card, and (b) spell the word of a given picture, using letters from a pack of letter-cards.
G. Margetis, X. Zabulis, S. Ntoa, P. Koutlemanis, E. Papadaki, M. Antona, C. Stephanidis (2015). Enhancing education through natural interaction with physical paper. Universal Access in the Information Society, Special Issue on “User Experience and Access using Augmented and Multimedia Technologies”, (pp. 427-447) http://dx.doi.org/10.1007/s10209-014-0365-0
Margetis, G., Antona, M., & Stephanidis, C. (2015). A Framework for Supporting Natural Interaction with Printed Matter in Ambient Intelligence Environments. In the Proceedings of the 5th International Conference on Ambient Computing, Applications, Services and Technologies (AMBIENT 2015), Nice, France, 19-24 July (pp. 72-78). USA: IARIA XPS Press. https://goo.gl/ysPHFm
Papadaki, E., Zabulis, X., Ntoa, S., Margetis, G., Koutlemanis, P., Karamaounas, P., & Stephanidis, C (2013). The book of Ellie: An interactive book for teaching the alphabet to children. In the Proceedings of the 2013 IEEE International Conference on Multimedia and Expo Workshops (ICMEW 2013), San Jose, California, USA, 15-19 July (6 pages). NY, USA: IEEE. http://dx.doi.org/10.1109/ICMEW.2013.6618341