Augmented Reality platforms employed on handheld devices, allowing users to program Intelligent Environments, by creating trigger-action rules about the behaviors of smart artifacts using 3D blocks.
This project presents users with a novel way of programming the behavior of smart artifacts in Intelligent Environments.
In more detail, users can define the behavior of smart artifacts in their surrounding Intelligent Environment using a portable device, e.g. a tablet. This is achieved via a novel a hybrid approach that enables rule-based programming with the use of 3D LEGO-like bricks and spatial computing. Rather than connecting the bricks in the physical world, this is done in AR on top of a virtual baseplate. This approach presents an innovative interaction paradigm, providing solutions to various challenges of previous 3D and tangible programming approaches (e.g. 3D bricks can easily be moved, scaled and reused to overcome physical space requirements since the entire augmented “world” is the user’s canvas). Moreover, via AR, the artifact collection task is simplified, by enabling users to easily explore their physical environment and directly select the object they need -instead of having to browse through extensive lists. The latter be-comes especially important in emerging intelligent spatial computing environments that include a plethora of IoT devices. Moreover, by being immersed, users can get a better understanding of all the available smart artifacts, their actions, and their locations. Finally, networked collaboration is inherently supported, so that another person can assist the user by means of remote help, while a Conversational Agent makes the experience more natural by enabling inter-action via natural language.
In the context of this project, two (2) applications have been created:
1) BricklAyeR, a novel collaborative platform for non-programmers, that allows to define the behavior of Intelligent Environments, through an intuitive, 3D building-block User Interface, following the Trigger-Action programming principle, in Augmented Reality, with the help of a Conversational Agent
2) MagiPlay, a serious game in AR, allowing children to program their Intelligent surroundings in a fun and engaging way, utilizing the same concepts as BricklAyeR, and aiming to help children acquire computational thinking skills. The Conversational Agent is accordingly designed for each of these applications, so that in BricklAyeR it is an adult character, while in MagiPlay it takes the form of a cartoon-style kid.
