This paper discusses the design and development of CreteAR, a system that integrates Extended Reality (XR) with a transformable physical model of the island of Crete to enhance learning experiences. CreteAR features an XR application for handheld devices that overlays information on a scale model, which is equipped with mechatronics to elevate or conceal showcases containing culturally significant items. Users can interact with both the digital content and the physical model, activating the showcases and accessing information through a companion display. The paper explores relevant literature, analyzes similar systems, and details the design process, including user requirements and expert feedback. CreteAR was initially conceived as a prototype, but it holds the potential to contribute to the development of broader applications across various domains beyond the scope of Crete’s geography and heritage. It was designed to deliver an immersive and handson learning experience, emphasizing social interaction and cooperative exploration, engaging users in dynamic and meaningful interactions.

Modern digitization technologies have created an increasing number of possibilities for capturing the physical dimensions and appearance of archaeological artifacts and sites in 3D. The usage of such data is usually targeted to the research, study, and documentation of our cultural heritage. At the same time, the increasing quality of the produced digitizations has opened new possibilities for the further exploitation of digitization outcomes in a wider context than initially expected. A pioneer in this direction was the gaming industry, where photogrammetry has been recently employed to achieve extreme photorealism. Of course, challenges still exist, especially when digitization accuracy is of importance, such as in the case of large-scale archaeological sites. Further challenges regard the need to combine indoor and outdoor scenes that pose requirements in the selection of the appropriate digitization modalities and post-processing strategies. In more detail, the challenges relate to the appropriate usage of existing technologies, organization issues in terms of digitization visits, the combination and registration of data, data acquisition, and data processing methodologies, etc. In this paper, we demonstrated a methodology for the digitization of archaeological sites that can be used for creating digital assets suitable for various scenarios including research, education, and entertainment.

With the ever-advancing availability of digitized museum artifacts, the question of how to make the vast collection of exhibits accessible and explorable beyond what museums traditionally offer via their websites and exposed databases has recently gained increased attention. This research work introduces the Invisible Museum: a user-centric platform that allows users to create interactive and immersive virtual 3D/VR exhibitions using a unified collaborative authoring environment. The platform itself was designed following a Human-Centered Design approach, with the active participation of museum curators and end-users. Content representation adheres to domain standards such as International Committee for Documentation of the International Council of Museums (CIDOC-CRM) and the Europeana Data Model and exploits state-of-the-art deep learning technologies to assist the curators by generating ontology bindings for textual data. The platform enables the formulation and semantic representation of narratives that guide storytelling experiences and bind the presented artifacts with their socio-historic context. Main contributions are pertinent to the fields of (a) user-designed dynamic virtual exhibitions, (b) personalized suggestions and exhibition tours, (c) visualization in web-based 3D/VR technologies, and (d) immersive navigation and interaction. The Invisible Museum has been evaluated using a combination of different methodologies, ensuring the delivery of a high-quality user experience, leading to valuable lessons learned, which are discussed in the article.

This paper presents COIN-O-RAMA, an interactive exhibit designed for exploring and further engaging with coins in physical exhibitions. The key motivation was to create an appealing and user friendly system, augmenting and complementing the overall experience of visiting the exhibition, also offering the means for creating a personalized digital souvenir (a coin with the user's head engraved on it), working as a post-visit link to the exhibition and to related information, as well as an artifact to share with friends and the social media. In this paper the requirements and preferences upon which the creation of COIN-O-RAMA was based are presented, followed by an account of the conceptual and interaction / user interface design of the system. Subsequently, a technical description of the process followed to achieve the engraving effect used for creating the personalized digital coin is offered. Then, the results of an expert evaluation of a fully working prototype of the system are reported, comprising issues raised by the evaluators and the corresponding re-design of the system. The paper wraps up with some conclusions and pointers to future work.

The case of mixed-reality projector-camera systems is considered and, in particular, those which employ hand-held boards as interactive displays. This work focuses upon the accurate, robust, and timely detection and pose estimation of such boards, to achieve high-quality augmentation and interaction. The proposed approach operates a camera in the near infrared spectrum to filter out the optical projection from the sensory input. However, the monochromaticity of input restricts the use of color for the detection of boards. In this context, two methods are proposed. The first regards the pose estimation of boards which, being computationally demanding and frequently used by the system, is highly parallelized. The second uses this pose estimation method to detect and track boards, being efficient in the use of computational resources so that accurate results are provided in real-time. Accurate pose estimation facilitates touch detection upon designated areas on the boards and high-quality projection of visual content upon boards. An implementation of the proposed approach is extensively and quantitatively evaluated, as to its accuracy and efficiency. This evaluation, along with usability and pilot application investigations, indicate the suitability of the proposed approach for use in interactive, mixed-reality applications.

Two dimensional paintings were exhibited in museums and art galleries in the same manner since at least three centuries. However, the emergence of novel interactive technologies provides the opportunity to change this status quo. By 2006, according to the Institute for Museum and Library Services, 43 % of museum visits in the U.S. were remote. According to the Institute for the Future, “Emerging technologies are transforming everything that constitutes our notion of “reality” – our ability to sense our surroundings, our capacity to reason, our perception of the world”. In the present age, that technology is becoming mixed to the fabric of reality to offer novel experiences in Cultural Heritage Institutions. This work presents the design and implementation of a technological framework based on ambient intelligence to enhance visitor experiences within Heritage Institutions by augmenting two dimensional paintings. Among the major contributions of this chapter is the support of personalized multi user access to exhibits, facilitating also adaptation mechanisms for altering the interaction style and content based on the requirements of each Heritage Institution’s visitor. A standards compliant knowledge representation and the appropriate authoring tools guarantee the effective integration of this approach in any relevant context. The developed applications have been deployed within a simulation space of the FORTH-ICS AmI facility and evaluated by users in the context of a pilot study.

Adaptation and content personalization in the context of multi user museum exhibits Conference Paper · June 2016 with 2 Reads Conference: 1st Workshop on Advanced Visual Interfaces for Cultural Heritage co-located with the International Working Conference on Advanced Visual Interfaces (AVI 2016), At Bari, Italy 1st Nikolaos Partarakis 4.18 · Foundation for Research and Technology - Hellas 2nd Margherita Antona 15.64 · Foundation for Research and Technology - Hellas 3rd Emmanouil Zidianakis 1.71 · Foundation for Research and Technology - Hellas 4th Constantine Stephanidis Abstract Two dimensional paintings are exhibited in museums and art galleries in the same manner since at least three centuries. However, the emergence of novel interaction techniques and metaphors provides the opportunity to change this status quo, by supporting mixing physical and digital Cultural Heritage experiences. This paper presents the design and implementation of a technological framework based on Ambient Intelligence to enhance visitor experiences within Cultural Heritage Institutions (CHIs) by augmenting two dimensional paintings. Among the major contributions of this research work is the support of personalized multi user access to exhibits, facilitating also adaptation mechanisms for altering the interaction style and content to the requirements of each CHI visitor. A standards compliant knowledge representation and the appropriate authoring tools guarantee the effective integration of this approach to the CHI context.

This work regards fingertip contact detection and localization upon planar surfaces, for the purpose of providing interactivity in augmented, interactive displays that are implemented upon these surfaces. The proposed approach differs from the widely employed approach where user hands are observed from above, in that user hands are imaged laterally. An algorithmic approach for the treatment of the corresponding visual input is proposed. The proposed approach is extensively evaluated and compared to the top view approach. Advantages of the proposed approach include increased sensitivity, localization accuracy, scalability, as well as, practicality and cost efficiency of installation.

Natural interaction refers to people interacting with technology as they are used to interact with the real world in everyday life, through gestures, expressions, movements, etc., and discovering the world by looking around and manipulating physical objects [16]. In the domain of cultural heritage research has been conducted in a number of directions including (a) Personalised Information in Museums, (b) Interactive Exhibits, (c) Interactive Games Installations in Museums, (d) Museum Mobile Applications, (e) Museums presence on the Web and (f) Museum Social Applications. Most museums target family groups and organize family-oriented events in their programs but how families choose to visit particular museums in response to their leisure needs has rarely been highlighted. This work exploits the possibility of extending the usage of AmI technology, and thus the user experience, within leisure spaces provided by museums such as cafeterias. The Museum Coffee Table is an augmented physical surface where physical objects can be used for accessing information about artists and their creations. At the same entertainment for children is facilitated through the integration of popular games on the surface. As a result, the entire family can seat around the table, drink coffee and complete their visit to the museum acquiring additional knowledge and playing games.

This paper reports on the design and implementation of BeThereNow, a public interactive information system where users are depicted immersed in various sceneries. The work is focused on the domain of info-tainment in public spaces using large displays and aims on short-time usage. The implemented system employs a mixed reality application through which users are informed about different sceneries and also create personalized digital postcards. This process is accomplished using computer vision algorithms in order to depict users and objects, while removing the background of the scene. Finally, the lessons learned from the long-term deployment of the system out-in-the-wild are presented, providing an insight on the users’ actions and reactions and feedback on future research directions.