Mixed Reality (MR) is the result of blending the physical world with the digital world. MR has been made relevant in the recent years with the leaps in advancements in processing power, computer vision and hardware. MR is the next evolution of AR and VR where the virtual spectrum blends with the physical one to create an experience beyond the already established – and in some ways mainstream – augmented reality (AR), where virtual objects are overlaid on graphic streams of the real world, and virtual reality (VR), where the virtual experience replaces or occludes the real world. In MR, the virtual objects are integrated to the natural world and exhibit responsiveness to real-world stimuli. MR, as a technology, is now embraced by the largest industries in the world, both commercial, such as Microsoft (Microsoft 2018), and community-driven open-sourced, such as the Mozilla Foundation (ShackNews 2018). The embracement is supported by specialized hardware as well as back-end APIs.

Numerous works have employed AR and VR as basic components for interaction to improve results in training (Weichel et al. 2014), education (Sommerauer and Müller 2014), places/buildings (Shatte et al. 2014, Olsson et al. 2011), personal development (Fonseca et al. 2014b), medical/clinical/special activities (Wilson et al. 2013; Poelman et al. 2012), collaboration (Wang and Dunston 2011; Liu et al. 2015), and so on. The list expands when the domains of application are included and span from tourism (Sylaiou et al. 2010) to learning (Fonseca et al. 2014a) and supporting people with disabilities (Yoo et al. 2013; Anderson and Bischof 2014) to space exploration (NASA 2018). Similarly, evaluation and usability studies have measured the user experience in terms of measurable objective indicators (Chiang et al. 2014; Wojciechowski and Cellary 2013) and subjective feedback (Cocciolo and Rabina 2013; Dey and Sandor 2014) from the experiencers. The main findings include the level of immersion, the user cognitive load, and the learning/educational impact. The challenges, as identified by the aforementioned studies as well as from professional organizations, include the need for special content, the monitoring of the user experience and the use of sensors and data.

Further to the above, the business sector (Martin 2018) has identified the top technology development challenges. The effort of manufacturing is the major challenge followed by device technology limitations and costs. The price for manufacturing, even when using very affordable consumer devices, skyrockets due to the need for special design of the environment content, the story line creation and the expert involvement to create the full experience. All the above can be summed to the challenge of enhanced content and expertly-directed user experience.

This paper aims to present a research approach to address these challenges and create future interactive technologies based on MR (including AR/VR) that are low-effort and low-cost, expert-driven but accessible to all businesses. The main ingredients to this research approach are the utilization of data/information from the Web, the Linked Open Data (LOD) cloud and the Internet of Things (IoT), and the concept of Shared MR Experiences Ecosystem (SMRE2), for enhanced user experience. IoT is already an “everywhere” technology that is also deemed to be the key factor of future interaction (Greenough and Camhi 2016) and the enabler of the vast semantically linked data/information. MR represents the controlled collision of the AR/VR and IoT trends (Analytics magazine 2017). MR technologies are the sole major qualifier to utilize the IoT-enabled semantic data/information to create advanced user experience. In order to do so, the user aspect has to be advanced to the point where the user behaviour can drive and be driven by the data/information, thus creating a novel world of seamless and immersive MR interaction with real-world and virtual entities.

The research approach, under the acronym ARTIST (for ReAl-time low-effoRt mulTi-entity Interaction System for creaTing reusable and optimized MR experiences), encapsulates two novel concepts; the concept of Shared MR Experiences Ecosystem (SMRE2) and the concept of Experience as a Trajectory (EaaT), which are briefly presented below (and in detail in the following sections).

• SMRE2 is an ecosystem built within ARTIST to contribute open and reusable MR content (MR experiences, data/information), MR applications and tools. Multiple users may interact with multiple entities at different times, thus creating a space of shared user experiences. The ARTIST ecosystem maintains the shared experiences that are complete (size and length is variable). The SMRE2 is the enabler for immersion for MR technologies since experts may interactively author experiences while the experiences themselves may dynamically intersect and interchange during user interaction, providing the users with unseen but relevant MR interaction.
• EaaT is introduced in ARTIST to map one or more semantic trajectories to one or more MR experiences. A semantic trajectory is a trajectory (segments of connected traces/points that represent movement of entities) that has been enhanced with annotations and/or one or several complementary segmentations. Authors argue that such an experience involves a number of scenes/episodes within segments of traces, where interconnected and interacting entities are moving and act in MR, situated in time and space sequences of particular application-specific interests, showing a number of different behaviours (virtual or real).

ARTIST introduces novel methods and tools for: i) real-time multi-entity interactions between human and non-human entities, to create reusable and optimized MR experiences with low effort, towards a SMRE2, ii) code-free implementation and deployment of open and reusable MR content, applications and tools, around the novel concept of an EaaT, iii) the tracking, monitoring and analysis of user behaviour and their interaction with the environment and with other users, towards optimizing MR experiences by recommending their reconfiguration either dynamically (at run-time) or statically (at development time), iv) synthesizing experiences into new mega (large number of smart entities, large virtual spaces, and rich interactions) and still reconfigurable EaaTs, enhancing them at the same time using semantically integrated related data/information available in disparate and heterogeneous resources.

ARTIST is integrating key Future Internet technologies such as Semantic Technologies (ST) and IoT, as well as technologies for User Interaction (UI), in order to advance MR (MR) technologies within a new Shared MR Experiences Ecosystem (SMRE2) that aims to contribute open and reusable content (MR experiences, data/information), applications and tools (Fig. 1).

The key concepts of ARTIST research approach are outlined below:

a)       Low-effort Content Creation: ARTIST aims to facilitate the design of MR experiences and the development of MR applications that, in a wider extend than existing frameworks or systems, will allow the:

i.    Low-effort rapid development of MR applications, via a code-free drag-and-drop style development, supporting non-expert (end-)users to actively participate in the early stages of the life-cycle of an MR application,

ii.   Low-effort management (select, add or remove) of virtual entities and external data/information at deployment and run-time.

b)      User behaviour: ARTIST aims to facilitate the integration and management of models for user behaviour using sensors’ feedback, attention and explicit user response, in order to allow the:

i.       Validation of user experience within EaaT interactions,

ii.      Prediction of user experience based on behaviour on similar or closely matched EaaTs.

c)       Analytics: ARTIST aims to facilitate the integration of analytic methods in order to allow the:

i.      Analysis of user behaviour and interactions during their participation in MR experiences,

ii.     Update/build of their psychological profile at run-time, based on the tracking and monitoring of user behaviour e.g. the user is bored, frustrated, confused, etc.,

iii.    Analysis of EaaTs, providing recommendations for synthesis and reconfiguration of experiences, towards optimizing future ones.

d)      Mega Virtual World: ARTIST aims to introduce the capability of an interaction system to further enhance/augment the experiences of an MR environment by combining virtual and physical entities of more than one virtual and/or physical world, thus enlarging the environment through integration (larger number of smart entities, larger virtual spaces, and richer interactions), linking physically or virtually separated sites and entities towards mega virtual worlds.

e)       Real-time Computations: ARTIST aims to support a number of real-time computations, towards introducing and preserving a dynamic aspect of MR applications in the SMRE2. Specifically, it will allow the:

i.      Support of interaction of users with up-to-date MR experiences’ related data/information gathered from the open Web and LOD cloud, as recommendations or additional information, triggered by the analysis of user’s behaviour at run-time (e.g. users are confused of what they see),

ii.     Track and monitor of real-time sensor data to analyse user’s behaviour within MR experiences that require higher precision/quality of data/information, properly understanding human behaviour and cognition, towards creating significant impact in improving MR environment and quality of the presentation and interaction with virtual objects e.g. purchasing experiences at a shopping environment that adapt to shoppers’ current biosensor data (pulses, temperature, motion, blood pressure).

f)       Semantic Technology: ARTIST aims to facilitate the integration of Semantic Technology to allow the:

i.        Integration of new structured/unstructured and heterogeneous data/information collected (from the Web/LOD cloud or from sensors) with the already encoded information, towards enhancing user experience/knowledge (e.g. real-time events related to the MR application, updated information related to an artefact position, real-time environmental/weather data that are integrated in a MR scene, or even historical (static) data that are discovered on-the-fly after have been ‘opened’ in the LOD cloud).

ii.      Interoperability between different virtual entities and MR experiences, synthesizing or extending MR experiences in more than one virtual space (e.g. mega virtual classes, mega virtual cultural sites or events), discovering links and synthesizing simple EaaTs into more complex ones within the same or cross-domain environments.

g)      IoT technology: ARTIST aims to support interactions between interconnected IoT entities (not only between users and groups), i.e. entities that will be trustfully deployed in an application domain environment (humans, things, sensors, actuators, applications, mobile devices). Their interaction in the MR environment and the tracking of such interaction and the behaviour of users will lead to higher quality experiences (optimizing the paths/trajectories of interactions in time and space), that will eventually enhance future MR experiences (through the reconfiguration of experiences).

h)     Openness: ARTIST aims to facilitate the design and development of open systems for sharing and reusing MR tools, content and     applications, towards a Shared MR Experiences Ecosystem (SMRE2), to provide relevant services in order to support the interaction of real and virtual world entities within the scope of a fully-experiential world. To achieve this, ARTIST allows open access to the following components: a) Virtual Experiences Store, b) Virtual Entities Store, c) MR apps, using well-known models, open standards and technology.

ARTIST aims to support the design and development of MR environments which are in principle virtual spaces where real world entities (human and non-human) are dynamically integrated into virtual worlds to produce new environments and visualizations. In these environments physical and digital entities co-exist and interact in real time. MR encompasses the whole spectrum of reality technologies, combining both VR and AR, with real and virtual entities. ARTIST embraces Future Interaction (FI) concept aiming to produce multi-entity interactions within the MR environment, between human and non-human entities. It is envisaged that more mature commercial VR systems, such as Samsung Gear VR for mobile phone platform, and HTC Vive Pro for PC platforms, as well as Microsoft Hololens headset*1 that allows its users to overlay holograms from virtual worlds on top of reality, will be used. The state-of-the art technology will be integrated in order to obtain measurements of user interaction (such as LooxidVR*2 in MR environments), to be used for user behaviour tracking and for real time presentation of high-quality content, towards enriching personal user experience.

ARTIST introduces new technology for developing MR software systems as well as advance the methods and tools available to develop MR applications with low effort. Additionally, authors envision ARTIST building a Shared MR Experiences Ecosystem (SMRE2) that aims to provide those services that will support the seamless interaction of real and virtual world entities within the scope of a fully-experiential world.

The main objectives and the correspondent means to achievement per objective of ARTIST are:

[O1] Create an environment for low-effort design and development of MR apps

Currently, design and development of MR apps is a laborious process requiring significant and specialized technical expertise in the domain, while integration of IoT devices in this context is impeded by device idiosyncrasies and incompatibilities. Further, due to the complexity of the domain, MR app designers may miss opportunities to further enhance user experiences and app functionalities, or deliver them in a suboptimal fashion. ARTIST provides the necessary methods and tools to: (a) allow for low-effort design and development of MR apps by leveraging the level of abstraction in the process of application creation and thus removing the barriers of technical expertise requirements and low-level device handling and (b) automatically identify opportunities for further enriching and optimizing MR app functionality to support MR app designers in enhancing and perfecting MR apps. The means to achieve this objective are the following:

• [O1.1] Implement a drag-and-drop style MR authoring environment for code-free development of apps.
• [O1.2] Implement a recommendation method for automatically suggesting the synthesis and reconfiguration of existing EaaTs in terms of creating mega virtual spaces that integrate new, optimized to user behaviour and profile, EaaTs.

[O2] Facilitate content creation for enhanced MR experiences through data/information

Creation and delivery of suitable, rich and comprehensive content for MR application is a challenging process due to the needs to cater for locating and extracting pertinent content from dispersed sources, satisfy the needs of diverse audience profiles and delivery contexts and maintain content timeliness. To support these needs, ARTIST exploits and advances state-of-the-art technologies from the domains of Linked Open Data, Deep Web, Semantics and Recommender systems to support the tasks of material identification, extraction and tailoring to the needs of individual user profiles and contexts. The means to achieve this objective are the following:

• [O2.1] Implement methods for collecting and integrating application-specific data/information from the open Web, Linked Open Data (LOD) cloud and from virtually accessed RDBMSs (virtual RDF graphs).
• [O2.2] Implement a recommendation method for automatically suggesting external data/info to be attached to related MR content for enhancing it.

[O3] Support the synthesis and reconfiguration of Experiences as a Trajectory (EaaT) within a mega virtual world

A mega (large) virtual world presents numerous challenges and opportunities related to MR experiences’ synthesis, configuration/reconfiguration and delivery; these challenges and opportunities stem from the magnitude and dynamicity of the mega virtual world (larger number of entities, larger virtual spaces, richer interactions), the variability of the experience delivery context (including the diversity of user profiles and user co-participation in experiences, since users are themselves virtualized as elements of the mega virtual world), the availability, richness and deliverability of suitable content, the feedback and metrics collected from previous experience deliveries and so forth. To this end, ARTIST provides a rich toolset allowing for (a) the identification of context, entities, pre-used and on-the-fly designed individual EaaTs and available related data & information (b) the provision of explicit feedback as well as collection and analysis of user traces towards the assessment of satisfaction and efficiency from EaaTs and (c) taking into account all the aforementioned information to create and suggest new, comprehensive EaaTs that transcend multiple virtual and/or physical worlds, towards the realization of the “mega” aspect introduced by ARTIST. The means to achieve this objective are the following:

• [O3.1] Implement methods for tracking and monitoring user behaviour and their interaction with the environment, with the system, and with other users.
• [O3.2] Implement methods for analysing user behaviour and interaction as well as the recorded EaaTs.
• [O3.3] Implement methods for the discovery of semantically related EaaTs (for discovering links between EaaTs with semantic, spatial, temporal relations).
• [O3.4] Implement methods for synthesizing EaaTs into mega ones, integrating virtual and physical entities of more than one virtual and/or physical world (larger number of smart entities, larger virtual spaces, and richer interactions).

[O4] Facilitate smart, multi-entity and trustful interactions between human and non-human entities

In the context of a “mega” virtual experience, users are virtualized into entities that can interact with each other towards some common goal, while additionally the number of entities that are available and have the potential to interact reaches unprecedented levels. The interaction between all mega world entities poses a number of significant challenges, not only related to the technical aspects of the interaction, but also to the level of trust between entities as well as to the perceived usefulness of the interaction in the context of the current experience. ARTIST addresses these challenges by providing a comprehensive framework for sensor data integration, seamless incorporation of both human and non-human entities into the virtual world and allowing for direct incorporation of trust as a core concept underpinning the selection and suggestion of entities that are candidate for interaction in the context of an experience. The means to achieve this objective are the following:

• [O4.1] Implement methods for collecting and integrating sensor data from mobile and smart devices (mobile phones, smart tags, smart bands, etc.).
• [O4.2] Implement methods for capturing and virtualizing human and non-human (mobile and smart) entities, such as users, smart rooms, smart phones, smart bands, smart tags, and storing them in a Virtual Entities (VE) store.
• [O4.3] Develop models and compute trust of virtual entities interaction, based on principles such as friendship, ownership, collaboration, as well as on environmental conditions etc.

[O5] Deliver reusable, sharable MR content and toolset towards a Shared Experiences Ecosystem (SEE)

The entities involved in the MR applications span across multiple domains, including IoT, Semantic web, MR application design & development and Behaviour Analytics. ARTIST uses data, information and knowledge models from the respective areas, extend them where needed, and provide the appropriate means to link elements of the models together, in order to provide full modelling and representation of MR applications. Necessary mechanisms will be also implemented by ARTIST. ARTIST focuses on open models and tools to promote interoperability with existing and forthcoming systems, towards the direction of promoting SMRE2. Finally, ARTIST contributes advances in models and tools to the open source community. The means to achieve this objective are the following:

• [O5.1] Develop models for data/information, virtual entities, user behaviour and EaaTs by reusing existing open and commonly agreed models.
• [O5.2] Develop the models and the ARTIST methods using open and standard technologies (languages, APIs, methods, tools) for MR, Semantic Web, IoT, Data Management, Behaviour Analytics.

[O6] Validate the ARTIST technology capacity-building potential and adaptability

ARTIST approach, including methodologies, models and tools developed within the framework, will be validated regarding their efficiency, comprehensiveness, usability and performance through the implementation of pilot MR applications in multiple and diverse domains. Furthermore, expert opinion will be sought regarding all the aforementioned aspects as well as visionary goals in the MR application and future interactions domains. The means to achieve this objective are the following:

• [O6.1] Develop evaluation methods and tools (front and back-end) for ARTIST, to collect related evaluation data/information in order to assess its usability and performance.
• [O6.2] Develop domain-specific MR apps, to assess the methods and tools integrated in ARTIST to meet objectives O1 to O5 in real use case environments.

ARTIST Architecture

The main components of ARTIST to be implemented for the achievement of the presented objectives are the following: a) EaaT management and analytics, b) IoT/VEs management, c) Data/Information management, and d) User behaviour monitoring and analysis, e) MR authoring environment. In detail, these components (Fig. 2) will encompass tools and methods for the technological means of objectives’ achievement:

1. EaaT management and analytics: This component will provide technological means for a) managing EaaTs (methods for efficiently representing, storing, querying, linking, synthesizing and enriching EaaTs), b) analysing EaaTs (methods for computing similarities between them, identifying different types of relations between one or more EaaTs, methods to support a variety of analytic tasks for EaaT behaviours, computation of statistics, and advanced mining methods for EaaT data co-movement pattern mining). This component utilizes input from the component of a) user behaviour analysis (for recommending reconfiguration of EaaTs based on user behaviour analytics), b) IoT/VE management (for VE integration in EaaTs), and c) Data/Information management (for EaaTs data/info enrichment).
2. IoT/VEs management: This component will provide technological means for a) facilitating connectivity between different types of entities (human and non-human) with ARTIST, b) virtualizing those entities according to the ARTIST ontology, c) managing the virtualized entities (VEs) by efficiently storing and querying the data/information they create during their experiences, d) modelling and computing trust between them, towards facilitating ARTIST trustful interactions.
3. Data/Information management: This component will provide technological means for a) searching and discovering disparate and heterogeneous domain-specific data and information related to EaaTs (from the Web, the LOD cloud, and even the RDBMSs via the utilization of virtual RDF graphs and Ontology-Based Data Access paradigm), b) transform to RDF and integrating this data under a common view, based on domain-specific ontological models and on the ARTIST ontology, c) delivering the integrated/unified data/information as recommendations for enhancing related EaaTs, using semantic matchmaking methods.
4. User behaviour monitoring and analysis: This component will provide technological means for: a) tracking and monitoring human entities in terms of their behaviour while interacting in an MR experience, b) analyse their behaviour to identify states such as boredom, frustration, confusion and uncertainty, c) taking certain intervening actions through comments, recommendations for changing path, additional content presentation, etc., according to the user affective  mental state and preferences, d) dynamically build users physiological profile on top of the initial profile already in ARTIST, e) providing feedback explicitly through an integrated interactive interface.
5. The MR authoring environment: ARTIST will develop and integrate an authoring environment for developing MR apps with low effort. This environment will provide the following functionality: a) the graphical drag-n-drop code-free authoring of EaaTs, b) the synthesis of open and reusable EaaTs, c) the integration of VEs into EaaTs, and d) the enhancement of EaaTs via their semantic enrichment. The output of this environment is a domain-specific MR application that is delivered to users for exploitation. During the use of the MR app, ARTIST is still ‘connected’ and following user’s MR experience via its ‘User behaviour and monitoring’ component. Several versions of the MR app will be delivered based on the optimization capabilities of ARTIST analytics.

Central to ARTIST system architecture is a) a triples store for storing and querying integrated data in Resource Description Framework (RDF) W3C standard data model (W3C 2014) and b) the ARTIST ontology (in OWL W3C ontology language (W3C 2012)). The ARTIST ontology will be designed and developed to represent EaaTs, Virtual Entities (VE) and the required domain specific data/information i.e. a) sensor data from the tracking and monitoring of human behaviour and their interactions with the environment and b) Web/LOD/RDBMS data/info that will be used for EaaTs enhancement (semantic enrichment). A scalable (due to volume of data) triple store will be integrated. Figure 2 depicts three data/information stores as a conceptual approach to the organization of ARTIST data/information management, however this is not necessarily the case of implementing three different physical stores.

The suggested architectural design of the ARTIST system allows for low-effort implementation and deployment of open and reusable MR applications and shared MR experience analytics.

ARTIST Validation

The technological developments in ARTIST will be validated and evaluated in user-defined challenges that aim at increasing the (re)usability of MR authoring tools and the developed MR content and applications within key EU-strategy domains of Retail, Education, Culture, and Space.

·      Domain I : Retail

The aim is to use ARTIST to support the seamless virtual linking of physically separated products and shops . Unique identification on item level and virtualization of physical product will provide asset tracking with benefits all over the product lifecycle: during production/recycle (origin of product, reuse of material), purchase (personalized buying and discounts based on the recycling data, users’ history, user “feeling” when interacting/looking at the product), use (when the product is used and how, was it repaired, how it can be used) and recycling (incentives for recycling for consumers; product passport with material being used for product for further re-usage after the end of the lifecycle). This will deliver enhanced higher quality user experiences leveraging product EaaTs (creating/analysing) of MR shopping experiences of shoppers that want to investigate, buy, consume and recycle products, get feedback on product that buyer interact with, or buy, but without really being present in the real shopping environment. The focus of the pilot will be to provide the service of unique identification and virtualization for as many products as possible and sensing wherever it makes sense (i.e. measuring temperature for guarantee purposes and for devices that are sensitive to temperature).

·      Domain II : Education

The aim of this use case is to support the virtual interconnection of spatially and temporally separated entities (e.g. teachers, students, scientists) while the features that will be designed will allow students to acquire enhanced learning experiences and endorse inquiry-based learning activities. ARTIST technologies will be blended to create and analyze educational EaaTs of MR learning experiences of students, supporting and promoting group collaboration and telepresence, exploiting virtualization of human resources to provide assistance and guidance as well as interaction with remote spaces. The MR courses will allow the students to immerse into a virtual learning/educational environment that would not be possible to be implemented in the real world (due to (a) cost or access to hazardous laboratory equipment and (b) limitations to methods for remote collaboration).

·      Domain III : Culture

ARTIST will be validated in culture domain with two pilot use cases: a) MR Samos Wines Experiences use case, b) A Remote Visit to Lesvos GeoPark (Petrified Forest).

III-A Wine Museum

The purpose of this pilot is to make use of the ARTIST technology to create apps to be used at the Wine Museum of Samos. The smart apps will target the enhancement of visitor EaaTs, providing them with knowledge and entertainment and become an incentive that will lead them to personized shopping. Two scenarios will be delivered: a) an interactive MR story and wander table, b) a taste smart walk at the museum.

III-B GeoPark (Petrified Forest)

The aim of this pilot is to validate ARTIST technology by enabling distant human entities (people with disabilities, hospital patients, elderly people, etc.) to interact with the GeoPark (Petrified Forest) environment (non-human entities) through another human entity that serves as a digital museum-real guide. The proposed scenario will be possible to be experienced in real-time by the remote entity using another VR Headset e.g. from the recovery bed of a patient's hospital. The distant entity will be able to passively follow the virtual path of the on-site entity or complementarily - invariably asking distant entity to insist on an object or to suggest a change to its path (aligned with the suggestions of ARTIST recommender functionality). This whole new experience of visitor EaaTs offered through ARTIST will be customized according to the final recipient. Two different scenarios will be delivered: one with patients/people-in-disability, and another with tourists in a distant hotel.

·  Domain IV : Space Exploration

The purpose of this pilot is to make use of ARTIST technology in the context of the ExoMars ROCC (Rover Operations Control Center) activities to be conducted in ALTEC to support the ExoMars2020 mission. The ARTIST system will be used to develop a suitable MR app to support space training EaaTs in a collaborative and interactive environment where ground operators (Engineers or Scientists) can interact to plan the Rover day by day operations on the Mars surface. ROCC Operators Trainees will use the MR app within this environment in order to be acquainted with Mission processes and procedures as part of their ARTIST-enabled advanced training procedure.

In order to create a sustainable ecosystem of shared MR experiences, ARTIST embraces the principal of Interactive Technologies sustainability, as founded by Blevis (2007) and argued under the concept of Sustainable Interaction Design. Mankoff et al. (2007)Mankoff et al. 2007 extended the sustainability on interactive technologies according to two major categories, sustainability through design (pertaining to the ways that interactive technologies can be designed to support sustainable behaviour) and sustainability in design (focusing on the design itself that ensures that the technology is sustainable). Huang (2011) expressed the importance of validation in real-world situations while DiSalvo et al. (2010) DiSalvo et al. 2010, through empirical analysis, strongly encouraged the use of formative user studies and “environment awareness” in the sense of exploiting the qualities of the associated environment for sustainability. Thus, the ARTIST offerings are based on a sustainable-in-design MR ecosystem that includes all stages of development and deployment of the integrated technologies, from authoring to synthesis to validation of MR experiences. Sustainability though design is ensured by the validation of the approach in real-world use cases involving both end-users as well as paired technology integrators from industry that aim not only to deploy the ARTIST offerings but also to create the experiences per situation and lead the formative user studies for usability evaluation focusing on the integration and long-term impact of the ARTIST technologies to the specifics of the use environment.

The guidelines of the World Economic Forum (2018)  for IoT technologies sustainability World Economic Forum 2018 identify that a major hindering to long-term sustainability of IoT technologies is fragmentation, which can be addressed by eliminating the negative effects through low cost development and deployment and increased interoperability by engaging the experts in all aspects of the technology lifecycle. ARTIST aims to create an ecosystem that is designed and used by both technical and application domain experts. The aim is to make low cost development of mainstream technology available to all industries. The technology will be available to all creators (end users and their technical business collaborators/integrators) to create their own MR experiences. The exact same type of use will be demonstrated, validated and evaluated by different pilots. Business Europe (2017) Business Europe 2017 presents the challenges and factors that will most probably drive the short- and long-term vision of Europe’s industry. Especially for the later, the rate of adaptability to new technologies is crucial, especially in the domains of education and training. The same is reported in the special session on the “social dimension of digitization” where IoT (and 5G) are at the centre of importance “allowing for a change in the paradigm of interaction between human being and technology” in situations such as e-learning, culture and others. To this extend the technologies proposed by ARTIST directly apply to the EU vision on both vertical and the horizontal axes (see table 1 below).

The overall value/impact of the presented approach in MR systems development and deployment can be outlined in the following offerings:

• Low-effort authoring of MR experiences
• Tracking, monitoring and analysis of user behaviour and interactions between human and non-human entities
• Discovery and integration of heterogeneous and disparate data/information related to the MR application content
• Synthesis of MR experiences towards a mega virtual world
• Higher quality, optimized MR experiences based on cognitive performance monitoring, psychological profiling and analysis of past experiences.

Vertical and horizontal topics were included in the roadmap for leveraging European assets, within the context of Think Virtual European initiative, presented by EuroVR (European Association for Virtual Reality and Augmented Reality) and the New European Media (NEM), while presenting the 'Future of AR/VR in Europe' (ICT Proposers' Day 2017). Think Virtual aims at facing the key challenges identified in Horizon 2020 of the European Commission that leads towards a Flagship Initiative for NGI (EuroVR and NEM 2017). ARTIST aims at delivering results with strong impact in EU vertical as well as horizontal dimensions of AR/VR/MR technology, as shown in the following table (Table 1):

This paper has presented ARTIST research approach that introduces novel methods and tools for real-time multi-entity interactions between human and non-human entities, to create reusable and optimized Mixed Reality experiences with low-effort, towards a Shared Mixed Reality Experiences Ecosystem. ARTIST will deliver high quality MR experiences, facilitating the interaction between a variety of entities which interact in a virtual and symbiotic way within a mega, virtual and fully-experiential world.  Specifically, the proposed approach introduces novel methods for code-free implementation and deployment of open and reusable MR content, applications and tools, around the novel concept of an Experience as a Trajectory (EaaT). In addition, it introduces tools for the tracking, monitoring and analysis of user behaviour and their interaction with the environment and with other users, towards optimizing MR experiences by recommending their reconfiguration, dynamically (at run-time) or statically (at development time). Finally, the presentred approach introduces tools for synthesizing experiences into new, mega, and still reconfigurable EaaTs, enhancing them at the same time using semantically integrated related data/information available in disparate and heterogeneous resources.

This research idea has been further elaborated by an international consortium within the context of an H2020 proposal.