Tactile Internet Service Requirements

Tactile Internet (TI) was defined as a new wave of innovation after the successful Internet of Things (IoT) . In fact, Tactile Internet (TI) can be regarded as a new ICT paradigm with extreme emphasises and service requirements on multiple performance metrics such as latency, availability, reliability, and security.TI finds its application in many emerging application scenarios, including, but not limited to, Industry, Robotics and Telepresence, eXtended Reality (e.g., Augmented Reality, Virtual Reality and Mixed Reality), Healthcare, Gaming, and Teleoperation. These extreme service requirements from TI applications pose new challenges to both communication and computing. Although existing networking architecture and protocols can support some of these service requirements partially (e.g., 5G URLLC ), a still pending question is whether and how a holistic and systematic approach should be developed in order to efficiently support TI applications. Moreover, IEEE 1918.1 standards working group on TI is formed to investigate aspects related to TI applications, architecture and haptic encoding.

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 .

TI - Tactile Internet TD - Tactile Devices UE - User Equipment URLLC - Ultra-Reliable Low-Latency Communications AR - Augmented Reality VR - Virtual Reality PPE - Personal Protective Equipment ISOBMFF - ISO Base Media File Format QoE - Quality of Experience QoS - Quality of Service AES - Advanced Encryption Standard WEP - Wired Equivalent Privacy WPA - Wi-Fi Protected Access

This section aims to introduce the reader to distinct, although not exhaustive, TI applications which are widely being discussed in the TI research community.

Automation, smart factories and remote operation are some of key industry use cases that are enabled by TI . Moreover, repair and maintenance in remote areas, in high-risk scenarios requiring high precision requires multi-modal and low latency communication provided by TI. For example, in such scenarios, human operators can control machinery (e.g., robots) remotely and perform complex operations , where either it is too dangerous for humans to be present, or it’s not possible for the experts to be physically present at the environment where the operations are conducted. The controlled machinery may be equipped with various sensors for providing information about the environment to the operator, while it may also be equipped with required actuators for performing corresponding tasks as instructed by the constructor over the TI. TI may also enable the transmission of critical information (e.g., alerts) to human users (e.g., through connected PPE as AR and haptic data) who perform operations in high-risk environments. Alerts may be automatically generated based on information gathered from sensors, or sent by human users, over the TI.

Key health applications of TI include, tele-surgery , tele-mentoring, tele-rehabilitation and tele-diagnosis . Specifically, minimising the invasive nature of surgery has been a focus of the heath technology industry and has currently been widely used due to the small tissue damage and fast recovery it incurs.Today, surgeons use surgical robots for performing highly precise operations. Providing tactile feedback is specifically critical for performing operations which require high precision manipulation. Although, it is not always possible to get specialist surgeons on site for performing operations on patients, TI enables surgeons to perform such critical operations remotely, where it requires only the machinery (high precision robots) to be co-located alongside the patient.

The advancements in Augmented Reality (AR) & Virtual Reality (VR) technology as well as the increased number of applications developed for user entertainment (e.g., VR gaming, VR tourism, VR art) have significantly increased the interest for further improving the immersive experiences those application provide. VR applications enable human users, or a collection of human users to interact with a virtual environment where the provided immersive experience is similar to that of a real physical interaction. Haptic feedback is a key element in such interactions, allowing the user to experience the sense of touch along with audio and visual(e.g., users perceiving the effect of each other’s actions in collaborative scenarios).

TI enables learning experiences where tactile feedback plays a crucial role. This may substantially improve both the learning as well as the teaching experiences in remote learning scenarios. The teacher will be able to experience (see, hear, feel) actions performed by the learner and correct any errors as if they are in a real physical (face-to-face) learning environment. Such applications include, remote military and sports training which requires problem solving by collaborating with remote team members, while incorporating feedback provided by the remote trainer in real-time. Furthermore, Internet of Skills application aims at training people in remote and diverse locations to improve their skills and capabilities. It combines advances in motor training and Tactile Internet with Human-in-the-loop to achieve the goal of transferring high quality skills to populations that otherwise do not have access to such training. Moreover, the goal of Surgical Assistance and training application is to develop a system that provides assistance to an expert surgeon during a surgery or to provide surgery training to students. Such a system is envisaged to be continuously learning and acquiring expert knowledge. To do this, the system interprets sensor data as it observes an expert surgeon performing their procedure.

Various sensors, actuators, display devices are used to provide a realistic haptic and multimodal interaction with the remote devices over a uni-directional or bi-directional communication. The sensor components capture the tele-manipulation instructions (e.g., kinaesthetic), and the resulting changes (e.g., haptic feedback). Actuators execute the user’s tele-manipulation instructions. The number of independent coordinates used for providing the end user experiences (using Human System Interfaces), and for controlling the velocity, position, and the orientation of the controlled devices is defined by their degree of freedom (DoF). Capabilities of UEs in collecting biometrics can enhance security solutions (such as user identification and authentication). While existing authentication mechanisms relay on SIM (subscriber identity module or subscriber identification module) cards in mobile devices, unique biometrics collected from the users can be used to enhance the security. Considering a scenario where the SIM card token is stolen, an alternative/complementary method of ensuring network connectivity for the genuine user would involve the use of biometrics as these cannot easily be stolen. Biometrics offers a solution to the weaknesses of knowledge and token-based systems. Examples of continuous biometrics are face, iris, keystroke dynamics, touchscreen gestures, behavioural profiling (e.g. Bluetooth/Wifi/GPS), gait, mood and one-shot biometrics are face, iris, and fingerprint that can be collected by the new UE.

As a result of the research and developments in TI, this section presents service requirements to be addressed by the networking community.

Unlike audio and video, there has not been any haptic media types in standards, until a very recent development in standards to register haptics as a top-level media type. A proposal to introduce haptics as a first-order media type in ISO Base Media File Format (ISOBMFF) was accepted by MPEG Systems File Format sub-group. This standardization process is expected to conclude in October 2021, making haptics a part of the ISO/IEC 14496-12 (ISOBMFF) standard. Providing this recent development, the authors make a case for haptics to be added to the list of top-level media types recognised by the IETF. The authors further argue that ‘application’ top-level type not suitable for haptics as, like audio/video haptics is related to a separate sensory system. Moreover, ‘application’ is historically used for application code, and haptics is not code but a property of a media stream (like audio and video). Therefore, we believe that the adoption of a top-level haptics media type in IETF is an important step towards further development of haptic communication.

Most Haptic applications demands stringent latency requirements from the underlying communication. Specifically, ultra-low latency, 1ms for haptic interaction , is demanded for providing timely delivery of messages between communicating devices by TI applications. The timely delivery of control messages is crucial for critical TI applications such as TI remote surgery. Moreover, timely delivery of messages also assists in playback of multi modal streams (audio, video, haptic) in a synchronous manner, providing a consistent experience that is devoid of cybersickness.

Ultra-high reliability is required by several TI applications. For example, it is not acceptable for communication reliability to be hindered during critical TI applications such as alert transmission for connected PPE (described Section ). Thus, it is crucial that ultra-reliable communication is a key enabler of TI applications.

The tactile applications often consist of several streams, e.g., audio, video, haptic, each stream with varying service requirements (bitrates, latency, level of reliability). Moreover, depending on the use case and the deployment scenario, streams of an application may be distributed among multiple tactile/terminal devices, e.g., video stream to display, audio stream to sound system, haptic stream to haptic suit. However, all such streams must be played back to the user in a synchronous manner when providing multi-sensory immersive experiences. Especially, in scenarios where a user uses multiple UEs/terminals for consuming the same user experience, media streams (haptic, audio, video) must be delivered and played to the user in a synchronous manner (e.g., avoiding Cybersickness ). Due to network conditions and the insufficient support/assistance for synchronization, related streams may arrive at different UEs/terminals out of synchronization (e.g., the lack of information related to inter-dependency among network flows ). Therefore, mechanisms for the coordination (see section for detailed discussion) and synchronization of multiple flows, for both the same destination/UE, and for multiple destinations/UEs must be introduced.

Emerging TI applications are highly diverse in terms of their use case requirements and constraints. For example, a TI application may comprise multiple streams (e.g., due to multi-modal nature), each of which may be required to be treated differently by the network based on their use case requirements and constraints; some streams may need high bandwidth and ultra-low latency while some others may require ultra-high reliability. The conventional interaction model between applications (end-hosts) and networks are insufficient to deliver the traffic of these emerging TI applications. In other words, applications should not consider the network as a black-box anymore and in turn they should not entirely rely on the end-to-end measurements for adapting their behaviour as the underlying network condition changes rapidly, mainly because the end-to-end measurements are implicit and thus coarse-grained. To this end, a new collaborative paradigm between applications and networks need to be realized. This way, applications and networks can express their desired use case requirements and constraints to one another, permitting applications in particular to adapt themselves to network constraints and the networks to orchestrate their resource distribution according to the applications' requirements if desired. This is particularly essential for TI terminals which have to run highly diverse applications/services often with conflicting requirements.

Applications in TI typically follow a multi-modal communication pattern in which the end-to-end communication between tactile devices (TDs) includes several modes of communication at the same time (e.g., video, audio and haptic). This results in generation of multiple coordinated streams in parallel which ultimately need to be presented to an end user in harmony. Otherwise, the quality of experience (QoE) of the user may not be satisfactory due to lack of precise synchronization across these parallel streams. For example, one stream may get delayed while others are delivered on time. Apart from the synchronization challenges (see also Section 5.4 for more detailed discussion), the instability of the underlying network condition of a stream may also impact the performance of the other coordinated parallel streams of the same TI application, which may ultimately reduce the overall QoE of users.Therefore, it is crucial to have mechanisms particularly tailored for coordination (e.g., data packet scheduling across multiple terminals and/or access networks) so that varying network condition across multiple networks can be intelligently handled. The key goal here is to distribute data packets without creating network congestion and/or increasing end-to-end delay. These type of communications can also significantly benefit when there is a feedback loop mechanism between TI applications (terminals) and networks (see Section for more details).

The TI use cases are highly dynamic in nature. Especially, in multi-user scenarios where user profiles, their dynamic relations and interactions are taken into consideration, e.g., virtual simulation environments used for training, where multiple users act upon the same virtual objects, the information received by individual 'trainee users' may differ due to 1) user preferences (e.g., with haptic feedback vs without) 2) specific user's perception (e.g., audio, video haptic) of objects and actions/events within the virtual environment (e.g., based on viewpoint, distance to objects/events, and properties of virtual objects). Moreover, the trainer (human or virtual) may choose to provide corresponding feedback (using audio-visual or audio-visual-haptic mediums) to an individual or a group/subset of trainees in real-time. Different users may receive different haptic feedback depending on the type of actions performed and therefore the experiences may differ for each user. When providing such experiences the resulting dynamicity must be considered. Therefore, the multi-modal information provided to each user, through data streams, may be personalised (e.g., based on distinct user perception and user profile).

This document requests no IANA actions.

Security and trust as well as communication latency are key challenges for delivering tele-surgery. Conventional internet security protocols (namely, AES, WEP, WPA) are used to make the data transfer prone to attack. Security and reliability of the haptic data locally/remotely are key to Tactile Internet use-cases such as telesurgery use-case. Further work is required on security/privacy aware haptic data/feedback encoding techniques to improve the reliability and security of the TI use-cases. Furthermore, continuous monitoring demands low-power and reliable operation to avoid any interruption in data collection from power restricted devices and therefore the service delivery .

This draft presents the emerging area of Tactile Internet, its key use cases and service requirements. The introduction of haptic communication, a new mode of communication, not only improves existing immersive experiences (e.g., AR/VR) while also facilitates new emerging Tactile immersive experiences (e.g., tele-surgery). Moreover, the resulting communication over the Tactile Internet demands for stringent service requirements on the underlying communication networks, e.g., ultra-high reliability, ultra-low latency transmission, security consideration and synchronization of multi-modal data (including haptic). Therefore, We believe IETF is a key forum for addressing some of the potential challenges described, for realizing the envisioned Tactile Internet, and for standardizing relevant aspects such as protocols.

The authors would like to thank Renan Krishna for reviewing and providing useful comments.