Olfactory Interaction based on ISO/IEC 23005 Standard

297 J. Sensor Sci. & Tech. Vol. 26, No. 5, 2017 Journal of Sensor Science and Technology

Vol. 26, No. 5 (2017) pp. 297-300 http://dx.doi.org/10.5369/JSST.2017.26.5.297 pISSN 1225-5475/eISSN 2093-7563

Olfactory Interaction based on ISO/IEC 23005 Standard

Jang-Sik Choi

, Sung-June Chang

, Hae-Ryong Lee

, and Hyung-Gi Byun

Abstract

Realistic media comprised of metadata of the five senses to provide enhanced experiences by stimulating our memory and sensations have had an increasingly pervading effect in our daily lives. Many researchers and companies are in the process of developing their own authoring systems running on different platforms to serve realistic media, resulting in compatibility issues among the systems. To tackle these issues, the International Organization for Standardization have standardized the interface, data format, protocol, API, etc. required to provide the realistic media. In particular, the ISO/IEC 23005 standard, which is called MPEG-V in SC29/WG 11, has defined XML schemas for olfaction interaction based on electronic nose (E-Nose), and scent display. In this paper, the MPEG-V standard for olfaction interaction is reviewed, and a data flow diagram that can be used for olfactory interaction based on the MPEG-V standard was designed.

In addition, the necessary schemas related to the E-Nose sensor for olfactory interaction was provided.

Keywords: Olfactory Information Standardization, E-Nose, MPEG-V, Olfactory Interaction

1. INTRODUCTION

Realistic media could be provided to a user through generation, processing, storage, transmission, and reproduction of metadata that defines the description schemes, and descriptors to represent sensory effects [1]. In particular, generation, and reproduction of the metadata are carried out using diverse sensory sensors, and actuators, respectively [2]. Different groups have implemented these sensors and actuators with different technologies, operated on different platforms. In addition, their command and data formats for sensory effects vary with their applications. Such differences affect the interoperability among the technologies, and make applying and creating realistic media difficult.

To tackle these problems, the MPEG-V (ISO/IEC 23005) standard was published in 2011 [3]. This provides standards for normative data format and command for the sensory effect of realistic media [4,5]. The MPEG-V standardization was conducted by a 3DG ad-hoc group in SC29/WG11.

This standard is used to define XML schema regarding data format, command, capability of sensor or actuator, preference, etc., for olfaction interaction based on the E-Nose and scent display. In this paper, we reviewed the MPEG-V standard for olfactory interaction, and designed a data flow diagram for olfactory interaction based on the MPEG-V standard. Moreover, we provided necessary schemas related to the E-Nose sensor for olfactory interaction.

2. MPEG-V(ISO/IEC 23005)

MPEG-V provides the system architecture to ensure interoperability between the virtual world, and the real world comprised of sensors, the device, and the user [6]. In addition, this standard specifies the representations of the associated sensory effects. The architecture of the MPEG-V framework is depicted in Fig. 1. Media exchange between the real world and the virtual world are accomplished using adaptation engines.

The MPEG-V standard specifies the elements and types of the data shown by the labeled arrows in Fig. 1, and consists of 7 parts:

1. Architecture: This part specifies the architecture of MPEG-V, and its three associated uses are: information adaptation from virtual world to real world, information adaptation from real world to virtual world, and information exchange between virtual worlds [7].

2. Control information: This part specifies the syntax and semantics of the tools required to provide interoperability in controlling devices in both the real and virtual worlds [8].

1

Division of Electronics, Infornation & Communication Eng., Kangwon National Unversity, Kangwon National University, 346 Joongang-no, Samcheok, Gangwon-Do, 25913, Korea

2

SW-Contents Laboratory, ETRI, 218 Gajeong-ro, Gaejeon, 34129, Korea

+

Corresponding author: [email protected]

(Received: Jul. 24, 2017, Revised: Sep. 25, 2017, Accepted: Sep. 26, 2017)

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License(http://creativecommons.org/

licenses/bync/3.0) which permits unrestricted non-commercial use, distribution,

and reproduction in any medium, provided the original work is properly cited.

Jang-Sik Choi, Sung-June Chang, Hae-Ryong Lee, and Hyung-Gi Byun

J. Sensor Sci. & Tech. Vol. 26, No. 5, 2017 298 3. Sensory information: This part specifies the syntax and semantics of description schemes and descriptors that represent sensory information. This is applied to enhance the experience of users consuming media resources by stimulating multiple human senses, such as tactile, olfactory, light sense, temperature sense, etc. [9]

4. Virtual world object characteristics: This part specifies the syntax and semantics of the description schemes and descriptors used to characterize the metadata of a virtual world object, making it possible to migrate the object—or only its characteristics—from one virtual world to another, or to control the object using real world devices [10].

5. Data formats for interaction devices: This part specifies the syntax and semantics of the data formats for interaction devices—

i.e. device commands, and sensed information—required for providing interoperability in controlling the devices, and in sensing information from them in both the real and virtual worlds [11].

6. Common types and tools: This part specifies the syntax and semantics of the data types and tools common to the tools defined in the other parts [12].

7. Reference software: This part specifies the conformance and reference software implementing the normative clauses of all parts [13].

3. Olfactory interaction using MPEG-V

We designed a data flow diagram for olfactory interaction by

considering the MPEG-V standards, as shown in Fig. 2.

In this figure, user A sets the configurations of a smartphone and the E-Nose sensor up for capturing odor-related media, and sensing olfactory information using DCDV:ScentCapabilityType, SCDV:EnoseCapabilityType, and SEPV:ScentPrefType, as defined in the control information part of the MPEG-V standards.

The description of each of these types is listed in Table 1. In particular, the SCDV:EnoseCapabilityType uses EnoseSensor- TechnologyCS—of which the binary representation is listed in Table 3—to characterize the capability of each E-Nose sensor.

The descriptions of the abbreviated terms in type names thus far are listed in Table 2.

Next, the user commands the smartphone, and the E-Nose sensor to receive the media data and olfactory information. The captured media and sensed olfactory information are described Fig. 1. MPEG-V system architecture.

Fig. 2. Data flow diagram for olfactory interaction using MPEG-V.

Table 1. MPEG-V control information types required for olfactory interaction

Type Description

DCDV:ScentCapabilityType Capabilities of scent generating devices.

SCDV:EnoseCapabilityType Capabilities for E-Nose SEPV:ScentPrefType User's preferences towards scent effects.

Table 2. Abbreviated terms in MPEG-V control information type names

Part Abbreviated

term Description

2

DCDV device capability description vocabulary SCDV sensor capability description vocabulary SEPV sensory effect preference vocabulary

3 SEV Sensory effect vocabulary

5 DCV Device Command Vocabulary

SIV Sensed Information Vocabulary

Olfactory Interaction based on ISO/IEC 23005 Standard

299 J. Sensor Sci. & Tech. Vol. 26, No. 5, 2017 using complex types supported by MPEG-7 and MPEG-V

(SIV:EnoseSensorType, defined in the supported data formats part of MPEG-V), respectively. The description of the SIV:EnoseSensorType is provided in Table 4. To describe the olfactory information sensed from the E-Nose, two classification schemes are used: ScentCS, which is used to describe the gas mixtures recognized by the E-Nose (listed in Table 5), and GasTypeCS, which is used to describe the individual gases that the E-Nose recognizes (listed in Table 6). These classification schemes provide 312 term IDs, and 15 term IDs for gas mixtures, and individual gases, respectively.

The olfactory-media composer encodes the captured and sensed data into olfaction-enhanced media. This olfaction-enhanced media contains scent effects represented by SEV:ScentType, as

defined in the sensory information part of the MPEG-V standards.

The description of SEV:ScentType is given in Table 7.

The olfaction-enhanced media is transmitted to a database for user B who wants to consume it. In the case where a user wants to consume the media, the olfaction-enhanced media is decoded to two complex types to be transmitted to a smartphone and a scent display. The DCV:ScentType defined in the supported data formats part of MPEG-V is used to control class, intensity (strength), and duration of the scent on the scent display.

4. Advanced MPEG-V schemas for olfactory interaction

It is important to set the sensor adaptation preferences of the E- Nose sensor of a user up to make user-centric olfaction-enhanced media. However, MPEG-V does not provide such schema yet.

We, therefore, propose EnoseSensorAdaptationPrefType to describe the E-Nose sensor adaptation preferences of the user, as shown in Fig. 3.

To describe the E-Nose sensor adaptation preferences of a scent, FavorableScent element is defined in the schema.

Most of the outputs of the E-Nose sensor are represented in a multi-dimensional data form. This data form—called fingerprint—represents the sensed olfactory information from the E-Nose. Hence, we propose EnosePatterDataType inside SIV:EnoseSensorType, as shown in Fig. 4.

Table 3. Binary representation of EnoseSensorTechnologyCS

Binary code Type

0000 MOS_sensor

0001 MOSFET_sensor

0010 CP_sensor

0011 SAW_sensor

0100 QMB_sensor

0101 - 1111 Reserved

Table 4. MPEG-V supported data format types required for olfactory interaction

Type Description

DCV:ScentType Device command type which can generate a scent effect.

SIV:EnoseSensorType Sensed information from E-Nose Table 5. Binary representation of ScentCS

Binary code Type

000000000 rose

000000001 acacia

000000010 chrysanthemum

…

101000001~111111111 Reserved

Table 6. Binary representation of GasTypeCS.

Binary code Type

0000000000000001 carbon monoxide 0000000000000010 carbon dioxide 0000000000000011 sulfurous acid

…

101000001~111111111 Reserved

Table 7. MPEG-V sensory information types required for olfactory interaction

Type Description

SEV:ScentType Scent effect to enhance the experience of users while consuming media resources.

Fig. 3. Diagram of SAPV:EnoseSensorAdaptationPrefType

Jang-Sik Choi, Sung-June Chang, Hae-Ryong Lee, and Hyung-Gi Byun

J. Sensor Sci. & Tech. Vol. 26, No. 5, 2017 300 We expect that the EnoseSensorType including proposed EnosePatternDataType could be used for various other applications involving olfactory interaction.

Fig. 5 shows an example of EnosePatternData. In the example, the pattern data is described on a 10×8 matrix. This matrix can be used as the input of the odor recognizer installed in the realistic media authoring tool for olfaction-enhanced media.

5. CONCLUSIONS

In this paper, we reviewed the MPEG-V standard for olfaction interaction, and designed a data flow diagram that can be used for olfactory interaction based on the MPEG-V standard. In addition, we provided necessary schemas for the E-Nose sensor for olfactory interaction.

Commercialization of olfaction-enhanced media as realistic media is being delayed due to insufficient development, and not having had their information standardized. Therefore, we believe that the proposed diagram and schemas for olfaction interaction in this paper could accelerate its commercialization, while ensuring its interoperability.

ACKNOWLEDGMENT

This work was supported by the Institute for Information &

Communications Technology Promotion (IITP) grant funded by the Korean government (MSIP) (No.2015-0-00318, Olfactory Bio Data-based Emotion Enhancement Interactive Content Technology Development).

REFERENCES

[1] S. Y. Kim, S. U. Yoon, and Y. S. Ho, “Realistic broadcasting using multi-modal immersive media”, Advances in Mul- timedia Information Processing-PCM, pp. 164-175, 2005.

[2] S. K. Kim, “Authoring multisensorial content”, Signal Pro- cessing: Image Communication, Vol. 28, No. 2, pp. 162- 167, 2013.

[3] https://en.wikipedia.org/wiki/Moving_Picture_Experts_Gr- oup (retrieved on July 14, 2017).

[4] B. Rainer, M. Waltl, E. Cheng, M. Shujau, C. Timmerer, S.

Davi, I. Burnett, C, Ritz, and H. Hellwagner, “Investigating the impact of sensory effects on the quality of experience and emotional response in web videos”, In Quality of Mul- timedia Experience (QoMEX), 2012 Fourth International Workshop on (pp. 278-283). IEEE.

[5] Y. S. Joo and S. K. Kim, “Sensory effect authoring tool for sensible media”, Journal of Broadcast Engineering, Vol. 16, No. 5, pp. 773-781, 2011.

[6] K. Yoon, S. K. Kim, J. J. Han, S. Han, and M. Preda,

“MPEG-V: bridging the virtual and real world”, Academic Press, 2015.

[7] ISO/IEC 23005-1:2016, Information technology - Media context and control - Part 1: Architecture, July 2016.

[8] ISO/IEC 23005-2:2016, Information technology - Media context and control - Part 2: Control information, Mar.

2016.

[9] ISO/IEC 23005-3:2016, Information technology - Media context and control - Part 3: Sensory information, July 2016.

[10] ISO/IEC 23005-4:2016, Information technology - Media context and control - Part 4: Virtual world object char- acteristics, Mar. 2016.

[11] ISO/IEC 23005-5:2016, Information technology - Media context and control - Part 5: Data formats for interaction devices, Mar. 2016.

[12] ISO/IEC 23005-6:2016, Information technology - Media context and control - Part 6: Common types and tools, Mar.

2016.

[13] ISO/IEC 23005-7:2014, Information technology - Media context and control - Part 7: Conformance and reference software, Jan. 2014.

Fig. 4. Diagram of SIV:EnoseSensorType

Fig. 5. Example for EnosePatternData