该文章提出了一种可触摸的3D视频系统，允许观众通过力反馈设备与视频场景互动。系统包括内容生成、内容传输和观看与交互三个功能组件。通过深度图像基的触觉表示方法（DIBHR）编码视频媒体的触感表面属性，结合高度图图像提供更丰富的交互体验。基于MPEG-4框架设计新的BIFS节点来描述触感场景，并采用一种计算力反馈算法来计算场景与观众之间的交互力。该算法在1kHz更新率下运行，提供稳定且平滑的接触力，尤其适用于具有高频几何噪声的深度图像。
摘要：
Multimedia technologies are reaching the limits of providing audio-visual media that viewers consume passively. An important factor, which will ultimately enhance the user's experience in terms of impressiveness and immersion, is interaction. Among daily life interactions, haptic interaction plays a prominent role in enhancing the quality of experience of users, and in promoting physical and emotional development. Therefore, a critical step in multimedia research is expected to bring the sense of touch, or haptics, into multimedia systems and applications. This article proposes a touchable 3D video system where viewers can actively touch a video scene through a force-feedback device, and presents the underlying technologies in three functional components: (1) contents generation, (2) contents transmission, and (3) viewing and interaction. First of all, we introduce a depth image-based haptic representation (DIBHR) method that adds haptic and heightmap images, in addition to the traditional depth image-based representation (DIBR), to encode the haptic surface properties of the video media. In this representation, the haptic image contains the stiffness, static friction, and dynamic friction, whereas the heightmap image contains roughness of the video contents. Based on this representation method, we discuss how to generate synthetic and natural (real) video media through a 3D modeling tool and a depth camera, respectively. Next, we introduce a transmission mechanism based on the MPEG-4 framework where new MPEG-4 BIFS nodes are designed to describe the haptic scene. Finally, a haptic rendering algorithm to compute the interaction force between the scene and the viewer is described. As a result, the performance of the haptic rendering algorithm is evaluated in terms of computational time and smooth contact force. It operates marginally within a 1 kHz update rate that is required to provide stable interaction force and provide smoother contact force with the depth image that has high frequency geometrical noise using a median filter.
展开
扫一扫
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
