Abstract
In order to keep track of the position and motion of our body in space, nature has given us a fascinating and very ingenious organ, the inner ear. Each inner ear includes five biological sensors—three angular and two linear accelerometers—which provide the body with the ability to sense angular and linear motion of the head with respect to inertial space. The aim of this paper is to present a dynamic virtual reality model of these sensors. This model, implemented in MATLAB/Simulink, simulates the rotary chair testing which is one of the tests carried out during a diagnosis of the vestibular system. High-quality 3D animations linked to the Simulink model are created using the export of CAD models into Virtual Reality Modeling Language (VRML) files. This virtual environment shows not only the test but also the state of each sensor (excited or inhibited) in real time. Virtual reality is used as a tool of integrated learning of the dynamic behavior of the inner ear using ergonomic paradigm of user interactivity (zoom, rotation, mouse interaction, etc.). It can be used as a learning and demonstrating tool either in the medicine field—to understand the behavior of the sensors during any kind of motion—or in the aeronautical field to relate the inner ear functioning to some sensory illusions.
1. Introduction
The vestibular apparatus is located in the inner ear and is vital for our dynamic equilibrium. It constitutes a three-dimensional inertial-guidance system. Since the 1950s, the advent of aerospace flight with its new demands has accelerated the pace of vestibular research. Furthermore, a full understanding of the mechanics of a healthy inner ear may contribute to the diagnosis and treatment of the vestibular part in a diseased state. This is the reason why several authors have studied the mechanics of the semicircular canals which detect changes in angular acceleration, and the otolith organs (the utricle and saccule) which are known to sense changes in linear acceleration and gravity. The first model regarding the canals was proposed by Steinhausen [1] and is known as the classical torsion pendulum system, which has been the benchmark for subsequent works (Groen [2], van Egmond et al. [3], Njeugna and kopp [4], Fernandez et al. [5]). Since Steinhausen, several models have been established, including, perhaps most effectively, the Navier-Stokes equations (Oman et al. [6], Rabbitt et al. [7], Steer [8], van Buskirk et al. [9]). Concerning the otolith organs, the works of Grant and Kondrachuk should be mentioned [10–13]. All these models lead to a transfer function between the output of each sensor and the angular/linear acceleration of the head with respect to an inertial space.
The scope of this paper is to design a dynamic virtual reality model, which simulates the inner ear sensory system. This numerical model takes into account not only the angular sensors (semicircular canals) but also the linear sensors (otolith organs). Furthemore, a Graphical User Interface (GUI) has been developed in order to simplify the use of this model. This modeling is linked to a virtual reality world in order to see the theoretical state of each sensor during any motion. Therefore, it offers the possibility to get a better overall understanding of the vestibular apparatus. Indeed, simulation using virtual reality tools affords flexible and versatile tools to improve learning, data gathering, and analysis.
This model simulates the rotary chair test which is one of the procedures usually performed by specialists during a diagnosis of the vestibular system; this will be explained in-depth later in this paper. To perform this numerically, the model follows different steps. First, it resolves the motion equations in each coordinate system. Second, it computes the angular acceleration vectors projected on the perpendicular of each canal plane and the linear acceleration vectors projected on the surface of the otolith organs. Third, the displacement of each sensor is derived using their transfer function (see Section 3). Finally, these data, which constitute the inputs of the virtual scene, are transferred to the virtual model.
2. Anatomy and Physiology
The “balance system” is a reflex system that allows us to maintain awareness of our spatial orientation at all times, and makes us react to it. Without it, we will not be capable to walk upright or follow objects with our eyes when we are moving. For purposes of illustration, we will think of the balance system as a “black box,” with inputs and outputs (Figure 1). This system has 3 categories of sensory inputs to provide spatial orientation cues: visual, somatosensory—of which sensory receptors cover the skin, muscles, bones, and joint to produce the sensory modalities such as touch and proprioception—and vestibular. The developed model in the present paper is devoted to the latter component, that is, the vestibular part.
The vestibular system is located within the temporal region of the skull (in the inner ear) and consists of two specialized types of sensory systems: the semicircular canals—which respond to angular acceleration, and two otolith organs—which primarily detect changes in linear acceleration and gravity (Roman [14], Sauvage [15]). These sensory systems consist of fluid-coupled structures that induce a motion-sensitive signal on the vestibular nerve. This signal is then transmitted to the nervous central system where other afferent systems such as vision and proprioception also converge for spatial orientation, postural stability, and gaze stabilization.
Anatomically, the semicircular canals consist bilaterally of three sets of membranous ducts suspended in a fluid called perilymph, and are oriented in almost mutually orthogonal planes (Figure 2). The membranous structure is filled with another Newtonian incompressible fluid called endolymph (Steer [8]). Each canal contains a gelatinous membrane known as the cupula that completely seals the semicircular canals (Hillman and Mclaren [16]). Angular motion sensation relies on inertial forces, caused by head accelerations, to generate endolymph fluid flow within the toroidal semicircular canals. More precisely, when the head rotates, the endolymph in the canals lags behind due to its inertia and produces a force across the cupula, deflecting it in the opposite direction of head movement. That deflection causes a sensation of motion. At a constant rotation rate, the endolymph in the canals tends to catch up with the rotation of the head due to the viscosity, eliminating the relative movement. Eventually, as long as the rotation rate remains constant, the cupula returns to a vertical position due to its elastic properties and the sensation of motion eventually ceases. Because of the imperfect orthogonality of the three canals, each canal can be stimulated by any rotation. However, it has been shown that each canal admits a specific direction of stimulation, which maximizes the excitation (Rabbitt [17]).
Concerning the otolith organs, they are approximately perpendicular to each other. They are flat layered structures. The top layer consists of calcium carbonate crystals (called otoconia), the middle layer consists of a gel structure, and the bottom layer is referred to as the sensory base containing receptor cells. The otoconia have a density of 2.71 whereas the gel layer consists of a highly deformable viscoelastic gel having a density of 1.0 . Consequently, during linear accelerations of the head, the inertia of the dense otoconial layer gives rise to relative displacement between it and the sensory base, which generates a stimulus (Figure 3).
3. Macromechanical Model of the Sensors
3.1. Single Semicircular Canal
Angular motion sensation relies on inertial forces, caused by head accelerations, to generate endolymph fluid flow within the toroidal semicircular canals. This fluid flow is described by the torsion pendulum model, which arises from the works of Steinhausen [1], Groen [2], and Rabbitt [7]. It describes the semicircular canals as a second-order overdamped system governed by
where is the endolymph volume displacement. The term represents the mass of the fluid contained in the canal, describes the viscous damping appearing in the duct, and defines the stiffness of the cupula which behaves as a restoring spring against the direction of fluid displacement. is an inertial forcing term [7]. In this model the membranous semicircular canal is represented as a rigid-walled structure, its stiffness being largely higher than the stiffness of the cupula. The cupula is also known as a quasi-incompressible gel with a Poisson ratio close to 0.48-0.49. Therefore, the cupula volume displacement can be approximated by the endolymph volume displacement.
However (1) is transformed to Laplace domain to obtain the transfer function defined in (2) between the cupula volume displacement and the angular head velocity :
As the system is highly overdamped, the two time constants are approximated by and . The values used in this model directly depend on the morphology of the canal and the physical properties of the fluid and the cupula (Table 1).
3.2. Otolith Organ
The otolith organs are dynamic as well as static sensors. Due to the density differences within their structure, any linear acceleration of the head due to linear translation or to a change of the orientation of the head with respect to gravity displaces the otoliths and generates a stimulus. By applying Newton’s second law of motion within the otoconial plane and then using Laplace transform, the transfer function between the otoconial layer displacement and the gravitoinertial acceleration is obtained:
with the mass of the otoconial layer, and are, respectively, the stiffness and the viscosity of the gel membrane supporting the otoconial layer. The system response is overdamped. The two time constants are approximated by and with and (Grant et al. [18, 19]).
4. Enhanced Formulation of the Kinematics Problem
The objective of this model is to simulate a real case of vestibular diagnosis carried out by a specialist. It takes into account the non-orthogonal coordinate system attached to the semicircular canals and the presence of the otolith organs. Figure 4(a) illustrates the procedure usually used. The patient is strapped into a rotary chair. Due to this rotational movement and the inertia of the endolymph, the cupula inside the canal perpendicular to the axis of rotation is deflected. A perception of rotation results (Figure 4(c). Furthermore, due to the distance of the inner ears from the axis of rotation, the components of normal and tangential acceleration stimulate the otolith organs. In addition, the patient can be made to undergo several head movements in order to stimulate others sensors.
In this first model the canals are assumed totally uncoupled, that is, there are no fluid flow interactions between them. That means a canal is stimulated if the component of the rotation vector along the perpendicular of this canal is nonnull. The different coordinate frames are defined in Figure 4(b).
The angular acceleration between the two frames are is defined by
The chair rotates at a constant velocity, which means is null. The differentiation in leads to the expression of the angular acceleration: Where is the angular acceleration of the head relative to the chair expressed in , and are, respectively, the angular velocities of the chair in and the angular velocity of the head in .
The general expression of head angular velocity relatively to the chair is . The movements of the head are separated into three nonsimultaneous distinct motions: case 1, is a rotation around the axis , case 2 is a rotation around and case 3 around. For each case, the acceleration of the point is calculated. Its general expression is
This expression describes linear acceleration in a rotating environment where the term is the Coriolis acceleration. (See Table 3 for the expression of the head angular velocity and the acceleration of the origin of the coordinate frame .)
5. Model Implementation
5.1. Simulink Model
The global functioning of the Simulink model taking into account the inputs (motion of the rotary chair head movements) and the sate of each sensor is explained in the schematic diagram (Figure 5).
The motions undertaken by an actual patient are integrated into the virtual modeling system. As soon as the simulation is started, equations of motion are solved in each coordinate system by using the various blocksets available in MATLAB/Simulink [20]. Finally, the state of each sensor in real time is derived according to their transfer function. The implementation of the simulation parameters is easily carried out by using the GUI, which is linked to the Simulink model (Figure 6).
These parameters define the rotation of the chair (constant, trapezoidal, or sinusoidal motion), the head movements of the subject, and exterior linear accelerations such as translation motion or gravity. For clarity purpose, the head movements are named (Figure 7):
For instance, the parameters of the model are the angular velocity of the chair, the time to reach this value, the magnitudes of the rotating movements of the head, six instants giving the starting signal of each head movement, the distance of the origin of from the axis of rotation, and so forth.
The model takes into account a coordinate frame attached to the canals. Indeed, it is important to determine the components of the angular acceleration vectors in this coordinate system as it permits to know whether each canal is stimulated or not. It is constructed by using experimental Euler angles which define the perpendicular of each canal plane (anterior, lateral, and posterior, resp.), that is, their orientation in a 3D space relatively to the head coordinate frame. These Euler angles were determined by several authors and more recently by Della Santina et al. [21]. They are summarized in Table 2.
These values clearly show that the canals do not define an orthogonal coordinate system. In physical sense that means if the head turn around a perpendicular axis of one canal, not only this canal but also the others will be stimulated, as well. So we can conclude that for any rotation of the head, all the angular sensors should provide a stimulus. However, it is important to note that each canal admits a specific direction of stimulation that maximizes the excitation: the lateral, anterior, and posterior canals primarily sense yaw, roll, and pitch, respectively. The component of any vector is defined in coordinate system with the transformation matrix:
5.2. Virtual Reality Model
A numerical demonstrating tool is developed to get a better understanding of the behavior of the inner ear for any kind of head motion. A potential application of this digital mockup could be the improvement of the existing clinical test or the development of new ones. A graphical user interface has been programmed in order to simplify the use of the model, the implementation of the simulation parameters, and the analysis of the results (curves plot, 3D animation, virtual reality). Furthermore a virtual reality world is linked to the kinematics and vestibular model using the virtual reality toolbox available in MATLAB [22]. This toolbox represents an interface between MATLAB and Simulink data on one hand, and virtual reality graphics one the other hand. Virtual reality graphics are based on VRML, an open standard for describing 3-D scenes [23, 24]. Virtual Reality Toolbox has been successfully used in multiple applications for visualizing results of Simulink simulations. However, it has been observed that system and control engineers who are unfamiliar with VRML find it difficult to create a VRML file describing a 3-D scene they would like to visualize. The solution to simplify the VRML file creation process is to start the design with CAD assemblies. In this modeling, the different parts of the virtual world are created using Solidworks (CAD software), which is very useful for specifying detailed three-dimensional design of a component [25]. The CAD models are then exported into Virtual Reality Modeling Language (VRML) files. The final virtual environment is created using the “V-Realm Builder” software where the VRML files are imported. In order to simulate the dynamics of the system, the CAD-to-SimMechanics translator from the MathWorks is used [26, 27]. It enables to translate CAD assemblies from a CAD platform into a Physical Modeling XML file compatible with SimMechanics. Then, a SimMechanics block diagram model is generated from this file to simulate the dynamics of the CAD assembly in the Simulink environment. In order to achieve this, Simulink and SimMechanics use a block diagram approach to model control systems around mechanical devices and simulate their dynamics. The block diagram approach does not include full geometric information, nor do CAD assemblies typically incorporate controllers or allow to perform dynamic simulations. Using this technique of CAD translation, the power of CAD and SimMechanics are combined.
Finally, this Simulink model is connected to the virtual scene in order to create a realistic high-quality animation. The outputs of the kinematics and vestibular model are linked to the inputs of the virtual reality toolbox in order to bring about progress in the virtual world (Figure 8).
6. Simulation and Visualization
6.1. Rotation Movement of the Chair
This experiment mimics the usual diagnosis procedure of the lateral semicircular canal. During this first experiment the patient sits down on the rotary chair. His head is kept fixed relatively to the device and tilted downward of to bring the lateral semircircular canal in the plane of rotation. Then, a constant angular velocity of is imposed to the chair. This motion starts at s and achieves its steady state in 1 s. This simulation lasts 40 seconds. The volume displacement of the cupula is shown on Figure 9. If the canals are considered to be orthogonal, the endolymph in the lateral canals lags behind—at the beginning of the rotation—due to its inertia. Consequently, the cupula of the lateral semicircular canal is deflected in the opposite direction of head movement (Figure 9(a)). This deflection causes a sensation of motion. The angular velocity of the chair being constant, the endolymph in the lateral canal tends to catch up with the rotation of the head eliminating the relative movement. Therefore, the cupula returns to a vertical position due to its elastic properties, and the sensation of motion ceases.
(a)
(b)
(c)
(d)
Figures 9(a) and 9(b) enable us to show the influence of the non-orthogonality of the canals. From these plots, a slight displacement of the anterior and posterior cupula is observed that does not appear in the case of an orthogonal system. However, the lateral canal is the most stimulated as its plane is quasi-perpendicular to the axis of rotation. The displacement of the lateral cupula generates a sensation of rotation, which lasts about thirty seconds at a constant angular velocity.
6.2. Rotation Movement of the Chair and then of the Head
The rotation movement of the chair is the same as above. In this case the subject does a downward and an upward head rotation at time and respectively. For the sake of simplicity, the amplitude of these movements is here equal to . This kind of head motion during a constant angular velocity of the chair involves the stimulation of the other canals. The displacements of the cupulas can be observed on Figures 9(c) and 9(d). Until being equal to 10 s, the movement of the cupulas is the same as the previous experiment. At time 10 s, the subject does a downward head rotation of from the previous head position. In the case of an orthogonal set of canals, this head motion brings the posterior canal into the plane of rotation. Therefore the cupula of the posterior canal is in turn deflected whereas the cupula of the lateral canal bends in the opposite direction as the fluid keeps moving relatively to the wall of the lateral canal. At time , the reverse phenomenon is produced as the subject makes an upward head rotation of the same magnitude.
It can be noticed that the succession of head movements, during a constant rotation of the body, creates erroneous motion sensations known as the Coriolis Effect in aeronautic terms. For example, at s the downward motion of the head engenders a positive displacement of the lateral cupula. This means that during a few seconds the subject has a sensation of rotation opposite to the rotation of the chair. This is due to the inertia of the fluid which is still in motion inside the canal.
6.3. VR as a Demonstration Tool
The aim of showing virtually the diagnosis test undertaken by the specialist is to allow a better comprehension of what happens inside the inner ear during a specific head movement. The state of each sensor is computed and visualized during the experimental protocol. Figure 10 illustrates the patient sitting on the rotary chair and undergoing a downward movement of his head. The displacement of the cupula of each canal can be observed.
Please note that a video of this simulation and the source code are available at the link: http://personnel.supaero.fr/morlier-joseph/Inner%20ear.html
7. Conclusion
In the vestibular diagnosis context, the model presented in this article simulates several rotation movements of the head which can be entirely defined by the user through the graphical user interface. It computes equations of motion in the coordinate system attached to the semicircular canals which underline the fact that all the canals are stimulated for any rotation. The virtual reality environment allows the user to observe what theoretically happens at the level of each sensor. This model also provides a better understanding of different kinds of erroneous motion sensations which may appear during combined rotation motions. This point might be of interest in the aeronautical field for the training of fighter or aerobatics pilots.
Improvements to the model presented in this article are already underway. In particular, a finite-element model of the cupulas and the utricle, which takes into account the fluid structural interaction, is in progress. The point of this modeling is to determine whether any coupling terms exist between the function of each canal, due to fluid flow.
At this moment, the major limitation of this model is that it does not entirely represent the diagnosis procedure of the vestibular system. Currently, the way to explore the vestibular component of the inner ear is to record the vestibulo-ocular-reflex (VOR) using different experiments including the rotary chair test. This reflex is directly linked to the stimulation of the vestibular sensors. By examining this ocular reflex relatively to the imposed rotational movements the specialists are able to detect any vestibular deficiencies. A potential future application of this model could be its use during diagnosis of the vestibular system in order to have a comparison between clinical results and those of the model. To achieve this goal, this first version has to be enhanced: by taking into account the link between the vestibular sensors and the vestibule-ocular-reflex, that is, the transfer functions between these two components (Zupan [28]), by incorporating eye movements in the virtual scene.
Finally, further works will focus on the development of a demonstration tool based on the technology of Augmented Reality (AR). The user’s view of the real world is enhanced with additional information generated from a computer model. By exploiting people’s visual and spatial skills, AR brings information into the user’s real world. That is, AR allows the user to stay in touch with the real environment. This is in contrast with virtual reality (VR) in which the user is completely immersed in an artificial world and cut off from the real world. Hence Augmented Reality is a promising tool for visualizing information from both dynamic models and experimentations (clinical tests, etc.) to develop an enhanced vestibular diagnosis tool.
Acknowledgments
The authors thank Dr. J-F. de Lauzun for providing valuable details about the anatomy and physiology of the inner ear, and F. Prieur for providing technical and dimensional data of the rotary chair used in the virtual world. The authors thank C. O’Keeffe for relevant advices and helpful comments on the manuscript. The author’s work was supported by F. Prieur, Integral Design.