Surface Optimization of a Three-Dimensional Eye Muscle - SEE-KID
Transcription
Surface Optimization of a Three-Dimensional Eye Muscle - SEE-KID
Fachhochschul-Diplomstudiengang SOFTWARE ENGINEERING FÜR MEDIZIN A-4232 Hagenberg, Austria Surface Optimization of a Three-Dimensional Eye Muscle Model Diplomarbeit zur Erlangung des akademischen Grades Diplom-Ingenieur (Fachhochschule) Eingereicht von Stefan Satzinger Betreuer: Begutachter: Dipl.-Ing. (FH) Thomas Kaltofen, UAR, Hagenberg FH-Prof. Dipl.-Ing. Dr. Herwig Mayr August 2003 Eidesstattliche Erklärung Ich erkläre, dass ich die Diplomarbeit selbstständig verfasst, andere als die angegebenen Quellen und Hilfsmittel nicht verwendet und mich auch sonst keiner unerlaubten Hilfe bedient habe. Hagenberg, August 2003 Stefan Satzinger Abstract Abstract This diploma thesis describes the development of optimization methods for a threedimensional extraocular eye muscle (EOM) model. All created methods were implemented in a software system called VISU, which is part of the SEE-KID research project that was initiated at the Upper Austria University of Applied Sciences in Hagenberg and is being continued at the UAR (Upper Austrian Research) company. The aim of this thesis is to improve a current three-dimensional EOM model generated from MR images with respect to the surface representation of the muscle for the purpose of simulating the movement of one EOM and to provide data about the dimensions of the eye muscles for use in an additional software system called SEE++, which is also part of the SEE-KID research project. In order to understand the anatomy of the human eye, a short description of the EOMs and the bulbus is given. Additionally, information about different models of the EOMs, including a comparison of the models with each other, is presented. The thesis describes the extension of the software system VISU by introducing a set of optimization methods customized for the properties of VISU. Thereby the main work for improvement of the three-dimensional EOM surface is accomplished by a spline interpolation method. After introducing and comparing different methods for spline interpolation, the NURBS interpolation method was used to improve the surface of the EOM model. The algorithms for the NURBS interpolation were included in VISU, which was extended to provide not only the three-dimensional EOM model as output but also data of the dimensions of an EOM. All the performed improvements were validated regarding their anatomical correctness and their compliance to the MR images which were the initial starting point for the generation of the EOM model. At the end of this thesis, an outlook to future improvements of VISU is given. iii Kurzfassung Kurzfassung Diese Diplomarbeit beschreibt die Entwicklung von Methoden für die Optimierung eines Modells der drei-dimensionalen extraokularen Augenmuskeln (EOM). Alle dabei entwickelten Methoden wurden in einem Softwaresystem mit dem Namen VISU implementiert, welches Teil des Forschungsprojektes SEE-KID ist. SEE-KID wurde an der Fachhochschule Hagenberg durchgeführt und wird von der Firma UAR (Upper Austrian Research) fortgesetzt. Es ist das Ziel dieser Diplomarbeit ein derzeitiges drei-dimensionales EOM Modell, welches aus MR Daten generiert wurde, hinsichtlich der Darstellung der Muskeloberfläche für die Aufgabe der Simulation eines EOMs zu optimieren. Zusätzlich sollen Dimensionsangaben der Augenmuskeln für den Gebrauch in einem weiteren Softwaresystem mit dem Namen SEE++, welches ebenfalls Teil des SEE-KID Forschungsprojektes ist, bereitgestellt werden. Um die Anatomie des menschlichen Auges zu verstehen, wird eine Beschreibung der EOMs sowie des Bulbus präsentiert. Zusätzlich wird Information über die unterschiedlichen Modelle der EOMs sowie ein Vergleich der Modelle untereinander präsentiert. Diese Diplomarbeit beschreibt die Erweiterung des Softwaresystems VISU durch die Einführung einer Reihe von Optimierungsmethoden, welche auf die Eigenschaften von VISU abgestimmt sind. Dabei wird die Hauptarbeit für die Optimierung der drei-dimensionalen EOM-Oberfläche durch eine Spline-Interpolationsmethode verrichtet. Nach dem Vergleich unterschiedlicher Methoden der Spline-Interpolation wurde die NURBS-Interpolationsmethode eingesetzt um die Oberfläche des EOMModells zu optimieren. Die Algorithmen für die NURBS-Interpolation wurden in VISU inkludiert, welches nicht nur erweitert wurde, um das drei-dimensionale EOMModell als Ausgabe zu liefern, sondern auch um Dimensionsangaben über die EOMs zu erhalten. Alle ausgeführten Verbesserungen wurden bezüglich ihrer Richtigkeit und Übereinstimmung mit den MR-Daten, welche den Ausgangspunkt für die Erzeugung des EOM-Modells darstellten, validiert. Am Schluss wird ein Ausblick auf zukünftige Verbesserungen von VISU gegeben. iv v Acknowledgements Acknowledgements I would like to put my “thank you” into words to all the people who made this work possible. First of all I would like to thank my girlfriend Gudrun for giving me the support I needed for finishing the four years at the Upper Austria University of Applied Sciences in Hagenberg. The next two people are my two colleagues at the UAR department for medical informatics, Michael Buchberger and Thomas Kaltofen. Michael helped me a lot with his advice, especially in times when I got stuck with my work he gave me the hints that pushed me back on the right way for finishing the thesis. Thomas, who was the supervisor for my thesis work at the UAR, always gave me best advice for improvements and corrections during my work. Of course I would also like to thank the other three colleagues at the UAR, especially Doris Siegl, for their help and assistance during the time at the company. A special “thank you” also goes to Gregory Curtis, my english teacher at the university, who was responsible for improving my english knowledge. A special thanks also goes to Herwig Mayr who took the task of correcting and grading my thesis and who gave me the chance to continue the work on the VISU project. Last but not least I would like to thank my family for the help they gave me during my studies in Hagenberg and all my friends from the department for Medical Software Engineering as well as all my other friends for whose names to mention there isn’t enough place on this page. Stefan Satzinger Contents 1 Optimizing Eye Muscle Models 1.1 Motivation . . . . . . . . . . 1.2 The SEE-KID Project . . . . 1.2.1 Structure . . . . . . . 1.2.2 Current Status . . . . 1.3 The VISU Project . . . . . . 1.3.1 Structure of VISU . . 1.3.2 Current Status . . . . 1.4 Deficits of Current Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Basic Notions 2.1 The DICOM Standard . . . . . . . . 2.2 Magnetic Resonance Imaging . . . . 2.2.1 History . . . . . . . . . . . . . 2.2.2 Magnets . . . . . . . . . . . . 2.2.3 Magnetic Influence on Atoms 2.2.4 MR Images . . . . . . . . . . 2.3 Mathematics . . . . . . . . . . . . . . 2.3.1 Analysis of Variance . . . . . 2.3.2 Algorithms for Interpolation 2.4 Medical Terms . . . . . . . . . . . . 2.4.1 Anatomy of the Human Eye 2.4.2 Rectus Muscle Pulleys . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Muscle Model Generation 3.1 Existing Approaches . . . . . . . . . . . 3.2 Limitations of Existing Approaches . . . 3.3 Possible Solutions for Optimization . . . 3.4 Generation of Muscle Data for Kinematic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1 2 3 3 4 5 5 6 . . . . . . . . . . . . 9 9 11 11 12 12 13 13 14 14 17 17 19 . . . . 20 20 21 23 27 4 Design of a Muscle Visualization System 29 4.1 Existing System Design . . . . . . . . . . . . . . . . . . . . . . . . . 29 vi vii Contents 4.2 4.3 Extension of the Existing Structure . 4.2.1 Picture Processing . . . . . . 4.2.2 Gravity Line Optimization . 4.2.3 Analysis of Variance . . . . . 4.2.4 Surface Optimization . . . . Modifications of the VISU Structure . . . . . . 5 Implementation 5.1 Muscle Gravity Line Optimization . . 5.2 Applying Analysis of Variance . . . . 5.3 Interpolation Using Splines . . . . . . 5.4 Muscle Data Generation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 32 33 35 36 37 . . . . 40 40 41 42 47 6 Validation 50 6.1 Comparing Reconstructed Volumes with MR Images . . . . . . . . . 50 6.2 Comparison with Existing Models . . . . . . . . . . . . . . . . . . . 52 6.3 Extraction of Muscle Dimension Data . . . . . . . . . . . . . . . . . 56 7 Software Development Aspects 61 7.1 Process View . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 7.2 Product View . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 8 Conclusion 63 8.1 Goals Achieved . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 8.2 Possible Future Improvements . . . . . . . . . . . . . . . . . . . . . 65 8.3 Open Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 Literature 66 List of Figures 1.1 1.2 1.3 1.4 1.5 1.6 SEE++ screenshot [Buchberger and Kaltofen, 2002] VISU structure . . . . . . . . . . . . . . . . . . . . EOM model [Miller, 1989] . . . . . . . . . . . . . . EOM model [Miller and Shamaeva, 1993] . . . . . . EOM model [Miller et al., 2003] . . . . . . . . . . . EOM model [Lacher, 2001] . . . . . . . . . . . . . . 2.1 2.2 2.3 2.4 2.5 2.6 2.7 2.8 DICOM scope [ACR-NEMA, 1988] . . DICOM file format . . . . . . . . . . . RF pulse . . . . . . . . . . . . . . . . . Flipped RF pulse . . . . . . . . . . . . Body planes [Haslwanter, 2003] . . . . Bézier curves [Ellman, 2002] . . . . . . Right eye (top view) [Sobotta, 1997] . Differences between conventional model [Miller and Demer, 1999] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 6 7 7 7 8 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . and pulley model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 11 12 12 13 16 17 . . . . . . . 19 3.1 3.2 3.3 3.4 3.5 3.6 3.7 Blausen EOM model [Blausen, 2003] . . . . . . . . . . . EOM model with errors on the surface . . . . . . . . . . Set of points to be interpolated . . . . . . . . . . . . . . Third degree interpolated curve . . . . . . . . . . . . . . Control polygon of the points . . . . . . . . . . . . . . . Bézier spline interpolation applied to the points of Figure NURBS interpolation applied to the points of Figure 3.3 4.1 4.2 4.3 4.4 Components of VISU . . . . . . . . . . . . . . . . . . . . . . . . . . Class diagram of the administration component . . . . . . . . . . . Data flow of the creation of a DXF model in VISU . . . . . . . . . Data flow of the creation of the EOM model in VISU out of the DXF file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Class diagram of the MR processing component [Lacher, 2001] . . . DXF muscle model with correct length calculation . . . . . . . . . . Partition of variance and amount of slices . . . . . . . . . . . . . . 4.5 4.6 4.7 viii . . . . . . . . . . 3.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 24 26 26 26 27 27 . 30 . 30 . 31 . . . . 31 32 34 36 List of Figures ix 4.8 4.9 Cross-section of DXF model with both halves interpolated . . . . . . 37 Expanded class diagram of the VISU structure . . . . . . . . . . . . . 39 5.1 Area centroid of a cross-section and the angle a that the point spans regarding the area centroid . . . . . . . . . . . . . . . . . . . . . . . NURBS curve with two equal control points P2 = P3 . . . . . . . . Linear interpolation of the NURBS-generated cross-section . . . . . SEE++ screenshot showing the EOM built of muscle data from initialization files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 6.1 6.2 6.3 6.4 6.5 6.6 6.7 6.8 6.9 6.10 6.11 6.12 6.13 6.14 6.15 6.16 6.17 6.18 6.19 6.20 6.21 6.22 6.23 6.24 6.25 EOM model without overlapping MR images . . . . . . . . . . EOM model with overlapping MR images . . . . . . . . . . . . DXF EOM model with overlapped MR images . . . . . . . . . Dynamic EOM model with overlapped MR images . . . . . . . DXF EOM model with overlapping MR images, magnified . . Dynamic EOM model with overlapping MR images, magnified DXF EOM model with overlapping MR images . . . . . . . . Dynamic EOM model with overlapping MR images . . . . . . DXF EOM model with overlapping MR images, magnified . . Dynamic EOM model with overlapping MR images, magnified DXF EOM model with overlapping MR images . . . . . . . . Dynamic EOM model with overlapping MR images . . . . . . DXF EOM model with overlapping MR images, magnified . . Dynamic EOM model with overlapping MR images, magnified Dynamic EOM model, non-optimized version . . . . . . . . . . Dynamic EOM model, optimized version . . . . . . . . . . . . EOM model with an innervation of 0 . . . . . . . . . . . . . . Optimized EOM model with an innervation of 0 . . . . . . . . EOM model with an innervation of 50 . . . . . . . . . . . . . Optimized EOM model with an innervation of 50 . . . . . . . EOM model with an innervation of 100 . . . . . . . . . . . . . Optimized EOM model with an innervation of 100 . . . . . . . Dynamic EOM model, non-optimized version . . . . . . . . . . Dynamic EOM model, optimized version . . . . . . . . . . . . Screenshot of the selected muscle in the MR images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.1 Screenshot of the rec. med. muscle and the corresponding MR images in VISU . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 5.2 5.3 5.4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 . 43 . 47 51 51 53 53 53 53 54 54 54 54 55 55 55 55 56 56 57 57 57 57 57 57 58 58 59 List of Tables 2.1 Primary, secondary, and tertiary functions of the six EOMs [Thömke, 2001] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 3.1 Overview on current EOM models . . . . . . . . . . . . . . . . . . . . 24 6.1 Comparison of muscle dimension data in millimeters, measured form the MR images and extracted from VISU . . . . . . . . . . . . . . . . 60 x Chapter 1 Optimizing Eye Muscle Models In modern medicine, methods for visualizing different parts of the human body have become state of the art. Technologies like magnetic resonance (MR) imaging or computed tomography (CT) build the basis for a deeper insight into the human body and offer the chance for getting a whole new understanding of how humans work. Thus, especially in the field of image processing, a lot of research is done to get the most out of imaging data. Today not only the processing of images taken from MR imaging or CT devices is carried out but also the three-dimensional anatomically correct reconstruction of certain parts of the human body. However, as new technologies find their way into the field of medicine, new methods for visualization also have to be discovered and developed to a status where doctors can use them in order to take better advantage of the results. 1.1 Motivation The movement and the spatial location of the six extraocular eye muscles (extraocular eye muscle = EOM) is of great importance to physicians, especially ophthalmologists who need information on certain muscles in order to plan surgeries or to diagnose defects of the oculomuscular apparatus. Thus, a system, preferably a software system, would be a good solution to plan surgeries and to simulate the movement of the different EOMs. Although the simulation of the movement of the EOMs is a rather new research area, some models already exist, differing in complexity, functionality and accuracy. Unfortunately none of these models is perfect. The motivation for this thesis work is to improve a current three-dimensional EOM model generated from MR images with respect to the surface representation of the muscle for the purpose of simulating the movement of one EOM. To manage this, different mathematical methods are applied in an underlying model of an EOM which has been realized in a software system called VISU (VIrtual SUgery). In addition, all of the new insights have to be added to this software system. The result should be a dynamic three-dimensional EOM model with improved muscle surface, as well as information about the generated 1 Chapter 1. Optimizing Eye Muscle Models muscle for the usage in another software system called SEE++ (Simulation Expert for Eyes + Diagnosis + Transposition Surgery). The VISU as well as the SEE++ software are the results of the research project—carried out at the Department of Medical Software Engineering at the Upper Austria University of Applied Sciences in Hagenberg—SEE-KID (Software Engineering Environment for Knowledge-based Interactive Eye Motility Diagnostics) [Buchberger and Mayr, 2000]. The research project SEE-KID as well as the two software systems VISU and SEE++ are continued by the UAR (Upper Austrian Research) company which is a partner of the university. The intended audience for this thesis consists of physicians and surgeons who are interested in a software system for simulating EOM movement as well as technicians who want to learn more about the technical aspects behind the system concerning implementation details or the algorithms used. 1.2 The SEE-KID Project Defects of the EOMs can develop into a pathological problem if they are not recognized and corrected already at an early state of human growth. This is already carried out at hospitals where the eye departments have specialized in the correction of certain pathologies, especially strabismus. Thus, the defects can be corrected by changing the position of at least one or several EOMs through surgery. The problem with these methods is that there is no possibility to simulate the surgery. As a result, the success of such surgery depends on the experience of the surgeon as well as on already documented surgeries originating from similar pathologies. Thus, a satisfying result might only be achieved by carrying out numerous surgeries, resulting in a higher burden on the patient. Enhancements were provided by the SEE++ software system which is one of the results of the SEE-KID project. The goal of the project is to build a biomechanical, interactive 3D simulation of the human eye, its globe and its muscles, to simulate common EOM surgeries (transposition, resection, splitting, etc.) in a graphical interactive way that is familiar to an experienced surgeon. Additionally, the system can be used for educational and training purposes, because through extensive possibilities for parameterizing the biomechanical model, common pathological cases of human eye motility disorders can be modeled and subsequently simulated [Buchberger et al., 2002]. As mentioned before, the main goal of the research project SEE-KID is the realization of a biomechanical model of the human eye. A mainly mathematical description of the oculomuscular apparatus forms the basis for a system which should give proper predictions on simulation and treatment of pathological cases [Buchberger and Mayr, 2000]. 2 Chapter 1. Optimizing Eye Muscle Models 1.2.1 Structure The core of the SEE++ software system is the biomechanical model which consists of three different parts [Kaltofen, 2002]: • geometrical model, • muscle force model, • kinematic model. The model characterizes the movement of the eye and its EOMs, with respect to the biological structures, according to the mechanical behavior of the eye and its muscles. Geometric Model The biomechanical model in SEE++ supports different geometrical models, whereby three of them are currently implemented, namely the string model [Krewson, 1950], the tape model [Robinson, 1975] and the pulley model [Demer et al., 1997]. The geometrical model describes the geometry of the human eye with its EOMs. Muscle Force Model The muscle force model is used to describe the muscle forces and the influence that these forces have on other parts of the overall model. Kinematic Model The kinematic model is responsable for combining the results of the geometrical model and the muscle force model. Of course there are also other parts of the SEE++ software which are important for the functionality. This is for example the graphical user interface (GUI), which is responsible for the visualization of the results as well as for the interaction with the user. 1.2.2 Current Status The user can navigate through the program in a tree view where he has quick access to the most important features of the program, hence he can manage different scenarios. Interactive surgeries with the EOMs can be simulated, e.g. by repositioning the EOMs on the globe which is displayed in three-dimensional view together with the six EOMs. Another feature are the different diagrams supported by SEE++: • the muscle force distribution diagram, which represents the current gaze position through a vertical line, • the muscle force vector diagram, which represents the current gaze position through a small cross, 3 Chapter 1. Optimizing Eye Muscle Models • the Hess diagram, which also represents the current gaze position through a small cross. It depends on gaze patterns, which can be changed in the gaze pattern dialog. Figure 1.1 shows a screenshot of the SEE++ software system. Figure 1.1: SEE++ screenshot [Buchberger and Kaltofen, 2002] 1.3 The VISU Project The VISU project had its origin as a student project at the Upper Austria University of Applied Sciences in Hagenberg in the year 2001. The goal was to provide the physician with a software system that could generate a three-dimensional eye muscle model out of a series of MR images taken from a patient, as well as to store information about the patient, who was the subject of an examination. With the VISU software system, the physician, especially the ophthalmologist, is able to analyze changes of lengths and volume shifts of an EOM. 4 Chapter 1. Optimizing Eye Muscle Models 1.3.1 Structure of VISU The system’s main parts consist of the following: • GUI, • patient data administration component, • MR analysis component, • MR processing component. The MR analysis and the MR processing component are the two most important parts of VISU, because they are responsible for the generation of the threedimensional muscle model. GUI The GUI is the part of the system which is responsible for the interaction with the user as well as for displaying the results. Administration Component All relevant information about the patient like name, address, date of birth etc., is managed in the administration component of the program. MR Analysis Component The MR analysis component uses coronal slices of MR images of a patient as input data and builds a three-dimensional static model of one EOM [Pirklbauer, 2001] using the marching cubes algorithm to reconstruct a muscle. The muscle is then stored in the DXF format (Drawing eXchange Format) [Degen, 2001] for further processing. MR Processing Component The MR processing component calculates a three-dimensional dynamic eye muscle model out of two DXF models coming from the previous component. It uses DXF models generated from data of one muscle in two different gaze positions: one where the muscle is in a primary position (cf. Chapter 2.4.1) and one where the muscle is in a secondary position. The result is a three-dimensional model which can be transformed interactively from one innervation to another [Buchberger and Kaltofen, 2003]. 1.3.2 Current Status The main input for the VISU software system consists of MR images. Thus, the muscles, from which the three-dimensional model is generated, have to be visible in the images. The better the visibility of the muscles is, the better the resulting muscle model will be. The user can store different picture series containing MR data of one patient. The following model generation is split into two steps, the 5 Chapter 1. Optimizing Eye Muscle Models generation of two or more static muscle models and the generation of a dynamic muscle model. All necessary information is stored in a data file so that different views on already generated models can be visualized. Figure 1.2 shows the procedure for generating a three-dimensional muscle model, starting with the MR images of a patient’s eye followed by the generation of a static muscle model using the marching cubes algorithm and resulting in a three-dimensional dynamic eye muscle model. Figure 1.2: VISU structure 1.4 Deficits of Current Models Since the creation of one of the first computer models for the simulation of eye movements (SQUINT, [Miller and Robinson, 1984]), three-dimensional visualization of the EOMs has always been a main feature in every software tool handling the topic of simulation of eye movements. One of the first EOM models was generated by Joel M. Miller (cf. Figure 1.3) in 1989. For this model MR images of some patients were used and for each set of image data the mean muscle position was calculated. Out of this information a three-dimensional EOM model was reconstructed. Later, a three-dimensional EOM model was included in the software tool Orbit [Miller and Shamaeva, 1993]. However, this model was not a very realistic one; instead, it showed EOMs in a three-dimensional but universal way, though it was possible to change the shape of the model (cf. Figure 1.4). Another EOM model was also developed by Joel M. Miller [Miller et al., 2003]; it focuses on the visualization of the tissue surrounding the EOMs, called EOM pulleys (cf. Chapter 2.4.2). In the accompanying study, MR data coming from patients and 6 Chapter 1. Optimizing Eye Muscle Models 7 Figure 1.3: EOM model [Miller, 1989] serially sectioned human cadavers was combined to calculate the muscle volumes (cf. Figure 1.5). All three models developed by Joel M. Miller have the disadvantage of not being modifiable in an anatomically correct way. This deficit was improved by [Lacher, 2001], who managed to create a modifiable three-dimensional EOM model using the data that resulted out of [Pirklbauer, 2001]. The model was fully dynamic and anatomically correct; it showed for the fist time a human EOM that could be watched in 3D and that could change its innervation within a certain range; however, only one of the EOM could be watched at the same time (cf. Figure 1.6). Figure 1.4: EOM model [Miller and Shamaeva, 1993] Figure 1.5: EOM model [Miller et al., 2003] Of course there exist other EOM models like the one created by [Simonsz et al., 2002]; it is based on the finite element method, or models made from various artists. The voxel-man project (cf. [Hoehne et al., 2003]) is also a good example. But most of these models have the disadvantage of not being anatomically correct, which means Chapter 1. Optimizing Eye Muscle Models Figure 1.6: EOM model [Lacher, 2001] that they are not able to reflect the EOM anatomy of one specific person. It is also not possible to change the EOMs interactively, which means to look at an EOM at different states of innervation. A closer look at the comparison of the different models will be given in Chapter 3. 8 Chapter 2 Basic Notions This chapter provides a short overview of the basic notions required. First the DICOM standard will be described, since this is one of the most frequently used protocols for data exchange in medicine. Chapter 2.2 provides the reader with fundamental knowledge on how magnetic resonance imaging works and what it is used for. In Chapter 2.3 the mathematics used in this thesis work is presented in a way that provides information on the different mathematical terms and methods and how they are applied. Finally the last section in Chapter 2 covers different medical terms regarding the human eye. 2.1 The DICOM Standard After the introduction of computed tomography (CT) in 1973, several other diagnostic imaging devices followed very quickly. This was the reason for the American College of Radiology (ACR) and the National Electrical Manufacturers Association (NEMA) to establish a standard for transferring images and associated information between devices manufactured by manifold vendors. The joint work of the ACR and the NEMA resulted in the DICOM standard (Digital Imaging and Communications in Medicine) [ACR-NEMA, 1988]. The DICOM standard ensures compatibility of medical imaging equipment by specifying: • a set of protocols to be followed by devices claiming conformance to the standard, • syntax and semantics of commands and associated information which can be exchanged using these protocols, • information that must be supplied with an implementation for which conformance to the standard is claimed. Figure 2.1 shows the scope of the DICOM format. 9 Chapter 2. Basic Notions 10 Medical Informatics . . . Patient Bedside Monitoring Administrative HIS/RIS Lab Data . . . Diagnostic Imaging Scope of DICOM Figure 2.1: DICOM scope [ACR-NEMA, 1988] However, there are lots of ways to extend the standard, which makes it difficult for DICOM software producers to implement DICOM viewers or readers that have the capability to read and display data from different DICOM files obtained from various imaging equipment. Structure of the DICOM File Format A DICOM file includes a data field as well as a header, where information about the patient, like the patient’s name, date of birth or the date when the study was conducted, is stored. The image information is stored in the data field which contains information in three dimensions. In the sample DICOM file shown in Figure 2.2, the header at the beginning shows information about the patient as well as information involving the modality which was used to generate the DICOM file. MR modalities, for example, store different technical information about the analysis on a patient than CT modalities do. The size of the header varies depending on how much information is stored in it. Chapter 2. Basic Notions 11 Figure 2.2: DICOM file format 2.2 Magnetic Resonance Imaging Magnetic resonance (MR) imaging is an imaging technique used primarily in medical fields to produce images of the inside of the human body. MR imaging is based on the magnetic properties of hydrogen protons which align themselves in a magnetic field; the alignment is greatly influenced by the strength of the external magnetic field and the thermal energy of the atoms. 2.2.1 History The magnetic resonance phenomenon of hydrogen protons was discovered by Felix Bloch and Edward Purcell independently in the year 1946. In 1952 both received the Nobel Prize for their discovery. Unfortunately, the usage in the medical field was not yet possible at that time. Then, in 1971, Raymond Damadian showed that the relaxation time of tissue and tumors differed. This was the breakthrough that allowed scientists to do research in the medical area in order to use it for diagnostic applications (cf. [Damadian, 1971]). Chapter 2. Basic Notions 2.2.2 12 Magnets Magnetism is an important part of MR imaging and requires field strengths from 0.4 up to 2.0 Tesla . Today’s devices, in most cases, use super conductive magnets; these consist of coils of wire which carry stable electric voltage. By cooling the coils with cryogenics (helium for example) it is possible to generate very strong magnetic fields because a higher amount of electric current can flow through the wire. Although super conductive magnets are currently the norm, there are two other possibilities, namely permanent magnets and resistive magnets, whereby both aren’t used very often because they only produce small magnetic fields. For example permanent magnets operate only up to 0.4 Tesla. 2.2.3 Magnetic Influence on Atoms Figure 2.3: RF pulse Figure 2.4: Flipped RF pulse Atoms consist of positively charged protons and neutrally charged neutrons, both form the so-called nucleus. Around the nucleus there is a cloud of negatively charged electrons which keep the atom in balance. Since more than 70% of the human body consists of water (H2 O), hydrogen is the fundamental element in MR imaging and thus all of the following considerations will focus on the hydrogen atom. Images of the inside of the human body are generated by exposing it to an external magnetic field B0 . This causes all the protons inside our body, which have their own magnetic field M0 , to align themselves in the direction of B0 (cf. Figure 2.3 [King, 2000]). By applying energy in the form of a radio frequency (RF) signal at 90◦ to the direction of B0 the direction of the magnetic field of the protons can be changed, with the result that all the protons flip their angle to the new layer xy (Figures 2.3 and 2.4 [King, 2000]). Switching off the RF signal causes the protons to realign with B0 . During this cycle signals can be measured by receiver coils inside the MR imaging device. An exact type of tissue can be assigned to each of the signals because signals coming from diverse tissues inside the human body have different properties called T1 and T2 relaxation times. Fat, for example, has a very efficient Chapter 2. Basic Notions energy exchange and therefore has a relatively short T2. Water is less efficient than fat in the exchange of energy, and therefore has a long T2. All these calculations are carried out by the MR imaging device and are transformed into images that can be processed further by other devices or that can just be viewed on a screen. 2.2.4 MR Images Each image is the result of data which was produced from signals that were the measured effect of an external magnetic field applied to the human body by the MR imaging device. One image does not just show a two-dimensional slice of the body; instead, it shows a whole volume in three dimensions, because the above-mentioned signals always emerge out of a slice which has a certain thickness. Images of the body can be taken in three different directions (cf. Figure 2.5) : Figure 2.5: Body planes [Haslwanter, 2003] • horizontal: all planes that develop parallel to the horizontal plane, • coronal: all layers that develop parallel to the plane going through the forehead, • sagittal: all planes that develop parallel to the nose to head plane. To get more information about MR imaging, refer to [Dössel, 2000] and [Morneburg, 1995]. 2.3 Mathematics While many mathematical techniques were used during the development of this thesis, only two of them leave their traces throughout the whole thesis work, namely the analysis of variance and algorithms for interpolation. Both of them are discussed in the following two chapters. 13 Chapter 2. Basic Notions 2.3.1 14 Analysis of Variance The basic principle is the comparison of the variation between groups of data with the variation in the groups. Data groups consist of measured values which have to be put into relation. An approach for the calculation is to determine the arithmetic mean of each group of values X0 , X1 , X2 , X3 , . . . , Xn and use it as the so-called target value Y . If the variation between the groups is bigger than the variation in the groups, then differences between the values (in the groups) can be supposed. First of all the variation s2 has to be calculated [Andress, 2001]: n P s2 = (xi − x̄)2 i=1 = n−1 SAQx , n−1 (2.1) where n = number of values per group, i = count index for the values (i = 1, . . . , n), xi = value of the variable X at the index i, x̄ = arithmetic mean of X, SAQx = sum of the squared deviation of X. 1 controls the value SAQx , because the more values Xi there are in one group n−1 of data, the higher the value for the variation gets. The result can be described as the “average squared deviation”. The standard deviation s s= √ s2 (2.2) can be calculated in order to interpret quantified spreading of a value in the groups. A more descriptive value is the variation coefficient v v= s ∗ 100, x̄ (2.3) because it brings the standard deviation in connection with the arithmetic mean, hence comparison of the groups is easier. 2.3.2 Algorithms for Interpolation We sometimes know the value of a function f (x) at a set of points x1 , x2 , x3 , . . . , xN where x1 < x2 . . . xN , but we don’t have the analytic expression for f (x) to calculate its value at an arbitrary point. The challenge is now to somehow approximate f (x) for arbitrary points by drawing a smooth curve through xi . If the desired x is between the largest and the smallest xi , the problem is called interpolation, if it is outside, it’s called extrapolation. In the following, three common methods for interpolation, which are also suitable for extrapolation, will be presented. Chapter 2. Basic Notions 15 Polynomial Interpolation This interpolation method uses a polynomial [Press et al., 2002], p(x) = a0 + a1 x + a2 x2 + · · · + an xn (2.4) to approximate f (x) for arbitrary x. The only problem is how to find the coefficients (base points) a0 , a1 , a2 , . . . , an that determine the behavior of the polynomial. One approach is to solve the linear equation system. But this leads to a very complex structure of equations if there is a high number of base points. A better way would be to use the interpolation polynomial according to Newton. pn (x) = k0 +k1 (x−x0 )+k2 (x−x0 )(x−x1 )+· · ·+kn (x−x0 )(x−x1 ) . . . (x−xn−1 ) (2.5) When doing so, the polynomial is built up piecemeal, thus offering more flexibility so that new base points can be added without changing the equation system. An algorithm for the polynomial interpolation can be looked up e.g., in [Press et al., 2002] Bézier Splines Using interpolation polynomials has the disadvantage of having very big oscillations between the base points, with the effect that further processing is harder to accomplish. These problems can be prevented using spline interpolation, where a function C(u) is built between the base points x1 , x2 , x3 , . . . , xN , the so-called spline knots, which only has minimal oscillations between these points. C(u) = n X Bi,n (u)Pi (2.6) i=0 Equation 2.6 characterizes the Bézier splines, where Bi,n (u) describes the blending functions and Pi are the given control points. The blending functions are the basis functions of the Bézier curve (cf. Figure 2.6 which shows the blending functions for cubic Bézier curves1 ). Bi,n (u) = n! ui (1 − u)n−i i!(n − i)! (2.7) The algorithm for the calculation of the desired Bézier curve is split into two parts. The first part calculates the blending functions and the second one calculates an arbitrary point on the resulting Bézier curve. For the complete algorithm, refer to [Hearn and Baker, 1997]. 1 The amount of control points determines the degree of a curve. Chapter 2. Basic Notions 16 1.0 0.9 B0,3(u) B3,3(u) 0.8 0.7 0.6 0.5 B1,3(u) B2,3(u) 0.4 0.3 0.2 0.1 u 0.0 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 Figure 2.6: Bézier curves [Ellman, 2002] NURBS NURBS stands for Non Uniform Rational B-Splines. This spline sort is probably the most generic form of splines because many other kinds of splines are just a special case of NURBS (e.g. Bézier splines). n P C(u) = Ni,p (u)wi Pi i=0 n P , a≤u≤b (2.8) Ni,p (u)wi i=0 Equation 2.8 describes a p-degree NURBS curve, [Piegl and Tiller, 1980]. {Pi } are the control points, {wi } the weights which influence the control points and {Ni,p (u)} are the p-degree B-spline basis functions defined on the nonperiodic (and nonuniform) knot vector U : U = {a, . . . , a, up+1 , . . . , um−p−1 , b, . . . , b}. | {z } | {z } p+1 (2.9) p+1 B-splines are another type of spline that are defined by a nonperiodic knot vector which determines the points where the curve must go through. For the calculation of the NURBS curve there are three algorithms necessary, one to check which part of the curve is determined by which knot points, one to calculate the blending functions and another one to calculate an arbitrary point on the resulting curve. All algorithms are fully described in [Piegl and Tiller, 1980]. Chapter 2. Basic Notions 2.4 Medical Terms The eyes reflect the visual part of our cognition. An eye consists of the bulbus oculi (globe), three inner and six outer eye muscles, all surrounded by the orbita, a dent in the cranial bones. The EOMs are responsible for moving the eye consciously in different directions. As this thesis concentrates on the EOMs, it is vital to have a basic understanding of the anatomy of the human eye, especially the EOMs. 2.4.1 Anatomy of the Human Eye Figure 2.7: Right eye (top view) [Sobotta, 1997] The six EOMs (cf. Figure 2.7) are of extreme interest, because MR images taken of them are one main starting point for this thesis work. These muscles are: • • • • • • medialis rectus musculus (med. rect. m.), lateralis rectus musculus (lat. rect. m.), inferior rectus musculus (inf. rect. m.), superior rectus musculus (sup. rect. m.), inferior obliquus musculus (inf. obl. m.), superior obliquus musculus (sup. obl. m.). The inf. obl. m. and the sup. obl. m. are the two oblique muscles and the others are the straight muscles. All straight eye muscles have their point of origin at the anulus tendineus communis, a small ring of sinews at the end of the orbita. From there the 17 Chapter 2. Basic Notions 18 muscles emerge first with a sinewy part followed by the muscle itself and then join the globe again with a sinewy part. The sup. obl. m. also has the anulus tendineus communis as its origin and is turned around at the trochlea before attaching to the globe. The inf. obl. m. originates at the front part of the orbita, near the nasal bone, passing across the m. rec. inferior. The purpose of each eye muscle is described in Table 2.1, where adduction and abduction describe a rotation toward the nose and away from the nose around the z-axis (which develops in dorsal direction from the center of the globe), respectively. Elevation and depression specify an upward and downward rotation of the eye around the x-axis (which develops in lateral direction from the center of the globe). Intorsion and extorsion define a rotation around the y-axis (which develops in rostal direction from the center of the globe) from the top of the bulbus toward the nose and away from it. Muscle med. rect. m. lat. rect. m. inf. rect. m. sup. rect. m. inf. obl. m. sup. obl. m. Primary Function adduction abduction depression elevation extorsion intorsion Secondary Function Tertiary Function extorsion intorsion elevation depression adduction adduction abduction abduction Table 2.1: Primary, secondary, and tertiary functions of the six EOMs [Thömke, 2001] Right in the middle of the anulus tendineus communis the nervous opticus passes through, with its origin in the bulbus oculi. The bulbus oculi itself consists of three skin layers: the tunica fibrosa, the sclera and the cornea. All six extra ocular eye muscles attach to the bulbus oculi, each at a different point called insertion point, which is the point where the muscle attaches to the globe. The pathway describes a muscle on its way from origin to insertion point is called muscle path, whereas the point where the muscle touches the bulbus oculi for the first time is called point of tangency. A point in three-dimensional space is fixed using all six eye muscles, whereby abduction and adduction are always executed by complementary muscles. Additionally, eyes can only be moved binocularly, which means that the movement of only one eye is not possible. According to [Kaufmann, 1995], three eye positions are feasible: • Primary position is the position where the eye is looking straight forward without fixing on any objects in space. The body and the head also have to be held straight. • Secondary positions are reached when, starting from the primary position, the eye rotates around exactly one axis, namely the x-axis or the z-axis. • Tertiary positions are accomplished by a consecutive rotation around the x-axis and the z-axis in any desired order. The starting point for these rotations is also the primary position. Chapter 2. Basic Notions 2.4.2 Rectus Muscle Pulleys Rectus muscle pulleys influence the muscle path of the four straight eye muscles. They consist of tissue sleeves composed of collagen and elastin stiffened by smooth muscle, enveloping the straight extra ocular eye muscles [Demer et al., 1997]. These tissue sleeves are firmly anchored to each other and to the walls of the orbita. Rectus muscle pulleys are a relatively new discovery [Miller and Demer, 1995]. Their appliance on the current bio-mechanical models of the eye lead to a dramatic change in the definition of the axis of rotation of a muscle. Several muscle models are available, all of them trying to describe the muscle movement, assuming that the eye muscles can move freely between the point of origin and the insertion point. The consequence of this is that the muscle path and thus the line of action is changed. To avoid this, the rectus muscle pulleys enclose the straight eye muscles in order to keep the primary direction. With this knowledge the muscle model can be extended so that the axis of rotation is not just determined by the point of origin and the point of tangency, but also the muscle’s effective pulley location. Figure 2.8 illustrates the differences between the conventional model and the pulley model in primary position and in up-gaze in a graphical way. Figure 2.8: Differences between conventional model and pulley model [Miller and Demer, 1999] 19 Chapter 3 Muscle Model Generation This chapter is devoted to the different three-dimensional EOM models. It outlines existing models and provides new ideas for improvement. Models will be compared based on their anatomical correctness, whether software programs can process them, and whether they can simulate muscle movement under real-life conditions. Of course the relation to the patients will also play a major role, because it makes a big difference, if an EOM model was generated out of patient data coming from one specific patient or if it uses data from many people where the mean values of all EOMs were calculated and then included into the model. 3.1 Existing Approaches Several of the currently available models were developed by Joel M. Miller. In [Miller, 1989] the creation of one of the first three-dimensional EOM reconstructions is outlined. Based on coronal slices of MR imaging data from four subjects, all of them with normal uncorrected vision; and ocular motility, a radiologist identified the orbital wall, globe and rectus muscles in the MR images using a pencil to mark the contours. The edited pictures were then digitized using a digitizing pad. Subsequently, every data set was transferred to the corresponding eye in order to combine the data of each subject into one result set. Finally, the normalized points were averaged by calculating the area centroid of any slice and subtracting it from its points. Through averaging the X and Y values of each set of four corresponding contour points and adding the mean area centroid an array describing the surface of four rectus muscles, an “average” eye was obtained. The data was then rendered using standard rendering software and the resulting pictures were redrawn by a medical illustrator (cf. Figure 1.3). The second model built by Miller (cf. [Miller et al., 2003]) also used MR images, but this time not from living human subjects but from human cadavers. Miller used a custom software tool called “TSR”, which was developed for the purpose of combining different slice data (e.g. image structures coming from MR scans and digitized images) to create another model. The result was an object representing a 20 Chapter 3. Muscle Model Generation three-dimensional EOM model including globe and optical nerve (cf. Chapter 1.4, Figure 1.5) as well as certain other tissue structures surrounding the EOMs. Franz Pirklbauer and Michael Lacher were the creators of another EOM model which has already been mentioned in Chapter 1.4 (cf. [Pirklbauer, 2001], [Lacher, 2001]). Their approach was also to use MR imaging data, not from human cadavers but from volunteers instead. The idea was to take two sets of MR images from one patient, each set showing coronal slices of the EOMs, one with the eyes in primary position and another with the eyes in secondary position. In each slice of a set the muscle of interest had to be isolated, using the algorithms developed by [Pirklbauer, 2001]. The result was a three-dimensional but very coarse muscle model which could be stored in the DXF format [Pirklbauer, 2001]. Pirklbauer and Lacher used the two DXF models of one muscle in two different states of innervation as input and calculated one dynamic three-dimensional model that could change its state within certain limits. Apart from these models another EOM model, based on finite-element analysis, was created by [Simonsz et al., 2002]. This model uses MR images of serial sections of orbits of fetuses and fits a mathematical surface through the contours on the images, resulting in a three-dimensional model consisting of the orbita, the globe and the EOMs. However, the search for the correct loading conditions for the EOM model proved to be difficult, therefore more research in that field will be necessary. The last EOM model, also based on MR imaging data, is part of a bigger project called “The Visible Human Project” which was issued by the National Library of Medicine (NLM) [Center for Human Simulation, 2001]. Its goal was to build a digital image library of volumetric data representing a complete, normal adult male and female, by means of MR and CT scans. These scans where performed on human cadavers. The resulting data set was the source for many commercial products like the “Voxel Man Series”, a set of programs dealing with the three-dimensional visualization of the human body. It also offers an environment where the user can segment certain areas in sets of MR images from the NLM and thus generate a threedimensional volume. In doing so, scan data from the EOMs can also be segmented and rendered. Finally there is the EOM model from the Blausen Medical Communications company [Blausen, 2003] depicted in Figure 3.1. It is a model that was created from the anatomical facts of the EOMs and the surrounding structures and is also available in digital form. This EOM model approach is one example for many other EOM models that have been created using the skills of artists. 3.2 Limitations of Existing Approaches As none of the above-presented models is perfect, Chapter 3.2 will compare their advantages and disadvantages as well as their limitations. The first two models we will focus on are the ones created by Joel M. Miller. His 21 Chapter 3. Muscle Model Generation Figure 3.1: Blausen EOM model [Blausen, 2003] intention for building the models of the EOMs was to learn more about the mechanical properties of the studied muscles and the extraocular pulleys, respectively. As a result he didn’t analyze MR images of single people in order to get information about, for example, pathological EOMs. Instead, he used data from several humans and created an “average” eye model, with the result of losing individual patient information. Millers second model was created out of data from human cadavers and again no correlation to living humans and their specific eye muscles could be made afterwards. However, with the results of his first model [Miller, 1989] it was possible to understand the mechanical properties of pulleys and how they influence the straight EOM movement. The second model [Miller et al., 2003] even made it possible to visualize extraocular tissues. Both models are static models, which means that the state of the EOMs could not be changed and thus no simulation of muscle movement was possible. Another drawback was that the models were rather complicated and time consuming to create. The third model explained in the previous chapter was the one created by Pirklbauer and Lacher. This model was the only one allowing the simulation of muscle movement. Thus, the model was more or less anatomically correct and rather simple to create on the basis of MR imaging data coming from a specific patient. This also allowed prediction on pathological properties of the EOMs. A possible disadvantage could be that this model only included one muscle, therefore it was only possible to reconstruct one muscle at a time. Additionally, the model had problems showing a smooth muscle surface at each reconstruction because there were certain edges visible on the created muscle that surely didn’t appear on the original muscle of the patient. Another problem is that it is not possible to reconstruct the two oblique eye muscles because they are not present at the original MR imaging data in a way that enables a reconstruction. This results in the fact that these two muscles cannot be reconstructed in Pirklbauer’s and Lacher’s model either. 22 Chapter 3. Muscle Model Generation Simonsz’s approach was limited in that he didn’t use data from specific persons either, but from serial sections of orbits of fetuses instead. So again no prediction about the situation of a certain muscle of a single person could be made. Additionally, his model only consisted of the four straight EOMs and didn’t take the oblique muscles into account. So a correct simulation of the eye movement was not possible. Another disadvantage was that only the impact of a change on one muscle on the EOM model could be visualized and not a simulation of the specific movement of an EOM. Finally, all models that were built upon the visible human project data were also not capable of reflecting anatomical data of a specific person, because again data was generated from more than one person. All models derived of data from the visible human project are only static models that cannot be changed in their state of innervation because MR imaging data was taken from many subjects in only one position due to the fact that the subjects had already been deceased. The model constructed by the Blausen Medical Communications company was built with the help of artists who used anatomical data of medical literature to recreate the model, with the purpose of simply visualizing the human EOMs, with no respect to their movement. This means that it cannot simulate the movement of an EOM. Another problem of this approach was that special patient anatomy could not be taken into account, thus leading to the fact that anatomically correct reconstruction was not possible. Table 3.1 shows all the discussed models in comparison. Our reason for building an EOM model is to analyze the movement of the EOMs with the purpose of detecting pathological changes of the oculomotor apparatus. Therefore, the basic features which mainly have to be supported by an EOM model in order to enable analysis of MR imaging data of the EOMs from certain patients are: • digital form which means that the model has to be in a form that software programs can process the data they consist of, • anatomical correctness means that the model has to reflect the anatomical circumstances which have been alleged by real life, • data from a specific person means that the data was obtained from a person, from whom we want to have a model reflecting the EOMs, • simulate muscle movements denotes if a model is able to simulate muscle movements, • show all muscles in one model means that all EOM muscles (at least the straight ones) can be viewed at the same time. 3.3 Possible Solutions for Optimization The improvement of a current three-dimensional eye muscle model will focus on the muscle model by Pirklbauer and Lacher, because this model is already implemented 23 Chapter 3. Muscle Model Generation Model Miller 1 Miller 2 Pirklbauer/Lacher Simonsz Blausen NLM Digital Form + + + + + + Anatomically Correct + + -1 -2 + 24 Data From Specific Patients + - Simulate EOM Movements + - Show All EOMs in One Model + + + + + Table 3.1: Overview on current EOM models in the VISU software and it is the easiest for generating data for a specific patient. Additionally, this model provides simulation of the movement of an EOM. The main problem with Pirklbauer’s and Lacher’s EOM model was that it didn’t show the correct anatomical representation of an EOM at each reconstruction but only in some cases. Instead, it had some sharp edges situated on the muscle surface. Figure 3.2 illustrates the problem with errors on the muscle surface of the EOM model by Pirklbauer and Lacher. So the starting point is to figure out techniques for the improvement of the surface structure of the EOM model. Figure 3.2: EOM model with errors on the surface A possible solution for optimization is to change the way the surface of the EOM model is calculated. The current generation process can be found in [Pirklbauer, 2001] and [Lacher, 2001]. First of all, the structure in which the muscle model exists has to be discussed. The raw data consists of a cloud of points in a three-dimensional coordinate system, the so-called base points. Each point represents a point on the surface of the muscle. These points can be arranged in cross-sections, which means that the cloud of points is cut into a number of slices which represent the cross1 Simonsz used data from sectioned human cadavers that had been conserved in paraffin, thus having the disadvantage of shrinking with time. The samples he used were from the 70’s. 2 Blausen used data from literature and, therefore, no relation to the person behind the data was possible. Chapter 3. Muscle Model Generation sections of a muscle [Pirklbauer, 2001]. These cross-sections and then the shape of the surface of the muscle along its length have to be calculated. Cross-sections are specified by the user who wants to have a reconstruction of an EOM. The more cross-sections there are specified, the higher the computational effort is. However, the points also represent disturbances in the DXF model, because it includes parts of tissue surrounding the muscle which don’t belong to the actual muscle. These artifacts can cause the muscle model to lose precision or even appear incorrect. Hence, a function has to be found which offers the possibility of calculating arbitrary points that lie between the points currently representing the surface. This function also has to be smooth enough to eliminate possible artifacts. Therefore, the model will neither appear more correct nor more incorrect but “more beautiful”. The most obvious way for optimization would be to find a spline function for the creation of the surface, because the formulation of the problem includes interpolation in order to get points on the surface of the muscle at arbitrary positions. But which spline function is the most suitable and is somehow easy to use in a computational form? The first possible approach for optimization could be the use of Hermite splines which were already used by Lacher for calculating parts of the EOM model. For the calculation of Hermite splines an additional point and a corresponding tangent are always needed. That spline type is not very suitable for our needs because the computational effort for finding a correct point with a corresponding tangent is much higher than the effort for using, e.g., the NURBS interpolation method, see Chapter 2.3.2. The second possible method is the polynomial interpolation [Press et al., 2002]. This type of spline curve interpolates between a set of given points by fitting a ndegree polynomial through the points (where n is the order of the polynomial). The resulting curve develops through every base point and characterizes a closed curve where each point on the curve can be calculated. The first point is the start point and the last one defines the end point of the curve. Figure 3.3 shows an example for a set of base points and Figure 3.4 shows the polynomial interpolation method applied to these base points. The disadvantage of this method is that the spline curve passes through every single point of the base points, because it uses each one of the them as a control point. This is a rather unsuitable method for our needs, because we need a curve that doesn’t exactly fit through all the given points in order to smooth out disturbances. So a better alternative has to be found. A more suitable curve can be created by using the Bézier spline interpolation method invented by Pierre Bézier [Bézier, 1977]. Bézier splines have the characteristic of always staying inside the complex hull, which is determined by the smallest polygon that includes all points of a set of points (cf. Figure 3.5) of the set of control points. The control points in the case of the Bézier splines are equal to the given set of points named base points in the previous interpolation method. If this interpolation method is used on the points shown in Figure 3.3, the curve passes in a much smoother way through the points (cf. Figure 3.6). The only disadvantage is that there is not much control over the curve, meaning that also small indentations on 25 Chapter 3. Muscle Model Generation Figure 3.3: Set of points to be interpolated 26 Figure 3.4: Third degree interpolated curve Figure 3.5: Control polygon of the points the surface as well as small bumps (both personated by the control points) will be flattened and so the display of the muscle will lose accuracy. This can be seen in Figure 3.6 where the fifth point (if counting starts at the left side) is rather far away from the rest of the points. A solution would be a curve function which behaves exactly like the Bézier splines but has the ability of changing its shape within a certain region. Introducing the NURBS interpolation method provides a solution to the problem above [Piegl and Tiller, 1980]. This kind of splines does not only use control points Chapter 3. Muscle Model Generation Figure 3.6: Bézier spline interpolation applied to the points of Figure 3.3 27 Figure 3.7: NURBS interpolation applied to the points of Figure 3.3 (the base points in the example of the polynomial interpolation) but also a so-called knot vector which determines the base points for the resulting spline curve. With the help of the knot vector and a weight vector (cf. Chapter 2.3.2) it is possible to change the shape of the curve within a certain region. Thus, small bumps or indentations can be taken into account during the curve calculation. Figure 3.7 shows the spline created using the points in Figure 3.3. It can be seen that the curve in the region under the fifth point is better fitted to the control points than the one in the example using Bézier splines. The improvement doesn’t only focus on the generation of the surface but also has to include optimization of the DXF model generation. A big disadvantage of the DXF generation process initiated by [Pirklbauer, 2001] is that it doesn’t extract data from the original DICOM files. Hence, additional information about a set of DICOM files taken from one patient are lost. A good example is the distance between the coronal MR images. This information is important for the determination of the length of the generated model. Also the information about how the image positions are situated in comparison to each other, in order to reposition the images if necessary, is important for further processing of the images. 3.4 Generation of Muscle Data for Kinematic Models Kinematic models are used to describe the behavior of parts of the human body. A kinematic model in the case of this thesis means to describe the movements of the eyes based on the mechanical properties of each EOM. The model for this purpose is already implemented in the SEE++ software system, namely the biomechanical Chapter 3. Muscle Model Generation model (cf. Chapter 1.2). The only thing missing when visualizing the model is the anatomically correct visualization of the EOMs. This is already done in SEE++, but the data for the muscles was only taken from average muscle data. So real-life conditions are not possible to visualize. The idea to improve this status is to extract muscle data with the VISU software for the usage in SEE++. 28 Chapter 4 Design of a Muscle Visualization System It is the purpose of this thesis to extend the existing software system VISU by introducing new methods of optimization to the current system structure in order to improve the surface of the three-dimensional eye muscle model. Thus, the following chapter gives a detailed description of the design of the VISU software system as well as the design of the extensions for the optimization. 4.1 Existing System Design The current VISU system architecture consists of four components (cf. Chapter 1.3). The GUI component, responsible for the display of the results as well as the interaction with the user, consists of a series of classes which are all inherited from the class TWindow. This class is a Borland C++ Builder 6- (cf. [Borland, 2002]) specific class which enables the creation of different GUI views. These views receive their data from the components described in Figure 4.1. However, a perfect capsulation as introduced with the Model View Controller pattern [Gamma et al., 1995] is not given because all the GUI views directly access the other two components. This makes an easy exchange of the components, with respect to other components, for example, with a better performance, more difficult to accomplish. The administration component manages data about a patient and stores it in the form of a directory hierarchy. Important information, e.g. name, date of birth or address of the patient, is serialized into a file. Image data in the form of DICOM files as well as bitmap data is stored in directories that correspond to each patient. The class structure of the component data administration is visualized in the class diagram in Figure 4.2. The most important components are MR analyzation and MR processing. Both were initially implemented by [Pirklbauer, 2001] and [Lacher, 2001]. Their purpose 29 Chapter 4. Design of a Muscle Visualization System 30 GUI component Data administration MR analysis MR processing Figure 4.1: Components of VISU Administration Patient string name Date birthdate string diagnosis string additionalComments Administration Map<string, Patient*>patMap addPatient() loadPatientList() savePatientList() removePatient() print() 1 0..* 1 addSeries() deleteSeries() calcDynmod() 1 0..* DynModel 0..* int stepsAlongThePath int stepsAroundThePath int pathThreshold int lastActivation Study Picture string side string description string fileName 1 * 1 1 0..1 0..* PictureSeries string muscle string activation string dxf3DRec Threshold 1 Palette 0..1 addPicture() removePicture() Polygon 0..1 Figure 4.2: Class diagram of the administration component 1 Chapter 4. Design of a Muscle Visualization System 31 was already explained in Chapter 1.3, but a closer look into the creation of the EOM model will be taken in the following. Figure 4.3 and Figure 4.4 illustrate the data flow of the creation of an EOM model. MR data from imaging device DXF file from MR analysis picture setup calculation of gravity line sort color table import DXF define polygons transform DXF format set threshold generate polyhedron calculate area centroids define data volume use scan-line/marchingcubes algorithms spline interpolation generate surface calculate volume and display surface storage set threshold interpolation between the two EOM models DXF file Figure 4.3: Data flow of the creation of a DXF model in VISU Figure 4.4: Data flow of the creation of the EOM model in VISU out of the DXF file As this thesis concentrates on the work by Lacher, it is important to be familiar with the structure of the MR processing component. Figure 4.5 shows the class structure of the dynamic muscle model. Lacher’s dynamic model is determined by the DynamicMuscle class, which holds instances of the MuscleParameters class and the DynamicModel class. These three classes are the most important ones for the following explanations, because they are responsible for the creation of the muscle surface. Chapter 4. Design of a Muscle Visualization System 32 dynamicMuscle PointList DXFModel #points filename name MuscleParameters 1 1 #midpoints 1 1 DXFModel() getName() ~DXFModel() getFilename() render() getPoints() getVertexCount... getIndexCount() 1 #model 1 MuscleImage MuscleImage() getModel() getModel() ~MuscleImage() getSourceImages() getSourceImages() getMuscleParameters() getMuscleParameters() getVertexCount() getIndexCount() #parameters 1 1 activation : float threshold : float max_threshold : float steps_u : int steps_v : int radii : float* length : float volume : float DynamicMuscle steps_u : int steps_v : int MuscleParameters() calculateMaxThreshold() ~MuscleParameters() generateMidpoints() generateSplines() generateRadii() recalcVolume() recalc() #dynamicParameters getActivation() setActivation() getThreshold() 1 1 getMaxThreshold() setThreshold() getSliceCount() getSizeU() getSizeV() getLength() setLength() setUVSize() getVolume() 1 getRadius() setRadius() getSplines() getSplines() render() #dynamicModel DynamicMuscle() DynamicMuscle() addImage() ~DynamicMuscle() getImage() replaceImage() removeImage() clearImages() getImageCount() getDynamicParameters() getDynamicModel() setInervation() getAverageVolume() getSizeU() getSizeV() setUVSize() render() getVertexCount() getIndexCount() 1 #muscleStates 1 1 #sourceImages DynamicModel 1 MuscleImageList 1 SourceImages siSizex : float siSizey : float SourceImages() getImageCount() ~SourceImages() getImageName() replaceImage() render() DynamicModel() DynamicModel() rebuildModel() ~DynamicModel() changeTexture() changeTexture() getTextureName() isResourceTexture() render() getVertexCount() getIndexCount() Figure 4.5: Class diagram of the MR processing component [Lacher, 2001] 4.2 Extension of the Existing Structure In Chapter 4.2 the necessary extensions to the VISU structure for the optimization of the muscle surface are depicted. First the picture processing is explained. Its main purpose is to extract data from DICOM files and process it. Then the calculation of the area centroids of the muscles’ cross-sections is outlined in Chapter 4.2.2 followed by the design of the analysis of variance. Finally, Chapter 4.2.4 deals with the improvements to the creation of the muscle surface. 4.2.1 Picture Processing The design for the picture processing part is kept very simple, it only consists of one class with the name PictureProcessing. This class contains an attribute which is of Chapter 4. Design of a Muscle Visualization System the type TDICOMBox which is an ActiveX component (cf. [Krug, 2000]) that has been adapted. TDICOMBox is a class that realizes the access to a native DICOM file, which enables the extraction of information like slice thickness or distance between the MR slices. Currently, methods for the extraction of the distance of the MR images between each other and for the extraction of the positions of the MR images between each other are supported. This information is stored in the header of a DICOM file and is important for the generation of the DXF model which is the result of the MR analysis component. Without the correct distance information, the muscle would appear too short and thus be incorrect according to the anatomical facts. The problem appears when using different types of MR imaging devices, but it can also appear when using only images created by one MR imaging device of the same vendor. The reason lies in the fact that the slice thickness can be chosen manually by the person who operates the MR imaging device. At the time when Pirklbauer wrote the code for the generation of the DXF EOM model he received MR images with a slice thickness of 3.6 millimeters. He implemented this value as a constant in the source code, not enabling the user to change it. Newer MR imaging devices are already able to create series of MR images with a slice thickness of 1.5 millimeters. However, the vast progress in the development of MR imaging devices made it necessary to allow a dynamic adaptation of the slice thickness stored in the DICOM files for the use in VISU. This results in the correct calculation of the muscles’ lengths. Another problem is that the images can be shifted mutually. Thus, for a better tracing of the muscle course the display window of the MR device can be changed. This results in MR images which, if put on top of each other, show an EOM muscle not reflecting the natural course of the muscle. This is a side effect of image generation with an MR imaging device (it would look as if it had huge steps in it). Getting rid of these disturbances is the main purpose of the class PictureProcessing. Methods There are three methods available: • OffsetPictures(. . . ) • GetDistanceBetweenSlices(. . . ) • GetVoxelDistance(. . . ) The first method realizes the correction of the shift of the images to each other, if necessary. It thereby uses the information gathered from the TDICOMBox component. The two other methods also use this information as they return the distance that lies between two consecutive images and the scaling factor for each voxel in millimeters. 4.2.2 Gravity Line Optimization The gravity line optimization characterizes methods for the calculation of the area centroid of each cross-section of the muscle model. As already mentioned in Chapter 33 Chapter 4. Design of a Muscle Visualization System 3.3, the cross-sections are slices of points having the same z-coordinate forming a user defined slice of the muscle. It is now vital for all further calculations of an EOM model to know the center of each muscle slice and its coordinates, respectively. Figure 4.6 shows the point cloud of the DXF muscle model with the calculated area centroids. Each area centroid is connected by a line with the one of the next slice. This line depicts the gravity line of the DXF muscle model and equals the gravity line of the real muscle of the patient. Of course this calculated line is too angular and thus has to be smoothened by means of interpolation, because at this point of improvement all the cross-sections include their original points coming from the DXF model. Thus some of the cross-sections haven’t got enough points in order to represent a cross-section that corresponds to the MR images of the muscle. These cross-sections have an area centroid which deflects from the ones that correspond to the MR images. Figure 4.6: DXF muscle model with correct length calculation The gravity line has already been calculated by Lacher in the class MuscleParameters. Starting with the DXF model he used a calculation method for the area centroids that treated each slice of points as if each slice included the same number of input points. But in fact most of the defined slices consist of not enough points to represent a muscle cross-section. This leads to false calculated area centroids, thus resulting in an incorrect muscle gravity line. Hence, the slices including too few points have to be identified and filled up with additional points to represent a correct muscle cross-section. The reason for the existence of muscle slices with too few points is that the user of the VISU software should have the possibility to choose the accuracy of the resulting EOM model by himself. This leads to the possibility of arbitrary sectioning of the DXF point cloud into slices even at “bad” positions, which can evolve because of artifacts in the underlying MR image set or because of misplaced triangles in the marching cubes algorithm [Pirklbauer, 2001]. The necessary optimization methods for improving the calculation of the gravity line of an EOM model are implemented in the class RadiusImprovement. This class interacts 34 Chapter 4. Design of a Muscle Visualization System with the class MuscleParameters, which already existed in VISU, by receiving all coordinates of the points of the DXF muscle model. It also receives the generated muscle gravity line which is calculated in the class Splines (which was also already implemented in VISU), because in RadiusImprovement only the optimization of the gravity line is done and not the actual calculation. Methods There’s only one method necessary, namely AddHelpPoints(. . . ), which improves the area centroid calculation by using the information from the coordinates of the area centroid of the nearest cross-section which corresponds to the MR images. 4.2.3 Analysis of Variance The purpose of the analysis of variance is to identify cross-sections which include either too few points or false points. The reason for their existence has already been explained above. Once identified, the cross-sections are corrected by using points from the cross-section nearest to the ones with not enough points. The result of the identification process is shown in Figure 4.7, where all the defined cross-sections (73 in this case) were analyzed and visualized in a diagram. The x-axis shows the number of arbitrary chosen cross-sections of one EOM and the y-axis the value assigned to each cross-section by means of analysis of variance. It can be seen that there are sections with low and high values, but only six of them reach a value which is higher than 80. Note that this value is only a guiding value to show that the value of those sections differs particularly from the other ones. These six sections are the cross-sections that consist of points which were derived directly from the MR images; all the others were extracted out of sections that lie between the MR images, thus not including enough information for further processing. Hence, the diagram shows which cross-sections have to be edited by adding additional points in order to suit our needs. The cross-sections between the MR images are included in the DXF model that formed a three-dimensional model out of the muscle which was visible in the MR images. Methods Two methods are important for the analysis of variance: • EvaluateVariance(), • CorrectSlices(. . . ). The first method evaluates the variance of each cross-section and the second one performs the correction by adding points, taken from the nearest complete slice, and adding them to the incomplete slice. 35 Chapter 4. Design of a Muscle Visualization System 120 100 variance 80 60 40 20 0 1 5 9 13 17 21 25 29 33 37 41 45 49 53 57 61 65 69 73 number of cross-sections Figure 4.7: Partition of variance and amount of slices 4.2.4 Surface Optimization This chapter covers the biggest part of the work that was done for the improvement, which is the spline interpolation. The interpolation method used is the NURBS interpolation. A class Nurbs takes care of the interpolation itself, while the RadiusImprovement class provides the points to be interpolated in the form of an STL vector that holds the points in three-dimensional space. This vector includes four elements: the (x, y, z)-coordinates of the point in three-dimensional space and a fourth value which represents the weight that assesses each coordinate. After each cross-section has been corrected, based on the analysis of variance, the improved slices are processed by calculating a NURBS curve for each cross-section, so that this curve fits exactly between the points used as control points (cf. Chapter 3.3). To do so each cross-section is split into two halves, one half containing all points in the range from 0 to 180 degrees in angle and another half containing points ranging from 181 to 360 degrees (cf. Figure 4.8). The angle always refers to the area centroid of the currently processed cross-section. Subsequently, each obtained half is interpolated in the class Nurbs. The result consists of two splines which form one cross-section of the EOM (cf. Figure 4.8). After repeating this process for each consecutive slice, the surface of the EOM model is created. The next step is to improve the surface also in the length of the model. This has to be done, because the 36 Chapter 4. Design of a Muscle Visualization System already-calculated shapes of the cross-sections have slightly different area centroids which result in a shift of the sections to each other and thus in a “bumpy” surface. By using the same spline interpolation again on the sum of all spline interpolated cross-sections, a totally homogenous surface with no disturbances is obtained. Figure 4.8: Cross-section of DXF model with both halves interpolated Methods The class RadiusImprovement includes one method CreateRadii(. . . ). This method starts the creation of the muscle model using NURBS and returns a data structure including the complete muscle surface. It also calls two private methods Interpolate(. . . ) and DoSplineOnMuscleLength(. . . ). The first of these methods uses the class Nurbs to generate the spline and the second one does the same interpolation again but this time on the whole length of the muscle. The class Nurbs includes mainly the method CurvePoint(. . . ) to calculate a NURBS curve of a set of control points which are received from RadiusImprovement. It also includes a helper method DoWeightOnPoints(. . . ) to format the control points for its needs. 4.3 Modifications of the VISU Structure The modifications of the VISU structure turned out to be more complicated than they appeared to be at first glance. The classes that were modified are part of nearly all components of VISU. The first modification was done in the GUI component where the class PictureProcessing was instantiated in the class TWizardGenerateStaticModel. This class takes 37 Chapter 4. Design of a Muscle Visualization System care of the extraction of the image position data out of the original DICOM files as well as the distance data of the MR images with respect to each other. Images were also shifted, when necessary, by the class PictureProcessing. The distance data was handed over to the MR analyzation component, where the correct length of the muscle could now be used for the calculation of the DXF model. However, the most extensive changes were done in the MR processing component. The generation of the muscle gravity line, which is the resulting line after connecting all calculated area centroids of ascending cross-sections (cf. Figure 4.6), was improved using the methods in the class RadiusImprovement. The next step was the creation of the surface of the EOM model. Thus, raw data for the interpolation had to be provided. This was done by handing over the set of points of one DXF model to the class RadiusImprovement, where the whole muscle model was generated using the NURBS interpolation method. This model was then returned to the class MuscleParameters in an adequate data structure and the whole process was repeated for the second DXF input model. The data structure consisted of an STL vector which included a pair of two vectors, the first one stored the spline point in three-dimensional space and the second one stored the area centroid of the cross-section to which the spline point belonged. An advantage was that the only extension of the class MuscleParameters was the addition of one attribute, which is an instance of the class RadiusImprovement. With this extension the creation of the EOM model was finished, but a big problem had yet to be solved, namely the problem of making the generated new surface model compatible with the existing visualization structures in VISU. The former model of an EOM was defined by the muscle gravity line and a radius function [Lacher, 2001]. The radius function was precalculated and stored in a vector array. This had the advantage of easy calculation and interpolation of the surface of the EOM model. However, the newly introduced concept of the NURBS interpolation made it impossible to retain the former definition of the EOM model. An adaptation was made by changing and adding methods to the classes MuscleParameters, DynamicModel and DynamicMuscle. Once finishing the creation of two muscle gravity lines and the associating radius function, the EOM model can be displayed and simulated. This is done by recalculating the radii stored in the radius vector with respect to the muscle volume and rerendering the already displayed model. However, a displayable model cannot be described only by its radius and that’s why the vertices of the model have to be calculated. Once this is done (in the class DynamicModel ) the vertices are handed over to a predefined data structure in the same class, whereby the change of the radii is accomplished in the classes MuscleParameters and DynamicMuscle. Therefore, each time the user changes the state of the EOM model, all three classes are involved. Figure 4.9 illustrates how the three additional classes where inserted into the existing VISU structure. The following methods were involved in the process of adapting the new spline interpolated model to the VISU structure: 38 Chapter 4. Design of a Muscle Visualization System component GUI TWizardGenerate StaticModel 39 PictureProcessing dicomViewer* :TDICOMBox fileNames* :TStringList PictureProcessing() ~PictureProcessing() OffsetPictures(...) GetDistBetweenSlices() GetVoxelDistance() RadiusImprovement component MR processing MuscleParameters midpoints* :std::vector<Vector3f> points* :PointList splines* :SplineGroup RadiusImprovement() RadiusImprovement(...) RadiusImprovement(...) ~RadiusImprovement() CreateRadii() Interpolate(...) DoSplineOnMuscleLength(...) EvaluateVariance() CorrectSlices(...) AdddHelpPoints(...) WriteMuscleData(...) Nurbs U*, W* ,u* :double n, p :int Nurbs(...) ~Nurbs() CurvePoint(...) DoWeightOnPoints(...) Figure 4.9: Expanded class diagram of the VISU structure • RebuildModel(. . . ) from class DynamicModel, • GetRadius(. . . ), SetRadius(. . . ), GetSplineVector(. . . ), GetSplinePair(. . . ), from class MuscleParameters, • SetInnervation(. . . ), from class DynamicMuscle. GetSplineVector(. . . ) and GetSplinePair(. . . ) were the only methods added to the class MuscleParameters. All other methods were already available and have just been modified. The adaptation also led to the introduction of an additional member variable in the class MuscleParameters, which is necessary in order to store the muscle model generated by RadiusImprovement. This data structure is used for the interpolation of the EOM model. Therefore, this is the structure that replaces the combination of muscle gravity line and radius function. Finally, data was generated for use in the SEE++ software. This was done in the class RadiusImprovement using the method WriteMuscleData(. . . ). Its job is to calculate the dimensions of the muscle and then write the obtained data into an initialization file. Chapter 5 Implementation The implementation of this thesis work includes mainly algorithms for the preparation and processing of images. Therefore, it is necessary to elucidate those algorithms with the intention to provide the reader with knowledge about the functionality of the software structures used. All algorithms have been implemented in C++. 5.1 Muscle Gravity Line Optimization The basic purpose of the muscle gravity line optimization has already been outlined in Chapter 4.2.2. This chapter will describe in detail how the actual algorithmic structures were implemented. The former calculation of the muscle gravity line was implemented in VISU using the source code in [Lacher, 2001]. Therefore, an array of points is built up which were extracted out of the DXF muscle model coming from the MR analyzation component. The array consists of a Standard Template Library (cf. [Prata, 1998]) vector containing points represented by three-dimensional vectors. The stored point cloud is then parted along its z-coordinates by storing the indices of those points having the same z-coordinates in a second vector containing arrays of integers. After inserting the indices into the second vector, this vector represents the cross-sections of the DXF muscle model. Lacher’s next step was to create the area centroid of each cross-section. However, the cross-sections didn’t contain enough points, so this is the starting point for the improvements. The area centroid is calculated by simply adding all coordinates of one cross-section together and then dividing the result by the number of cross-sections. The newly improved calculation method uses the same concept with the refinement of adding additional points to cross-sections having too few points. This is implemented in the method AddHelpPoints(. . . ) of the class RadiusImprovement. The method gets a reference to the currently calculated area centroid and an array which contains the indices to the points ordered by the muscle cross-sections as well as the index of the currently processed cross-section. Every time a cross-section doesn’t contain enough points, the AddHelpPoints method is called, thus detecting the nearest cross-section which consists of enough points to calculate a correct area centroid. If such a cross-section 40 Chapter 5. Implementation is found, the points are combined with the currently calculated area centroid. The return value of the method is the number of points that has been added. The fact that the first cross-section always corresponds to an MR image ensures that there is always a cross-section to gather points from. After finishing the calculation of one area centroid, a second improvement is carried out. The idea is to sort out area centroids that have an ascending slope of more than 45 degrees with respect to the previous area centroid. If one is found, this area centroid and the one which has already been calculated are averaged. This way the area centroid having a slope that is too high gets one that is lower. After the area centroids of all cross-sections have been calculated, the points that represent the muscle gravity line are interpolated using the Hermite spline interpolation to get a smoother gravity line. This is already implemented in VISU and doesn’t have to be changed. 5.2 Applying Analysis of Variance The analysis of variance is the basis for the interpolation methods following in the next chapter. The first step is to create a vector storing not only the points belonging to one cross-section but also the angle in the x-y plane that the vector of the point spans regarding to the area centroid of the currently processed cross-section (cf. Figure 5.1). The direction of the z-axis is assumed to develop through all crosssections of a muscle, whereby the x- and y-axes lie in the same plane in which one cross-section lies. The correct angle for each point is already created in VISU; it only has to be mapped for our needs. This is done in the method FillBuckets(. . . ) of the class RadiusImprovement. The second step is the evaluation of the variance of each cross-section in order to sort out those without enough points or faulty points. This has already been explained in Chapter 5.1. But this time it is also taken into account that there are cross-sections without enough points which form a suitable cross-section for the spline interpolation anyway. Finding such cross sections results in a more correct EOM model. The method EvaluateVariance(. . . ) sorts the points of all cross-sections according to their ascending angles. This is done by using a helper method, Compare(. . . ), which is called by the STL method sort(. . . ) at the beginning of EvaluateVariance. Afterwards the variance is calculated according to the equations outlined in Chapter 2.3.1, then the result is stored in an STL vector, which is the return value of the EvaluateVariance method. The next step is to correct those cross-sections which have points that do not correspond to an MR image or which don’t have enough points to satisfy the NURBS interpolation. The correction takes place in the method CorrectSlices(. . . ) of the class RadiusImprovement. It receives the vector with the values coming from the analysis of variance as input. Based on these values, the slices that are below a certain value are corrected by adding the points of the nearest cross-section which 41 Chapter 5. Implementation 42 y-axis points of the cross-section a area centroid 0/0 x-axis Figure 5.1: Area centroid of a cross-section and the angle a that the point spans regarding the area centroid consists of correct points in the sense of the NURBS interpolation. The following steps are performed in order to add points: • Iterate over the values received from the analysis of variance. • If a value is below a certain threshold, start searching for a slice which has a better variance. • Find the best variance by searching left and right, take the first slice found which has correct points and the shortest distance to the faulty slice. • Add the index of the found slice to the result, which is a vector representing all cross-sections with each index entry representing the index of the correction slice. 5.3 Interpolation Using Splines This chapter describes the spline interpolation in detail using NURBS with RadiusImprovement and Nurbs as the participating classes. The main methods involved in the interpolation concerning the class RadiusImprovement are: • GenerateSplineArray(. . . ) • CreateRadii(. . . ) • Interpolate(. . . ) Chapter 5. Implementation 43 • CalculateNewResult(. . . ) • DoSplineOnMuscleLength(. . . ) Nurbs uses all included methods for the interpolation. But before describing the interpolation itself, it is necessary to determine the input points over which the iteration should take place. The first step is to rebuild the cross-sections by adding points according to the output vector of the method CorrectSlices (cf. Chapter 5.2). Additionally, the points of one cross-section are also parted in those having an angle which is lower than 180 degrees and those whose angle is greater than 180 degrees. The result of this partitioning is stored in a data structure consisting of an STL vector which contains a pair of lists each containing the points belonging either to the first half of a cross-section or to the second half. All this takes place in the method GenerateSplineArray, which is called in the class MuscleParameters by the method GenerateRadii. Now the partitioned points of each cross-section have to be processed in order to contain only unique points, which means that each point representing a point on the surface of the muscle has to follow the criteria x1 < x2 < x3 < · · · < xn with n representing the number of points in one half of a cross-section. The existence of two points having the same coordinates is possible because of the insertion of points into cross-sections with too few points or faulty points. The reason why all points of one slice have to be different from each other is that in some cases it is possible that a spline curve is created which has a cusp (visual discontinuity) (cf. Figure 5.2), although the resulting spline curve is always c1 continuous1 . P2 = P3 P1 P4 P0 P5 Figure 5.2: NURBS curve with two equal control points P2 = P3 Such cusps have to be avoided in order to generate a surface that looks smooth enough, although the curve is c1 continuous in such a point. After the points in each cross-section have been prepared in the above-described way, they can be used as input for the generation of the splines by handing over 1“c1 continuous” means that the first derivative is continuous. Chapter 5. Implementation 44 the points of one half of a cross-section to the class Nurbs. When finished with the generation of one half, the next half is processed. The method Interpolate is responsible for the execution of the interpolation. Therefore, it first instantiates the class Nurbs and then calls the method CurvePoint after creating a weighted version of the input points using DoWeightOnPoints. These weighted points are necessary for the adaptation of NURBS, since they can influence the shape of the resulting curve. However, in this thesis the vector for the determination of the weighted points is set in a way that each point is being amplified with the same number. In the method CurvePoint, the actual calculation of one point on the spline curve is done in the following way: • determine the span in the knot vector U, • calculate the nonzero blending functions (cf. Chapter 2.3.2), • calculate the point using Equation 2.8. Finding the span where a point on the curve lies is important because otherwise all the blending functions of a NURBS curve would be generated. As a NURBS curve is the sum of many smaller splines, it is a big time saver to only calculate those blending functions that are nonzero and thus contribute to the generation of the resulting curve. The FindSpan algorithm is basically a binary search algorithm that searches through the knot vector U (cf. Equation 2.9). int Nurbs : : FindSpan ( double u , int n ){ i f ( u == U[ n +1]) return ( n ) ; int low = p ; int high = n+1; int mid = ( int ) ( low+high ) / 2 ; while ( u < U[ mid ] | | u >= U[ mid +1]){ i f ( u < U[ mid ] ) high = mid ; else low = mid ; mid = ( low+high ) / 2 ; } return mid ; } Listing 5.1: Method FindSpan Listing 5.1 shows the method FindSpan of the class Nurbs where u is the input point for the spline function, n the number of control points, U the knot vector and p the degree of the NURBS curve. The result is the index in the knot vector where the blending functions are nonzero. The next important algorithm is the one to calculate the blending functions. It is implemented in the method BasisFunctions(. . . ) and implements Equation 5.1 Chapter 5. Implementation 45 (from [Piegl and Tiller, 1980]) to compute the nonzero blending functions. ½ 1 : if ui ≤ u < ui+1 0 : otherwise u − ui ui+p − u Ni,p (u) + Ni+1,p−1 (u) Ni,p (u) = ui+p ui+p+1 − ui+1 Ni,0 (u) = (5.1) The variable ui in Equation 5.1 indicates the ith knot point and the variable u the value which is the input for the interpolation. void Nurbs : : B a s i s F u n c t i o n s ( int i , double u , int m, double ∗ N){ N[ 0 ] = 1 . 0 ; double ∗ l e f t , ∗ r i g h t , saved , temp ; l e f t = new double [m] ; r i g h t = new double [m] ; for ( int j = 1 ; j <= p ; j ++){ l e f t [ j ] = u − U[ i +1−j ] ; r i g h t [ j ] = U[ i+j ] − u ; saved = 0 . 0 ; for ( int r = 0 ; r < j ; r++){ temp = N[ r ] / ( r i g h t [ r +1]+ l e f t [ j−r ] ) ; N[ r ] = ( double ) saved + ( double ) r i g h t [ r +1]∗ temp ; saved = l e f t [ j − r ] ∗ temp ; } N[ j ] = saved ; } delete [ ] l e f t ; delete [ ] r i g h t ; } Listing 5.2: Method BasisFunctions Listing 5.2 shows the method BasisFunctions which returns an array containing the values of the blending functions in N at the value u. i is the index to the knot vector U , and m is the size of U . Once the blending functions are determined, the point on the spline can be calculated. This calculation is done over the whole range that one cross-section specifies. This means that the calculation starts at the point which has the smallest value on the x-axis of a cross-section and stops at the point having the highest value on the x-axis. Moreover, it is also ensured that the transition between the two curves (the upper half and the lower half of a cross-section) is c1 continuous, because both splines have the same start and endpoints by using the adjacent point of the other half. Finally, all the generated points are stored in a map together with their angle regarding the area centroid of each cross-section. This is necessary for mapping the generated points to the visualization methods which are already implemented in VISU. Chapter 5. Implementation 46 To display the muscle’s surface and to realize a correct mapping of the texture to the surface of the muscle, it is vital that all spline points have a defined distance to each other. This includes equally spaced distances in one cross-section as well as in the next, leading to the result that points of one cross-section correspond to points of a consecutive cross-section by lying in the same plane. This property is, however, not realized after the NURBS interpolation. So the previously calculated splines have to be processed in order to obtain equally spaced points. The responsible method for this process is CalculateNewResult(. . . ) of the class RadiusImprovement. It receives the points of one spline curve, generated over one cross-section, as well as the position of the area centroid of the currently processed cross-section as input. The output is a vector containing equally spaced points around the area centroid of each cross-section. The points should be distributed having a distance of an adjustable amount of degrees to each other. As already mentioned above, all points of a spline are stored together with their angle regarding the area centroid of a cross-section. The idea is to have a counter c1 counting up an angle in equal distances δ which can be adjusted by the user. This angle can be used to find points of the spline curve that fit the criteria β ≤ α < γ, where α is the angle determined by the counter and β and γ, respectively, are the points belonging to two consecutive points of the spline curve. Figure 5.3 shows a single cross-section which has already been interpolated using NURBS. It can be seen that the points which were generated have no equal distance to each other. V~1 and V~2 are two vectors representing two consecutive spline points of a cross-section. Vector ~ represents the area-centroid of a cross-section, thus having its starting point at M the center of the coordinate system represented by the two axes x and y. Vector P~ is the one calculated from the current angle a = α which is consequently increased by δ as well as elongated to intersect vector V~12 which is the resulting vector from ~ n−i is the new vector determined by the intersection subtracting V~1 from V~2 . Vector X ◦ ~ ~ point of V12 and P (n = 360 /number of points on the spline, i = 0 . . . n − 1). X~n−i has to be calculated by subsequently increasing α from 0 degrees to 360 degrees ~ n−i (drawn in dashed lines in Figure 5.3) can be in equal distances δ. Vectors X calculated each having the same increase of angle (represented by bn , bn−1 , bn−2 where bn−2 + δ = bn−1 , bn−1 + δ = bn ). Vector X~n can be calculated by solving the linear system of equations: ~n = M ~ + s ∗ P~ X ~ n = V~1 + t ∗ V~12 X V~12 = V~2 − V~1 ~ |) P~ = P~0 ∗ (| V~1 | + | M (5.2) (5.3) (5.4) (5.5) When substituting Equation 5.2 into 5.3 (cf. Equation 5.6), the two variables s and t can be calculated (from Equations 5.7 and 5.8): Chapter 5. Implementation 47 Y V 12 P X n-2 X n-1 Xn V2 V1 a M bn b n-2 b n- 1 -X X -Y Figure 5.3: Linear interpolation of the NURBS-generated cross-section ~ + s ∗ P~ = V~1 + t ∗ V~12 M ~ x + s ∗ P~x = V~1x + t ∗ V~12x M ~ y + s ∗ P~y = V~1y + t ∗ V~12y M (5.6) (5.7) (5.8) This results in: t = ~ y + s ∗ P~y − V~1y M V~12y s = ~ x ∗ V~12y − M ~ y ∗ V~12x + V~12x ∗ V~1y − V~12y ∗ V~1x M V~12x ∗ P~y − V~12y ∗ P~x (5.9) (5.10) ~ n−i for each angle a from Equations 5.2 or Now it is possible to calculate vector X 5.3. 5.4 Muscle Data Generation The generation of muscle data has the purpose of providing the SEE++ software system with geometrical muscle data. This data is used in SEE++ to display the Chapter 5. Implementation EOMs with higher accuracy than in the current version. The interface between VISU and SEE++ are two files where data is stored in special number-value pairs that represent information about each muscle. The files are generated in the method WriteMuscleData(. . . ) in the class RadiusImprovement. Its input parameters are the name of the file, which is “MuscleDataLeft.ini” or “MuscleDataRight.ini” for the left and the right eye, respectively and the surface data of the generated EOM model. The structure of the data files which is required by the SEE++ software system is shown in Listing 5.3. 1 : [RM] 2 : NumberOfArcOfContactSlices=20 3 : N u m b e r O f S p l i n e S l i c e s =30 4 : NumberOfSlicePoints =20 5 : S p l i n e T e n s i o n =0.7 6 : Width−X−Values = 0 , 1 , 5 , 1 1 , 1 3 , 1 6 , 2 0 , 2 5 , . . . 7 : Width−Y−Values = 2 , 2 . 6 , 3 . 0 , 3 . 2 , 3 . 4 , 3 . 8 , . . . 8 : Height−X−Values = 0 . 0 0 , 1 . 0 , 5 . 0 , 1 1 . 0 , 1 3 . 0 , . . . 9 : Height−Y−Values = 0 . 0 1 , 0 . 6 , 1 . 0 , 1 . 3 , 1 . 2 , . . . 1 0 : I n n e r v a t i o n P o s i t i o n =78 1 1 : I nne rvationW idth =30 1 2 : B u l b u s R a d i u s D e l t a T o l e r a n c e =0.1 Listing 5.3: SEE++ initialization file specification Each file is divided into sections which specify the type of muscle followed by numbervalue pairs which provide the actual muscle dimension data. The section name specifies the type of muscle. In the example of Listing 5.3 [RM] the rec. med. muscle is specified. Pair number 2 defines the number of cross-sections that specify the part of the muscle which touches the bulbus. This part of the muscle consists mainly of the muscle tendon and is not visible on the MR images. The result of this is that VISU is not able to calculate this part of the muscle and can’t thus display the sinewy part. The next pair enumerates the number of cross-sections on the muscle gravity line that don’t touch the bulbus, so the whole length of the muscle can be calculated by adding pair 2 and 3. Pair number 4 specifies the number of points per spline in a cross-section. Pair number 5 is a SEE++ specific value. Pairs 6 to 9 specify the dimensions of the EOM. They include the actual data which is generated by VISU. Finally, pairs 10 to 12 are again SEE++ specific tags. Figure 5.4 shows the rec. med. muscle of a right eye in the SEE++ software system. This muscle image was generated from data produced by VISU. The black lines mark the point from where the muscle was generated by VISU. The part before the left black line and the part of the muscle after the right black line were filled up according to average muscle dimension data. It can be seen that the rec. med. muscle of the right eye has now nearly the same dimensions as the muscle that can be seen underneath the SEE++ muscle in the same figure which was generated with VISU. The reason for the differences is that SEE++ produces symmetric muscles and doesn’t take asymmetric muscles into consideration. 48 Chapter 5. Implementation Figure 5.4: SEE++ screenshot showing the EOM built of muscle data from initialization files 49 Chapter 6 Validation An EOM model is only useful for the person who works with it, if it matches anatomically with the MR images. However, this important premise is much harder to accomplish than it might seem at first glance. The most important question concerns, how it can be ensured that the EOM model, which has been reconstructed from MR images, is anatomically correct. The answer to this question will be the focus of this chapter. But before starting with the validation, it must be made clear why the name of this chapter is only Validation and not Validation and Verification. Validation determines whether a practically useful system was built, whereas verification determines whether a system was built correctly. Thus, the verification of the VISU software system, where the algorithms for the optimization of the surface of the EOM model were implemented, would mean to provide formal evidence that the EOM model shown is correct for each possible set of MR images used as input. Another possibility for verification would be to compare the current EOMs of the patients, who were used to get the MR images, with the calculated EOM model in VISU. But since this is not possible, a verification is also not possible. Additionally, it is not feasible to develop a mathematical model which describes the whole EOM generation process in VISU. The conclusion is that only a validation of the created EOM model is possible. Therefore, it has to be shown that the generated EOMs match with the MR images which served as an initial point for the creation of the models. 6.1 Comparing Reconstructed Volumes with MR Images The output of VISU, namely the EOM model, is compared with the MR images which were the starting point for the creation. Therefore, the images which are available in the DICOM format are converted into bitmap files. Only those images used for the creation of the EOM model are converted and stored by VISU together with the original files. After the EOM model is created in VISU, it is displayed on the screen using OpenGL (cf. [Claussen, 1997]). Screenshots of the EOM models are made which show one reconstructed model with the eyes in primary position 50 Chapter 6. Validation together with the bitmaps showing the MR images. Each screenshot shows one MR image representing one cross-section of a muscle and is overlapped with the EOM model at the corresponding cross-section of the muscle. Figure 6.1 shows an EOM model of the med. rec. muscle of the right eye without any overlapping MR images. The same model including one specific MR image is shown in Figure 6.2. The model was properly scaled in order to provide a better perspective of the overlapping MR image. The muscle model is only partially visible because the rest is covered by the MR image. Figure 6.1: EOM model without overlapping MR images Figure 6.2: EOM model with overlapping MR images The next step is to compare screenshots of the EOM model and screenshots of the 51 Chapter 6. Validation DXF EOM model both in different positions together with the accompanying MR images. The reason is that one DXF EOM model (cf. [Pirklbauer, 2001]) provides the input data for one EOM model. However, since this model cannot change its innervation, a second EOM model is needed. This is generated from a second DXF EOM model showing the muscle with the eyes in a different position as in the first DXF EOM model. VISU can now interpolate between these two EOM models by changing the innervation and this results in the final EOM model which is visualized. When one DXF EOM model of the muscle and the EOM model with the same innervation as the DXF EOM model both comply with the MR images, one part of the validation is finished with respect to the matching of the MR images to the optimized EOM model. The models shown in Figures 6.3 and 6.4 are magnified in Figures 6.5 and 6.6 for better comparison. The models are shown only in wireframe mode, because otherwise the contrast to the MR image would not be very good, with the result that the differences would be harder to evaluate. Figures 6.3 and 6.4 as well as the magnified Figures 6.5 and 6.6 show the med. rec. muscle of the right eye in frontal view, whereas the following pictures show this muscle together with the accompanying EOM models from the side for the sake of better analyzing of the differences. It can be seen that the EOM model in Figure 6.6 matches very well with the black contours of those parts of the MR images that represent the muscle. The reason the boundaries of both models in Figures 6.5 and 6.6 don’t match exactly with the darker parts of the MR images is that during the reconstruction process the muscles on the MR images have to be identified “by hand”. This identification process is not easy since the muscle in an MR image has to be differentiated from the surrounding tissue and all this differentiation has to be performed by the user (cf. [Pirklbauer, 2001]). The dynamic EOM model doesn’t match exactly with the DXF EOM model either, but this is actually quite good because the DXF EOM model only shows a very coarse EOM model with lots of edges on its surface. Since a real muscle doesn’t have any edges on the surface either, the dynamic model is now much closer to a real muscle. Figures 6.7 and 6.8 as well as the magnified Figures 6.9 and 6.10 show the med. rec. muscle of the right eye with the MR image, whereby other cross-sections are shown in order to compare the models with other MR images of the same muscle. Figures 6.11 and 6.12 as well as Figures 6.13 and 6.14 show another muscle of the same patient, the sup. rec. muscle. Again, screenshots were taken from the side to enable a better comparison. In those screenshots the optimized EOM model matches exactly with the contours defined by the DXF model. 6.2 Comparison with Existing Models The comparison with existing models focuses on the optimized EOM model and the EOM model which was implemented in the earlier versions of VISU (the model 52 Chapter 6. Validation 53 Figure 6.3: DXF EOM model with overlapped MR images Figure 6.4: Dynamic EOM model with overlapped MR images Figure 6.5: DXF EOM model with overlapping MR images, magnified Figure 6.6: Dynamic EOM model with overlapping MR images, magnified built by [Lacher, 2001]). All the other models which were described in Chapter 3 do not match the criteria stated in Table 3.1 and as a consequence are not subject to comparison. As each of the EOM models is only capable of showing a single muscle at the same time, one muscle of a series of MR images is chosen and an EOM model of the optimized version and one of the same muscle in the non-optimized version are created in VISU. These models are compared based on their surface structure, their simulation behavior (i.e. which means how they react to a change of the state Chapter 6. Validation 54 Figure 6.7: DXF EOM model with overlapping MR images Figure 6.8: Dynamic EOM model with overlapping MR images Figure 6.9: DXF EOM model with overlapping MR images, magnified Figure 6.10: Dynamic EOM model with overlapping MR images, magnified of innervation) and their matching to the MR images. First, we will compare the surface of the two EOM models, because the optimization of the surface of the EOM model was the main goal of our work. Figure 6.15 shows the non-optimized EOM model built from the rec. med. muscle of the right eye. It can be seen that the surface of the model in Figure 6.15 has a much “bumpier” structure than the surface of the model in Figure 6.16, which shows the optimized version of the same EOM model. The conclusion of this is that the surface has been Chapter 6. Validation 55 Figure 6.11: DXF EOM model with overlapping MR images Figure 6.12: Dynamic EOM model with overlapping MR images Figure 6.13: DXF EOM model with overlapping MR images, magnified Figure 6.14: Dynamic EOM model with overlapping MR images, magnified optimized in this particular situation. The next step is the comparison of the two models according to their simulation behavior. To accomplish this, screenshots of both models showing the muscle in different innervation states are compared. Figure 6.18 shows the optimized model with an innervation of 0 and Figure 6.17 shows the non-optimized model, also with an innervation of 0. The following Figures 6.19 and 6.20 show both models with an innervation of 50 and Figures 6.21 and 6.22 show the models with an innervation Chapter 6. Validation Figure 6.15: Dynamic EOM model, non-optimized version 56 Figure 6.16: Dynamic EOM model, optimized version of 100. An innervation of 0 means that the muscle is displayed with the eyes in primary position. An innervation of 50 means that the EOM model is exactly in the middle between the primary and the secondary position. An innervation of 100 means that the eye is in secondary or tertiary position and the models show the muscle’s corresponding shape. It can be seen that the optimized EOM model shows the same behavior as the non-optimized because no heavy aberrations are visible. This leads to the conclusion that the simulation of the muscle movements, which was already implemented before the optimization took place, hasn’t changed. The last comparison covers the compliance of the EOM models (the optimized as well as the non-optimized) with the MR images. Figures 6.23 and 6.24 show the med. rec. muscle of the right eye in primary position. Again, the visualization in wireframe mode was selected for a better determination of the muscle position on the MR images. Both models match with the surface of the muscle structures on the MR images. In Figure 6.23 the part of the EOM model, which in Figure 6.24 is covered by the MR image, is visible through the MR image because the version of VISU with the non-optimized EOM model includes transparent MR images which could not be disabled. Nevertheless, a clear conclusion about the compliance of the MR images with the EOM model can be taken, namely that the optimized model matches with the MR images. 6.3 Extraction of Muscle Dimension Data The validation of the extraction of muscle dimension data cannot be realized by simply comparing pictures. Instead, the extracted data has to be compared with Chapter 6. Validation 57 Figure 6.17: EOM model with an innervation of 0 Figure 6.18: Optimized EOM model with an innervation of 0 Figure 6.19: EOM model with an innervation of 50 Figure 6.20: Optimized EOM model with an innervation of 50 Figure 6.21: EOM model with an innervation of 100 Figure 6.22: Optimized EOM model with an innervation of 100 real-life dimensions, which means to use data measured on a muscle not by VISU but by some other program or by hand. A possible way of doing this is to compare the extracted data from VISU with muscle dimensions data that can be measured Chapter 6. Validation Figure 6.23: Dynamic EOM model, non-optimized version Figure 6.24: Dynamic EOM model, optimized version in the MR images. The standard software Adobe Photoshop (cf. [Adobe, 2002]) was used to measure the pixel dimensions of the muscles shown in the MR images. The MR images for the comparison were taken from screenshots (cf. Figure 6.25) from 58 Chapter 6. Validation VISU because in those images the muscle of interest was already segmented by a surrounding polygon and the threshold was changed for better differentiation of the tissues surrounding the muscle (cf. [Pirklbauer, 2001]). The screenshots were taken with the MR images in their real dimensions, not in the magnified version seen in Figure 6.25. Figure 6.25: Screenshot of the selected muscle in the MR images The measured pixel dimensions were then recalculated to receive the dimensions of the muscle in millimeters which is the output unit of VISU concerning the muscle data. The factor for the recalculation of the pixels was achieved from an additional file which is already stored by VISU during the creation of the DXF model. For the measurement the muscle visualized in Figure 6.1 was chosen. The measured lengths and heights from eleven MR images are listed in Table 6.1, whereby the height of the muscle is the longest dimension of the muscle on the current MR image in vertical direction (the axis that develops vertically in the sagittal plane) and the width is the dimension of the muscle in horizontal direction (the axis that develops horizontally in the coronal plane). The space between the images is 1.8 mm. This value is gathered from the file generated by VISU. Therefore, the length of the muscle that can be displayed according to the MR images is 10 * 1.8 mm = 18 mm. 59 Chapter 6. Validation muscle image cross-section 1 2 3 4 5 6 7 8 9 10 11 60 height MR VISU 9.4 8.44 9.4 8.48 9.4 9.07 9.8 9.17 9.8 9.51 10.2 9.70 9.8 9.87 9.8 9.83 9.4 9.77 9.4 9.63 9.4 9.59 width MR VISU 5.1 4.94 4.7 4.14 4.3 4.43 4.7 4.46 4.3 4.35 4.3 4.27 4.3 4.12 3.9 4.00 3.5 3.83 3.9 3.78 3.9 3.96 length MR VISU 0.0 0.0 1.8 1.41 3.6 3.19 5.4 5.43 7.2 6.99 9.0 8.77 10.8 10.84 12.6 12.20 14.4 14.40 16.2 16.17 18.0 17.92 Table 6.1: Comparison of muscle dimension data in millimeters, measured form the MR images and extracted from VISU Table 6.1 also shows the values that were generated by VISU for use in SEE++. Although it is rather difficult to determine the contours of the muscle in the MR images, a good extraction of the dimensions has been achieved. It can be seen that most of the values are within the range of at least +/- 1 mm. Chapter 7 Software Development Aspects The development of the different algorithms in this thesis proved to be rather difficult, but this, of course, is inherent in the nature of algorithms. One part of the accomplished work that really appeared to be more difficult than one could think in the first place was the inclusion of the algorithms into the existing system structures of VISU. The reason for this is that the design of VISU is not very good concerning the exchangeability and the extensibility of existing components. This led to a lot of troubles during the inclusion of the methods for optimization. 7.1 Process View The project VISU began as a student project at the Upper Austria University of Applied Sciences in Hagenberg. It was created in order to combine the functionality of two thesis works as well as to add data storage for patient specific information. In the end VISU was a fully functional program which had to be extended by the functionality of optimizing the EOM model. The connection to the SEE++ software system was made by providing data of the dimensions of the EOMs in the form of a pre-defined file. The whole VISU project is encompasses research and hence will be continued as a research project. The design of the classes that had to be added to the existing design of VISU for implementing the optimization was kept very simple. The idea was to provide methods for interpolation and optimization, respectively, that could easily be exchanged. This was necessary because, as already mentioned at the beginning of the chapter, the design of VISU has severe deficits concerning the strict separation of the different components from each other. The only way of changing the methods for the generation of an EOM was to add the classes for optimization into the existing code in VISU. However, although this was not the best solution, it was the only way to solve this problem within the time that was available. A better way to improve VISU would be to redesign the whole VISU software application and consider the presented techniques for optimization already in the design. 61 Chapter 7. Software Development Aspects 7.2 Product View The purpose of this thesis was the development of algorithms for the optimization of an existing three-dimensional EOM model as well as the generation of muscle data for SEE++, and all this within an existing software system. From a product standpoint, a concrete product was not created; instead, an existing product was improved by adding functionality. The product engineering which was practiced in this thesis did not differ very much from the traditional product creation cycle. However, there were some differences. Since the planning didn’t include the planning of a whole software system, only the planning of the adaptation of the optimization had to be considered. The same applies to the execution of the planned optimization methods. They were adapted to the requirements that emerged out of the fact that a current system had to be improved. As already stated, since the optimization of the EOM model was the forefront, checking of the optimization methods was the main criterion. Every time a possible solution was found for the optimization, the review of the newly generated output was the criterion that gave the input for the next step in the development of the optimization. The last topic, which, of course, relates to the previous criteria, was the validation—also an important criterion in all product development. With this the whole set of additional methods could be validated. 62 Chapter 8 Conclusion The last chapter in this thesis provides a summary of the goals which have been achieved. It also outlines possible future improvements and enumerates open problems. 8.1 Goals Achieved The main goal of this thesis was the improvement of a three-dimensional eye muscle model. This goal was achieved by solving the following key problems: • extraction of MR imaging data from DICOM files, • determination of the correct spline interpolation method, • preparation of the DXF model data, • inclusion of the optimized model into VISU, • extraction of muscle dimension data for SEE++. The first point was achieved with the help of the DICOMBox component. Behind this component lies a framework written in the programming language Delphi (cf. [Heijlsberg, 1995]). The important methods for the extraction of the needed data were included in a wrapper class written in C++ in the Borland C++ Builder environment and then packed into a Borland C++ Builder component. The component can be extended by adding additional functionality to extract more information out of a DICOM file. The second point in the list above was solved with the help of a test application. In this application the methods for interpolation improvement were tested to evaluate their abilities with respect to an optimal interpolation method for the underlying problem of generating a surface for an EOM model. The result was to choose the NURBS interpolation method because of its high parametrization abilities. This interpolation method was finally implemented. 63 Chapter 8. Conclusion The preparation of the DXF model data was again a main part of the work for this thesis, because of the inconsistency of the received data. Therefore, methods for restructuring the received points (coming from the DXF model) had to be introduced using the evaluation of the variance. A point that appeared to be more difficult than assumed in the first place was the inclusion of the optimized EOM model into VISU. The main drawback was the change of the visualization which made the introduction of different data structures into already existing code necessary. However, after some experimentation the visualization finally appeared to be satisfying. The last point on the list was rather easy to achieve because the dimensions of the EOM model just needed to be extracted to an external file since they were already available in VISU. Figure 8.1 shows the VISU software system in the optimized version. As an example the rec. med. muscle of the right eye is shown. Figure 8.1: Screenshot of the rec. med. muscle and the corresponding MR images in VISU 64 Chapter 8. Conclusion 8.2 Possible Future Improvements Possible improvements in the future will concentrate on redesigning VISU. Requirement that were suggested by users are the extension of the EOM model with respect to the inclusion of all six EOMs and eventually the orbita as well as the bulbus. This would provide a better overview of the anatomical circumstances which means that the interaction of the six EOMs with each other as well as the bulbus and the orbita could be evaluated. Another improvement would be to find a solution for a better differentiation of the EOMs from the surrounding tissue. This was already partially achieved by using MR images that have a higher resolution and smaller spaces between each other. A further point is the calculation of the two oblique EOMs. VISU offers no possibility for the generation of a DXF model for these two muscles because no suitable MR data is available by standard routines. Therefore, other methods for calculating a DXF model out of MR images which were not taken from a coronal or sagital view but from views that have an angle that lies between these two planes have to be found. 8.3 Open Problems The biggest problem is the design of VISU, because it prevents a flexible exchange of functionality. This inflexibility develops throughout all components of VISU and was one of the main problems for the inclusion of the methods for optimization. Therefore a redesign of VISU is strongly recommended. Another problem was the preparation of the DXF data, namely the improving of cross-sections with points from cross-sections that have a better variance. Since some slices received points from cross-sections that were situated before them, the mapping to the MR images could not be 100% exact. Thus some, although very small, differences were present. However, these differences didn’t influence the anatomy of the EOM model in a wide range since there were also other strong differences in the DXF model which still have to be eliminated. At first the inclusion of the optimized EOM model into the existing MR processing component also was afflicted with problems. One was the possibility of choosing the number of processed cross-sections by the user as well as the determination of the spline points for one cross-section. However, this makes only sense for reducing calculational effort, but since today’s computer technology develops in such a fast way, the reduction of calculation time may not play a big role in this part of the program in the future. 65 Literature [ACR-NEMA, 1988] ACR-NEMA (1988). DICOM Version 2.0. ACR-NEMA 300-1988, Digital Imaging and Communications, Engineering Department, National Electrical Manufacturers Association, 2101 L Street, N.W. Suite 300 Washington, D.C. 20037 USA. [Adobe, 2002] Adobe, S. I. (2002). Adobe Photoshop 7.0. http://www.adobe.com. [Andress, 2001] Andress, H. (2001). Glossar zur Datenerhebung und statistischen Analyse (Glossary for Data Gathering and Statistical Analysis; in German). http://www.homes.unibielefeld.de/hjawww/glossar, University of Bielefeld, Germany. [Bézier, 1977] Bézier, P. (1977). Essay de Définition Numérique des Courbes et des Surfaces Expérimentales (Essay on the Numerical Definition of Experimental Curves and Surfaces; in French). PhD thesis, University of Paris VI. [Blausen, 2003] Blausen Medical Communications. (2003). Eye-muscles http://www.blausen.com/BMChom.html, Blausen Medical Communications. and Nerves 3D. [Borland, 2002] Borland Software Corporation. (2002). Borland C++ Builder. http://www.borland.com. [Buchberger and Kaltofen, 2002] Buchberger, M. and Kaltofen, T. (2002). Software Engineering Environment for Knowledge-based Interactive Eye Motility Diagnostics. http://www.see-kid.at, Upper Austrian Research GmbH, Hagenberg, Austria. [Buchberger and Kaltofen, 2003] Buchberger, M. and Kaltofen, T. (2003). Ophtalmologic Diagnostic Tool Using MR Images for Biomechanically Based Muscle Volume Deformation. In M. Sonaka, J. M. F., editor, Proceedings of SPIE Medical Imaging 2003: Image Processing, volume 5032, pages 60–71, SPIE, Bellingham, WA, 2003. [Buchberger et al., 2002] Buchberger, M., Kaltofen, T., Priglinger, S., and Hörantner, R. (2002). Construction and Application of an Object-oriented Computer Model for Simulating Ocular Position Defects. In Proceedings of the International Symposium of the German Ophthalmological Society, Heidelberg, Germany, Department of Ophthalmology, University of Heidelberg, Germany. [Buchberger and Mayr, 2000] Buchberger, M. and Mayr, H. (2000). SEE-KID: Software Engineering Environment for Knowledge-based Interactive Eye Motility Diagnostics. In Proceedings of the International Symposium on Telemedicine, Gothenburg, Sweden, pages 67–79, Department of Medical Software Engineering, Upper Austria University of Applied Sciences, Hagenberg, Austria. [Center for Human Simulation, 2001] Center for Human Simulation, U. o. C. (2001). The Visible Human Project. http://www.nlm.nih.gov, Center for Human Simulation, University of Colorado, Keystone, Colorado. [Claussen, 1997] Claussen, U. (1997). Programmieren mit OpenGL (Programming with OpenGL; in German). Springer Verlag, Berlin, 1nd edition. [Damadian, 1971] Damadian, R. (1971). Tumor detection by nuclear magnetic resonance. Science. [Degen, 2001] Degen, C. (2001). Das Praxisbuch zu AutoCAD 2002 (The Exercise Book for AutoCAD 2002; in German). SmartBooks Publishing AG, 1st edition. [Demer et al., 1997] Demer, J., Miller, J., Poukens, V., Vinters, H., and Glasgow, B. (1997). Evidence for Fibromuscular Pulleys of the Recti Extraocular Muscles. Invest. Ophthalmol. Vis. Sci., 36:1125–1136. [Dössel, 2000] Dössel, O. (2000). Bildgebende Verfahren in der Medizin: Von der Technik zur Medizinischen Anwendung (Imaging Methods in Medicine: From Technics to Medical Application; in German). Springer-Verlag, Berlin Heidelberg, Germany, 2nd edition. 66 Literature [Ellman, 2002] Ellman, T. (2002). Modeling and Animation. Lecture 2 handout winter semester 2002, Computer Science, Vassar College, Poughkeepsie, New York. [Gamma et al., 1995] Gamma, E., Helm, R., Johnson, R., and Vlissides, J. (1995). Design Patterns: Elements of Reusable Object-oriented Software. Addison-Wesley, 2nd edition. [Haslwanter, 2003] Haslwanter, T. (2003). Biophysik und Signalverarbeitung Sensorischer Systeme (Biophysics and Signal Processing of Sensoric Systems; in German). Die Physik Sensorischer Systeme ein Überblick, Lecture notes summer semester 2002, Upper Austria University of Applied Sciences, Hagenberg, Austria. [Hearn and Baker, 1997] Hearn, D. and Baker, M. P. (1997). Computer Graphics - C Version. Prentice Hall, USA, 2nd edition. Section 10-8. [Heijlsberg, 1995] Heijlsberg, A. (1995). Delphi. http://www.borland.com/delphi/. [Hoehne et al., 2003] Hoehne, K., Pflesser, B., Pemmert, A., et al. (2003). The Visible Human Project. DVD-ROM for Windows 95/98/NT/2000/ME/XP, English and German, Institute of mathematics and computer science in medicine, Universitätsklinikum Hamburg-Eppendorf, Germany. [Kaltofen, 2002] Kaltofen, T. (2002). Design and Implementation of a Mathematical Pulley Model for Biomechanical Eye Surgery. Master’s Thesis, Upper Austria University of Applied Sciences, Hagenberg, Austria. [Kaufmann, 1995] Kaufmann, H. (1995). Strabismus (in German). Enke Verlag, Stuttgart, Germany, 2nd edition. [King, 2000] King, M. M. (2000). Module nr. 2: Basic Principles of MRI. http://www.erads.com. [Krewson, 1950] Krewson, E. (1950). The Action of the Extraocular Muscles. Transactions of the American Ophthalmological Society, 48:443–486. [Krug, 2000] Krug, W. (2000). ezDICOM Software. http://www.psychology.nottingham.ac.uk/staff/cr1/ ezdicom.html. [Lacher, 2001] Lacher, M. (2001). A Volume Preserving Model for Biomechanically-based Muscle Volume Deformation. Master’s Thesis, Upper Austria University of Applied Sciences, Hagenberg, Austria. [Miller, 1989] Miller, J. (1989). Functional Anatomy of Normal Human Rectus Muscles. Vision Res, 29(2):223–240. [Miller and Demer, 1995] Miller, J. and Demer, J. (1995). Magnetic Resonance Imaging of the Functional Anatomy of the Superior Oblique Muscle. Invest Ophthalmol Vis Sci, 36(5):906–13. [Miller et al., 2003] Miller, J., Demer, J., Poukens, V., Pavlovski, D., Nguyen, H., and Rossi, E. (2003). Extraocular Connective Tissue Architecture. Vision Res, 3(3):240–251. [Miller and Robinson, 1984] Miller, J. and Robinson, D. (1984). A Model of the Mechanics of Binocular Alignment. Comput Biomed Res, 17(5):436–470. [Miller and Shamaeva, 1993] Miller, J. and Shamaeva, I. (1993). Orbit 1.0 Gaze Mechanics Simulation User’s Manual. Eidactics, Suite 404; 1450 Greenwich Street; San Francisco, CA 94109; USA. [Miller and Demer, 1999] Miller, J. M. and Demer, J. L. (1999). Clinical Applications of Computer Models for Strabismus. In Rosenbaum, A. L. and Santiago, A. P., editors, Clinical Strabismus Management: Principles and Surgical Techniques. W. B. Saunders, Philadelphia, USA. [Morneburg, 1995] Morneburg, H. (1995). Bildgebende Systeme für die Medizinische Diagnostik (Imaging Systems for Medical Diagnostics; in German). Publicis-MCDVerlag, Munich, Germany, 3rd edition. [Piegl and Tiller, 1980] Piegl, L. and Tiller, W. (1980). The NURBS Book. Springer, Germany, 2nd edition. [Pirklbauer, 2001] Pirklbauer, F. (2001). 3D-Rekonstruktion Extraokulärer Augenmuskeln aus MRBilddaten (3D-Reconstruction of Extraocular Eye Muscles; in German). Master’s thesis, Upper Austria University of Applied Sciences, Hagenberg, Austria. [Prata, 1998] Prata, S. (1998). C++ Primer Plus. The Waite Group’s, 3nd edition. [Press et al., 2002] Press, W., Vetterling, W., Teukolsky, S., and Flannery, B. (2002). Numerical Recipes in C++. Cambridge University Press, UK, 2nd edition. [Robinson, 1975] Robinson, D. (1975). A Quantitative Analysis of Extraocular Muscle Cooperation and Squint. Investigative Ophthalmology and Visual Science, 14(11):801–825. 67 Literature [Simonsz et al., 2002] Simonsz, H., Spekreijse, H., van der Helm, F., Willekens, B., van Keulen, A., van den Schutte, S., and Bedem, S. (2002). The Development of a Finite-element Model for Orbital Mechanics. In Proceedings of the International Symposium of the German Ophthalmological Society, Department of Ophthalmology, University of Heidelberg, Germany. [Sobotta, 1997] Sobotta (1997). Atlas der Anatomie des Menschen (Atlas of the Human Anatomy; in German). Urban und Schwarzenberg Verlag, Munich, Germany, 20th edition. [Thömke, 2001] Thömke, F. (2001). Augenbewegungsstörungen: Ein klinischer Leitfaden (Interferences of the Eyes: A Clinical Guideline; in German). Georg Thieme Verlag, Stuttgart, Germany, 2nd edition. 68