Surface Optimization of a Three-Dimensional Eye Muscle - SEE-KID

Transcription

Fachhochschul-Diplomstudiengang
SOFTWARE ENGINEERING FÜR MEDIZIN
A-4232 Hagenberg, Austria
Surface Optimization of a Three-Dimensional Eye
Muscle Model
Diplomarbeit
zur Erlangung des akademischen Grades
Diplom-Ingenieur (Fachhochschule)
Eingereicht von
Stefan Satzinger
Betreuer:
Begutachter:
Dipl.-Ing. (FH) Thomas Kaltofen, UAR, Hagenberg
FH-Prof. Dipl.-Ing. Dr. Herwig Mayr
August 2003
Eidesstattliche Erklärung
Ich erkläre, dass ich die Diplomarbeit selbstständig verfasst, andere als die angegebenen Quellen und Hilfsmittel nicht verwendet und mich auch sonst keiner unerlaubten
Hilfe bedient habe.
Hagenberg, August 2003
Stefan Satzinger
Abstract
Abstract
This diploma thesis describes the development of optimization methods for a threedimensional extraocular eye muscle (EOM) model. All created methods were implemented in a software system called VISU, which is part of the SEE-KID research
project that was initiated at the Upper Austria University of Applied Sciences in
Hagenberg and is being continued at the UAR (Upper Austrian Research) company.
The aim of this thesis is to improve a current three-dimensional EOM model generated from MR images with respect to the surface representation of the muscle for
the purpose of simulating the movement of one EOM and to provide data about
the dimensions of the eye muscles for use in an additional software system called
SEE++, which is also part of the SEE-KID research project.
In order to understand the anatomy of the human eye, a short description of the
EOMs and the bulbus is given. Additionally, information about different models of
the EOMs, including a comparison of the models with each other, is presented.
The thesis describes the extension of the software system VISU by introducing a set
of optimization methods customized for the properties of VISU. Thereby the main
work for improvement of the three-dimensional EOM surface is accomplished by a
spline interpolation method. After introducing and comparing different methods
for spline interpolation, the NURBS interpolation method was used to improve the
surface of the EOM model. The algorithms for the NURBS interpolation were
included in VISU, which was extended to provide not only the three-dimensional
EOM model as output but also data of the dimensions of an EOM.
All the performed improvements were validated regarding their anatomical correctness and their compliance to the MR images which were the initial starting point
for the generation of the EOM model. At the end of this thesis, an outlook to future
improvements of VISU is given.
iii
Kurzfassung
Kurzfassung
Diese Diplomarbeit beschreibt die Entwicklung von Methoden für die Optimierung
eines Modells der drei-dimensionalen extraokularen Augenmuskeln (EOM). Alle
dabei entwickelten Methoden wurden in einem Softwaresystem mit dem Namen
VISU implementiert, welches Teil des Forschungsprojektes SEE-KID ist. SEE-KID
wurde an der Fachhochschule Hagenberg durchgeführt und wird von der Firma
UAR (Upper Austrian Research) fortgesetzt. Es ist das Ziel dieser Diplomarbeit
ein derzeitiges drei-dimensionales EOM Modell, welches aus MR Daten generiert
wurde, hinsichtlich der Darstellung der Muskeloberfläche für die Aufgabe der Simulation eines EOMs zu optimieren. Zusätzlich sollen Dimensionsangaben der Augenmuskeln für den Gebrauch in einem weiteren Softwaresystem mit dem Namen
SEE++, welches ebenfalls Teil des SEE-KID Forschungsprojektes ist, bereitgestellt
werden.
Um die Anatomie des menschlichen Auges zu verstehen, wird eine Beschreibung
der EOMs sowie des Bulbus präsentiert. Zusätzlich wird Information über die unterschiedlichen Modelle der EOMs sowie ein Vergleich der Modelle untereinander
präsentiert.
Diese Diplomarbeit beschreibt die Erweiterung des Softwaresystems VISU durch die
Einführung einer Reihe von Optimierungsmethoden, welche auf die Eigenschaften
von VISU abgestimmt sind. Dabei wird die Hauptarbeit für die Optimierung der
drei-dimensionalen EOM-Oberfläche durch eine Spline-Interpolationsmethode verrichtet. Nach dem Vergleich unterschiedlicher Methoden der Spline-Interpolation
wurde die NURBS-Interpolationsmethode eingesetzt um die Oberfläche des EOMModells zu optimieren. Die Algorithmen für die NURBS-Interpolation wurden in
VISU inkludiert, welches nicht nur erweitert wurde, um das drei-dimensionale EOMModell als Ausgabe zu liefern, sondern auch um Dimensionsangaben über die EOMs
zu erhalten.
Alle ausgeführten Verbesserungen wurden bezüglich ihrer Richtigkeit und Übereinstimmung mit den MR-Daten, welche den Ausgangspunkt für die Erzeugung des
EOM-Modells darstellten, validiert. Am Schluss wird ein Ausblick auf zukünftige
Verbesserungen von VISU gegeben.
iv
v
Acknowledgements
Acknowledgements
I would like to put my “thank you” into words to all the people who made this
work possible. First of all I would like to thank my girlfriend Gudrun for giving me
the support I needed for finishing the four years at the Upper Austria University
of Applied Sciences in Hagenberg. The next two people are my two colleagues
at the UAR department for medical informatics, Michael Buchberger and Thomas
Kaltofen. Michael helped me a lot with his advice, especially in times when I got
stuck with my work he gave me the hints that pushed me back on the right way for
finishing the thesis. Thomas, who was the supervisor for my thesis work at the UAR,
always gave me best advice for improvements and corrections during my work. Of
course I would also like to thank the other three colleagues at the UAR, especially
Doris Siegl, for their help and assistance during the time at the company. A special
“thank you” also goes to Gregory Curtis, my english teacher at the university, who
was responsible for improving my english knowledge.
A special thanks also goes to Herwig Mayr who took the task of correcting and
grading my thesis and who gave me the chance to continue the work on the VISU
project. Last but not least I would like to thank my family for the help they gave me
during my studies in Hagenberg and all my friends from the department for Medical
Software Engineering as well as all my other friends for whose names to mention
there isn’t enough place on this page.
Stefan Satzinger
Contents
1 Optimizing Eye Muscle Models
1.1 Motivation . . . . . . . . . .
1.2 The SEE-KID Project . . . .
1.2.1 Structure . . . . . . .
1.2.2 Current Status . . . .
1.3 The VISU Project . . . . . .
1.3.1 Structure of VISU . .
1.3.2 Current Status . . . .
1.4 Deficits of Current Models . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
2 Basic Notions
2.1 The DICOM Standard . . . . . . . .
2.2 Magnetic Resonance Imaging . . . .
2.2.1 History . . . . . . . . . . . . .
2.2.2 Magnets . . . . . . . . . . . .
2.2.3 Magnetic Influence on Atoms
2.2.4 MR Images . . . . . . . . . .
2.3 Mathematics . . . . . . . . . . . . . .
2.3.1 Analysis of Variance . . . . .
2.3.2 Algorithms for Interpolation
2.4 Medical Terms . . . . . . . . . . . .
2.4.1 Anatomy of the Human Eye
2.4.2 Rectus Muscle Pulleys . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
3 Muscle Model Generation
3.1 Existing Approaches . . . . . . . . . . .
3.2 Limitations of Existing Approaches . . .
3.3 Possible Solutions for Optimization . . .
3.4 Generation of Muscle Data for Kinematic
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
. . . . .
. . . . .
. . . . .
Models
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
1
1
2
3
3
4
5
5
6
.
.
.
.
.
.
.
.
.
.
.
.
9
9
11
11
12
12
13
13
14
14
17
17
19
.
.
.
.
20
20
21
23
27
4 Design of a Muscle Visualization System
29
4.1 Existing System Design . . . . . . . . . . . . . . . . . . . . . . . . . 29
vi
vii
Contents
4.2
4.3
Extension of the Existing Structure .
4.2.1 Picture Processing . . . . . .
4.2.2 Gravity Line Optimization .
4.2.3 Analysis of Variance . . . . .
4.2.4 Surface Optimization . . . .
Modifications of the VISU Structure
.
.
.
.
.
.
5 Implementation
5.1 Muscle Gravity Line Optimization . .
5.2 Applying Analysis of Variance . . . .
5.3 Interpolation Using Splines . . . . . .
5.4 Muscle Data Generation . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
32
32
33
35
36
37
.
.
.
.
40
40
41
42
47
6 Validation
50
6.1 Comparing Reconstructed Volumes with MR Images . . . . . . . . . 50
6.2 Comparison with Existing Models . . . . . . . . . . . . . . . . . . . 52
6.3 Extraction of Muscle Dimension Data . . . . . . . . . . . . . . . . . 56
7 Software Development Aspects
61
7.1 Process View . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
7.2 Product View . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
8 Conclusion
63
8.1 Goals Achieved . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
8.2 Possible Future Improvements . . . . . . . . . . . . . . . . . . . . . 65
8.3 Open Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
Literature
66
List of Figures
1.1
1.2
1.3
1.4
1.5
1.6
SEE++ screenshot [Buchberger and Kaltofen, 2002]
VISU structure . . . . . . . . . . . . . . . . . . . .
EOM model [Miller, 1989] . . . . . . . . . . . . . .
EOM model [Miller and Shamaeva, 1993] . . . . . .
EOM model [Miller et al., 2003] . . . . . . . . . . .
EOM model [Lacher, 2001] . . . . . . . . . . . . . .
2.1
2.2
2.3
2.4
2.5
2.6
2.7
2.8
DICOM scope [ACR-NEMA, 1988] . .
DICOM file format . . . . . . . . . . .
RF pulse . . . . . . . . . . . . . . . . .
Flipped RF pulse . . . . . . . . . . . .
Body planes [Haslwanter, 2003] . . . .
Bézier curves [Ellman, 2002] . . . . . .
Right eye (top view) [Sobotta, 1997] .
Differences between conventional model
[Miller and Demer, 1999] . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
4
6
7
7
7
8
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
and pulley model
. . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
10
11
12
12
13
16
17
. . . . . . . 19
3.1
3.2
3.3
3.4
3.5
3.6
3.7
Blausen EOM model [Blausen, 2003] . . . . . . . . . . .
EOM model with errors on the surface . . . . . . . . . .
Set of points to be interpolated . . . . . . . . . . . . . .
Third degree interpolated curve . . . . . . . . . . . . . .
Control polygon of the points . . . . . . . . . . . . . . .
Bézier spline interpolation applied to the points of Figure
NURBS interpolation applied to the points of Figure 3.3
4.1
4.2
4.3
4.4
Components of VISU . . . . . . . . . . . . . . . . . . . . . . . . . .
Class diagram of the administration component . . . . . . . . . . .
Data flow of the creation of a DXF model in VISU . . . . . . . . .
Data flow of the creation of the EOM model in VISU out of the DXF
file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Class diagram of the MR processing component [Lacher, 2001] . . .
DXF muscle model with correct length calculation . . . . . . . . . .
Partition of variance and amount of slices . . . . . . . . . . . . . .
4.5
4.6
4.7
viii
. .
. .
. .
. .
. .
3.3
. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
22
24
26
26
26
27
27
. 30
. 30
. 31
.
.
.
.
31
32
34
36
List of Figures
ix
4.8
4.9
Cross-section of DXF model with both halves interpolated . . . . . . 37
Expanded class diagram of the VISU structure . . . . . . . . . . . . . 39
5.1
Area centroid of a cross-section and the angle a that the point spans
regarding the area centroid . . . . . . . . . . . . . . . . . . . . . . .
NURBS curve with two equal control points P2 = P3 . . . . . . . .
Linear interpolation of the NURBS-generated cross-section . . . . .
SEE++ screenshot showing the EOM built of muscle data from initialization files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. 49
6.1
6.2
6.3
6.4
6.5
6.6
6.7
6.8
6.9
6.10
6.11
6.12
6.13
6.14
6.15
6.16
6.17
6.18
6.19
6.20
6.21
6.22
6.23
6.24
6.25
EOM model without overlapping MR images . . . . . . . . . .
EOM model with overlapping MR images . . . . . . . . . . . .
DXF EOM model with overlapped MR images . . . . . . . . .
Dynamic EOM model with overlapped MR images . . . . . . .
DXF EOM model with overlapping MR images, magnified . .
Dynamic EOM model with overlapping MR images, magnified
DXF EOM model with overlapping MR images . . . . . . . .
Dynamic EOM model with overlapping MR images . . . . . .
DXF EOM model with overlapping MR images . . . . . . . .
Dynamic EOM model with overlapping MR images . . . . . .
Dynamic EOM model, non-optimized version . . . . . . . . . .
Dynamic EOM model, optimized version . . . . . . . . . . . .
EOM model with an innervation of 0 . . . . . . . . . . . . . .
Optimized EOM model with an innervation of 0 . . . . . . . .
EOM model with an innervation of 50 . . . . . . . . . . . . .
Optimized EOM model with an innervation of 50 . . . . . . .
EOM model with an innervation of 100 . . . . . . . . . . . . .
Optimized EOM model with an innervation of 100 . . . . . . .
Dynamic EOM model, non-optimized version . . . . . . . . . .
Dynamic EOM model, optimized version . . . . . . . . . . . .
Screenshot of the selected muscle in the MR images . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
8.1
Screenshot of the rec. med. muscle and the corresponding MR images
in VISU . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
5.2
5.3
5.4
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
. 42
. 43
. 47
51
51
53
53
53
53
54
54
54
54
55
55
55
55
56
56
57
57
57
57
57
57
58
58
59
List of Tables
2.1
Primary, secondary, and tertiary functions of the six EOMs [Thömke,
2001] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
3.1
Overview on current EOM models . . . . . . . . . . . . . . . . . . . . 24
6.1
Comparison of muscle dimension data in millimeters, measured form
the MR images and extracted from VISU . . . . . . . . . . . . . . . . 60
x
Chapter 1
Optimizing Eye Muscle Models
In modern medicine, methods for visualizing different parts of the human body have
become state of the art. Technologies like magnetic resonance (MR) imaging or computed tomography (CT) build the basis for a deeper insight into the human body
and offer the chance for getting a whole new understanding of how humans work.
Thus, especially in the field of image processing, a lot of research is done to get the
most out of imaging data. Today not only the processing of images taken from MR
imaging or CT devices is carried out but also the three-dimensional anatomically
correct reconstruction of certain parts of the human body. However, as new technologies find their way into the field of medicine, new methods for visualization also
have to be discovered and developed to a status where doctors can use them in order
to take better advantage of the results.
1.1
Motivation
The movement and the spatial location of the six extraocular eye muscles (extraocular eye muscle = EOM) is of great importance to physicians, especially ophthalmologists who need information on certain muscles in order to plan surgeries or
to diagnose defects of the oculomuscular apparatus. Thus, a system, preferably a
software system, would be a good solution to plan surgeries and to simulate the
movement of the different EOMs.
Although the simulation of the movement of the EOMs is a rather new research
area, some models already exist, differing in complexity, functionality and accuracy.
Unfortunately none of these models is perfect. The motivation for this thesis work is
to improve a current three-dimensional EOM model generated from MR images with
respect to the surface representation of the muscle for the purpose of simulating the
movement of one EOM. To manage this, different mathematical methods are applied
in an underlying model of an EOM which has been realized in a software system
called VISU (VIrtual SUgery). In addition, all of the new insights have to be added
to this software system. The result should be a dynamic three-dimensional EOM
model with improved muscle surface, as well as information about the generated
1
Chapter 1. Optimizing Eye Muscle Models
muscle for the usage in another software system called SEE++ (Simulation Expert
for Eyes + Diagnosis + Transposition Surgery). The VISU as well as the SEE++
software are the results of the research project—carried out at the Department of
Medical Software Engineering at the Upper Austria University of Applied Sciences
in Hagenberg—SEE-KID (Software Engineering Environment for Knowledge-based
Interactive Eye Motility Diagnostics) [Buchberger and Mayr, 2000]. The research
project SEE-KID as well as the two software systems VISU and SEE++ are continued by the UAR (Upper Austrian Research) company which is a partner of the
university.
The intended audience for this thesis consists of physicians and surgeons who are
interested in a software system for simulating EOM movement as well as technicians
who want to learn more about the technical aspects behind the system concerning
implementation details or the algorithms used.
1.2
The SEE-KID Project
Defects of the EOMs can develop into a pathological problem if they are not recognized and corrected already at an early state of human growth. This is already
carried out at hospitals where the eye departments have specialized in the correction
of certain pathologies, especially strabismus. Thus, the defects can be corrected by
changing the position of at least one or several EOMs through surgery. The problem
with these methods is that there is no possibility to simulate the surgery. As a result,
the success of such surgery depends on the experience of the surgeon as well as on
already documented surgeries originating from similar pathologies. Thus, a satisfying result might only be achieved by carrying out numerous surgeries, resulting in a
higher burden on the patient.
Enhancements were provided by the SEE++ software system which is one of the
results of the SEE-KID project. The goal of the project is to build a biomechanical,
interactive 3D simulation of the human eye, its globe and its muscles, to simulate
common EOM surgeries (transposition, resection, splitting, etc.) in a graphical interactive way that is familiar to an experienced surgeon. Additionally, the system
can be used for educational and training purposes, because through extensive possibilities for parameterizing the biomechanical model, common pathological cases of
human eye motility disorders can be modeled and subsequently simulated [Buchberger et al., 2002].
As mentioned before, the main goal of the research project SEE-KID is the realization of a biomechanical model of the human eye. A mainly mathematical description
of the oculomuscular apparatus forms the basis for a system which should give proper
predictions on simulation and treatment of pathological cases [Buchberger and Mayr,
2000].
2
1.2.1
Structure
The core of the SEE++ software system is the biomechanical model which consists
of three different parts [Kaltofen, 2002]:
• geometrical model,
• muscle force model,
• kinematic model.
The model characterizes the movement of the eye and its EOMs, with respect to
the biological structures, according to the mechanical behavior of the eye and its
muscles.
Geometric Model
The biomechanical model in SEE++ supports different geometrical models, whereby
three of them are currently implemented, namely the string model [Krewson, 1950],
the tape model [Robinson, 1975] and the pulley model [Demer et al., 1997]. The
geometrical model describes the geometry of the human eye with its EOMs.
Muscle Force Model
The muscle force model is used to describe the muscle forces and the influence that
these forces have on other parts of the overall model.
Kinematic Model
The kinematic model is responsable for combining the results of the geometrical
model and the muscle force model.
Of course there are also other parts of the SEE++ software which are important for
the functionality. This is for example the graphical user interface (GUI), which is
responsible for the visualization of the results as well as for the interaction with the
user.
1.2.2
Current Status
The user can navigate through the program in a tree view where he has quick access
to the most important features of the program, hence he can manage different scenarios. Interactive surgeries with the EOMs can be simulated, e.g. by repositioning
the EOMs on the globe which is displayed in three-dimensional view together with
the six EOMs. Another feature are the different diagrams supported by SEE++:
• the muscle force distribution diagram, which represents the current gaze position through a vertical line,
• the muscle force vector diagram, which represents the current gaze position
through a small cross,
3
• the Hess diagram, which also represents the current gaze position through a
small cross. It depends on gaze patterns, which can be changed in the gaze
pattern dialog.
Figure 1.1 shows a screenshot of the SEE++ software system.
Figure 1.1: SEE++ screenshot [Buchberger and Kaltofen, 2002]
1.3
The VISU Project
The VISU project had its origin as a student project at the Upper Austria University of Applied Sciences in Hagenberg in the year 2001. The goal was to provide the
physician with a software system that could generate a three-dimensional eye muscle
model out of a series of MR images taken from a patient, as well as to store information about the patient, who was the subject of an examination. With the VISU
software system, the physician, especially the ophthalmologist, is able to analyze
changes of lengths and volume shifts of an EOM.
4
1.3.1
Structure of VISU
The system’s main parts consist of the following:
• GUI,
• patient data administration component,
• MR analysis component,
• MR processing component.
The MR analysis and the MR processing component are the two most important parts of VISU, because they are responsible for the generation of the threedimensional muscle model.
GUI
The GUI is the part of the system which is responsible for the interaction with the
user as well as for displaying the results.
Administration Component
All relevant information about the patient like name, address, date of birth etc., is
managed in the administration component of the program.
MR Analysis Component
The MR analysis component uses coronal slices of MR images of a patient as input
data and builds a three-dimensional static model of one EOM [Pirklbauer, 2001]
using the marching cubes algorithm to reconstruct a muscle. The muscle is then
stored in the DXF format (Drawing eXchange Format) [Degen, 2001] for further
processing.
MR Processing Component
The MR processing component calculates a three-dimensional dynamic eye muscle
model out of two DXF models coming from the previous component. It uses DXF
models generated from data of one muscle in two different gaze positions: one where
the muscle is in a primary position (cf. Chapter 2.4.1) and one where the muscle
is in a secondary position. The result is a three-dimensional model which can be
transformed interactively from one innervation to another [Buchberger and Kaltofen,
2003].
1.3.2
Current Status
The main input for the VISU software system consists of MR images. Thus, the
muscles, from which the three-dimensional model is generated, have to be visible
in the images. The better the visibility of the muscles is, the better the resulting
muscle model will be. The user can store different picture series containing MR
data of one patient. The following model generation is split into two steps, the
5
generation of two or more static muscle models and the generation of a dynamic
muscle model. All necessary information is stored in a data file so that different
views on already generated models can be visualized. Figure 1.2 shows the procedure
for generating a three-dimensional muscle model, starting with the MR images of a
patient’s eye followed by the generation of a static muscle model using the marching
cubes algorithm and resulting in a three-dimensional dynamic eye muscle model.
Figure 1.2: VISU structure
1.4
Deficits of Current Models
Since the creation of one of the first computer models for the simulation of eye
movements (SQUINT, [Miller and Robinson, 1984]), three-dimensional visualization
of the EOMs has always been a main feature in every software tool handling the
topic of simulation of eye movements. One of the first EOM models was generated by
Joel M. Miller (cf. Figure 1.3) in 1989. For this model MR images of some patients
were used and for each set of image data the mean muscle position was calculated.
Out of this information a three-dimensional EOM model was reconstructed. Later,
a three-dimensional EOM model was included in the software tool Orbit [Miller
and Shamaeva, 1993]. However, this model was not a very realistic one; instead, it
showed EOMs in a three-dimensional but universal way, though it was possible to
change the shape of the model (cf. Figure 1.4).
Another EOM model was also developed by Joel M. Miller [Miller et al., 2003]; it
focuses on the visualization of the tissue surrounding the EOMs, called EOM pulleys
(cf. Chapter 2.4.2). In the accompanying study, MR data coming from patients and
6
7
Figure 1.3: EOM model [Miller, 1989]
serially sectioned human cadavers was combined to calculate the muscle volumes (cf.
Figure 1.5). All three models developed by Joel M. Miller have the disadvantage
of not being modifiable in an anatomically correct way. This deficit was improved
by [Lacher, 2001], who managed to create a modifiable three-dimensional EOM
model using the data that resulted out of [Pirklbauer, 2001]. The model was fully
dynamic and anatomically correct; it showed for the fist time a human EOM that
could be watched in 3D and that could change its innervation within a certain range;
however, only one of the EOM could be watched at the same time (cf. Figure 1.6).
Figure 1.4: EOM model [Miller and
Shamaeva, 1993]
Figure 1.5: EOM model [Miller et al.,
2003]
Of course there exist other EOM models like the one created by [Simonsz et al.,
2002]; it is based on the finite element method, or models made from various artists.
The voxel-man project (cf. [Hoehne et al., 2003]) is also a good example. But most of
these models have the disadvantage of not being anatomically correct, which means
Figure 1.6: EOM model [Lacher, 2001]
that they are not able to reflect the EOM anatomy of one specific person. It is also
not possible to change the EOMs interactively, which means to look at an EOM
at different states of innervation. A closer look at the comparison of the different
models will be given in Chapter 3.
8
Chapter 2
Basic Notions
This chapter provides a short overview of the basic notions required. First the
DICOM standard will be described, since this is one of the most frequently used
protocols for data exchange in medicine. Chapter 2.2 provides the reader with
fundamental knowledge on how magnetic resonance imaging works and what it is
used for. In Chapter 2.3 the mathematics used in this thesis work is presented in
a way that provides information on the different mathematical terms and methods
and how they are applied. Finally the last section in Chapter 2 covers different
medical terms regarding the human eye.
2.1
The DICOM Standard
After the introduction of computed tomography (CT) in 1973, several other diagnostic imaging devices followed very quickly. This was the reason for the American
College of Radiology (ACR) and the National Electrical Manufacturers Association
(NEMA) to establish a standard for transferring images and associated information
between devices manufactured by manifold vendors. The joint work of the ACR and
the NEMA resulted in the DICOM standard (Digital Imaging and Communications
in Medicine) [ACR-NEMA, 1988].
The DICOM standard ensures compatibility of medical imaging equipment by specifying:
• a set of protocols to be followed by devices claiming conformance to the standard,
• syntax and semantics of commands and associated information which can be
exchanged using these protocols,
• information that must be supplied with an implementation for which conformance to the standard is claimed.
Figure 2.1 shows the scope of the DICOM format.
9
Chapter 2. Basic Notions
10
Medical Informatics
. . .
Patient
Bedside
Monitoring
Administrative
HIS/RIS
Lab Data
. . .
Diagnostic
Imaging
Scope of
DICOM
Figure 2.1: DICOM scope [ACR-NEMA, 1988]
However, there are lots of ways to extend the standard, which makes it difficult
for DICOM software producers to implement DICOM viewers or readers that have
the capability to read and display data from different DICOM files obtained from
various imaging equipment.
Structure of the DICOM File Format
A DICOM file includes a data field as well as a header, where information about
the patient, like the patient’s name, date of birth or the date when the study was
conducted, is stored. The image information is stored in the data field which contains
information in three dimensions.
In the sample DICOM file shown in Figure 2.2, the header at the beginning shows
information about the patient as well as information involving the modality which
was used to generate the DICOM file. MR modalities, for example, store different
technical information about the analysis on a patient than CT modalities do. The
size of the header varies depending on how much information is stored in it.
11
Figure 2.2: DICOM file format
2.2
Magnetic Resonance Imaging
Magnetic resonance (MR) imaging is an imaging technique used primarily in medical
fields to produce images of the inside of the human body. MR imaging is based on
the magnetic properties of hydrogen protons which align themselves in a magnetic
field; the alignment is greatly influenced by the strength of the external magnetic
field and the thermal energy of the atoms.
2.2.1
History
The magnetic resonance phenomenon of hydrogen protons was discovered by Felix
Bloch and Edward Purcell independently in the year 1946. In 1952 both received
the Nobel Prize for their discovery. Unfortunately, the usage in the medical field
was not yet possible at that time. Then, in 1971, Raymond Damadian showed that
the relaxation time of tissue and tumors differed. This was the breakthrough that
allowed scientists to do research in the medical area in order to use it for diagnostic
applications (cf. [Damadian, 1971]).
2.2.2
12
Magnets
Magnetism is an important part of MR imaging and requires field strengths from 0.4
up to 2.0 Tesla . Today’s devices, in most cases, use super conductive magnets; these
consist of coils of wire which carry stable electric voltage. By cooling the coils with
cryogenics (helium for example) it is possible to generate very strong magnetic fields
because a higher amount of electric current can flow through the wire. Although
super conductive magnets are currently the norm, there are two other possibilities,
namely permanent magnets and resistive magnets, whereby both aren’t used very
often because they only produce small magnetic fields. For example permanent
magnets operate only up to 0.4 Tesla.
2.2.3
Magnetic Influence on Atoms
Figure 2.3: RF pulse
Figure 2.4: Flipped RF pulse
Atoms consist of positively charged protons and neutrally charged neutrons, both
form the so-called nucleus. Around the nucleus there is a cloud of negatively charged
electrons which keep the atom in balance. Since more than 70% of the human body
consists of water (H2 O), hydrogen is the fundamental element in MR imaging and
thus all of the following considerations will focus on the hydrogen atom.
Images of the inside of the human body are generated by exposing it to an external
magnetic field B0 . This causes all the protons inside our body, which have their own
magnetic field M0 , to align themselves in the direction of B0 (cf. Figure 2.3 [King,
2000]). By applying energy in the form of a radio frequency (RF) signal at 90◦ to
the direction of B0 the direction of the magnetic field of the protons can be changed,
with the result that all the protons flip their angle to the new layer xy (Figures 2.3
and 2.4 [King, 2000]). Switching off the RF signal causes the protons to realign
with B0 . During this cycle signals can be measured by receiver coils inside the
MR imaging device. An exact type of tissue can be assigned to each of the signals
because signals coming from diverse tissues inside the human body have different
properties called T1 and T2 relaxation times. Fat, for example, has a very efficient
energy exchange and therefore has a relatively short T2. Water is less efficient than
fat in the exchange of energy, and therefore has a long T2. All these calculations
are carried out by the MR imaging device and are transformed into images that can
be processed further by other devices or that can just be viewed on a screen.
2.2.4
MR Images
Each image is the result of data which was produced from signals that were the
measured effect of an external magnetic field applied to the human body by the MR
imaging device. One image does not just show a two-dimensional slice of the body;
instead, it shows a whole volume in three dimensions, because the above-mentioned
signals always emerge out of a slice which has a certain thickness. Images of the
body can be taken in three different directions (cf. Figure 2.5) :
Figure 2.5: Body planes [Haslwanter, 2003]
• horizontal: all planes that develop parallel to the horizontal plane,
• coronal: all layers that develop parallel to the plane going through the forehead,
• sagittal: all planes that develop parallel to the nose to head plane.
To get more information about MR imaging, refer to [Dössel, 2000] and [Morneburg,
1995].
2.3
Mathematics
While many mathematical techniques were used during the development of this
thesis, only two of them leave their traces throughout the whole thesis work, namely
the analysis of variance and algorithms for interpolation. Both of them are discussed
in the following two chapters.
13
2.3.1
14
Analysis of Variance
The basic principle is the comparison of the variation between groups of data with
the variation in the groups. Data groups consist of measured values which have to
be put into relation. An approach for the calculation is to determine the arithmetic
mean of each group of values X0 , X1 , X2 , X3 , . . . , Xn and use it as the so-called
target value Y . If the variation between the groups is bigger than the variation in
the groups, then differences between the values (in the groups) can be supposed.
First of all the variation s2 has to be calculated [Andress, 2001]:
n
P
s2 =
(xi − x̄)2
i=1
=
n−1
SAQx
,
n−1
(2.1)
where
n = number of values per group,
i = count index for the values (i = 1, . . . , n),
xi = value of the variable X at the index i,
x̄ = arithmetic mean of X,
SAQx = sum of the squared deviation of X.
1
controls the value SAQx , because the more values Xi there are in one group
n−1
of data, the higher the value for the variation gets. The result can be described as
the “average squared deviation”. The standard deviation s
s=
√
s2
(2.2)
can be calculated in order to interpret quantified spreading of a value in the groups.
A more descriptive value is the variation coefficient v
v=
s
∗ 100,
x̄
(2.3)
because it brings the standard deviation in connection with the arithmetic mean,
hence comparison of the groups is easier.
2.3.2
Algorithms for Interpolation
We sometimes know the value of a function f (x) at a set of points x1 , x2 , x3 , . . . , xN
where x1 < x2 . . . xN , but we don’t have the analytic expression for f (x) to calculate
its value at an arbitrary point. The challenge is now to somehow approximate f (x)
for arbitrary points by drawing a smooth curve through xi . If the desired x is
between the largest and the smallest xi , the problem is called interpolation, if it
is outside, it’s called extrapolation. In the following, three common methods for
interpolation, which are also suitable for extrapolation, will be presented.
15
Polynomial Interpolation
This interpolation method uses a polynomial [Press et al., 2002],
p(x) = a0 + a1 x + a2 x2 + · · · + an xn
(2.4)
to approximate f (x) for arbitrary x. The only problem is how to find the coefficients
(base points) a0 , a1 , a2 , . . . , an that determine the behavior of the polynomial. One
approach is to solve the linear equation system. But this leads to a very complex
structure of equations if there is a high number of base points. A better way would
be to use the interpolation polynomial according to Newton.
pn (x) = k0 +k1 (x−x0 )+k2 (x−x0 )(x−x1 )+· · ·+kn (x−x0 )(x−x1 ) . . . (x−xn−1 ) (2.5)
When doing so, the polynomial is built up piecemeal, thus offering more flexibility
so that new base points can be added without changing the equation system. An
algorithm for the polynomial interpolation can be looked up e.g., in [Press et al.,
2002]
Bézier Splines
Using interpolation polynomials has the disadvantage of having very big oscillations
between the base points, with the effect that further processing is harder to accomplish. These problems can be prevented using spline interpolation, where a function
C(u) is built between the base points x1 , x2 , x3 , . . . , xN , the so-called spline knots,
which only has minimal oscillations between these points.
C(u) =
n
X
Bi,n (u)Pi
(2.6)
i=0
Equation 2.6 characterizes the Bézier splines, where Bi,n (u) describes the blending
functions and Pi are the given control points. The blending functions are the basis
functions of the Bézier curve (cf. Figure 2.6 which shows the blending functions for
cubic Bézier curves1 ).
Bi,n (u) =
n!
ui (1 − u)n−i
i!(n − i)!
(2.7)
The algorithm for the calculation of the desired Bézier curve is split into two parts.
The first part calculates the blending functions and the second one calculates an
arbitrary point on the resulting Bézier curve. For the complete algorithm, refer
to [Hearn and Baker, 1997].
1 The
amount of control points determines the degree of a curve.
16
1.0
0.9
B0,3(u)
B3,3(u)
0.8
0.7
0.6
0.5
B1,3(u)
B2,3(u)
0.4
0.3
0.2
0.1
u
0.0
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
Figure 2.6: Bézier curves [Ellman, 2002]
NURBS
NURBS stands for Non Uniform Rational B-Splines. This spline sort is probably the
most generic form of splines because many other kinds of splines are just a special
case of NURBS (e.g. Bézier splines).
n
P
C(u) =
Ni,p (u)wi Pi
i=0
n
P
,
a≤u≤b
(2.8)
Ni,p (u)wi
i=0
Equation 2.8 describes a p-degree NURBS curve, [Piegl and Tiller, 1980]. {Pi }
are the control points, {wi } the weights which influence the control points and
{Ni,p (u)} are the p-degree B-spline basis functions defined on the nonperiodic (and
nonuniform) knot vector U :
U = {a, . . . , a, up+1 , . . . , um−p−1 , b, . . . , b}.
| {z }
| {z }
p+1
(2.9)
p+1
B-splines are another type of spline that are defined by a nonperiodic knot vector
which determines the points where the curve must go through. For the calculation
of the NURBS curve there are three algorithms necessary, one to check which part
of the curve is determined by which knot points, one to calculate the blending
functions and another one to calculate an arbitrary point on the resulting curve. All
algorithms are fully described in [Piegl and Tiller, 1980].
2.4
Medical Terms
The eyes reflect the visual part of our cognition. An eye consists of the bulbus oculi
(globe), three inner and six outer eye muscles, all surrounded by the orbita, a dent
in the cranial bones. The EOMs are responsible for moving the eye consciously in
different directions. As this thesis concentrates on the EOMs, it is vital to have a
basic understanding of the anatomy of the human eye, especially the EOMs.
2.4.1
Anatomy of the Human Eye
Figure 2.7: Right eye (top view) [Sobotta, 1997]
The six EOMs (cf. Figure 2.7) are of extreme interest, because MR images taken of
them are one main starting point for this thesis work. These muscles are:
•
•
•
•
•
•
medialis rectus musculus (med. rect. m.),
lateralis rectus musculus (lat. rect. m.),
inferior rectus musculus (inf. rect. m.),
superior rectus musculus (sup. rect. m.),
inferior obliquus musculus (inf. obl. m.),
superior obliquus musculus (sup. obl. m.).
The inf. obl. m. and the sup. obl. m. are the two oblique muscles and the others are
the straight muscles. All straight eye muscles have their point of origin at the anulus
tendineus communis, a small ring of sinews at the end of the orbita. From there the
17
18
muscles emerge first with a sinewy part followed by the muscle itself and then join
the globe again with a sinewy part. The sup. obl. m. also has the anulus tendineus
communis as its origin and is turned around at the trochlea before attaching to the
globe. The inf. obl. m. originates at the front part of the orbita, near the nasal bone,
passing across the m. rec. inferior. The purpose of each eye muscle is described in
Table 2.1, where adduction and abduction describe a rotation toward the nose and
away from the nose around the z-axis (which develops in dorsal direction from the
center of the globe), respectively. Elevation and depression specify an upward and
downward rotation of the eye around the x-axis (which develops in lateral direction
from the center of the globe). Intorsion and extorsion define a rotation around the
y-axis (which develops in rostal direction from the center of the globe) from the top
of the bulbus toward the nose and away from it.
Muscle
med. rect. m.
lat. rect. m.
inf. rect. m.
sup. rect. m.
inf. obl. m.
sup. obl. m.
Primary Function
adduction
abduction
depression
elevation
extorsion
intorsion
Secondary Function
Tertiary Function
extorsion
intorsion
elevation
depression
adduction
adduction
abduction
abduction
Table 2.1: Primary, secondary, and tertiary functions of the six EOMs [Thömke, 2001]
Right in the middle of the anulus tendineus communis the nervous opticus passes
through, with its origin in the bulbus oculi. The bulbus oculi itself consists of three
skin layers: the tunica fibrosa, the sclera and the cornea. All six extra ocular eye
muscles attach to the bulbus oculi, each at a different point called insertion point,
which is the point where the muscle attaches to the globe. The pathway describes
a muscle on its way from origin to insertion point is called muscle path, whereas the
point where the muscle touches the bulbus oculi for the first time is called point of
tangency.
A point in three-dimensional space is fixed using all six eye muscles, whereby abduction and adduction are always executed by complementary muscles. Additionally,
eyes can only be moved binocularly, which means that the movement of only one
eye is not possible. According to [Kaufmann, 1995], three eye positions are feasible:
• Primary position is the position where the eye is looking straight forward without fixing on any objects in space. The body and the head also have to be held
straight.
• Secondary positions are reached when, starting from the primary position, the
eye rotates around exactly one axis, namely the x-axis or the z-axis.
• Tertiary positions are accomplished by a consecutive rotation around the x-axis
and the z-axis in any desired order. The starting point for these rotations is
also the primary position.
2.4.2
Rectus Muscle Pulleys
Rectus muscle pulleys influence the muscle path of the four straight eye muscles.
They consist of tissue sleeves composed of collagen and elastin stiffened by smooth
muscle, enveloping the straight extra ocular eye muscles [Demer et al., 1997]. These
tissue sleeves are firmly anchored to each other and to the walls of the orbita.
Rectus muscle pulleys are a relatively new discovery [Miller and Demer, 1995]. Their
appliance on the current bio-mechanical models of the eye lead to a dramatic change
in the definition of the axis of rotation of a muscle.
Several muscle models are available, all of them trying to describe the muscle movement, assuming that the eye muscles can move freely between the point of origin
and the insertion point. The consequence of this is that the muscle path and thus
the line of action is changed. To avoid this, the rectus muscle pulleys enclose the
straight eye muscles in order to keep the primary direction. With this knowledge the
muscle model can be extended so that the axis of rotation is not just determined by
the point of origin and the point of tangency, but also the muscle’s effective pulley
location. Figure 2.8 illustrates the differences between the conventional model and
the pulley model in primary position and in up-gaze in a graphical way.
Figure 2.8: Differences between conventional model and pulley model [Miller and Demer, 1999]
19
Chapter 3
Muscle Model Generation
This chapter is devoted to the different three-dimensional EOM models. It outlines
existing models and provides new ideas for improvement. Models will be compared
based on their anatomical correctness, whether software programs can process them,
and whether they can simulate muscle movement under real-life conditions. Of
course the relation to the patients will also play a major role, because it makes a
big difference, if an EOM model was generated out of patient data coming from one
specific patient or if it uses data from many people where the mean values of all
EOMs were calculated and then included into the model.
3.1
Existing Approaches
Several of the currently available models were developed by Joel M. Miller. In [Miller,
1989] the creation of one of the first three-dimensional EOM reconstructions is outlined. Based on coronal slices of MR imaging data from four subjects, all of them
with normal uncorrected vision; and ocular motility, a radiologist identified the orbital wall, globe and rectus muscles in the MR images using a pencil to mark the
contours. The edited pictures were then digitized using a digitizing pad. Subsequently, every data set was transferred to the corresponding eye in order to combine
the data of each subject into one result set. Finally, the normalized points were averaged by calculating the area centroid of any slice and subtracting it from its points.
Through averaging the X and Y values of each set of four corresponding contour
points and adding the mean area centroid an array describing the surface of four
rectus muscles, an “average” eye was obtained. The data was then rendered using
standard rendering software and the resulting pictures were redrawn by a medical
illustrator (cf. Figure 1.3).
The second model built by Miller (cf. [Miller et al., 2003]) also used MR images,
but this time not from living human subjects but from human cadavers. Miller
used a custom software tool called “TSR”, which was developed for the purpose of
combining different slice data (e.g. image structures coming from MR scans and
digitized images) to create another model. The result was an object representing a
20
Chapter 3. Muscle Model Generation
three-dimensional EOM model including globe and optical nerve (cf. Chapter 1.4,
Figure 1.5) as well as certain other tissue structures surrounding the EOMs.
Franz Pirklbauer and Michael Lacher were the creators of another EOM model which
has already been mentioned in Chapter 1.4 (cf. [Pirklbauer, 2001], [Lacher, 2001]).
Their approach was also to use MR imaging data, not from human cadavers but
from volunteers instead. The idea was to take two sets of MR images from one
patient, each set showing coronal slices of the EOMs, one with the eyes in primary
position and another with the eyes in secondary position. In each slice of a set the
muscle of interest had to be isolated, using the algorithms developed by [Pirklbauer,
2001]. The result was a three-dimensional but very coarse muscle model which could
be stored in the DXF format [Pirklbauer, 2001]. Pirklbauer and Lacher used the
two DXF models of one muscle in two different states of innervation as input and
calculated one dynamic three-dimensional model that could change its state within
certain limits.
Apart from these models another EOM model, based on finite-element analysis, was
created by [Simonsz et al., 2002]. This model uses MR images of serial sections of
orbits of fetuses and fits a mathematical surface through the contours on the images,
resulting in a three-dimensional model consisting of the orbita, the globe and the
EOMs. However, the search for the correct loading conditions for the EOM model
proved to be difficult, therefore more research in that field will be necessary.
The last EOM model, also based on MR imaging data, is part of a bigger project
called “The Visible Human Project” which was issued by the National Library of
Medicine (NLM) [Center for Human Simulation, 2001]. Its goal was to build a digital
image library of volumetric data representing a complete, normal adult male and
female, by means of MR and CT scans. These scans where performed on human
cadavers. The resulting data set was the source for many commercial products
like the “Voxel Man Series”, a set of programs dealing with the three-dimensional
visualization of the human body. It also offers an environment where the user can
segment certain areas in sets of MR images from the NLM and thus generate a threedimensional volume. In doing so, scan data from the EOMs can also be segmented
and rendered.
Finally there is the EOM model from the Blausen Medical Communications company
[Blausen, 2003] depicted in Figure 3.1. It is a model that was created from the
anatomical facts of the EOMs and the surrounding structures and is also available
in digital form. This EOM model approach is one example for many other EOM
models that have been created using the skills of artists.
3.2
Limitations of Existing Approaches
As none of the above-presented models is perfect, Chapter 3.2 will compare their
advantages and disadvantages as well as their limitations.
The first two models we will focus on are the ones created by Joel M. Miller. His
21
Figure 3.1: Blausen EOM model [Blausen, 2003]
intention for building the models of the EOMs was to learn more about the mechanical properties of the studied muscles and the extraocular pulleys, respectively. As
a result he didn’t analyze MR images of single people in order to get information
about, for example, pathological EOMs. Instead, he used data from several humans
and created an “average” eye model, with the result of losing individual patient
information. Millers second model was created out of data from human cadavers
and again no correlation to living humans and their specific eye muscles could be
made afterwards. However, with the results of his first model [Miller, 1989] it was
possible to understand the mechanical properties of pulleys and how they influence
the straight EOM movement. The second model [Miller et al., 2003] even made
it possible to visualize extraocular tissues. Both models are static models, which
means that the state of the EOMs could not be changed and thus no simulation of
muscle movement was possible. Another drawback was that the models were rather
complicated and time consuming to create.
The third model explained in the previous chapter was the one created by Pirklbauer and Lacher. This model was the only one allowing the simulation of muscle
movement. Thus, the model was more or less anatomically correct and rather simple
to create on the basis of MR imaging data coming from a specific patient. This also
allowed prediction on pathological properties of the EOMs. A possible disadvantage
could be that this model only included one muscle, therefore it was only possible to
reconstruct one muscle at a time. Additionally, the model had problems showing
a smooth muscle surface at each reconstruction because there were certain edges
visible on the created muscle that surely didn’t appear on the original muscle of the
patient. Another problem is that it is not possible to reconstruct the two oblique
eye muscles because they are not present at the original MR imaging data in a way
that enables a reconstruction. This results in the fact that these two muscles cannot
be reconstructed in Pirklbauer’s and Lacher’s model either.
22
Simonsz’s approach was limited in that he didn’t use data from specific persons
either, but from serial sections of orbits of fetuses instead. So again no prediction
about the situation of a certain muscle of a single person could be made. Additionally, his model only consisted of the four straight EOMs and didn’t take the oblique
muscles into account. So a correct simulation of the eye movement was not possible.
Another disadvantage was that only the impact of a change on one muscle on the
EOM model could be visualized and not a simulation of the specific movement of
an EOM.
Finally, all models that were built upon the visible human project data were also
not capable of reflecting anatomical data of a specific person, because again data
was generated from more than one person. All models derived of data from the
visible human project are only static models that cannot be changed in their state
of innervation because MR imaging data was taken from many subjects in only one
position due to the fact that the subjects had already been deceased.
The model constructed by the Blausen Medical Communications company was built
with the help of artists who used anatomical data of medical literature to recreate
the model, with the purpose of simply visualizing the human EOMs, with no respect
to their movement. This means that it cannot simulate the movement of an EOM.
Another problem of this approach was that special patient anatomy could not be
taken into account, thus leading to the fact that anatomically correct reconstruction
was not possible.
Table 3.1 shows all the discussed models in comparison. Our reason for building an
EOM model is to analyze the movement of the EOMs with the purpose of detecting
pathological changes of the oculomotor apparatus. Therefore, the basic features
which mainly have to be supported by an EOM model in order to enable analysis
of MR imaging data of the EOMs from certain patients are:
• digital form which means that the model has to be in a form that software
programs can process the data they consist of,
• anatomical correctness means that the model has to reflect the anatomical
circumstances which have been alleged by real life,
• data from a specific person means that the data was obtained from a person,
from whom we want to have a model reflecting the EOMs,
• simulate muscle movements denotes if a model is able to simulate muscle
movements,
• show all muscles in one model means that all EOM muscles (at least the
straight ones) can be viewed at the same time.
3.3
Possible Solutions for Optimization
The improvement of a current three-dimensional eye muscle model will focus on the
muscle model by Pirklbauer and Lacher, because this model is already implemented
23
Model
Miller 1
Miller 2
Pirklbauer/Lacher
Simonsz
Blausen
NLM
Digital
Form
+
+
+
+
+
+
Anatomically
Correct
+
+
-1
-2
+
24
Data From
Specific Patients
+
-
Simulate EOM
Movements
+
-
Show All EOMs
in One Model
+
+
+
+
+
Table 3.1: Overview on current EOM models
in the VISU software and it is the easiest for generating data for a specific patient.
Additionally, this model provides simulation of the movement of an EOM. The
main problem with Pirklbauer’s and Lacher’s EOM model was that it didn’t show
the correct anatomical representation of an EOM at each reconstruction but only in
some cases. Instead, it had some sharp edges situated on the muscle surface. Figure
3.2 illustrates the problem with errors on the muscle surface of the EOM model
by Pirklbauer and Lacher. So the starting point is to figure out techniques for the
improvement of the surface structure of the EOM model.
Figure 3.2: EOM model with errors on the surface
A possible solution for optimization is to change the way the surface of the EOM
model is calculated. The current generation process can be found in [Pirklbauer,
2001] and [Lacher, 2001]. First of all, the structure in which the muscle model exists
has to be discussed. The raw data consists of a cloud of points in a three-dimensional
coordinate system, the so-called base points. Each point represents a point on the
surface of the muscle. These points can be arranged in cross-sections, which means
that the cloud of points is cut into a number of slices which represent the cross1 Simonsz used data from sectioned human cadavers that had been conserved in paraffin, thus having the disadvantage of shrinking with time. The samples he used were from the 70’s.
2 Blausen used data from literature and, therefore, no relation to the person behind the data was possible.
sections of a muscle [Pirklbauer, 2001]. These cross-sections and then the shape of
the surface of the muscle along its length have to be calculated. Cross-sections are
specified by the user who wants to have a reconstruction of an EOM. The more
cross-sections there are specified, the higher the computational effort is. However,
the points also represent disturbances in the DXF model, because it includes parts
of tissue surrounding the muscle which don’t belong to the actual muscle. These
artifacts can cause the muscle model to lose precision or even appear incorrect.
Hence, a function has to be found which offers the possibility of calculating arbitrary
points that lie between the points currently representing the surface. This function
also has to be smooth enough to eliminate possible artifacts. Therefore, the model
will neither appear more correct nor more incorrect but “more beautiful”. The most
obvious way for optimization would be to find a spline function for the creation of
the surface, because the formulation of the problem includes interpolation in order
to get points on the surface of the muscle at arbitrary positions. But which spline
function is the most suitable and is somehow easy to use in a computational form?
The first possible approach for optimization could be the use of Hermite splines
which were already used by Lacher for calculating parts of the EOM model. For
the calculation of Hermite splines an additional point and a corresponding tangent
are always needed. That spline type is not very suitable for our needs because the
computational effort for finding a correct point with a corresponding tangent is much
higher than the effort for using, e.g., the NURBS interpolation method, see Chapter
2.3.2.
The second possible method is the polynomial interpolation [Press et al., 2002].
This type of spline curve interpolates between a set of given points by fitting a ndegree polynomial through the points (where n is the order of the polynomial). The
resulting curve develops through every base point and characterizes a closed curve
where each point on the curve can be calculated. The first point is the start point
and the last one defines the end point of the curve. Figure 3.3 shows an example
for a set of base points and Figure 3.4 shows the polynomial interpolation method
applied to these base points.
The disadvantage of this method is that the spline curve passes through every single
point of the base points, because it uses each one of the them as a control point.
This is a rather unsuitable method for our needs, because we need a curve that
doesn’t exactly fit through all the given points in order to smooth out disturbances.
So a better alternative has to be found.
A more suitable curve can be created by using the Bézier spline interpolation method
invented by Pierre Bézier [Bézier, 1977]. Bézier splines have the characteristic of
always staying inside the complex hull, which is determined by the smallest polygon
that includes all points of a set of points (cf. Figure 3.5) of the set of control points.
The control points in the case of the Bézier splines are equal to the given set of
points named base points in the previous interpolation method. If this interpolation
method is used on the points shown in Figure 3.3, the curve passes in a much
smoother way through the points (cf. Figure 3.6). The only disadvantage is that
there is not much control over the curve, meaning that also small indentations on
25
Figure 3.3: Set of points to be interpolated
26
Figure 3.4: Third degree interpolated
curve
Figure 3.5: Control polygon of the points
the surface as well as small bumps (both personated by the control points) will be
flattened and so the display of the muscle will lose accuracy. This can be seen in
Figure 3.6 where the fifth point (if counting starts at the left side) is rather far away
from the rest of the points. A solution would be a curve function which behaves
exactly like the Bézier splines but has the ability of changing its shape within a
certain region.
Introducing the NURBS interpolation method provides a solution to the problem
above [Piegl and Tiller, 1980]. This kind of splines does not only use control points
Figure 3.6: Bézier spline interpolation
applied to the points of Figure 3.3
27
Figure 3.7: NURBS interpolation applied to the points of Figure 3.3
(the base points in the example of the polynomial interpolation) but also a so-called
knot vector which determines the base points for the resulting spline curve. With
the help of the knot vector and a weight vector (cf. Chapter 2.3.2) it is possible
to change the shape of the curve within a certain region. Thus, small bumps or
indentations can be taken into account during the curve calculation. Figure 3.7
shows the spline created using the points in Figure 3.3. It can be seen that the
curve in the region under the fifth point is better fitted to the control points than
the one in the example using Bézier splines.
The improvement doesn’t only focus on the generation of the surface but also has to
include optimization of the DXF model generation. A big disadvantage of the DXF
generation process initiated by [Pirklbauer, 2001] is that it doesn’t extract data from
the original DICOM files. Hence, additional information about a set of DICOM files
taken from one patient are lost. A good example is the distance between the coronal
MR images. This information is important for the determination of the length of the
generated model. Also the information about how the image positions are situated in
comparison to each other, in order to reposition the images if necessary, is important
for further processing of the images.
3.4
Generation of Muscle Data for Kinematic Models
Kinematic models are used to describe the behavior of parts of the human body. A
kinematic model in the case of this thesis means to describe the movements of the
eyes based on the mechanical properties of each EOM. The model for this purpose
is already implemented in the SEE++ software system, namely the biomechanical
model (cf. Chapter 1.2). The only thing missing when visualizing the model is the
anatomically correct visualization of the EOMs. This is already done in SEE++,
but the data for the muscles was only taken from average muscle data. So real-life
conditions are not possible to visualize. The idea to improve this status is to extract
muscle data with the VISU software for the usage in SEE++.
28
Chapter 4
Design of a Muscle Visualization
System
It is the purpose of this thesis to extend the existing software system VISU by
introducing new methods of optimization to the current system structure in order to
improve the surface of the three-dimensional eye muscle model. Thus, the following
chapter gives a detailed description of the design of the VISU software system as
well as the design of the extensions for the optimization.
4.1
Existing System Design
The current VISU system architecture consists of four components (cf. Chapter 1.3).
The GUI component, responsible for the display of the results as well as the interaction with the user, consists of a series of classes which are all inherited from the
class TWindow. This class is a Borland C++ Builder 6- (cf. [Borland, 2002]) specific
class which enables the creation of different GUI views. These views receive their
data from the components described in Figure 4.1. However, a perfect capsulation
as introduced with the Model View Controller pattern [Gamma et al., 1995] is not
given because all the GUI views directly access the other two components. This
makes an easy exchange of the components, with respect to other components, for
example, with a better performance, more difficult to accomplish.
The administration component manages data about a patient and stores it in the
form of a directory hierarchy. Important information, e.g. name, date of birth or
address of the patient, is serialized into a file. Image data in the form of DICOM
files as well as bitmap data is stored in directories that correspond to each patient.
The class structure of the component data administration is visualized in the class
diagram in Figure 4.2.
The most important components are MR analyzation and MR processing. Both
were initially implemented by [Pirklbauer, 2001] and [Lacher, 2001]. Their purpose
29
Chapter 4. Design of a Muscle Visualization System
30
GUI component
Data administration
MR analysis
MR processing
Figure 4.1: Components of VISU
Administration
Patient
string name
Date birthdate
string diagnosis
string additionalComments
Administration
Map<string, Patient*>patMap
addPatient()
loadPatientList()
savePatientList()
removePatient()
print()
1
0..*
1
addSeries()
deleteSeries()
calcDynmod()
1
0..*
DynModel
0..*
int stepsAlongThePath
int stepsAroundThePath
int pathThreshold
int lastActivation
Study
Picture
string side
string description
string fileName
1
*
1
1
0..1
0..*
PictureSeries
string muscle
string activation
string dxf3DRec
Threshold
1
Palette
0..1
addPicture()
removePicture()
Polygon
0..1
Figure 4.2: Class diagram of the administration component
1
31
was already explained in Chapter 1.3, but a closer look into the creation of the EOM
model will be taken in the following. Figure 4.3 and Figure 4.4 illustrate the data
flow of the creation of an EOM model.
MR data from imaging
device
DXF file from MR
analysis
picture setup
calculation of gravity line
sort color table
import DXF
define polygons
transform DXF format
set threshold
generate polyhedron
calculate area centroids
define data volume
use scan-line/marchingcubes algorithms
spline interpolation
generate surface
calculate volume and display
surface
storage
set threshold
interpolation between
the two EOM models
DXF file
Figure 4.3: Data flow of the creation of
a DXF model in VISU
Figure 4.4: Data flow of the creation
of the EOM model in VISU out of the
DXF file
As this thesis concentrates on the work by Lacher, it is important to be familiar
with the structure of the MR processing component. Figure 4.5 shows the class
structure of the dynamic muscle model. Lacher’s dynamic model is determined by
the DynamicMuscle class, which holds instances of the MuscleParameters class and
the DynamicModel class. These three classes are the most important ones for the
following explanations, because they are responsible for the creation of the muscle
surface.
32
dynamicMuscle
PointList
DXFModel
#points
filename
name
MuscleParameters
1
1
#midpoints
1
1
DXFModel()
getName()
~DXFModel()
getFilename()
render()
getPoints()
getVertexCount...
getIndexCount()
1
#model
1
MuscleImage
MuscleImage()
getModel()
getModel()
~MuscleImage()
getSourceImages()
getSourceImages()
getMuscleParameters()
getMuscleParameters()
getVertexCount()
getIndexCount()
#parameters
1
1
activation : float
threshold : float
max_threshold : float
steps_u : int
steps_v : int
radii : float*
length : float
volume : float
DynamicMuscle
steps_u : int
steps_v : int
MuscleParameters()
calculateMaxThreshold()
~MuscleParameters()
generateMidpoints()
generateSplines()
generateRadii()
recalcVolume()
recalc()
#dynamicParameters
getActivation()
setActivation()
getThreshold()
1
1
getMaxThreshold()
setThreshold()
getSliceCount()
getSizeU()
getSizeV()
getLength()
setLength()
setUVSize()
getVolume()
1
getRadius()
setRadius()
getSplines()
getSplines()
render()
#dynamicModel
DynamicMuscle()
DynamicMuscle()
addImage()
~DynamicMuscle()
getImage()
replaceImage()
removeImage()
clearImages()
getImageCount()
getDynamicParameters()
getDynamicModel()
setInervation()
getAverageVolume()
getSizeU()
getSizeV()
setUVSize()
render()
getVertexCount()
getIndexCount()
1
#muscleStates
1
1
#sourceImages
DynamicModel
1
MuscleImageList
1
SourceImages
siSizex : float
siSizey : float
SourceImages()
getImageCount()
~SourceImages()
getImageName()
replaceImage()
render()
DynamicModel()
DynamicModel()
rebuildModel()
~DynamicModel()
changeTexture()
changeTexture()
getTextureName()
isResourceTexture()
render()
getVertexCount()
getIndexCount()
Figure 4.5: Class diagram of the MR processing component [Lacher, 2001]
4.2
Extension of the Existing Structure
In Chapter 4.2 the necessary extensions to the VISU structure for the optimization
of the muscle surface are depicted. First the picture processing is explained. Its main
purpose is to extract data from DICOM files and process it. Then the calculation of
the area centroids of the muscles’ cross-sections is outlined in Chapter 4.2.2 followed
by the design of the analysis of variance. Finally, Chapter 4.2.4 deals with the
improvements to the creation of the muscle surface.
4.2.1
Picture Processing
The design for the picture processing part is kept very simple, it only consists of one
class with the name PictureProcessing. This class contains an attribute which is of
the type TDICOMBox which is an ActiveX component (cf. [Krug, 2000]) that has
been adapted. TDICOMBox is a class that realizes the access to a native DICOM
file, which enables the extraction of information like slice thickness or distance between the MR slices. Currently, methods for the extraction of the distance of the
MR images between each other and for the extraction of the positions of the MR
images between each other are supported. This information is stored in the header
of a DICOM file and is important for the generation of the DXF model which is the
result of the MR analysis component. Without the correct distance information, the
muscle would appear too short and thus be incorrect according to the anatomical
facts. The problem appears when using different types of MR imaging devices, but
it can also appear when using only images created by one MR imaging device of
the same vendor. The reason lies in the fact that the slice thickness can be chosen
manually by the person who operates the MR imaging device. At the time when
Pirklbauer wrote the code for the generation of the DXF EOM model he received
MR images with a slice thickness of 3.6 millimeters. He implemented this value as a
constant in the source code, not enabling the user to change it. Newer MR imaging
devices are already able to create series of MR images with a slice thickness of 1.5
millimeters. However, the vast progress in the development of MR imaging devices
made it necessary to allow a dynamic adaptation of the slice thickness stored in
the DICOM files for the use in VISU. This results in the correct calculation of the
muscles’ lengths.
Another problem is that the images can be shifted mutually. Thus, for a better tracing of the muscle course the display window of the MR device can be changed. This
results in MR images which, if put on top of each other, show an EOM muscle not
reflecting the natural course of the muscle. This is a side effect of image generation
with an MR imaging device (it would look as if it had huge steps in it). Getting rid
of these disturbances is the main purpose of the class PictureProcessing.
Methods
There are three methods available:
• OffsetPictures(. . . )
• GetDistanceBetweenSlices(. . . )
• GetVoxelDistance(. . . )
The first method realizes the correction of the shift of the images to each other, if
necessary. It thereby uses the information gathered from the TDICOMBox component. The two other methods also use this information as they return the distance
that lies between two consecutive images and the scaling factor for each voxel in
millimeters.
4.2.2
Gravity Line Optimization
The gravity line optimization characterizes methods for the calculation of the area
centroid of each cross-section of the muscle model. As already mentioned in Chapter
33
3.3, the cross-sections are slices of points having the same z-coordinate forming a
user defined slice of the muscle. It is now vital for all further calculations of an
EOM model to know the center of each muscle slice and its coordinates, respectively.
Figure 4.6 shows the point cloud of the DXF muscle model with the calculated area
centroids. Each area centroid is connected by a line with the one of the next slice.
This line depicts the gravity line of the DXF muscle model and equals the gravity
line of the real muscle of the patient. Of course this calculated line is too angular
and thus has to be smoothened by means of interpolation, because at this point
of improvement all the cross-sections include their original points coming from the
DXF model. Thus some of the cross-sections haven’t got enough points in order to
represent a cross-section that corresponds to the MR images of the muscle. These
cross-sections have an area centroid which deflects from the ones that correspond to
the MR images.
Figure 4.6: DXF muscle model with correct length calculation
The gravity line has already been calculated by Lacher in the class MuscleParameters. Starting with the DXF model he used a calculation method for the area
centroids that treated each slice of points as if each slice included the same number
of input points. But in fact most of the defined slices consist of not enough points
to represent a muscle cross-section. This leads to false calculated area centroids,
thus resulting in an incorrect muscle gravity line. Hence, the slices including too
few points have to be identified and filled up with additional points to represent a
correct muscle cross-section. The reason for the existence of muscle slices with too
few points is that the user of the VISU software should have the possibility to choose
the accuracy of the resulting EOM model by himself. This leads to the possibility
of arbitrary sectioning of the DXF point cloud into slices even at “bad” positions,
which can evolve because of artifacts in the underlying MR image set or because of
misplaced triangles in the marching cubes algorithm [Pirklbauer, 2001]. The necessary optimization methods for improving the calculation of the gravity line of an
EOM model are implemented in the class RadiusImprovement. This class interacts
34
with the class MuscleParameters, which already existed in VISU, by receiving all
coordinates of the points of the DXF muscle model. It also receives the generated
muscle gravity line which is calculated in the class Splines (which was also already
implemented in VISU), because in RadiusImprovement only the optimization of the
gravity line is done and not the actual calculation.
Methods
There’s only one method necessary, namely AddHelpPoints(. . . ), which improves
the area centroid calculation by using the information from the coordinates of the
area centroid of the nearest cross-section which corresponds to the MR images.
4.2.3
Analysis of Variance
The purpose of the analysis of variance is to identify cross-sections which include
either too few points or false points. The reason for their existence has already been
explained above. Once identified, the cross-sections are corrected by using points
from the cross-section nearest to the ones with not enough points. The result of
the identification process is shown in Figure 4.7, where all the defined cross-sections
(73 in this case) were analyzed and visualized in a diagram. The x-axis shows the
number of arbitrary chosen cross-sections of one EOM and the y-axis the value
assigned to each cross-section by means of analysis of variance. It can be seen that
there are sections with low and high values, but only six of them reach a value which
is higher than 80. Note that this value is only a guiding value to show that the value
of those sections differs particularly from the other ones. These six sections are
the cross-sections that consist of points which were derived directly from the MR
images; all the others were extracted out of sections that lie between the MR images,
thus not including enough information for further processing. Hence, the diagram
shows which cross-sections have to be edited by adding additional points in order to
suit our needs. The cross-sections between the MR images are included in the DXF
model that formed a three-dimensional model out of the muscle which was visible
in the MR images.
Methods
Two methods are important for the analysis of variance:
• EvaluateVariance(),
• CorrectSlices(. . . ).
The first method evaluates the variance of each cross-section and the second one
performs the correction by adding points, taken from the nearest complete slice,
and adding them to the incomplete slice.
35
120
100
variance
80
60
40
20
0
1 5 9 13 17 21 25 29 33 37 41 45 49 53 57 61 65 69 73
number of cross-sections
Figure 4.7: Partition of variance and amount of slices
4.2.4
Surface Optimization
This chapter covers the biggest part of the work that was done for the improvement,
which is the spline interpolation. The interpolation method used is the NURBS
interpolation. A class Nurbs takes care of the interpolation itself, while the RadiusImprovement class provides the points to be interpolated in the form of an STL
vector that holds the points in three-dimensional space. This vector includes four
elements: the (x, y, z)-coordinates of the point in three-dimensional space and a
fourth value which represents the weight that assesses each coordinate. After each
cross-section has been corrected, based on the analysis of variance, the improved
slices are processed by calculating a NURBS curve for each cross-section, so that
this curve fits exactly between the points used as control points (cf. Chapter 3.3).
To do so each cross-section is split into two halves, one half containing all points in
the range from 0 to 180 degrees in angle and another half containing points ranging from 181 to 360 degrees (cf. Figure 4.8). The angle always refers to the area
centroid of the currently processed cross-section. Subsequently, each obtained half
is interpolated in the class Nurbs. The result consists of two splines which form
one cross-section of the EOM (cf. Figure 4.8). After repeating this process for each
consecutive slice, the surface of the EOM model is created. The next step is to improve the surface also in the length of the model. This has to be done, because the
36
already-calculated shapes of the cross-sections have slightly different area centroids
which result in a shift of the sections to each other and thus in a “bumpy” surface.
By using the same spline interpolation again on the sum of all spline interpolated
cross-sections, a totally homogenous surface with no disturbances is obtained.
Figure 4.8: Cross-section of DXF model with both halves interpolated
Methods
The class RadiusImprovement includes one method CreateRadii(. . . ). This method
starts the creation of the muscle model using NURBS and returns a data structure
including the complete muscle surface. It also calls two private methods Interpolate(. . . ) and DoSplineOnMuscleLength(. . . ). The first of these methods uses the
class Nurbs to generate the spline and the second one does the same interpolation
again but this time on the whole length of the muscle.
The class Nurbs includes mainly the method CurvePoint(. . . ) to calculate a NURBS
curve of a set of control points which are received from RadiusImprovement. It also
includes a helper method DoWeightOnPoints(. . . ) to format the control points for
its needs.
4.3
Modifications of the VISU Structure
The modifications of the VISU structure turned out to be more complicated than
they appeared to be at first glance. The classes that were modified are part of nearly
all components of VISU.
The first modification was done in the GUI component where the class PictureProcessing was instantiated in the class TWizardGenerateStaticModel. This class takes
37
care of the extraction of the image position data out of the original DICOM files
as well as the distance data of the MR images with respect to each other. Images
were also shifted, when necessary, by the class PictureProcessing. The distance data
was handed over to the MR analyzation component, where the correct length of the
muscle could now be used for the calculation of the DXF model. However, the most
extensive changes were done in the MR processing component. The generation of
the muscle gravity line, which is the resulting line after connecting all calculated
area centroids of ascending cross-sections (cf. Figure 4.6), was improved using the
methods in the class RadiusImprovement.
The next step was the creation of the surface of the EOM model. Thus, raw data
for the interpolation had to be provided. This was done by handing over the set of
points of one DXF model to the class RadiusImprovement, where the whole muscle
model was generated using the NURBS interpolation method. This model was then
returned to the class MuscleParameters in an adequate data structure and the whole
process was repeated for the second DXF input model. The data structure consisted
of an STL vector which included a pair of two vectors, the first one stored the spline
point in three-dimensional space and the second one stored the area centroid of the
cross-section to which the spline point belonged. An advantage was that the only
extension of the class MuscleParameters was the addition of one attribute, which
is an instance of the class RadiusImprovement. With this extension the creation of
the EOM model was finished, but a big problem had yet to be solved, namely the
problem of making the generated new surface model compatible with the existing
visualization structures in VISU.
The former model of an EOM was defined by the muscle gravity line and a radius function [Lacher, 2001]. The radius function was precalculated and stored in
a vector array. This had the advantage of easy calculation and interpolation of the
surface of the EOM model. However, the newly introduced concept of the NURBS
interpolation made it impossible to retain the former definition of the EOM model.
An adaptation was made by changing and adding methods to the classes MuscleParameters, DynamicModel and DynamicMuscle.
Once finishing the creation of two muscle gravity lines and the associating radius
function, the EOM model can be displayed and simulated. This is done by recalculating the radii stored in the radius vector with respect to the muscle volume and
rerendering the already displayed model. However, a displayable model cannot be
described only by its radius and that’s why the vertices of the model have to be
calculated. Once this is done (in the class DynamicModel ) the vertices are handed
over to a predefined data structure in the same class, whereby the change of the
radii is accomplished in the classes MuscleParameters and DynamicMuscle. Therefore, each time the user changes the state of the EOM model, all three classes are
involved.
Figure 4.9 illustrates how the three additional classes where inserted into the existing
VISU structure.
The following methods were involved in the process of adapting the new spline
interpolated model to the VISU structure:
38
component GUI
TWizardGenerate
StaticModel
39
PictureProcessing
dicomViewer* :TDICOMBox
fileNames* :TStringList
PictureProcessing()
~PictureProcessing()
OffsetPictures(...)
GetDistBetweenSlices()
GetVoxelDistance()
RadiusImprovement
component MR
processing
MuscleParameters
midpoints* :std::vector<Vector3f>
points* :PointList
splines* :SplineGroup
RadiusImprovement()
RadiusImprovement(...)
RadiusImprovement(...)
~RadiusImprovement()
CreateRadii()
Interpolate(...)
DoSplineOnMuscleLength(...)
EvaluateVariance()
CorrectSlices(...)
AdddHelpPoints(...)
WriteMuscleData(...)
Nurbs
U*, W* ,u* :double
n, p :int
Nurbs(...)
~Nurbs()
CurvePoint(...)
DoWeightOnPoints(...)
Figure 4.9: Expanded class diagram of the VISU structure
• RebuildModel(. . . ) from class DynamicModel,
• GetRadius(. . . ), SetRadius(. . . ), GetSplineVector(. . . ), GetSplinePair(. . . ), from
class MuscleParameters,
• SetInnervation(. . . ), from class DynamicMuscle.
GetSplineVector(. . . ) and GetSplinePair(. . . ) were the only methods added to the
class MuscleParameters. All other methods were already available and have just
been modified. The adaptation also led to the introduction of an additional member
variable in the class MuscleParameters, which is necessary in order to store the
muscle model generated by RadiusImprovement. This data structure is used for the
interpolation of the EOM model. Therefore, this is the structure that replaces the
combination of muscle gravity line and radius function.
Finally, data was generated for use in the SEE++ software. This was done in the
class RadiusImprovement using the method WriteMuscleData(. . . ). Its job is to
calculate the dimensions of the muscle and then write the obtained data into an
initialization file.
Chapter 5
Implementation
The implementation of this thesis work includes mainly algorithms for the preparation and processing of images. Therefore, it is necessary to elucidate those algorithms
with the intention to provide the reader with knowledge about the functionality of
the software structures used. All algorithms have been implemented in C++.
5.1
Muscle Gravity Line Optimization
The basic purpose of the muscle gravity line optimization has already been outlined
in Chapter 4.2.2. This chapter will describe in detail how the actual algorithmic
structures were implemented. The former calculation of the muscle gravity line was
implemented in VISU using the source code in [Lacher, 2001]. Therefore, an array
of points is built up which were extracted out of the DXF muscle model coming
from the MR analyzation component. The array consists of a Standard Template
Library (cf. [Prata, 1998]) vector containing points represented by three-dimensional
vectors. The stored point cloud is then parted along its z-coordinates by storing the
indices of those points having the same z-coordinates in a second vector containing
arrays of integers. After inserting the indices into the second vector, this vector
represents the cross-sections of the DXF muscle model. Lacher’s next step was to
create the area centroid of each cross-section. However, the cross-sections didn’t
contain enough points, so this is the starting point for the improvements. The area
centroid is calculated by simply adding all coordinates of one cross-section together
and then dividing the result by the number of cross-sections. The newly improved
calculation method uses the same concept with the refinement of adding additional
points to cross-sections having too few points. This is implemented in the method
AddHelpPoints(. . . ) of the class RadiusImprovement. The method gets a reference
to the currently calculated area centroid and an array which contains the indices to
the points ordered by the muscle cross-sections as well as the index of the currently
processed cross-section. Every time a cross-section doesn’t contain enough points,
the AddHelpPoints method is called, thus detecting the nearest cross-section which
consists of enough points to calculate a correct area centroid. If such a cross-section
40
Chapter 5. Implementation
is found, the points are combined with the currently calculated area centroid. The
return value of the method is the number of points that has been added. The fact
that the first cross-section always corresponds to an MR image ensures that there
is always a cross-section to gather points from.
After finishing the calculation of one area centroid, a second improvement is carried
out. The idea is to sort out area centroids that have an ascending slope of more
than 45 degrees with respect to the previous area centroid. If one is found, this area
centroid and the one which has already been calculated are averaged. This way the
area centroid having a slope that is too high gets one that is lower.
After the area centroids of all cross-sections have been calculated, the points that
represent the muscle gravity line are interpolated using the Hermite spline interpolation to get a smoother gravity line. This is already implemented in VISU and
doesn’t have to be changed.
5.2
Applying Analysis of Variance
The analysis of variance is the basis for the interpolation methods following in the
next chapter. The first step is to create a vector storing not only the points belonging
to one cross-section but also the angle in the x-y plane that the vector of the point
spans regarding to the area centroid of the currently processed cross-section (cf.
Figure 5.1). The direction of the z-axis is assumed to develop through all crosssections of a muscle, whereby the x- and y-axes lie in the same plane in which one
cross-section lies. The correct angle for each point is already created in VISU; it
only has to be mapped for our needs. This is done in the method FillBuckets(. . . )
of the class RadiusImprovement. The second step is the evaluation of the variance of
each cross-section in order to sort out those without enough points or faulty points.
This has already been explained in Chapter 5.1. But this time it is also taken into
account that there are cross-sections without enough points which form a suitable
cross-section for the spline interpolation anyway. Finding such cross sections results
in a more correct EOM model.
The method EvaluateVariance(. . . ) sorts the points of all cross-sections according
to their ascending angles. This is done by using a helper method, Compare(. . . ),
which is called by the STL method sort(. . . ) at the beginning of EvaluateVariance.
Afterwards the variance is calculated according to the equations outlined in Chapter
2.3.1, then the result is stored in an STL vector, which is the return value of the
EvaluateVariance method.
The next step is to correct those cross-sections which have points that do not correspond to an MR image or which don’t have enough points to satisfy the NURBS
interpolation. The correction takes place in the method CorrectSlices(. . . ) of the
class RadiusImprovement. It receives the vector with the values coming from the
analysis of variance as input. Based on these values, the slices that are below a
certain value are corrected by adding the points of the nearest cross-section which
41
42
y-axis
points of the
cross-section
a
area
centroid
0/0
x-axis
Figure 5.1: Area centroid of a cross-section and the angle a that the point spans regarding the area
centroid
consists of correct points in the sense of the NURBS interpolation. The following
steps are performed in order to add points:
• Iterate over the values received from the analysis of variance.
• If a value is below a certain threshold, start searching for a slice which has a
better variance.
• Find the best variance by searching left and right, take the first slice found
which has correct points and the shortest distance to the faulty slice.
• Add the index of the found slice to the result, which is a vector representing
all cross-sections with each index entry representing the index of the correction
slice.
5.3
Interpolation Using Splines
This chapter describes the spline interpolation in detail using NURBS with RadiusImprovement and Nurbs as the participating classes. The main methods involved in the interpolation concerning the class RadiusImprovement are:
• GenerateSplineArray(. . . )
• CreateRadii(. . . )
• Interpolate(. . . )
43
• CalculateNewResult(. . . )
• DoSplineOnMuscleLength(. . . )
Nurbs uses all included methods for the interpolation. But before describing the
interpolation itself, it is necessary to determine the input points over which the
iteration should take place. The first step is to rebuild the cross-sections by adding
points according to the output vector of the method CorrectSlices (cf. Chapter
5.2). Additionally, the points of one cross-section are also parted in those having
an angle which is lower than 180 degrees and those whose angle is greater than 180
degrees. The result of this partitioning is stored in a data structure consisting of
an STL vector which contains a pair of lists each containing the points belonging
either to the first half of a cross-section or to the second half. All this takes place
in the method GenerateSplineArray, which is called in the class MuscleParameters
by the method GenerateRadii. Now the partitioned points of each cross-section
have to be processed in order to contain only unique points, which means that each
point representing a point on the surface of the muscle has to follow the criteria
x1 < x2 < x3 < · · · < xn with n representing the number of points in one half of a
cross-section. The existence of two points having the same coordinates is possible
because of the insertion of points into cross-sections with too few points or faulty
points. The reason why all points of one slice have to be different from each other
is that in some cases it is possible that a spline curve is created which has a cusp
(visual discontinuity) (cf. Figure 5.2), although the resulting spline curve is always
c1 continuous1 .
P2 = P3
P1
P4
P0
P5
Figure 5.2: NURBS curve with two equal control points P2 = P3
Such cusps have to be avoided in order to generate a surface that looks smooth
enough, although the curve is c1 continuous in such a point.
After the points in each cross-section have been prepared in the above-described
way, they can be used as input for the generation of the splines by handing over
1“c1
continuous” means that the first derivative is continuous.
44
the points of one half of a cross-section to the class Nurbs. When finished with
the generation of one half, the next half is processed. The method Interpolate is
responsible for the execution of the interpolation. Therefore, it first instantiates the
class Nurbs and then calls the method CurvePoint after creating a weighted version
of the input points using DoWeightOnPoints. These weighted points are necessary
for the adaptation of NURBS, since they can influence the shape of the resulting
curve. However, in this thesis the vector for the determination of the weighted
points is set in a way that each point is being amplified with the same number. In
the method CurvePoint, the actual calculation of one point on the spline curve is
done in the following way:
• determine the span in the knot vector U,
• calculate the nonzero blending functions (cf. Chapter 2.3.2),
• calculate the point using Equation 2.8.
Finding the span where a point on the curve lies is important because otherwise all
the blending functions of a NURBS curve would be generated. As a NURBS curve
is the sum of many smaller splines, it is a big time saver to only calculate those
blending functions that are nonzero and thus contribute to the generation of the
resulting curve. The FindSpan algorithm is basically a binary search algorithm that
searches through the knot vector U (cf. Equation 2.9).
int Nurbs : : FindSpan ( double u , int n ){
i f ( u == U[ n +1])
return ( n ) ;
int low = p ;
int high = n+1;
int mid = ( int ) ( low+high ) / 2 ;
while ( u < U[ mid ] | | u >= U[ mid +1]){
i f ( u < U[ mid ] )
high = mid ;
else
low = mid ;
mid = ( low+high ) / 2 ;
}
return mid ;
}
Listing 5.1: Method FindSpan
Listing 5.1 shows the method FindSpan of the class Nurbs where u is the input point
for the spline function, n the number of control points, U the knot vector and p the
degree of the NURBS curve. The result is the index in the knot vector where the
blending functions are nonzero.
The next important algorithm is the one to calculate the blending functions. It
is implemented in the method BasisFunctions(. . . ) and implements Equation 5.1
45
(from [Piegl and Tiller, 1980]) to compute the nonzero blending functions.
½
1 : if ui ≤ u < ui+1
0 : otherwise
u − ui
ui+p − u
Ni,p (u) +
Ni+1,p−1 (u)
Ni,p (u) =
ui+p
ui+p+1 − ui+1
Ni,0 (u) =
(5.1)
The variable ui in Equation 5.1 indicates the ith knot point and the variable u the
value which is the input for the interpolation.
void Nurbs : : B a s i s F u n c t i o n s ( int i , double u , int m, double ∗ N){
N[ 0 ] = 1 . 0 ;
double ∗ l e f t , ∗ r i g h t , saved , temp ;
l e f t = new double [m] ;
r i g h t = new double [m] ;
for ( int j = 1 ; j <= p ; j ++){
l e f t [ j ] = u − U[ i +1−j ] ;
r i g h t [ j ] = U[ i+j ] − u ;
saved = 0 . 0 ;
for ( int r = 0 ; r < j ; r++){
temp = N[ r ] / ( r i g h t [ r +1]+ l e f t [ j−r ] ) ;
N[ r ] = ( double ) saved + ( double ) r i g h t [ r +1]∗ temp ;
saved = l e f t [ j − r ] ∗ temp ;
}
N[ j ] = saved ;
}
delete [ ] l e f t ;
delete [ ] r i g h t ;
}
Listing 5.2: Method BasisFunctions
Listing 5.2 shows the method BasisFunctions which returns an array containing the
values of the blending functions in N at the value u. i is the index to the knot vector
U , and m is the size of U .
Once the blending functions are determined, the point on the spline can be calculated. This calculation is done over the whole range that one cross-section specifies.
This means that the calculation starts at the point which has the smallest value
on the x-axis of a cross-section and stops at the point having the highest value on
the x-axis. Moreover, it is also ensured that the transition between the two curves
(the upper half and the lower half of a cross-section) is c1 continuous, because both
splines have the same start and endpoints by using the adjacent point of the other
half. Finally, all the generated points are stored in a map together with their angle
regarding the area centroid of each cross-section. This is necessary for mapping the
generated points to the visualization methods which are already implemented in
VISU.
46
To display the muscle’s surface and to realize a correct mapping of the texture to the
surface of the muscle, it is vital that all spline points have a defined distance to each
other. This includes equally spaced distances in one cross-section as well as in the
next, leading to the result that points of one cross-section correspond to points of a
consecutive cross-section by lying in the same plane. This property is, however, not
realized after the NURBS interpolation. So the previously calculated splines have
to be processed in order to obtain equally spaced points. The responsible method
for this process is CalculateNewResult(. . . ) of the class RadiusImprovement. It
receives the points of one spline curve, generated over one cross-section, as well as
the position of the area centroid of the currently processed cross-section as input.
The output is a vector containing equally spaced points around the area centroid
of each cross-section. The points should be distributed having a distance of an
adjustable amount of degrees to each other.
As already mentioned above, all points of a spline are stored together with their
angle regarding the area centroid of a cross-section. The idea is to have a counter c1
counting up an angle in equal distances δ which can be adjusted by the user. This
angle can be used to find points of the spline curve that fit the criteria β ≤ α < γ,
where α is the angle determined by the counter and β and γ, respectively, are the
points belonging to two consecutive points of the spline curve. Figure 5.3 shows a
single cross-section which has already been interpolated using NURBS. It can be seen
that the points which were generated have no equal distance to each other. V~1 and V~2
are two vectors representing two consecutive spline points of a cross-section. Vector
~ represents the area-centroid of a cross-section, thus having its starting point at
M
the center of the coordinate system represented by the two axes x and y. Vector P~
is the one calculated from the current angle a = α which is consequently increased
by δ as well as elongated to intersect vector V~12 which is the resulting vector from
~ n−i is the new vector determined by the intersection
subtracting V~1 from V~2 . Vector X
◦
~
~
point of V12 and P (n = 360 /number of points on the spline, i = 0 . . . n − 1). X~n−i
has to be calculated by subsequently increasing α from 0 degrees to 360 degrees
~ n−i (drawn in dashed lines in Figure 5.3) can be
in equal distances δ. Vectors X
calculated each having the same increase of angle (represented by bn , bn−1 , bn−2 where
bn−2 + δ = bn−1 , bn−1 + δ = bn ).
Vector X~n can be calculated by solving the linear system of equations:
~n = M
~ + s ∗ P~
X
~ n = V~1 + t ∗ V~12
X
V~12 = V~2 − V~1
~ |)
P~ = P~0 ∗ (| V~1 | + | M
(5.2)
(5.3)
(5.4)
(5.5)
When substituting Equation 5.2 into 5.3 (cf. Equation 5.6), the two variables s and
t can be calculated (from Equations 5.7 and 5.8):
47
Y
V 12
P
X n-2
X n-1
Xn
V2
V1
a
M
bn
b n-2 b n- 1
-X
X
-Y
Figure 5.3: Linear interpolation of the NURBS-generated cross-section
~ + s ∗ P~ = V~1 + t ∗ V~12
M
~ x + s ∗ P~x = V~1x + t ∗ V~12x
M
~ y + s ∗ P~y = V~1y + t ∗ V~12y
M
(5.6)
(5.7)
(5.8)
This results in:
t =
~ y + s ∗ P~y − V~1y
M
V~12y
s =
~ x ∗ V~12y − M
~ y ∗ V~12x + V~12x ∗ V~1y − V~12y ∗ V~1x
M
V~12x ∗ P~y − V~12y ∗ P~x
(5.9)
(5.10)
~ n−i for each angle a from Equations 5.2 or
Now it is possible to calculate vector X
5.3.
5.4
Muscle Data Generation
The generation of muscle data has the purpose of providing the SEE++ software
system with geometrical muscle data. This data is used in SEE++ to display the
EOMs with higher accuracy than in the current version. The interface between
VISU and SEE++ are two files where data is stored in special number-value pairs
that represent information about each muscle. The files are generated in the method
WriteMuscleData(. . . ) in the class RadiusImprovement. Its input parameters are
the name of the file, which is “MuscleDataLeft.ini” or “MuscleDataRight.ini” for the
left and the right eye, respectively and the surface data of the generated EOM model.
The structure of the data files which is required by the SEE++ software system is
shown in Listing 5.3.
1 : [RM]
2 : NumberOfArcOfContactSlices=20
3 : N u m b e r O f S p l i n e S l i c e s =30
4 : NumberOfSlicePoints =20
5 : S p l i n e T e n s i o n =0.7
6 : Width−X−Values = 0 , 1 , 5 , 1 1 , 1 3 , 1 6 , 2 0 , 2 5 , . . .
7 : Width−Y−Values = 2 , 2 . 6 , 3 . 0 , 3 . 2 , 3 . 4 , 3 . 8 , . . .
8 : Height−X−Values = 0 . 0 0 , 1 . 0 , 5 . 0 , 1 1 . 0 , 1 3 . 0 , . . .
9 : Height−Y−Values = 0 . 0 1 , 0 . 6 , 1 . 0 , 1 . 3 , 1 . 2 , . . .
1 0 : I n n e r v a t i o n P o s i t i o n =78
1 1 : I nne rvationW idth =30
1 2 : B u l b u s R a d i u s D e l t a T o l e r a n c e =0.1
Listing 5.3: SEE++ initialization file specification
Each file is divided into sections which specify the type of muscle followed by numbervalue pairs which provide the actual muscle dimension data. The section name
specifies the type of muscle. In the example of Listing 5.3 [RM] the rec. med. muscle
is specified. Pair number 2 defines the number of cross-sections that specify the part
of the muscle which touches the bulbus. This part of the muscle consists mainly
of the muscle tendon and is not visible on the MR images. The result of this is
that VISU is not able to calculate this part of the muscle and can’t thus display the
sinewy part. The next pair enumerates the number of cross-sections on the muscle
gravity line that don’t touch the bulbus, so the whole length of the muscle can be
calculated by adding pair 2 and 3. Pair number 4 specifies the number of points
per spline in a cross-section. Pair number 5 is a SEE++ specific value. Pairs 6 to 9
specify the dimensions of the EOM. They include the actual data which is generated
by VISU. Finally, pairs 10 to 12 are again SEE++ specific tags.
Figure 5.4 shows the rec. med. muscle of a right eye in the SEE++ software system.
This muscle image was generated from data produced by VISU. The black lines
mark the point from where the muscle was generated by VISU. The part before
the left black line and the part of the muscle after the right black line were filled
up according to average muscle dimension data. It can be seen that the rec. med.
muscle of the right eye has now nearly the same dimensions as the muscle that can
be seen underneath the SEE++ muscle in the same figure which was generated with
VISU. The reason for the differences is that SEE++ produces symmetric muscles
and doesn’t take asymmetric muscles into consideration.
48
Figure 5.4: SEE++ screenshot showing the EOM built of muscle data from initialization files
49
Chapter 6
Validation
An EOM model is only useful for the person who works with it, if it matches anatomically with the MR images. However, this important premise is much harder to accomplish than it might seem at first glance. The most important question concerns,
how it can be ensured that the EOM model, which has been reconstructed from MR
images, is anatomically correct. The answer to this question will be the focus of
this chapter. But before starting with the validation, it must be made clear why the
name of this chapter is only Validation and not Validation and Verification.
Validation determines whether a practically useful system was built, whereas verification determines whether a system was built correctly. Thus, the verification of
the VISU software system, where the algorithms for the optimization of the surface
of the EOM model were implemented, would mean to provide formal evidence that
the EOM model shown is correct for each possible set of MR images used as input.
Another possibility for verification would be to compare the current EOMs of the
patients, who were used to get the MR images, with the calculated EOM model in
VISU. But since this is not possible, a verification is also not possible. Additionally,
it is not feasible to develop a mathematical model which describes the whole EOM
generation process in VISU. The conclusion is that only a validation of the created
EOM model is possible. Therefore, it has to be shown that the generated EOMs
match with the MR images which served as an initial point for the creation of the
models.
6.1
Comparing Reconstructed Volumes with MR Images
The output of VISU, namely the EOM model, is compared with the MR images
which were the starting point for the creation. Therefore, the images which are
available in the DICOM format are converted into bitmap files. Only those images
used for the creation of the EOM model are converted and stored by VISU together
with the original files. After the EOM model is created in VISU, it is displayed on
the screen using OpenGL (cf. [Claussen, 1997]). Screenshots of the EOM models
are made which show one reconstructed model with the eyes in primary position
50
Chapter 6. Validation
together with the bitmaps showing the MR images. Each screenshot shows one MR
image representing one cross-section of a muscle and is overlapped with the EOM
model at the corresponding cross-section of the muscle. Figure 6.1 shows an EOM
model of the med. rec. muscle of the right eye without any overlapping MR images.
The same model including one specific MR image is shown in Figure 6.2. The model
was properly scaled in order to provide a better perspective of the overlapping MR
image. The muscle model is only partially visible because the rest is covered by the
MR image.
Figure 6.1: EOM model without overlapping MR images
Figure 6.2: EOM model with overlapping MR images
The next step is to compare screenshots of the EOM model and screenshots of the
51
DXF EOM model both in different positions together with the accompanying MR
images. The reason is that one DXF EOM model (cf. [Pirklbauer, 2001]) provides
the input data for one EOM model. However, since this model cannot change
its innervation, a second EOM model is needed. This is generated from a second
DXF EOM model showing the muscle with the eyes in a different position as in
the first DXF EOM model. VISU can now interpolate between these two EOM
models by changing the innervation and this results in the final EOM model which
is visualized. When one DXF EOM model of the muscle and the EOM model with
the same innervation as the DXF EOM model both comply with the MR images,
one part of the validation is finished with respect to the matching of the MR images
to the optimized EOM model.
The models shown in Figures 6.3 and 6.4 are magnified in Figures 6.5 and 6.6
for better comparison. The models are shown only in wireframe mode, because
otherwise the contrast to the MR image would not be very good, with the result
that the differences would be harder to evaluate. Figures 6.3 and 6.4 as well as
the magnified Figures 6.5 and 6.6 show the med. rec. muscle of the right eye in
frontal view, whereas the following pictures show this muscle together with the
accompanying EOM models from the side for the sake of better analyzing of the
differences.
It can be seen that the EOM model in Figure 6.6 matches very well with the black
contours of those parts of the MR images that represent the muscle. The reason
the boundaries of both models in Figures 6.5 and 6.6 don’t match exactly with the
darker parts of the MR images is that during the reconstruction process the muscles
on the MR images have to be identified “by hand”. This identification process is not
easy since the muscle in an MR image has to be differentiated from the surrounding
tissue and all this differentiation has to be performed by the user (cf. [Pirklbauer,
2001]). The dynamic EOM model doesn’t match exactly with the DXF EOM model
either, but this is actually quite good because the DXF EOM model only shows a
very coarse EOM model with lots of edges on its surface. Since a real muscle doesn’t
have any edges on the surface either, the dynamic model is now much closer to a
real muscle.
Figures 6.7 and 6.8 as well as the magnified Figures 6.9 and 6.10 show the med. rec.
muscle of the right eye with the MR image, whereby other cross-sections are shown
in order to compare the models with other MR images of the same muscle.
Figures 6.11 and 6.12 as well as Figures 6.13 and 6.14 show another muscle of the
same patient, the sup. rec. muscle. Again, screenshots were taken from the side to
enable a better comparison. In those screenshots the optimized EOM model matches
exactly with the contours defined by the DXF model.
6.2
Comparison with Existing Models
The comparison with existing models focuses on the optimized EOM model and
the EOM model which was implemented in the earlier versions of VISU (the model
52
53
Figure 6.3: DXF EOM model with
overlapped MR images
Figure 6.4: Dynamic EOM model with
overlapped MR images
overlapping MR images, magnified
built by [Lacher, 2001]). All the other models which were described in Chapter 3 do
not match the criteria stated in Table 3.1 and as a consequence are not subject to
comparison. As each of the EOM models is only capable of showing a single muscle
at the same time, one muscle of a series of MR images is chosen and an EOM model
of the optimized version and one of the same muscle in the non-optimized version
are created in VISU. These models are compared based on their surface structure,
their simulation behavior (i.e. which means how they react to a change of the state
54
overlapping MR images
of innervation) and their matching to the MR images.
First, we will compare the surface of the two EOM models, because the optimization
of the surface of the EOM model was the main goal of our work. Figure 6.15 shows
the non-optimized EOM model built from the rec. med. muscle of the right eye.
It can be seen that the surface of the model in Figure 6.15 has a much “bumpier”
structure than the surface of the model in Figure 6.16, which shows the optimized
version of the same EOM model. The conclusion of this is that the surface has been
55
optimized in this particular situation.
The next step is the comparison of the two models according to their simulation
behavior. To accomplish this, screenshots of both models showing the muscle in
different innervation states are compared. Figure 6.18 shows the optimized model
with an innervation of 0 and Figure 6.17 shows the non-optimized model, also with
an innervation of 0. The following Figures 6.19 and 6.20 show both models with an
innervation of 50 and Figures 6.21 and 6.22 show the models with an innervation
Figure 6.15: Dynamic EOM model,
non-optimized version
56
Figure 6.16: Dynamic EOM model, optimized version
of 100. An innervation of 0 means that the muscle is displayed with the eyes in
primary position. An innervation of 50 means that the EOM model is exactly in
the middle between the primary and the secondary position. An innervation of 100
means that the eye is in secondary or tertiary position and the models show the
muscle’s corresponding shape. It can be seen that the optimized EOM model shows
the same behavior as the non-optimized because no heavy aberrations are visible.
This leads to the conclusion that the simulation of the muscle movements, which
was already implemented before the optimization took place, hasn’t changed.
The last comparison covers the compliance of the EOM models (the optimized as
well as the non-optimized) with the MR images. Figures 6.23 and 6.24 show the
med. rec. muscle of the right eye in primary position. Again, the visualization in
wireframe mode was selected for a better determination of the muscle position on
the MR images. Both models match with the surface of the muscle structures on
the MR images. In Figure 6.23 the part of the EOM model, which in Figure 6.24 is
covered by the MR image, is visible through the MR image because the version of
VISU with the non-optimized EOM model includes transparent MR images which
could not be disabled. Nevertheless, a clear conclusion about the compliance of the
MR images with the EOM model can be taken, namely that the optimized model
matches with the MR images.
6.3
Extraction of Muscle Dimension Data
The validation of the extraction of muscle dimension data cannot be realized by
simply comparing pictures. Instead, the extracted data has to be compared with
57
Figure 6.17: EOM model with an innervation of 0
Figure 6.18: Optimized EOM model
with an innervation of 0
real-life dimensions, which means to use data measured on a muscle not by VISU
but by some other program or by hand. A possible way of doing this is to compare
the extracted data from VISU with muscle dimensions data that can be measured
Figure 6.23: Dynamic EOM model, non-optimized version
Figure 6.24: Dynamic EOM model, optimized version
in the MR images. The standard software Adobe Photoshop (cf. [Adobe, 2002]) was
used to measure the pixel dimensions of the muscles shown in the MR images. The
MR images for the comparison were taken from screenshots (cf. Figure 6.25) from
58
VISU because in those images the muscle of interest was already segmented by a
surrounding polygon and the threshold was changed for better differentiation of the
tissues surrounding the muscle (cf. [Pirklbauer, 2001]). The screenshots were taken
with the MR images in their real dimensions, not in the magnified version seen in
Figure 6.25.
Figure 6.25: Screenshot of the selected muscle in the MR images
The measured pixel dimensions were then recalculated to receive the dimensions of
the muscle in millimeters which is the output unit of VISU concerning the muscle
data. The factor for the recalculation of the pixels was achieved from an additional
file which is already stored by VISU during the creation of the DXF model. For the
measurement the muscle visualized in Figure 6.1 was chosen. The measured lengths
and heights from eleven MR images are listed in Table 6.1, whereby the height of the
muscle is the longest dimension of the muscle on the current MR image in vertical
direction (the axis that develops vertically in the sagittal plane) and the width is the
dimension of the muscle in horizontal direction (the axis that develops horizontally
in the coronal plane). The space between the images is 1.8 mm. This value is
gathered from the file generated by VISU. Therefore, the length of the muscle that
can be displayed according to the MR images is 10 * 1.8 mm = 18 mm.
59
muscle image
cross-section
1
2
3
4
5
6
7
8
9
10
11
60
height
MR VISU
9.4
8.44
9.4
8.48
9.4
9.07
9.8
9.17
9.8
9.51
10.2
9.70
9.8
9.87
9.8
9.83
9.4
9.77
9.4
9.63
9.4
9.59
width
MR VISU
5.1
4.94
4.7
4.14
4.3
4.43
4.7
4.46
4.3
4.35
4.3
4.27
4.3
4.12
3.9
4.00
3.5
3.83
3.9
3.78
3.9
3.96
length
MR VISU
0.0
0.0
1.8
1.41
3.6
3.19
5.4
5.43
7.2
6.99
9.0
8.77
10.8 10.84
12.6 12.20
14.4 14.40
16.2 16.17
18.0 17.92
Table 6.1: Comparison of muscle dimension data in millimeters, measured form the MR images
and extracted from VISU
Table 6.1 also shows the values that were generated by VISU for use in SEE++.
Although it is rather difficult to determine the contours of the muscle in the MR
images, a good extraction of the dimensions has been achieved. It can be seen that
most of the values are within the range of at least +/- 1 mm.
Chapter 7
Software Development Aspects
The development of the different algorithms in this thesis proved to be rather difficult, but this, of course, is inherent in the nature of algorithms. One part of the
accomplished work that really appeared to be more difficult than one could think in
the first place was the inclusion of the algorithms into the existing system structures
of VISU. The reason for this is that the design of VISU is not very good concerning
the exchangeability and the extensibility of existing components. This led to a lot
of troubles during the inclusion of the methods for optimization.
7.1
Process View
The project VISU began as a student project at the Upper Austria University of
Applied Sciences in Hagenberg. It was created in order to combine the functionality
of two thesis works as well as to add data storage for patient specific information.
In the end VISU was a fully functional program which had to be extended by the
functionality of optimizing the EOM model. The connection to the SEE++ software
system was made by providing data of the dimensions of the EOMs in the form of
a pre-defined file. The whole VISU project is encompasses research and hence will
be continued as a research project.
The design of the classes that had to be added to the existing design of VISU for
implementing the optimization was kept very simple. The idea was to provide methods for interpolation and optimization, respectively, that could easily be exchanged.
This was necessary because, as already mentioned at the beginning of the chapter,
the design of VISU has severe deficits concerning the strict separation of the different components from each other. The only way of changing the methods for the
generation of an EOM was to add the classes for optimization into the existing code
in VISU. However, although this was not the best solution, it was the only way to
solve this problem within the time that was available. A better way to improve
VISU would be to redesign the whole VISU software application and consider the
presented techniques for optimization already in the design.
61
Chapter 7. Software Development Aspects
7.2
Product View
The purpose of this thesis was the development of algorithms for the optimization
of an existing three-dimensional EOM model as well as the generation of muscle
data for SEE++, and all this within an existing software system. From a product
standpoint, a concrete product was not created; instead, an existing product was
improved by adding functionality. The product engineering which was practiced
in this thesis did not differ very much from the traditional product creation cycle.
However, there were some differences. Since the planning didn’t include the planning
of a whole software system, only the planning of the adaptation of the optimization
had to be considered. The same applies to the execution of the planned optimization
methods. They were adapted to the requirements that emerged out of the fact that
a current system had to be improved. As already stated, since the optimization of
the EOM model was the forefront, checking of the optimization methods was the
main criterion. Every time a possible solution was found for the optimization, the
review of the newly generated output was the criterion that gave the input for the
next step in the development of the optimization. The last topic, which, of course,
relates to the previous criteria, was the validation—also an important criterion in
all product development. With this the whole set of additional methods could be
validated.
62
Chapter 8
Conclusion
The last chapter in this thesis provides a summary of the goals which have been
achieved. It also outlines possible future improvements and enumerates open problems.
8.1
Goals Achieved
The main goal of this thesis was the improvement of a three-dimensional eye muscle
model. This goal was achieved by solving the following key problems:
• extraction of MR imaging data from DICOM files,
• determination of the correct spline interpolation method,
• preparation of the DXF model data,
• inclusion of the optimized model into VISU,
• extraction of muscle dimension data for SEE++.
The first point was achieved with the help of the DICOMBox component. Behind this component lies a framework written in the programming language Delphi
(cf. [Heijlsberg, 1995]). The important methods for the extraction of the needed
data were included in a wrapper class written in C++ in the Borland C++ Builder
environment and then packed into a Borland C++ Builder component. The component can be extended by adding additional functionality to extract more information
out of a DICOM file.
The second point in the list above was solved with the help of a test application. In
this application the methods for interpolation improvement were tested to evaluate
their abilities with respect to an optimal interpolation method for the underlying
problem of generating a surface for an EOM model. The result was to choose the
NURBS interpolation method because of its high parametrization abilities. This
interpolation method was finally implemented.
63
Chapter 8. Conclusion
The preparation of the DXF model data was again a main part of the work for this
thesis, because of the inconsistency of the received data. Therefore, methods for
restructuring the received points (coming from the DXF model) had to be introduced
using the evaluation of the variance.
A point that appeared to be more difficult than assumed in the first place was the
inclusion of the optimized EOM model into VISU. The main drawback was the
change of the visualization which made the introduction of different data structures
into already existing code necessary. However, after some experimentation the visualization finally appeared to be satisfying.
The last point on the list was rather easy to achieve because the dimensions of the
EOM model just needed to be extracted to an external file since they were already
available in VISU.
Figure 8.1 shows the VISU software system in the optimized version. As an example
the rec. med. muscle of the right eye is shown.
Figure 8.1: Screenshot of the rec. med. muscle and the corresponding MR images in VISU
64
Chapter 8. Conclusion
8.2
Possible Future Improvements
Possible improvements in the future will concentrate on redesigning VISU. Requirement that were suggested by users are the extension of the EOM model with respect
to the inclusion of all six EOMs and eventually the orbita as well as the bulbus.
This would provide a better overview of the anatomical circumstances which means
that the interaction of the six EOMs with each other as well as the bulbus and the
orbita could be evaluated. Another improvement would be to find a solution for a
better differentiation of the EOMs from the surrounding tissue. This was already
partially achieved by using MR images that have a higher resolution and smaller
spaces between each other. A further point is the calculation of the two oblique
EOMs. VISU offers no possibility for the generation of a DXF model for these two
muscles because no suitable MR data is available by standard routines. Therefore,
other methods for calculating a DXF model out of MR images which were not taken
from a coronal or sagital view but from views that have an angle that lies between
these two planes have to be found.
8.3
Open Problems
The biggest problem is the design of VISU, because it prevents a flexible exchange
of functionality. This inflexibility develops throughout all components of VISU and
was one of the main problems for the inclusion of the methods for optimization.
Therefore a redesign of VISU is strongly recommended. Another problem was the
preparation of the DXF data, namely the improving of cross-sections with points
from cross-sections that have a better variance. Since some slices received points
from cross-sections that were situated before them, the mapping to the MR images
could not be 100% exact. Thus some, although very small, differences were present.
However, these differences didn’t influence the anatomy of the EOM model in a wide
range since there were also other strong differences in the DXF model which still
have to be eliminated. At first the inclusion of the optimized EOM model into the
existing MR processing component also was afflicted with problems. One was the
possibility of choosing the number of processed cross-sections by the user as well as
the determination of the spline points for one cross-section. However, this makes
only sense for reducing calculational effort, but since today’s computer technology
develops in such a fast way, the reduction of calculation time may not play a big
role in this part of the program in the future.
65
Literature
[ACR-NEMA, 1988] ACR-NEMA (1988). DICOM Version 2.0. ACR-NEMA 300-1988, Digital Imaging
and Communications, Engineering Department, National Electrical Manufacturers Association, 2101 L
Street, N.W. Suite 300 Washington, D.C. 20037 USA.
[Adobe, 2002] Adobe, S. I. (2002). Adobe Photoshop 7.0. http://www.adobe.com.
[Andress, 2001] Andress, H. (2001).
Glossar zur Datenerhebung und statistischen Analyse (Glossary for Data Gathering and Statistical Analysis; in German).
http://www.homes.unibielefeld.de/hjawww/glossar, University of Bielefeld, Germany.
[Bézier, 1977] Bézier, P. (1977). Essay de Définition Numérique des Courbes et des Surfaces Expérimentales
(Essay on the Numerical Definition of Experimental Curves and Surfaces; in French). PhD thesis,
University of Paris VI.
[Blausen, 2003] Blausen Medical Communications. (2003).
Eye-muscles
http://www.blausen.com/BMChom.html, Blausen Medical Communications.
and
Nerves
3D.
[Borland, 2002] Borland Software Corporation. (2002). Borland C++ Builder. http://www.borland.com.
[Buchberger and Kaltofen, 2002] Buchberger, M. and Kaltofen, T. (2002). Software Engineering Environment for Knowledge-based Interactive Eye Motility Diagnostics. http://www.see-kid.at, Upper Austrian
Research GmbH, Hagenberg, Austria.
[Buchberger and Kaltofen, 2003] Buchberger, M. and Kaltofen, T. (2003). Ophtalmologic Diagnostic Tool
Using MR Images for Biomechanically Based Muscle Volume Deformation. In M. Sonaka, J. M. F.,
editor, Proceedings of SPIE Medical Imaging 2003: Image Processing, volume 5032, pages 60–71, SPIE,
Bellingham, WA, 2003.
[Buchberger et al., 2002] Buchberger, M., Kaltofen, T., Priglinger, S., and Hörantner, R. (2002). Construction and Application of an Object-oriented Computer Model for Simulating Ocular Position Defects. In
Proceedings of the International Symposium of the German Ophthalmological Society, Heidelberg, Germany, Department of Ophthalmology, University of Heidelberg, Germany.
[Buchberger and Mayr, 2000] Buchberger, M. and Mayr, H. (2000). SEE-KID: Software Engineering Environment for Knowledge-based Interactive Eye Motility Diagnostics. In Proceedings of the International
Symposium on Telemedicine, Gothenburg, Sweden, pages 67–79, Department of Medical Software Engineering, Upper Austria University of Applied Sciences, Hagenberg, Austria.
[Center for Human Simulation, 2001] Center for Human Simulation, U. o. C. (2001). The Visible Human
Project. http://www.nlm.nih.gov, Center for Human Simulation, University of Colorado, Keystone,
Colorado.
[Claussen, 1997] Claussen, U. (1997). Programmieren mit OpenGL (Programming with OpenGL; in German). Springer Verlag, Berlin, 1nd edition.
[Damadian, 1971] Damadian, R. (1971). Tumor detection by nuclear magnetic resonance. Science.
[Degen, 2001] Degen, C. (2001). Das Praxisbuch zu AutoCAD 2002 (The Exercise Book for AutoCAD
2002; in German). SmartBooks Publishing AG, 1st edition.
[Demer et al., 1997] Demer, J., Miller, J., Poukens, V., Vinters, H., and Glasgow, B. (1997). Evidence for
Fibromuscular Pulleys of the Recti Extraocular Muscles. Invest. Ophthalmol. Vis. Sci., 36:1125–1136.
[Dössel, 2000] Dössel, O. (2000). Bildgebende Verfahren in der Medizin: Von der Technik zur Medizinischen Anwendung (Imaging Methods in Medicine: From Technics to Medical Application; in German).
Springer-Verlag, Berlin Heidelberg, Germany, 2nd edition.
66
Literature
[Ellman, 2002] Ellman, T. (2002). Modeling and Animation. Lecture 2 handout winter semester 2002,
Computer Science, Vassar College, Poughkeepsie, New York.
[Gamma et al., 1995] Gamma, E., Helm, R., Johnson, R., and Vlissides, J. (1995). Design Patterns: Elements of Reusable Object-oriented Software. Addison-Wesley, 2nd edition.
[Haslwanter, 2003] Haslwanter, T. (2003). Biophysik und Signalverarbeitung Sensorischer Systeme (Biophysics and Signal Processing of Sensoric Systems; in German). Die Physik Sensorischer Systeme ein
Überblick, Lecture notes summer semester 2002, Upper Austria University of Applied Sciences, Hagenberg, Austria.
[Hearn and Baker, 1997] Hearn, D. and Baker, M. P. (1997). Computer Graphics - C Version. Prentice
Hall, USA, 2nd edition. Section 10-8.
[Heijlsberg, 1995] Heijlsberg, A. (1995). Delphi. http://www.borland.com/delphi/.
[Hoehne et al., 2003] Hoehne, K., Pflesser, B., Pemmert, A., et al. (2003). The Visible Human Project.
DVD-ROM for Windows 95/98/NT/2000/ME/XP, English and German, Institute of mathematics and
computer science in medicine, Universitätsklinikum Hamburg-Eppendorf, Germany.
[Kaltofen, 2002] Kaltofen, T. (2002). Design and Implementation of a Mathematical Pulley Model for
Biomechanical Eye Surgery. Master’s Thesis, Upper Austria University of Applied Sciences, Hagenberg,
Austria.
[Kaufmann, 1995] Kaufmann, H. (1995). Strabismus (in German). Enke Verlag, Stuttgart, Germany, 2nd
edition.
[King, 2000] King, M. M. (2000). Module nr. 2: Basic Principles of MRI. http://www.erads.com.
[Krewson, 1950] Krewson, E. (1950). The Action of the Extraocular Muscles. Transactions of the American
Ophthalmological Society, 48:443–486.
[Krug, 2000] Krug, W. (2000). ezDICOM Software. http://www.psychology.nottingham.ac.uk/staff/cr1/
ezdicom.html.
[Lacher, 2001] Lacher, M. (2001). A Volume Preserving Model for Biomechanically-based Muscle Volume
Deformation. Master’s Thesis, Upper Austria University of Applied Sciences, Hagenberg, Austria.
[Miller, 1989] Miller, J. (1989). Functional Anatomy of Normal Human Rectus Muscles. Vision Res,
29(2):223–240.
[Miller and Demer, 1995] Miller, J. and Demer, J. (1995). Magnetic Resonance Imaging of the Functional
Anatomy of the Superior Oblique Muscle. Invest Ophthalmol Vis Sci, 36(5):906–13.
[Miller et al., 2003] Miller, J., Demer, J., Poukens, V., Pavlovski, D., Nguyen, H., and Rossi, E. (2003).
Extraocular Connective Tissue Architecture. Vision Res, 3(3):240–251.
[Miller and Robinson, 1984] Miller, J. and Robinson, D. (1984). A Model of the Mechanics of Binocular
Alignment. Comput Biomed Res, 17(5):436–470.
[Miller and Shamaeva, 1993] Miller, J. and Shamaeva, I. (1993). Orbit 1.0 Gaze Mechanics Simulation
User’s Manual. Eidactics, Suite 404; 1450 Greenwich Street; San Francisco, CA 94109; USA.
[Miller and Demer, 1999] Miller, J. M. and Demer, J. L. (1999). Clinical Applications of Computer Models
for Strabismus. In Rosenbaum, A. L. and Santiago, A. P., editors, Clinical Strabismus Management:
Principles and Surgical Techniques. W. B. Saunders, Philadelphia, USA.
[Morneburg, 1995] Morneburg, H. (1995). Bildgebende Systeme für die Medizinische Diagnostik (Imaging
Systems for Medical Diagnostics; in German). Publicis-MCDVerlag, Munich, Germany, 3rd edition.
[Piegl and Tiller, 1980] Piegl, L. and Tiller, W. (1980). The NURBS Book. Springer, Germany, 2nd edition.
[Pirklbauer, 2001] Pirklbauer, F. (2001). 3D-Rekonstruktion Extraokulärer Augenmuskeln aus MRBilddaten (3D-Reconstruction of Extraocular Eye Muscles; in German). Master’s thesis, Upper Austria
University of Applied Sciences, Hagenberg, Austria.
[Prata, 1998] Prata, S. (1998). C++ Primer Plus. The Waite Group’s, 3nd edition.
[Press et al., 2002] Press, W., Vetterling, W., Teukolsky, S., and Flannery, B. (2002). Numerical Recipes
in C++. Cambridge University Press, UK, 2nd edition.
[Robinson, 1975] Robinson, D. (1975). A Quantitative Analysis of Extraocular Muscle Cooperation and
Squint. Investigative Ophthalmology and Visual Science, 14(11):801–825.
67
Literature
[Simonsz et al., 2002] Simonsz, H., Spekreijse, H., van der Helm, F., Willekens, B., van Keulen, A., van den
Schutte, S., and Bedem, S. (2002). The Development of a Finite-element Model for Orbital Mechanics.
In Proceedings of the International Symposium of the German Ophthalmological Society, Department of
Ophthalmology, University of Heidelberg, Germany.
[Sobotta, 1997] Sobotta (1997). Atlas der Anatomie des Menschen (Atlas of the Human Anatomy; in
German). Urban und Schwarzenberg Verlag, Munich, Germany, 20th edition.
[Thömke, 2001] Thömke, F. (2001). Augenbewegungsstörungen: Ein klinischer Leitfaden (Interferences of
the Eyes: A Clinical Guideline; in German). Georg Thieme Verlag, Stuttgart, Germany, 2nd edition.
68

Surface Optimization of a Three-Dimensional Eye Muscle - SEE-KID

Transcription

Documents pareils

Link to the AGENDA

muscle mass gainer

Summer 2014 Starts June 23 Été 2014 Début 23 juin - YM-YWHA

Trial Description

Lloyds Yacht Application

Cellular aspects of breast muscle development in chickens with high

Auswirkungen eines Power Plate-Trainings im Vergleich zu einem

Dsi Xl Geant Casino

Optogenetic skeletal muscle-powered adaptive biological machines

A phase I trial of adeno-associated virus serotype 1