Accès et adaptation de contenus multimédia pour les

Transcription

No d’ordre 06 ISAL 0056
Année 2006
Thèse
Accès et adaptation de contenus multimédia pour les systèmes pervasifs
Présentée devant
L’Institut National des Sciences Appliquées de Lyon
Pour obtenir
Le grade de docteur
Ecole Doctorale: Informatique et Information pour la Société (EDIIS)
Spécialité: Documents Multimédia, Images et Systèmes D’Information
Communicants (DISIC)
Par
GIRMA BERHE HAGOS
Soutenue le 25 Septembre 2006 devant la Commission d’examen
Composition du jury
Président:
Günter Haring, Professor, University of Vienna, Austria
Rapporteur:
Harald Kosch, Professeur, Université de Passau, Allemagne
Hervé Martin, Professeur, LSR, Université Joseph Fourier
Examinateur:
Günter Haring, Professeur, Université de Vienne, Autriche
Ahmed Mostefaoui, Maître de conférences, (LIFC) UFR-STGI, Montbéliard
Directeur de thèse:
Lionel Brunie, Professeur, INSA de Lyon
Co-directeur de thèse: Jean-Marc, Pierson, Professeur, IRIT, Toulouse
Ordering number 06 ISAL 0056
Year 2006
Thesis
Access and adaptation of multimedia content for pervasive systems
Submitted to
National Institute of Applied Sciences of Lyon
In fulfilment of the requirements for
Doctoral Degree
Doctoral school of Computer Science and Informatics for Society (EDIIS)
Specialized in Multimedia Documents, Images and Communication Information Systems
Prepared by
GIRMA BERHE HAGOS
Defended on 25 September 2006 in front of the Examination Committee
Committee Members
President:
Reviewer:
Examiner:
Supervisor:
Co-supervisor:
Harald Kosch, Professor, University of Passau, Germany
Hervé Martin, Professor, LSR, University Joseph Fourier
Ahmed Mostefaoui, Associate Professor, (LIFC) UFR-STGI, Montbéliard
Lionel Brunie, Professor, INSA de Lyon
Jean-Marc, Pierson, Professor, IRIT, Toulouse
In memory of my father
Acknowledgements
Many people assisted in the preparation of this thesis both directly and indirectly. I would like
to thank all of them for their contribution and support.
First and foremost, I would like to forward my gratitude to my first advisor Prof. Lionel
Brunie for accepting me as one of his PhD students. His valuable guidance and support was
very important for the progress of my study. I always had very interesting and important
discussions with him which contributed a lot for my research. His help was not limited to
academic works, his friendly and lovely characteristics made me to spend very memorable
events with him and his family. In this regard I do not want to pass without mentioning the
hospitality I have got from his wife Benedicit and their children. I would like to thank them.
My thanks also go to my second advisor Prof. Jean-Marc Pierson. He provided me with a lot
of ideas and concepts to realize my research. He was always there to help me advance my
study and provided me valuable ideas to bring up the important issues in my research and
finally enable me to publish many articles. I would also like to thank his family for the
hospitality they provided me during my stay in France.
Thanks should be given to French Embassy in Ethiopia, Addis Ababa and Dr. Dawit Bekele
for establishing a very good contact and cooperation between the Department of Computer
Science in AAU and INSA de Lyon that made possible for Ethiopian Students including me
to pursuit our PhD with the financial support of the Embassy. I would like to thank the
Embassy for the support and request for their cooperation in the future.
I would like to thank Prof. Harald Kosch and Prof. Hervé Martin for reviewing my work and
being member of the examination committee. I would also like to extend my thanks to Prof.
Günter Haring and Dr. Ahmed Mostefaoui who are also members of the examination
committee.
I wish to thank all my colleagues and staffs in the LIRIS/INSA with whom I have had very
good scientific discussions and have spent very memorable times: Solomon Atnafu, Dejene
Ejigu, Marian Scuturici, Faiza Najjar, Tarak Chaari, Ludwig Seitz, David Coquil, Yonny
Cardenas, Ny-Haingo Andrianarisoa, Rachid Saadi, Julien Gossa, Yassir Fawaz, Addisalem
Negash, Talar Atechian, Kader Keita, Rami Rifaieh. I would like to thank undergraduate
computer science students Yann Gripay and Christope Litzinger for their participation in the
development of the prototype to validate my research. I would also like to thank Frederique
ex-secretary of Ecole Doctoral, Mabrouka secretary of Ecole Doctoral and Mme. Ripon
secretary of LIRIS/INSA.
I would like to thank all my family and friends in Europe, America and Ethiopia for their
indispensable encouragement they provided me to work hard on my research. Lastly but not
least, my thanks go to all the Ethiopian family leaving in France for their enjoyable time they
have given me.
Remerciements
De nombreuses personnes m'ont assisté pour la préparation de cette thèse, à la fois
directement et indirectement. Je voudrais les remercier tous pour leurs contributions et leur
soutien.
Tout d'abord, je voudrais tout particulièrement transmettre ma gratitude à mon directeur de
thèse Pr. Lionel Brunie pour m'avoir accepté comme doctorant. Ses conseils précieux et son
soutien ont été très importants pour la progression de ma thèse. J'ai toujours eu avec lui des
discussions importantes et très intéressantes qui ont beaucoup contribué à mon travail de
recherche. Son aide ne s'est pas limitée au travail académique, son amitié et sa gentillesse
m'ont fait passer des moments vraiment inoubliables avec lui et sa famille. A cet égard, je
tiens à mentionner l'hospitalité que j'ai reçue de la part de sa femme Bénédicte et de leurs
enfants. Je souhaite les remercier tous.
Mes remerciements s'adressent également à mon deuxième directeur de thèse Prof. Jean-Marc
Pierson. Il m'a apporté beaucoup d'idées et de concepts pour la réalisation de mes recherches.
Il a toujours été présent pour m'aider à avancer dans mon travail et m'a apporté de précieuses
idées pour aborder les points importants de mes recherches, et m'a finalement permis de
publier de nombreux articles. Je voudrais également remercier sa famille pour leur hospitalité
à mon égard durant mon séjour en France.
Je tiens à remercier l'Ambassade de France en Éthopie, à Addis Ababa, ainsi que le Dr. Dawit
Bekele pour avoir établi de très bons contact et une très bonne coopération entre le
Département d'Informatique de l'Université d’Addis Ababa et l'INSA de Lyon, qui ont rendu
possible pour des Étudiants Éthiopiens, dont je fais partie, de poursuivre notre thèse avec le
soutien financier de l'Ambassade. Je voudrais remercier l'Ambassade pour leur soutien et
souhaiterais poursuivre cette collaboration pour l'avenir.
Je voudrais remercier Pr. Harald Kosch and Pr. Hervé Martin pour avoir évalué mon travail et
pour être membres du jury de thèse. Je voudrais également élargir mes remerciements au Pr.
Günter Haring et au Dr. Ahmed Mostefaoui qui sont également membres du jury de thèse.
Je souhaite remercier tous mes collègues et autres membres du LIRIS à l'INSA de Lyon avec
qui j'ai eu des discussions scientifiques de très bon niveau et ai passé des moments très
agréables : Solomon Atnafu, Dejene Ejigu, Marian Scuturici, Faiza Najjar, Tarak Chaari,
Ludwig Seitz, David Coquil, Yonny Cardenas, Ny-Haingo Andrianarisoa, Rachid Saadi,
Julien Gossa, Yassir Fawaz, Addisalem Negash, Talar Atechian, Kader Keita, Rami Rifaieh.
Je voudrais remercier les étudiants en Informatique Yann Gripay et Christophe Litzinger pour
leur participation au développement du prototype afin de valider mes recherches. Je voudrais
également remercier Frederique, ancienne secrétaire de l'École Doctorale, Mabrouka,
secrétaire de l'École Doctorale, et Mme Ripon, secrétaire du LIRIS à l'INSA de Lyon.
Je voudrais remercier toute ma famille et mes amis en Europe, en Amérique et en Éthiopie
pour leurs encouragements indispensables qu'ils m'ont donnés pour que je puisse travailler dur
sur mes recherches. Enfin, mes remerciements s'adressent à toutes les familles éthiopiennes
vivant en France pour les bons moments que nous avons passés ensemble.
Résumé
L’information numérique et les réseaux informatiques occupent aujourd'hui une place centrale dans
l’ensemble des composants de la vie professionnelle et, même, personnelle. Le très rapide
développement de l'informatique mobile (téléphones mobiles, assistants personnels, ordinateurs
portables…) permet aujourd'hui d'envisager la mise en œuvre de systèmes d'information ubiquitaires :
quelle que soit sa localisation, l'utilisateur dispose, à tout moment, d'un accès au réseau mondial (accès
“anywhere - anytime”). L'enjeu des systèmes pervasifs (ou ubiquitaires) est, dans cette perspective, de
fournir les cadres méthodologiques et les mécanismes protocolaires à même de permettre une
utilisation fiable, pertinente et efficace de ces systèmes.
La gestion des données en représente clairement l’une des composantes-clefs, recouvrant de nombreux
aspects: extensibilité du système, accès et intégration des données (en termes d’indexation et
d’exécution de requêtes dans un environnement distribué fortement hétérogène), optimisation de
l’utilisation des ressources et de la qualité de service (bande passante réduite, discontinuité du service),
cohérence de l’information (l’utilisateur pouvant se trouver en mode déconnecté, comment assurer la
cohérence et la pérennité des données qu'il manipule ?). Enfin, l'adaptation de contenus multimédias
constitue certainement l'un des mécanismes centraux dans la mise en œuvre effective des systèmes
d'information pervasifs. La très grande hétérogénéité des moyens de connexion et des conditions de
communication (bande passante notamment), la très grande variabilité des préférences utilisateur
imposent en effet de fournir des outils d'adaptation des contenus transmis aux utilisateurs. Dans ce
contexte, beaucoup de travaux de recherche ont été effectués que l’on peut classer en trois approches
principales selon le site d’exécution des mécanismes d'adaptation.
Dans une première approche, dite approche “orientée client”, le dispositif du client est responsable
d'adapter le contenu selon les caractéristiques du dispositif, les préférences de l'utilisateur et le
contexte opérationnel (e.g. bande puissante disponible). Néanmoins, en raison de la capacité des
performances limitées des dispositifs client, cette approche est difficile à mettre en place et souvent
impraticable.
Dans l’approche “orientée serveur”, le serveur de contenus fournit lui-même des fonctionnalités
d'adaptation. Cette approche a comme conséquence une consommation accrue des ressources du
serveur, ralentissant de facto le traitement des requêtes.
Enfin, alternative aux approches précédentes, l’approche “orientée proxy” place un proxy entre le
client et le serveur de contenus. Ce proxy est alors responsable de l'adaptation des contenus avant
qu'ils soient fournis à l'utilisateur. Même si cette approche résout à un certain degré les problèmes
II
rencontrés avec les approches orientées serveur et client, elle présente des limitations en termes
d’extensibilité en raison du coût élevé des opérations d’adaptation. En outre, les techniques
d'adaptation évoluent et l’intégration de ces nouvelles techniques au proxy peut s’avérer difficile,
celui-ci n’étant pas extensible.
Dans cette thèse, nous présentons une architecture logicielle d'adaptation de contenus multimédia
(nommée DCAF: Distributed Content Adaptation Framework). Cette architecture est fondée sur
l’utilisation de services d'adaptation accessibles sur l'Internet. A l’inverse des approches précédentes
où le client, le serveur de contenu ou un proxy est responsable du processus d'adaptation, ce sont ici
des services développés extérieurement qui effectuent la transformation des contenus. DCAF fournit
ainsi une solution générique d'adaptation de contenus offrant flexibilité, extensibilité et
interopérabilité.
La principale problématique introduite est la détermination du scénario optimal d’adaptation
(également appelé chemin optimal d'adaptation), plusieurs services d'adaptation pouvant fournir les
mêmes fonctionnalités. Cette thèse propose d’adopter une technique issue de l’intelligence artificielle
pour optimiser la planification, la composition et l’exécution des services impliqués dans l’adaptation
d’un contenu. L’algorithme proposé prend en compte le profil des utilisateurs, les caractéristiques des
dispositifs, la connectivité du réseau, le format des contenus échangés et les spécifications (coût,
qualité) des services d'adaptation disponibles pour construire un graphe d'adaptation modélisant le
scénario d’adaptation choisi.
Parallèlement, nous avons développé une ontologie dédiée à la description des services d'adaptation.
Cette ontologie permet de décrire la sémantique des services d’adaptation enfin de faciliter la
découverte et la composition des services. Elle permet également de passer facilement de la
description d’un service à la représentation de l’opérateur d’adaptation.
Pour démontrer la faisabilité de l’architecture DCAF, un prototype a été développé et évalué. Les
expériences conduites montrent que le temps de construction du graphe d'adaptation et la durée
moyenne d'adaptation sont compatibles avec les contraints des environnements pervasifs.
Mots-clés : Systèmes Pervasifs, Adaptation de Contenu Multimédia, Composition de Services,
Planification, Contexte.
Abstract
Nowadays, computing technologies are part of our daily life. The advancement of mobile computing
(mobile telephone, PDA, Portable) enables users to access data wherever and whenever they want
them: at work, at home, in a meeting, or even when travelling. In this perspective, pervasive
(ubiquitous) systems provide the necessary tools and protocols to allow reliable, appropriate and
effective content access by heterogeneous users and devices using different network connections.
In pervasive systems, the management of data represents one of the key components covering many
aspects: extensibility of the system, data access and integration (in terms of indexing and query
execution in heterogonous distributed environment), optimization of the use of the resources and
quality of services (low band-width and disconnection), consistency of information (e.g how to ensure
consistency in case of disconnection?). Finally, the adaptation of multimedia contents constitutes
certainly one of the central mechanisms in implementing effective pervasive information systems.
Heterogeneity of network connections and conditions of communication (in particular bandwidth),
variability of user’s preference and heterogeneity and limitation of access devices demand tools of
adaptation to transmit the content to the user. Many research works were carried out to develop
adaptation tools which can generally be categorized into three major approaches based on the location
of the adaptation process. These approaches are: client-based, proxy-based and server-based.
In the first approach, client device is responsible to adapt the contents according to the characteristics
of the device, the preferences of the user and the operational context (e.g. available bandwidth).
Nevertheless, because of limited capacity the client device, this approach is difficult to implement and
often impractical.
In the second approach, the content server provides the functionalities of adaptation. This approach
results in computational load and resource consumption on the server so decreases performance of the
content delivery.
To tackle the problems of both approaches, the third approach uses proxy. In this approach, a proxy
between the client and the content server is responsible for the adaptation of the content before it is
delivered to the user. While to some degree this approach solves some of the problems with the two
approaches, it has limited scalability because of high computational needs of adaptation operations.
Secondly, the list of content adaptation tools is not exhaustive and might include other in the future
and accommodating these new tools may not be possible if especially the proxy is not extensible.
II
Finally, putting all adaptation functionality into the proxy requires a machine with powerful CPUs and
a lot of memory.
In this thesis, we present a software architecture of multimedia content adaptation (called DCAF:
Distributed Content Adaptation Framework). This architecture is based on the use of Internet
accessible adaptation services. Unlike in the previous approaches where the client, content server or a
proxy is responsible for the adaptation process, in this architecture externally developed services carry
out the content transformation. DCAF thus provides a general content adaptation solution offering
flexibility, scalability, extensibility and interoperability.
The principal problem is the determination of optimal service configuration (also called optimal
adaptation path) provided that there are a number of possible service configurations and several
adaptation services can provide the same functionalities. This thesis proposes to adopt an Artificial
Intelligence (AI) technique to optimize the planning, the composition and the execution of the services
to adapt the content. The algorithm proposed takes into account the profile of the user, the
characteristics of the devices, the connectivity of the network, the format of the exchanged contents
and the specifications (cost, quality) of the available adaptation services to build a graph of adaptation
modelling the selected scenario of adaptation.
In parallel, we developed an ontology for the description of adaptation services. This ontology makes
it possible to describe the adaptation services semantically and finally to facilitate the discovery and
the composition of the services. It also makes it possible to translate easily the services description to
planning operator representation.
To demonstrate the feasibility of DCAF, a prototype has been implemented and tested. In addition to
the openness, scalability, flexibility and interoperability of the framework, experiments show that the
average processing time for adaptation services and the adaptation graph construction time suggest
that with some performance enhancement such as optimization of message exchange the service-based
framework is of a reasonable performance and can be used in pervasive computing environments.
Keywords: Pervasive Systems, Multimedia Content Adaptation, Service Composition, Planning,
Context.
Table of Contents
Table of Contents ............................................................................................................I
List of Figures .................................................................................................................I
List of Tables...................................................................................................................I
Résumé étendu ................................................................................................................... 1
1. Introduction ............................................................................................................ 1
2. État de l’art ............................................................................................................. 3
3. Description de métadonnées................................................................................... 7
4. Description des services d’adaptation de contenu multimédia .............................. 8
4.1
Hiérarchie de classes ...................................................................................... 8
4.2
Propriétés des services ................................................................................. 11
5. Proposition d’une architecture d’adaptation de contenu orientée service............ 13
5.1
L’Architecture DCAF .................................................................................. 13
5.2
Module de négociation et d’adaptation de contenu (CNAM) ...................... 15
5.2.1 Générateur du graphe d’adaptation multimédia (MAGG) ....................... 15
5.2.1.1 Définitions et notations utilisées dans MAGG...................................... 16
5.2.1.2 Algorithme ............................................................................................ 19
5.3
Recherche d’un chemin /plan d’adaptation optimal..................................... 26
5.4
Evaluation de performances ......................................................................... 29
6. Implémentation du prototype ............................................................................... 31
6.1
Composants du prototype............................................................................. 31
6.2
Bases de données utilisées dans le prototype ............................................... 35
6.3
Interfaces visuelles ....................................................................................... 37
6.3.1 Interface de saisie du profil client ............................................................ 37
6.3.2 Interface d’affichage de données adaptées............................................... 38
7. Conclusion et perspectives ................................................................................... 40
Chapter 1 Introduction and Motivation...................................................................................... 1
1.1
Introduction ........................................................................................................ 1
1.2
What is Pervasive Computing? .......................................................................... 6
1.3
The Need for Content Adaptation ...................................................................... 8
1.4
Motivation ........................................................................................................ 10
1.5
A Short Overview of Our Approach ................................................................ 12
1.6
Organization of the Thesis ............................................................................... 14
Chapter 2 Related Work on Content Adaptation ..................................................................... 15
2.1
Introduction ...................................................................................................... 15
2.2
Content Adaptation .......................................................................................... 17
2.2.1
Content Adaptation Techniques ............................................................... 18
2.2.1.1 Semantic adaptation .............................................................................. 20
2.2.1.2 Physical adaptation................................................................................ 21
2.2.2
Content Adaptation Approaches .............................................................. 26
2.2.2.1 Server-Based (or Source) Adaptation Approach................................... 26
2.2.2.2 Client-Based (or Receiver) Adaptation Approach ................................ 28
2.2.2.3 Proxy-Based Adaptation Approach....................................................... 30
II
2.2.2.4 Distributed and Agent Based Adaptation Approach ............................. 32
2.2.2.5 Ongoing Standardization for Content Adaptation................................. 35
2.2.3
Content Adaptation Systems .................................................................... 36
2.2.3.1 Commercial/Open Source Products ...................................................... 36
2.2.3.2 Research Prototypes .............................................................................. 40
2.3
Summary .......................................................................................................... 49
Chapter 3 Service-Oriented Architecture & Web Services...................................................... 51
3.1
Introduction ...................................................................................................... 51
3.2
Benefits of Service-Based Development.......................................................... 53
3.3
Web Services: Standards and Related Technologies ....................................... 54
3.3.1
Web Service Languages ........................................................................... 56
3.3.2
Semantic Web Service Description.......................................................... 57
3.4
Web Service Composition................................................................................ 60
3.4.1
Workflow Technique................................................................................ 61
3.4.2
AI planning............................................................................................... 63
3.5
Summary .......................................................................................................... 66
Chapter 4 Content Metadata Description ................................................................................. 67
4.1
Introduction ...................................................................................................... 67
4.2
Dublin Core ...................................................................................................... 70
4.3
SMPTE Meta-Dictionary ................................................................................. 72
4.4
MPEG 7............................................................................................................ 73
4.5
Other standards................................................................................................. 79
4.6
Summary .......................................................................................................... 80
Chapter 5 Context Metadata Description ................................................................................. 81
5.1
Introduction ...................................................................................................... 81
5.2
Composite Capabilities/Preference Profiles..................................................... 84
5.3
IETF Media Feature Sets.................................................................................. 87
5.4
Comprehensive Structured Context Profiles .................................................... 89
5.5
MPEG 21.......................................................................................................... 90
5.6
Summary .......................................................................................................... 94
Chapter 6 Service-Based Content Adaptation.......................................................................... 97
6.1
Introduction ...................................................................................................... 97
6.2
Why Service-Based Content Adaptation?........................................................ 98
6.3
Functional Model of Adaptation System ......................................................... 98
6.4
Overview of the DCAF Architecture ............................................................... 99
6.5
Context and Content Descriptions.................................................................. 101
6.5.1
Context Description................................................................................ 102
6.5.2
Content Description (Metadata) ............................................................. 103
6.6
Multimedia Content Adaptation Services Description................................... 104
6.6.1
Adaptation Service and Multimedia Data Entities................................. 105
6.6.2
Class Hierarchy ...................................................................................... 106
6.6.3
Properties of Entities .............................................................................. 109
6.7
Content Negotiation and Adaptation Module ................................................ 112
6.7.1
Constructing Content Adaptation Graph................................................ 113
6.7.2
Multimedia Adaptation Graph Generator (MAGG)............................... 122
6.7.2.1 Definitions and Notations Used in MAGG ......................................... 122
6.7.2.2 Algorithm to Construct Adaptation Graph.......................................... 125
6.7.3
Searching an Optimal Adaptation Path/Plan.......................................... 130
6.7.4
Performance Evaluation ......................................................................... 133
6.8
Summary ........................................................................................................ 137
Table of Contents
III
Chapter 7 Prototype Implementation ..................................................................................... 139
7.1
Components of the Prototype......................................................................... 139
7.2
Sample Database Used in the Prototype ........................................................ 146
7.3
Visual Interfaces............................................................................................. 148
7.3.1
Client Profile Entry Interface ................................................................. 148
7.3.2
Patient Record Search Interface ............................................................. 151
7.3.3
Adapted Data Display Interface ............................................................. 151
7.4
Summary ........................................................................................................ 155
Chapter 8 Discussion.............................................................................................................. 157
8.1
DCAF Architecture ........................................................................................ 157
8.2
Semantic Adaptation Services Description .................................................... 159
8.3
Adaptation Service Composition ................................................................... 160
8.4
Prototype Implementation .............................................................................. 162
Chapter 9 Conclusion and Future Work................................................................................. 163
9.1
Conclusion and Contributions........................................................................ 163
9.2
Future Work ................................................................................................... 165
Annexes.............................................................................................................................. 169
Annex A: Context Profile........................................................................................... 171
Annex B: Proposed Services Description Ontology .................................................. 183
Annex C: OWL-S Profile Schema ............................................................................. 193
Annex D: Modified Dijkstra’s Algorithm.................................................................. 201
Annex E: Classes Hierarchy of Local Proxy.............................................................. 205
Annex F: Glossary...................................................................................................... 211
Bibliography....................................................................................................................... 213
Bibliography............................................................................................................... 215
Curriculum Vitae........................................................................................................ 237
Liste des publications ................................................................................................. 239
I
List of Figures
List of Figures
Figure 1 : Hiérarchie de classes (services d’adaptation de multimédia) .................................... 9
Figure 2 : Représentation de la classe MultimediaAdaptationServices en OWL....................... 9
Figure 3 : Hiérarchie de classes pour les services d’adaptation d’audio.................................. 10
Figure 4 : Extrait de la représentation de la classe AudioAdaptationServices en OWL .......... 11
Figure 5 : Exemple d’une description de service de transcodage (audio vers texte) .............. 12
Figure 6: Architecture DCAF................................................................................................... 13
Figure 7: Comparaison de différents ordres d'exécution.......................................................... 30
Figure 8: Temps de construction du graphe ............................................................................. 31
Figure 9: Schéma fonctionnel du proxy local .......................................................................... 33
Figure 10: Modèle conceptuel des données ............................................................................. 37
Figure 11: Capture d’écran de l’interface de sélection du matériel. ........................................ 38
Figure 12 (a) : Capture d’écran d’une donnée-image sur un PC standard (sans adaptation)... 39
Figure 12 (b) : Capture d’écran d’une donnée-image sur un Smartphone (après adaptation).39
Figure 1.1: Goal of Content Adaptation................................................................................... 10
Figure 2.1: Server-Based or Source Adaptation....................................................................... 27
Figure 2.2: Client-Based or Receiver Adaptation .................................................................... 29
Figure 2.3: Proxy-Based Adaptation........................................................................................ 31
Figure 3.1: A service-Oriented Architecture (SOA) [Ort05] ................................................... 53
Figure 4.1: Dublin Core Metadata Element Set ....................................................................... 72
Figure 4.2: Scope of the MPEG Standard ................................................................................ 75
Figure 4.3 Schema Definition of VideoSegmentType ............................................................. 76
Figure 4.4: MPEG-7 Description Example .............................................................................. 78
Figure 5.1: CC/PP Context Representation Model .................................................................. 85
Figure 5.2: Example of CC/PP Profile in XML/RDF Format.................................................. 86
Figure 5.3: UAProf Main Components. ................................................................................... 87
Figure 5.3: CSCP Profile Example .......................................................................................... 90
Figure 5.4: A Representation of the MPEG-21 Multimedia Framework [Maga04]................ 92
Figure 5.5: The Adaptation of Digital Items ............................................................................ 94
Figure 6.1: Functional Model................................................................................................... 99
Figure 6.3: Partial Structure of Context Profile Representation ............................................ 102
Figure 6.4 Partial Example of Context Profile....................................................................... 103
Figure 6.5 Sample Multimedia Content Description in XML Format ................................... 103
Figure 6.6: Class Hierarchy of Top-level of Multimedia Content Adaptation Services........ 107
Figure 6.7: OWL Representation of MultimediaAdaptationServices Class........................... 108
Figure 6.9: A Small Excerpt of OWL Representation for AudioAdaptationServices Class .. 109
Figure 6.10: An Example of Service Description for Audio to Text Adaptation Service ..... 111
Figure 6.11: Content Adaptation Process............................................................................... 112
Figure 6.12: Sample Transformation Rules in XML Format................................................. 117
Figure 6.13: An XML Encoding of a Transformation Prescript ............................................ 118
Figure 6.14: A Graphical Representation for the Transformation Prescript .......................... 119
Figure 6.15: An example of Adaptation Graph for Text to Audio Transformation............... 120
Figure 6.16: A Fully-Connected Adaptation Graph............................................................... 121
Figure 6.17: Algorithm for Adaptation Graph Construction (AGC) ..................................... 129
Figure 6.18: Comparison of Different Execution Orders....................................................... 134
Figure 6.20 Example of an adaptation graph ......................................................................... 135
Figure 6.19: Graph Construction Time .................................................................................. 136
Figure 7.1: Components and Interaction of DCAF Architecture ........................................... 141
II
Figure 7.2: Block Diagram of the Local Proxy...................................................................... 144
Figure 7.3: Classes of Local Proxy Implementation. ............................................................. 145
Figure 7.4: Conceptual data model. ....................................................................................... 148
Figure 7.5: Screenshot of the Device Selection Interface ...................................................... 149
Figure 7.6: Screenshot of the New Device Entry Interface.................................................... 149
Figure 7.7: Screenshot of the Network Characteristics Entry Interface................................. 150
Figure 7.8: Screenshot of User Preferences Entry Interface .................................................. 150
Figure 7.9: Screenshot of Patient Record Search Interface.................................................... 151
Figure 7.10: Graphical Assistant: AWD graphical representation......................................... 152
Figure 7.11 (a): Screenshot of an Image data on a Standard PC (without adaptation) ......... 153
Figure 7.12 Screenshot of text data (a) original in French (b) the translated data in English 154
I
List of Tables
List of Tables
Tableau 1 : Propriétés de MultimediaAdaptationServices...................................................... 11
Tableau 2 : Propriétés de la classe MultimediaDatatypeFormat............................................. 12
Table 2.1: Media Types and Content Adaptation Techniques [Lei01 and Shaha01] .............. 19
Table 2.2: Summary of Content Adaptation Systems Features ............................................... 47
Table 2.3: Architectural Characteristics of Adaptation Approaches........................................ 49
Table 6.1: Types of Audio Adaptation Services .................................................................... 105
Table 6.2: Multimedia Data Types......................................................................................... 105
Table 6.3: Properties of MultimediaAdaptationServices ....................................................... 110
Table 6.4: Properties of MultimediaDatatypeFormat Class................................................... 111
Table 6.5: Sample Transformation Rules............................................................................... 115
Table 6.6: Constraint Processing............................................................................................ 117
Résumé étendu
1.
Introduction
Le très rapide développement de l'informatique mobile (téléphones mobiles, assistants
personnels, ordinateurs portables…) permet aujourd'hui d'envisager la mise en œuvre de
systèmes d'information ubiquitaires : quelle que soit sa localisation, l'utilisateur dispose, à
tout moment, d'un accès au réseau mondial (accès “anywhere - anytime”). L'enjeu des
systèmes pervasifs (ou ubiquitaires) est, dans cette perspective, de fournir les cadres
méthodologiques et les mécanismes protocolaires à même de permettre une utilisation fiable,
pertinente et efficace de ces systèmes.
La gestion des données en représente clairement l’une des composantes-clefs, recouvrant de
nombreux aspects : extensibilité du système, accès et intégration des données (en termes
d’indexation et d’exécution de requêtes dans un environnement distribué fortement
hétérogène), optimisation de l’utilisation des ressources et de la qualité de service (bande
passante réduite, discontinuité du service), cohérence de l’information (l’utilisateur pouvant
se trouver en mode déconnecté, comment assurer la cohérence et la pérennité des données
qu'il manipule ?).
Enfin, l'adaptation de contenus multimédias constitue certainement l'un des mécanismes
centraux dans la mise en œuvre effective des systèmes d'information pervasifs. La très grande
hétérogénéité des moyens de connexion et des conditions de communication (bande passante
notamment), l’hétérogénéité des dispositifs (en termes d'affichage, de stockage, de puissance
de calcul, d’environnement logiciel), la très grande variabilité des préférences utilisateur
imposent en effet de fournir des outils d'adaptation des contenus transmis aux utilisateurs.
Considérons le cas d’un médecin de garde dans un établissement de soins à Lyon. Ce centre
est équipé d’un système informatique de gestion de données patients moderne (dossier
médical, images, résultats d’analyses…), connecté à d’autres centres de soins en Europe. Ce
système permet aux médecins, infirmiers, personnels de soins et autres équipes hospitalières
d’accéder aux données relatives aux patients. Le médecin de garde se rend au chevet d’un
2
patient originaire d’un Etat européen tiers. Ce patient présente des signes d’infection
pulmonaire. Avant d’établir une nouvelle prescription, le médecin veut, à partir de son
dispositif sans fil de poche (par exemple : un assistant personnel numérique), consulter le
dossier médical du patient, dans lequel figurent des données multimédias intégrant du texte,
des images, des séquences dynamiques d’images (ex : coronarographies). Il se connecte au
système, récupère les données répertoriées lors des précédents séjours dans d’autres
établissements, note que le patient est, d’une part, déjà sous traitement pour une hypertension
artérielle et des problèmes cardiaques et, d’autre part, allergique à la pénicilline. Il se sert
alors de son dispositif de poche pour consulter une base de données de médicaments en ligne
afin de prescrire un traitement compatible avec les médicaments actuellement observés par le
patient. Le médecin remplit enfin sur l’écran de son assistant personnel un petit formulaire
visant à autoriser et émettre une prescription auprès d’une pharmacie par un réseau sans fil.
Le scénario précédent illustre la mobilité d’un utilisateur, l’hétérogénéité des dispositifs de
connexion, l’hétérogénéité des connexions réseaux utilisées, la grande variété des sources
d’information mises en œuvre stockant des données multimédias (texte, images, séquences
vidéo et audio).
L’accès à l’information, dans ce scénario soulève, en pratique, des problématiques de
compatibilité et de pertinence : compatibilité du format des données avec les formats
supportés par le dispositif de connexion, pertinence vis-à-vis des préférences de l’utilisateur
(ex : langues parlées par le médecin), compatibilité des données transmises vis-à-vis de la
capacité du réseau et du dispositif de connexion, ergonomie de l’affichage sur le dispositif de
connexion.
L'adaptation de contenu est considérée comme une solution efficace et intéressante à cette
problématique [Held02]. Elle consiste en la transformation et la personnalisation du contenu
afin qu’il soit compatible avec les préférences de l‘utilisateur et les contraintes du contexte
d’exploitation [Lum02b, Hori00].
Dans cette thèse, nous proposons un cadre architectural d’adaptation de contenu distribué
fondé sur l’utilisation de services tiers. Les idées à la base de l’architecture proposée sont :
3
État de l’art
d’une part, de placer le moteur décisionnel chargé d’optimiser et de
planifier l’adaptation à mettre en œuvre sur un proxy (mandataire) situé entre
le consommateur et le fournisseur d’informations ;
d’autre part, d’utiliser des services-tiers accessibles sur Internet (services
Web) pour exécuter physiquement l’adaptation ;
enfin, de proposer une ontologie décrivant les caractéristiques
(sémantique, coût, qualité…) des services d’adaptation multimédia. Cette
ontologie est utilisée pour sélectionner et composer les services d’adaptation
les plus appropriés au contexte d’utilisation.
Cette architecture constitue une solution ouverte et générique pour les problèmes d'adaptation
de contenus qui peut être déployée dans le cadre de différentes applications. Afin de valider
la faisabilité de notre proposition, nous avons développé et évalué un prototype ciblé sur
l’accès au dossier médical réparti, domaine d’application privilégié de l’équipe.
Ce résumé étendu est organisé comme suit. Dans la section 2, nous présentons un état de l’art
sur l’adaptation de contenu. Dans la section 3, nous analysons les modèles de métadonnées
utilisés pour la description du contexte d’exécution et du contenu des données. Ces
métadonnées sont essentielles pour identifier les caractéristiques des données et les
contraintes contextuelles et, in fine, décider du nombre et du type d’adaptations ou de
transformations requises. Une ontologie des services d’adaptation multimédia est proposée en
section 4. La section 5 présente notre proposition d’architecture distribuée d’adaptation de
contenus fondée sur des services tiers et en donne une description formelle.
L’implémentation d’un prototype et les résultats expérimentaux sont analysés dans la section
6. Enfin, La section 7 conclut ce document et présente les principales perspectives à nos
travaux.
2.
État de l’art
Depuis l’émergence des systèmes pervasifs, l'adaptation de contenu multimédia a fait l’objet
d’importantes recherches et plusieurs techniques d'adaptation ont été développées. Les
techniques actuellement disponibles reposent principalement sur la transformation textuelle
4
[Nagao01, Hous96, Lilj95], le transcodage d’image [Wee03, Lee03, Chandra00, Smith98] et
le traitement vidéo et audio [Libse04, Shan00, Vetro02, Bolot98].
Une des questions majeures en terme d’architecture d’adaptation est de savoir où se réalise la
prise de décision et, surtout, où s’exécutent les opérations d’adaptation. Trois approches
élémentaires ont été proposées dans la littérature pour la localisation des processus
d’adaptation sur le parcours de transmission des données, entre la source des données
(serveur de contenu) et la destination des données (client final) : (i) au niveau de la source,
(ii) au niveau du destinataire ou client et (iii) au niveau d’un intermédiaire, quelque part sur le
réseau, sur des sites spécifiques présentant des propriétés intéressantes, par exemple, à la
frontière des parties filaire et non filaire du réseau.
Dans la première approche, où l’adaptation est opérée au niveau de la source - approche dite
centrée serveur - [Marg01, Mogul01, Mohan99, Bhrag98, Bolot98], le serveur se charge de
découvrir les capacités du client et la bande passante disponible. Il décide alors de la
meilleure stratégie d’adaptation. Cette approche permet davantage de contrôle car
l’adaptation, liée au traitement du contenu stocké sur le serveur, permet à l’auteur (créateur)
du contenu de formuler des conseils ou des contraintes sur l’adaptation. Un auteur peut ainsi
pré-visualiser et valider le résultat de l’adaptation selon différentes préférences et conditions.
De plus, les solutions centrées serveur permettent la mise en œuvre de mécanismes
d’adaptation de contenu dynamique (à la volée) ou statique (hors ligne). Cependant, ces
approches présentent deux inconvénients majeurs : d’une part, elles imposent aux
fournisseurs de contenu d’intégrer des mécanismes d’adaptation ; d’autre part, les processus
de transformation induisent une charge de calcul et une consommation de ressources
importantes sur le serveur que celui-ci n’a souvent pas la capacité de fournir.
Dans la seconde approche, où l’adaptation est réalisée au niveau du destinataire - approche
dite centrée client - [Marr02, Lei01, Björk99, Yosh97, Fisher97], le processus d’adaptation
est effectué par le terminal client, en fonction des capacités de celui-ci. Le serveur demeure
donc inchangé et transmet ses contenus dans leur format d’origine. Le terminal du client
utilisateur, à la réception du contenu, transforme celui-ci de manière à pouvoir l’afficher
correctement. Deux méthodes d’adaptation peuvent être envisagées : sélection par le terminal
des contenus (ou des versions de contenus) compatibles avec ses caractéristiques, ou
transformation ad hoc effectuée par le terminal lui-même. La sélection des contenus
État de l’art
5
compatibles peut être réalisée dans le cadre d’un protocole de négociation avec les serveurs
de contenus : en réponse à une requête client, le(s) serveur(s) transmette(nt) la liste des
variantes disponibles pour les contenus répondant à la requête ; le terminal client sélectionne
les (variantes des) contenus compatibles (et pertinentes) ; enfin, le(s) serveur(s)
transmette(nt) les contenus sélectionnés. Notons que certains dispositifs clients transforment
par défaut certains contenus ; par exemple, certains terminaux Windows CE (comme le
SmartPhone) modifient la résolution chromatique des images (passage d’une profondeur de
couleur de 24 bits à un niveau de gris sur 4 bits).
L’adaptation centrée client requiert des modifications minimales au niveau du terminal de
connexion et peut être extrêmement efficace dans la personnalisation du contenu et
l’adaptation aux caractéristiques du terminal. Toutefois, cette solution est très mal adaptée
aux situations où les contraintes de transmission réseaux sont prégnantes, ce qui est
typiquement le cas de nombreuses applications pervasives intégrant des dispositifs nomades.
En outre, la complexité souvent élevée des mécanismes d'adaptation interdit l’utilisation
générique de cette solution [Perk01] dans la mesure où une proportion considérable des
dispositifs pervasifs reste limitée en termes de puissance de calcul, de mémoire, d’énergie et
de stockage. Ainsi, les approches centrées client ne sont en pratique pertinentes que vis-à-vis
de problématiques simples d’affichage lorsque les contraintes réseaux sont traitées par
ailleurs [Nand99].
La troisième approche tente de constituer un compromis entre les deux solutions précédentes
; l’adaptation est ainsi exécutée sur un nœud intermédiaire habituellement qualifié de proxy,
(ou mandataire)1. Des exemples de systèmes à base de proxys se trouvent dans [Sing04,
Wee03, Kim03, Chandra00, Fox98a Angin98, Bick97, Noble97]. L’idée de base de cette
approche est la suivante : le proxy intercepte la réponse du serveur, décide et effectue
l’adaptation en fonction des préférences de l’utilisateur, des capacités des terminaux et de
l’état du réseau, puis envoie le contenu adapté au client.
L’adoption d’une solution centrée proxy présente plusieurs avantages. D’abord, la charge de
calcul du serveur de contenus est déplacée vers le proxy. En second lieu, le proxy peut être
positionné au point le plus critique du chemin de données. On considère habituellement que
1
On parle également de passerelle (gateway).
6
la bande passante entre un proxy et un serveur est plus large que celle qui est disponible entre
un client (souvent mobile) et un proxy [Antonio97]. Le placement sur le dernier maillon de la
chaîne permet au proxy de disposer d’une vue globale sur les ressources de l’environnement,
telles que la latence du réseau, la bande passante, la taille du contenu à transporter avant la
livraison finale, les préférences de l’utilisateur, toute modification survenant dans le temps,
etc. Le serveur proxy joue aussi un autre rôle majeur de mise en cache des résultats de
l’adaptation de contenus afin d’éviter des allers-retours vers le serveur de contenu et la
répétition d’opérations de transcodage coûteuses lorsque les ressources peuvent être
accessibles depuis le cache [Singh04, Fox97a].
Les approches orientées proxy présentent aussi des inconvénients. En premier lieu, elles
passent mal à l’échelle du fait de coûts de calcul considérables induits par les opérations
d’adaptation [Lumb02b] imposant des unités de calcul puissantes et beaucoup de mémoire.
En second lieu, le fait de passer par un intermédiaire pour la transmission d’un contenu
soulève des problèmes de sécurité. Le tiers manipulant le proxy doit être digne de confiance
auprès de l’utilisateur et de la source. De plus, le proxy est susceptible de facturer son service
et l’utilisation des ressources sollicitées pour la réalisation de l’adaptation. Des mécanismes
de traçage (accounting) doivent dans cette hypothèse être intégrés au système afin
d’enregistrer la quantité de ressources utilisées et l’usage des données. Enfin, la liste des
outils d’adaptation de contenu n’est pas exhaustive et est amenée à évoluer ; l’intégration de
ces nouveaux outils risque de ne pas être possible si le proxy n’est pas fonctionnellement
extensible.
En résumé, de nombreux efforts ont été menés pour développer différentes approches et
techniques d’adaptation de contenu multimédia dans les environnements pervasifs (cf.
chapitre 2 du manuscrit pour une analyse complète de l’état de l’art). Des produits
commerciaux (IBM Transcoding [IBM00], Web Clipping [3Com], WebSphere [Rodr01],
etc.) et des prototypes issus de travaux de recherche (TranSend [Fox98b], Odyssey
[Noble00], Conductor [Yarvis01], Digestor [Bick99], etc.) ont été parallèlement développés.
Ces solutions ont contribué à l’élaboration de nombreuses stratégies, d’architecture et de
techniques d’adaptation. Cependant aucune ne résout de manière satisfaisante les
problématiques d’interopérabilité, d’extensibilité, de flexibilité, de passage à l’échelle qui
sont des points essentiels dans un environnement pervasif (voir les tableaux 2.2 et 2.3
(chapitre 2) pour les résultats de cette analyse). En outre, ces systèmes sont conçus pour la
Description de métadonnées
7
plupart pour des besoins spécifiques ; les types d’adaptation de contenu qui ont été abordés
sont ainsi principalement focalisés sur la transformation d’image, et ne fournissent pas de
solution générique d’adaptation.
3.
Description de métadonnées
La motivation pour l’adaptation de contenu multimédia est de fournir à l’utilisateur final le
meilleur contenu possible dans un contexte hétérogène, avec des contraintes, des
caractéristiques et des préférences utilisateur fortement personnalisables. La réussite de
l’adaptation de contenu dépend de la qualité et de la quantité de connaissances recueillies sur
le contenu et l’environnement d’utilisation (contexte). Les modèles et systèmes de
métadonnées fournissent un mécanisme de représentation, de stockage et de traitement de
cette connaissance.
Nous considérons trois catégories de métadonnées : les métadonnées de contenu (par
exemple, le format, la taille, la dimension d’une image, etc.), les métadonnées de contexte
(taille de l’écran, contenus pris en charge par le dispositif de connexion, capacité de la bande
passante, préférences linguistiques, etc.), et les métadonnées ou la description des services
d’adaptation (cf. section 4).
Le modèle de métadonnées de contenu que nous avons choisi est le format MPEG-7. Notre
choix (cf. chapitre 4) repose sur le fait que MPEG-7 est indépendant du média, qu’il est
actuellement développé en qualité de standard international, qu’il représente le standard qui
fournit l’ensemble le plus riche et le plus souple de descriptions de propriétés multimédias, et
enfin qu’il couvre une famille de standards comme SMPTE, Dublin Core et d’autres ; il
constitue de fait le standard le plus interopérable.
Des métadonnées de contexte sont utilisées pour décrire l’utilisateur (par exemple : sa langue
préférée), le matériel (résolution d’affichage, contenus supportés, etc.), le réseau (bande
passante disponible…), etc. Un certain nombre d’outils de description ont été développés
pour représenter les métadonnées de contexte ; parmi ces outils figurent CC/PP, MPEG-21,
CSCP, etc. Si CC/PP est décomposable, uniforme et extensible, il manque cependant de
fonctionnalités de structuration. Sa hiérarchie stricte en deux niveaux n’est pas appropriée à
8
la capture de structures de profils complexes. Les outils MPEG-21/DIA garantissent une
adaptation efficace du contenu multimédia ; mais leur restriction aux contenus MPEG-21
limite leur application. CSCP fournit une structure multi-niveaux qui permet une
représentation de tous les types d’information contextuelle. CSCP est basé sur RDF et hérite
ainsi des propriétés de substitution et d’extensibilité. Les noms d’attribut sont interprétés en
fonction du contexte et selon leur position dans la structure de profil. Ainsi un nommage non
ambigu pour les attributs d’un profil n’est pas nécessaire. La description CSCP a été choisie
dans notre travail pour sa simplicité, quoique notre structure d’adaptation demeure ouverte à
l’intégration d’autres modèles de description.
4.
Description des services d’adaptation de contenu multimédia
Un modèle de description de services tel que WSDL ne suffit pas pour automatiser la
découverte, la sélection, la composition et l’invocation des services. En effet, si la syntaxe du
service est importante pour son invocation et sa composition, une information sémantique sur
l’objet et le fonctionnement du service (sémantique des paramètres d’entrée et de sortie, coût
du service, préconditions d’exécution, postconditions, temps d’exécution, qualité du
service…) est essentielle à son utilisation pratique.
Nous proposons dans ce manuscrit (chapitre 5) une ontologie qui formalise la description
sémantique des services d’adaptation multimédia. Cette ontologie facilite la découverte, la
sélection, la composition et l’invocation automatique des services. Nous présentons dans ce
résumé étendu la sous-ontologie qui concerne les données audio. L’ontologie complète est
décrite en Annexe B.
4.1
Hiérarchie de classes
Pour déterminer la structure de la hiérarchie des services d’adaptation, nous avons utilisé la
classification des techniques d’adaptation définie par Lei et al. [Lei01]. Les différents
services d’adaptation sont représentés comme étant des classes hiérarchiquement organisées
dans l’ontologie (Figure 1). Cette hiérarchie de classes et les propriétés des services (cf.
Section 1.2) décrivent l’ontologie complète des services d’adaptation multimédia.
9
MultimediaAdaptationServices
f
su
bC
las
sO
ImageAdaptationServices
O
la
ss
su
bC
O
f
O
f
las
s
f
la
ss
O
su
bC
O
f
Of
la
ss
su
bC
f
AudioLanguageAdaptation
Services
ss
la
bC
su
AudioFormatAdaptation
Services
O
ss
la
bC
su
subClassOf
su
bC
TextAdaptationServices
UnimodalAudioAdaptationServices
AudioReductionAdaptation
Services
StereoToMonoAdaptation
Services
sO
f
ss
la
bC
su
f
AudioAdaptationServices
MultimodalAudioAdaptationServices
AudioToTextAdaptation
Services
Cla
s
f
VideoAdaptationServices
su
b
sO
la s
bC
su
Of
ss
Cl a
b
su
AudioSamplingRateReduction
AdaptationServices
Figure 1 : Hiérarchie de classes (services d’adaptation de multimédia)
La super-classe dans l’ontologie est la classe MultimediaAdaptationServices. Cette classe
regroupe, pour chaque type de données multimédias, différentes sous-classes de services
d’adaptation. Ces services peuvent être des services d’adaptation audio, des services
d’adaptation d’images, etc. La Figure 2 montre la représentation en OWL de la classe
MultimediaAdaptationServices et ses sous-classes.
<owl:Class rdf: ID= “MultimediaAdaptationServices”>
<owl:Class rdf:ID= “AudioAdaptationServices”>
<rdfs:subClassOf rdf:resource= “MultimediaAdaptationServices”/>
</owl:Class>
<owl:Class rdf:ID= “TextAdaptationServices”>
</owl:Class>
<owl:Class rdf:ID= “ImageAdaptationServices”>
</owl:Class>
<owl:Class rdf:ID= “VideoAdaptationServices”>
</owl:Class>
Figure 2 : Représentation de la classe MultimediaAdaptationServices en OWL
10
La Figure 3 représente, de manière plus spécifique, la hiérarchie de classes des services
d’adaptation audio.
O
ss
la
bC
su
su
bC
la
ss
O
f
f
O
f
la
ss
O
f
Services
ss
la
su
bC
bC
su
Services
subClassOf
su
bC
la
ss
O
f
Services
Services
Of
su
bC
l
ss
la
as
sO
f
bC
su
Services
AdaptationServices
Figure 3 : Hiérarchie de classes pour les services d’adaptation d’audio
La
(sous-)classe
hérite
des
propriétés
de
la
classe
MultimediaAdaptationServices ; cependant, la valeur de la propriété d’entrée hasInputType est
logiquement
limitée
à
« audio ».
et
Cette
classe
a
deux
MultiModalAudioAdaptationServices.
sous-classes:
La
classe
UnimodalAudioAdaptationServices impose le type de donnée de sortie (audio) mais aussi le type
de donnée d’entrée (audio). La classe UnimodalAudioAdaptationServices intègre trois sousclasses
:
AudioReductionAdaptationServices,
AudioLanguageAdaptationServices
et
AudioFormatAdaptationServices. La classe AudioReductionAdaptationServices a deux sous-classes :
StereoToMonoAdaptationServices et AudioSamplingRateReductionAdaptationServices. Enfin, la
classe MultiModalAudioAdaptationServices a une sous-classe, AudioTextAdaptationServices.
11
La Figure 4 décrit un extrait de la représentation de la classe AudioAdaptationServices en OWL.
<owl:Class rdf:about="#AudioAdaptationServices">
<rdfs:subClassOf rdf:resource="#MultimediaAdaptationServices"/>
<rdfs:subClassOf>
<owl:Restriction>
<owl:hasValue rdf:resource="#Audio"/>
<owl:onProperty>
<owl:ObjectProperty rdf:ID="hasInputType"/>
</owl:onProperty>
</owl:Restriction>
</rdfs:subClassOf>
</owl:Class>
Figure 4 : Extrait de la représentation de la classe AudioAdaptationServices en OWL
4.2
Propriétés des services
Les services d’adaptation partagent tous un certain nombre de propriétés générales
(Tableau 1), souvent similaires aux propriétés de la classe OWL-S (cf. Annexe C). Après les
avoir identifiées, nous avons regroupé ces propriétés (attributs) dans la super-classe
MultimediaAdaptationServices.
Nom de la propriété
hasServiceName
hasServiceTime
hasServiceQuality
hasEffect
Valeur
String
Float
String
Instance de la classe
MultimediaDataTypeFormat
String
Définition
Nom du service
Temps d’exécution du service
Qualité de donnée adaptée
Effets de l’exécution de
service
hasInputType
Nom du type de donnée en
entrée
hasOutputType
String
Nom du type de donnée en
sortie
hasServicePrice
Float
Coût du service
hasPrecondition
Instance de la classe
Condition préalable pour
MultimediaDataTypeFormat
exécuter le service, par
exemple format de l’image
hasServiceProvider
String
Nom et adresse du fournisseur
de service
hasServiceDescription String
Autres descriptions de services
Tableau 1 : Propriétés de MultimediaAdaptationServices
12
Les
types
de
format
de
données
multimédias
sont
décrits
par
la
classe
MultimediaDatatypeFormat. Le tableau 2 décrit les propriétés de cette classe.
MultimediaDatatypeFormat
Nom de la propriété
hasDatatype
hasFormat
Valeur
String
String
Définition
Nom du type de média
Nom du type de format (ex.
MPEG)
Tableau 2 : Propriétés de la classe MultimediaDatatypeFormat
La Figure 5 présente un exemple de description de service d’adaptation (transcodage audio
vers texte) utilisant l'ontologie décrite ci-dessus.
<AtomicProcess rdf:ID="AudioToTextAdapationService ">
<hasInput rdf:ID="#InputAudio">
<parameterType datatype="&xsd;#anyURI">
</hasInput>
<hasOutput rdf:ID="#OutputText">
</hasOutput>
<hasPrecondition>
<hasExpression rdf:ID="InputAudioFormat">
<hasSubject rdf:resource="#InputAudio"/>
<hasProperty rdf:resource="#Audio"/>
<hasFormat rdf:datatype="&xsd; #string">#WAV</hasFormat>
</hasExpression>
</hasPrecondition>
<hasEffect>
<hasExpression rdf:ID="OutputTextFormat">
<hasSubject rdf:ID="#OutputText"/>
<hasProperty rdf:resource="#Text"/>
<hasFormat rdf:datatype="&xsd;#string">#PlainText</hasFormat>
</hasExpression>
</hasEffect>
<hasServicePrice>
<parameterType datatype="&xsd;#float">25
</hasServicePrice>
<hasServiceTime>
</hasServiceTime>
</AtomicProcess>
Figure 5 : Exemple d’une description de service de transcodage (audio vers texte)
Nous renvoyons le lecteur, pour plus de détails sur l’ontologie de description de services
d’adaptation multimédia, au chapitre 5.
13
Proposition d’une architecture d’adaptation de contenu orientée service
5.
Proposition d’une architecture d’adaptation de contenu
orientée service
Cette section décrit l’architecture générale d’adaptation de contenu proposée dans cette thèse.
Les différents composants de l’architecture et leurs rôles sont tout d’abord analysés avec un
accent plus particulièrement porté sur le module de négociation et d’adaptation de contenu
(Content Negotiation and Adaptation Module - CNAM). Puis nous formalisons la notion de
graphe d’adaptation de contenu et donnons un algorithme de construction de ce graphe.
5.1
L’Architecture DCAF
À la différence des systèmes existants d’adaptation de contenu, lesquels sont restreints à une
application particulière [Lemlouma et al 2003] ou une transformation spécifique [Buchholz et
al 2002], nous visons à fournir une architecture d’adaptation de contenu générique, extensible
et interopérable. L’architecture que nous proposons (Figure 6) est composée de six
composants principaux décrits ci-dessous :
Adaptation Services
Adaptation
Services
Registry
Content Servers
Internet
User Devices
Local Proxies
Content Proxies
Context Profile
Repository
Figure 6: Architecture DCAF
14
Serveurs de contenus : Ce sont des entrepôts standards de données, tels que des sites Web,
des bases de données et des serveurs multimédias.
Proxys de contenus (Content Proxies - CP) : Les proxys de contenus fournissent un accès
aux serveurs de contenus, formulent les requêtes des utilisateurs dans un format adéquat aux
sources, gèrent et livrent (en réponse à des requêtes utilisateur) des descriptions de contenu
(métadonnées).
Gestionnaire de profils utilisateurs (Context Profile Repository - CPR) : ce serveur
stocke les informations du contexte utilisateur, c’est-à-dire les caractéristiques et préférences
de l’utilisateur, les capacités et propriétés du terminal et les propriétés du réseau. Les
utilisateurs peuvent mettre à jour et modifier leur profil à tout moment. Les informations
contextuelles dynamiques telles que la localisation d’un utilisateur ou les paramètres
dynamiques du réseau sont déterminées lors de l’exécution des requêtes de données.
Répertoire des services d’adaptation (Adaptation Service Registry - ASR) : Notre
approche de l’adaptation est basée sur des services d’adaptation tiers. Cependant, la
recherche des services Web d’adaptation disponibles est un problème qui croît de manière
exponentielle avec un nombre croissant de services. Pour faire face à ce problème, nous
avons introduit un annuaire de services d’adaptation. L’ASR est semblable à un annuaire
UDDI (Universal Description, Discovery and Integration). Il stocke les descriptions
fonctionnelles (profils) et non fonctionnelles (ex : coût) des services d’adaptation multimédia
et offre des API de recherche de services.
Services d’adaptation (Adaptation Services - AS) : serveurs hébergeant un ou plusieurs
services d’adaptation. Dans notre approche, les services d’adaptation sont implémentés en
tant que services Web et sont développés de manière indépendante de l’architecture DCAF.
Proxy locaux (Local Proxies - LP) : Ces proxies prennent en charge la récupération et le
traitement des profils de contexte. Ils décident du type et du nombre de traitements adaptatifs,
découvrent les services d’adaptation, planifient l’exécution des services et les invoquent. Un
élément fondamental du proxy local est le module de négociation et d’adaptation de contenu
(CNAM). Ce module détermine un schéma d’adaptation optimal et invoque les services
d’adaptation ad hoc.
5.2
15
Module de négociation et d’adaptation de contenu (CNAM)
La partie fondamentale du proxy local est le module de négociation et d’adaptation de
contenu (CNAM). Le CNAM a deux principaux composants : le moteur de décision et le
planificateur de l’adaptation. Le moteur de décision identifie les types d’adaptation à
effectuer en utilisant les paramètres de l’environnement : profil de l’utilisateur, profil du
terminal, profil de réseau et métadonnées de contenu. Une fois les types d’adaptation
déterminés, l’étape suivante consiste à identifier les services d’adaptation qui vont les
exécuter. Le planificateur se charge de la sélection et de la composition des services
appropriés. La sélection de services est basée sur des critères de qualité comme le coût du
service et le temps d’exécution. Le planificateur récupère ces informations via les ASR (cf.
plus haut).
5.2.1 Générateur du graphe d’adaptation multimédia (MAGG)
Dans un scénario d’application typique en environnement pervasif, un client demande un
contenu multimédia à un serveur au moyen d’un dispositif présentant des propriétés
spécifiques, connecté au serveur par un réseau donné. L’objectif de l’adaptation de contenu
est de transformer le contenu multimédia, stocké sur le serveur dans un format précisément
défini, en fonction des capacités du terminal, de la connexion réseau et des préférences de
l’utilisateur avant de le transmettre au client. Selon le cas, il peut s’avérer nécessaire
d’appliquer un ensemble de transformations afin de faire correspondre le contenu à
l’environnement d’exploitation. Par exemple, dans le scénario 1 du chapitre 1 (section 1.4),
en considérant que le contenu est rédigé en anglais et que la langue préférée de l’utilisateur
est le français, deux transformations au moins sont nécessaires : une conversion du format
textuel en séquence audio et une traduction de l’anglais au français. Comme chaque
transformation peut être effectuée par un ou plusieurs services d’adaptation, nous sommes
amenés à rechercher un scénario d’adaptation optimal. Pour ce faire, l’ensemble des
compositions de services possibles sont modélisées sous la forme d’un graphe, un scénario
d’adaptation correspondant à un chemin de la racine à l’extrémité du graphe. Ce « graphe
d’adaptation » est construit par le module « Générateur de graphes d’adaptation multimédia »
(Multimedia Adaptation Graph Generator - MAGG) et exécuté par le proxy local.
16
5.2.1.1 Définitions et notations utilisées dans MAGG
Définition 1 : Objet média.
Un objet média est une donnée multimédia qui peut être un texte, une image, un son ou une
vidéo. Il est représenté par
M (m1, m2, …, mn)
où m1, m2, …, mn désignent des propriétés du média ou des métadonnées.
Définition 2 : État d’un objet média.
L’état S d’un objet média M, noté S (M), est décrit par les valeurs courantes des métadonnées
caractérisant l’objet M.
Exemple : Pour un objet-image, l’état comporte les valeurs du format, de la couleur, de la
hauteur, de la largeur, etc. Ex :
S(M) = (bmp, 24 bits, 245 pixels, 300 pixels).
Définition 3 : Tâche d’adaptation.
Une tâche d’adaptation est une expression de la forme t(α1 α2 … αn) où t est une
transformation et α1, α2, …, αn, sont les paramètres de la transformation t.
Exemple : ImageFormatConversion(imageIn, imageOut, oldFormat, newFormat)
avec :
imageIn : fichier-image en entrée,
imageOut : fichier-image en sortie,
oldFormat : ancien format de fichier,
newFormat : nouveau format de fichier.
17
Définition 4 : Service d’adaptation.
Un service d’adaptation est un service décrit en termes d’entrées, de sorties, de pré-conditions
et d’effets. Il est dénoté par s = (R I O Pre Eff Q)
avec :
R : processus qui effectue une tâche d’adaptation,
I : paramètres d’entrée du processus,
O : paramètres de sortie du processus,
Pre : pré-conditions du processus,
Eff : effets du processus,
Q : critères de qualité du service.
Définition 5 : Opérateur (opérateur de plan d’adaptation).
Un opérateur de plan d’adaptation est une expression de la forme o = (h (a1, a2, …an, b1, b2,
…bm), Pre Eff Q)
avec :
h : tâche d’adaptation réalisée par un service d’adaptation ayant pour paramètres d’entrée a1,
a2, …, an et pour paramètres de sortie b1, b2, …, bm. h est appelé l’entête de l’opérateur.
Pre : pré-conditions de l’opérateur.
Eff : effets de l’exécution de l’opérateur.
Q = {q1, q2,…, qn}: critères de qualité (par exemple : coût, temps, etc.).
Un opérateur représente de manière abstraite un service d’adaptation qui réalise une tâche
d’adaptation. Les opérateurs sont utilisés (cf. paragraphe ci-dessous) pour construire des
graphes d’adaptation.
Soit S un état, t une tâche d’adaptation et M un objet média. Soit o un opérateur d’entête h qui
réalise t tel que Pre de o est satisfait dans S. Alors on dit que o est applicable à t et le nouvel
état est donné par :
S ( M o ) = Executing (o, M , S )
18
Exemple : Pour la tâche d’adaptation mentionnée ci-dessus (cf. déf. 3), on peut rencontrer
une instance d’opérateur comme suit :
Opérateur : ImageFormatConversionOperator (http://mediaadaptation/imagefiles/image1, http:// media-adaptation/imagefiles/image2,
mpeg, bmp)
Entrée : http://media-adaptation/imagefiles/image1, mpeg, bmp
Sortie : http:// media-adaptation/imagefiles/image2
Pré-condition : hasFormat (http://media-adaptation/imagefiles/image1,
mpeg)
Effet : hasFormat (http:// media-adaptation/imagefiles/image2, bmp)
Q (qualité) : (cost=30 units, time=100 ms)
Note :
Une tâche d’adaptation t peut être réalisée par plusieurs opérateurs. Les attributs de qualité Q
sont utilisés pour déterminer un scénario (appelé également plan ou chemin) optimal.
Définition 6 : Graphe d’adaptation.
Un graphe d’adaptation G (V, E) est un graphe orienté acyclique (Directed Acyclic Graph DAG) dans lequel :
V, ensemble des nœuds, représentent des opérateurs d’adaptation ;
E, ensemble des arcs, représentent les compositions possibles entre les opérateurs
d’adaptation.
Le nœud initial A ∈ V est un pseudo-opérateur avec effet (état initial) mais sans précondition.
Le nœud terminal Z ∈ V est un pseudo-opérateur avec pré-condition (état final) mais sans
effet.
Remarque :
Soient oi ∈ V et oj ∈ V. Une liaison ou un arc eij existe de oi vers oj si la condition suivante
est satisfaite :
oj.Pre ⊆ oi.Eff
où :
19
oj.Pre dénote les pré-conditions de oj ;
oi.Eff dénote les effets de oi.
Définition 7 : Problème de planification d’adaptation.
Un problème de planification d’adaptation est un 4-uplet (SA, SZ, T, D) où SA est l’état initial
de l’objet média, SZ est l’état final de l’objet média, T est une liste (particulière) de tâches
d’adaptation et D est la liste (complète) des opérateurs d’adaptation disponibles. Le résultat
est un graphe G = (V, E).
5.2.1.2 Algorithme
Pour construire le graphe d’adaptation, nous avons d’abord besoin de traduire les descriptions
des services en opérateurs d’adaptation. Dans ce but, nous avons utilisé un analyseur XML
qui prend en entrée le fichier de description d’un service et qui crée l’opérateur d’adaptation
correspondant (cf. chapitre 6).
Disposant des opérateurs d’adaptation, nous avons développé l’algorithme MAGG
(Multimedia Adaptation Graph Generator) qui génère le graphe d’adaptation (cf. ci-dessous).
Algorithm: graph (SA, SZ, T, D)
Input: Initial state SA, final state SZ, adaptation task list T and adaptation operators D
Output: an adaptation graph G (V, E)
// Global constant
// Limit
maximum number of neutral operators allowed in a connection
// Global variables
// V
a set of nodes in a graph
// E
a set of edges in a graph
// ao
start node
// zo
end node
// NO
a set of neutral operators available in the system
// Local variables
// T
a set of adaptation tasks
// t
an adaptation task element of T
20
// O
a set of nodes for adaptation operators realizing an adaptation task
// PO
a set of parent nodes
// Pz
a set containing the end node
var
V, E, T, t, O, PO, ao, zo, Pz, NO
begin
ao= ConstructStartNode(SA) // constructs the start node from the initial state
zo= ConstructEndNode(SZ) // constructs the end node from the goal state
NO= ConstructNeutralOperators( ) // returns the list of the neutral operators available in
//the system
V= {ao} //initialization
E= Ø
//initialization
PO= {ao}//initialization
for each t ∈ T
begin
Construct nodes O from D with operators realizing t
//several operators can realize a task
Connect(O, PO)
//after the process PO holds the value of O
end //T is processed
Pz={zo}
Connect(Pz, PO) // connects the end node
return G (V, E)
end // graph
---------------------------------------------------------------------------------------------------------------// procedure Connect: connects child nodes (O) with parent nodes (PO)
// O a set of nodes-input variable
// PO a set of nodes-input and output variable
---------------------------------------------------------------------------------------------------------------procedure Connect(in O, inout PO)
//Local variables
// CO a set of nodes-temporal variable used to store child nodes (O)
// TPO a set of nodes-temporal variable used to store parent nodes (PO)
// CPO a set of nodes-temporal variable used to store connected parent nodes
// po a node in PO
// o a node in O
// UNO a set of operators-variable to store already used neutral operators
// Link a Boolean variable to check if a node has a connection
// Connected a Boolean variable, true if a node is connected by neutral operator
// LimitC an integer- number of neutral operators in a connection
//Notations
// po.Pre preconditions of node po
// po.Eff effects of node po
// o.Pre preconditions of node o
// o.Eff effects of node o
var
CO, TPO, CPO, UNO, o, po, LimitC, Link, Connected
begin
CO= Ø
//initialization
CPO= Ø //initialization
Link = false
// connect each o ∈ O with each node of the parent nodes PO
for each o ∈ O
begin
for each po ∈ PO
begin
if (o.Pre ⊆ po. Eff) then // direct connection of node o with node po
begin
if(o∉ V) then
begin
21
22
V= V ∪ {o}
end
E= E ∪ {(po, o)}
Link = true // o is connected in the graph
end
else // for indirect connection of node o with po using neutral operators
begin
LimitC=0
UNO= Ø
Connected=ConnectWithNeutralOperators (o, po, LimitC, UNO)
Link=Link or Connected // Link is true if o has at least one connection
// if po is connected add it to the connected parent nodes list
if (Connected) then
CPO=CPO ∪ {po}
end
end // PO is processed
if (not Link) then // checks if o has at least one direct or indirect connection
begin
if(o=zo) then // it is the end node so the graph construction fails
begin
V= Ø
E= Ø
return
end
else
O=O-{o}// o can not be connected so remove it
end
end // O is processed
// remove parent nodes which have no connection
for each po ∈ PO
begin
if (po ∉ CPO ) then
23
begin
V=V-{po}
for each (x, po) ∈ E
begin
E=E-{(x, po)}
Remove(x) // removes x with its ancestors
end
end
end
PO= O //child nodes become parent nodes for the next process
end // end connect procedure
---------------------------------------------------------------------------------------------------------------// function ConnectWithNeutralOperators it connects o and op using neutral operators
//recursively
// o a node- input variable
// op a node- input variable
//LimitC an integer- input variable
// UNO a set of nodes- input and output variable
---------------------------------------------------------------------------------------------------------------function Boolean ConnectWithNeutralOperators (in o, in op, in LimitC, inout UNO)
//Local variables
// TNO a set of operators-a temporal variable for neutral operators
// TE a set of edges-a temporal variable to store connectable edges
// TV a set of nodes-a temporal variable to store connectable nodes
// no a node
var
TNO, TE, TV, no
begin
TNO=NeutralOperators(o, UNO)// it returns neutral operators connectable with o
for each no ∈ TNO
begin
if (no.Pre ⊆ op.Eff) then
24
begin
TE= TE ∪ {(op, no)}
TE= TE ∪ {(no, o)}
TV= TV ∪ {no}
TV= TV ∪ {o}
E= E ∪ TE
V= V ∪ TV
return true
end
else
begin
TE= TE ∪ {(no, o)}
TV= TV ∪ {no}
TV= TV ∪ {o}
if (LimitC+ 1=Limit) or (UNO ∪ {no}=NO) then //terminating condition
return false
else
return ConnectWithNeutralOperators(no, op, LimitC+1, UNO ∪ {no})
end
end // TNO is processed
end // ConnectWithNeutralOperators
---------------------------------------------------------------------------------------------------------------// function NeutralOperators returns the set of neutral operators satisfying the preconditions
//of o
// o a node-input variable
// UNO a set of nodes- input variable
---------------------------------------------------------------------------------------------------------------function Operators NeutralOperators(in o,in UNO)
//Local variables
// TNO a set of operators-temporal variable for neutral operators
// no a node
//Notations
25
// no.Eff effects of node no
var
TNO, no
begin
TNO= Ø
for each no ∈ NO
begin
if (o.Pre ⊆ no.Eff) then
if (no ∉ UNO) then
TNO=TNO ∪ {no}
end
return TNO
end // NeutralOperators
---------------------------------------------------------------------------------------------------------------// procedure Remove it removes the node x recursively until it finds a node with a branch
//connection or it reaches the start node
// x a node- input variable
---------------------------------------------------------------------------------------------------------------procedure Remove(in x)
//Local variables
// z a node for which (z, x) is an edge in E
// y a node for which (x, y) is an edge in E
var
z, y
begin
if (x=ao) then // it is the start node make V empty and return
begin
V= Ø
26
return
end
else if ((y ∈ V) and ((x, y) ∈ E)) then // x has a branch connection
begin
return
end
else
begin
for each (z, x) ∈ E
begin
E=E-(z, x)
Remove(z)
end
V=V-{x}
end // each (z, x) is processed
end //Remove
5.3
Recherche d’un chemin /plan d’adaptation optimal
Définition : Plan /chemin d’adaptation.
Un plan (ou chemin) d’adaptation est un chemin dans le graphe d’adaptation G qui relie le
nœud initial au nœud terminal. Il est représenté par une liste de la forme p = (A o1 o2 … on Z)
où
A et Z sont les nœuds initial et terminal et oi est une instance d’opérateur d’adaptation.
Si p = (A o1 o2 … on Z) est un plan et que S est un état, alors S(p) est l’état de l’objet média
produit suite à l’exécution de la suite ordonnée d’opérateurs o1, o2, …, on sur l’objet média en
entrée.
Un plan d’adaptation est caractérisé également par une qualité.
27
Modèle de qualité de service (QoS)
Comme une tâche d’adaptation peut être accomplie par plus d’un service, le choix d’un
service n’est pas neutre. Une fois que le graphe d’adaptation, qui modélise toutes les
compositions possibles, est généré, le choix de la meilleure alternative, c’est-à-dire du
chemin optimal dans le graphe, est fait sur la base des critères de QoS spécifiés par
l’utilisateur. Le modèle de QoS de base que nous utilisons s’appuie sur deux critères : le
temps de réponse du service et son coût.
Soit p = (A s1 s2 … sN Z) un chemin dans le graphe d’adaptation où N est le nombre de
services. On définit la QoS du chemin, dénotée par Q(p), par :
Q( p) = (QCOST ( p), QTIME ( p ))
(1)
où
QCOST ( p ) désigne le coût total du chemin,
QTIME ( p ) désigne le temps total d’exécution du chemin.
avec
QCOST ( p ) défini par
N
QCOST ( p ) = ∑ QCOST ( s i )
i =1
(2)
où
QCOST ( si ) désigne le coût intégrant l’exécution du service d’adaptation et la
transmission des données.
et
QTIME ( p ) défini par
N
QTIME ( p) = ∑ QTIME ( s i )
i =1
(3)
28
où
QTIME ( si ) désigne la somme du temps d’exécution du service d’adaptation et du
temps de transmission des données.
Pour agréger ces valeurs quantitatives hétérogènes, il est nécessaire de les ramener sur une
échelle unique (scaled values).
QsCOST ( s i ) et QsTIME ( s i ) sont ainsi définies par :
max
⎧ QCOST
− QCOST ( si )
⎪
max
min
QsCOST ( s i ) = ⎨ QCOST
− QCOST
⎪
1
⎩
max
min
if QCOST − QCOST ≠ 0
min
Q max − QCOST
=0
if COST
max
⎧ QTIME
− QTIMe ( s i )
max
min
⎪
if QTIME − QTIME ≠ 0
max
min
QsTIME ( si ) = ⎨ QTIMe − QTIME
max
min
QTIME
− QTIME
=0
⎪
1
if
⎩
(4)
(5)
où
max
min
QCOST
et QCOST sont les valeurs respectives de coûts maximum et minimum et
max
min
QTIME
et QTIME sont les valeurs respectives de temps maximum et minimum.
Note : Les valeurs maximum et minimum sont calculées à partir des services disponibles.
Les utilisateurs peuvent indiquer leurs préférences sur la QoS (ex : choix d’un service rapide
vs choix d’un service peu coûteux) en spécifiant des valeurs de pondération sur chaque
critère. Le score d’un chemin avec des valeurs pondérées s’établit ainsi comme suit :
N
Score( p ) = ∑ (QsCOST ( si ) ∗ wCOST + QsTIME ( si ) ∗ wTIME )
i =1
(6)
où wCOST ∈ [0, 1] et wTIME ∈ [0, 1] représentent les valeurs de poids affectées
respectivement au coût et au temps avec
wCOST + wTIME = 1
(7)
29
Soit Pset = {p1, p2, …, pK} l’ensemble de tous les chemins possibles dans un graphe
d’adaptation. Alors le chemin optimal est le chemin dont la valeur du score est maximale.
Cette valeur, dénotée scoremax, est définie comme suit :
Scoremax = max {Score( pi )}
iε {1, K }
(8)
Alors que l’algorithme de Dijkstra [Dijkstra59] peut être directement appliqué pour trouver le
chemin optimal, l’algorithme peut être rendu plus efficace pour un graphe orienté acyclique
(ce qui est le cas du graphe d’adaptation) en effectuant un tri topologique du graphe.
Le temps d’exécution pour l’implémentation la plus simple de l’algorithme de Dijkstra est en
O (|V|2). Avec un tas binaire, l’algorithme requiert un temps en O ((|E|+|V|) log (|V|)) ; un tas
de Fibonacci améliore ce temps en O (|E| + |V|log (|V|). Enfin, l’algorithme appliqué à un
graphe trié topologiquement présente un temps d’exécution en O (|E|+|V|). Il est décrit en
Annexe D.
5.4
Evaluation de performances
Deux principaux tests de performance ont été conduits. Le premier visait à étudier l’impact
du ré-ordonnancement des transformations sur le temps total d’exécution. Des services
d’adaptation ont été publiés sur un PC intégrant deux processeurs Pentium 4 à 3.00 GHz,
2.00 Go de mémoire vive et l’édition standard du système d’exploitation Microsoft Windows
Server 2003. Nous avons utilisé des images JPEG de taille 570x760 pixels. Afin de
minimiser l’effet de la variation de la charge du système, l’expérience a été répétée et la
moyenne des résultats a été considérée. Nous avons observé que les différents ordres
d’exécution de services d’adaptation ont produit différents temps d’exécution. Si on met le
service de réduction de couleur en dernier il produit un meilleur résultat comparé aux autres
ordres. Dans la légende R est la redimensionnent de taille, F est la conversion de format et C
est la réduction de couleur. Donc, le meilleur ordre d’exécution est RFC (redimensionnent de
taille, conversion de format, réduction de couleur). La résultat de cette expérience est décrit
Figure 7.
30
Temps d’exécution (100
ms)
Comparaison d’ordre d'exécution des
transformations
50
RFC
40
CFR
30
FRC
20
RCF
CRF
10
FCR
0
32
42
43,3 43,9 45,7 49,4 52,6 56,7 74,4
Taille d’image (Kb)(570*760 pixels)
Figure 7: Comparaison de différents ordres d'exécution
La seconde expérience (Figure 8) étudiait le comportement de l’algorithme de génération du
graphe d’adaptation par rapport à la profondeur du graphe et au nombre de services par
transformation. Elle a été réalisée sur une machine disposant d’un processeur Pentium 4
cadencé à 1.9 GHz, avec 256 Mo de mémoire vive et équipé du système d’exploitation
Microsoft Windows 2000. Alors que la relation entre la construction du graphe et le nombre
de services par transformation est linéaire, comparativement à la profondeur du graphe, le
temps de construction progresse lentement avec le nombre de services par transformation.
Nous avons également observé que la pente pour la profondeur et la largeur (nombre de
services par transformation) était quasiment constante avec une augmentation moyenne de 40
ms pour chaque profondeur et de 10 ms pour chaque tranche additionnelle de 10 services.
Cela signifie que le nombre de services par transformation n’affecte pas significativement le
temps total de construction, tandis que cela permet de choisir le meilleur service parmi les
services candidats. Par exemple, pour 300 services (profondeur de 10 et largeur de 30) dans
un graphe, configuration extrême d’une adaptation réaliste, le temps total de construction
n’est que de 335 ms - résultat en pratique très acceptable.
31
Implémentation du prototype
Temps(ms)
Temps de construction du graphe
400
350
300
250
200
150
100
50
0
10 Services par
transformation
20 Services par
transformation
30 Services par
transformation
2
3
4
5
6
7
8
9
10
Profondeur du graphe (# des
transformations)
Figure 8: Temps de construction du graphe
6.
Nous avons développé un prototype de l’architecture logicielle DCAF (cf. section 5 Figure 6
ci-dessus) afin de montrer la faisabilité et la validité de nos propositions. Le prototype a été
implémenté avec Borland Jbuilder 9 personal. Nous avons utilisé MySQL-4.0.12 pour les
différentes bases de données mises en œuvre dans le prototype. Le paquetage logiciel
comprend plus de 10 000 lignes de code.
6.1
Composants du prototype
Notre prototype intègre les fonctionnalités principales suivantes :
Moteur de décision d’adaptation : un moteur de décision analysant les contraintes
contextuelles et les préférences utilisateur est chargé de la construction du graphe
d’adaptation et du calcul d’un chemin optimal d’adaptation.
32
Proxys : Trois types de proxys ont été développés. Des proxys de contenu chargés de la
gestion des serveurs de contenus distribués et des métadonnées décrivant les contenus ont été
implémentés. Des bases de données MySQL ont été déployées pour le stockage des contenus
et des métadonnées. Des proxys locaux qui constituent le cœur de l’architecture ont été mis
en place. Le moteur de décision d’adaptation et d’autres fonctionnalités cruciales font partie
intégrante de ces proxys. Enfin, nous avons implémenté des proxys (services) d’adaptation.
Ceux-ci ont été programmés en tant que services Web afin de montrer la faisabilité
d’adaptations complexes à partir de services tiers. C# et ASP.NET ont été utilisés pour
développer ces services.
Annuaires de services d’adaptation : Pour la découverte de services d’adaptation, des
annuaires ont été mis en place afin de stocker les informations relatives aux services
disponibles. Ces annuaires ne s’intéressent qu’aux services d’adaptation. Les fournisseurs de
services d’adaptation peuvent publier la description de leurs services dans ces annuaires que
les proxys locaux viennent consulter en fonction des besoins d’adaptation.
Répertoire de profils client : Ce répertoire stocke les profils client. Un profil client consiste
en les préférences de l’utilisateur, les caractéristiques et capacités des dispositifs et la
connexion au réseau. Alors que, dans le prototype, le profil client est créé lors de la
souscription d’un utilisateur et que ce profil peut être modifié à tout moment, dans une
application réelle, le système récupérerait le profil du terminal à partir de son identifiant dans
les bases de données mises à disposition par le vendeur du dispositif ; le profil réseau est
connu en utilisant les informations de contrôle du réseau. Une base de données MySQL a été
utilisée pour stocker les informations du compte utilisateur ainsi que son profil.
L’élément au cœur du mécanisme d’adaptation est le proxy local.
33
Interface module
Content Negotiation & Adaptation Module
Context Profile and
Metadata Processor
Adaptation Plan Generator
Communication module
Client Profile agent
Adaptation service agent
Service discovery
Service invocation
Figure 9: Schéma fonctionnel du proxy local
Le schéma fonctionnel de la Figure 9 présente les composants du proxy local, ainsi que les
interactions et les flux d’information entre ces composants. Un proxy local est composé de
cinq éléments principaux : un module de négociation et d’adaptation de contenu, un agent de
profils, un agent de services d’adaptation, un module de communication et un module
d’interface :
Le module d’interface est utilisé pour communiquer avec l’utilisateur. Quatre interfaces
sont définies : une interface de connexion, une interface de saisie de requêtes, une
interface d’affichage des résultats et une interface de saisie/mise à jour du profil
utilisateur. L’interface de connexion sert à la saisie de l’identifiant utilisateur et de
son mot de passe nécessaires pour l’accès au système. L’interface de requêtes permet
de formuler les requêtes de contenu. L’interface de saisie/mise à jour du profil
utilisateur permet l’ajout et la modification du profil client. L’interface d’affichage
des résultats est chargée de la présentation du contenu après adaptation.
Le module de communication est utilisé pour les communications avec les proxys de
contenu. Deux types de données sont échangés entre le proxy local et les proxys de
contenus : des métadonnées et du contenu. Le proxy local envoie au proxy de
34
contenus une requête pour le serveur de contenus ; le proxy de contenus fait parvenir
la réponse du serveur de contenus (des métadonnées) au proxy local. L’utilisateur
choisit un contenu parmi les réponses et en informe le proxy local qui retransmet cette
requête vers le proxy de contenus. Ce dernier demande au serveur de contenus de lui
envoyer les données demandées puis transfère celles-ci au proxy local. L’ensemble
des proxys sont programmés en tant que services Web et SOAP est utilisé comme
protocole de communication.
L’agent de profil est chargé de stocker et récupérer le profil client à partir du répertoire
de profils. Quoique dans notre prototype un seul répertoire de profils client est
déployé, dans une application réelle plusieurs répertoires peuvent exister ; par
exemple, les répertoires de profils matériel des distributeurs. Ceux-ci sont d’ailleurs
susceptibles d’implémenter différents protocoles de communication, ce qui peut
constituer une source de problèmes d’interopérabilité. Nous avons choisi SOAP,
protocole largement répandu, comme protocole de communication entre le proxy
local et les répertoires de profils.
L’agent de service d’adaptation a deux composants : le module de découverte de
services d’adaptation et le module d’invocation de services d’adaptation. Le module
de découverte de services d’adaptation a pour rôle d’identifier tous les services
d’adaptation pertinents enregistrés dans l’annuaire de services. Le module
d’invocation de services d’adaptation est chargé d’invoquer les services distants en
fonction du plan optimal d’adaptation. Dans les deux modules, le protocole de
communication choisi est SOAP.
Le module de négociation et d’adaptation de contenu est la partie centrale du proxy
local. Ces deux principaux composants sont : le processeur de profils et métadonnées
de contexte et le générateur de plan d’adaptation. Le processeur de profils et
métadonnées de contexte est chargé d’analyser et de décider du nombre et des types
de transformation nécessaires pour adapter le contenu en tenant compte des
contraintes et caractéristiques du contexte, des préférences utilisateur et du contenu. Il
établit une règle de transformation (voir chapitre 6, Figure 6.13). Le générateur de
plan d’adaptation prend en entrée cette règle de transformation, construit le graphe
35
d’adaptation puis génère un plan optimal d’adaptation. Ce plan est alors transmis au
module d’invocation de services d’adaptation.
6.2
Bases de données utilisées dans le prototype
Quatre principales bases de données ont été développées dans le prototype : le répertoire de
profils client, l’annuaire des services d’adaptation, une base de contenus et une base de
métadonnées. Le répertoire des profils stocke les informations relatives aux utilisateurs telles
que le nom, l’identifiant, le mot de passe et un lien vers le fichier du profil contextuel
(CSCP). L’annuaire des services d’adaptation stocke les informations relatives aux services
d’adaptation telles que les identifiants et le lien vers le fichier de description de service
(OWL-S). La base de contenus héberge les données d’application et les contenus
téléchargeables par l’utilisateur. Enfin, la base de métadonnées accueille les informations de
recherche sur le contenu et un lien vers le fichier MPEG-7 associé à chaque contenu Les
bases de données sont déployées sous MySQL-4.0.12.
L’architecture DCAF d’adaptation de contenus multimédias est générique et indépendante du
domaine applicatif et des configurations matérielle et réseau. Cependant, dans le but de
valider concrètement nos concepts et propositions, nous avons développé une implémentation
typée de DCAF pour une application médicale, domaine d’application privilégié de l’équipe.
Notre base de données stocke ainsi des modèles de dossiers médicaux. Le modèle conceptuel
de données utilisé dans le prototype est donné Figure 10. Les tableaux suivants décrivent les
champs des différentes tables.
Hospital
Champ
Description
HCode
Identificateur unique de l’hôpital
Name
Nom de l’hôpital
Address
Adresse de l’hôpital
Section
Liste des services de l’hôpital
36
PatientData
Champ
Description
PatientID
Identificateur unique du patient
FirstName
Prénom
LastName
Nom de famille
DateOfBirth
Date de naissance
Address
Adresse
MedicalRecord
Champ
Description
MedRecId
Identificateur unique
PatientID
Identificateur unique du patient
DocId
Identificateur unique du document
Title
Titre du document
Description
Description du contenu du document
Keywords
Mots-clés décrivant le document
MedicalDoc
Champ
Description
MedDocId
DocId
Identificateur du document
MediaObject
Champ
Description
MediaId
Identificateur unique de l’objet média (contenu)
MedDocId
MediaType
Type du contenu
Title
Titre du contenu
Description
Description du contenu
Language
Langue du contenu
MetaDataLoc
Adresse URL du fichier de métadonnées
37
MedicalRecord
Hospital
PK
PK
HospCode
Name
Address
Section
PatientData
MedRecId
PK
1:1
N:1
PatientID
DocID
Title
Description
Keywords
HospCode
N:1
1 :1
PatientID
FirstName
LastName
DataOfBirth
Address
1:1
MediaObject
PK
1:1
MedicalDoc
PK
1:1
N:1
MedDocId
DocId
MediaID
MedDocId
MediaType
Title
Description
Language
MetaDataLoc
Figure 10: Modèle conceptuel des données
6.3
Interfaces visuelles
Plusieurs interfaces visuelles ont été développées pour la communication entre le système et
l’utilisateur : interface de connexion, interface de saisie de profils, interface de saisie de
requêtes et interface de présentation des résultats. Nous présentons ci-dessous les interfaces
de saisie de profil et d’affichage de résultats qui sont les plus significatives.
6.3.1 Interface de saisie du profil client
L’interface de saisie du profil client (Figure 11) est composée de trois panneaux : le panneau
de sélection du type de terminal, le panneau de spécification du réseau et le panneau de saisie
des préférences de l’utilisateur (cf. chapitre 7 pour le détail de chaque panneau).
L’organisation de l’interface de saisie est basée sur la structure de représentation du profil
client (cf. Annexe A). L’interface permet de capturer l’ensemble des informations du profil
client.
38
Figure 11: Capture d’écran de l’interface de sélection du matériel.
6.3.2 Interface d’affichage de données adaptées
Dans un système pervasif, l’adaptation revêt deux principaux aspects : adaptation des
données/du contenu et adaptation de l’interface utilisateur. Notre travail s’est focalisé sur
l’adaptation de contenu. L’adaptation et la gestion de l’interface utilisateur dans un
environnement pervasif constitue un domaine de recherche à part entière qui nécessite un
approfondissement ne faisant pas l’objet de cette thèse. Comme la présentation de
l’information à l’utilisateur requiert à la fois une adaptation de contenu et de l’interface, nous
avons collaboré avec une autre équipe de recherche du laboratoire qui travaille sur un projet
d’adaptation d’interfaces appelé SEFAGI (Simple Environment for Adaptable Graphical
Interfaces) [Chaari05]. Le système SEFAGI fournit un environnement qui permet la
description des interfaces utilisateur spécifiées en utilisant un langage abstrait de description
des fenêtres, un langage de description de plate-forme et des mécanismes de génération
automatique de code.
Notre collaboration avec le projet SEFAGI nous a permis de disposer d’un système complet
d’adaptation qui délivre des informations médicales adaptées au contexte de l’utilisateur en
termes de contenu et d’interface. Comme le système SEFAGI est en phase de développement,
nous avons généré uniquement des interfaces pour les PC et les Smartphones à des fins de
test (voir Figures 12 (a) et (b)). Nous sommes en cours de rédaction d’un article pour une
revue sur les résultats de cette collaboration. Nous espérons poursuivre ce travail commun.
Figure 12 (a) : Capture d’écran d’une donnée-image sur un PC standard (sans adaptation).
Figure 12 (b) : Capture d’écran d’une donnée-image sur un Smartphone (après adaptation).
39
40
7.
Conclusion et perspectives
S’appuyant sur les progrès récents dans le monde de la communication sans fil et des
technologies mobiles, les environnements pervasifs (ou ubiquitaires) suscitent un intérêt
croissant. Cependant les limitations liées aux ressources des terminaux mobiles et aux
capacités des réseaux sans fil, l’hétérogénéité des terminaux et des données, la variabilité des
conditions réseaux, la multiplicité des demandes et des préférences utilisateurs génèrent des
problèmes que les systèmes multimédias distribués classiques ne traitent qu’imparfaitement.
L’adaptation des contenus (données) transmis(es) aux utilisateurs offre un cadre de solution à
ces problématiques nouvelles.
Dans ce cadre, les principales contributions de cette thèse sont les suivantes :
• Nous avons introduit une nouvelle architecture répartie d’adaptation de contenu
multimédia fondée sur l’utilisation de services tiers (architecture DCAF). Cette
architecture est très facilement extensible en termes fonctionnels (intégration de
nouveaux services d’adaptation) et non-fonctionnels (passage à l’échelle).
L’interopérabilité de cette architecture avec les services d’adaptation est garantie via
l’utilisation de protocoles et de modèles de description standards (e.g. SOAP, XML,
MPEG-7).
• La technologie des services Web est considérée comme étant une solution appropriée
pour l’informatique distribuée, en particulier pour Internet. Son application à
l’informatique pervasive, notamment pour l’adaptation de contenu, a été explorée
dans cette thèse.
• Nous avons proposé une définition et une description formelles du processus
d’adaptation de contenu multimédia (e.g. concepts d’opérateur et de graphe
d’adaptation).
• Nous avons spécifié une ontologie pour la description de services d’adaptation
multimédia. Cette ontologie permet une description sémantique des services, qui
facilite la découverte, la composition et l’exécution automatiques de services
d’adaptation.
• L’approche proposée est fondée sur la composition de services d’adaptation. Une
méthodologie d’optimisation de compositions de services a été proposée.
Conclusion et perspectives
41
• Enfin, un prototype a été réalisé pour évaluer nos propositions. Des expérimentations
ont été menées avec ce prototype qui ont montré la faisabilité et la pertinence de
l’approche proposée.
Nos perspectives se déclinent comme suit :
• Dans cette thèse, des mécanismes distincts de représentation sont utilisés pour décrire
le contexte (utilisateur, matériel, réseau), le contenu et les services d’adaptation.
Cependant, l’interopérabilité du système pourrait être renforcée par la construction
d’une ontologie unifiée et globale. Cette ontologie unifiée admettra des ontologies de
haut et bas niveaux (parente et enfant). L’ontologie parente (de haut niveau) sera
chargée de capturer la connaissance générale sur le monde physique dans les
environnements pervasifs. L’ontologie enfant (de bas niveau) capturera l’information
relative à chaque composant (par exemple : les services d’adaptation).
• Dans l’implémentation actuelle, l’algorithme MAGG prend en considération le format
des données d’entrée et de sortie pour l’analyse de la compatibilité de service. Mais
d’autres paramètres, comme la taille des données et/ou le taux de rafraîchissement
d’une séquence vidéo, sont aussi importants. L’algorithme devrait idéalement tenir
compte de tous les paramètres dans la détermination de la compatibilité des services.
Cela peut être implémenté en modifiant la partie de l’algorithme qui effectue la
correspondance entre les pré-conditions et les effets. Au lieu de ne manipuler qu’un
seul paramètre, l’algorithme vérifiera tous les paramètres.
• À un certain niveau d’abstraction, les systèmes pervasifs et les grilles se rapportent
tous les deux à des unités de traitement distribuées et à grande échelle. La
combinaison des deux aspects, au sein de ce qu’on peut appeler la grille pervasive
[Pierson05], peut favoriser l’essor de la grille de calcul dans le monde réel. Une grille
pervasive est une architecture orientée services. . Notre proposition d’adaptation de
contenu basée sur des services s’inscrit donc parfaitement dans cette orientation. Dans
ce contexte, nous envisageons de déployer les services d’adaptation multimédia
comme des services de grille pour fournir des données multimédias adaptées dans des
environnements de grille pervasive.
Le système de proxy implémenté est, à notre connaissance le premier prototype visant à
étudier l’adaptation de contenu dans un environnement pervasif. Plusieurs axes
42
d’amélioration sont envisageables. La liste suivante expose les limites et les travaux futurs
relatifs au prototype :
• Notre moteur de décision n’utilise pas de « conseils d’adaptation » lors de la décision
des types d’adaptation à implémenter. Par exemple, un fournisseur de contenu peut
vouloir spécifier par le biais des métadonnées liées au contenu la liste restrictive des
adaptations autorisées sur ses données. Les « conseils d’adaptation » peuvent ainsi
limiter ou faciliter la prise de décision.
• Notre prototype implémente une stratégie simple de gestion de cache de contenus
adaptés. Pour leur part, les profils de contexte ne sont pas cachés (ce qui amène le
proxy local à récupérer le profil auprès du répertoire de profils à chaque nouvelle
requête). Il est clair qu’il serait tout à fait pertinent de mettre en place des mécanismes
de gestion de cache (cache de contenus, cache de contextes) plus évolués.
Y. Cardenas, doctorant au sein de l’équipe, s’intéresse aux caches coopératifs dans un
environnement de grille [Carden05]. Son travail s’attache au développement d’un grid
service de cache virtuel (agrégation de multiples caches répartis sur une grille). Nous
comptons rapidement étudier l’intégration de ce service dans notre système.
• Enfin, nous envisagerons le développement de plus nombreuses interfaces à
destination de plusieurs terminaux potentiels en sus de ceux actuellement utilisés dans
le prototype (la collaboration avec le projet SEFAGI).
Chapter
1
Introduction and Motivation
In this chapter, we motivate the need of content adaptation for pervasive computing
environments. In Section 1.1, we present a brief historical of the development of information
technology with a particular emphasis on multimedia information and its major application
areas. In Section 1.2 we give a general definition of pervasive computing and of its major
characteristics. In Section 1.3, we present the need of content adaptation for pervasive
computing environments. Three driving scenarios that illustrate situations in which content
adaptation is desirable are presented in Section 1.4. A Short overview of our proposed content
adaptation approach is presented in Section 1.5. Finally we present the organization of the
thesis in Section 1.6.
1.1 Introduction
Human society has passed through different development ages, defined by the dominant
technology of that particular age, each with profound impacts on the way the society lives,
works and communicates. In the course of technological ascent, society has traversed the
Stone Age, Bronze Age, Iron Age, Agrarian Age, Industrial Age, and most recently - the
Information Age [Aspy93].
Information Age is a term applied to the period where the movement of information became
faster than the physical movement, more narrowly applying to the 1980s onward. One could
argue, though, that it actually began during the latter half of the 19th Century with the
invention of the telephone and telegraphy [Manu04]. Advocates of the concept of the
Information Age maintain that we have embarked on a journey in which information and
2
Chapter 1. Introduction and Motivation
communications will become the dominant forces in defining and shaping human actions,
interactions, activities, and institutions. In contrast to the Industrial Age that proceeded this
era, information is rather than natural resources or capital -- has become the strategic
resource upon which the economy revolves. Information has been defined in innumerable
ways. One definition offered by the American Library Association [List02] is:
"All ideas, facts, and imaginative works of the mind which have been communicated,
recorded, published and/or distributed formally or informally in any format".
Under this broad definition, information is everything that has been created or studied by the
human mind and recorded and communicated in some way. Since the start of civilization,
humans have found ways to record, store, and communicate what they have discovered and
created. In fact, human progress would not have been possible without all of the ingenious
ways humans have recorded, transmitted, stored, retrieved, and used information.
Before the invention of writing, humans depended entirely on the spoken word to record
thoughts and ideas. Known as the oral tradition, this was the process by which "culture,
tradition, and historical accounts were passed from generation to generation in stories,
folklore, songs, and poetry" [Bolner01]. But after writing evolved into the phonetic stage,
completed with symbols and alphabets, it was common for information to be written down.
Scholars believe that humans first wrote their ideas by carving or painting on objects they
found around them: cave walls, bones, and pieces of bark. Clay tablets and large pieces of
stone were used a bit later, and by 500 BC peoples of Egypt, Greece, and Rome were writing
on papyrus2, a substance similar to paper made from a plant that grew along the Nile River
[Wolb02]. Papyrus was gradually replaced by parchment, a substance made from specially
treated animal skins thought to have been invented around 200 BC in what is now modernday Turkey. Parchment was widely used for writing until it was replaced in the 1400's by the
introduction of paper in Europe and the invention of the movable type printing press in 1450.
The invention of the printing press is a milestone in human history because for the first time
books became available to people outside the upper classes. It was in the 17th century that the
first newspapers and journals were published in Europe [List02]. This provided an
2
Papyrus : an Egyptian word which is the base of the English word paper
Introduction
3
opportunity for the average persons to learn about science, politics, and culture. The Industrial
Revolution of the 18th century saw improvements in printing technologies and reductions in
the price of paper, thus initiating the mass marketing of books and journals.
The 19th and 20th centuries saw the development of many new ways to record and store
information other than paper: microfilm, microfiche, phonograph records, audio tape, film
strips, and video tape [List02]. The invention of computers which dated back to 1940’s
[More84] marked the most dramatic advance in recording, storing, and retrieving information.
Although people continue to produce information in handwritten, typewritten, and printed
form, it can now also be generated in electronic format, i.e. recorded as bits and bytes of
computer data and stored on magnetic tape or computer disk. Vast amounts of information are
stored in electronic format and retrievable using computers. The digital electronic computer
made it possible to process information instantly. As the computer developed and matured,
communication and processing technologies were joined into networks that now stretch
around the world, affecting all areas of global society.
The latest advance in recording and distributing information has been the rise of the Internet.
The Internet has its roots in the early 1970’s when the US government was funding research
into a computer network that could survive a nuclear attack [Leiner03]. In 1991 Berners-Lee
[Mosch99] who was working at CERN (“Centre Européen pour la Recherche Nucléaire”European Laboratory for Particle Physics), in Geneva, Switzerland, developed the World
Wide Web (WWW), the hypertext system he had first proposed in 1989. He envisioned the
WWW as a shared information space - a web of hypertext documents - within which people
communicate with each other and with computers. In the early 1990’s, Internet browsers first
appeared, making it easier for the general public to access an astonishing variety and amount
of information.
Further advance in computing and communication technologies devices especially during the
past few years has made possible the creation, storing, processing and dissemination of
multimedia information. Unlike the alphanumeric data which require much less processing,
storage and transmission resources, multimedia data require far more powerful computing and
communication facilities. Multimedia information is a combination of text, graphics, image,
audio and video [Lu99].
4
Multimedia provides the possibility for a spectrum of new applications, many of which are in
place today. Multimedia applications differ from traditional software in many ways: up until
recently it has been relatively easy to predict the areas of use of traditional packages but
multimedia applications have the potential to be used in a diverse range of areas from
entertainment, education, and commerce through to communication and art. The following are
some of the application areas:
Multimedia entertainment: the field of entertainment uses multimedia extensively. One of the
earliest applications of multimedia was for games. Multimedia made possible innovative and
interactive games that greatly enhanced the learning experience. Games could come alive with
sounds and animated graphics.
Multimedia training: multimedia presentations are a great way to introduce new concepts or
explain a new technology. In companies, this reduces the training time. Individuals find it
easy to understand and use.
Multimedia in business: even basic office applications like a word processing package or a
spreadsheet tool becomes a powerful tool with the aid of multimedia data. Pictures, animation
and sound can be added to these applications, emphasizing important points in the documents.
Multimedia in healthcare: in the healthcare industry, software developers have implemented
many solutions to deliver the patient management in the following main areas – clinical,
appointments and billing. In clinical section, the healthcare system provides very basic
standard clinical related functionalities. For instance, tools for handling prescription-writing,
managing patient consultation record, requesting pathology tests and importing pathology
results. In order for the doctor to have full and complete patient information, healthcare
systems incorporate multimedia information into the patient records which may include
image, audio or video. This enables to improve the quality and efficiency of the patient
treatment.
Multimedia libraries: different education institutions have developed multimedia or digital
libraries to facilitate the teaching-learning process. Multimedia libraries provide web-based
access to instructional resources such as course note, books, lecture presentations, sample
Introduction
5
exercises and tests. The multimedia library saves time and costs. Many users can have access
to the resources at any time which is not the case in traditional library.
Multimedia museums: the ability to capture and store information in formats other than
structured text provides new opportunities for the documentation and interpretation of works
in museum collections. New kinds of information can be recorded, stored and communicated.
Multimedia databases- which store two or more different types of information enable the
recording of structured text, unstructured text, images, sound and video. All of these data
types can be integrated to form a comprehensive archive which offers a more robust picture of
the context and meaning embodied in artifacts, held in collections. The multimedia data
present the museum visitor with more than formatted data and textual information. It
encompasses interactive multimedia, hypermedia, imaging applications, digital video,
computer graphics, virtual reality, and computer-controlled interactive displays and
participatory exhibitions. The construction of multimedia archives plays a significant role in
the documentation strategies of many museums.
Another multimedia application is videoconferencing. It allows conducting a conference
between two or more participants at different sites by using computer networks to transmit
audio and video data. For example, homecare nurses can use video conferencing to care for
their patients, saving time and costs and helping them cope with increasing workloads.
The rapid advancements in computing and communication technologies combined with the
widespread multimedia applications led to many research topics such as storage, indexing,
searching, delivery and consumption. While the storage, index and searching aspects of
multimedia data have been addressed by several researches during the last decades, the
delivery and consumption of multimedia data have become an interesting research issue with
the emergence of heterogeneous devices and network connections. In this thesis, we present
the important issues related to multimedia content delivery in heterogeneous environment, the
related challenges and the possible solutions.
6
1.2 What is Pervasive Computing?
Today, computing is no longer limited to a particular location using some stationary
computing devices, such as Personal Computers (PCs) at home or in office. With the
advances in the computing technologies, computing can now be done on heterogeneous
mobile devices, such as laptop computers and information appliances like Personal Digital
Assistants (PDAs), pagers, smart phones, etc. People can access different multimedia services
using these mobile devices wherever and whenever they want them: at work, at home, in a
meeting, or even when traveling.
The term Pervasive computing [Agoston00] is used to describe the kind of computing that can
be done anytime, anywhere, and on any device. Computing can be done whenever a
computing device is available; e.g. with the PC at home and in office, the PDA you carried
during traveling, the laptop brought into the meeting room, or even the thin client inside a
coffee shop. All these devices help accessing information or services in one seamless,
integrated system. There is no fixed association between a device and a user, meaning that
these devices do not need to be owned, and can be any device that is come across. In
pervasive computing, users are free to move and compute without the need to carry any
device. It is clear that pervasive computing environment is very different from traditional
computing environments.
The essential features and demands, which set the pervasive computing environment apart,
include [Kwan02, Henr01]:
Device heterogeneity: computing is carried out on a wide range of devices, from small
mobile devices, such as smart phones and PDAs, to tablet PCs, laptops, and stationary devices
such as PCs. Each of these devices has a different hardware configuration and capability, and
possibly a different software platform for program execution.
Device resource limitations: miniaturization of devices is one of the propelling factors for
the trend towards mobile computing. Many mobile devices are designed to be small for easy
portability. They are small not only in terms of form factor, but also computing power,
memory, display screen size and battery power. All these limit the type of computing that can
What is Pervasive Computing?
7
be done on these devices. Although the advance in technology will increase the capabilities of
these devices with no doubt, they will still have limited capabilities compare to desktop PC.
In the case where a screen size of a device is less compared to the original media size (e.g.
image), an image resizing adaptation mechanism can be applied to adjust the image size so
that it can fit with the device screen size. Another limitation can be a device without color
capability. In this case, it is possible to stream a variation scaled in the color domain (color-togray conversion, for instance).
Network limitation: the wireless connection used in the mobile devices is usually
bandwidth-limited and unstable when compared with the wired connection. At present, the
wireless technologies standards include Bluetooth, Wi-Fi, General Packet Radio Service
(GPRS), IEEE 802.11a, IEEE 802.11b, IEEE 802.11g and several others under development.
The comparatively slow speed of these technologies compared to wired network is always a
barrier for running large applications and moving large data to/from mobile devices.
High mobility: users in pervasive computing environments are highly mobile. Without the
need to have a fixed association with the computing devices, users can move freely from
place to place with or without any mobile device. For example, a user is carrying out a video
conference on his way home. He may be using the digital display in his car for the
conference. While walking from his car to his home, he switches the conference to his PDA.
Once he is at home, in his room, he continues the video conference on his personal computer.
This high mobility nature results in a continuous change of the computing environment, or
execution context that can affect the execution.
User preferences: while this is not a new phenomen specific to pervasive computing
environment, the increase mobility and the use of several user devices result in dynamic
preferences related to the user device, location, activity, role, knowledge or some other
factors.
The rest of the Chapter is organized as follows. Section 1.3 motivates the thesis by identifying
some scenarios that may benefit from the work in this thesis. Section 1.4 presents a short
overview of the proposed approach. It also lists some advantages and limitations of the
approach. Finally, Section 1.5 provides a road map to the rest of the thesis.
8
1.3 The Need for Content Adaptation
There is a huge amount of multimedia information being captured and produced for different
multimedia applications and the speed of generation is constantly increasing. While providing
the right information to a user is already difficult for structured information, it is much harder
in the case of large volume multimedia information. This is further complicated with the
emergence of new computing environments. For example, a user may want to access content
on the network through a handheld device connected by wireless link or from a high-end
desktop machine connected by broadband network. Content that can be presented on one
device might not be necessarily viewable on another device unless some content
transformation operations are applied.
Therefore, a content delivery system must be able to send the appropriate content for every
client in every situation. Efficient content delivery systems must fulfill some requirements to
address the wide range of clients, namely, content adaptation, minimal bandwidth
requirement, and fast real-time delivery [Chi02]. Content delivery is one of the research
topics that attracted a number of multimedia research community. We will limit our
discussion on the issues related to the problem of content adaptation and show how it solves
the problems arisen due to the inherent characteristics of pervasive computing. Content
adaptation is the process of customizing content so that it fits the user and environment
context.
Device heterogeneity: it would be very expensive and time consuming to create different
versions and modalities of the same multimedia content for different device types. This
problem is aggravated by the fact that multimedia contents have different modalities and each
with different formats. However, with a content adaptation facility, the same original content
can be delivered to different devices through a proper adaptation mechanism to convert the
format or modality of the content. For example, having a JPEG image data, we can send it to
a device which supports JPEG directly and after conversion to GIF to a device supporting
GIF.
Device resource limitation: in the case where a screen size of a device is less compared to
the original media size for example image, an image resizing adaptation mechanism can be
The Need for Content Adaptation
9
applied that adjusts the image size so that it can fit with the device screen size. Another
limitation can be a device without color capability: in this case, it is possible to stream a
variation scaled in the color domain (for instance, color-to-gray conversion).
Network limitation: one solution is to scale the network bandwidth to cope up the demand
but this is expensive and difficult; hence we need to minimize the demand for network
bandwidth. This can be achieved by creating a variation that fits the available bandwidth
through different size reduction methods such as color reduction, size reduction, format
conversion etc.
User preferences: for example, a user may be interested to see only the goal scenes of a
football game instead of the whole video. This can be achieved by extracting those particular
scenes and delivering them to the user. Another case scenario is that a user can only read
information in his language; if the pertinent information is written in another language, one
must apply an appropriate language translation to the information before it is delivered.
In summary, the need of content adaptation in pervasive environment is motivated by the
limited availability of resources, the user preferences, and the usage context. Figure 1.1 shows
the tradeoff between the Quality of Service (QoS) with respect to resources in a content
delivery system. As it can be seen from the figure, the goal of content adaptation is to deliver
the content using the available resources.
10
Best QoS
Adaptation
Application resource
requirements
Acceptable QoS
Available
resources
Low QoS
Guaranteed
Figure 1.1: Goal of Content Adaptation
1.4 Motivation
Concurrent with the developments in pervasive computing, advances in storage, networking
and authoring tools are driving the production of large amounts of rich multimedia content.
The result is a growing mismatch between the available rich content and the capabilities of
the client devices to access and process it. The mismatch between content requirements and
client capabilities impacts a number of multimedia applications.
Therefore, a solution to the mismatch problems, which is presented in this thesis, is to provide
a context aware content adaptation (or transformation) system which allows clients to access
to a range of media formats and within each media data to convert on the fly if necessary
these multimedia data to more appropriate formats. In this thesis we present a distributed
service-based content adaptation approach. To demonstrate the importance of content
adaptation in pervasive environments, we present three different application scenarios.
Motivation
11
Scenario I: Cooking guide
One of Mrs. Catherine’s hobbies is cooking. She watches her favorite cooking program on
TV. She gets interested in the recipe being explained so she decides to prepare a meal of the
recipe. The program uses a multimedia database system to store video clips on cooking
procedures, image and textual description of recipes and ingredients list. She logs to the
programs web site with her PC to see details of the recipe. Then, she goes to supermarket to
buy the ingredients. She connects to the system with her cell-phone and gets the list of the
ingredients. She comes back home, connects to the system with her cell-phone and prepares
the meal by listening to the recipe description.
Scenario II: Map and route service
Mr. Bob is a tourist who likes visiting museums. He is in Paris for a vacation and wants to
visit “Le Louvre”. Before going to the museum he wants to know the direction and location.
He connects to a location map and routing service system with his PDA. He discovers that the
system provides route guiding service. He activates the route guiding service with his
smartphone and starts driving to the museum. After the visit he likes to walk around the
district. He connects again to the location mapping and routing service with his PDA and
downloads the districts map. He visits the district appreciating architecture of the ancient
building. Since he wants to know history of those ancient buildings, he uses his mobile phone
to connect to a tourism site and walks around while listing a commentary about the sites he
traverses.
Scenario III: Medical Prescription
Consider a situation wherein a physician makes rounds at health center in Paris. The health
center is built with the state-of-art of modern health care system. This center is networked
with other health centers throughout Europe and allows anywhere/anytime patient data access
by doctors, nurses, caretakers and other hospital staffs. The physician is visiting a patient
from one of the European cities. The patient shows signs of a bronchial infection. Before
prescribing any new medication, the doctor wants to check the patient’s medical records
which may consist of media data such as text, image, video or audio on his wireless handheld
device (e.g. PDA). He connects to the system and gets her historical data from centers she has
12
previously visited and notes that she is already taking medications for high blood pressure and
heart problems and she is also allergic to penicillin. He then uses the handheld device to
check an online drug database for the most effective treatment compatible with the patient’s
other medications. The doctor fills out a short form on the handheld device screen to authorize
and send the prescription via a wireless network to a pharmacy.
The above three scenarios shows that users are mobile, they have different access devices
with varying processing capacities, display capabilities etc. The information sources are
multimedia, that is, they consist of different media types such as text, image, video, and
audio. On one hand we have limited device and network connections. On the other hand
multimedia requires large resource consumption. Moreover, the user needs may vary with
device and location. To fill the gap between these two requirements, content adaptation or
transformation becomes crucial.
1.5 A Short Overview of Our Approach
One of the architectural issues for content adaptation is the decision where the adaptation
processes resides. Based on the location of the adaptation processes, three approaches have
been proposed: server-based, client-based and proxy-based.
In server-based approaches, the functionality of the content server (provider) is extended so
that it also performs content transformation activities. This approach might produce better
adaptation results as it has more knowledge on the content, however, the processing overhead
of the transformation leads to poor user response experience and server overload.
In client-based approaches, the transformation is done on the client devices after receiving the
content from the server. It is mostly effective in situations where the transmission
characteristics of the end-to-end path are less important than the limitations of the displaying
device. However, this solution performs poorly in situations where the transmission
characteristics of the end-to-end path are also the point of consideration rather than only the
displaying ones and it has poor extensibility (adding an adaptation tool requires modifying all
the clients).
A Short Overview of Our Approach
13
In proxy-based appraoches, an intermediate proxy which is implemented usually at the access
point, intercepts the content coming from the server and applies different transformations
according to the device characteristics and the network connection. By introducing the proxy,
users will experience better performance as the server is relieved from the transformation
tasks. But, this approach has still a limited scalability because of significant computational
costs of adaptation operations such as key-frame extraction from video [Lum02a].
In this thesis we propose a distributed service-based content adaptation approach. This
approach uses adaptation services that perform basic transformation operations and are the
building blocks for processing multimedia media. Through chaining of these services,
complex adaptation scenarios can be achieved. The decision on the number and the type of
transformations is done by a proxy placed between the client and the server and more
appropriately close to the client. Because of their flexibility and distributed nature, web
services are a good choice for the adaptation services. This approach tries to solve
performance problems of content adaptation as the adaptation is done by third party external
services each implementing a particular transformation tool. It also provides a general
solution of content adaptation that can be deployed to any type of application. Moreover, it
addresses the interoperability, extensibility, scalability and flexibility issues of content
adaptation which are important requirements in pervasive application systems by proposing a
general architectural framework.
The major contributions of this thesis are the following:
•
We introduce a novel service-based content adaptation architectural framework for
distributed multimedia systems. Using this architecture, third-party adaptation services
can be easily included into the architecture which makes the framework widely
extensible. Moreover, the framework is scalable and interoperable so that we can use
it for wide application areas.
•
Though in many instances web service technologies are mentioned as good solutions
for distributed computing especially on the Interent, for the best of our knowledge
their application for pervasive computing, particularly for content adaptation is
investigated for the first time in this thesis.
•
We provide a formal definition and description of multimedia content adaptation
process. This formalism enables to understand the different elements involved in
deciding the adaptation process so that different mechanisms can be developed. We
14
also define transformation rules that are easily extensible to add new transformation
functionalities and user constraints.
•
The design of the framework allows the composition of adaptation services to provide
optimal service for a given computational context. Therefore, any adaptation need is
achievable.
•
We propose an ontology for multimedia adaptation services description. The ontology
helps to describe services semantically. The semantic description facilitates automatic
discovery, composition and execution of the adaptation services.
•
A planning based algorithm is developed for adaptation service composition.
•
A prototype has been developed that implement the framework. The prototype
demonstrated not only the validity of our proposal but that our solution could be used
for real application systems.
•
Various experiments were conducted on the prototype to measure the overhead of
applying one or more transformations and of the adaptation graph construction
algorithm. The experiments showed that the overhead of the graph construction
algorithm is not significant compared to the total latency time introduced by the
adaptation process.
1.6 Organization of the Thesis
This dissertation is organized as follows:
•
Chapter 2 gives a literature review on content adaptation systems and service oriented
architectures.
•
Chapter 3 gives a brief overview of Service-Oriented Architecture and web services
•
Chapter 4 discusses about content metadata description tools and standards
•
Chapter 5 discusses about context metadata description tools and standards
•
Chapter 6 presents our service-based architecture framework and its key concepts
•
Chapter 7 details the prototype implementation of our architecture
•
Chapter 8 provides a brief discussion on our appraoch
•
Chapter 9 summarizes major contributions of the thesis and describes our future work.
Chapter
2
Related Work on Content
Adaptation
The objective of this chapter is to review the works done on the multimedia content
adaptation for pervasive application systems. More specifically different adaptation
techniques and adaptation approaches proposed in the literature will be discussed. The rest of
the chapter is organized as follows. In section 2.1 we cite main academic and commercial
pervasive computing projects, discuss the difficulties of multimedia access in heterogeneous
environments and the importance of content adaptation to solve these difficulties. Section 2.2
first assesses works in the area of multimedia content adaptation including issues related to
adaptation techniques, adaptation architectures and adaptation systems. Then a short summary
of actually implemented adaptation systems, their functionalities and their drawbacks are
presented. Section 2.3 is devoted to service-oriented architectures and their features. Web
service languages standards and related technologies are presented in Section 2.4. Section 2.5
deals with works related to web service composition particularly workflow techniques and AI
planning approaches. Finally, section 2.6 summarizes the chapter.
2.1 Introduction
Many citations do not distinguish between “pervasive” and “ubiquitous” when it comes to
computing visions. Some citations argue that the difference exists: pervasive computing aims
to make information available everywhere while ubiquitous computing requires information
to be available everywhere [Goff04]. Although the pervasive-computing essentials have
emerged, the ubiquitous-computing age is still yet to come. On the other hand, we have seen
major pervasive/ubiquitous computing projects booming in academia, such as the project
16
Chapter 2. Related Work on Content Adaptation
Aura at Carnegie Mellon University (CMU)3, Endeavour at UC Berkeley4, Oxygen at MIT5,
and Portolano at the University of Washington6. Industry examples include work at AT&T
Research, HP Labs, IBM Research, Intel Research, and many others. Each of these efforts
addresses a different mix of issues and a different blend of near-term and far-term goals. In
this thesis both terms are used synonyms.
Pervasive computing systems try to make life easier for users by deploying various devices
and supporting middlewares. But real difficulties still lie at the application front. Today,
applications are typically developed for specific device classes or system platforms, leading to
separate versions of the same application for handled, desktops, and cluster-based servers. As
heterogeneity increases, developing applications that run across all platforms will become
exceedingly difficult [Saha03]. Firstly, wireless terminals for examples, cell phones, PDAs,
notebook computers etc. are inherently resource-limited. Comparing them to desktop
computers with LAN (Local Area Network) connections, the wireless terminals have smaller
display, limited computation powers and energy consumption, and narrower bandwidths on
their network links. These limited resources can easily be overwhelmed by bulky multimedia
data that are originally intended for wirelined computers.
Secondly, the collection of wireless user terminals is heterogeneous. These terminals run
different operating systems, have different capacity, and may have different installed sets of
applications. Furthermore, they may use some proprietary multimedia formats that other
terminals do not use. As a result, it becomes more and more difficult for the providers of
multimedia contents to decide a priori what file formats are suitable for the terminals of the
target audience.
Thirdly, different users may have different needs. They use different terminals and have their
own preferences as to how multimedia data should be received and presented. Users may have
also different language preferences depending on their background or education.
There are two extreme solutions to these problems, which, however, can be easily
disqualified. The first solution is to have multiple devices and multiple connections, one for
3
Aura project: http://www.cs.cmu.edu/~aura/
Endeavour project http://endeavour.cs.berkeley.edu/
5
Oxygen project: http://oxygen.lcs.mit.edu/
6
Portolano: http://portolano.cs.washington.edu/
4
Content Adaptation
17
each media type. This is far from an optimal solution. Users prefer to operate as few devices
as possible, but they also have specific priorities for some of them (e.g., weight and energy
issues for mobile devices). The other extreme solution would require one device to be capable
of presenting every type of multimedia content. This approach is also disqualified for similar
reasons, in addition to limited resources and cost issues that the production of such a device
would introduce.
The concept of content adaptation has been introduced to bridge the gap between these two
extreme solutions. Instead of having each medium stored in multiple representations, each
matching the characteristics of a probable end-device, the source can provide only one or a
few representations and rely on adaptation functionalities to deliver the content in the
appropriate format at the receiver. In addition, devices with transmission and display
characteristics that prevent them from accessing certain types of media are able to do so by
having the representation of the content adapted to a form suitable for them, with as small an
impact to its perceptual value as possible. Thus, users gain access to a great variety of media,
virtually with any possible combination of connection and device type. In the following
section we will present techniques and approaches of content adaptation and examples of
application systems implementing adaptation functionalities.
2.2 Content Adaptation
Content adaptation or multimedia content adaptation refers to the process of customizing or
tailoring content to suit to the user’s computing environment such as location, device
capability, network bandwidth, user’s preferences and usage context [Lum02b, Hori00].
Content adaptation is considered an effective and attractive solution to the problem of
mismatch in content format, network connectivity, device capability and user’s preferences in
pervasive computing environment [Held02]. Using an adaptation technique, content can be
filtered, transformed, converted or reformed to make it accessible by several devices, to
exploit specific application requirements for customized content and to personalize general
content for different devices and users. By tailoring content, users have become accustomed
to particular interfaces, presentations and preferences. For example, a user is receiving video
broadcast football on his mobile phone while driving to his home. At this time, the system
detects that the user does not have enough network connection to send the video of the match
18
so it sends him key-frames showing goals. While arriving in his home the user switches to PC
and continue getting the video but this time the full video since the user has good network
connection. What we can see from this scenario is that the system sends the key-frames by
applying adaptation it provides the user and then sends him the full video when the condition
is possible.
The adaptation process does not only tackle the problem of resource constraints but it also
increases user satisfaction and reachablity. For example, a visual impaired person can not
access visual information unless the system provides him/her in audio form; on the other hand
for hearing impaired person visual form is more appropriate. Another interesting example is
the case where a French speaking person accesses to information sources written in English.
In this case the person does not understand English hence can not understand the content
unless there is a mechanism to produce the content in his/her language that is French. All
these example scenarios show us the importance of content adaptation for universal
information access in the world of pervasive computing.
2.2.1 Content Adaptation Techniques
The introduction of pervasive computing systems in the last decade has made multimedia
content adaptation an important research focus. Since then, several adaptation techniques have
been developed to deliver multimedia data to users in heterogeneous environments. Currently
available techniques apply textual transformation [Nagao01, Hous96, Lilj95], image
transcoding [Wee03, Lee03, Chandra00, Smith98], video and audio processing [Libse04,
Shan00, Vetro02, Bolot98,]. A list of content adaptation technologies that can be applied to
the basic media types: text, image, audio and video have been identified by Lei [Lei01] and
Narendra [Shaha01]. The list is presented in Table 2.1.
Content Adaptation
19
Category
Text
Image
Video
Audio
Transcoding
- format
- data size
- frame rate
- audio stereo-
conversion
- font size
reduction
reduction
- dimension
reduction
- spatial
reduction
resolution
- color-depth
reduction
reduction
resolution
grayscale
reduction
- format
conversion
conversion
- format
conversion
- temporal
- color-to-
transformation
to-mono
- color-depth
reduction
- format
conversion
Transmoding
- text-to-audio
transformation
- image to text
- video-to-image
transformation
- audio-to-text
transformation
- video-to-text
transformation
- video-to-audio
transformation
Summarization - text
summarization
Translation
- language
translation
- key frame
- audio highlight
extraction
- language
translation
- language
translation
Table 2.1: Media Types and Content Adaptation Techniques [Lei01 and Shaha01]
Classifying the adaptation technologies is mandatory for developing a general decision
making strategy for delivering adapted content to users according to their context.
In a more general sense, content adaptation techniques can be broadly classified as semantic
adaptation and physical adaptation. We briefly present each classification as follows:
20
2.2.1.1
Semantic adaptation
According to [Lei01], semantic adaptation is a selective process. Based on available and
requested information, multiple pieces of information are combined into a complete
presentation. Semantic adaptation is affected by the semantic structure of a presentation,
which determines the relative data in the final presentation. For example, most current ecommerce web sites contain many images of banners, logos, and advertisements. These data
often consume a good deal of network bandwidth, and are non relevant or not of interest to a
user. We can drop less important data under bandwidth constraints or, we can provide
progressive delivery to send out the more important data first (such as low resolution images)
and then deliver the less important data to enhance the information later (such as
reconstruction of high resolution images). In this way, we can improve the user’s access
experience by efficiently utilizing available network bandwidth. [Ma00] have identified data
prioritization and purpose classification for semantic content adaptation.
Data prioritization: the aim is to distinguish the more important part of the data from the less
important part so that different quality service levels can be provided when delivering the data
through the network [Shaha01]. Data prioritization can be achieved within a single media type
by using special encoding schemes such as layered coding and multi-resolution
comprehension [McCan97, Taub94]. For example, in case of video content, the adaptation
may create a smaller duration variation of the content, usually summary, by selecting the
temporal (and may be also spatial) segments that are more relevant according to some criteria
(e.g. user preferences).
Purpose classification: a typical web page contains a lot of information and media objects
that are redundant or may not be of interest to a user. For example, an e-commerce web site
may have multiple images for linking to the same product site on the top, bottom and side of
the page. Purpose classification [Shaha01, Peak98] deals with classifying the purpose of each
media object in a Web page, to improve the efficiency of information delivery by either
removing redundant objects (assuming the related copyright issues have been properly
addressed) or prioritizing them according to importance.
Data appropriateness is also important for semantic adaptation. A typical application case is
a 10 year old kid who would like to visit a museum. He is really interested in dolphins and
Content Adaptation
21
consequently he proceeds to the ocean section, where he receives on his mobile device the
catalogue. Browsing the catalogue, he selects the chapter related to the dolphins in order to
watch the corresponding video. The original video with a length of 5 minutes consists of
different themes: (1) the search for food, (2) dolphin’s enemies and (3) games. The kid has no
particular interest on specific themes. He just wants to see dolphin’s images. Therefore, he
receives an adapted video, where particularly violent and bloody scenes (e.g., sharks wildly
devouring dolphins) are dropped. Emphasis is instead put on nicer scenes, e.g., dolphins
playing with each other, mother with its little dolphin.
To facilitate semantic adaptation it is important to add semantic information (priority, purpose
and appropriateness) through annotations during the content creation or later. Annotation
refers to the processing of content in order to provide additional information and metainformation, such as summaries, keywords, highlights, etc. Multimedia annotation is time
consuming and requires a large storage space which makes semantic adaptation sometimes
difficult. Consequently most of the adaptation techniques developed so far focus on physical
content adaptation. This thesis is within this context. In the following section we present some
of the physical adaptation technologies.
2.2.1.2
Physical adaptation
Physical adaptation of media is defined as the combination of conversion, scaling and
distillation processes guided by the characteristics of media format and physical QoS [Lei01].
Physical adaptation technologies can further be classified based on target context, time (when
the adaptation is performed), and number of media types involved in the adaptation. The
different classifications are presented in the following section (A-D).
(A)
Based on the target adaptation context, adaptation techniques can be divided into
technical infrastructure (terminal and network), user preferences or user context/location:
•
Terminal and network driven adaptation: The consumption device and the network
capabilities may pose a serious barrier to the content distribution and consumption.
Current pervasive devices vary widely in their features in terms of hardware (screen
size, resolution, color depth, computing power, storage space, …) and software
(applications, codecs, …). They also use a variety of network connections from cable
22
to wireless, with different effective bandwidth and network latency. That implies that
the data transmitted must be adapted to fit to the terminal and network connection
capacity. This category of adaptations aims to reduce the content consumption
requirements by adjusting the source content to the device and network capabilities.
For example, in order to display images on devices with a small screen and limited
display capability, reducing the size or resolution for each image will help to fit the
image on the small screens of devices. Techniques such as transcoding, transmoding
and scalable coding are used for terminal and network driven adaptations [Maga04].
Using transcoding content resource can be reduced so that it matches the available
terminal and network capabilities [Lei01b]. Some examples of transcoding are format
conversion, temporal/spatial resolution reduction and color reduction. Transcoding
requires computational power and memory and thus it may be quite expensive to
achieve a large scale adaptation deployment when many on-the-fly adaptations are
required [Maga04, Lum02b].
Transmoding is defined to be the adaptation of a digital media by transformation of its
original modality to another modality [Pere05, Asadi04]. Transmoding (or modality
transformation) is necessary when the usage environment conditions do not allow
consuming the content with its original media types. Some examples of transmoding
are text to speech, video to still image and video to text.
Scalable coding is used to compress video contents in a scalable fashion. Scalable
video usually consists of a series of layered streams. There are three basic scalabilities
on video content [Shen06]: Signal Noise Ratio (SNR) scalability for picture quality,
spatial scalability for frame size, and temporal scalability for frame rate. In many
practical applications, one scalable stream may contain more than one kind of basic
scalability. Each layer can be partially and even completely dropped during
transmission and decoding.
A.Fox et al. [Fox98] developed an architecture that adapts Internet content to network
and client variations. The architecture uses datatype-specific lossy compression to
increase the quality of service for the client and reduce the end-to-end latency
perceived by the client. The Odyssey system [Noble95] supports a form of end-to-end
Content Adaptation
23
bandwidth management by providing a fixed number of representations of data objects
on a server and specifying an Application Program Interface (API) by which clients
track their network characteristics and negotiate for a representation that is appropriate
for their current connectivity. Since the data representations are not exhaustive, that is
they do not represent all possible network connections, a user request which requires a
different representation can not be answered by the server. Moreover, generating
different representations may require large storage space.
•
User characteristics/preferences driven adaptation: Constraints concerning
terminals and networks are technical, yet it exists one more criterion to consider, the
user [Zang00]. It is certain that the user is the one who works with the data, so it is
very important to consider his/her needs, preferences, priorities, role, skills, interests,
characteristics…etc. For a specific multimedia presentation, individual users may have
different requirements on the level of details or some other parameters [Boll01]. For
example, consider a professor on campus who is interested to see in-depth multimedia
material on coronary artery bypass grafting, and an undergraduate student at home
who needs to get only an abstraction of the same material. Users may also have
different language preferences. For example, a user may be more comfortable to
access data in French than English. In this case a language translator can be used to
translate the data from French to English. Content adaptation might also be essential to
facilitate data access for handicapped users [Pere05, Maga04]. For example, a hearing
impaired user may need transformation of speech to text. Conversely a vision impaired
user may need transformation of text to speech.
•
User location/context driven adaptation: Due to the terminal mobility, it is also
important to consider content adaptation based on the user location and/or context.
This is because the natural environment may limit the user capabilities and thus the
user may need a specific adaptation to maximize his/her access conditions. For
instance, if a user is driving, his/her attention (and his/her eyes) is focused on the road,
and thus voice content (text to voice) must be provided. Conversely if a user is in a
meeting he may favor silent content delivery that is transforming voice to text. It is
important to note that the user location/context driven adaptation is not the same as
location aware queries (e.g. where is the nearest gaz station?).
24
(B)
Based on the number of media types involved in the adaptation process, adaptation
technologies can be divided into single or cross-media adaptation [Lei01]:
•
Single media adaptation: In single media adaptation, different versions of a media
element are produced with varying features, qualities and resource requirements. Most
of current best-effort adaptation techniques consider switching between different
qualities or formats of single media elements in order to fit the receiving device
capabilities and user interest. For example, most images can be significantly
compressed without an appreciable decrease in quality. Another example is GIF-toJPEG or color-to-grayscale transformations that can be used to reduce the physical
size of an image.
•
Cross-media adaptation: In cross-media adaptation (also called transmoding)
[Boll99], content is transformed from one media type to another so that the content
can be processed by a particular device. For instance, some handheld computers may
not be capable to handle video data due to their hardware and software constraints or
network connections. Thus, transforming video into sets of images and extracting
audio or caption will enable the devices to access to the information contained in the
video. In this case, users will be able to receive useful information in whatever form
their devices can handle. Transmoding also helps disabled people to access data as
mentioned above (section A).
(C)
We can also classify adaptation techniques as structural transformation or media
transformation [Leml03]:
•
Structural transformation: techniques in this category relate to the transformations
applied to the total organization of a document. Some examples of such
transformations are HTML to XHTML BASIC for the mobile terminals, filtering of
HTML documents, transformation of textual contents written in XML to a graphical
representation in SVG, etc. Structural transformation can preserve the media resources
used by the document source as it can filter them.
Content Adaptation
•
25
Media transformation: this category consists of adaptation and encoding of media
objects. For example adaptation of image and video by applying color reduction or
color-to-gray, resizing, changing format encoding etc. Several works have developed
techniques for adaptation of media resources such as image [Wee03, Lee03] and video
[Libse04, Shan00, Vetro02].
(D)
Finally, adaptation techniques can be classified into static or dynamic depending on
the time when the adapted content is created [Lei01, Lum02a, and Chang02]:
•
Static (off-line) adaptation: the content transformation is generated when the
resource (the content) is created or updated. This often involves a human designer to
hand-tailor the content to some specific requirements. The multiple variants of the
same resource are stored in the server and selected to match the client specifications.
For example, the server can store several versions of an original video with different
sizes, formats and resolutions and according to different connection QoS links. The
appropriate version will be selected at runtime depending on the usage context. For
instance, in order to eliminate extra processing overhead at presentation time, most
current web sites create multiple versions of information at authoring time. For
example, a bilingual site is a typical case where an off-line approach may be
convenient. The Odyssey system [Noble97] uses a static adaptation approach. The
Odyssey server has several pre-generated versions of the data with different fidelity
levels, and the appropriate one is chosen at run-time according to the resources, or
even energy, available.
Static adaptation has three main advantages: (1) it is highly customized to specific
classes of client devices (2) it does not require any runtime processing, so no delay is
incurred, and (3) the content creator has the full control on how the content is
formatted and delivered to the client. On the other hand, static adaptation has a number
of disadvantages, mainly related to the management and maintenance of different
variants of the same content [Lum02a, Leml01]: (1) different content formats need to
be created for each sort of device or class of devices, and new content versions should
be created when new devices are introduced, (2) it is unsuitable when the content is
dynamically generated or highly volatile, and (3) it requires large storage space to
keep all variants of the same content.
26
•
Dynamic (on-the-fly) adaptation: In dynamic adaptation [Huan04, Sing04, Card03,
Chang02, Mahe02], the desired content is generated on the fly just before delivering to
the client according to the current context of the user and computing environment. The
dynamic approach has the maximum flexibility and generality, and frees the provider
from creating in advance and keeping consistent multiple versions of its resources.
However, as stated by [Lum02b], the adaptation overhead introduces latency in the
delivery.
Classifying adaptation techniques is important as it helps to understand the different
techniques and use them in different application scenarios. It is also useful for developing a
general decision-making strategy to optimize the adapted content delivery.
2.2.2 Content Adaptation Approaches
One major architectural design issue for developing adaptation functionality in pervasive
computing environment is to decide where the decision making and the adaptation operations
reside. There are three basic approaches for locating adaptation functionality on the path of
data delivery from the source to the destination: (i) at the source, (ii) at the destination or
client, and (iii) in between, somewhere in the network, with some special locations having
special properties, e.g., at the boundary between the wireless and the wired parts of the
network. In this section we will examine in detail each design approach with its advantages
and drawbacks.
2.2.2.1
Server-Based (or Source) Adaptation Approach
The first approach puts the adaptation functionality at the source (or server) as shown in
Figure 2.1. In server based approaches such as [Marg01, Mogul01, Mohan99, Bhrag98,
Bolot98], the server is responsible for discovering what the client capabilities are and how
much bandwidth is available. It then decides on the best adaptation strategy.
Content Adaptation
27
Adaptation
Point
Wireless Device Base Station
Application
Server
Figure 2.1: Server-Based or Source Adaptation
The server based approach provides more control since the adaptation can be tied to the
content authoring process, allowing the author to provide hints on the adaptation for different
circumstances. An author can also preview the adaptation outcome under different viewer
preferences and conditions. For secured environment, such as in e-commerce applications,
contents are usually encrypted in which case only the server or client can perform the
adaptation. In addition to technical issues, it is necessary to consider the copyright and
business implications of adaptive content delivery. Modifying content rewritten by someone
else may incur copyright infringement. In a server based apporach, this liability is avoided
since the content provider has control over how the content is to be transformed. Moreover,
server-side solutions allow an additional alternative for content adaptation that is dynamic or
static solutions.
The major drawback to server-based adaptation is its limited scope. Deploying adaptation in
the content server only affects targeted content from that server. It is not appropriate when a
client is viewing content from a range of servers. Performing adaptation on the content server
will also impose an additional load on the resources of the server, and so may require
additional infrastructure enhancements. Furthermore, the main problem of content delivery in
a pervasive environment lies at the last hop, that is, where the wireless connection is, hence
having the transformation process at this point will be more appropriate that is what proxy-
28
based approach is. Finally, though the server-based approach allows off-line adaptation,
generation of multiple versions of the content makes content management more cumbersome
and requires more storage. In conclusion, the simplicity of the server-based solution makes it
efficient for simple cases with small variability, but it can not stand as a viable solution under
extremely variable conditions.
2.2.2.2
Client-Based (or Receiver) Adaptation Approach
The second approach in locating the adaptation functionality is to have it reside at the receiver
(or client) as depicted in Figure 2.2. In client-based adaptation [Marr02, Lei01, Björk99,
Yosh97, Fisher97], the required transcoding is performed by the client device taking into
account the device’s capabilities which might not otherwise be available to the server. The
source remains unchanged and continues to transmit the same quality content to the receiver.
The receiver, upon reception of the content, transforms it to suitable form that it is capable of
presenting.
Client-based adaptation can be done in two ways, either by selecting the best representation or
by performing transformation by the client device. Selection of the best representation for a
response is performed by the user agent after receiving an initial response from the server.
The server sends a list of available content variants with their resource requirements, then the
agent selects, based on client information. Some client devices transform the content at the
device, for example, Windows-CE devices (e.g. SmartPhone) change the color-depth (for
example, from 24 bit color to 4 bit gray-level) of images. Opera Software [Opera04] uses
proprietary client-side adaptation technology called Small-Screen Rendering in its mobile
browsers. This technology reformats web sites to fit a smaller screen eliminating the need for
horizontal scrolling. Similarly, Opera’s Medium-Screen Rendering identifies a web page’s
content and adapts its elements to fit on medium-sized screens ranging from PDAs used in
landscape mode to low-resolution TV screens.
Content Adaptation
29
Adaptation
Point
Wireless Device
Base Station
Application
Server
Figure 2.2: Client-Based or Receiver Adaptation
The client-based approach requires minimal changes at the receiver and can be extremely
efficient in customizing the content to meet exactly its capabilities. The user profile which
includes preferences, location and role is an important parameter in determining the
adaptation. In the case of server-based approach, the profile information should be
communicated to the server and this may pose privacy issues. However, in client-based
approach, it is the client who manages the profile and other context-sensitive data so it avoids
the risk. The client-based approach is mostly effective in situations where the transmission
characteristics of the end-to-end path are less of a concern than the limitations of the
displaying device [Nand99].
Indeed, this solution performs poorly in situations where the transmission characteristics of
the end-to-end path are also the point of consideration. Having a desktop connected to the
Internet using a low speed modem, for example, renders the receiver adaptation mechanism
useless, since it resides after the critical part of the end-to-end path; the adaptation mechanism
is applied to the stream after the stream has already congested the low-speed modem link.
Therefore, whatever transformation it applies, it can not prevent the Internet Service
Provider’s (ISP’s) modem from dropping a significant amount of packets of this stream which
overflows the link. Since both the device display and the transmission characteristics are
30
important factors to determine content adaptation in pervasive environment, applicability of
this solution as a generic adaptation approach is limited.
Moreover, the complexity of adaptation mechanisms which is usually high also hinders the
wide acceptance of this approach [Perk01]. Since a significant proportion of pervasive devices
have limited CPU power, memory, energy, and storage, implementing such resourcedemanding processes on them might not be feasible or efficient. In particular, the complexity
of algorithms for transformations of text and image are fairly low, but transcoding algorithms
for audio and video are complex and demanding, making their implementation on small and
not powerful devices, such as PDAs and mobile phones, extremely difficult.
2.2.2.3
Proxy-Based Adaptation Approach
The third approach in locating the adaptation mechanism is a compromise between the two
extremes discussed above. It places the mechanism within the end-to-end path, at an
intermediate node identified as the most appropriate for performing effectively the adaptation.
Some examples of proxy-based systems are [Sing04, Wee03, Kim03, Chandra00, Fox98a
Angin98, Bick97, Noble97]. The intermediate node usually denoted as proxy or gateway (See
Figure 2.3), intercepts the reply from the server, decides on and performs the adaptation
according to the user’s preferences, device capabilities and network conditions, and sends the
adapted content to the client.
Content Adaptation
31
Application
Server
Wireless Device Base Station
Adaptation
Point
Proxy
Server
Figure 2.3: Proxy-Based Adaptation
There are several advantages that come with the adoption of the proxy solution. First,
intermediate adaptation shifts computational load from content server to proxy server.
Second, the proxy can be located at the most critical position in the end-to-end path. It is
usually assumed that the bandwidth between the proxy and the server is much higher than
between the client and the proxy [Antonio97], so that the time to download the original
content from the server to the proxy is less compared to the time required from proxy to the
client. For example, let us consider a case that a wireless device accesses to the Internet
through a base station (router) which is connected by wired line. In this case, the last
(wireless) hop is expected to be the bottleneck, which indicates that the proxy will perform
ideally if it resides at the router or as close to it as possible. Putting at the last hop enables the
proxy to have a global picture of the environment resources such as network latency and
bandwidth, content size to be transported before the final delivery, user preferences, any
changes that may occur, etc. Therefore, the flexibility of locating the adaptation mechanism at
the best position in the end-to-end path is an important advantage of the proxy solution over
the other two approaches. The proxy server also plays another important role, that is, it can
cache the results of content adaptation, thus avoiding some roundtrips to the content server
and costly transcoding operations when resources can be served from the cache [Singh04,
Fox97].
32
Third, adapting on the proxy means that there is no need to change the existing clients and
servers, and it achieves economy of scale more than a server-based approach since each proxy
can transform content for many servers. One potential problem of this approach is that the
proxy might not have access to context information needed for a satisfactory operation during
transcoding. This problem can be solved by providing context information containing implicit
or explicit guidance to the proxy as in the case of context-aware adaptation [Lei01a, Yau01,
Dey99a]. For example, in server-directed transcoding [Mogul01], the server provides the
context information.
Naturally, the proxy solution also presents some disadvantages. First, it has limited scalability
because of significant computational costs of adaptation operations [Lum02b]. Second,
having an intermediate intercepting the content raises security issues. The (usually) thirdparty operating the proxy must be trusted by the receiver and the source. In the absence of a
trusted third party the proxy approach has no way of adapting the multimedia content. In
addition, the third party may charge for the service it provides and the resources it employs to
perform the adaptation for the receiver. Therefore, accounting mechanisms should be
incorporated in the proxy solution in order to keep track of the amount of resources utilized
and the usage of the data. Third, the list of content adaptation tools is not exhaustive and
might include other in the future so accommodating these new tools may not be possible if
especially the proxy is not extensible. Finally, putting all adaptation functionalities into the
proxy requires powerful CPUs and a lot of memory.
2.2.2.4
Distributed and Agent Based Adaptation Approach
A distributed based approach for content adaptation is quite recent. There are very few works
such as MARCH [Ardon03], Ninja [Gribb01] and DANAE [Hutter05] that propose platforms
based on this approach. The Mobile Aware server aRCHitecture (MARCH) [Ardon03] project
enables the creation of a service-based overlay network, by deploying several proxies at
various points along the data path between the client and the server. MARCH is a servercentric approach in which the decision on the adaptation is done at Mobile Aware Server
(MAS) which is the front end to the content servers. The archicteure has a dynamic proxy
execution environment called Compute Servers (CS) that facilitates uploading of proxies on
session basis and allowing the placement of the required proxies at optimal points along the
data path. The Mobile Aware Server (MAS) is responsible for determining what proxies are
Content Adaptation
33
required to provide the optimum service for a set of operating conditions and deciding where
to execute these proxies. MARCH allows using several proxies to compose the required
service. In the MARCH solution, a client entity needs to be installed in the client device in
order to route applications’ traffic to proxies. This makes MARCH less transparent and also
lacks interoperability and flexibility. Moreover, the decision for the adaptation is done at the
server hence it shares the drawbacks of the server-based approaches.
In the Ninja project [Gribb01], several proxies, each with a specific transcoding capability are
put at different locations in the end-to-end path and the data passes through the path while
being adapted by each proxy according to the need. The project conceives a mechanism to
dynamically establish an adaptation path through several proxies to compose a complex
adaptation service. While this resolves the computational problem, it has its drawbacks. First,
Ninja is intended to provide robust cluster-based services, and it does not consider dynamic
adaptation of data paths or of the paths’ components [Bust01]. Second, it can handle only a
sub-set of adaptation techniques such as image transcoding. Third, it is not easily deployable
as one needs to put the proxies at different points in the delivery path. Finally, no mechanism
is developed to select an optimal adaptation path.
The Dynamic and distributed Adaptation of scalable multimedia coNtent in a context-Aware
Environment (DANAE) [Hutter05] is an IST7 European co-funded project that proposes a
platform for dynamic and distributed adaptation of scalable multimedia content in a contextaware environment. The project covers three major works: the definition of scalable media
formats, the adaptation of multimedia content to session context and the transport and
delivery of multimedia content to the end user. Specific feature of the DANAE platform is
that it provides distributed adaptation by offering adaptation both at the content server and at
intermediate network nodes. This is achieved by implementing the adaptation decision engine
at the different nodes (adaptation nodes). The adaptation nodes collect context information
from the client nodes, merge the context information (by enriching with the adaptation nodespecific descriptors such as supported adaptations) and send the merged context information
to the content server. The content server adapts the content to the quality level requested by
the adaptation node based on the merged context information. The adapted content
corresponds to the highest media quality which satisfies the requests of all terminals
7
IST: Information Society Technology
34
connected to the adaptation node. The content server sends the adapted content together with
its content-related metadata to the adaptation node. The adaptation node may further adapt the
content to provide it in optimum way to each terminal.
DANAE is designed based on MPEG-21. This limits its use other than MPEG-21 formats.
While DANAE allows distributed adaptation by using several adaptation nodes, both the
adaptation tool(s) and the adaptation decision engine should be implemented at each node.
Moreover, each node may not have the same type of adaptation tool(s) which further
complicates the implementation of the adaptation nodes.
Angela et al. [Ramos05] developed agent-based framework called PUMAS (Peer Ubiquitous
Multi-Agent Systems) to provide adapted Web content. The framework consists of four
Multi-Agent Systems (MAS) each providing particular function and communicates with other.
The agents are in charge of connection, communication, information transmission and
adaptation. The Connection MAS provides mechanisms for facilitating the connection from
different kinds of devices to the system. The Communication MAS ensures a transparent
communication between the user device and the system. It also adapts information according
to the technical constraints of the user’s device. The Information MAS receives user queries,
redirects them to the right source of information (e.g. the one which is the most likely to
answer the query). This MAS adapts information according to the user’s profile in the system
(preferences) and returns filtered result to the communication MAS. In charge of the
adaptation, the agents of the Adaptation MAS communicate with the agents of the three other
MAS in order to exchange information about user and about the context. The system considers
only one aspect of adaptation which is content filtering while other adaptations are very
important to deliver multimedia content such as transcoding, transmoding etc. Moreover the
system does not have a mechanism to provide a complex adaptation need that can be realized
by chaining of adaptation agents. For example to deliver an MPEG-1 format video with high
frame rate to a device supporting only MPEG-4 video and with less bandwidth connection, we
need to apply at least format conversion (MPEG-1 to MPEG-4) and frame rate reduction. This
type of adaptation scenario can not be realized without chaining adaptation agents but the
system does not have such mechanism.
Content Adaptation
2.2.2.5
35
Ongoing Standardization for Content Adaptation
The IETF8 Working Group has recently proposed the Open Pluggable Edge Services (OPES)
[Barbir02] and the Internet Content Adaptation Protocol (iCAP) [Elson01] defining the basic
functions of future caching proxy. iCAP defined how various caching objects can be
transported from one cache to another. OPES provides some interesting capabilities to
caching proxies. An OPES proxy can be equipped with message parsers, rule modules, and
proxylets library. When messages flow through an OPES proxy, not only it is cached but they
can also be automatically parsed and processed with these rules. The OPES proxylets can
execute the processing in the caching proxies, or optionally can call for remote callout
services via protocols such as iCAP [Lee01]. This approach does not provide enough
flexibility in accommodating various service arrangements that may arise in the real service
deployments, which often restricts where, when and how the service can be performed,
redirected, and who may provide the service specifications. Second, OPES is limited to Web
content and can not be used with any distributed multimedia applications. Finally, OPES is
not developed particularly for content adaptation purpose, it also includes localized
advertisements, virus scanning, personalized data insertion, etc. hence it is not optimized.
The MPEG-21 [Burn03] is a work in progress aiming to enable the use of multimedia
resources across a wide range of network devices by developing an open framework for
delivery and consumption. One part of MPEG-21 is the Digital Item Adaptation, that defines
description tools for usage environment and content format features for the purpose of
transparent access to the multimedia content. These tools enable the adaptation engine to
make better decision in order to provide the best content to the user. The implementation of
the engine that actually performs the required adaptation steps to the multimedia content is
left to tool vendors and is beyond the scope of standardization. The MPEG-21 DIA tools
guarantee an efficient adaptation of multimedia content but its focus on the MPEG-21 content
limits its application [Leml04].
8
Internet Engineering Task Force (IETF), http ://www.ietf.org.
36
2.2.3 Content Adaptation Systems
Research towards content adaptation solutions has been the focus of several academic
communities, commercial companies, and standard organizations in order to achieve
ubiquitous access to multimedia content. In the following section, we describe the main
application systems and research prototypes which support content adaptation.
2.2.3.1
Commercial/Open Source Products
Since the importance of adaptation has been recognized by the Internet community, several
commercial products have emerged that try to incorporate the concepts and techniques of
adaptation in order to provide multimedia data access to heterogeneous terminals and users.
The most notable products are presented in the following section.
IBM Transcoding Proxy
The Mobile Networking Group at IBM has developed a transcoding proxy for multimedia
adaptation in order to be displayed in small devices like PDAs and mobile phones [IBM00].
The transcoding proxy was first developed as a research prototype in 1999 for adapting
Internet traffic to PDAs and mobile phones. Later it became a commercial product. For PDAs,
text is summarized, images are transcoded into grayscale equivalents and audio and video is
analyzed and presented as text whenever possible. For mobile phones, text is summarized into
single headline, images are omitted and audio and video are transcoded into speech describing
them. The transcoding proxy is commercially available and one can download for a 90-day
trial from the IBM site (http://www.alphaworks.ibm.com/). One of the problems of the
transcoding proxy is that it is the user that must select what transcoding he/she needs to be
carried on the content by configuring the proxy. This limits the dynamicity of the system as
parameters such as network conditions can vary continuously. Moreover the transcoding
options are limited, for example for html text, there is no language translation, and for images,
there is not format conversion.
Content Adaptation
37
WebSphere Transcoding Publisher
The experience gained from the transcoding proxy developed for the Internet access helped
IBM to develop a product called WebSphere Transcoding Publisher [Rodr01]. The product
was first released in 2000 and went through two additional releases before becoming a
technology embedded in WebSphere Portal and the webSphere EveryPlace Server. It is a
server-based software that dynamically translates Web content and applications into multiple
markup languages and optimizes it for delivery to mobile devices based on user preferences
and device capabilities. Content adaptation for many devices and languages eliminates the
need to maintain multiple versions of a web site. The product can convert images to links to
retrieve images, convert tables to lists, and remove comments and features not supported by
the device. The product comes with a wide variety of standard transcoders including:
- HTML to WML, cHTML, HDML, PalmOS HTML, and VoiceXML
- XML to wide variety of formats through XSLT style sheets
- JPEG/GIF/WBMP image transcoding and rescaling
- Natural language translation
One important feature of WebSphere transcoding is that it provides a pluggable infrastructure
to add transcodings for new data types. However, its plug-and-play architecture is more
tightly-coupled. Specifically, to add a new transcoding capability, a developer has to use the
developer’s toolkit to develop or re-develop a program so as to make the program pluggable
into the system. This limits the interoperability of the system. Moreover, since the transcoidng
is done at the content server it has limited scalability.
Web Clipping
Palm Computing introduced “Web clipping” in 1998 as an HTML transformation process for
its products Palm VII [3Com]. The goal of Web Clipping is to minimize both display
requirements (to fit on the Palm’s screen), and bandwidth usage by optimizing page layouts,
graphics and contents. During an HTML access, the request is redirected to the central server
of the company, which acts as proxy for the PDA. The proxy retrieves the web page, discards
the redundant objects and transforms the content to fit in the PDAs screen. It then starts
transmitting only the objects that will immediately be displayed, while it holds the rest of
them for later transmission upon request. Thus, the power consumption is reduced to
38
minimum. Web clipping which was first introduced to adapt Internet content for Palm VII
device which was basically text-based display later extended to other palm devices.
On 2001 Maporama9, the European leader in online cartography and geocentric information,
and Palm, announce their international content partnership for the distribution of Maporama’s
cartographic and geocentric content onto all new Palm products via Palm’s Web clipping
technology. Geographic content is one of the commodities Internet users seek the most
frequently. Since the customers are, by default, mobile users, this service becomes a real need
for them. Palm and Maporama have teamed up to provide users value-added geocentric
services compatible with their hardware. Maporama’s three most essential functionalities maps (both country and street level detail), itineraries (both door-to-door and city-to-city), and
pan-European directories - are accessible from all Palm’s new handheld computers. Locationbased weather forecasts are also available.
Web Clipping does not have transcoding tools so it does not solve the format and modality
mismatches between content and devices. Moreover, it is specifically designed for Palm
handheld devices.
Morphis Wireless Content Transcoder
Morphis10 is an open source java based Web content transcoding, transformation, translation
and aggregation framework. The first version of the product was released in 2001. It was
primarily built for retrieving and translating XML based documents, but it is also capable of
performing any type of translation: binary, plain text, or text markup. Therefore, Morphis is
able to convert XML into HTML or WML, while also being able to scale, crop and convert
images on the fly. Central to the Morphis solution is WAX (Wireless Abstract XML), a set of
tools and an abstract markup language used to author content for wireless applications.
Content is written once in WAX language, and then translated to various wireless languages
via XSL style sheets. The limitations of Morphis are (1) it is server-based so it has limited
scalability (2) WAX is used for content authoring hence it does not support other content
formats (3) the transcoding tools are limited to images.
9
Maporama SA, Paris, develops and markets Maporama.com, an Internet portal dedicated to providing detailed
geographic information (for example town plans) across the world from any Internet enabled device such as a
PC, Palm-Pilot, WAP or WebTV (www.maporama.com).
10
Morphis: http://www.morphis.org/
Content Adaptation
39
Oracle Application Server Wireless
Oracle Application Server Wireless [Oracle03] is the mobile component of the Oracle
Application Server. It includes a multi-channel server which abstracts the underlying
networks, protocols, devices and gateways to one protocol and one language, namely HTTP
and XML. The server enables applications to be accessed through multiple delivery methods
and devices such as 2-way pagers, devices with SMS, MMS, voice, or WAP capabilities, and
handheld computers. The server automatically translates applications written in Oracle
Wireless XML, XHTML Mobile Profile, or XHTML+XForms for any device and network.
For example, an XHTML+XForms application passed through the Multi-Channel Server will
be translated to VoiceXML if a phone is accessing the application through a voice interface or
to WML if a WAP phone issues the request. The system also supports device-specific
adaptation of images, ring tones, voice grammars, audio and video streams. For example,
images can be dynamically adapted to suit the image format, color depth, size and aspect ratio
requested by the device. Ring tone adaptation allows for conversation of ring tone to formats
supported by the most popular phones.
The major limitations of the Oracle Wireless server are: (1) the server is placed on the content
so it has limited scalability; (2) it does not support content formats other than XML; (3) it is
developed for applications developed in oracle.
Sun Java System Portal Server Mobile Access
The Mobile Access pack is an extension to the Sun Java System Portal Server11. It enables
wireless access to the portal and extends the core portal services (session handling,
authentication, logging, e-mail, calendar, address book, discussions etc.) to mobile devices.
Java System Portal Server Mobile Access can dynamically render and deliver personalized
and aggregated content to users with wireless mobile devices, such as cell phones, PDAs, and
smart phones, over any wireless network connection. The server dynamically renders content
into different markups such as XML, HTML or WML and tailors it to make the best use of
the requesting device’s features. The Mobile Access pack uses a templating system based on
JSP technology for accessing applications or XSL style sheets for accessing XML content.
11
http://wwws.sun.com/software/products/portal_ma/index.html
40
XSL templates are used to transform the XML data into any markup language including
XHTML, WML and VoiceXML [Sun].
The major limitations of the Sun Mobile Access are: (1) it has no transcoding tools and apply
only structural transformation such as XML to WML; (2) it is server-based approach so it has
limited scalability; (3) it has limited application since it supports only XML content format
Summary on commercial/open products
To conclude, the commercial application systems that have been presented above show the
importance of content adaptation to deliver content in heterogeneous and wireless
environments. The systems are developed mostly for web contents and they are specific, for
example, the Oracle Wireless Server is developed for Oracle applications. The systems are
more interested in structural transformation such XML to WML. These works did not deal
with the problem of content adaptation from the perspective of service that is providing
content adaptation as a service accessible on Internet. Instead, they looked into the
deployment of content adaptation technology at an origin server or a transcoding proxy hence
they have limited scalability. Moreover, they do not consider adaptation paths where we can
have several adaptation chains to be applied on the content and no path optimization is used.
2.2.3.2
Research Prototypes
The emergence of pervasive computing as a result of advance in computing and
communication technologies opens several new research projects in different academic and
research institutions. The focus of these projects is mainly centered on developing adaptation
system platforms for delivering adapted content in heterogeneous access environment. There
are many platforms proposed by the research community. Here, we present the most
important platforms for purpose of comparison.
Content Adaptation
41
TranSend
UC Berkeley’s TranSend Web accelerator proxy [Fox98b] was one of the earliest projects to
explore adaptation proxies aggressively. TranSend intercepts HTTP requests from standard
web clients and applies datatype-specific lossy compression when possible; for example,
images can be scaled down or down sampled in the frequency domain, long HTML pages can
be broken up into series of short pages, …,etc. TranSend supports a wireless vertical handoff
mechanism. When a client equipped switches between wireless networks, the client-side
vertical handoff software (which is completely independent of TranSend) generates a
notification packet containing some essential characteristics (e.g., estimated expected
throughput) of the new network. This packet would be sent to a special UDP port on
TranSend where the notification would be processed and stored in a per-client profile.
TranSend would then process future requests from that client in accordance with the new
network type; for example, very aggressive image downsampling will be performed for
clients connecting with an expected throughput of 15-25 Kb/s, whereas compression will be
much less aggressive (and in some cases disabled) for WaveLAN clients connecting at about
1 Mb/s. Because HTTP is a “stackable” protocol (i.e. it is possible to have several HTTP
“hops” in a request chain), TranSend-based adaptations are naturally composable, allowing a
multilevel system with some “baseline” compression performed far upstream, and additional
compression performed near the clients. Later on, TranSend has been commercialized (mostly
to adapt Web content for display on personal digital assistants (PDAs), for example, for 3Com
PalmPilot) by Proxynet (http://www.proxynet.com).
TranSend adapts contents on-the-fly in a proxy between the client and the Web Server.
Composability support of TranSend enables to apply several adaptations on the content.
Major drawbacks of the system include:
•
The design assumes a single point of adaptation, at the proxy server, this limits its
scalability.
•
The adaptation is primarily limited to image compression and reduction of image size
and color space.
•
It does not provide a mechanism to adapt content to user preferences.
42
Odyssey
Odyssey is a system built at CMU to support challenging network applications by adjusting
the quality of accessed data to match the available resources [Noble00]. Odyssey particularly
focuses on resource management for multiple applications running on the same machine.
Odyssey was designed primarily to run in wireless environments characterized by changing
and frequently limited bandwidth, but the model is sufficiently general to handle many other
kinds of challenging resource management issues, such as battery power or cache space. The
goal of the system is to provide all applications on the portable machine with the best quality
of service consistent with available resources and the needs of other applications.
Odyssey is an application-aware approach to adaptation intended primarily to assist
client/server interactions. The Odyssey system consists of a viceroy, an operating system
entity in charge of managing the limited resources for multiple processes; a set of data typespecific wardens that handle the intercommunications between clients and servers; and
applications that negotiate with Odyssey to receive the best level of service available.
Applications request the resources they need from Odyssey, specifying a window of tolerance
required to operate in a desired manner. If resources within that window are currently
available, the request is granted and the client application is connected to its server through
the appropriate warden for the data type to be transmitted. Wardens can handle issues like
caching or pre-fetching in manners specific to their data type to make the best use of the
available resources.
If resources within the requested window are not available, then the application is notified and
can request a lower window of tolerance and corresponding level of service. As conditions
change and previously satisfied requests can no longer be met (or, more happily, conditions
improve dramatically), the viceroy uses upcalls registered by the applications to notify them
that they must operate in a different window of tolerance, possibly causing them to alter their
behavior.
Odyssey became part of the Aura project [Garl02] that was developed by the research
community at Carnegie Mellon University. The drawbacks of the Odyssey system are:
•
The Odyssey system assumes that content in different versions is always pre-created.
But it is difficult to generate all possible versions that match different contexts.
Content Adaptation
•
43
It is specifically designed for adaptation to network characteristics but not to device
capabilities and user preferences.
•
The need to modify the operating system kernel and its specificity to the Odyssey file
system limits its application.
•
It requires the addition of a module on the client.
Conductor
The UCLA Conductor system allows deployment of cooperating adaptive agents at specially
enabled nodes throughout a network [Yarvis01]. Conductor is an application-transparent
adaptation mechanism. Applications can benefit from Conductor without being recoded or
explicitly requesting its services. Instead, the underlying system is configured to indicate what
kinds of data flows Conductor is capable of assisting and the Conductor system automatically
traps and adapts those data flows. Conductor also handles issues of composing adaptations in
support of a single flow at multiple nodes. Conductor determines the characteristics of the
data path from source to destination and estimates if the path will meet the needs of the
applications using it. If not, Conductor automatically deploys adapters at one or several of the
available nodes along the path to adapt the data flow to network conditions, allowing better
application-visible network behavior. Conductor plans the cooperative behavior of the agents
and handles problems of transient or long-term failure of particular adapter nodes.
Conductor is designed to handle both lossy and lossless adaptations. Combining lossy
adaptations and reliability is especially challenging, since a lossy adapter may drop part of the
data or may transform several data packets into fewer packets. If an adapter or its node fails,
some of the adapted packets could be delivered while others were not. Without the lossy
adapter’s state to determine which original packets were dropped or coalesced, the system
may find it difficult to resume transmission without either duplicating already received
information or failing to deliver required information. Unaware applications are generally
unprepared for either problem, so Conductor must hide these problems from such
applications. Conductor attaches numbers to pieces of semantic content that do not vary when
adapted. For example, if every other packet is dropped, the undropped packets are
renumbered to include the dropped packets. The system is thus able to determine which
information has and has not been delivered despite failures.
44
Conductor permits the introduction of arbitrary adaptors in the data path between applications
and end services. This solves the incompatibility issues between heterogeneous networks.
However, the system has three major drawbacks: (1) it does not handle device limitation and
user preferences (2) the lack of application input limits its flexibility and (3) a significant
change to existing infrastructure is required for its deployment that is the network nodes must
be
conductor-enabled.
The
software
package
is
available
on
the
net
(http://lasr.cs.ucla.edu/Conductor/).
Digestor
Digestor [Bick99] is a software system that automatically re-authors documents from the Web
to display them appropriately on small screen devices such as PDAs and mobile phones. It
can be run on HTTP proxies so that online documents can be adapted to different client
devices. Re-authoring is the re-structuring of the documents such that they can be presented
on the resource-constrained devices. Techniques for re-authoring includes outlining (e.g.
section headers made into hyperlinks with the contents elided from the document), first
sentence elision (the first sentence of each block made into a hyperlink), and image reduction
and elision (images are scaled down and the reduced images have hyperlinks back to their
originals).
Digestor makes use of an automatic re-authoring system to select the best combination of the
re-authoring techniques for the document/display-size pair. The system uses heuristics to
understand the web documents to enable re-authoring decisions. However, the potentials of
such heuristics are limited [Buch03]. In order to overcome those limitations, other work
considers generic XMLbased representations of documents that are augmented by meta
information to allow smarter adaptation (e.g. [Goeb01]).
The drawbacks associated with the Digestor system are: (1) the concept of re-authoring
requires semantic understanding of the structure of the document which is not always evident
especially to adapt content on-the-fly; (2) the system focuses on different display size rather
than device capabilities or network condition; (3) it applies only structural transformation and
no support for media transcoding; (4) Digestor users are required to specify the size of display
for their device and the size of their default browser font to estimate the screen area
requirements of the text blocks.
Content Adaptation
45
Mowser
Mowser (Mobile Browser) [Joshi00, Bhra98] is a transcoding proxy that modifies the HTTP
stream and changes its content dynamically. It allows users to set their preferences based on
their needs and the capabilities of the devices, such as video resolution, sound and color
capabilities, maximum sizes for each type of files, etc. These preferences are stored in an
Apache server. When the proxy receives a request from the client, it looks up the preferences
that are stored with the IP address of the client, and modifies the HTTP request according to
the preferences before sending to the server. Mowser introduces what is known as content
negotiation. Different variants (or fidelities) of the web contents are stored in the server. The
proxy asks the server for one of the fidelities that is available, and which satisfies the
requirements of the client. If the one that is returned by the server is still too large for the
client, transcoding is applied. The kind of transcoding that is applied depends on the file type.
For example, the size or color depth of an image is reduced, and representative frames of a
video file are selected. The proxy can also parse the HTML stream to remove active contents
and tags that the clients cannot handle. Any contents that cannot be processed by the client are
not returned. Different from other transcoding proxies, Mowser also considers the user
preferences apart from the capabilities of the client devices. The authors claimed that the
proxy is available on the net (http://nirvana.cecs.missouri.edu:8001) but it was not working
during verification.
Since users may have dynamic IP addresses, the use of IP address to store user preferences is
not a good solution. Another limitation of the system is creating content variants require
additional storage.
ADMITS (Adaptation in Distributed Multimedia IT Systems)
ADMITS [Bosz03] [Bosz02] project was initiated at the University of Klagenfurt, Austria,
and is still under development. It realizes a distributed multimedia architecture, which adapts
its behavior to changing resource availabilities. The goal in the ADMITS project is to access
multimedia resources, from different types of terminals and/or over diverse networks and
make this access interoperable and transparent. This distributed multimedia system comprises
several components, such as media servers, meta-databases, adaptive proxy cache, and
adapative routers. The proxy cache stores full videos with their respective metadata
46
descriptions in the form of MPEG-7 Variations Descriptors, and it reduces their quality (and
thus their size), before fully deleting them. The essential innovation is that the quality
reduction process is driven by metadata in a standardized format (MPEG-7), which is readily
provided in ADMITS by the meta-database.
The routers can cope up with dynamically varying network load and multicast media stream
along heterogeneouse links clients with different capabilities by applying some adaptation
operations such as dropping of video frames, of certain enhancement layers, or of entire
MPEG-4 Elementary Streams.
The ADMITS project builds an experimental distributed multimedia system for investigations
into adaptation. The MPEG-7 technology is widely used and provides tools, such as the
variation description scheme, which enable the components to communicate adaptation
metadata in a standardized form. The major distinguishing feature of the ADMITS approach
is that (MPEG-conforming) meta-data are used to steer the adaptation processes in
components on the delivery path to the client (e.g., proxies, routers).
Major limitations of ADMITS are (1) the need of processing and storing content variants at
the server (2) the routers must be MPEG-7 compatible to apply some adaptations on the data.
Summary of Content Adaptation Systems
Table 2.2 summarizes the adaptation approach or architecture, transcoding type, and
adaptation target (network, device and user preferences) of the content adaptation systems
discussed above both commercial/open and research.
Content Adaptation
Adaptation
system
IBM
Transcoding
Oracle
Adaptation
approach
Proxy-based
server-based
Sun
Web Clipping
WebSphere
47
Transcoding type
Network
conditions
Yes
Device
capabilities
Yes
User
preferences
Yes
Remark
Images, ring
tones
Yes
Yes
Yes
Server-based
Proxy-based
Server-based
No
No
JPEG/GIF/WBM
P images
Not known
Yes
Yes
Yes
Yes
Yes
Yes
No
Yes
Uses
proprietary
XML for
authoring
XML to any
MobiXstar
Proxy-based
Yes
Yes
Not known
Morphis
Server-based
Image,
animation, audio
and video
Images
No
Yes
No
TranSend
Proxy-based
Yes
No
No
Odyssey
Middleware
Image and text
compressors
Selection
Yes
No
No
Conductor
Data filtration
and compression
Image reduction
Yes
No
No
Digestor
Multiple
network nodes
Proxy-based
No
Yes
No
MOWSER
Proxy-based
Image and video
No
Yes
Yes
ADMITS
Multiple
network nodes
Video
Yes
Yes
Yes
Image and text
XML to wide
variety
Proprietary
XML used
for authoring
HTML
Uses preadapted
version
considers
only display
size
content
variants
stored at the
server
content
variants
stored at the
server
Table 2.2: Summary of Content Adaptation Systems Features
There are at least four general requirements that a good content adaptation framework should
meet to be effective in a pervasive computing environment. Characteristics of these
requirements are described below:
Extensibility: one major difficulty with content adaptation is that all the adaptation tools can
not be exhaustively determined beforehand. It is difficult to develop a system that can answer
all adaptation demands which are multidimensional. The framework should therefore be
extensible to handle new data object types and content adaptation tools, including those not
yet known. To that end it should comprise of independent, modular and pluggable tools that
48
can be easily added, replaced or modified. The tools should perform a single function to
enable reusability and enforce robustness.
Flexibility: pervasive computing is the ability to extending applications and services normally
conducted on office computers to handheld and wireless devices enabling anyway, anytime
with any device access to information systems. This means we can create pervasive systems
from existing applications. Thus, the adaptation framework must be flexible enough to be
easily integrated to existing applications without the need of major alteration on the
applications.
Interoperability: today, application developers use a wide range of programming models,
languages and development environments, and this heterogeneity will continue in the future,
particularly as the range of use for computing technology expands. Thus, the framework must
support diverse types of software components; one of them is the adaptation tools. The
framework must integrate adaptation tools residing in fundamentally different environments,
into compositions that can successfully interact and cooperate to achieve an adaptation need.
This will require interoperability at the component level in addition to interoperability that
overcomes heterogeneity of the environment and components.
Scalability: one of the features of pervasive computing is the increasing availability of
devices and software. Thus, the framework, the interactions between components, and the
software services provided in the pervasive computing environment must all be scalable. Most
importantly, the adaptation framework should perform incremental scalability with an
expanding user base.
Table 2.3 shows a comparison of each of the architecture with respect to the requirements for
good adaptation framework that are presented above.
Summary
Properity
49
Extensibility
Flexibility
Interoperability
Scalability
Server
+
-
+
+
Client
-
-
-
-
Proxy
++
+
++
++
MARCH
++
+
-
+
Ninja
++
+
+
++
DANAE
+
+
-
++
PUMAS
+
+
+
+
Architecture
Table 2.3: Architectural Characteristics of Adaptation Approaches
To summarize, there have been many efforts done to develop different approaches and
techniques of multimedia content adaptation for pervasive environments. A number of
commercial products and research prototypes have been developed that provides different
adaptation techniques to deliver multimedia data in a heterogeneous environment. These
products have contributed a lot to propose different architectures and techniques of adaptation
strategies. However, none of them have addressed well the interoperability, extensibility,
flexibility, and scalability issues of content adaptation which are important requirements in
pervasive computing environment. Moreover, most of the systems are designed only for
narrow needs and do not provide a general adaptation solution. In this thesis, we try to address
these issues.
2.3 Summary
In this chapter we presented the most important and relevant works to the subject of this
thesis. We have discussed major adaptation techniques and their classification. We have also
discussed the different adaptation approaches that have been proposed in the literature their
advantages and drawbacks. Commercial and research adaptation systems and their features
were also presented. Since each of these systems fall in one of the approaches, comparison of
50
the different approaches with respect to the requirements of a general adaptation framework
for pervasive computing was done.
Chapter
3
Service-Oriented
Architecture & Web
Services
One of the aims of our research was to develop a generic distributed content adaptation
framework. To achieve this we chose a Service-Oriented Architecture (SOA) based on web
services. In addition to providing a generic framework, the SOA enables to realize different
adaptation scenarios by using externally developed adaptation tools (implemented with web
service technologies). The purpose of this chapter is to present briefly the SOA and its
associated technologies. Section 3.1 presents the major concepts of the SOA. Some features
of the SOA are presented in Section 3.2. Web Service standards and related technologies are
presented in Section 3.3. Discussion on web service composition is presented in Section 3.4
and finally the chapter ends with a short summary in Section 3.5.
3.1 Introduction
In traditional approach to software design and packaging, all the components of a software
system are packaged together, even though most of them are used rarely. Over the last four
decades, software architectures have attempted to deal with increasing levels of software
complexity. But the level of complexity continues to increase, and traditional architectures
seem to be reaching the limit of their ability to deal with the problem. We have gone through
multiple computing architectures designed to allow fully distributed processing, programming
languages designed to run on any platform, greatly reducing implementation schedules, and a
myriad of connectivity products designed to allow better and faster integration of applications.
However, the complete solution continues to elude us. A service-based computing paradigm
52
Chapter 3. Service-Oriented Architecture & Web Services
approach was introduced to handle the ever increasing complexity of modern software
systems.
The service-based paradigm is a more flexible approach in which software components are
loaded on demand, i.e., only when their services are actually requested by the application, and
removed when these services are not needed any more. Such components will be referred to
as service components, in order to distinguish them from the more traditional view of
components as independent entities in their own right [Szyp02].
A SOA [Ort05] is an information technology approach or a strategy in which applications
make use of (perhaps more accurately, rely on) services available in a network such as the
World Wide Web. As shown in Figure 3.1, a SOA has three types of elements: service
provider, service user and service publisher. A service provider develops the services and
publishes them on a directory or registry. The service user also called customer or client
searches for services that provide the task he/she wants to achieve in the directory. The
service registry can also be coupled with a repository component that stores additional
information about each service such as business process information or service use costs.
A service provides a specific function, typically a business function, such as analyzing an
individual's credit history, processing a purchase order or image format conversion (in case of
adaptation services). In this thesis each service implements a particular adaptation tool, for
example, image resizing, language translation, text to speech conversion etc.
Multiple services can be used together in a coordinated way. The aggregated, or composite,
service can be used to satisfy a more complex user requirement. In fact, one way of looking at
a SOA is as an approach to connecting applications (exposed as services) so that they can
communicate with (and take advantage of) each other. In other words, a SOA is a way of
sharing functions (typically business functions) in a widespread and flexible way.
Benefits of Service-Based Development
53
Customers
Client
Service
Business
partners
Service
Directory
Service
Service
Communicate
Publish
Find
Figure 3.1: A service-Oriented Architecture (SOA) [Ort05]
Though the SOA is not a new concept, the emergence of web services made it important tool
to develop distributed application systems [Rao04]. A web service is a service that
communicates with clients through a set of standard protocols and technologies. These web
services standards are implemented in platforms and products from all the major software
vendors, making it possible for clients and services to communicate in a consistent way across
a wide spectrum of platforms and operating environments. This universality has made web
services the most prevalent approach to implementing a SOA.
3.2
Benefits of Service-Based Development
Complexity is a fact of life in information technology (IT). Dealing with the complexity while
building new applications, replacing existing applications, and keeping up with all the
maintenance and enhancement requests represents a major challenge [Lomo04].
Service-based development provides the following benefits which are more appropriate to our
work:
54
Interoperability: the objective of SOA is to enable clients and services to communicate and
understand each other no matter what platform they run on. This objective can be met only if
clients and services have a standard way of communicating with each other - a way that is
consistent across platforms, systems, and languages. In fact, Web services provide exactly
that. Web services comprise a maturing set of protocols and technologies that are widely
accepted and used, and that are platform, system, and language independent.
Scalability: because services in a SOA are loosely coupled, applications that use these
services tend to scale easily, certainly more easily than applications in a more tightly-coupled
environment. That's because there are few dependencies between the requesting application
and the services it uses. On the other hand, in a tightly-coupled environment it is more
complex to make the services accessible by many applications.
Flexibility: loosely-coupled services are typically more flexible than more tightly-coupled
applications. In a tightly-coupled architecture, the different components of an application are
tightly bound to each other, sharing semantics, libraries, and often sharing state. This makes it
difficult to evolve the application to keep up with changing technology or business
requirements. The loosely-coupled nature of services in SOA allows applications to be
flexible, and easy to evolve with changing requirements.
3.3
Web Services: Standards and Related Technologies
The term “Web services” has been used very often nowadays. According to IBM [Gard01],
“Web Services are self-contained, modular applications, accessible via the Web through open
standard languages, which provide a set of functionalities to businesses or individuals”. This
definition places the emphasis on two points. The first point is that a Web service is seen as an
application accessible to other applications over the Web. Secondly, Web services are open,
which means that services have published interfaces that can be invoked by message passing
standards. This definition is very simple, but not precise enough. For instance, components in
Common Object Request Broker Architecture (CORBA) are also modular and self-contained
but they are not web services. A step further in refining the definition of Web service is the
one provided by the World Wide Web consortium (W3C) [Aust02]:
55
“A Web service is a software system identified by a URI, whose public interfaces
and bindings are defined and described using XML. Its definition can be discovered
by other software systems. These systems may then interact with the Web service in
a manner prescribed by its definition, using XML-based messages conveyed by
Internet protocols.”
The W3C definition stresses that web services should be capable of being defined, described,
and discovered [Alonso04]. It should also be emphasized that web services do not merely
provide static information, but allow one to affect some action or change in the world. For
example, in the case of content adaptation, services such as image color reduction, image
format conversion, video frame dropping etc. change the state of the media data.
What makes the web services attractive is the ability to integrate the web services developed
by different organizations together to fulfill the user's requirement. Such integration is based
on the common standards of web service interfaces, regardless of the languages that are used
to implement the web services, and the platforms where the web services are executed. In
general, the web services have the following features that make them better in integration
inside the heterogeneous environments:
Loosely coupled: in software development, coupling typically refers to the degree to which
software components/modules depend upon each other. Comparing with the tightly coupled
components (such as the Distributed Component Object Model (DCOM) or the CORBA)
[OMG02], the web services are autonomous and can operate independently from one another.
The loosely coupled feature enables Web services to locate and communicate with each other
dynamically at runtime.
Universal accessibility: the web services can be defined, described and discovered through
the Web that enables an easy accessibility. Not only the web services users can locate
appropriate services, but services can describe and advertise themselves so that they are
possible to bind and interact with each other.
Standard languages: although the cores of web services may be implemented by different
programming languages, the interfaces of web services are described by uniform standard
XML languages.
56
3.3.1 Web Service Languages
A web service provider offers services on the Web. He may choose to register his service at
an online registry of a service broker. The registry also provides standardized description
facilities, e.g. taxonomies that allow the description of 1) the functionality of a service, 2) the
information about the service provider, and 3) the way to access and interact with the service.
The corresponding information about a particular service is registered by the provider at a
broker.
The service requester searches for a service at the registry. He finds one by browsing or
querying the registry. Then he uses the service description to create a binding for his
application. The last step is to invoke or interact with the web service using standard
communication language. Standards such as XML, UDDI, WSDL and SOAP support the
above procedure.
UDDI [Bell02] provides a registry where service providers can register and publish their
services. The registry consists of three parts: white pages, yellow pages and green pages.
Contact information, human readable information and related information can be registered in
the white pages. Keywords that characterize the service are registered in the yellow pages.
Service rules and descriptions for application invocations are registered in the green pages
(technical). UDDI does not support semantic descriptions of services and no content language
for advertising is provided.
WSDL [Chinnici03] is a proposed W3C standard for describing network services as a set of
end points operating on messages containing either document-oriented or procedure-oriented
information. The operations and messages are described abstractly, and then bound to a
concrete network protocol and message format to define an endpoint. Related concrete
endpoints are combined into abstract endpoints (services). SOAP is typically used to deploy
the operations.
SOAP [Box00] is a proposed W3C standard for exchanging information in a decentralized
and distributed environment. SOAP consists of three parts: envelope, encoding rules and
Remote Procedure Call (RPC) representation. The envelope sets up a framework for what the
57
message contains and responsibility. The encoding rules provide a serialization mechanism
for exchanging instances of application specific data types. Finally, RPC makes it possible to
encapsulate and represent remote procedure calls and responses.
Both UDDI and WSDL do not include semantic information about the services. Currently
UDDI provides key-word based search. Keywords are not enough to discover the services as
service providers may use different words though they provide the same functionalities.
Moreover, UDDI does not enforce a relationship between the service names and their
functionalities. While WSDL describes the exchange data, it provides no semantic
information such as the service use, input output characteristics, the effect/consequence,
service cost and execution time, etc. However, semantic information is important for
automated adaptation service discovery and composition. Our adaptation services
composition bases on service compatibility which depends on the input, output, and quality
criteria such as service cost and execution time. Hence, UDDI and WSDL fail to provide
enough information about a web service (or an adaptation service in our case) in order to be
discovered and used automatically in real applications. In the following section we present
some of the tools that facilitate semantic service description.
3.3.2 Semantic Web Service Description
In IBM’s proposition, web services have simple rather than rich descriptions, the environment
is closed rather than open, the service requester is human rather than machine and data
exchange are syntactic rather than semantic [Solla02]. However, facilities to put machineunderstandable web service on the web are becoming a high priority for many communities.
To implement reliable and large-scale interoperation of web services, the services should be
computer interpretable. To have computer programs or agents implementing reliable and
large-scale interoperation of web services, the services should be computer interpretable. This
implies to have semantic web of services whose properties, capabilities, interfaces, and effects
are encoded in an unambiguous, machine-understandable form. The Semantic Web language
is a vision: the idea of having semantic service description that can be used by machines for
automation, integration and reuse across various applications. Semantic service description
should enable automatic service discovery, execution, composition and interoperation.
58
Automatic discovery involves automatically locating web services that provide a particular
service and that adhere to requested properties. Currently, a human must perform the
discovery task, first using a search engine to find a service and then either reading the web
page associated with that service or executing the service to see whether it adheres to the
requested properties. With semantic markup of services, we can specify the information
necessary for Web service discovery as computer-interpretable semantic markup at the service
Web sites, and a service registry or (ontology-enhanced) search engine can automatically
locate appropriate services.
Automatic execution involves a computer program or agent automatically executing an
identified web service. To execute a particular service on today’s web a user generally must
go to the web site offering that service, fill out a form, and click a button to execute the
service. Alternately, the user might send an http request directly to the service URL with the
appropriate parameters encoded. Either case requires a human to understand what information
is required to execute the service and to interpret the information the service returns. Semantic
markup of web services provides a declarative, computer-interpretable API for executing
services. The markup tells the agent what input is necessary, what information will be
returned, and how to execute—and potentially interact with—the service automatically.
Automatic composition and interoperation involves the automatic selection, composition,
and interoperation of appropriate web services to perform some task, given a high-level
description of the task’s objective. Currently, if some task requires a composition of web
services that must interoperate, then the user must select the web services, manually specify
the composition, ensure that any software for interoperation is custom-created, and provide
the input at choice points. With semantic markup of web services, the information necessary
to select, compose, and respond to services is encoded at the service web sites. We can write
software to manipulate this markup, together with a specification of the task’s objectives, to
achieve the task automatically.
Nowadays, there are many services accessible on the Internet. It is difficult to identify a
particular service in order to execute a task. This is due to the problem that there is no
common structure or framework in using terms or names to describe services. For example, to
describe a service that provides image resizing function, we may find different names. In
order to facilitate service composition and interoperation it is imperative to have a common
59
description framework. Ontologies have been introduced to solve these problems. As stated in
[Usch96], ontology is the term used to refer to the shared understanding of some domain of
interest which may be used as a unifying framework. It is also defined as a formal explicit
description of concepts in a domain of discourse (classes sometimes called concepts),
properties of each concept describing various features and attributes of the concept (slots
sometimes called roles or properties), and restrictions on slots (facets sometimes called role
restrictions) [Grub95]. Today there are different ontology specification languages with
varying degree of expressiveness, like RDF (Resource Description Framework), DAML
(DARPA Agent Markup Language), DAML+OIL and OWL (Web Ontology Language).
Here, we present OWL and OWL-S because these are the tools that we have used to develop
our multimedia content adaptation services description.
OWL (Web Ontology Language)
OWL is a W3C general purpose recommendation language for defining web ontologies
[Smith04]. This language incorporates the lessons learned from the design and application of
DAML + OIL. OWL, like DAML + OIL, uses description logic to provide an unambiguous
meaning of terms defined using it. This can be used by applications that require
interoperability. It is also compatible with XML and RDF.
OWL-S (Web Ontology Language for Services)
OWL-S (formerly DAML-S) [Ankol01] is an ontology that provides constructs for describing
web services. With OWL-S the properties of services and their capabilities are less ambiguous
and described in computer-interpretable format. It can be regarded as a semantic-based effort
for service description, service publication and service flow [Solla02]. OWL-S descriptions
enable improved matching of services [Paol02]. The top level of the ontology has a service
construct,
related
to
three
basic
concepts:
ServiceProfile,
ServiceModel
and
ServiceGrounding. The ServiceProfile describes what the service does (what is provided), the
ServiceModel describes how it works (what happens) and the ServiceGrounding describes
how to access it (how to use). In the current version, the ServiceGrounding is defined as the
mapping between the Service- Model and the service's interface description, WSDL. OWL-S
is one of the approaches developed to semantically describe Web services [Cabral]. The other
main similar approaches are IRS (Internet Reasoning Service) [Motta03] and WSMF (Web
60
Service Modeling Framework) [Fensel02]. IRS provides a strong user and application
integration support but it does not have semantic description of composed services. WSMF is
an effort towards developing a comprehensive conceptual architecture, which covers
requirements of e-commerce. Compared to these two, OWL-S has a richer service ontology
which builds on the semantic web stack standards. In addition, the service ontology of OWLS is public.
3.4
Web Service Composition
In service-oriented computing (SOC), developers use services as fundamental elements in
their application-development processes. Services are platform- and network-independent
operations that clients or other services invoke. In SOC, we can identify two distinguished
abstraction levels of proposed services. On one hand, basic services are elementary
functionalities, usually provided by the third parties. On the other hand, composite services
aggregate a set of functionalities into higher level applications, closer to the user’s actual
needs. Thus, service composition enables to fill the abstraction gap between the user’s need
and elementary service levels. Developers and users can solve complex problems by
combining available basic services and ordering them to best suit their problem requirements.
Service composition accelerates rapid application development, service reuse, and complex
service consummation [Milan04].
Web service composition in SOAs has been an active research area lately, as a result of the
increasing impact the web services deployment paradigm on today’s World Wide Web.
Despite of many efforts in developing different languages and standards for service discovery,
description and invocation over the past years, the Web service composition remains highly
complex and beyond the human capability to deal with the whole process manually. Jinghai et
al. [Rao04] have listed three main sources for Web service composition complexity.
•
The number of services available over the web increases dramatically during the
recent years, and one can expect to have a huge web service repository to be searched.
•
Web services can be created and updated on the fly, thus the composition system
needs to detect the updating at runtime and the decision should be made based on the
up to date information.
•
61
Web services can be developed by different organizations, which use different
semantic models to describe the services, however, a unique language does not exist to
define and evaluate the web services in an identical means. Many authors have argued
that web service composition will be the dominating paradigm on the web in the years
to come, thus changing its structure from a web of information and manuallyaccessible applications, to that of a web of automatic services [e.g. Casati01].
According to Nikola et al. [Milan04] a web service composition mechanism must satisfy
several requirements such as connectivity, nonfunctional quality-of-service properties,
correctness, and scalability. Every composition approach must guarantee connectivity. With
reliable connectivity, we can determine which services are composed and reason about the
input and output messages. Composition correctness requires verification of the composed
service’s properties, such as security or dependability. Finally, because complicated business
transactions are likely to involve multiple services in a complex invocation chain,
composition frameworks must scale with the number of composed services.
Building composite web services with an automated or semi-automated tool is critical to the
success of the web service applications. To that end, several composition methods have been
proposed recently. Most of them fall into one of the following two categories [Rao04]:
methods based on pre-defined workflow model and methods based on Artificial Intelligence
(AI) planning.
3.4.1 Workflow Technique
In a pre-defined workflow service composition technique, a user should specify the workflow
of the required composite service, including both nodes and the control flow and the data flow
between the nodes. The nodes are regarded as abstract services that contain search recipes.
The concrete services are selected and bound at runtime according to the search recipes. This
approach is widely adopted by members of the Information Systems community [Rao04].
The definition of a composite service includes a set of atomic services together with control
and data flow among the services. Similarly, a workflow has to specify the flow of work
items. In the workflow-based composition methods, we should distinguish the static and
62
dynamic workflow generation. The requestor builds an abstract process model before the
composition planning starts. The abstract process model includes a set of tasks and their data
dependency. Each task contains a query clause that is used to search the real atomic web
service to fulfill the task. In that case, only the selection and binding of atomic web service is
done automatically by the program.
Various web service flow specification standards have been developed in the industry such as
IBM’s Web Service Flow Language (WSFL) [Leym01], BEA Systems’ Web Services
Choreography Interface (WSCI) [Arkin02], and Business Process Execution Language for
Web Services (BPEL4WS, or BPEL for short) [Weer02]. Since the WSFL and WSCI were
incompatible, researchers have developed the BFEL which combines WSFL and WSCI with
Microsoft’s XLANG specification [That01].
The workflow composition systems provide manual and semi-automatic service composition
tools but they present several drawbacks:
•
First, the user is required to specify the workflow which may not be always practical
as in the case of content adaptation services. The type and number of adaptation
services required are determined during request execution.
•
Second, the discovery and selection of services is un-scalable as the number of
services increases.
•
Third, they require the user to have low-level knowledge, e.g. in the case of
BPWS4J12, the user is expected to set up a workflow at the XML level which is not
feasible for a large workflow. Although some systems provide a graphical drag and
drop interface to construct the workflow, it is not feasible for a large workflow.
•
Fourth, if the service is no longer available, the execution will fail.
Because of the above major drawbacks, the manual or semi-automatic workflow composition
is not appropriate for our work. In the following section we present automatic service
composition methods which are basically related to AI planning.
12
BPWS4J is java implementation of BPEL4WS.
63
3.4.2 AI planning
The second category includes methods related to AI plainning. The service composition
methods provide dynamic service composition where both the process model and the
selection of atomic services are done automatically. Our composition method proposed in this
work falls in this category. The basic assumption of these methods is that each web service is
an action which alters the state of the world as a result of its execution. Since the services
(actions) are software components, input and output parameters of a service act as
preconditions and effects in the planning context. If the user can specify inputs and outputs
required by the composite service then the service (as a composition of available services) is
generated automatically by AI planners without any knowledge of a predefined workflow.
Since the introduction of Semantic web, several Web service composition methods using AI
planning have been reported in the literature [Rao04, Medj03, McIl02]. In general, a planning
problem can be described as a four-tuple (S, So, G, A), where
- S is the set of all possible states of the world
- So Є S denotes the initial state of the world
- G Є S denotes the goal state of the world
- A is the set of actions that change one state to another state in the world
Given a representation of services as actions, we can exploit AI planning techniques for
service composition by treating service composition as a planning problem. Ideally, given a
user’s objective and a set of web services, a planner would find a collection of web services
requests that achieves the objective. This resemblance demonstrates the relevance of AI
planning to tackle the problem of Web service composition. However, DAML-S (also called
OWL-S in the most recent versions) is the only web language that announces the direct
connection with AI planning [Wu03]. The state change produced by the execution of the
service is specified through the precondition and effect properties of the ServiceProfile in
DAML-S. A precondition presents logical conditions that should be satisfied prior to the
service being requested. Effects are the result of the successful execution of a service. In the
following section, we present some of the web service composition method based on AI
planning. These methods use DAML-S as external Web service description language.
64
Golog
S. McIlraith and T. C. Son [McIl02] proposed a method to compose Web services by applying
logical inferencing techniques on pre-defined plan templates. The service capabilities are
annotated in DAML-S/RDF and then manually translated into Prolog. Now, given a goal
description, the logic programming language of Golog [Leve97] (which is implemented over
Prolog) is used to instantiate the appropriate plan for composing the Web services. Golog is
based on the situation calculus [Gross00] (a logical language for reasoning about action and
change) and it supports the specification and execution of complex actions in dynamical
systems.
PDDL
Jinghai Rao [Rao04] noted that the strong interest to Web service composition from AI
planning community comes from the similarity between DAML-S and Planning Domain
Definition Language (PDDL) representations. Since its release in 1998 by McDermott
[Dermott, 2000], PDDL has become a community standard for the representation and
exchange of planning domain models. Moreover, since DAML-S has been strongly influenced
by PDDL language, mapping from one representation to another is straightforward (as long as
only declarative information is considered). DAML-S descriptions could be translated to
PDDL format and different planners could be exploited to plan service composition.
Rule-based planning
B. Medjahed et. al. [Medj03] presents a technique to generate composite services from highlevel declarative description. The method uses composability rules to determine whether two
services are composable. B. Medjahed et. al. [B. Medjahed et. al.2003] identifies two
composability rules: syntactic and semantic. Syntactic rules include the rules for operation
modes and the rules for binding protocols of interacting services. Semantic rules include (1)
message composability, defines that two Web services are composable only if the output
message of one service is compatible with the input message of another service (2) operation
semantic composability, defines the compatibility between the domain, categories and
purpose of two services (3) qualitative composability, defines the requester’s preferences
regarding the quality of operations for the composite service, and (4) composition soundness
65
considers whether a composition of services is reasonable. To this end, the authors introduce
the notion of composition templates that define the dependency between the different kinds of
services. A service composition is considered to be sound if its template is the subset of a
stored template.
The composition approach consists of four phases called specification, matchmaking,
selection and generation. For the specification phase, Medjahed et. al. introduces a high level
language – CSSL (Composite Service Specification Language) – which is a semantic
description language for web service compositions. CSSL extends WSDL and represents the
ontology-based graph. In the matchmaking phase, a special matchmaking algorithm,
integrating the composability rules described above is applied to generate composition plans
that meet the service requester’s specifications. QoS constraints are used in the selection
phase to select the most proper composition plan from the output.
The main contribution of this method is the composability rules, because they define the
possible Web service’s attributes that could be used in service composition. Those rules can
be used as a guideline for other Web service composition methods based on planning.
HTN
Hierarchical Task Network (HTN) [Sirin04a] planning is an AI planning methodology that
creates plan by task decomposition. This is a process in which the planning system
decomposes tasks into smaller and smaller subtasks, until primitive tasks are found that can be
performed directly. HTN includes methods that describe how a complex task can be
decomposed into subtask and operators describing how a primitive task can be achieved.
The appropriateness of HTN planning for the problem of Web service composition is
suggested by the many operational similarities between OWL-S and HTN planning:
- OWL-S atomic processes correspond to primitive actions
- OWL-S composite processes correspond to complex actions or methods
- Both OWL-S and HTN planning use hierarchical decomposition to cope with
complexity. Specifically OWL-S use control constructs to decompose composite
processes to sub processes, and HTN planning use methods to decompose complex
actions to simpler subtask.
66
The similarity between OWL-S and HTN planning leads to mapping of composite processes
to methods and atomic processes to operators. Once this mapping has been performed, given
an objective (goal) and initial state, a HTN planner will find a sequence of web services that
achieves this goal from the initial state. Because of these reasons, a HTN planning approach
was adopted to develop the adaptation service composition algorithm in this thesis.
3.5 Summary
The emergence of web services makes the SOA a standard solution for distributed computing
on the Internet. This chapter presented the SOA and its related technologies with particular
focus on web services. We presented some of the benefits of SOA for distributed computing
such as flexibility, scalability and interoperability. Web services standards and related
technologies such as UDDI, WSDL and SOAP were discussed. We also presented the use of
OWL for semantic web services description. Finally, we presented two major categories web
services composition techniques (work flow and AI) and their comparisons. The HTN which
is one of the AI planning techniques was adopted in this thesis. The similarity between the
HTN and the OWL-S facilitates the representation of the OWL-S in planning operator form
so that a planning algorithm can be used to compose the adaptation services.
Chapter
4
Content Metadata
Description
The purpose of this chapter is to present a brief overview of the most important initiatives for
content metadata standards related to multimedia content adaptations. We start with a formal
definition of the term metadata and its basic characteristics in Section (4.1.). Next we will
present the Dublin Core standard in Section (4.2), the SMPTE Meta-Dictionary in Section
(4.3), the MPEG-7 standard in Section (4.4), and other standards in Section (4.5). We will end
this chapter with a summary in Section (4.6).
4.1 Introduction
Content adaptation enables pervasive access to multimedia resources through the process of
tailoring content. This requires explicit and machine-processable knowledge about the content
and the usage environment (context). Solutions for metadata representation provide a
mechanism for representing and processing this knowledge. Two types of metadata should be
considered: content metadata (e.g. format, size, dimension etc.) and context metadata (e.g.
display size, supported contents, language preference, bandwidth capacity etc.). Content
adaptation can be done without the use of content metadata [Kazai03]. For example, it is
possible to transcode MPEG-2 video to MPEG-4 based on features such as scene changes and
closed captions. However, this requires automatic video analysis which is of course, time
consuming and very difficult to correctly identify meaningful segments. Thus, it is important
that the content metadata should be created before delivering the content and stored with the
content or separately.
68
Chapter 4. Content Metadata Description
Metadata is a term originating from the Greek word ‘meta’ denoting words such as ‘along
with, after, among, between’. Within information technology, the term metadata has been
variously described as 'structured data about data' [Dublin99], or 'information about data',
even 'data about data'. These are technological descriptions. A further definition has been
provided by Lorcan Dempsey [Demp97]: “metadata is data associated with objects which
relieve their potential users having to have full advanced knowledge of their existence or
characteristics”. A user might be a program or a person, and metadata may support a variety
of uses or operations.
The term metadata is used differently in different communities [NISO04]. Some use it to refer
to machine understandable information, while others use it only for records that describe
electronic resources. In the library environment, metadata is commonly used for any formal
scheme of resource description, applying to any type of object, digital or non-digital.
Traditional library cataloging is a form of metadata.
A metadata description characterizes the information or content by a set of attributes. The
attributes may not only describe the actual content in terms of raw data, meaning, and/or key
concepts, but also characterise the content in terms of its author, its quality, its production
time, its format, etc. In short, metadata is information about all relevant aspects of information
objects in terms of the complete content provision chain.
There are three main types of metadata used for multimedia data [NISO04]:
•
Descriptive or Content metadata: information about the object captured in the media
data. It is used for discovery of the media objects. It includes:
o object's name: the name of the object captured in the data (e.g. heart in an
image data)
o title: the title given to the data by the content creator (e.g. heart cancer
treatment)
o materials: form of the media (e.g paper, electronic, film, etc.)
o publication date
o edition or version
o physical characteristics: physical aspects of the object (e.g for a book, its
material, form, color, etc.)
o etc.
Introduction
•
69
Structural or Technical metadata: data about the media data itself (not about the
object in the media). It is used primarily for the storage of objects in a repository and
for presentation. It includes information about:
o digitisation or compression specifications
o encryption
o data format
o file size, etc.
•
Administrative metadata: information related to the management of the media data. It
is used for managing and preserving the media objects in the repository. It includes:
o Location information
o Acquisition information
o Ownership, rights, permission, reproduction information
o Usage information, etc.
Metadata is used for describing resources and thereby improving access to them [Tayl03].
Apart from its application for resource discovery, metadata facilitates a wide range of
purposes. For examples in distributed multimedia systems, metadata can be used for the
following purpose [Kosch03, Adela01]:
•
To search multimedia data by content, e.g., ‘list all video clips from an online video
shop where Roger Moore plays 007’. But, before multimedia data can be searched, the
data have to be indexed; that means content metadata has to be extracted from the
video either automatically or annotated manually.
•
To describe intellectual properties of the multimedia data. These may guarantee a fair
use of the data in commercial applications.
•
To adapt content according to the usage context. For example, if metadata such as
frame rate, spatial resolution and bit rate are easily accessible, decisions about what
media resources to send can be made. The metadata provide media properties so that
appropriate transformation (s) can be applied to match the content with the usage
environment.
70
Over the years there have been several initiatives to develop standards for content metadata
such as Dublin Core, SMPTE13 Metadata Dictionary, and MPEG-7. These standards are
general, i.e. they do not target a particular industry or application domain, and are supported
by well-recognized organizations [Kosch03]. In the following sections we give a brief
overview of these standards.
4.2 Dublin Core
The Dublin Core14 is a simple content description model for electronic resources. It is a
metadata element set intended to facilitate the discovery of electronic resources. Originally
conceived for author-generated description of web resources, it has attracted the attention of
formal resource description communities such as museums, libraries, government agencies,
and commercial organizations [Adela01]. The Dublin Core Metadata Initiative (DCMI) began
in 1995 with an invitational workshop in Dublin, Ohio, that brought together experts from the
library world, the networking and digital library research communities, and a variety of
content specialties [ISO03]. The DCMI is an organization dedicated to promoting the
widespread adoption of interoperable metadata standards and developing specialized metadata
vocabularies for describing resources that enable more intelligent information discovery
systems. At the end of the workshop, the original Dublin Core metadata elements emerged.
Since the original workshop there has been a steadily growing interest in resource descriptions
that are easy to create and that almost anyone can understand. The interest in cross-domain
discovery fueled growing participation in a series of subsequent DCMI workshops. These
workshops enabled to create a standard Dublin Core metadata consists of fifteen elements (see
Figure 5.1). The Dublin Core metadata elements describe the content of the document,
information about the intellectual property contained in the document and information about
the particular instances of the document [ISO03, Hill03]. The elements in the Dublin Core are
optional, repeatable, modifiable, and can appear in any order.
Advantages associated with Dublin Core are simplicity, interoperability, scalability,
international consensus and flexibility [Hunt98]. The simplicity of the Dublin Core lowers the
cost of creating metadata and promotes interoperability. On the other hand, simplicity does
13
14
SMPTE: Society of Motion Picture and Television Engineers (http://www.smpte.org)
http://purl.org/DC
Dublin Core
71
not accommodate the semantic and functional richness supported by complex metadata
schemes such as MPEG 7 [Libse04].
Dublin Core was designed specifically for generating metadata for textual documents.
Although a number of workshops have been held to discuss the applicability of Dublin Core
to non-textual documents such as images, sound and moving images, they have primarily
focused on extensions to the 15 core elements through the use of subelements and schemes
specific to audiovisual data, to describe bibliographic-type information rather than the actual
content [Hunt98]. For example, metadata about video such as frame rate, resolution, etc. are
important to adapt the video according to the usage context. But Dublin core does not have
elements to specify those metadata. Therefore, Dublin core does not provide all necessary
metadata information for the purpose of content adaptation.
72
E
Figure 4.1: Dublin Core Metadata Element Set
4.3
SMPTE Meta-Dictionary
The SMPTE Metadata Dictionary is a collection of registered names and data types,
developed mostly for the television and video industry that forms the SMPTE membership. Its
hierarchical structure allows endless expansion and mechanisms for data formatting in TV
and video-signals, and provides a common method of implementation. Most metadata are
media-specific attributes, such as timing information.
MPEG 7
4.4
73
MPEG 7
MPEG-7 [Mart02, Day02, Bekker00] formally named “Multimedia Content Description
Interface” is an ISO/IEC standard developed by MPEG15 that provides a rich set of tools for
completely describing multimedia content so that users can search, browse and retrieve that
content more efficiently and effectively. The effort is being driven by specific requirements
taken from a large number of applications related to image, video and audio databases, media
filtering and interactive media services (radio, TV programs), image libraries and so forth
[Smith00]. MPEG-1, MPEG-2, and MPEG-4 represent the content itself but MPEG-7
represents information about the content. While earlier MPEG versions were focusing on
making multimedia content available, MPEG-7 allows the localization, filtering, managing
and processing of desired media files by tagging them with information regarding their
content and origin [Libse04].
Major guiding principles of MPEG-7 include [Kosch03, Martinez03]:
- It is not specific to any particular application. It is intended to be applicable to content
associated to any application domain. To ensure that the needs of as many applications
as possible are included, MPEG-7 has been developed by experts representing
broadcasters, electronics manufacturers, content creators and managers, publishers,
intellectual property rights managers, telecommunication service providers and
academia.
- For requirements that are not covered by the standard, MPEG-7 is extensible to
address new application domains. It allows the extension of the core set of description
tools in a standard way so that interoperability is guaranteed.
- It addresses a variety of data types (or modalities) such as speech, audio, image, video,
graphics, 3-D models, and synthetic audio.
- It is applicable independently of the medium that carries the content. Media can
include paper, film, tape, CD, hard disk, digital broadcast, Internet streaming, etc.
- It is also applicable independently of the content representation format, whether
analogue or digital, compressed or uncompressed, etc.
15 MPEG (Moving Picture Experts Group) is a working group of the Geneva-based ISO/IEC standards organization (International Standards
Organization/ International Electro-technical Committee) that developed, in particular MPEG-1, MPEG-2 and MPEG-4 standard [Henny
Bekker, 2000].
74
-
It includes description capabilities with different levels of abstraction, from low-level
features such as color, texture, and motion to high-level features conveying semantic
meaning. Often low-level features are extracted automatically whereas semantic
features need to be extracted manually or semi-automatically.
Using the MPEG-7 description tools one can create different types of descriptions for a
multimedia content. These descriptions contain information related to [Mart02]:
- the content’s creation and production processes (such as director, title, actors, and
location)
- usage of the content (such as broadcast schedules and copyright pointers)
- storage and representation of the content (such as storage format and encoding format)
- the content’s spatial, temporal, or spatio–temporal structure (for example, scene cuts,
segmentation in regions, and region motion tracking)
- the low-level features in the content (for example, colors, textures, sound timbres, and
melody descriptions)
- semantic information of the reality captured by the content (for example, objects,
events, and interactions between objects).
- how objects are related and gathered in collections
- browsing content in an efficient way (such as summaries, variations, and transcoding
information)
- the interaction of the user with the content (such as user preferences and usage
history), etc.
MPEG-7 standardized descriptors for defining the syntax and semantics of each feature
representation and description schemes for specifying relationships between components
(both descriptors and description schemes). Standardizing the multimedia content description
addresses the interoperability issue by allowing content and its descriptions to be exchanged
across different systems [Libse04].
Figure 4.2 [Smith00] below shows the scope of the MPEG-7 standard within the multimedia
content processing chain. The description generation (feature extraction, indexing process,
annotation and authoring, etc.) and the description consumption (search engine, filtering,
retrieval process, browsing etc.) tools are not within the scope of MPEG-7 standard.
MPEG 7
75
This means that only the description format—the minimum needed to ensure interoperability
among applications—is standardized, which is the standard’s goal.
Figure 4.2: Scope of the MPEG Standard
MPEG-7 has four major building blocks [Mart02]:
- Descriptor (D): describes features (multimedia information), attributes, or groups of
attributes of multimedia content. A Descriptor defines the syntax and the semantics of
the feature representation (metadata element). MPEG-7 Descriptors describe for
example the color, the texture, the shape, the motion or the audio features.
- Description Scheme (DS): is the structure and semantics of the relationships between
its components, which may be both Descriptors and Description Schemes. An example
of Video Segment Type Scheme is shown in Figure 3.3.
- Description Definition Language (DDL): is a language that allows the creation of new
DSs and possibly Ds. It also allows the extension and modification of existing DSs.
The W3C's XML schema language is adopted as the MPEG-7 DDL. The DDL adds
extensions to the XML Schema to address specific MPEG-7 requirements such as
support for vectors, matrices and typed references. The adoption of XML Schema as
the basis for MPEG-7 DDL and the resulting XML-compliant instances (which
contains Descriptions in MPEG-7 textual format) eases interoperability by using a
common, generic and powerful presentation format. It also facilities the adoption of
MPEG-7 tools for extending existing XML applications with multimedia content
description functionalities.
-
Systems Tools: are tools to support synchronization of descriptions with content,
delivery mechanisms, and coded representations (both textual and binary format) for
efficient storage and transmission and the management and protection of intellectual
property in MPEG-7 Descriptions. One can represent descriptions in text format also
76
called TeM (Textual Format for MPEG-7) and binary format called BiM (Binary
Format for MPEG-7) or using a mixture of the two, depending on the usage scenario.
The text format is defined with the DDL and the instances are XML files. It is suitable
for editing, searching and filtering, whereas the binary format is suitable for storage,
transmission, and streaming delivery. The binary descriptions have a more efficient
compression than the textual descriptions and provide flexibility for streaming and
efficient random search and access to description elements without the need of parsing
the complete stream.
<complexType name="VideoSegmentType">
<complexContent>
<extension base="mpeg7:SegmentType">
<sequence>
<choice minOccurs="0">
<element name="MediaTime" type="mpeg7:MediaTimeType" />
<element name="TemporalMask" type="mpeg7:TemporalMaskType" />
</choice>
<choice minOccurs="0" maxOccurs="unbounded">
<element name="VisualDescriptor" type="mpeg7:VisualDType" />
<element name="VisualDescriptionScheme" type="mpeg7:VisualDSType" />
<element name="VisualTimeSeriesDescriptor"
type="mpeg7:VisualTimeSeriesType" />
</choice>
<element name="MultipleView" type="mpeg7:MultipleViewType" minOccurs="0" />
<element name="Mosaic" type="mpeg7:MosaicType" minOccurs="0"
maxOccurs="unbounded" />
<choice minOccurs="0" maxOccurs="unbounded">
<element name="SpatialDecomposition"
type="mpeg7:VideoSegmentSpatialDecompositionType" />
<element name="TemporalDecomposition"
type="mpeg7:VideoSegmentTemporalDecompositionType" />
<element name="SpatioTemporalDecomposition"
type="mpeg7:VideoSegmentSpatioTemporalDecompositionType" />
<element name="MediaSourceDecomposition"
type="mpeg7:VideoSegmentMediaSourceDecompositionType" />
</choice>
</sequence>
</extension>
</complexContent>
</complexType>
Figure 4.3 Schema Definition of VideoSegmentType
A Description consists of a Description Scheme (structure) and the set of Descriptor values
(instantiations) that describe the data. A Descriptor value is an instantiation of a Descriptor for
MPEG 7
77
a given data set (or subset). The Description Schemes provide a way to describe in XML the
important concepts related to multimedia data in order to facilitate the searching, indexing and
filtering of multimedia data [Smith00]. The Description Schemes are designed to describe
both the generic aspects of multimedia data management and the specific content features of
the multimedia data. Figure 4.4 shows MPEG-7 document file for video data.
MPEG-7 standard consists of several parts [Manj02]. The different parts of the MPEG-7
standard collectively offer a comprehensive set of multimedia description tools to create
descriptions that can be used by applications that enable fast and efficient access to content.
The multimedia descriptions can also be used for searching, filtering and adapting the content
[Smith00]. For the purpose of content adaptation, MPEG-7 standard contributes at least the
following:
- MPEG-7 standard provides tools (the navigation and access part) for describing
variations of the content such as summaries and abstracts; scaled, compressed and low
resolution versions, and versions with different languages and modalities-audio, video,
image, text, and so forth.
- MPEG-7 standard provides tools (the content management part) for describing media
information such as storage media, coding format, size, quality and transcoding hints
for adapting content to different networks and terminals [Mart02].
78
Figure 4.4: MPEG-7 Description Example
MPEG-7 is currently used in applications such as video annotation and browsing, Internet
audio/video search and spoken content description. Some industries have already developed
applications using MPEG-7. IBM has developed an MPEG-7 Annotation Tool that assists in
annotating MPEG-1 video sequences with MPEG-7 metadata [Smith00]. The Annotation
Tool is a stand-alone Windows application and it can be freely downloaded from IBM
alphaWorks16. Canon Research Centre Europe Ltd (CRE)17 has developed a spoken content
transcription service that uses MPEG-7 to describe the spoken content. The application uses a
Web interface where the user can upload an audio file in wav format and the service creates
and returns an XML file conforming to the MPEG-7 SpokenContent description scheme. The
audio part of MPEG-7 contains a SpokenContent high-level tool targeted for spoken data
management applications. The MPEG-7 SpokenContent tool provides a standardized
representation of an automatic speech recognition output, i.e. of the semantic information (the
spoken content) extracted by an automatic speech recognition system from a spoken signal. It
consists of a compact representation of multiple word and/or sub-word hypotheses produced
by an automatic speech recognition engine.
16
17
http://www.alphaworks.ibm.com/tech/videoannex
http://www.cre.canon.co.uk/mpeg7asr/
Other standards
79
Thomson has developed a streaming media search engine called Singingfish18. Singingfish is
one of the search engines that currently utilize MPEG-7. It is text-based search engine that
uses MPEG-7 Multimedia Description Schemes to model the metadata characteristics of
Internet streaming media.
A query by humming (QBH) system enables a user to hum a melody into a microphone
connected to a computer in order to retrieve a list of possible song titles that match the query
melody [Batke04]. The system analyzes the melodic and rhythmic information of the input
signal. The extracted data set is used as a database query. The result is presented as a list of
e.g. ten best matching results. A QBH system is a typical application of the MPEG-7
description standard. Different QBH systems are already available on the World Wide Web
(WWW). Musicline19 is a commercial QBH system developed by Fraunhofer. Musicline’s
MPEG-7 powered search engine allows finding music by humming. The search engine
extracts the melody from the user’s singing or from the instrument the user is playing and
compares the result with a database. Currently, the database contains about 3500 melodies of
mainly pop music.
4.5
Other standards
There are also other standardization activities that are focused to more specific applications or
application domains. These standards include TV Anytime Metadata20, SMEF21, and
DICOM22.
The Standard Media Exchange Framework (SMEF) is developed by BBC to support and
enable media asset management ("MAM") as an end-to-end process across its business areas,
from commissioning to delivery to the home. The SMEF Data Model (SMEF-DM) provides a
set of definitions for the information required in production, distribution and management of
media assets, currently expressed as a data dictionary and set of Entity Relationship
Diagrams.
18
http://www.singingfish.com
http://www.musicline.de/de/melodiesuche/input
20
TV-Anytime (http://www.tv-anytime.org/).
21
SMEF: Standard Media Exchange Framework (http://www.bbc.co.uk/guidelines/smef/).
22
DICOM is developed by the American College of Radiology (ACR) and the National Electrical Manufactures
Association (NEMA).
19
80
TV-Anytime [TV02] is a metadata standard developed by the TV-Anytime Forum23 to define
specifications for programme level content descriptions to allow viewers to find, navigate and
manage contents from a variety of sources, including enhanced broadcast, interactive TV,
Internet and local storage. Such metadata includes attractors (title, synopsis, genre, cast and
awards etc.) that aid the acquisition of available content organised in EPGs (Electronic
Programme Guide).
The Digital Imaging and Communications in Medicine (DICOM) is developed to create an
aid for the distribution, transmission and viewing of medical images such as CT scans, MRIs,
and ultrasound with their associated information [DICOM01]. Unlike other metadata
standards, DICOM stores both the metadata and the content in one file. The metadata
information which contains patient’s name, the type of scan, image dimensions, etc. is stored
in the file header.
4.6 Summary
In this chapter we presented an overview of the most important initiatives to develop
standards for metadata. Our interest in this thesis is to develop an open general architectural
framework for adapting multimedia contents utilizing external adaptation tools developed as
services. One major component of the architecture is the content adaptation module. This
module analysis the context information, the content and the available adaptation services in
order to determine on the number and type of adaptations required to deliver the content. In
view of this context, there is a need for metadata in order to describe the multimedia content.
Reviewing the different standards proposed up to know, the best candidate as a metadata
standard at the moment is MPEG-7: it is media independent, it is currently being developed as
an international standard, it is the standard that provides the richest and most versatile set of
multimedia feature descriptions, and it embraces a family of standards such as SMPTE,
Dublin Core and more, so it is the highest interoperable standard among well-known industry
standard.
23
TV Anytime Forum: association of organizations which seeks to develop specifications to enable audio-visual and other services based on
mass-market high volume digital storage in consumer platforms – simply referred to as local storage.
Chapter
5
Context Metadata
Description
In this chapter we present some of the commonly used context descriptions. We also discuss
the advantages and limitations of each description. First we start by giving a definition of
context in Section 5.1. Then we present the most important context representation developed
for the purpose of content adaptation: CC/PP (Section 5.2), IETF Media Feature Sets (Section
5.3), CSCP (Section 5.4), and MPEG-21 (Section 5.5). We end the chapter with a summary in
Section 4.6.
5.1 Introduction
Not only the availability of content metadata but also context metadata related to user, device,
network, environment, etc. are essential to perform appropriate content adaptations. Although
there are a number of clear benefits for the use of context metadata in other issues of
pervasive computing environments, in this chapter the primary focus is on their use for
content adaptation. To facilitate content adaptation for pervasive computing applications, the
context metadata need to be described, maintained and processed. There are several
description tools that have been developed for context metadata. Before we see these
description tools and their characteristics, we first give a definition of “context”24. It is quite
unlikely that a single definition of context will be accepted by all researchers. From time to
time, from application to application this definition varies. Since it was coined in 1994
[Schi94b], numerous definitions of context have been provided in the literature. Historically
[Wing01], “context” has been adapted from linguistics, referring to the meaning that must be
24
In this work, the use of the notion of “context” is limited to pervasive computing systems and in particular its use for content adaptation
purpose.
82
Chapter 5. Context Metadata Description
inferred from the adjacent text. Here, we are interested with the definition of context with
respect to the computing world. Context definitions are far away from mathematical precision
and a particular definition often strongly depends on an author’s subjectiveness [Reim02].
Most of the definitions are given by list of examples or by choosing synonyms for context
[Dey01].
Schilit and Theimer [Schi94b] define context as location, identities of nearby people and
objects, and changes to those objects. Brown et al. [Brown97] define context in a similar
manner as location, identities of the people around the user, the time of day, season,
temperature, etc. Ryan et al. [Ryan97] define context as the user’s location, environment,
identity and time. Dey [Dey98] enumerates context as the user’s emotional state, focus of
attention, location and orientation, date and time, objects, and people in the user’s
environment. These are definitions by examples. As noted by [Dey01], definitions by
examples are difficult to use since it is difficult to determine whether a type of information
which is not listed in the definition is part of the context or not. It is not clear how the
definition can be used to solve the dilemma.
Synonym definitions of context are provided by Brown [Brown96], Rodden [Rodd98] and
Ward [Ward97]. For these researchers the term context refers to an environment or a
situation. While Brown [Brown96] considers context to be the elements of the user
environment that the user’s computer knows about, Ward et al. [Ward97] view the context as
the state of the application’s surrounds and Rodden et al. [Rodd98] define it to be the
application’s settings. The synonym definitions can provide an indication for the meaning of
the term context but they are at abstract level and it can be extremely difficult to apply them
in a concrete application for instance in the case of content adaptation.
Further refined definitions of context were provided by Schilit et al. [Schi94a], Dey et al.
[Dey99b] and Pascoe [Pascoe98]. According to Schilit et al. context is the execution
environment which changes constantly and consists of computing environment (available
processors, accessible devices, network capacity and connectivity, and computing costs), user
environment (location, collection of nearby people, and social situation), and physical
environment (lighting and noise level). Dey et al. define context to be the user’s physical,
social, emotional or informational state. Pascoe defines context to be the subset of physical
and conceptual states of interest to a particular entity.
Introduction
83
Though Pascoe tried to define context in a more general way, his focus on the need of a single
entity makes it very specific. Context is all about the whole situations relevant to an
application and its set of users. Thus, it is not possible to list which aspects of all situations
are important since it changes from situation to situation. Moreover, an element of context
important in one case may be immaterial for another.
A comprehensive and more general definition of context that can be adapted according to the
particular application scenario was provided by Dey [Dey01]. According to this definition,
context is any information that can be used to characterize the situation of an entity (where an
entity can be a person, place, or physical or computational object) that are considered relevant
to the interaction between a user and an application, including the user and the application
themselves. Context is typically the location, identity, and state of the people, groups and
computational and physical objects.
Based on this definition, application developers can easily enumerate the context for a given
application scenario. A good understanding of the general context concept helps application
designers to determine what context to be considered in their applications. For example, in the
case of context-aware25 content adaptation for pervasive systems, the more relevant context
information consists of the following [Chaari06, Sun03, Held02]:
•
User and environment context: characteristics, preferences, role, activity (e.g meeting,
home, driving, etc.), physical environment (e.g. location, weather, time etc.)
•
Device context: hardware characteristics (e.g. screen resolution, color depth, display
size, etc.) and software characteristics (operating system vender and version, content
type and format capabilities, etc.)
•
Network context: type of connection, available bandwidth, delay, etc.
This context information also called context profiles26 allows adapting content to device
capabilities, user preferences and characteristic, and network conditions. To facilitate the
programming of context-aware content adaptation systems, an infrastructure is necessary to
gather, manage and disseminate context information. This requires that the context-aware
adaptation system should be capable of acquiring and transforming context information and of
25
Context-aware is a term used to refer to systems, devices or applications that have information about the circumstances under which they
operate and can react accordingly.
26
The term context profiles or context metadata refers to the collection of user, environment, device and network context information.
84
reasoning on the basis of this information, thus showing the need for a uniform and
interchangeable context representation.
Use of context information demands appropriate standards for applications to express and
exchange context information. A representation of context information should be applicable
throughout the process of gathering, transferring, storing and interpreting the context
information. According to Held et al. [Held02], a context representation should be structured,
interchangeable, composable/decomposable, uniform, extensible and standardized. Up to
now, a number of efforts have been done to develop context representation mechanisms such
as UPS [Leml02], CC/PP [Klyne01], IETF [Klyne99], CSCP [Held02], MPEG-21 [Burn03],
etc. In the following sections we will give a brief overview of the most important
representation mechanisms: CC/PP, IETF Media Feature Sets, CSCP and MPEG-21.
5.2
Composite Capabilities/Preference Profiles
The Composite Capabilities/Preference Profiles (CC/PP) is a context description standard for
content negotiation and adaptation proposed by the W3C [Klyne01]. CC/PP offers an XMLbased format for exchanging context descriptions, as well as vocabularies for expressing
context information related to device capabilities and user preferences. The intended use of
such a profile is to allow an origin server to deliver content customized to a device capability
and user’s preferences. CC/PP is based on the Resource Description Framework (RDF)
[Lass99], which was designed by the W3C as a general purpose metadata description
language. The CC/PP specification defines a basic profile structure. A CC/PP profile is
organized as a two-level tree containing components and each component having a number of
attributes (shown in Figure 5.1). The particular components and attributes are not defined by
the CC/PP specification. The CC/PP architecture model uses RDF to define schemas upon
which vocabularies can be independently developed and implemented. This flexible
mechanism ensures CC/PP extensibility but limits the interoperability of the descriptions. In
this model, statements are made about the components (the subject) that have named attribute
(the predicate) and values for those attributes (the object).
85
Profile
Attribute
(e.g. displayWidth)
Component
(e.g TerminalHardware)
Attribute
Component
Component
Figure 5.1: CC/PP Context Representation Model
The CC/PP representation more or less fulfils the requirements for context representation that
are listed above; nevertheless it has some disadvantages such as it is a strict two-level
structure [Held02]. For example, if we consider the device profile, it consists of components
such as hardware and software where each component is further composed of different
components. In the case of the software component it is composed of an operating system and
a user agent where a user agent has attributes such as name, version, content types, etc. From
this example, we can understand that the device context naturally has more than two-level.
The CC/PP representation provides extensibility but with limited structure. Furthermore,
CC/PP requires attributes names to be unambiguous even if they are used in different
components. Figure 5.2 shows an example of CC/PP profile in XML form.
<?xml version="1.0"?>
<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:ccpp="http://www.w3.org/2002/11/08-ccpp-schema#"
xmlns:ex="http://www.example.com/schema#">
<rdf:Description rdf:about="http://www.example.com/profile#MyProfile">
<ccpp:component>
<rdf:Description rdf:about="http://www.example.com/profile#TerminalHardware">
<rdf:type rdf:resource="http://www.example.com/schema#HardwarePlatform" />
<ex:displayWidth>320</ex:displayWidth>
<ex:displayHeight>200</ex:displayHeight>
</rdf:Description>
</ccpp:component>
<ccpp:component>
86
<rdf:Description rdf:about="http://www.example.com/profile#TerminalSoftware">
<rdf:type rdf:resource="http://www.example.com/schema#SoftwarePlatform" />
<ex:name>EPOC</ex:name>
<ex:version>2.0</ex:version>
<ex:vendor>Symbian</ex:vendor>
</rdf:Description>
</ccpp:component>
<ccpp:component>
<rdf:Description rdf:about="http://www.example.com/profile#TerminalBrowser">
<rdf:type rdf:resource="http://www.example.com/schema#BrowserUA" />
<ex:name>Mozilla</ex:name>
<ex:version>5.0</ex:version>
<ex:vendor>Symbian</ex:vendor>
<ex:htmlVersionsSupported>
<rdf:Bag>
<rdf:li>3.2</rdf:li>
<rdf:li>4.0</rdf:li>
</rdf:Bag>
</ex:htmlVersionsSupported>
</rdf:Description>
</ccpp:component>
</rdf:Description>
</rdf:RDF>
Figure 5.2: Example of CC/PP Profile in XML/RDF Format
This sample profile is a collection of the capabilities and preferences associated with either a
user or the hardware platform or a software component. Each collection of capabilities and
preferences are organized within the “ccpp:component”. These description blocks may
contain subordinate description blocks to describe other collections of attributes.
It is also possible to use multiple namespaces. This might be useful to define application
specific capabilities and preferences. In Figure 5.2, there are three namespace declarations.
The first namespace (http://www.w3.org/1999/02/22-rdf-syntax-ns#) declaration is for RDF
usage. The second namespace declaration (http://www.w3.org/2002/11/08-ccpp-schema#) is for
CC/PP core structural vocabulary, which includes "component" and other properties that are
intrinsic
to
the
CC/PP
framework.
The
third
namespace
declaration
(http://www.example.com/schema#) is for CC/PP components properties vocabulary.
CC/PP profiles are intended to provide information necessary to adapt web content to best fit
the device capabilities and user’s preferences. The CC/PP exchange protocol is based on the
HTTP extension framework. During a content request the user agent sends his request with
the URI that references his profile. The URI is included in the HTTP header. When a content
server receives the request, it inquires the CC/PP repositories using the URI.
IETF Media Feature Sets
87
Up to now, the only implementation of CC/PP is UAProf (or User Agent Profile), by the
Open Mobility Alliance (formerly the WAP-Forum) which is targeted to mobile devices
[Vela04]. UAProf has a very specific scope and does not cover the whole spectrum of devices
that can access nowadays Internet services and applications, nor even considers user
characteristics. Figure 5.3 show the structure of the UAProf with its components.
Figure 5.3: UAProf Main Components.
5.3
IETF Media Feature Sets
The IETF27 Content Negotiation Working Group [Klyne99] has developed a standardized
syntax called the Media Feature Sets (MFSs) that allows protocol-independent content
negotiation. The MFSs have been developed to enhance access to internet applications such as
the World Wide Web. The MFSs specifies device capabilities and user preferences by
unstructured attribute/value pairs. In contrary to CC/PP, it specifies the features of the
supported and preferred content representation rather than simply the device capabilities.
The Media Feature Sets description uses Boolean expressions for complex capabilities and
preferences. For example, the following expression describes a data resource that can be
displayed either:
27
IETF (Internet Engineering Task Force): a large open international community of network designers, operators, vendors, and researchers
concerned with the evolution of the Internet architecture and the operation of the Internet. (http:// http://www.ietf.org/overview.html).
88
(a) as a 750x500 pixel image using 15 colors, or
(b) at 150dpi on an A4 page.
(| (& (pix-x=750) (pix-y=500) (color=15) )
(& (dpi>=150) (papersize=iso-A4) ) )
Note that “|’ is used for “or” and “&” is used for “and”. In the above expression dpi, pix-x,
pix-y, color, papersize are feature tags. One interesting feature of this description is that it is
possible to assign quality values (from 0 to 1) to capabilities and preferences. For example the
expression given above can be written as follows:
(| (& (Pix-x=750) (pix-y=500) (color=15)); q=0.8
(& (dpi>=150) (papersize=iso-A4)) ; q=0.7)
This expression shows that the image with 750x500 pixels using 15 colors is preferred over
the printed form of 150 dpi and A4 size.
This Media Feature Sets has been originally motivated by the requirements from web
browsers to send the browser's display characteristics to the web server to allow the server to
choose an appropriate representation. The media feature sets are included in the request
header like the accept-content header filled by web browsers. Up on receiving the request, the
content server retrieves the feature information from the header and sends a variant that
matches the features.
Drawbacks of the Media Feature Sets are: they have no structure (they use attribute/value
pairs only), there is no formal extension mechanism to add new features, there is no means to
reference external Media Feature Sets, and the Media Feature Tags currently-registered
[MFTR05] cover client capabilities, but not the relevant network properties.
Comprehensive Structured Context Profiles
5.4
89
Comprehensive Structured Context Profiles
To addresse the problems of context profile representation, Held et al. [Held02] have
developed the Comprehensive Structured Context Profiles (CSCP) framework. Just as CC/PP,
CSCP is based on RDF and thus inherits its interoperability and extensibility. Yet CSCP
overcomes the deficits of CC/PP regarding structuring. It does not impose any fixed
hierarchy. It rather supports the full flexibility of RDF to express natural structures of context
information. Attribute names are interpreted context-sensitively according to their position in
the profile structure. Thus, unambiguous attribute naming throughout the profile is not
required.
CSCP profiles are queried by referencing attribute values of entities in the profile. Attribute
values are referenced using CSCP path expressions, for example the expression
ContextProfile.DeviceProfile.Software.UserAgent.ContentTypes refers to content types
accepted by the user agent. A CSCP path expression always starts at a named resource (i.e.
root node) identified by its URI. The attributes of this resource are addressed by simply
appending a query component to the URI containing the attribute name. As the value of the
selected attribute can be a resource itself, attribute names can be chained in a dot-separated
list, thus forming a path traversing a sub-tree of the root node of the CSCP profile. Unlike
plain RDF, CSCP semantics allows for only a single valid attribute value at a time. Hence,
CSCP paths are unambiguous even if there are multiple uses of an attribute (e.g. the attribute
ContentTypes is used both in UserAgent and Preferences elements). The main limitation of
CSCP is that the description has not been standardized so it makes interoperability difficult.
For the device described in Figure 5.2 by CC/CP, the corresponding description in CSCP is
given in Figure 5.3.
<?xml version="1.0" encoding="UTF-8"?>
<rdf:RDF
xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:cscp="context-aware.org/CSCP/CSCPProfileSyntax#"
xmlns:dev="context-aware.org/CSCP/DeviceProfileSyntax#"
xmlns:net="context-aware.org/CSCP/NetworkProfileSyntax#"
xmlns="context-aware.org/CSCP/SessionProfileSyntax#"
<SessionProfile rdf:ID="Session">
<cscp:defaults rdf:resource= "http://Context/CSCPProfile/previous#Session"/>
90
<device>
<dev:DeviceProfile>
<dev:Hardware>
<dev:ScreenWdith>320</dev:ScreenWidth>
<dev:ScreenHeight>200</dev:ScreenHeight>
</dev:Hardware>
<dev:Software>
<dev:OperatingSystem>
<dev:Name>EPOC</dev:Name>
<dev:Version>2.0</dev:Version>
<dev:Vendor>Symbian</dev:Vendor>
</dev:OperatingSystem>
<dev:UserAgent>
<dev:Type>Browser</dev:Type>
<dev:Name>Mozilla</dev:Name>
<dev:ContentTypes>
<rdf:Bag>
<rdf:li>text/html</rdf:li>
<rdf:Bag>
</dev:ContentTypes>
</dev:UserAgent>
</dev:Software>
</dev:DeviceProfile>
</device>
</SessionProfile>
</rdf:RDF>
Figure 5.3: CSCP Profile Example
5.5
MPEG 21
MPEG-21 [Burn03], formally known as ISO/IEC 21000-Multimedia Framework is one of a
series of standards being produced by the Moving Picture Experts Group. MPEG has a long
history of producing multimedia standards: MPEG-1 (1993), MPEG-2 (1996), MPEG-4
(1999), and MPEG-7 (2001). The MPEG 21 standard aims to enable a transparent and
augmented use of multimedia resources across a wide range of networks and devices. In order
to realize this goal, MPEG-21 provides a normative open framework for multimedia delivery
and consumption. This framework will be of use for all players in the multimedia delivery and
consumption chain. It will provide content creators, producers, distributors, and service
providers with equal opportunities in the MPEG-21 enabled open market. This will also be to
the benefit of content consumers, providing them access to a large variety of contents in an
interoperable manner. From a content adaptation point of view, content consumers can be
MPEG 21
91
services that perform content transformation, for example, in our approach they are adaptation
services.
The content description has been addressed by MPEG-7 but the description of content format
and usage environments has not been addressed and it has been the target of the MPEG-21
Digital Item Adaptation (DIA) specification.
MPEG-21 is based on two essential concepts: the definition of a fundamental unit of
distribution and transaction (the Digital Item) and the concept of Users interacting with
Digital Items. The Digital Items can be considered as the “what” of the Multimedia
Framework (e.g., a video collection, a music album) and the Users can be considered as the
“who” of the Multimedia Framework. Figure 5.4 shows the MPEG-21 multimeda
frameowork.
In MPEG-21, a User is an entity that interacts within the multimedia framework: it can be a
content creator, a content distributor, or a content consumer (end user). Users include
individuals,
consumers,
communities,
organizations,
corporations,
consortia,
and
governments. Users are identified specifically by their relationships to other users for a certain
interaction. From a purely technical perspective, MPEG-21 makes no distinction between
“content provider”, and “consumer”-both are Users in the MPEG-21 multimedia framework.
MPEG-21 introduced the notion of Digital Item to represent a multimedia content. It is the
fundamental unit of distribution and transaction among Users in the MPEG-21 multimedia
framework. It is a structured digital object with resources (the content), a unique identification
and corresponding metadata (e.g. MPEG-7 description). The structure relates to the
relationships among the parts of the Digital Item, both resources and metadata.
92
Figure 5.4: A Representation of the MPEG-21 Multimedia Framework [Maga04]
MPEG-21 provides a framework for Users interaction through the Digital Item. Here, the goal
is to develop the necessary technologies that support the access, the exchange, the use and the
manipulation of the Digital item in an effective, transparent and interoperable way. In order to
achieve the goal, the framework defines the elements necessary and the operations applicable
to these elements which at the end guarantee an effective content transmission chain. To
realize these elements, the MPEG-21 standard is organized into several more or less
independent parts primarily to allow various slices of the technology to be useful as standalones [Burn03]. Here, we discuss two of these parts: the Digital Item Declaration (DID) (part
2) and Digital Item Adaptation (DIA-part 7) because they are highly related to our work.
Digital Item Declaration (DID)
The Digital Item Declaration (ISO/IEC 21000-2) [ITMF03] specifies a uniform and flexible
abstraction and interoperable schema for declaring the structure and makeup of Digital Items.
Through the Digital Item Declaration Language (DIDL), a DI can be declared by specifying
its resources (content), metadata, and their interrelationships. The MPEG-21 standard
describes the DID using a model, a representation and a schema. The DID model describes a
set of abstract terms and concepts for defining Digital Item. The representation (in XML)
MPEG 21
93
contains the normative description of the syntax and semantic of each of the DID elements.
The DIDL schema comprises the entire grammar of the DID representation in XML.
Digital Item Adaptation (DIA)
One of the goals of MPEG-21 is to achieve an interoperable transparent access to distributed
multimedia contents by shielding users from network and terminal installation, management,
and implementation issues. Achieving this goal requires the adaptation of Digital Items (see
Figure 5.5). The Digital Item Adaptation (ISO/IEC 21000-7) specifies tools for the adaptation
of Digital Items [MDS03]. Figure 5.5 illustrates a Digital Item that is subject to a resource
adaptation engine and a description adaptation engine, which produces the adapted Digital
Item. For the adaptation, both the description of the content and a description of its format and
the usage environment are essential. The content description has been addressed by MPEG-7
but the description of the content format and the usage environments has not been addressed
and it is the target of the MPEG-21 Digital Item Adaptation (DIA) specification. The major
component of the DIA specification is the usage environment description tool which consists
of [ISO04]:
•
Terminal characteristics: media resource encoding and decoding formats, hardware,
software and communication protocols, etc.
•
Network characteristics: bandwidth, delay and error characteristics
•
Natural environment characteristics: location and time, illumination, altitude,
temperature etc.
•
User preferences: media preferences, presentation preferences, usage history, and
location characteristics.
94
Digital Item
Adaptation Engine
Digital
Item
Resource Adaptation
Engine
Adapted
Digital
Item
Description
Adaptation Engine
DIA Descriptions
Scope of
standardization
DIA Tools
Figure 5.5: The Adaptation of Digital Items
MPEG-21 is more than a metadata description standard. MPEG-21 standardizes the DIA tools
for adapting multimedia contents according to usage environment descriptors. The adaptation
engines are non-normative tools. The implementation of an engine that actually performs the
required adaptation steps to the multimedia content is left to tool vendors and is beyond the
scope of standardization.
5.6 Summary
This chapter presented the context metadata representation mechanism. We have discussed
different representation mechanisms: CC/PP, IETF Media Feature Sets, CSCP and MPEG-21.
CC/PP is a context description framework proposed by W3C for describing device
capabilities and user preferences based on RDF. CC/PP defines the basic structure of context
profiles and introduces a mechanism for profile decomposability. Whereas CC/PP is
interchangeable, decomposable, uniform and extensible, it lacks sufficient structuring. Its
strict two-level-hierarchy is not appropriate to capture complex profile structures.
Furthermore, CC/PP does not allow for context-sensitive interpretation of attributes requiring
globally unambiguous attribute naming.
Summary
95
The IETF Media Feature Sets specify device capabilities and user preferences by unstructured
attribute/value pairs which is not appropriate. Complex capabilities and preferences are
expressed by Boolean expressions of predicates about single attributes. The Media Feature
Sets are not decomposable and there are no formal, machine readable means for extensions.
The MPEG-21 DIA tools guarantee an efficient adaptation of multimedia content but the
focus only on the MPEG-21 contents limits their application. Exploiting the full potential of
the standard requires MPEG-21 compatibility by all users of the Digital Item which may not
be always the case and also limits its openness. Since there are several formats other than
MPEG-21 in a heterogeneous environment, it is important to have an adaptation strategy for a
broader range of content and conditions.
CSCP is an extension of CC/PP. It provides a multilevel structure that allows representing all
types of context information. CSCP is based on RDF and thus inherits its interchangeability
and extensibility. Attribute names are interpreted context-sensitively according to their
position in the profile structure. Thus, unambiguous attribute naming throughout the profile is
not required. Furthermore, CSCP provides mechanisms to attach conditions and priorities to
attributes. This extends the means to express user preferences. The CSCP description has
been chosen in our work for three reasons (1) the CSCP description is simple as it is used to
describe only the context information unlike the MPEG-21 which is used to describe the
context and the content format (2) the MPEG-21 context description does not have
QoSAdapationService specification such as execution time and prices which are important
components in our work (3) the efficient use of MPEG-21 requires that all the components
participating in the content delivery should be MPEG-21 compatible in other words, not only
the context but the content description should also be in MPEG-21 and this limits openness of
the system.
Chapter
6
Service-Based Content
Adaptation
In this chapter we describe the design and implementation of our proposal of Distributed
Content Adaptation Framework. The chapter starts with an introduction in Section 6.1.
Section 6.2 presents the reasons of service-based content adaptation. Section 6.3 presents a
functional model of a service-based adaptation system. Section 6.4 describes the Distributed
Content Adaptation Framework (DCAF) architecture which is a service-based approach and
its components. Content and context metadata descriptions used by DCAF are given in
section 6.5. The multimedia content adaptation services description is presented in Section
6.6. Section 6.7 describes the Content Adaptation and Negotiation Module (CNAM), the
construction of the adaptation graph, the adaptation graph generator, and its algorithm.
Finally, the chapter ends with a summary in Section 6.8.
6.1 Introduction
As it has been discussed in Chapter 2, context aware content adaptation has been considered
as a fundamental requirement for pervasive computing to meet the dynamic heterogeneous
environment. Up to now, three important approaches have been proposed for content
adaptation namely server-based, client-based and proxy-based (Chapter 2). In this thesis we
present a service-based approach. The main idea behind this proposal is that content
adaptations are developed as services such as web service that will be easily integrated and
reused in our work. In this proposal, a service composition algorithm based on AI (Artificial
Intelligence) planning technique is developed to compose the adaptation services. For this
purpose we have developed a Distributed Content Adaptation Framework (DCAF). Before the
discussion of the concept of our service-based content adaptation, we first briefly present the
98
Chapter 6. Service-Based Content Adaptation
reasons for using services for content adaptation and the general functional model of
adaptation system.
6.2
Why Service-Based Content Adaptation?
In a server or proxy based adaptation approaches, the adaptation systems are developed for
particular or specific application. This (1) limits the usability of the adaptation tool only by
that application (2) does not allow the application to access and use other tools with the same
functionalities that provides better performance and (3) does not permit to change the code of
the tool easily in order to get better performance and better result as it is might affect the
application and (4) lacks scalability and interoperability. By using SOA, adaptation tools can
be developed as services and can be widely accessible through proper communication
interface. Since the services are developed to implement a particular adaptation, they are
efficient. New adaptation tools can be developed by using more powerful algorithm and then
implemented as services. Using the new services does not require modification of the
application system. Moreover, the SOA provides a level of flexibility in deploying and
reusing services not previously attained. Finally, the scalability and interoperability issues can
be solved as the SOA paradigm tends to provide these features. Our proposition of distributed
service-based content adaptation approach bases on these facts.
The use of SOA enables a generic application independent adaptation framework which can
be adapted to different application scenarios. As presented in Chapter 1, there are different
scenarios where we need content adaptations to deliver services to users. For example, an
adaptation framework developed for the medical application can be adapted to tourism or
cooking guide application scenarios with little modifications.
6.3
Functional Model of Adaptation System
The motivation for multimedia content adaptation is to provide with the best possible content
to a user given the heterogeneity of data and device, the limitation of resources, the
characteristics of device and the user’s preferences. A functional overview of our adaptation
system is depicted in Figure 6.1. The usage environment context and the content metadata are
Overview of the DCAF Architecture
99
used to select the adaptation service (s) required to deliver the multimedia content. The
adaptation services description is used to determine the appropriate adaptation services.
Context Metadata
Adaptation service
description
Collects
Collects
Adaptation
system
Processes
Content Metadata &
data
Delivers
Adapted data
Figure 6.1: Functional Model
•
Context Metadata: it consists of the device capabilities, user characteristics and
preferences, and network conditions.
•
Adaptation service description: it consists of the functional and non-functional service
descriptions.
•
Content metadata: it consists of the content description.
•
Content data: the multimedia content to be adapted
6.4
Overview of the DCAF Architecture
The DCAF aims to provide an extensible, scalable, flexible and interoperable adaptation
architecture for pervasive computing applications. The proposed approach is service-based,
i.e. it makes use of Internet accessible adaptation services. Adaptation services provide
adaptation functionalities or tools. An adaptation need or requirement can be broken up into
adaptation steps each fulfilling a certain function. There may be more than one service which
fulfills the same functionality. It is at the adaptation decision time that the appropriate
services are selected, invoked and executed. In our model, in addition to the functional
requirements, non-functional requirements such as service execution time and price are used
100
to determine which services would be more suitable to fulfill the required adaptation task(s).
In this section, we present the general architecture of the DCAF architecture, the different
components involved and their roles (see Figure 6.2). The architecture is composed of six
components. We describe these components and their functional relationships as follows:
Content Servers (CSs): they are standard data repositories such as web sites, databases, and
media servers.
Content Proxies (CPs): the content proxies provide access to Content Servers, formulate user
request to source format, manage and provide content description (metadata).
Client Profile Repository (CPR): stores usage environment context information that is, the
user characteristics and preferences, the terminal capabilities and characteristics. Users can
update and modify their profiles at any time. Dynamic context information such as user
location and network conditions are determined at data request execution.
Adaptation Service Registry (ASR): searching the available Web Services space to find
matching services for content adaptation is a problem that with an increasing number of
services displays exponential growth. To cope with this problem, we introduce an adaptation
service registry. The ASR is like a Universal Description, Discovery and Integration (UDDI)
registry specialized for multimedia adaptation services. In additiona to the functional and nonfunctional information of the mulltimedia adaptation services, it also stores semantic
information (e.g. the type of media the service handles) and allows for service look up.
Adaptation Service Proxies (ASPs): the adaptation service proxies are the servers that host
one or more adaptation tools or functionalities. In our framework ASPs are implemented as
Web Services and are developed independently of the DCAF.
Local Proxies (LPs): access point to information delivery systems. They are in charge of
retrieving and processing context profile. They decide the type and number of adaptation
processes, discover adaptation services, plan the execution of the adaptation services and
finally invoke the services. The core part of the local proxy is a content negotiation and
adaptation module (CNAM). This module computes an optimal execution plan of the
Context and Content Descriptions
101
adaptation process and invokes the adaptation services accordingly (see Section 6.6 for
details).
Adaptation Services
Adaptation
Services
Registry
Content Servers
Internet
User Devices
Local Proxies
Content Proxies
Context Profile
Repository
Figure 6.2: General architecture of the Distributed Content Adaptation Framework (DCAF)
6.5
As depicted in Figure 6.1, the decision of the adaptation process depends on the extent and
quality of information gathered from various sources. This information consists of usage or
context description, content description or metadata and adaptation services description. The
context description (or context profile) includes the device profile, the network profile, the
user profile and possibly other context information such as environmental information (e.g.
location, sensor information, etc.). The content description (or content metadata) contains
information about the content data or media such as title, description etc., and the media
profile such as format, size, etc. The adaptation service description (or service profile)
contains information about the service such as the name, location, function it provides, etc.
The context and content descriptions are presented in the following sections and the
adaptation services description is presented in Section 6.6.
102
6.5.1 Context Description
The context description must give a complete description of the environment and provide an
easy way of processing it. For the reasons discussed in Chapter 5, we have chosen the CSCP
to represent the context information in the DCAF architecture. The context description
(whose partial structure is shown in Figure 6.3) is made of three schemas: the user profile
schema, the device profile schema and the network profile schema. Figure 6.4 shows an
example of partial context profile for a sample device represented in XML. The full context
profile structure and its schema definition can be found in Annex A.
ContextProfile
DeviceProfile
Hardware
DeviceType
ScreenWidth
Software
OperatingSystem
Name
Version
UserAgent
Type
Name
Figure 6.3: Partial Structure of Context Profile Representation
<ContextProfile rdf:ID=”Context”>
<device>
<dev:DeviceProfile>
<dev:Hardware>
<dev:ScreenWidth>320</dev:ScreenWidth>
<dev:ScreenHeight>200</dev:ScreenHeight>
</dev:Hardware>
<dev:Software>
<dev:Name>EPOC </dev:Name>
<dev:UserAgent>
<dev:Type>Browser</dev:Type>
<dev:Name>Mozilla</dev:Name>
</dev:UserAgent>
</dev:Software>
103
</device>
</ContextProfile>
Figure 6.4 Partial Example of Context Profile
6.5.2 Content Description (Metadata)
The architecture consists of content servers and content proxies. The content servers store
media objects such as text, image, audio and video. The content proxies manage the content
metadata. Metadata contain both the information about the object(s) captured in the mediathe object’s title, description, language, etc.- and feature data about the media itself (not about
the object(s) in the media). Feature data include the media type, the file format, the file size,
the media dimensions, the color information, the location, etc. For content description the
XML form of MPEG-7 description is used. Figure 6.5 shows an example of the description
for an image data. Content metadata are generated during content creation and stored at
Content Proxies.
<ContentDescription>
<MultimediaContent type=“Image”>
<MediaInformation ID=“x_ray_image”>
<MediaUri> http:/localhost/imagefiles/gorge.jpeg </MediaUri>
<MediaTitle>Examen de la gorge</MediaTitle>
<MediaDescription> Patient atteint d’un cancer de la gorge. A ce que nous avons pu
constater, le patient ne peut s’empêcher de fumer un paquet de cigarettes par
jour</MediaDescription>
<MediaLanguage>French</MediaLanguage>
</MediaInformation>
<MediaProfile>
<MediaFormat>JPEG</MediaFormat>
<MediaSize>61.6 </MediaSize>
<MediaWidth>660</MediaWidth>
<MediaHeight>445</MediaHeight>
<MediaColor>24</MediaColor>
</MediaProfile>
</MultimediaContent>
</ContentDescription>
Figure 6.5 Sample Multimedia Content Description in XML Format
104
6.6 Multimedia Content Adaptation Services Description
Since the adaptation approach proposed in this thesis uses externally developed adaptation
services, it is important to use or develop appropriate service description mechanism(s). A
service description specifies what a service does, how the service works and how the service
is implemented. The service description enables the following tasks: discovery, i.e. locating
services that provide a particular service (typically through a registry, in our architecture the
adaptation service registry) and that adhere to specified constraints, invocation or activation
and execution of an identified service by an agent or other service, and composition of new
services through automatic selection, composition and interoperation of existing services.
As discussed in Chapter 3, a syntactic service description such as WSDL is not adequate for
automatic service discovery, selection and composition. To enhance the service discovery and
composition, the description should also include semantic information. Semantic description
gives meanings and relationships of the terms and concepts used in describing services. For
example in multimedia adaptation services, the semantic information consists of the meaning
of the input/output, effect of executing the service, cost of the service, service execution time,
etc. Semantic service description can be achieved using ontology.
As part of the thesis work, we propose to build a multimedia adaptation service ontology.
This ontology facilitates describing adaptation services semantically so that they can be
discovered, selected, composed and invoked automatically. To build the ontology, the first
step was to determine the basic multimedia adaptation services and multimedia data entities.
Then, structural hierarchies were determined from the entities. Some entity properties were
adopted from OWL-S profile [Ankol01]. Here for the sake of understanding, we present the
part of the ontology for audio adaptation services as an example. The full ontology can be
found in Annex B.
Multimedia Content Adaptation Services Description
105
6.6.1 Adaptation Service and Multimedia Data Entities
Based on audio adaptation techniques discussed in various papers [Lei01a, Shaha01,
Mohan99], we have identified the following possible audio adaptation services (Table 6.1).
Name
AudioReductionAdaptationServices
Definition
Services that reduce the bit rate of an
audio file
AudioSamplingRateReductionAdaptationServices Services that reduce the rate at which an
audio file is sampled.
StereoToMonoAdaptationServices
Services that convert an audio which has
two channels to single channel
AudioFormatTranscodingAdaptationServices
Services that change the audio file format
(e.g. WAV to MP3)
AudioLanguageTranslationAdaptationServices
Services that translate an audio in one
language to another language (e.g.
French to English).
AudioToTextAdaptationServices
Services that convert an audio file to a
text document.
Table 6.1: Types of Audio Adaptation Services
The above definitions are used to represent the functionalities of audio content adaptation
services. The services have input and output of multimedia data type. Formalizing the
multimedia data types helps to represent the input and output of the services in a standard
way. Table 6.2 shows the multimedia data type entities identified in our prototype (Chapter
7).
Name
Description
MultimediaDatatype
It represents multimedia data types such as audio,
video, image, text, etc.
ImageDatatypeFormat
It represents image data type formats.
AudioDatatypeFormat
It represents audio data type formats.
VideoDatatypeFormat
It represents video data type formats.
TextDatatypeFormat
It represents text data type formats.
GraphicsDatatypeFormat
It represents graphical data type formats.
AnimationDatatypeFormat It represents animation data type formats.
Table 6.2: Multimedia Data Types
106
6.6.2 Class Hierarchy
To determine the structural hierarchy of the adaptation service entities, we used the content
adaptation technique classification defined by Lei et al. [Lei01]. According to this definition,
the adaptation techniques are classified based on the different criteria. Here, we are interested
in audio adaptation techniques. We can classify audio adaptation techniques as unimodal or
multimodal. The first class consists of techniques used to create different versions of the
audio element with different qualities, formats and resource requirements. For example,
techniques that are used to convert an audio from one format, e.g. WMA to MP3. The second
class consists of techniques that are used to convert audio to another media type, for example
audio to text conversion.
The different adaptation services are represented as classes in the ontology. These classes can
be arranged in hierarchy that is useful for service discovery. This hierarchy, shown in Figure
6.6, is based on the broad classifications described above. For the sake of simplicity, only the
audio adaptation services are presented in detail the figure. The hierarchical classification
together with the service properties (see section 6.6.3) gives a complete ontology of the
adaptation services.
107
f
su
bC
l as
sO
ImageAdaptationServices
O
la
ss
su
bC
f
O
f
las
s
f
su
bC
O
f
Of
las
s
su
bC
f
Services
ss
la
bC
su
Services
O
ss
la
bC
su
subClassOf
la
ss
O
TextAdaptationServices
O
su
bC
Of
Services
Services
ss
ss
la
bC
su
f
Services
Cla
f
VideoAdaptationServices
su
b
O
ss
la
bC
su
bC
su
f
sO
las
AdaptationServices
Figure 6.6: Class Hierarchy of Top-level of Multimedia Content Adaptation Services
The super class in the ontology is MultimediaAdaptationServices. This class can have different
sub-classes of adaptation services that are specific to each media element like audio
adaptation services, image adaptation services, etc. Figure 6.7 shows the OWL representation
of the super class (MultimediaAdaptationServices).
<owl:Class rdf: ID= “MultimediaAdaptationServices”>
<owl:Class rdf:ID= “AudioAdaptationServices”>
</owl:Class>
<owl:Class rdf:ID= “TextAdaptationServices”>
</owl:Class>
<owl:Class rdf:ID= “ImageAdaptationServices”>
</owl:Class>
<owl:Class rdf:ID= “VideoAdaptationServices”>
108
</owl:Class>
Figure 6.7: OWL Representation of MultimediaAdaptationServices Class
Here, we present as an example the audio adaptation services. Therefore, the description of
the sub-classes and properties are restricted to audio adaptation services. Figure 6.8 shows
specifically the class hierarchy of the audio adaptation services.
bC
su
su
bC
f
O
ss
la
la
ss
O
f
bC
su
subClassOf
f
O
ss
la
Services
bC
su
ss
la
su
bC
Services
f
O
Services
la
ss
O
f
Services
su
bC
su
bC
la
ss
O
f
la
ss
O
f
Services
AdaptationServices
Figure 6.8: Excerpt of Class Hierarchy of Audio Adaptation Service Classes
The AudioAdaptationServices class inherits all properties of MultimediaAdaptationServices, but it
restricts the value of the input property, hasInputType to be Audio. This is because, all audio
adaptation services take an audio file as their input. This class has two sub-classes:
MultiModalAudioAdaptationServices.
and
The
UnimodalAudioAdaptationServices has an additional restriction on its output data type property
other than the restrictions inherited on its input data type. The instance of this property is
restricted
to
be
Audio.
AudioReductionAdaptationServices,
This
class
has
three
subclasses
AudioLanguageAdaptationServices
under
it:
and
AudioFormatAdaptationServices. The AudioReductionAdaptationServices has two sub-classes:
109
StereoToMonoAdaptationServices and AudioSamplingRateReductionAdaptationServices. The other
sub-class of AudioAdaptationServices is the MultimodalAudioAdaptationServices which has one
sub-class, AudioTextAdaptationServices.
The
Figure
6.9
shows
a
small
excerpt
of
the
OWL
representation
of
the
AudioAdaptationServices class. The full description of the ontology can be found in Annex B.
<rdfs:subClassOf>
<owl:Restriction>
<owl:onProperty>
</owl:onProperty>
</owl:Restriction>
</rdfs:subClassOf>
</owl:Class>
Figure 6.9: A Small Excerpt of OWL Representation for AudioAdaptationServices Class
6.6.3 Properties of Entities
For audio adaptation services we have identified the following properties, most of which are
similar to those of the OWL-S profile class (See Annex C). These properties are shown in
Table 6.3. These properties are general to all multimedia adaptation services so to make it
extensible to other media elements easier we have put them as properties of the general class
MultimediaAdaptationServices.
110
Property name
Value
Definition
hasServiceName
String
Name of the service
hasServiceTime
Float
Service execution time
hasServiceQuality
String
Quality value for the adapted data
hasEffect
Instance of
Effect of service execution e.g.
MultimediaDataTypeFormat language change or format change
class
hasInputType
String
Name of the input data type
hasOutputType
String
Name of the output data type
hasServicePrice
Float
Service charge
hasPrecondition
Instance of
Conditions to be satisfied to execute
MultimediaDataTypeFormat the service for example, format of an
class
image data, frame rate of a video data,
etc.
hasServiceProvider
String
Name of the service provider that may
include address and other related
information
hasServiceDescription String
Any description about the service
Table 6.3: Properties of MultimediaAdaptationServices
All types of multimedia data type formats have similar properties. Therefore, we put them as
properties of the general class that is MultimediaDatatypeFormat. Table 6.4 shows the properties
with their description.
111
MultimediaDatatypeFormat
Property name
Value
Definition
hasDatatype
String
Name of the media type
hasFormat
String
Name of the format type
Table 6.4: Properties of MultimediaDatatypeFormat Class
In Figure 6.10 we present an example of a description of audio to text adaptation service
using the described ontology.
<AtomicProcess rdf:ID="AudioToTextAdapationService ">
<hasInput rdf:ID="#InputAudio">
</hasInput>
<hasOutput rdf:ID="#OutputText">
</hasOutput>
<hasPrecondition>
<hasExpression rdf:ID="InputAudioFormat">
<hasSubject rdf:resource="#InputAudio"/>
<hasProperty rdf:resource="#Audio"/>
<hasFormat rdf:datatype="&xsd; #string">#WAV</hasFormat>
</hasExpression>
</hasPrecondition>
<hasEffect>
<hasExpression rdf:ID="OutputTextFormat">
<hasSubject rdf:ID="#OutputText"/>
<hasProperty rdf:resource="#Text"/>
<hasFormat rdf:datatype="&xsd;#string">#PlainText</hasFormat>
</hasExpression>
</hasEffect>
<hasServicePrice>
</hasServicePrice>
<hasServiceTime>
</hasServiceTime>
</AtomicProcess>
Figure 6.10: An Example of Service Description for Audio to Text Adaptation Service
112
6.7
Content Negotiation and Adaptation Module
The building block of the local proxy is the Content Negotiation and Adaptation Module
(CNAM). CNAM has two main components: the adaptation decision engine and the
adaptation planner. The decision engine computes the needed adaptation types to run using
the environments parameters: user profile, device profile, network profile, and content
metadata. Once the adaptation types are known, the next step is to determine the adaptation
services to execute them. The adaptation planner is concerned with the selection and the
composition of appropriate adaptation services. The selection of the adaptation services is
based on quality criteria such as service cost and execution time. The adaptation planner finds
these quality criteria from the service description (hasServicePrice, hasServiceTime). Figure
6.11 illustrates the process of content adaptation.
Client
ListDoc
Client
SelDoc
Req
Document filtering
Selected Doc
Doc.Req
ListDoc
Content Proxy
eq
C. R
s
e
C.R
Content Extractor
Content & metadata
Client Profile
Adaptation Decision
Transformation List
Adaptation Planner
Adaptation
Service Profile
Plan of Execution
Proxy Service 1
Plan Executor
Adapted Content
Client
Req
DocReq
ListDoc
SelDoc
C.Req
C.Res
Request
Document Request
List of documents
Selected document
Content Request
Content Response
Figure 6.11: Content Adaptation Process
Proxy Service 2
Proxy Service N
113
The above figure shows the different steps during the process of content negotiation and
adaptation. The boxes in continues line (e.g. Document Filtering) represent processes. The
boxes in broken line (e.g. Transformation List) represent output and input of the processes.
6.7.1 Constructing Content Adaptation Graph
In a typical pervasive application scenario, a client requests a multimedia content from a
server using a device with specific properties connected to the server through a given
network. In such scenario, the idea of content adaptation is that the multimedia content stored
at the server in exactly one defined format is transformed according to the device capabilities,
the network connections and the user preferences before it is delivered to the client.
Considering the usage context and the multimedia metadata, it may be necessary to apply a
set of transformations28 so that the content matches the usage environment. For example, the
three scenarios presented in Chapter 1 (Section 1.4), demonstrates the need of several
transformations or multi-step adaptations to deliver the response content based on the usage
context, characteristics of the stored multimedia contents, and preferences of the user as
discussed below.
In Scenario I, assuming that the content is written in English language and the user’s
preference is French language we need text to audio and English to French transformations.
In Scenario II, assuming that the map is stored in colored GIF format and the device supports
non color BMP images and in addition, the dimension of the map is greater than the device’s
screen display, we need at least three transformations to display the map on the user’s device.
The needed transformations are: removing the color (changing to black and white), changing
the dimension and changing the format.
In Scenario III, the doctor might prefer to listen rather than read the response on the past
medical history of the patient. In this case, it may be necessary to summarize, translate and
convert the text to audio.
28
In this work the term transformation and adaptation are equivalent
114
Transformation mapping rules
As indicated in Figure 6.11, one of the components in the content adaptation process is the
adaptation decision. To facilitate and speed up the computation of the needed transformations
we defined a constraint to the transformation mapping rules. These rules are used to
determine what transformations are required to meet the environment constraints such as the
device capabilities, user preferences and network conditions etc. A description file specifies
each transformation process along with a set of constraints. Table 6.5 shows a sample of these
transformation rules. The transformation rules are interpreted as follows: For example, to
apply image-resizing, the multimedia content type must be image, the device hardware must
be image capable and the image size must be greater than the device size.
For the purpose of simplicity and interoperability XML format is used to describe the rules.
The XML representation of the transformation rules contains Boolean expressions for
aggregating constraints; and, or, and not. It has also a number of conditionals: lessthan,
lessthanequal, greaterthan, greaterthanequal, equal, and contains.
Transformation
text-languagetranslation
text-to-audio
image-resizing
image-formatconversion
video-to-imageconversion
video-to-text
audio-to-text
115
Context constraint
(MultimediaContent.Type="Text"
^ MultimediaContent.MediaInformation.MediaLanguage
∉UserProfile.Preferences.Languages)
^ ((DeviceProfile.Hardware.TextCapable="Yes" ^
"Text" ∈ DeviceProfile.Software.UserAgent.ContentTypes)
v (DeviceProfile.Hardware.AudioCapable="Yes" ^
"Audio" ∈ DeviceProfile.Software.UserAgent.ContentTypes))
(MultimediaContent.Type="Text"
^ DeviceProfile.Hardware.AudioCapable="Yes"
^ "Audio" ∈ DeviceProfile.Software.UserAgent.ContentTypes)
^ ("Audio" ∈ UserProfile.Preferences.ContentTypes
v DeviceProfile.Hardware.TextCapable="No")
(MultimediaContent.Type="Image"
^ DeviceProfile.Hardware.ImageCapable="Yes"
^ "Image" ∈ DeviceProfile.Software.UserAgent.ContentTypes)
^ ((MultimediaContent.MediaProfile.MediaWidth
> DeviceProfile.Hardware.ScreenWidth)
v (MultimediaContent.MediaProfile.MediaHeight
>DeviceProfile.Hardware.ScreenHeight))
(MultimediaContent.Type="Image"
^ "Image" ∈ DeviceProfile.Software.UserAgent.ContentTypes
^ MultimediaContent.MediaProfile.MediaFormat ∉
DeviceProfile.Software.UserAgent.ContentFormats)
(MultimediaContent.Type="Video"
^ "Image" ∈ DeviceProfile.Software.UserAgent.ContentTypes)
^ (DeviceProfile.Hardware.VideoCapable="No"
v "Image" ∈ UserProfile.Preferences.ContentTypes
v (NetworkProfile.Bandwidth<=UserProfile.QoSNetwork.Medium.L2 ^
NetworkProfile.Bandwidth>=UserProfile.QoSNetwork.Medium.L1))
(MultimediaContent.Type="Video"
^ DeviceProfile.Hardware.TextCapable="Yes"
^ "Text" ∈ DeviceProfile.Software.UserAgent.ContentTypes)
^ (DeviceProfile.Hardware.VideoCapable="No"
v DeviceProfile.Hardware.ImageCapable="No"
v "Text" ∈ UserProfile.Preferences.ContentTypes
v (NetworkProfile.Bandwidth<=UserProfile.QoSNetwork.Low.L2 ^
NetworkProfile.Bandwidth>=UserProfile.QoSNetwork.Low.L1))
(MultimediaContent.Type="Audio"
^ DeviceProfile.Hardware.TextCapable="Yes"
^ "Text" ∈ DeviceProfile.Software.UserAgent.ContentTypes)
^ (DeviceProfile.Hardware.AudioCapable="No"
v "Text" ∈ UserProfile.Preferences.ContentTypes
v (NetworkProfile.Bandwidth<=UserProfile.QoSNetwork.Low.L2 ^
NetworkProfile.Bandwidth>=UserProfile.QoSNetwork.Low.L1))
Table 6.5: Sample Transformation Rules
116
Usually transformations that apply transmoding should come at the end, that is, after all the
other transformations are applied on the original data. For example in Scenario III where
summarization, translation and text-to-audio are required, the text-to-audio transformation
should be done at the end while summarization and translation can be done in any order. Of
course applying translation after summarization is more efficient than the other way. In this
case summarization, translation and text to audio, is the more efficient ordering. To capture
such ordering information so that to produce efficient transformation ordering, we defined
two tags: before and after. Optionally each transformation may have before and after tags to
specify its order with respect to other transformations. Taking the same example, Scenario III,
for the translation, the after tag contains summarization. Figure 6.12 gives an example of
transformation rules represented in XML format.
<transformations>
<transformation name="text-language-translation">
<constraint>
<and>
<and>
<equal value="Text">MultimediaContent.Type</equal>
<not>
<contains
ref="MultimediaContent.MediaInformation.MediaLanguage">UserProfile.Preferences.Languages</contains>
</not>
</and>
<or>
<and>
<equal value="Yes">DeviceProfile.Hardware.TextCapable</equal>
<contains value="Text ">DeviceProfile.Software.UserAgent.ContentTypes</contains>
</and>
<and>
<equal value="Yes">DeviceProfile.Hardware.AudioCapable</equal>
<contains value="Audio">DeviceProfile.Software.UserAgent.ContentTypes</contains>
</and>
</or>
</and>
</constraint>
<after>
<transformation>text-summary</transformation>
</after>
</transformation>
<transformation name="text-to-audio">
<constraint>
<and>
<equal value="text">MultimediaContent.Type</equal>
<equal value="Yes">DeviceProfile.Hardware.AudioCapable</equal>
<contains value="audio">DeviceProfile.Software.UserAgent.ContentTypes</contains>
<or>
<contains
value="audio">UserProfile.Preferences.ContentTypes</contains>
<equal
value="No">DeviceProfile.Hardware.TextCapable</equal>
</or>
</and>
</constraint>
<after>
117
<transformation>text-summary</transformation>
<transformation>text-language-translation</transformation>
</after>
</transformation>
<transformation name="image-resizing">
<constraint>
<and>
<equal value="image">MultimediaContent.Type</equal>
<equal value="Yes">DeviceProfile.Hardware.ImageCapable</equal>
<contains value="image">DeviceProfile.Software.UserAgent.ContentTypes</contains>
<or>
<lessthan ref="MultimediaContent.MediaProfile.MediaWidth">DeviceProfile.Hardware.ScreenWidth</lessthan>
<lessthan ref="MultimediaContent.MediaProfile.MediaHeight">DeviceProfile.Hardware.ScreenHeight</lessthan>
</or>
</and>
</constraint>
<before>
<transformation>image-format-conversion</transformation>
<transformation>image-color-reduction</transformation>
</before>
</transformation>
</transformations>
Figure 6.12: Sample Transformation Rules in XML Format
The decision engine produces the transformations needed using the context profile, the
multimedia metadata and the transformation rules. The engine processes the mapping rules as
follows. It parses the file and constructs a post fix description of each set of constraints. It
stores this postfix description in a vector for later evaluation. For example, the text-to-audio
transformation is represented in Table 6.6.
expression
Variables
value
equal
MultimediaContent.Type
"Text"
equal
DeviceProfile.Hardware.AudioCapable
"Yes"
contains
DeviceProfile.Software.UserAgent.ContentTypes "Audio"
contains
UserProfile.Preferences.ContentTypes
"Audio"
equal
DeviceProfile.Hardware.TextCapable
"No"
or
and
Table 6.6: Constraint Processing
The decision engine evaluates the postfix description of a set of constraints by retrieving each
expression, evaluating it and then writing the result back to a result stack. In the case of textto-audio transformation it examines audio capability of the device, user’s content type
preference, content types supported by the device and writes true or false accordingly. The
118
engine repeats this process for each transformation and returns a vector containing the names
of transformations necessary to meet the context constraints. As a result of the process, the
engine returns a file (in XML format) that consists of a list of transformations. We call this
file transformation prescript. The transformations are ordered using the information in before
and after tags of the description file, if there exist, otherwise randomly. For the scenario III
described in Chapter 1, the transformation prescript is given in Figure 6.13. We will use this
example in the following to illustrate the adaptation process before formalizing it in Section
6.7.2.
<transformationPrescription>
<source>
<MediaContent type="text" identifier="http://localhost/text-data /text-test.txt"
format="text/plain" language="EN"/>
</source>
<transformation name="text-summary">
<input type="text"/>
<output type="text"/>
</transformation>
<transformation name="text-language-translation" language="“FR”">
<output type="text"/>
</transformation>
<transformation name="text-to-audio">
<output type="audio"/>
</transformation>
</transformationPrescription>
Figure 6.13: An XML Encoding of a Transformation Prescript
The transformation prescript provides the list of transformations needed to deliver the data in
the current context. Figure 6.14 is the graphical representation of the transformation prescript
shown in Figure 6.13. It does not specify the actual service executing the transformation. To
generate an adaptation execution plan, we construct graph of services by replacing each
transformation with the appropriate adaptation service(s).
Summary
Text data
(in English)
A: Start
Where
Data (text)
Language (English)
A
Translation
t1
t1: text-summary
Input: text
Output: text
119
Text to audio
t2
t2: text-languagetranslation
Input: text in English
Output: text in French
t3
t3: text-to-audio
Input: text
Output: audio
Z
Audio data
(in French)
Z: End
Where
Data (Audio)
Language (French)
Figure 6.14: A Graphical Representation for the Transformation Prescript
We can see from Figure 6.11, the transformation prescript and the adaptation services
description are inputs to the planner. Considering the transformation prescript given in Figure
6.13 and 6.14, T= {t1, t2, t3}, where t1, t2, and t3 are the transformation processes or operators,
then we construct the graph as follows:
Step 1:
First, we associate the transformation operators with the available services that can implement
them. This gives us GS, a group of services for each transformation.
GS= {<t1, s11, s12,>, <t2, s21, s22, s23, s33>, <t3, s31, s32>}, where t1 (text summary), t2 (language
translation) and t3 (text to audio) can be realized by the adaptation services {s11, s12}, {s21, s22,
s23} and {s31, s32, s33} respectively.
Step 2:
Then, we connect the services according to the transformation prescript.
We assume that each adaptation service accepts input data with a quality value (Qin) and
generates output data with a quality value (Qout). The quality parameters may include the data
format, the resolution etc. To connect two services, we evaluate the quality compatibility, that
is, the output quality value of one service should match with the input quality value of the
120
next service. We find the quality values in the precondition and effect tags of the service
description. Figure 6.15 shows the graph where Fi represents service input/output format.
A
F1
F1
s11
F2
s12
F3
s21
F5
F4
s22
s23
F6
F7
s31
F8
s32
s33
F10
F9
F11
Z
F12
Figure 6.15: An example of Adaptation Graph for Text to Audio Transformation
Step 3:
By looking at the graph in Figure 6.15, we can see that the two services s21 and s33 are
connected only to one service. This is because the output (F5) and the input (F8) formats of
services s21 and s33 respectively do not match with the input and output formats of the next and
predecessor services in the graph. It is necessary to find one or more so called “neutral”
operators so that s21 and s33 will be connected to one of {s31, s32, s33} and {s21, s22, s23}
respectively or alternatively remove them from the graph. Neutral operators are in charge of
transforming the output format of a service into a format compatible with the formats
accepted as input by another service. Removing the unconnected services helps to reduce the
computational time of the algorithm to find the best path. But by doing this, we might exclude
an interesting path through s21 or s33 in terms of global execution time from A to Z.
121
Assuming that no neutral operator (s) is/are found to connect s33 we remove s33 from the
graph. On the other hand, suppose that the output of s21 is Ms-Word data and the input of s31 is
rtf data. Let us say there exist two services with neutral operators sn1 (Ms-Word to pdf) and
sn2 (pdf to rtf). We can thus connect service s21 to s31 through a chain of the two services (sn1 > sn2). By doing this we add the path {A, s11, s21, sn1, sn2 s31, Z} that can be competitive with
other paths in the graph. The full adaptation graph is depicted in Figure 6.16.
A
F1
F1
s11
F2
s12
F3
F4
s21
s22
F5
F4
s23
F7
F6
sn1
sn2
s31
s32
F9
F10
Z
F12
Figure 6.16: A Fully-Connected Adaptation Graph
Defintion: We say a graph is fully connected when (1) all the services in the graph are
instantiable (2) no input/output quality value mismatches and (3) all paths contain at least the
start node A and the end node Z.
Once the graph is constructed quality criteria such as service execution and price are used to
find an optimal path in the graph. Detailed discussion of the QoS model and the optimal path
selection are presented in Section 6.7.2.3.
122
6.7.2 Multimedia Adaptation Graph Generator (MAGG)
As discussed in Chapter 3, there are two approaches for service composition: workflow
technique and AI planning. In this thesis the AI planning approach is used. We developed an
algorithm called Multimedia Adaptation Graph Generator. This algorithm constructs an
adaptation graph. In the following section we present definitions and notations used in the
algorithm.
6.7.2.1
Definitions and Notations Used in MAGG
Definition 1: Media object
A media object is a multimedia data which can be a text, an image, an audio or a video
represented as
M (m1, m2, …,mn)
where m1, m2, …,mn are media features or metadata
Definition 2: State
The state S of a media object M say S (M) is described by the metadata values
Example: for an image object the state holds the values for the format, the color, the height,
the width, etc.
e.g. S(M) =(bmp, 24 bits, 245 pixels, 300 pixels).
Definition 3: Adaptation task
An adaptation task is an expression of the form t (α1 α2 … α n) where t is a transformation and
α1, α2, … α n, are parameters
For example, ImageFormatConversion(imageIn, imageOut, oldFormat, newFormat), where
imageIn: image input file
imageOut: image output file
oldFormat: old file format
123
newFormat: new file format
Definition 4: Adaptation service
An adaptation service is a service described in terms of inputs, outputs, preconditions and
effects. An adaptation service is represented as s= (R I O Pre Eff Q) where
R: an atomic process that Realizes an adaptation task
I: input parameters of the process
O: output parameters of the process
Pre: preconditions of the process
Eff: effects of the process
Q: quality criteria of the service
Definition 5: Operator (plan operator)
A plan operator is an expression of the form o= (h (a1, a2, …an, b1, b2, …bm), Pre Eff Q) where
h: an adaptation task realized by an adaptation service with input parameters a1, a2, …an and
output parameters b1, b2, …bm. h is called the head or identification of the operator.
Pre: represents the operator’s preconditions.
Eff: represents the effect of executing the operator.
Q= {q1, q2,…,qn}: represents quality attributes (e.g. cost, time, etc.)
Let S be a state, t be an adaptation task and M a media object. Suppose that there is an
operator o with head h that realizes t such that Pre of o is satisfied in S. Then we say that o is
applicable to t and the new state is given by:
M o = S ( M ) = Executing (o, M , S )
Example: for the above given adaptation task, we can have an adaptation operator instance as
follows:
Operator: ImageFormatConversionOperator (http://media-adaptation/imagefiles/image1,
http:// media-adaptation/imagefiles/image2, mpeg, bmp)
Input: http://media-adaptation/imagefiles/image1
Output: http:// media-adaptation/imagefiles/image2
124
Precondition: hasFormat (http://media-adaptation/imagefiles/image1, mpeg)
Effect: hasFormat (http:// media-adaptation/imagefiles/image2, bmp)
Quality: (cost=30 units, time=100 ms)
Note:
An adaptation task t can be realized by several adaptation operators hence Q will be used to
compute an optimal plan or path.
Definition 6: Adaptation graph
An adaptation graph G (V, E) is a Directed Acyclic Graph (DAG) where
-
V is the set of nodes that represent the adaptation operators
-
E is the set of edges that represent the possible connections between the adaptation
operators.
The start node A ∈ V is a pseudo operator with effect (initial state) but no precondition
The end node Z ∈ V is a pseudo operator with precondition (goal state) but no effect
Remark
Let oi ∈ V and oj ∈ V, a link or an edge eij exist from oi to oj if the following condition is
satisfied:
oj.Pre ⊆ oi.Eff
Where
-
oj.Pre denotes preconditions of oj
-
oi.Eff denotes effects of oi
Definition 7: Adaptation planning problem
An adaptation planning problem is a four-tuple (SA, SZ, T, D) where SA is the initial state of the
media object, SZ is the goal state of the media object, T is an adaptation task list and D is the
adaptation operators. The result is a graph G= (V, E).
6.7.2.2
125
Algorithm to Construct Adaptation Graph
To construct the adaptation graph, first we need to translate the service description to
adaptation operator representation. For this purpose, we used an XML parser that accepts the
service description file and then creates an object for the adaptation operator. The algorithm
that generates the adaptation graph is represented in Figure 6.17.
Algorithm: graph (SA, SZ, T, D)
Input: Initial state SA, final state SZ, adaptation task list T and adaptation operators D
Output: an adaptation graph G (V, E)
// Global constant
// Limit
maximum number of neutral operators allowed in a connection
// Global variables
// V
a set of nodes in a graph
a set of edges in a graph
// E
start node
// ao
end node
// zo
a set of neutral operators available in the system
// NO
// Local variables
// T
a set of adaptation tasks
an adaptation task element of T
// t
a set of nodes for adaptation operators realizing an adaptation task
// O
a set of parent nodes
// PO
a set containing the end node
// Pz
var
V, E, T, t, O, PO, ao, zo, Pz, NO
begin
ao= ConstructStartNode(SA) // constructs the start node from the initial state
zo= ConstructEndNode(SZ) // constructs the end node from the goal state
NO= ConstructNeutralOperators( ) // returns the list of the neutral operators available in
//the system
V= {ao} //initialization
E= Ø
//initialization
PO= {ao}//initialization
for each t ∈ T
begin
Construct nodes O from D with operators realizing t
//several operators can realize a task
Connect(O, PO)
//after the process PO holds the value of O
end //T is processed
Pz={zo}
Connect(Pz, PO) // connects the end node
return G (V, E)
end // graph
126
----------------------------------------------------------------------------------------------------------------// procedure Connect: connects child nodes (O) with parent nodes (PO)
// O a set of nodes-input variable
// PO a set of nodes-input and output variable
----------------------------------------------------------------------------------------------------------------procedure Connect(in O, inout PO)
//Local variables
// CO a set of nodes-temporal variable used to store child nodes (O)
// TPO a set of nodes-temporal variable used to store parent nodes (PO)
// CPO a set of nodes-temporal variable used to store connected parent nodes
// po a node in PO
// o a node in O
// UNO a set of operators-variable to store already used neutral operators
// Link a Boolean variable to check if a node has a connection
// Connected a Boolean variable, true if a node is connected by neutral operator
// LimitC an integer- number of neutral operators in a connection
//Notations
// po.Pre preconditions of node po
// po.Eff effects of node po
// o.Eff effects of node o
var
CO, TPO, CPO, UNO, o, po, LimitC, Link, Connected
begin
CO= Ø //initialization
CPO= Ø //initialization
Link = false
// connect each o ∈ O with each node of the parent nodes PO
for each o ∈ O
begin
for each po ∈ PO
begin
if (o.Pre ⊆ po. Eff) then // direct connection of node o with node po
begin
if(o∉ V) then
begin
V= V ∪ {o}
end
E= E ∪ {(po, o)}
Link = true // o is connected in the graph
end
else // for indirect connection of node o with po using neutral operators
begin
LimitC=0
UNO= Ø
Connected=ConnectWithNeutralOperators (o, po, LimitC, UNO)
Link=Link or Connected // Link is true if o has at least one connection
127
end
// if po is connected add it to the connected parent nodes list
if (Link) then
CPO=CPO ∪ {po}
end // PO is processed
if (not Link) then // checks if o has at least one direct or indirect connection
begin
if(o=zo) then // it is the end node so the graph construction fails
begin
V= Ø
E= Ø
return
end
else
O=O-{o}// o can not be connected so remove it
end
end // O is processed
// remove parent nodes which have no connection
for each po ∈ PO
begin
if (po ∉ CPO ) then
begin
V=V-{po}
for each (x, po) ∈ E
begin
E=E-{(x, po)}
Remove(x) // removes x with its ancestors
end
end
end
PO= O //child nodes become parent nodes for the next process
end // end connect procedure
----------------------------------------------------------------------------------------------------------------// function ConnectWithNeutralOperators it connects o and po using neutral operators
//recursively
// o a node- input variable
// po a node- input variable
//LimitC an integer- input variable
// UNO a set of nodes- input and output variable
----------------------------------------------------------------------------------------------------------------function Boolean ConnectWithNeutralOperators (in o, in po, in LimitC, inout UNO)
//Local variables
// TNO a set of operators-a temporal variable for neutral operators
// TE a set of edges-a temporal variable to store connectable edges
// TV a set of nodes-a temporal variable to store connectable nodes
// no a node
var
TNO, TE, TV, no
128
begin
TNO=NeutralOperators(o, UNO)// it returns neutral operators connectable with o
for each no ∈ TNO
begin
if (no.Pre ⊆ po.Eff) then
begin
TE= TE ∪ {(po, no)}
TE= TE ∪ {(no, o)}
TV= TV ∪ {no}
TV= TV ∪ {o}
E= E ∪ TE
V= V ∪ TV
return true
end
else
begin
TE= TE ∪ {(no, o)}
TV= TV ∪ {no}
TV= TV ∪ {o}
if (LimitC+ 1=Limit) or (UNO ∪ {no}=NO) then //terminating condition
return false
else
return ConnectWithNeutralOperators(no, po, LimitC+1, UNO ∪ {no})
end
end // TNO is processed
end // ConnectWithNeutralOperators
----------------------------------------------------------------------------------------------------------------// function NeutralOperators returns the set of neutral operators satisfying the preconditions
//of o
// o a node-input variable
// UNO a set of nodes- input variable
----------------------------------------------------------------------------------------------------------------function Operators NeutralOperators(in o,in UNO)
//Local variables
// TNO a set of operators-temporal variable for neutral operators
// no a node
//Notations
// no.Eff effects of node no
var
TNO, no
begin
TNO= Ø
for each no ∈ NO
begin
if (o.Pre ⊆ no.Eff) then
if (no ∉ UNO) then
TNO=TNO ∪ {no}
end
129
return TNO
end // NeutralOperators
----------------------------------------------------------------------------------------------------------------// procedure Remove it removes the node x recursively until it finds a node with a branch
//connection or it reaches the start node
// x a node- input variable
----------------------------------------------------------------------------------------------------------------procedure Remove(in x)
//Local variables
// z a node for which (z, x) is an edge in E
// y a node for which (x, y) is an edge in E
var
z, y
begin
if (x=ao) then // it is the start node make V empty and return
begin
V= Ø
return
end
else if ((y ∈ V) and ((x, y) ∈ E)) then // x has a branch connection
begin
return
end
else
begin
for each (z, x) ∈ E
begin
E=E-(z, x)
Remove(z)
end
V=V-{x}
end // each (z, x) is processed
end //Remove
Figure 6.17: Algorithm for Adaptation Graph Construction (AGC)
130
6.7.3 Searching an Optimal Adaptation Path/Plan
Definition: Adaptation plan/path
An adaptation plan/path is a path in the adaptation graph G that connects the start node to the
end node. It is represented as a list of the form p= (A o1 o2 … on Z)
Where
A and Z are the start and the end nodes and oi is an adaptation operator instance.
If p= (A o1 o2 … on Z) is a plan and S is a state, then S(p) is the state of the media object
produced by executing o1, o2, …, on in the given order on the input media object. Q(p) is the
aggregate quality value of the path computed according to equation (6).
QoS model
Since an adaptation task can be achieved by more than one service29, choosing an appropriate
service is an obvious requirement. Once the adaptation graph that consists of all possible
compositions is generated, then the choice of the best alternative i.e. the optimal path in the
graph is done based on user specified QoS criteria. The QoS model used contains two
features: the response time of the service and its cost (charge). This model can be extended to
include other features such as reliability, availability and security. The response time is
composed of the execution time and the transmission time. The execution time represents the
necessary time to execute a service. The transmission time corresponds to the communication
of data between two services. Since the services are provided by third parties there is a cost
associated with each service. This cost consists of service execution charge and transmission
charge.
Let p= (A s1 s2 … sN Z) be a path in an adaptation graph where N is the number of services.
We define the QoS of the path, denoted as Q(p), as follows:
Q( p) = (QCOST ( p), QTIME ( p))
Where
29
The services are represented by operators in the graph
(1)
131
QCOST ( p ) : the total cost of the path
QTIME ( p ) : the total execution time of the path
N
QCOST ( p ) is defined as QCOST ( p) = ∑ QCOST ( si )
(2)
i =1
Where
QCOST ( si ) : the adaptation service execution cost and the data transmission cost
N
QTIME ( p ) is defined as QTIME ( p) = ∑ QTIME ( si )
(3)
i =1
Where
QTIME ( si ) : the adaptation service execution time and the data transmission time
To aggregate the quality values we define scaled qualities QsCOST ( s i ) and QsTIME ( s i ) as
follows:
max
⎧ QCOST
− QCOST ( si )
⎪
max
min
QsCOST ( s i ) = ⎨ QCOST − QCOST
⎪
1
⎩
max
min
if QCOST − QCOST ≠ 0
min
Q max − QCOST
=0
if COST
max
⎧ QTIME
− QTIMe ( s i )
max
min
⎪
if QTIME − QTIME ≠ 0
max
min
QsTIME ( si ) = ⎨ QTIMe
− QTIME
min
Q max − QTIME
=0
⎪
1
if TIME
⎩
(4)
(5)
Where
max
min
QCOST
and QCOST
are the maximum and the minimum cost values respectively and
max
min
QTIME
and QTIME
are the maximum and the minimum time values respectively
Note: the maximum and the minimum values are calculated from the services currently
available.
132
Users can give their preference on QoS (i.e by balancing the impact of the different criteria)
such as fastest or cheapest by specifying weight values for each criteria. The score of a path
with weighted values is calculated as follows:
N
Score( p ) = ∑ (QsCOST ( si ) ∗ wCOST + QsTIME ( si ) ∗ wTIME )
(6)
i =1
Where wCOST ∈ [0, 1] and wTIME ∈ [0, 1] represent the weight value assigned to the cost and
the weight value assigned to the time respectively and
wCOST + wTIME = 1
(7)
Let Pset= {p1, p2, …, pK} be the set of all possible paths in an adaptation graph, then the
optimal path is the path with the maximum score value scoremax where scoremax is defined as
follows:
Scoremax = max {Score( pi )}
iε {1, K }
(8)
To find the optimal path one possible solution is to perform an exhaustive search that is,
computing all possible adaptation paths and selecting the optimal path. Obviously, such an
approach is of combinatorial complexity (in dependency of the number of adaptation
operators n in the adaptation graph) and only feasible for a small number of n.
The Dijkstra’s algorithm [Dijkstra59] can be used to find the optimal path. We can express
the running time of Dijkstra's algorithm on a graph with E edges and V nodes as a function of
|E| and |V| using the Big-O notation. The running time for the simplest implementation of the
Dijkstra’s algorithm is O (|V|2). With a binary heap, the algorithm requires O ((|E|+|V|)log
(|V|)) time, and the Fibonacci heap improves this to O (|E| + |V|log(|V|). The algorithm can be
made more efficient in the special case when the directed graph is acyclic. This can be
achieved by modifying the algorithm to include a step for sorting the graph topologically.
Since the adaptation graph is a DAG, the modified algorithm was used. The algorithm with a
topologically sorted graph has a running time of O (|E|+|V|). The algorithm can be found in
Annex D.
133
6.7.4 Performance Evaluation
6.7.4.1
Adaptation services execution time
In this section we present the experiments we made to measure the execution time of image
adaptation services. The goal of the test was to examine feasibility of the adaptation services.
This was done by measuring the execution time required by the service with respect to the
size of the images. It was also to observe the impact of ordering the transformation services
on the total execution time. The adaptation services have been implanted in the DCAF
prototype. The adaptation services were published on double 3.00 GHZ Pentium 4 with 2.00
GB RAM running Microsoft Windows Server 2003 standard edition service pack 1. For the
purpose of testing we have used JPEG images with 570*760 pixels. To minimize the effect of
varying system load during the adaptation execution we made repeated experiments and took
the average execution time.
Our experiment demonstrated that the execution time of the adaptation services increases with
the size of the images which is logical. One interesting observation of the experiment was that
the format conversion and the image resizing services took less time compared to the color
reduction services. Moreover, we observed that varying order of the adaptation services
produced different execution times. Putting the color reduction services at the end produced
better result compared to the other orderings. In the legend R is Resizing, F is Format
conversion and C is Color reduction. The best ordering was Resizing-Format conversionColor reduction (RFC). The experimental result is shown in Figure 6.18.
134
Execution Time (in 100 ms)
Comparison of Transformation Execution Order
50
RFC
40
CFR
30
FRC
20
RCF
CRF
10
FCR
0
32
42
43,3 43,9 45,7 49,4 52,6 56,7 74,4
Image Size(Kb)(570*760 pixels)
Figure 6.18: Comparison of Different Execution Orders
6.7.4.2
Graph construction time
Definition 1: Link Ratio
A link ratio ri of a node ni is defined as the number of links divided by the number of child
nodes.
Remark
-link ratio of the end node is zero
-link ratio of the start node is 1
For Figure 6.20, the link ratio values are:
3
2
1
1
r1 = , r2 = , r3 = , r4 = , etc.
3
4
4
4
135
A
n1
n4
n2
n3
n5
n6
n8
n7
n9
Z
Figure 6.20 Example of an adaptation graph
Definition 2: Average Link Ratio (ALR)
Average link ratio ALR of a graph G is defined as follows:
k
ALR =
∑r
i
i =1
k
with ri ≠ 0 .
, Where ri is the link ratio of node ni and k is the number of nodes in the graph
For Figure 6.20, the average link ratio is
10
ALR =
∑r
i =1
10
i
=
1
2
The first analysis of the algorithm shows that there is no significant difference in the graph
construction time for the different average link ratios. This is due to the fact that the algorithm
performs the checking condition for each child and parent node whether there is a link or not.
Here, we present the experimental result on the performance of the algorithm with respect to
the depth of the graph and the number of services per transformation. Our aim in this
experiment was to see the progress of the graph construction time with respect to the graph
depth and the number of services per transformation. As cited by Libse [Libse04] and our
136
experience with different adaptation scenarios (see Table 2.1) show that in practice the path
length is not more than ten levels. Therefore, we did the experiment with maximum graph
depth of 10 levels. We varied the number of services per transformation from ten to thirty.
The experiment was performed on a 1.9 GHZ Pentium 4 with 256 MB RAM running
Microsoft Windows 2000.
Time(ms)
Graph Construction Time
360
340
320
300
280
260
240
220
200
180
160
140
120
100
80
60
40
20
0
10 Services per
transformation
20 Services per
transformation
30 Services per
transformation
2
3
4
5
6
7
8
9
10
Graph Depth (# of Transformations)
Figure 6.19: Graph Construction Time
As can be seen in the Figure 6.19, the graph construction time increases linearly with the
graph depth (# transformations in the path). It also shows that the relationship between graph
construction and the number of services per transformation is linear, as the gap between each
curve remains constant. Compare to the graph depth, the construction time progress slowly
with number of services per transformation. Of course, it was observed that the progress both
for the depth and width (number of services per transformation) was almost constant with
average increase of 40 ms when we increased the depth by one and average increase of 10 ms
when we increased the width by 10 services. This implies that having several services per
transformation does not affect much on the total construction time, while it provides the
possibility to select the best service among the candidates. For example, for 300 (depth of 10
and width of 30) services in a graph which are large enough to realize an adaptation need, the
total construction time is only 335 ms which is really acceptable.
Summary
137
Optimal path search algorithm
Our measurement on the performance of the optimal path search algorithm showed that the
average link ratio has not significant impact on the execution time. For a graph with depth of
10 and 10 services per transformation, the algorithm’s execution time was 18 ms.
6.8 Summary
By enabling people to access information and services independent of time and location
pervasive computing promises to facilitate work, play, and every day activities. The pervasive
computing environment is characterized, however, by devices with widely varying
capabilities (e.g., display size, network connectivity), and a variety of data formats and
supported services. Given this extreme diversity, providing effortless access to data and
services anywhere and anytime requires the seamless customization of content and
applications to the user's context. Context-aware content adaptation is a key concept to meet
the varying requirements of different clients in a dynamic heterogeneous environment, such
as pervasive computing. On one hand, content adaptations are resource intensive processes
and the tools required to implement them are diversified. On the other hand, the emergence of
web services brought the advantages of publishing Internet accessible functionalities which
includes content adaptation tools. In this chapter we proposed several concepts for a novel
content adaptation approach-the distributed service-based framework of multimedia content
adaptation for pervasive computing applications. We particularly:
•
described the architectural framework and its components
•
defined content and context metadata description mechanisms
•
developed an ontology for multimedia content adaptation services description
•
described the process and components of a content adaptation and negotiation module
•
described the construction of the content adaptation graph
•
described a Qos model for content adaptation services
•
presented a formalism for multimedia adaptation graph generator and its algorithm
•
presented a performance evaluation of some of the adaptation services and the MAGG
algorithm to demonstrate the feasibility of the approach
Chapter
7
Prototype Implementation
Based on the concepts mentioned in the last few chapters, we have implemented a prototype
of content adaptation system as a proof-of-concept. The system demonstrates the idea of
multimedia content adaptation by selecting Internet accessible content adaptation service(s)
that provide(s) the desired functionality satisfying the execution context. This chapter
describes a prototype implementation of our framework and reports our initial experience.
Section 7.1 describes major functionalities of the prototype system, the components of the
local proxy and their description. Sample databases used in the prototype are presented in
Section 7.2. Discussion on the visual interface of the prototype is given in Section 7.3. Finally
the chapter ends with a short summary in Section 7.4.
7.1
Components of the Prototype
We developed a prototype implementation of the DCAF to show the feasibility, validity and
practicability of our proposals. The prototype was built using Borland Jbuilder 9 personal. For
the different databases developed in the prototype we used MySQL-4.0.12. The software
package consists of more than 10,000 lines of code. Our prototype system is based on the
architecture described in Chapter 6 (Section 6.3). The architectural framework consists of
local Content Servers, Content Proxies, Client Profile Repository, Adaptation Service
Registry, Adaptation Service Proxies, and Local Proxies. Figure 7.1 represents an overview of
a DCAF session which is described as follows:
Step 1
Client login information: the client enters his login information
Step 2
Client profile request: profile request from the local proxy to the client profile repository
140
Chapter 7. Prototype Implementation
Step 3
Client profile response: profile response from the client profile repository
Step 4
User interface: the local proxy provides a user interface for the user to enter his content
request
Step 5
User request: content request from the user
Step 6
Local proxy request: the request can not be served from the cache and the local proxy
contacts the content proxy
Step 7
Content proxy response: metadata response from the content proxy
Step 8
Local proxy response: the response from the cache/the content proxy is sent back to the user
Step 9
User request for selected data: the user selects from the list and sends to the local proxy
Step 10 Local proxy request for selected data: the selected data request is sent to the content proxy
Step 11 Content request: the content proxy contacts the content server
Step 12 Content server response: the content server sends the content to local proxy
Step 13 Content response: the response from the content server is sent back to the local proxy.
Step 14 Adaptation service description request: a description request from the local proxy is sent to
the adaptation service registry
Step 15 Adaptation service registry response: description response from the registry to the local
proxy
Step 16 Adaptation request: adaptation request sent from local proxy to the adaptation service proxy
Step 17 Adaptation service proxy response: the adapted content from the adaptation service proxy is
sent to the local proxy
Step 18 Local proxy response: the adapted content received from the adaptation service proxy
finally sent to the user by the local proxy
Note: While the adaptation service registry response (Step 15) may be cached to avoid the next request
(Step 14) so that to improve performance of the system, it would be useful to execute Step 14 each
time to get the newly added services in the registry.
141
2
3
Client Profile
1
Local Proxy
Content Proxy
10
11
12
Content Server
Adaptation
Service Registery
*
16
*
7
1
15
13
14
User Agent
6
7
4
5
8
9
18
Adaptation
Service Proxy
* this communication can occur several times , if several
adaptations are required
Figure 7.1: Components and Interaction of DCAF Architecture
Our prototype offers the following major functionalities:
Adaptation decision engine: The aim of our system is to have the intelligence of making
suitable decisions on the adaptation services that are needed for adapting a content so that this
latter matches the execution environment constraints. The decision engine uses a planning
algorithm to find suitable adaptation services to construct the adaptation graph and finally
produce an optimal adaptation path from the graph. It uses the framework described in
Chapter 6.
Proxies modules: three categories of proxies were developed as parts of the prototype
implementation. One group represents content proxies which were developed to manage
distributed content servers and their content metadata. MySQL was used for storing the
content and metadata databases. The second group represents local proxies that are the core
parts of the architecture. The adaptation decision engine and other important functionalities
are parts of these proxies.
The third group represents proxies that provide specific content adaptation functionalities.
These were implemented as Web service to show the feasibility of providing complex
adaptations using third party adaptation services. C# and ASP.NET were used to develop the
adaptation services. The following adaptation services were used/developed in the prototype:
142
•
A language translation service which translates a text written in one language to
another language (e.g. French to English). A widely used language translator
(BabelFish)30 was used to perform the actual translation. No specific coding was
needed in using the service. We only developed the equivalent description of the
service. This shows the framework is easy to extend.
•
A Text-to-audio transcoding service which transcodes a text document to audio
format.
•
An Image resizing service which changes the size of an image
•
A Color to greyscale conversion service
•
An Image format conversion service which converts the format of an image to png,
gif, etc.
Adaptation Service registries: for adaptation service discovery, registries were used to store
the information about available adaptation services. The registries are specialized in the sense
that they store only adaptation services. Service providers publish their service description in
the registries and local proxies consult the registries when matching for particular adaptation
service(s) is required. For a given adaptation service, the identifier of the service and its
description are stored. The description contains both functional and non-functional properties
(see Table 6.3 in Chapter 6).
Client profile repositories: they are used to store client profile. The client profile consists of
user’s preferences, device characteristics and capabilities and network connection. While in
the propotype, the client profile is created during user subscription and it can be modified any
time, in real application, the system captures the device profile from device vendors databases
using the device’s identification and the network profile is captured using network
monitoring. MySQL database was used to hold user login information and its associated
profile.
The block diagram in Figure 7.2 shows the components of the local proxy and the interaction
and information flow between them. It has five major components: a Content Negotiation and
Adaptation Module, a Profile Agent, an Adaptation Service Agent, a Communication Module,
30
Babelfish: http://www.babelfish.org/
143
and an Interface Module. In the following section, we describe the components and their
roles:
•
The Interface Module is used to communicate with the user. There are four interfaces
defined in this module: user login interface, user request input interface, result display
interface and user profile entry/modification interface. The user login interface is used
to enter user id and password information to access the system. The user request
interface is used to enter user content requests. The user profile entry/modification
interface is used to enter/modify the client profile. And finally the result display
interface is used to display the adapted content.
•
The Communication Module is used to communicate with content proxies. There are
two types of data exchanged between the local proxy and content proxies: metadata
and content. The content proxy first sends local proxy request to content server and
then sends back the response (metadata) to the local proxy. The user selects a
particular data from the response and sends to the local proxy. The local proxy sends
the selected data request to the content proxy. The content proxy requests the content
server for the data and sends back to the local proxy. We used web services and SOAP
as a communication protocol.
•
The Profile Agent is used to store and get client profile to/from the profile repository.
SOAP is used as communication protocol between the Local Proxy and the profile
repository. Though in our prototype, one client profile repository is used, in real
applications, there may be several repositories for example, vendors’ device profile
repositories. Vendors may use different protocols and this creates the problem of
interoperability. We use SOAP as a solution to this problem.
•
The Adaptation Service Agent has two components: the adaptation service discovery
module and the adaptation service invocation module. The Adaptation Service
Discovery is used to find all the possible adaptation services from the adaptation
service registry. The Adaptation Service Invocation is used to invoke remote
services according to the optimal adaptation plan. SOAP is used as a communication
144
protocol for both Adaptation Service Discovery and Adaptation Service Invocation
modules.
•
The Content Negotiation and Adaptation Module is the central part of the local
proxy. It has two main components: the Adaptation Decision and the Adaptation
Planner. The Adaptation Decision is responsible to analyse and decide the number
and types of transformations required to send the content providing the constraints and
characteristics of the context and the content. It finally creates a transformation
prescript as described in Chapter 6 (see Figure 6.13). The Adaptation Planner uses
the output of the Adaptation Decision as input (the transformation prescript),
constructs the adaptation graph and finally generates an optimal adaptation plan. The
optimal plan is communicated to the Adaptation Service Invocation.
Figure 7.2: Block Diagram of the Local Proxy
Several classes and data structures have been developed to implement the different
components of the Local Proxy (as Shown in Figure 7.3). Two important data structures have
been created: the OperatorLayer and LinkLayer. The OperatorLayer stores the operators
145
corresponding to the transformations or the adaptation tasks. The LinkLayer stores the links
of the operators. The adaptation graph discussed in Chapter was implemented using these data
structures. While the OperatorLayer was used for the nodes of the graph, the LinkLayer
was used for the edges of the graph. Detailed description of the classes with their purpose,
instance variables and methods can be found in Annex E.
Classes
MediaObject
PlanElement
PlanHandler
MediaState
ContextProfile
OWLHandler
Predicate
ProfileHandler
Operator
MediaHandler
MappingHandler
Vector*
AdaptationTasks
OperatorsList
OperatorLayer
LinkLayer
OpInput
OpOutput
OpPrecondition
OpEffect
Plan
OptimalPlan
Plans
Figure 7.3: Classes of Local Proxy Implementation.
* Vector is a built-in Java class used for representing list of values
146
7.2
Sample Database Used in the Prototype
Four major databases were developed in the prototype: profile database, adaptation service
registry, content and metadata. The profile database is used to store user related information
such as the user name, identification, password and a link to the context profile (CSCP) file.
The adaptation service registry database is used to store adaptation service related information
such as the identification and a link to the service description file (OWL-S). The content
database was used to store the application data or content to be provided to the user. Finally
the metadata database was used to store content search information and a link to the MPEG-7
file. All the databases used in the prototype were developed with MYSQL-4.0.12.
Our proposal for distributed service-based multimedia content adaptation is very generic so
that it can be used for any multimedia content delivery application domain in heterogeneous
environments such as pervasive systems. However, for the purpose of testing our concepts
and proposals we developed a sample database for medical application. Our choice for the
medical application comes from the fact that these are multimedia data and we have already
sample data because some members of the laboratory work on them (e.g. Flory06, Vedier06).
Our sample database stores medical records of patients. The conceptual data model is shown
in Figure 7.4. The following tables describe the fields of the various tables.
Hospital
Field
Description
HCode
Unique identification code of the hospital
Name
Name of the hospital
Address
Full address of the hospital
Section
List of services (departments) of the hospital
Sample Database Used in the Prototype
PatientData
Field
Description
PatientID
Unique identification of the patient
FirstName
First name
LastName
Last name
DateOfBirth
Birth date
Address
Full address
MedicalRecord
Field
Description
MedRecId
Unique identification for the record
PatientID
Patient identification
DocId
Document identification
Title
Title of the document
Description
Description of document content
Keywords
Key words used to search the document
MedicalDoc
Field
Description
MedDocId
Unique identification of the document
DocId
Document identification
MediaObject
Field
Description
MediaId
Unique identification of the media object
MedDocId
Unique identification of the document
MediaType
Type of media
Title
Title for the content of the media
Description
Description for the content of the media
Language
Language of the media content
MetaDataLoc
URL address of the metadata location
147
148
MedicalRecord
Hospital
PK
PK
HospCode
Name
Address
Section
PatientData
MedRecId
PK
1:1
N:1
PatientID
DocID
Title
Description
Keywords
HospCode
N:1
1 :1
PatientID
FirstName
LastName
DataOfBirth
Address
1:1
MediaObject
PK
1:1
MedicalDoc
PK
1:1
MedDocId
DocId
N:1
MediaID
MedDocId
MediaType
Title
Description
Language
MetaDataLoc
Figure 7.4: Conceptual data model.
7.3
Visual Interfaces
We developed various visual interfaces for the communication between the system and the
user. These interfaces include a user login interface, a profile entry/modification interface, a
user request entry interface and an adaptation data display interface. Here we present the
profile entry/modification and the adapted data display interfaces since they are most
important ones.
7.3.1 Client Profile Entry Interface
The client profile entry interface is composed of three panels: Device Type selection panel,
Network Specification panel and User Preferences entry panel. The organization of the entry
interface is based on the client profile representation structure (See Annex A). The interface
enables to capture client profile information. Each of these panels is described below.
Device Type Selection Panel
The device type selection panel allows selecting the type of device the user wants to use to
access the data from an existing list for which the profile already exists (See Figure 7.5).
Visual Interfaces
149
Figure 7.5: Screenshot of the Device Selection Interface
The Device Characteristics Entry Panel
The device characteristics entry panel allows entering characteristics of the device the user
uses to access data if the profile of this device has not yet described. Using this panel the user
enters the hardware and software characteristics of the device (See Figure 7.6).
Figure 7.6: Screenshot of the New Device Entry Interface
150
Network characteristics Entry Panel
The network specification panel allows the specification of the network connection the user is
connected to (See Figure 7.7). Using this panel the user enters the network type, bandwidth
and delay of the network connection.
Figure 7.7: Screenshot of the Network Characteristics Entry Interface
User Preferences Panel
The user preferences entry panel allows specifying user preference information so that data
will be sent to accordingly (See Figure 7.8). Using this panel the user provides his language
preference and the QoS value in terms of time and cost.
Figure 7.8: Screenshot of User Preferences Entry Interface
Visual Interfaces
151
7.3.2 Patient Record Search Interface
The patient record search interface permits patient record search using key words (See Figure
7.9). The user enters patient identification and key words related to a type of disease, a
treatment or a medical examination. The result of the search provides the list of associated
documents for that patient. Then the user selects the interest document for detail information.
Figure 7.9: Screenshot of Patient Record Search Interface
7.3.3 Adapted Data Display Interface
In pervasive systems, adaptation consists of at least two things: the data/content and the user
interface. Our work deals with the adaptation of content. The user interface adaptation and
management in pervasive environment by itself is another important research domain that
needs further investigation which is not investigated in this thesis. Since presenting
information to users requires both content and interface adaptation, we collaborated with
another research group in the laboratory. This group works on an interface adaptation project
called SEFAGI (Simple Environment for Adaptable Graphical Interfaces).
The SEFAGI project [Chaari05] aims to design and build a platform that allows using the
same application services with different terminal types. The system provides an environment
that permits the description of expected user interfaces using an abstract window description
language, the description of target platforms using a platform description language and the
automatic generation of user interface code based on the previous descriptions.
152
The SEFAGI system developed an interface server that provides tools (a) to create a new
Abstract Windows Descriptions (AWD) (b) modify or reuse abstract windows descriptions
and (c) to generate the code corresponding to the abstract windows descriptions for each kind
of terminal. The system provides a graphical interface to help making the first two activities
(a and b) and a code generator.
Figure 7.10: Graphical Assistant: AWD graphical representation
The XML AWD which is the output of the graphical interface serves as an input for the code
generator. The graphical interface offers (See Figure 7.10) an interactive design process and
hides the language syntax. This avoids the writing of the abstract description by the users. To
design a window, the end user chooses in one click a group of graphical components (a
standard SEFAGI panels) in a library. Then, he links the component to a service by
navigating in the service repository. In SEFAGI the services are implemented to manage and
display different information parts, for example, in a medical application, the patients list, the
medical records (textual information), the medical records (images), etc. are considered as
different services. The platform descriptions are provided by computer scientists. Such
descriptions are made using the XML grammar provided by the system. Generated interfaces
Visual Interfaces
153
are deployed on terminals as dedicated runtime environment. Each type of platform requires a
different runtime environment code, as it depends on the available platform API. Platforms
descriptions are used to build automatically the code of the runtime environments.
Our collaboration enabled us to have a complete adaptation system that provides medical
information adapted to the user context. The system provides adapted information both for the
content and the interface. Since the SEFAGI system is still a project in development, we
generated only interfaces for standard PC and smart phone for the purpose of testing (see
Figure 7.11 (a) and Figure 7.11 (b)). We are in the process of writing an article on the result
of this collaboration work to be sent for a publication in a journal. We also hope to continue
working with the group in the future. This will finally enable us to have a fully complete
adaptation system that allows medical information access with any type of terminal.
For the case of scenario III (Chapter 1), where a medical doctor requests the medical history
of a patient on his PDA or Smartphone and the system has medical records written in a
language for example French but the doctor’s language preference is English, the system uses
a language translation service and returns the adapted data (in English). Figure 7.12 (a) and
Figure 7.12 (b) are screenshots of the data in French and English respectively.
Figure 7.11 (a): Screenshot of an Image data on a Standard PC (without adaptation)
154
Figure 7.11 (b): Screenshot of an Image data on a Smartphone (after adaptation)
Figure 7.12 Screenshot of text data (a) original in French (b) the translated data in English
Summary
155
7.4 Summary
In this chapter, we presented a prototype that implements the DCAF framework to evaluate
the validity and practicability of our proposals. We first described the different components of
the architecture and their interaction. We then presented the major functionalities of the
prototype and their implementations. We also presented the implementation of the local proxy
and its sample class hierarchy. While our proposal for distributed service-based multimedia
content adaptation is very generic so that it can be used for any multimedia content delivery
application domain, a sample database for medical application was developed for the purpose
of testing our concepts. Two computer science students at the laboratory (Yann Gripay and
Christope Litzinger) were involved in the development of the prototype and the integration of
the SEFAGI system to the prototype. Our framework is based on Internet accessible
adaptation services. We connected our prototype to language translation services and
multimedia adaptation services which were implemented by our means.
The different visual interfaces developed in the prototype facilitate the communication
between the system and the user. The visual interfaces include Client Profile Entry, Patient
Record Search, and Adapted Data Display interfaces. The client profile entry interface is used
to capture client profile information. The patient record search interface is used to search
patient records using patient identification and keywords. The medical data adapted by the
system is sent to the SEFAGI interface server [Chaari05] which provides an adapted data
display interface so that the data will be presented according to the terminal characteristics.
Chapter
8
Discussion
In this thesis, we proposed a novel distributed service based content adaptation system for
pervasive environments. The important issues addressed in this thesis are: the design of
distributed adaptation architectural framework, the development of an ontology for semantic
content adaptation service description, and the development of an algorithm for content
adaptation service composition. Moreover, we developed a prototype that demonstrates the
practicality of our proposals and concepts. The purpose of this chapter is to briefly present the
advantages and limitations of the framework, the service description, the composition
algorithm and the prototype. We also try to give ideas how these limitations can be solved.
8.1
DCAF Architecture
The type of content adaptation to be performed depends on the context information gathered
and the characteristics of the content to be adapted. In this thesis we proposed an architectural
framework to deliver adapted content in pervasive environments. The architecture takes into
consideration the characteristics of the client device, the user’s preferences, the client
network, the content description (also called content metadata), and the properties of available
adaptation services to determine the number and type of adaptations to be performed on the
content. The principal strengths of our architecture are extensibility, scalability, flexibility and
interoperability.
Extensibility: three fundamental design principles of our architecture aid its extensibility.
First, the DCAF architecture uses externally developed adaptation services. Adaptation
services needed to process new data types and/or satisifiying new client requirements can be
158
Chapter 8. Discussion
easily added to the adaptation service registry just by publishing their description. New
adaptation services implementing more effective algorithms can be added easily to the
system. Second, the structure of the architecture is modular. Therefore, adding or removing
components to the system is made easy, for example, device profile databases provided by
device vendors can be added to the system and used by the system to retrieve device
characteristics. New content servers can be added to the system by creating the corresponding
metadata databases containing the content search information and link of the MPEG-7 files.
Third, our design utilizes XML as a standard means of information exchange between
different components of the architecture and different stages of the adaptation process. The
XML technology facilitates the extensibility. For example, the transformation rules are
encoded in XML that enables to add and modify rules to incorporate new client constraints.
Finally, the system can use several adaptation service registries. New content servers can be
added to the system by just creating the corresponding metadata databases containing the
content search information and link of the MPEG-7 files. Third, our design utilizes XML as a
standard means of information exchange between different components of the architecture
and different stages of the adaptation process. The XML technology facilitates the
extensibility. For example, the transformation rules are encoded in XML that enables to add
and modify the rules to incorporate new client constraints. Finally, the system can use several
adaptation service registries.
Scalability: as discussed in Chapter 2, in the proxy-based approach, the proxy is responsible
for adaptation decision and execution of the adaptation functions. However, such an approach
is not scalable and impedes performance under several conditions. Externalizing the
adaptation process minimizes the load of the local proxy so the local proxy can handle several
requests and several clients. Moreover, adapted contents can be cached so that requests for the
same content with the same contextual situation can be served from the cache.
Interoperability: one of the bases for architectural interoperability is to implement a standard
way of communication between the components of the architecture. Web services technology
is used to implement the different components of our architecture (e.g. Client profile
repository, adaptation service registry, etc.) and SOAP as communication protocol. The third
party adaptation services are developed with web services (e.g. [Babelfish]). Web services
have a mature set of protocols and technologies that are widely accepted and used. They are
platform, system and language independent. Hence, they provide interoperability.
Semantic Adaptation Services Description
159
Flexibility: our architecture is flexible in two ways. First, it is generic and content independent
so it can be adapted easily to different content delivery application environments. For
example, the architecture can be used for patient record access in medical application as
demonstrated in our application or it can be used to access recipes from a cooking guide
information system. Second, it permits components of the architecture to evolve easily with
changing requirements. The different components of the architecture are developed
independently and loosely- coupled, hence they can be updated without affecting the other
components. For example, the separation of content adaptation and presentation (user
interface) adaptation enabled to easily integrate the SEFAGI’s [Chaari05] adaptable graphical
interface to the prototype.
Since the architecture relays on third party adaptation services, its application may be limited
as many of the adaptation services have not yet been developed. For example, in developing
our prototype we found only the language translator services and we implemented other
services by ourselves. However, the increase popularity of Web Service and the need of an
open and interoperable content adaptation system will open the venue for developing more
external adaptation services.
8.2
Semantic Adaptation Services Description
Fully automatic adaptation decision process and adaptation service composition require a
formal and semantic description of the adaptation services. The significant advantages of
semantic descriptions is that they enable to identify the appropriate adaptation services for the
needed adaptation functionalities, facilitate the use of planning techniques for adaptation
service composition and finally invoking the selected adaptation services. However the
commonly used service description such as WSDL lacks the semantic information which
makes automatic service composition difficult.
To solve these problems, we developed semantic adaptation services description ontology.
This ontology defines the different multimedia data types and their associated adaptation
techniques. Moreover, it formalizes the input/output, precondition and effects of the
adaptation services. This formalization enables to automatically verify the compatibility of
160
two adaptation services for composition. It increases the robustness of the system by
providing global view of the adaptation functionalities of the system. This is also good for
maintenance. The ontology provides specification for developers for implementing adaptation
services at the same time wrappers can be easily developed for other service descriptions.
Classifying the adaptation services into different category based on the media type simplifies
the difficulty in describing services by the service developers.
Our semantic service description is not complete (e.g. no description for video adaptation
services). However, the extensibility of the ontology enables addition of new adaptation
techniques and new data types easily in the future. While the system can not use services
described in other form, an automatic wrapper can be developed to transform other
descriptions into the system’s semantic description platform.
8.3
Adaptation Service Composition
Our service composition algorithm implements automatic service composition by using
planning methodology. We first developed a formal representation of multimedia adaptation
service composition. This formalism enabled us to define the algorithm as a planning problem
with four-tuple (SA, SZ, T, D) input where SA is the initial state of the media object, SZ is the
goal state of the media object, T is an adaptation task list and D is the adaptation operators.
We developed a parser that translates the service description to operator representation (D)
which is the input for the algorithm. The algorithm produces an adaptation graph with all
possible adaptation paths as a result of the composition. The experiments conducted to see the
performance of the algorithm demonstrated that the algorithm performs well even when there
are several transformations and several services per transformation. For example, for a graph
with the depth level (no of transformations) of 10 and width (services per transformation) of
30 (a total of 300 services) which is enough to realize any type of adaptation scenario, the
construction time was only 335 ms which is acceptable. One of the strong features of the
algorithm is that its execution time progress very slowly with the depth and width of the
graph. For example, for each increases of 10 services it only has 10 ms additional time.
The algorithm generates a DAG and the Dijkstar’s algorithm [Dijkstra59] with a topological
sort step is used to find the optimal adaptation path in the graph. This algorithm is based on
Adaptation Service Composition
161
QoS criteria to find the optimal path based on the user’s preference on service time and
charge. The QoS criteria are indication of user’s interest on the time and/or the charge. The
user indicates his/her interest by giving more weight on the specific criteria. For example, if
the user gives 0.7 for time and 0.3 for charge 0.3, it means that the user is interested in the
response time of the service than the cost of the service.
The MAGG algorithm considers only compatibility of the input format and the output format
of the adaptation services in the composition. But in reality there can be other parameters that
should be compatible such as the size of the input/output data, the frame rate of the
input/output data, etc. therefore; we need a composition algorithm that considers several
compatibility parameters in addition to the format of the input/output data. Moreover, the
algorithm does not consider different ordering possibilities of the transformations. In the
prototype implementation one ordering is used. While all ordering might not be efficient (e.g.
applying text language translation before text summary is less efficient than the other way
order) or not possible, some ordering constraints may be added during the time of registry of
the adaptation services. The ordering constraints of a transformation can be provided by the
administrator who is responsible in registering the services or the service provider.
One of the problems with the optimal path search algorithm is that it assumes that the services
available during composition will be available during the execution. But a service which was
active during the composition might not be available later on when the graph was generated.
In this case the path execution fails because of this service. The algorithm does not consider
such situation and leads to system failure. Since the composition algorithm generates all
possible paths, the problem can be fixed by taking another optimal path or replacing the
service by another service with similar function and cost. This recovery mechanism is not
implemented in the algorithm.
The path cost which consists of the service execution time and the service charge is assumed
to be static. But in the case where the cost varies with network condition or other parameters,
the optimal path might not be the same. The algorithm does not consider such issues. One
possible solution could be to include some possible variation value in calculating the path cost
or check the cost after the optimal path is found to verify if the cost is the same. Such
contingency mechanism is not implemented in the algorithm.
162
To select the optimal path only service time and charge are considered. However, other
parameters such as reliability, security, etc can also be essential from the user’s point of view.
In such case we need to develop a metrics to combing them with the existing ones. One
possible solution can be to store some history information of the services with respect to these
QoS. This historic information will finally enables us to represent these parameters
numerically so that they can be combined with the time and the charge to calculate overall
QoS of a service.
It was not implemented in the prototype but the construction time can be minimized using
caches. Since the same path might be used for other request, the path can be cached for further
use. If there is a request with similar contextual settings (e.g. context profile and content
metadata), then the path can be applied without the need of constructing the graph.
8.4
Prototype Implementation
In developing the prototype, we used a modular approach. Each components of the system are
developed independently and communicate through proper interface. The modularity of the
system enables to add or evolve the components without difficulty and without affecting other
components of the system. Its modularity helps us to integrate easily adaptable user interface
systems such as SEFAGI [Chaari05].
The modularity of the system also facilitates to use the prototype in other application
scenarios but it requires creating the metadata databases, the content servers, and modifying
the search and the presentation interfaces. The SEFAGI system provides the necessary tools
to generate automatically the search and the presentation interfaces. Therefore, by using
SEFAGI we can create the interfaces for the new application.
To make the prototype fully functional, the ontology is required to be completed and we need
more externally available adaptation services. It will also be necessary to include interface for
other terminals other than PC and smart phone. This hopefully will be achieved by future
collaboration with the SEFAGI project.
Chapter
9
Conclusion and Future
Work
9.1 Conclusion and Contributions
Due to recent communication and technological advances such as wireless and mobile
devices, computing has moved towards pervasive or ubiquitous environment enabling
anywhere and anytime data access in distributed multimedia systems. However, resource
limitation of mobile device and limited capacity of wireless networks have created many
issues that have not been adequately addressed in traditional distributed multimedia systems.
One of these issues is content delivery to applications and users with varying terminal and
network characteristics, preferences and requirements. Delivering content in such
heterogeneous and dynamic environments requires understanding the different elements
involved such as the user, the terminal, the network and send data accordingly. In order to
solve the heterogeneity of the client devices used in pervasive computing, content should be
adapted to the clients such that it would be usable. Content adaptation is, therefore, of great
significance in pervasive computing environments.
Content adaptation attracted much attention in the pervasive community and several
approaches have been proposed by research groups which can be generally classified as
client-based, server-based and proxy-based. Client based content adaptation is sometimes
impossible due to the resource requirement of the adaptation processing on one hand and the
inherent resource limitation of client devices on the other hand. Server-based content
adaptation is good in terms of producing acceptable adapted content as it is close to the
content and has knowledge about the content but the adaptation process results in an overload
of the server which in turn diminishes its performance. Proxy-based is the most employed
164
Chapter 9. Conclusion and Future Work
approach for content adaptation because it relives the server load and its implementation
simplicity. However, putting all adaptation operations for which some of them may be
computationally expensive requires a proxy with powerful CPUs and a lot of memory. To
summarize, all the three approaches do not provide a complete and general content adaptation
solution that can be usable in a wide distributed heterogeneous environment.
In this thesis, we presented a service-based distributed content adaptation framework called
DCAF for adapting multimedia content in pervasive computing environments. The
framework addressed the problem of content adaptation in a complete and more general way.
We have also showed the validity of the concepts proposed by implementing and evaluating a
simple prototype for content adaptation.
The major contributions of this thesis are the following:
•
We introduced a new service-based content adaptation architectural framework for
distributed multimedia systems. Using this architecture third-party adaptation services
can be easily included into the architecture what makes the framework widely
extensible. Moreover, the framework is scalable and interoperable so that we can use
it for wide application areas.
•
Web Service technologies are considered to be good solutions for distributed
computing especially on the Internet. Their application for pervasive computing,
particularly for content adaptation has been investigated in this thesis.
•
We provided a formal definition and description of multimedia content adaptation
process. This formalism enables to understand the different elements involved in the
adaptation process so that different mechanisms can be developed (e.g. new
mechanisms can be added to the transformation rules). We have also defined
transformation rules that are easily extensible to add new transformation
functionalities and user constraints.
•
The design of the framework allows the composition of adaptation services to provide
an optimal service for a given computational context. Therefore, any adaptation need
is achievable.
•
We developed an ontology for multimedia adaptation services description. The
ontology helps to describe services semantically. The semantic description facilitates
Future Work
165
the discovery, the composition and the execution of the adaptation services
automatically.
•
An algorithm was developed for the optimization of the adaptation service
composition.
•
A prototype was developed to implement the framework. Various experiments were
conducted on the prototype. The prototype demonstrated not only the validity of our
proposal but our solution could be used for real application systems.
9.2 Future Work
Some of the future works include:
•
In this thesis separate representation mechanisms are used to describe context (e.g.
user, device and network), content, and adaptation services. However, the
interoperability of the system could be enhanced by building a unified and
comprehensive ontology that can be used to describe each of these components. This
unified ontology will have upper and lower ontologies. The upper ontology captures
the general knowledge about the physical world in pervasive computing environments.
The lower ontologies capture information about each component (e.g. the adaptation
services).
•
While CSCP overcomes the shortcomings of other representations (e.g. CC/PP), it
can’t
represent
complex
relationships
and
constraints.
Moreover,
the
component/attribute model becomes clumsy if there are many layers. On the other
hand, future’s full fledged pervasive systems will require much more sophisticated
context models in order to support seamless adaptation to changes in the
computational environment. The context models will need to specify a range of
characteristics/quality of context information including temporal characteristics as
well as various types of dependencies among the different context information. One
possible solution for context modelling could be an ontology based modeling, but
more investigation is required in this direction. D. Ejigu, a PhD student in the group is
currently working on context modelling for pervasive systems [Chaari06]. Therefore,
one of our future works will be to integrate his work in our prototype.
166
•
Chapter 9. Conclusion and Future Work
In the current implementation the MAGG algorithm considers the format of the input
and the output data for service compatibility. But other parameters for example, the
size of data and/or frame rate (for video) are also important. Therefore, the algorithm
should use all the parameter in determining the compatibility of the services. This can
be implemented by modifying the part of the algorithm that performs the precondition
and effect matching. Instead of checking only one parameter, it will check all the
parameters.
•
Pervasive computing is about distributed computing devices in the physical world,
such as personal devices, wearable computers, devices embedded in everyday objects
and sensors in the environment - it is about both the devices and the infrastructures
needed to support pervasive computing applications. Grid computing is about large
scale distributed computation, large scale data and large scale collaborations - applied
to solving large scale problems [Foster99]. Sometimes the Grid application demands
the pervasive computing, for example, in the grid-enabled devices in a laboratory-the
pieces of scientific equipment connected directly to the gird-and handheld devices
with which users gather information and access results [Roure03]. On the other hand,
in pervasive computing where we have arrays of sensors we acquire data with higher
temporal or spatial resolution, and this increasing bulk of (often real-time) data
demands the computational power of the Grid. At an appropriate layer of abstraction
both Pervasive and Grid are about large scale distributed processing elements. By
combining the two aspects together, that is, Pervasive Grid [Pierson05], we can bring
Grid computing to the physical world. Pervasive Grid is a service-oriented architecture
which involves research challenges including: service description, discovery and
composition, and security, authentication and trust. Our proposition of service-based
content adaptation maps well into this realm. In this context, we envisage that,
multimedia adaptation services can be deployed as Grid services31 for delivering
adapted multimedia data in pervasive grid environments. Hence, one of our future
works will be investigating the deployment of our framework into Pervasive Grid
systems.
The proxy system that has been implemented is first prototype to demonstrate content
adaptation in pervasive computing environments and there are still rooms for improvements.
31
A Grid service is a Web Service that conforms to a set of conventions (interfaces and behaviours) that define how a client interacts with
the Grid service [Geldof04]
167
We have identified the following list of limitations and future works concerning the
prototype:
•
Our decision engine does not utilize adaptation hints in deciding the adaptation types.
For example, a content provider can specify in the content metadata some allowed
adaptations on the data. The adaptation hints facilitate informed decision. Therefore,
we will investigate the incorporation of the adaptation hints.
•
Our prototype implements a simple cache strategy for adapted content. However, it
would be important to use caching strategy that guarantees high performance. Since
caching was not used for context profile, each time a user request a content, the local
proxy demands the profile from the repository. However, using cache for context
profile enables to increase the performance of the system. Therefore, further
investigation would be important to implant appropriate cache strategy for both the
adapted content and context profile. Y.Cardenas, a PhD student in the group is
working on cooperative caches in Grid environment [Carden05]. His work deals with
developing cache service for Grid computing. Our framework is developed in such a
way that enables easily integration of components/services to the system. Therefore,
we will study, how his cache service can be integrated and used by our system.
•
To develop interfaces for several possible terminals other than those used in the
prototype through collaboration with SEFAGI project.
169
Annexes
170
Annexes.
Annex A: Context Profile
171
Annex A: Context Profile
I Structure of the Context Profile
ContextProfile
DeviceProfile
Hardware
DeviceType
DeviceName
ScreenWidth
ScreenHeight
ScreenCharSize
ColorDepth
Memory
Processor
Storage
TextCapable
ImageCapable
VideoCapable
AudioCapable
Software
OperatingSystem
Name
Version
Vendor
Type
Name
UserAgent
Version
Vendor
ContentTypes
ContentFormats
UserProfile
Preferences
ContentTypes
Languages
QoSAdaptationService
Time
Price
QoSNetwork
Medium
L1
L2
Low
NetworkProfile
Type
Bandwidth
Delay
L1
L2
172
II XML Representation of the Context Profile
<rdf:RDF
xmlns:rdf="http://www.w3.org/TR/WD-rdf-syntax#"
xmlns="http://Perse.insa-lyon.fr/DCAF/Schema/ContextProfile#">
xmlns:dev="http://Perse.insa-lyon.fr/DCAF/Schema/DeviceProfile#">
xmlns:use="http://Perse.insa-lyon.fr/DCAF/Schema/UserProfile#">
xmlns:net="http://Perse.insa-lyon.fr/DCAF/Schema/NetworkProfile#">
<ContextProfile rdf:ID="Context">
<device>
<dev:DeviceProfile>
<dev:Hardware>
<dev:DeviceType></dev:DeviceType>
<dev:DeviceName></dev:DeviceName>
<dev:ScreenWidth></dev:ScreenWidth>
<dev:ScreenHeight></dev:ScreenHeight>
<dev:ScreenCharSize></dev:ScreenCharSize>
<dev:ColorDepth></dev:ColorDepth>
<dev:Memory></dev:Memory>
<dev:Processor></dev:Processor>
<dev:Storage></dev:Storage>
<dev:TextCapable></dev:TextCapable>
<dev:ImageCapable></dev:ImageCapable>
<dev:AudioCapable></dev:AudioCapable>
<dev:VideoCapable></dev:VideoCapable>
</dev:Hardware>
<dev:Software>
<dev:Name></dev:Name>
<dev:Version></dev:Version>
<dev:Vendor></dev:Vendor>
<dev:UserAgent>
<dev:Type> </dev:Type>
<dev:Name></dev:Name>
<dev:Version></dev:Version>
<dev:Vendor></dev:Vendor>
<dev:ContentTypes>
<rdf:Bag>
<rdf:li></rdf:li>
</rdf:Bag>
</dev:ContentTypes>
<dev:ContentFormats>
<rdf:Bag>
<rdf:li></rdf:li>
</rdf:Bag>
</dev:ContentFormats>
</dev:UserAgent>
</dev:Software>
Annexes.
173
</device>
<user>
<use:UserProfile>
<use:Preferences>
<use:ContentTypes>
<rdf:Bag>
<rdf:li></rdf:li>
</rdf:Bag>
</use:ContentTypes>
<use:Languages>
<rdf:Bag>
<rdf:li></rdf:li>
</rdf:Bag>
</use:Languages>
</use:Preferences>
<use:QoSAdaptationService>
<use:Time></use:Time>
<use:Price></use:Price>
</use:QoSAdaptationService>
<use:QoSNetwork>
<use:Medium>
<use:L1></use:L1>
<use:L2></use:L2>
</use:Medium>
<use:Low>
<use:L1></use:L1>
<use:L2></use:L2>
</use:Low>
</use:QoSNetwork>
</use:UserProfile>
</user>
<network>
<net:NetworkProfile>
<net:Type></net:Type>
<net:Bandwidth></net:Bandwidth>
<net:Delay></net:Delay>
</net:NetworkProfile>
</network>
</ContextProfile>
</rdf:RDF>
174
III Schema Definition of the Context Profile
<rdf :RDF xmlns :rdf="http ://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns :rdfs="http ://www.w3.org/2000/01/rdf-schema#"
xmlns="http://Perse.insa-lyon.fr/DCAF/Schema/ContextProfile">
<! Context Profile Schema>
<rdf :Description rdf :ID="ContextProfile">
<rdf :type rdf :resource="http ://www.w3.org/2000/01/rdf-schema#Class"/>
<rdfs :subClassOf rdf :resource="http ://www.w3.org/2000/01/rdfschema#Resource"/>
<rdfs :comment>
Device, user and network profile descriptions
</rdfs :comment>
</rdf :Description>
<rdf :Description rdf :ID="device">
<rdfs :subClassOf rdf :resource="#ContextProfile"/>
<rdfs :comment>
Device characteristics
</rdfs :comment>
</rdf :Description>
<rdf :Description rdf :ID="user">
<rdfs :comment>
User profile and preferences
</rdfs :comment>
</rdf :Description>
<rdf :Description rdf :ID="network">
<rdfs :comment>
Network charactertsics
</rdfs :comment>
</rdf :Description>
</rdf :RDF>
Annexes.
175
IV Schema Definition of the Device Profile
<rdf:RDF xmlns:rdf="http ://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:rdfs="http ://www.w3.org/2000/01/rdf-schema#"
xmlns="http://Perse.insa-lyon.fr/DCAF/Schema/DeviceProfile">
<rdf:Description rdf:ID="DeviceProfile">
<rdf:type rdf:resource="http ://www.w3.org/2000/01/rdf-schema#Class"/>
<rdfs:subClassOf rdf:resource="http ://www.w3.org/2000/01/rdfschema#Resource"/>
<rdfs:comment>
Contains device, user and network descriptions
</rdfs:comment>
</rdf:Description>
<rdf:Description rdf:ID="Hardware">
<rdfs:subClassOf rdf:resource="#DeviceProfile"/>
<rdfs:comment>
Hardware characteristics of the user device
</rdfs:comment>
</rdf:Description>
<rdf:Description rdf:ID="Software">
<rdfs:subClassOf rdf:resource="#DeviceProfile"/>
<rdfs:comment>
Software characteristics of the user device
</rdfs:comment>
</rdf:Description>
<rdf:Description rdf:ID="DeviceType">
<rdf:type rdf:resource="http ://www.w3.org/2000/01/rdf-schema#Property"/>
<rdfs:domain rdf:resource="#Hardware"/>
<rdfs:comment>
The type of user device: PC, Laptop, Phone, PDA etc.
</rdfs:comment>
</rdf:Description>
<rdf:Description rdf:ID="DeviceName">
<rdfs:comment>
The name of user device: Qtek 9100, Nokia E61, HP iPAQ hx2790, etc.
</rdfs:comment>
</rdf:Description>
<rdf:Description rdf:ID="ScreenWidth">
<rdfs:comment>
Width of the screen
</rdfs:comment>
</rdf:Description>
<rdf:Description rdf:ID="ScreenHeight">
176
<rdfs:comment>
Height of the screen
</rdfs:comment>
</rdf:Description>
<rdf:Description rdf:ID="ScreenCharSize">
<rdfs:comment>
Character lines supported by the device"
</rdfs:comment>
</rdf:Description>
<rdf:Description rdf:ID="ColorDepth">
<rdfs:comment>
Number of colors supported by the screen
</rdfs:comment>
</rdf:Description>
<rdf:Description rdf:ID="Memory">
<rdfs:comment>
Memory capacity of the device in MB
</rdfs:comment>
</rdf:Description>
<rdf:Description rdf:ID="Processor">
<rdfs:comment>
Processor speed in MGHZ
</rdfs:comment>
</rdf:Description>
<rdf:Description rdf:ID="Storage">
<rdfs:comment>
Hard disk capacity of the device in GB
</rdfs:comment>
</rdf:Description>
<rdf:Description rdf:ID="TextCapable">
<rdfs:comment>
Text capability of the device “Yes” or “No”
</rdfs:comment>
</rdf:Description>
<rdf:Description rdf:ID="ImageCapable ">
Annexes.
177
<rdfs:comment>
Image capability of the device “Yes” or “No”
</rdfs:comment>
</rdf:Description>
<rdf:Description rdf:ID="VideoCapable">
<rdfs:comment>
Video capability of the device “Yes” or “No”
</rdfs:comment>
</rdf:Description>
<rdf:Description rdf:ID="AudioCapable">
<rdfs:comment>
Audio capability of the device “Yes” or “No”
</rdfs:comment>
</rdf:Description>
<rdf:Description rdf:ID="OperatingSystem">
<rdfs:subClassOf rdf:resource="#Software"/>
<rdfs:comment>
Describe operating system name, version and vendor
</rdfs:comment>
</rdf:Description>
<rdf:Description rdf:ID="Name">
<rdfs:domain rdf:resource="#OperatingSystem"/>
<rdfs:comment>
Name of the operating system
</rdfs:comment>
</rdf:Description>
<rdf:Description rdf:ID="Version">
<rdfs:comment>
Version of the operating system
</rdfs:comment>
</rdf:Description>
<rdf:Description rdf:ID="Vendor">
<rdfs:comment>
Vendor of the operating system
</rdfs:comment>
</rdf:Description>
<rdf:Description rdf:ID="UserAgent">
<rdfs:subClassOf rdf:resource="#Software"/>
178
<rdfs:comment>
Describe the user agent used
</rdfs:comment>
</rdf:Description>
<rdf:Description rdf:ID="Type">
<rdfs:domain rdf:resource="#UserAgent"/>
<rdfs:comment>
Type of the user agent
</rdfs:comment>
</rdf:Description>
<rdf:Description rdf:ID="Name">
<rdfs:comment>
Name of the user agent
</rdfs:comment>
</rdf:Description>
<rdf:Description rdf:ID="Version">
<rdfs:comment>
Version of the user agent
</rdfs:comment>
</rdf:Description>
<rdf:Description rdf:ID="Vendor">
<rdfs:comment>
Vendor of the user agent
</rdfs:comment>
</rdf:Description>
<rdf:Description rdf:ID="ContentTypes">
<rdf:type rdf:resource="http ://www.w3.org/2000/01/rdf-schema#Bag"/>
<rdfs:comment>
Content types supported by the user agent
</rdfs:comment>
</rdf:Description>
<rdf:Description rdf:ID="ContentFormats">
<rdfs:comment>
Content formats supported by the user agent
</rdfs:comment>
</rdf:Description>
</rdf:RDF>
Annexes.
179
V Schema Definition of the User Profile
xmlns:rdfs="http ://www.w3.org/2000/01/rdf-schema#">
xmlns="http://Perse.insa-lyon.fr/DCAF/Schema/UserProfile"
<rdf:Description rdf:ID="UserProfile">
<rdfs:comment>
Describe user profile and preferences
</rdfs:comment>
</rdf:Description>
<rdf:Description rdf:ID="Preferences">
<rdfs:subClassOf rdf:resource="#UserProfile"/>
<rdfs:comment>
Describe user preferences
</rdfs:comment>
</rdf:Description>
<rdf:Description rdf:ID="ContentTypes">
<rdfs:domain rdf:resource="#Preferences"/>
<rdfs:comment>
User content preference
</rdfs:comment>
</rdf:Description>
<rdf:Description rdf:ID="Languages">
<rdfs:domain rdf:resource="#Preferences"/>
<rdfs:comment>
Describe user language preference
</rdfs:comment>
</rdf:Description>
<rdf:Description rdf:ID="QoSAdaptationService">
<rdfs:comment>
Describe user Qos for adaptation services in terms of time and price
</rdfs:comment>
</rdf:Description>
<rdf:Description rdf:ID="Time">
<rdfs:domain rdf:resource="#QoSAdaptationService"/>
<rdfs:comment>
Describe user preference for service delay (waiting time) in terms of weight values
between 0 and 1
180
Annexes.
</rdfs:comment>
</rdf:Description>
<rdf:Description rdf:ID="Price">
<rdfs:domain rdf:resource="#QosAdaptationService"/>
<rdfs:comment>
Describe user preference for service charge in terms of weight values between 0 and 1
</rdfs:comment>
</rdf:Description>
<rdf:Description rdf:ID="QosNetwork">
<rdfs:comment>
Describe network Qos specification in terms of bandwidth values
</rdfs:comment>
</rdf:Description>
<rdf:Description rdf:ID="Medium">
<rdfs:subClassOf rdf:resource="#QoSNetwork"/>
<rdfs:comment>
Describe medium network bandwidth range
</rdfs:comment>
</rdf:Description>
<rdf:Description rdf:ID="L1">
<rdfs:domain rdf:resource="#Medium"/>
<rdfs:comment>
The lower bound value for medium network bandwidth
</rdfs:comment>
</rdf:Description>
<rdfs:domain rdf:resource="#Medium"/>
<rdfs:comment>
The upper bound value for medium network bandwidth
</rdfs:comment>
</rdf:Description>
<rdf:Description rdf:ID="Low">
<rdfs:domain rdf:resource="#QoSNetwork"/>
<rdfs:comment>
Describe low network bandwidth range
</rdfs:comment>
</rdf:Description>
<rdfs:domain rdf:resource="#Low"/>
<rdfs:comment>
The lower bound value for low network bandwidth
</rdfs:comment>
181
</rdf:Description>
<rdfs:domain rdf:resource="#Low"/>
<rdfs:comment>
The upper bound value for low network bandwidth
</rdfs:comment>
</rdf:Description>
</rdf:RDF>
VI Schema Definition of the Network Profile
xmlns:rdfs="http ://www.w3.org/2000/01/rdf-schema#"
xmlns="http://Perse.insa-lyon.fr/DCAF/Schema/NetworkProfile">
<rdf:Description rdf:ID="NetworkProfile">
<rdfs:comment>
Describe network characterstics
</rdfs:comment>
</rdf:Description>
<rdf:Description rdf:ID="Type">
<rdfs:subClassOf rdf:resource="#NetworkProfile"/>
<rdfs:comment>
Describe the network type
</rdfs:comment>
</rdf:Description>
<rdf:Description rdf:ID="Bandwidth">
<rdfs:subClassOf rdf:resource="#NetworkProfile"/>
<rdfs:comment>
Describe the network bandwidth in KB per second
</rdfs:comment>
</rdf:Description>
<rdf:Description rdf:ID="Delay">
<rdfs:domain rdf:resource="#NetworkProfile"/>
<rdfs:comment>
Describe the network delay
</rdfs:comment>
</rdf:Description>
</rdf:RDF>
Annex B: Proposed Services Description Ontology
183
<rdf:RDF
xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#"
xmlns:owl="http://www.w3.org/2002/07/owl#"
xmlns:dc="http://purl.org/dc/elements/1.1/"
xmlns="http://perse.insa-lyon.fr/DCAF/Ontology/MMASO.owl#"
xml:base="http://perse.insa-lyon.fr/DCAF/Ontology/MMASO.owl">
<owl:Ontology rdf:about=""/>

<owl:Class rdf:ID="MultimediaAdaptationServices">
<rdfs:subClassOf rdf:resource="http://www.w3.org/2002/07/owl#Thing"/>
<rdfs:subClassOf>
<owl:Restriction>
<owl:allValuesFrom>
<owl:DataRange>
<owl:oneOf rdf:parseType="Resource">
<rdf:rest rdf:parseType="Resource">
<rdf:first
rdf:datatype="http://www.w3.org/2001/XMLSchema#string"
>Good</rdf:first>
<rdf:first
rdf:datatype="http://www.w3.org/2001/XMLSchema#string"
>Bad</rdf:first>
<rdf:rest
rdf:resource="http://www.w3.org/1999/02/22-rdfsyntax-ns#nil"/>
</rdf:rest>
<rdf:first
rdf:datatype="http://www.w3.org/2001/XMLSchema#string">Poor</rdf:first>
</rdf:rest>
<rdf:first
rdf:datatype="http://www.w3.org/2001/XMLSchema#string">Fair</rdf:first>
</rdf:rest>
</rdf:rest>
<rdf:first
rdf:datatype="http://www.w3.org/2001/XMLSchema#string">Excellent</rdf:first>
</owl:oneOf>
</owl:DataRange>
</owl:allValuesFrom>
<owl:onProperty>
<owl:DatatypeProperty rdf:ID="hasServiceQuality"/>
</owl:onProperty>
</owl:Restriction>
184
Annexes.
</rdfs:subClassOf>
</owl:Class>
<owl:DatatypeProperty rdf:ID="hasServiceName">
<rdfs:domain rdf:resource="#MultimediaAdaptationServices"/>
<rdfs:range rdf:resource="http://www.w3.org/2001/XMLSchema#string"/>
</owl:DatatypeProperty>
<owl:DatatypeProperty rdf:ID="hasServiceTime">
<rdfs:range rdf:resource="http://www.w3.org/2001/XMLSchema#float"/>
<owl:DatatypeProperty rdf:about="#hasServiceQuality">
<owl:ObjectProperty rdf:about="#hasEffect">
<rdfs:range rdf:resource="http://www.w3.org/2002/07/owl#Thing"/>
</owl:ObjectProperty>
<owl:ObjectProperty rdf:about="#hasInputType">
<rdfs:range rdf:resource="#MultimediaDatatype"/>
<owl:ObjectProperty rdf:about="#hasOutputType">
<owl:ObjectProperty rdf:ID="hasServicePrice">
<rdfs:range rdf:resource="http://www.w3.org/2001/XMLSchema#float"/>
<owl:ObjectProperty rdf:about="#hasPrecondition">
<rdfs:range rdf:resource="http://www.w3.org/2002/07/owl#Thing"/>
<owl:ObjectProperty rdf:ID="hasServiceProvider">
<owl:DatatypeProperty rdf:ID="hasServiceDescription">

<rdfs:subClassOf>
<owl:Restriction>
<owl:onProperty>
</owl:onProperty>
</owl:Restriction>
</rdfs:subClassOf>
</owl:Class>
<owl:Class rdf:about="#UnimodalAudioAdaptationServices">
<rdfs:subClassOf>
<owl:Class rdf:ID="AudioAdaptationServices"/>
</rdfs:subClassOf>
<rdfs:subClassOf>
<owl:Restriction>
<owl:onProperty>
<owl:ObjectProperty rdf:ID="hasOutputType"/>
</owl:onProperty>
<owl:hasValue>
<owl:Thing rdf:ID="Audio"/>
</owl:hasValue>
</owl:Restriction>
</rdfs:subClassOf>
<owl:disjointWith>
<owl:Class rdf:ID="MultimodalAudioAdaptationServices"/>
</owl:disjointWith>
<rdfs:subClassOf rdf:resource="#AudioAdaptationServices"/>
</owl:Class>
<owl:Class rdf:about="#MultimodalAudioAdaptationServices">
<rdfs:subClassOf rdf:resource="#AudioAdaptationServices"/>
<owl:disjointWith rdf:resource="#UnimodalAudioAdapationServices"/>
</owl:Class>
<owl:Class rdf:ID="AudioLanguageAdaptationServices">
<owl:disjointWith>
<owl:Class rdf:ID="AudioFormatAdaptationServices"/>
</owl:disjointWith>
<owl:disjointWith>
<owl:Class rdf:ID="AudioReductionAdaptationServices"/>
</owl:disjointWith>
<rdfs:subClassOf>
<owl:Class rdf:ID="UnimodalAudioAdaptationServices"/>
</rdfs:subClassOf>
</owl:Class>
<owl:Class rdf:ID="AudioToTextAdaptationServices">
<rdfs:subClassOf>
<owl:Restriction>
<owl:onProperty>
<owl:ObjectProperty rdf:about="#hasOutputType"/>
</owl:onProperty>
<owl:hasValue>
<owl:Thing rdf:ID="Text"/>
</owl:hasValue>
</owl:Restriction>
185
186
Annexes.
</rdfs:subClassOf>
<rdfs:subClassOf>
<owl:Class rdf:ID="MultimodalAudioAdaptationServices"/>
</rdfs:subClassOf>
</owl:Class>
<owl:Class rdf:about="#AudioReductionAdaptationServices">
<rdfs:subClassOf rdf:resource="#UnimodalAudioAdapationServices"/>
<owl:disjointWith>
<owl:Class rdf:about="#AudioFormatAdaptationServices"/>
</owl:disjointWith>
<owl:disjointWith rdf:resource="#AudioLanguageAdaptationServices"/>
</owl:Class>
<owl:Class rdf:ID="StereoToMonoAdaptationServices">
<rdfs:subClassOf>
<owl:Class rdf:about="#AudioReductionAdaptationServices"/>
</rdfs:subClassOf>
<owl:disjointWith>
<owl:Class rdf:ID="AudioSamplingRateReductionAdaptationServices"/>
</owl:disjointWith>
</owl:Class>
<owl:Class rdf:about="#AudioSamplingRateReductionAdaptationServices">
<rdfs:subClassOf rdf:resource="#AudioReductionAdaptationServices"/>
<owl:disjointWith rdf:resource="#StereoToMonoAdaptationServices"/>
</owl:Class>
<owl:Class rdf:about="#AudioFormatAdaptationServices">
<rdfs:subClassOf>
<owl:Restriction>
<owl:onProperty>
<owl:ObjectProperty rdf:ID="hasPrecondition"/>
</owl:onProperty>
<owl:allValuesFrom>
<owl:Class rdf:ID="AudioDatatypeFormat"/>
</owl:Restriction>
</rdfs:subClassOf>
<owl:disjointWith rdf:resource="#AudioLanguageAdaptationServices"/>
<owl:disjointWith rdf:resource="#AudioReductionAdaptationServices"/>
<rdfs:subClassOf rdf:resource="#UnimodalAudioAdapationServices"/>
<rdfs:subClassOf>
<owl:Restriction>
<owl:onProperty>
<owl:ObjectProperty rdf:ID="hasEffect"/>
</owl:onProperty>
<owl:allValuesFrom>
<owl:Class rdf:about="#AudioDatatypeFormat"/>
</owl:Restriction>
</rdfs:subClassOf>
</owl:Class>

<owl:Class rdf:about="#ImageAdaptationServices">
<rdfs:subClassOf>
<owl:Restriction>
<owl:onProperty>
</owl:onProperty>
<owl:hasValue rdf:resource="#Image"/>
</owl:Restriction>
</rdfs:subClassOf>
</owl:Class>
<owl:Class rdf:about="#UnimodalImageAdaptationServices">
<rdfs:subClassOf>
<owl:Class rdf:ID="ImageAdaptationServices"/>
</rdfs:subClassOf>
<rdfs:subClassOf>
<owl:Restriction>
<owl:onProperty>
</owl:onProperty>
<owl:hasValue>
<owl:Thing rdf:ID="Image"/>
</owl:hasValue>
</owl:Restriction>
</rdfs:subClassOf>
<owl:disjointWith>
<owl:Class rdf:ID="MultimodalImageAdaptationServices"/>
</owl:disjointWith>
<rdfs:subClassOf rdf:resource="#ImageAdaptationServices"/>
</owl:Class>
<owl:Class rdf:about="#MultimodalImageAdaptationServices">
<rdfs:subClassOf rdf:resource="#ImageAdaptationServices"/>
<owl:disjointWith rdf:resource="#UnimodalImageAdapationServices"/>
</owl:Class>
<owl:Class rdf:ID="ImageResizingAdaptationServices">
<owl:disjointWith>
<owl:Class rdf:ID="ImageColorReductionAdaptationServices"/>
</owl:disjointWith>
<owl:disjointWith>
<owl:Class rdf:ID="ImageColorToGrayAdaptationServices"/>
</owl:disjointWith>
<owl:disjointWith>
<owl:Class rdf:ID="ImageFormatAdaptationServices"/>
</owl:disjointWith>
<rdfs:subClassOf>
<owl:Class rdf:ID="UnimodalImageAdaptationServices"/>
</rdfs:subClassOf>
</owl:Class>
<owl:Class rdf:ID="ImageToTextAdaptationServices">
187
188
Annexes.
<rdfs:subClassOf>
<owl:Restriction>
<owl:onProperty>
</owl:onProperty>
<owl:hasValue>
</owl:hasValue>
</owl:Restriction>
</rdfs:subClassOf>
<rdfs:subClassOf>
<owl:Class rdf:ID="MultimodalImageAdaptationServices"/>
</rdfs:subClassOf>
</owl:Class>
<owl:Class rdf:about="#ImageResizingAdaptationServices">
<rdfs:subClassOf rdf:resource="#UnimodalImageAdapationServices"/>
<owl:disjointWith rdf:resource="#ImageColorReductionAdaptationServices"/>
<owl:disjointWith rdf:resource="#ImageColorToGrayAdaptationServices"/>
<owl:disjointWith>
<owl:Class rdf:about="#ImageFormatAdaptationServices"/>
</owl:disjointWith>
</owl:Class>
<owl:Class rdf:about="#ImageFormatAdaptationServices">
<rdfs:subClassOf>
<owl:Restriction>
<owl:onProperty>
</owl:onProperty>
<owl:allValuesFrom>
<owl:Class rdf:ID="ImageDatatypeFormat"/>
</owl:Restriction>
</rdfs:subClassOf>
<owl:disjointWith rdf:resource="#ImageResizingAdaptationServices"/>
<owl:disjointWith rdf:resource="#ImageColorReductionAdaptationServices"/>
<owl:disjointWith rdf:resource="#ImageColorToGrayAdaptationServices"/>
<rdfs:subClassOf>
<owl:Restriction>
<owl:onProperty>
</owl:onProperty>
<owl:allValuesFrom>
<owl:Class rdf:about="#ImageDatatypeFormat"/>
</owl:Restriction>
</rdfs:subClassOf>
</owl:Class>

<owl:Class rdf:about="#TextAdaptationServices">
<rdfs:subClassOf>
<owl:Restriction>
<owl:onProperty>
</owl:onProperty>
<owl:hasValue rdf:resource="#Text "/>
</owl:Restriction>
</rdfs:subClassOf>
</owl:Class>
<owl:Class rdf:about="#UnimodalTextAdaptationServices">
<rdfs:subClassOf>
<owl:Class rdf:ID="TextAdaptationServices"/>
</rdfs:subClassOf>
<rdfs:subClassOf>
<owl:Restriction>
<owl:onProperty>
</owl:onProperty>
<owl:hasValue>
</owl:hasValue>
</owl:Restriction>
</rdfs:subClassOf>
<owl:disjointWith>
<owl:Class rdf:ID="MultimodalTextAdaptationServices"/>
</owl:disjointWith>
<rdfs:subClassOf rdf:resource="#TextAdaptationServices"/>
</owl:Class>
<owl:Class rdf:about="#MultimodalTextAdaptationServices">
<rdfs:subClassOf rdf:resource="#TextAdaptationServices"/>
<owl:disjointWith rdf:resource="#UnimodalTextAdapationServices"/>
</owl:Class>
<owl:Class rdf:ID="TextLanguageAdaptationServices">
<owl:disjointWith>
<owl:Class rdf:ID="TextFormatAdaptationServices"/>
</owl:disjointWith>
<owl:disjointWith>
<owl:Class rdf:ID="TextSummaryAdaptationServices"/>
</owl:disjointWith>
<rdfs:subClassOf>
<owl:Class rdf:ID="UnimodalTextAdaptationServices"/>
</rdfs:subClassOf>
</owl:Class>
<owl:Class rdf:ID="TextToAudioAdaptationServices">
<rdfs:subClassOf>
<owl:Restriction>
<owl:onProperty>
189
190
Annexes.
</owl:onProperty>
<owl:hasValue>
<owl:Thing rdf:ID="Audio"/>
</owl:hasValue>
</owl:Restriction>
</rdfs:subClassOf>
<rdfs:subClassOf>
<owl:Class rdf:ID="MultimodalTextAdaptationServices"/>
</rdfs:subClassOf>
</owl:Class>
<owl:Class rdf:about="#TextSummaryAdaptationServices">
<rdfs:subClassOf rdf:resource="#UnimodalTextAdapationServices"/>
<owl:disjointWith>
<owl:Class rdf:about="#TextFormatAdaptationServices"/>
</owl:disjointWith>
</owl:Class>
<owl:Class rdf:about="#TextFormatAdaptationServices">
<rdfs:subClassOf>
<owl:Restriction>
<owl:onProperty>
</owl:onProperty>
<owl:allValuesFrom>
<owl:Class rdf:ID="TextDatatypeFormat"/>
</owl:Restriction>
</rdfs:subClassOf>
<owl:disjointWith rdf:resource="#TextLanguageAdaptationServices"/>
<owl:disjointWith rdf:resource="#TextSummaryAdaptationServices"/>
<rdfs:subClassOf>
<owl:Restriction>
<owl:onProperty>
</owl:onProperty>
<owl:allValuesFrom>
<owl:Class rdf:about="#TextDatatypeFormat"/>
</owl:Restriction>
</rdfs:subClassOf>
</owl:Class>
<owl:Class rdf:ID="MultimediaDatatype">
<owl:equivalentClass>
<owl:Class>
<owl:oneOf rdf:parseType="Collection">
<owl:Thing rdf:about="#Audio"/>
<owl:Thing rdf:about="#Image"/>
<owl:Thing rdf:about="#Text"/>
<owl:Thing rdf:ID="Video"/>
<owl:Thing rdf:ID="Graphics"/>
<owl:Thing rdf:ID="Animation"/>
</owl:oneOf>
</owl:Class>
</owl:equivalentClass>
</owl:Class>

<owl:Class rdf:ID="#MultimediaDatatypeFormat">
<rdfs:subClassOf rdf:resource="http://www.w3.org/2002/07/owl#Thing"/>
</owl:Class>
<owl:ObjectProperty rdf:about="#hasDatatype">
<rdfs:domain rdf:resource="#MultimediaDatatypeFormat"/>
<owl:DatatypeProperty rdf:ID="hasFormat">
<rdfs:domain rdf:resource="#MultimediaDatatypeFormat"/>

<owl:Class rdf:about="#AudioDatatypeFormat">
<rdfs:subClassOf>
<owl:Restriction>
<owl:onProperty>
<owl:ObjectProperty rdf:about="#hasDatatype"/>
</owl:onProperty>
</owl:Restriction>
</rdfs:subClassOf>
<rdfs:subClassOf rdf:resource="#MultimediaDatatypeFormat"/>
</owl:Class>

<owl:Class rdf:ID="ImageDatatypeFormat">
<rdfs:subClassOf>
<owl:Class rdf:ID="MultimediaDatatypeFormat"/>
</rdfs:subClassOf>
<rdfs:subClassOf>
<owl:Restriction>
<owl:onProperty>
<owl:ObjectProperty rdf:ID="hasDatatype"/>
</owl:onProperty>
<owl:hasValue>
<owl:Thing rdf:ID="Image"/>
</owl:hasValue>
</owl:Restriction>
</rdfs:subClassOf>
</owl:Class>
191
192
Annexes.
<owl:Class rdf:ID="AnimationDatatypeFormat">
<rdfs:subClassOf rdf:resource="#ImageDatatypeFormat"/>
</owl:Class>
<owl:Class rdf:ID="GraphicsDatatypeFormat">
<rdfs:subClassOf rdf:resource="#ImageDatatypeFormat"/>
</owl:Class>

<owl:Class rdf:ID="TextDatatypeFormat">
<rdfs:subClassOf>
<owl:Restriction>
<owl:onProperty>
</owl:onProperty>
<owl:hasValue rdf:resource="#Text"/>
</owl:Restriction>
</rdfs:subClassOf>
</owl:Class>

<owl:Class rdf:ID="VideoDatatypeFormat">
<rdfs:subClassOf>
<owl:Restriction>
<owl:onProperty>
</owl:onProperty>
<owl:hasValue rdf:resource="#Video"/>
</owl:Restriction>
</rdfs:subClassOf>
</owl:Class>
</rdf:RDF>
Annex C: OWL-S Profile Schema
193
<?xml version='1.0' encoding='ISO-8859-1'?>
<!DOCTYPE uridef [
<!ENTITY rdf "http://www.w3.org/1999/02/22-rdf-syntax-ns">
<!ENTITY rdfs "http://www.w3.org/2000/01/rdf-schema">
<!ENTITY owl "http://www.w3.org/2002/07/owl">
<!ENTITY expr "http://www.daml.org/services/owl-s/1.1/generic/Expression.owl">
<!ENTITY xsd "http://www.w3.org/2001/XMLSchema">
<!ENTITY service "http://www.daml.org/services/owl-s/1.1/Service.owl">
<!ENTITY process "http://www.daml.org/services/owl-s/1.1/Process.owl">
<!ENTITY DEFAULT "http://www.daml.org/services/owl-s/1.1/Profile.owl">
]>
<!-This document uses entity types as a shorthand for URIs.
For a version with unexpanded entities, try loading this source into Internet Explorer.
-->
<rdf:RDF
xmlns:rdf=
"&rdf;#"
xmlns:rdfs=
"&rdfs;#"
xmlns:owl=
"&owl;#"
xmlns:service="&service;#"
xmlns:process="&process;#"
xmlns= "&DEFAULT;#"
xml:base=
"&DEFAULT;">
<owl:Ontology rdf:about="">
<owl:versionInfo>
$Id: Profile.owl,v 1.51 2004/11/03 03:27:20 martin Exp $
</owl:versionInfo>
<rdfs:comment>
OWL ontology for Advertisements (i.e. Profiles).
This file belongs to the OWL-S Release.
Initial version created by Terry Payne ([email protected]).
Modified by Massimo Paolucci ([email protected])
Modified by David Martin and other members of the OWL-S Coalition.
</rdfs:comment>
<rdfs:seeAlso
rdf:resource="http://www.daml.org/services/owls/1.1/ProfileAdditionalParameters.owl" />
<rdfs:seeAlso
rdf:resource="http://www.daml.org/services/owls/1.1/ProfileDeprecatedElements.owl" />
<rdfs:seeAlso rdf:resource="http://www.daml.org/services/owl-s/1.1/ActorDefault.owl"
/>

<!-Profile is a subclass of ServiceProfile. It is used to acknowledge that there may be
different ways to profile services that are different from the way we expressed it so far.
-->
<owl:Class rdf:ID="Profile">
<rdfs:label>Profile</rdfs:label>
<rdfs:subClassOf rdf:resource="&service;#ServiceProfile" />
<rdfs:comment>
Definition of Profile
</rdfs:comment>
</owl:Class>

NON FUNCTIONAL PROPERTIES
########### -->
<!-The Service Name refers to the name of the service that is being offered.
-->
<owl:DatatypeProperty rdf:ID="serviceName">
<rdfs:domain rdf:resource="#Profile"/>

<owl:Class rdf:about="#Profile">
<rdfs:comment>
A profile can have only one name
</rdfs:comment>
<rdfs:subClassOf>
<owl:Restriction>
<owl:onProperty rdf:resource="#serviceName"/>
<owl:cardinality rdf:datatype="&xsd;#nonNegativeInteger">1</owl:cardinality>
</owl:Restriction>
</rdfs:subClassOf>
</owl:Class>
<!-The TextDescription provides a brief description of the service.
It summarisese what the service offers, or to describe what
service is requested.
-->
195
<owl:DatatypeProperty rdf:ID="textDescription">

<owl:Class rdf:about="#Profile">
<rdfs:comment>
A profile can have only one text description
</rdfs:comment>
<rdfs:subClassOf>
<owl:Restriction>
<owl:onProperty rdf:resource="#textDescription"/>
</owl:Restriction>
</rdfs:subClassOf>
</owl:Class>
<!-Contact information is used to record contact information about the entity that issued
the Profile. It once refered to Actor, that records address and other info that allows a
receiver of the profile to contact directly the issuer. However, this class has migrated to a
separate ActorDefault.daml ontology.
-->
<owl:ObjectProperty rdf:ID="contactInformation">
<!-has_process is a pointer the process that is associated with the service.
-->
<owl:ObjectProperty rdf:ID="has_process">
<rdfs:range rdf:resource="&process;#Process"/>
<owl:FunctionalProperty rdf:about="#has_process"/>
<owl:ObjectProperty rdf:ID="serviceCategory">
<rdfs:range rdf:resource="#ServiceCategory"/>
<owl:ObjectProperty rdf:ID="serviceParameter">
<rdfs:range rdf:resource="#ServiceParameter"/>
196
Annexes.

<owl:ObjectProperty rdf:ID="hasParameter">
<rdfs:range rdf:resource="&process;#Parameter"/>
<owl:ObjectProperty rdf:ID="hasInput">
<rdfs:subPropertyOf rdf:resource="#hasParameter"/>
<rdfs:range rdf:resource="&process;#Input"/>
<owl:ObjectProperty rdf:ID="hasOutput">
<rdfs:subPropertyOf rdf:resource="#hasParameter"/>
<rdfs:range rdf:resource="&process;#Output"/>
<owl:ObjectProperty rdf:ID="hasPrecondition">
<rdfs:range rdf:resource="&expr;#Condition"/>
<owl:ObjectProperty rdf:ID="hasResult">
<rdfs:range rdf:resource="&process;#Result"/>
<owl:Class rdf:ID="ServiceCategory"/>

<owl:DatatypeProperty rdf:ID="categoryName">
<rdfs:domain rdf:resource="#ServiceCategory"/>

<owl:Class rdf:about="#ServiceCategory">
<rdfs:comment>
a ServiceCategory is restricted to refer to only onename
</rdfs:comment>
<rdfs:subClassOf>
<owl:Restriction>
197
<owl:onProperty rdf:resource="#categoryName"/>
</owl:Restriction>
</rdfs:subClassOf>
</owl:Class>

<owl:DatatypeProperty rdf:ID="taxonomy">

<rdfs:comment>
a ServiceCategory is restricted to refer to only one taxonomy
</rdfs:comment>
<rdfs:subClassOf>
<owl:Restriction>
<owl:onProperty rdf:resource="#taxonomy"/>
</owl:Restriction>
</rdfs:subClassOf>
</owl:Class>

<owl:DatatypeProperty rdf:ID="value">

<rdfs:comment>
</rdfs:comment>
<rdfs:subClassOf>
<owl:Restriction>
<owl:onProperty rdf:resource="#value"/>
198
Annexes.
</owl:Restriction>
</rdfs:subClassOf>
</owl:Class>

<owl:DatatypeProperty rdf:ID="code">

<rdfs:comment>
</rdfs:comment>
<rdfs:subClassOf>
<owl:Restriction>
<owl:onProperty rdf:resource="#code"/>
</owl:Restriction>
</rdfs:subClassOf>
</owl:Class>
<!-ServiceParameter describes service parameters.
In general we can think this class as the root of an ontology of
Service Parameters of different types. Other types of
ServiceParameters may expand this definition by adding other
properties.
-->
<owl:Class rdf:ID="ServiceParameter"/>

<owl:DatatypeProperty rdf:ID="serviceParameterName">
<rdfs:domain rdf:resource="#ServiceParameter"/>
<owl:Class rdf:about="#ServiceParameter">
<rdfs:comment>
A ServiceParameter should have at most 1 name (more precisely only
one serviceParameterName)
</rdfs:comment>
<rdfs:subClassOf>
<owl:Restriction>
<owl:onProperty rdf:resource="#serviceParameterName"/>
</owl:Restriction>
</rdfs:subClassOf>
</owl:Class>

<owl:ObjectProperty rdf:ID="sParameter">
<rdfs:domain rdf:resource="#ServiceParameter"/>
<rdfs:range rdf:resource="&owl;#Thing"/>
<owl:Class rdf:about="#ServiceParameter">
<rdfs:comment>
a Parameter is restricted to refer to only one concept in some
ontology
</rdfs:comment>
<rdfs:subClassOf>
<owl:Restriction>
<owl:onProperty rdf:resource="#sParameter"/>
</owl:Restriction>
</rdfs:subClassOf>
</owl:Class>

<owl:DatatypeProperty rdf:ID="serviceClassification">
<rdfs:range rdf:resource="&xsd;#anyURI"/>

<owl:DatatypeProperty rdf:ID="serviceProduct">
<rdfs:range rdf:resource="&xsd;#anyURI"/>
</rdf:RDF>
199
200
Annexes.
Annex D: Modified Dijkstra’s Algorithm
201
function: Dijkstra (G)
Input: Adaptation Graph G= (V, E) and W
Output: Optimal path OPath
V
E
os
oe
v
u
D[v]
W ( v)
Pred
SV
A set of nodes representing operators
A set of edges representing connection between operators
A start node
An end node
A node
A node
Stores total quality value (price and execution time) of the path from
start node os to node v
stores quality value of v
A set of nodes to store predecessor nodes, e.g. Pred[v] stores
predecessor of v
A set of nodes-stores topologically sorted nodes
begin
SV=SortTopologiocally(V, E) // sort V topologically
for each v ∈ V
begin
D[v] = negative infinity
Pred[v] =NIL//parent of v in the optimal path
end
D[os] =0
for each u ∈ SV
begin
for each v ∈ SV
begin
if ((u, v) ∈ E) then
begin
if (D[v] < D[u]+W(v) then
begin
D[v] =D[u]+W(v)
Pred[v] =u
end
end
end
end
//reading the optimal path by iteration
Opath =Ø //contains list of nodes on the optimal path from os to oe
u =oe //e is the end node
for each Pred[u] ≠ NIL
begin
Opath ={u} ∪ Opath //Insert u in the beginning of Opath
end
202
Annexes.
return Opath
end //Optimal-path search
// this procedure sorts a graph topologically and returns a set of sorted nodes
// in a topologically sorted graph, if there is a path from node x to node y, the node x comes
// before the node y in the node set.
// V a set of nodes-input
// E a set of edges-input
function Vector SortTopologically(in V, in E)
// TE a set of edges- temporal variable
// TV a set of nodes-temporal variable
// TSV a set of nodes- stores topologically sorted nodes
// v a node- temporal variable
begin
TE =E
TV =V
TSV =Ø
while (TV ≠ Ø)
begin
for each v ∈ TV
begin
if (Not Predecessor(v, TV, TE)) then // v does not have predecessor
begin
TSV =TSV ∪ {v}
Delete(v, TV, TE)
end
end
end
return TSV
end //SortTopologically
//this function returns true if a node has a predecessor otherwise false
// v a node-input
// TV a set of nodes-input
//TE a set of edges-input
function Boolean Predecessor(in v, in TV, in TE)
//y a node
var
y
begin
for each y ∈ TV
begin
if ((y, v) ∈ TE) then // the node has predecessor
return true
end
return false
end //Predecessor
//this procedure deletes a node
//v a node to be deleted-input
//TV a set of nodes-input/output
//TE a set of edges-input/output
procedure Delete(in v, inout TV, inout TE)
// y a node
var
y
begin
for each y ∈ TV
begin
if ((v,y) ∈ TE) then
TE=TE-{(v, y)}
end
TV=TV-{v}
end //Delete
203
Annex E: Classes Hierarchy of Local Proxy
205
Class::MediaObject
This class represents a media object such as text, image, video or audio data.
MediaObject: instance variables
Vector mediaInfo
Vector mediaProfile
MediaObject: Methods
public void setMediaInformation(vector mediaInfo)
sets the media information of the media object
public void setMediaProfile(vector mediaProfile)
sets the media profile of the media object
public vector getMediaInformation()
returns the media information of the media object
public vector getMediaProfile()
returns the media profile of the media object
Note: Hashtable was used to implement the media object
Class: AdaptationTasks
This class extends the Vector class. It stores the transformation tasks.
Class: Operator
This class stands for an operator that corresponds to an adaptation service. It has a constructor
to initialize the instance variables.
Operator: Instance variables
protected String OpName; the name of the operator
206
Annexes.
protected int OpId; the identification of the operator
protected Vector Input; the input list of the operator
protected Vector Output; the output list of the operator
protected Vector Precondition; the preconditions of the operator
protected Vector Effect; the effect of the operator
protected double Cost; cost of the operator
protected double Time; execution time of the operator
Operator: Methods
public void SetOpId(int id)
public void SetOpName(String op)
public void AddInput(Vector in)
public void AddOutput(Vector out)
public void SetPrecond(Vector pre)
public void SetEffect(Vector effect)
public void SetCost(double c)
public void SetTime(double t)
public int GetOpId()
public String GetOpName()
public Vector GetPrecond()
public Vector GetEffect()
public double GetCost()
public double GetTime()
Class: Predicate
This class stores the predicate information to define precondition and effect of an operator
Predicate: Instance variables
protected String PredName; the name of the predicate
protected String PredValue; the value of the predicate
Predicate: Methods
public void SetPred(String preName, String preValue)
207
public String GetPredName()
public String GetPredValue()
Class: PlanElement
This is a data structure that has two main variables of type list (Vector) of Operator and list
(Vector) of String. The first list stores the operators that realize an adaptation task and the
second list stores the links of each operator with next operators. It has a constructor to
initialize the instance variables.
PlanElement: Instance variables
protected Vector opLayer; the list of operators
protected Vector linkLayer; the link or connection of each operator in the opLayer with the
next operators
PlanElement: Methods
public void setOpLayer(Vector ops)
public void setLinkLayer(Vector link)
public Vector GetOpLayer()
public Vector GetLinkLayer()
Class: Plans:
This is a data structure that has a list (Vector) of PlanElement objects. Its purpose is to hold
all possible adaptation plans (also called adaptation graph).
Class: OptimalPlan (also called optimal path)
This class extends the Vector class. Its purpose is to hold the operators of the optimal plan or
path.
Class: Predicate
This class stores the predicate information to define precondition and effect of an operator
Predicate: Instance variables
protected String PredName; the name of the predicate
208
Annexes.
protected String PredValue; the value of the predicate
Predicate: Methods
public void SetPred(String preName, String preValue)
public String GetPredName()
public String GetPredValue()
Class: ContextProfile
This class stores the context profile/metadata which includes user profile, device profile and
network profile.
ContextProfile:Methods
public Void setUserProfile()
public Void setDeviceProfile()
public Void setNetworkProfile()
public Vector getUserProfile()
public Vector getDeviceProfile()
public Vector getNetworkProfile()
Note: Hashtable was used to implement ContextProfile
Class: ContextProfileHandler
This class extends the DefaultHandler class. It parses the XML description of context profile
and stores the data in the ContextProfile object.
Class: MediaHandler
This class extends the DefaultHandler class. It parses the Multimedia content description (in
XML format) and stores the data in the MediaObject object.
Class: OperatorMappingHandler
209
This class extends the DefaultHandler class. It takes ContextProfile object, MediaObject
object and transformation rules (in XML format) and returns the transformations prescript in
XML format.
Class: OWLHandler
This class extends the DefaultHandler class. It parses an OWL description of an adaptation
service and returns its corresponding Operator object.
Class: AdaptationProcessor
This class is responsible for constructing the adaptation graph and the finding the optimal path
of the graph. It has a constructor that takes ContextProfile, MediaObject and a file containing
the transformation rules and returns the transformation tasks.
AdaptationProcessor: Instance variables
AdaptationTasks tasks;
MediaState iState;
MediaState fSate;
AdaptationProcessor: Methods
public
Void
AdaptationGraphConstructor
(MediaState
iState,
MediaState
AdaptationTasks tasks)
public Vector SearchOptimalPath (Vector plans)
this method returns the optimal adaptation plan if there is one otherwise empty.
fState,
Annex F: Glossary
211
Annex F: Glossary
API
Application Program Interface
CC/PP
CSS
Cascading Style Sheets
cHTML
Compact HTML
GPRS
General Packet Radio Service
GIF
Graphics Interchange Formats
HDML
Handheld Device Markup Language
HTML
Hypertext Markup Language
HTTP
Hypertext Transfer Protocol
ISP
Internet Service Provider
JPEG
Joint Photographic Experts Group, a lossy compression technique for color
images
MMS
Multimedia Messaging Service, a method of transmitting multimedia and short
text messages over wireless networks using WAP protocol
MPEG
Moving Picture Experts Group, also refers to family of digital video
compression standards and file formats developed by the group
PDA
Personal Digital Assistant
PDF
Portable Document Format
PNG
Portable Network Graphics
SMIL
Synchronized Multimedia Integration Language
SOAP
Simple Object Access Protocol
RIM
Remote Invocation Method
RPC
Remote Procedure Call
Smartphone
A combination of mobile phone and PDA
SMS
Short Message Service
SVG
Scalable Vector Graphics
TIFF
Tagged Image File Format
UAProf
User Agent Profile
W3C
World Wide Consortium
WAP
Wireless Application Protocol
212
WAX
Annexes.
Wireless Abstract XML a set of tools and an abstract markup language used to
author content for wireless applications.
WBMP
Wireless Bitmap, a graphic format optimized for mobile devices
WML
Wireless Markup Language, an XML language used to specify content and
user interface for WAP devices
XHTML
eXtensible HyperText Markup Language
XML
eXtensible Markup Language
XSL
eXtensible Style Language
XSLT
eXtensible Style Language Transformations, the language used in XSL style
sheets to transform XML documents into other XML documents
Windows CE (sometimes abbreviated WinCE) is a variation of Microsoft's Windows
operating system for minimalistic computers and embedded systems.
Bibliography
Bibliography
215
Bibliography
[3Com00]
3Com
Inc.
Palm
VII
[On
line].
Available
on:
<http://www.palm.com/products/palmvii> (Accessed on 11 December 2003).
[Adela01]
T. Adelaar, F. Aldershoff, E. Goedvolk, et al. Engineering Content for a
New Competitive Era: The GigaPort Content Engineering Periodical 2000. TI/RS/2000/164
[on
line].
Netherlands:
GigaCE,
2001.
Available
on:
<http://extranet.telin.nl/docuserver/dscgi/ds.py/ViewProps/File-12331> (Accessed on 14
December 2003).
[Agoston00]
T.C. Agoston, T.Ueda, Y.Nishimura. Pervasive Computing in a Networked
World. In Global Distributed Intelligence for Everyone, INET2000: 10th Annual Internet
Society Conference, 18-21 July 2000, Yokohama, Japan [on line]. Available on:
<http://www.isoc.org/inet2000/cdproceedings/3a/3a_1.htm> (Accessed on 11 December
2003).
[Alonso04]
G. Alonso, F. Casati, H. Kuno et al. Web Services: Concepts, Architectures
and Applications. New York: Springer-Verlag, 2004, 480 p. ISBN 3540440089.
[Angin98]
O. Angin, AT. Campbell, ME. Kounavis et al. The mobiware Toolkit:
Programmable Support for Adaptive Mobile Networking. IEEE Personal Communication,
1998, Vol.5, No.4, pp. 32-43.
[Ankol01]
A. Ankolekar, M. Burstein, JR. Hobbs et al. DAML-S: Semantic Markup
for Web services. In proceedings of the International Semantic Web Working Symposium,
July 30-August 1, 2001, Stanford, CA, pp. 411-430.
[Apache]
The Apache Software Foundation. The Apache Cocoon Project [on line].
Available on: <http://cocoon.apache.org/> (Accessed on 10 April 2004).
[Ardon03]
S. Ardon, P. Gunningberg, B. LandFeldt et al. MARCH: A distributed
content adaptation architecture. International Journal of Communication Systems, Special
Issue: Wireless Access to the Global Internet: Mobile Radio Networks and Satellite Systems,
2003, Vol 16, No. 1, pp. 97-115.
[Arkin02]
A. Arkin .Business Process Modeling Language. Working Draft June 24,
2002 [on line]. Available on: <http://xml.coverpages.org/BPML-WD-200206.pdf> (Accessed
on 13 April 2004).
[Arkin02b]
A. Arkin, S. Askary, S. Fordin et al. Web Service Choreography Interface
(WSCI) 1.0. World Wide Web Consortium, 2002, Boston, USA.
216
Bibliography.
[Asadi04]
M. K. Asadi and J-C. Dufourd. Multimedia Adaptation by Transmoding in
MPEG-21. In 5th International Workshop on Image Analysis for Multimedia Interactive
Services (WIAMIS), April 21-23, 2004, Lisbon, Portugal.
[Aspy93]
C. B. Aspy D. N. Aspy. The Human Age in Education. Journal of
Invitational Theory and Practice, 1993, Vol. 2, No. 1, pp.5-12.
[Aura]
Aura (CMU) [on line]. Available on: <http://www-2.cs.cmu.edu/aura/>
(Accessed on 10 September 2004).
[Aust02]
D. Austin. Web Services Architecture Requirements [on line]. W3C
Working Draft 14 November 2002. Available on: <http://www.w3.org/TR/wsa-reqs>
(Accessed on 13 October 2003).
[BabelFish]
BabelFish [on line]. Available on: <http://babelfish.altavista.com/>
[Bane02]
Language
A. Banerji, C. Bartolini, D. Beringer et al. Web Services Conversation
(WSCL)
1.0,
W3C
Note,
Mar.
2002
[on
line].
Available
on:
<http://www.w3.org/TR/2002/NOTE-wscl10-20020314/> (Accessed on 19 March 2003).
[Barbir0202]
A. Barbir02, R. Chen, M. Hofmann et al. An Architecture for Open
Pluggable Edge Services (OPES), Internet Draft, IETF Network Working Group, December
11, 2002 [on line]. Available on: <http://www.ietf.org/internet-drafts/draft-ietf-opesarchitecture-04.txt> (Accessed on 9 December 2003).
[Batke04]
J-M. Batke, G. Eisenberg, P. Weishaupt et al. A QBH system using
MPEG-7 Descriptor. In Proceedings of the 116th Audio Engineering Society Convention, May
8–11 2004, Berlin, Germany.
[Bekker00]
H. Bekker, I. Belgers, P. Valkenburg. Inventory of Metadata for
Multimedia
[on
line].
September
2000.
Available
on:
http://www.surfnet.nl/innovatie/surfworks/doc/mmmetadata/. (Accessed on 24 May 2003).
[Bell02]
T. Bellwood, L. Clément, D. Ehnebuske et al. Universal Description,
Discovery and Integration specification (UDDI) 3.0. UDDI Spec Technical Committee
Specification [on line], 19 July 2002. Available on: <http://uddi.org/pubs/uddi-v3.00published-20020719.htm> (accessed on 10 June 2003).
[Bhra98]
H. Bharadvaj, A. Joshi, and S. Auephanwiriyakul. An active transcoding
proxy to support mobile web access. In Proceedings of 17th IEEE Symposium on Reliable
Distributed System, 1998, West Lafayette, IN, USA, pp. 118--123.
Bibliography
[Bhrag97]
217
V. Bharghavan, V. Gupta. A Framework for Application Adaptation in
Mobile Computing Environments. In Proceedings of the IEEE Compsoc’97, 11-15 Aug 1997,
Washington, D.C., USA.
[Bick97]
T. Bickmore and B. Schilit. Digestor: Device-independent Access to the
World Wide Web. In Proceedings of the 6th World Wide Web Conference (WWW6), April
7-11, 1997, Santa Clara, California USA, pp. 655–663.
[Bick99]
T. Bickmore, A. Girgensohn, J.W. Sullivan. Web Page Filtering and Re-
Authoring for Mobile Users. In The Computer Journal, 199, vol. 42, no. 6, pp. 534—546.
[Björk99]
S. Björk, L.E. Holmquist, J. Redström et al. WEST: a Web browser for
small terminals. In Proceedings of the 12th annual ACM symposium on User interface
software and technology, 07-10 November 1999, Asheville, North Carolina, USA, pp.187196.
[Boll01]
S. Boll and W. Klas. ZYX - A Multimedia Document Model for Reuse and
Adaptation. In IEEE Transactions on Knowledge and Data Engineering, May/June 2001,
Vol.13, No. 3, pp. 361-382.
[Boll99]
S.Boll, W.Klas, J.Wandel. A cross-media adaptation strategy for
multimedia presentation. In Proceedings of the 7th ACM International Conference on
Multimedia, Oct. 1999, Orlando, Florida, USA, pp. 37-46. ISBN: 1-58113-151-8.
[Bolner01]
M. S. Bolner and G. A. Poirier. The Research Process: Books and Beyond.
2nd Ed. Dubuque, IA: Kendall/Hunt, 2001, 450 p. ISBN: 0787294489.
[Bolot98]
J.C. Bolot, T. Turletti. Experience with rate control mechanisms for packet
video in the Internet, Computer Communication Review, ACM, January 1998, Vol. 28, No. 1;
pp.4-15.
[Bosz02]
L. Böszörményi, M. Döller, H. Hellwagner, H. Kosch et al. Comprehensive
treatment of adaptation in distributed multimedia systems in the ADMITS project. In
Proceedings of the 10th ACM international conference on Multimedia, December 1-6, 2002,
Juan Les pins, France, pp. 429-430. ISBN: 1-58113-620-X.
[Bosz03]
L. Böszörményi, H. Hellwagner, H. Kosch et al. Metadata Driven
Adaptation in the ADMITS Project. EURASIP Signal Processing: Image Communication
Journal (Special Issue on Multimedia Adaptation), Sept. 2003, Vol. 18, No. 8 pp. 749-766.
[Box00]
D. Box, D. Ehnebuske, G. Kakivaya et al. Simple Object Access Protocol
(SOAP) 1.1. Tech. rep., W3C Note, 08 May 2000.
218
[Brown96]
Bibliography.
J.P. Brown. The Stick-e Document: a Framework for Creating Context-
Aware Applications. In Proceedings of Electronic Publishing (EP’96), 1996, Palo Alto, pp.
259-272.
[Brown97]
P.J Brown, J.D. Bovey, X. Chen. Context-Aware Applications: From the
Laboratory to the Marketplace. IEEE Personal Communications, Oct. 1997, Vol. 4, No. 5,
pp.58-64.
[Buch03]
S. Buchholz, A. Schill. Adaptation-Aware Web Caching: Caching in the
Future Pervasive Web. In 13th GI/ITG Conference Kommunikation in verteilten Systemen
(KiVS), Feb. 25.-28, 2003, Leipzig, Germany, pp. 55-66.
[Burn03]
I. Burnett, R. Van de Walle, K. Hill et al. MPEG-21: Goals and
Achievements. IEEE Multimedia, October-December 2003, Vol.10, No.4, pp.60-70.
[Bust01]
F. E. Bustamante. The Active Streams Approach to Adaptive Distributed
Applications and Services. Ph.D. Thesis. Atlanta, GA: Georgia Institute of Technology,
November 2001, 112 p.
[Cabral04]
L. Cabral, J. Domingue, E. Motta et al. Approaches to Semantic Web
Services: An Overview and Comparisons. In Lecture Notes in Computer Science Volume
3053: Proceedings of the First European Semantic Web Symposium (ESWS2004); 10-12
May 2004, Heraklion, Crete, Greece, Berlin / Heidelberg: Springer, pp. 225-239. ISBN:
978-
3-540-21999-6.
[Carde03]
V. Cardellini, M. Colajanni, R. Lancellotti et al. A distributed architecture
of edge proxy servers for cooperative transcoding. In Proc. of 3rd IEEE Workshop on Internet
Applications, June 2003, San Jose, CA, pp. 66–70.
[Carden05]
Y. Cardenas, J. Pierson, L. Brunie. Uniform Distributed Cache Service for
Grid Computing. In Sixteenth International Workshop on Database and Expert Systems
Applications (DEXA 2005), IEEE ed. Copenhagen, Denmark. pp. 351-355.
[Casati00]
F. Casati, S. Ilnicki, and L. Jin. Adaptive and dynamic service composition
in EFlow. In Lecture Notes in Computer Science, Vol. 1789: Proceedings of the 12th
International Conference on Advanced Information Systems Engineering (CAiSE), June
2000, Stockholm, Sweden, London (UK): Springer Verlag pp. 13-31. ISBN: 3-540-67630-9.
[Casati01]
F.Casati, M.Sayal, and M.-C.Shan. Developing e-services for composing e-
services. In Proceedings of the 13th International Conference on Advanced Information
Systems Engineering (CAiSE), June 2001, Interlaken, Switzerland, pp.171-186.
[Chaari05]
T Chaari, F. Laforest. SEFAGI: Simple Environment For Adaptable
Graphical Interfaces -Generating user interfaces for different kinds of terminals. In
Bibliography
219
Proceedings of the Seventh International Conference on Enterprise Information Systems
(ICEIS), 25-28 May 2005, Miami, USA, pp. 232-237. ISBN 972-8865-19-8.
[Chaari06]
T. Chaari, D. Ejigu, F. Laforest et al. Modeling and using context in
adapting applications to pervasive enviroments. In Proceedings of the IEEE International
Conference on Pervasive Services (ICPS’06), 26-29 June 2006, Lyon, France, pp.111-120.
[Chandra00]
S. Chandra, C.S. Ellis, A. Vahdat. Differentiated Multimedia Web Services
Using Quality Aware Transcoding. 19th Annual Joint Conference of the IEEE Computer and
Communication Societies (INFOCOM 2000), Mar. 2000, Tel Aviv, Israel, pp. 961–969.
[Chandra99]
S. Chandra and C.S. Ellis. JPEG Compression Metric as a Quality Aware
Image Transcoding. In 2nd Usenix Symposium on Internet Technologies and Systems
(USITS ’99), Oct. 1999, Berkeley, CA, USA, pp.81-92.
[Chang02]
C.-Y. Chang and M.-S. Chen. Exploring Aggregate effect with weighted
transcoding graphs for efficient cache replacement in transcoding proxies. In Proceedings of
the 18th IEEE International Conference on Data Engineering (ICDE-02), February 26 - March
1 2002, San Jose, CA, USA, pp. 383-392.
[Cheng02]
S.-W. Cheng, D. Garlan, B. R. Schmerl et al. Software architecture-based
adaptation for pervasive systems. In Lecture Notes In Computer Science, Vol. 2299:
Proceedings of the International Conference on Architecture of Computing Systems: Trends
in Network and Pervasive Computing, London, UK: Springer-Verlag, 2002, pp. 67-82. ISBN:
3-540-43409-7.
[Chi02]
C.-H. Chi, Y. Cao, and T. Luo. Scalable Multimedia Content Delivery on
Internet. In Proceedings 2002 IEEE International Conference on Multimedia and Expo
(ICME’02), August 2002, Lusanne, Switzerland, Vol. 1, pp. 1-4. ISBN: 0-7803-7304-9.
[Chinnici03]
R. Chinnici, M. Gudin, J. Moreau et al. Web Services Description
Language (WSDL) [on line], Version 1.2. W3C Working Draft 24 January 2003. Available
on: <http://www.w3.org/TR/wsdl/> (accessed on June 2003).
[Chris99]
L. Christianson and K. Brown. Rate Adaptation for Improved Audio
Quality in Wireless Networks, 1999 IEEE International Workshop on Mobile Multimedia
Communications (MoMuC’99), November 15-17, 1999, San Diego, CA, pp. 363-367.
[Cooltown]
Cooltown (HP) [on line]. Available on: <http://www.cooltown.hp.com>.
(Accessed on 21 May 2003).
[Cowan95]
C. Cowan, S. Cen, J. Walpole et al. Adaptive methods for distributed video
presentation. In ACM Computing Surveys, Special Issue on Multimedia Systems Symposium on Multimedia, December 1995, Vol. 27, No. 4, pp. 580-583.
220
Bibliography.
[Day02]
N. Day and J. M. Martinez. Introduction to MPEG-7 (v4.0). ISO/IEC
JTC1/SC29/WG11 N4675. MPEG Requirements Group, Jeju, March 2002.
[Dean02]
M. Dean, D. Connoly, F. Harmelen et al. Web Ontology Language (OWL)
Reference Version 1.0 Recent Trends and Developments [on line]. W3C Working Draft 12
November 2002. Available on: <http://www.w3.org/TR/2002/WD-owl-ref-20021112/>
[Demp97]
L. Dempsey, R. Russell and R. Heery. In at the shallow end: metadata and
cross-domain resource discovery, In Discovering online resources across the humanities: a
practical implementation of the Dublin Core, Paul Miller and Daniel Greenstein, eds. Bath:
UKOLN on behalf of the Arts and Humanities Data Service, 1997, pp. 63-71. ISBN 09516856-4-3.
[Dey01]
A. K. Dey. Understanding and Using Context. Personal and Ubiquitous
Computing Journal, 2001, Vol. 5, No. 1, pp. 4-7.
[Dey98]
A.K Dey. Context-Aware Computing: The CyberDesk Project. AAAI 1998
Spring Symposium on Intelligent Environments, Technical Report SS-98-02, 1998, pp. 51-54.
A.K. Dey, G.D. Abowd and P. J. Brown. Towards a Better Understanding
[Dey99a]
of Context and Context-Awareness. In Lecture Notes in Computer Science Vol. 1707: In
Proceedings of the 1st international symposium on Handheld and Ubiquitous Computing
(HUC’99), 27-29 Sept. 1999, Karlsruhe, Germany, pp. 304-307.
[Dey99b]
A.K. Dey, G.D Abowd, A. Wood. CyberDesk: A Framework for Providing
Self-Integrating Context-Aware Services. Knowledge-Based Systems, 1999, Vol. 11, pp. 313.
[DICOM01]
The Digital Imaging and Communications in Medicine (DICOM) Standard
Part 1-16 [on line]. Published by National Electrical Manufactures Association, 1300 N. 17th
Street,
Rosslyn,
Virginia
22209
USA,
2001.
Avaialble
on:
<http://medical.nema.org/dicom/2001.html>. (Accessed on 16 May 2004).
[Dijkstra59]
E. W. Dijkstra. A note on two problems in connection with graphs. Journal
of NumerMath, 1959, Vol. 1, pp. 269-271.
[Dogan04]
S. Dogan, S. Eminsoy, A. H. Sadka et al. Video Content Adaptation Using
Transcoding for Enabling UMA over UMTS. 5th Workshop on Image Analysis for
Multimedia Interactive Services, April 21-23, 2004, Instituto Superior Técnico, Lisboa,
Portugal.
Bibliography
[Dublin99]
221
Dublin Core. The Dublin Core: A Simple Content Description Model for
Electronic Resources [on line]. 1999. Available on: <http://purl.oclc.org/dc/>. (Accessed on
13 October 2004).
[EasyLiving]
EasyLiving
(Microsoft)
[on
line].
Available
on:
<http://www.research.microsoft.com/easyliving>. (Accessed on 21 May 2003).
[Elson01]
J. Elson, A. Cerpa. ICAP the Internet Content Adaptation Protocol [on
line]. Internet Draft, The ICAP Protocol Group, Jun 2001. Available on: <http://www.icap.
org/spec/icap_specification.txt>. (Accessed on 20 February 2003).
[Esler99]
M. Esler, J. Hightower, T. Anderson et al. Next Century Challenges: Data-
Centric Networking for Invisible Computing: The Portolano Project at the University of
Washington. In Proceedings of the 5th annual ACM/IEEE international conference on Mobile
computing and networking (MOBICOM'99), August 1999, Seattle, Washington, pp.256-262.
ISBN: 1-58113-142-9.
[Fensel02]
D. Fensel and C. Bussler. The Web Service Mode ling Framework WSMF.
In Proceeding of the NSF-EU Workshop on Database and Information Systems Research for
Semantic Web and Enterprises, April 2002, Georgia, USA, pp. 15-20.
[Fisher97]
B. Fisher, G. Agelidis, J. Dill et al. CZWeb: Fish-Eye Views for
Visualizing the World-Wide Web. Proc. of the 7th Int. Conf. on Human-Computer Interaction
(HCI International '97), 1997, Elsevier, Amsterdam, pp. 719-722.
[Flory06]
A. Flory, C. Verdier, S SASSI. Nouvelles Interfaces pour la représentation
de l'information médicale. In INFORSID 2006 (Systèmes d’information et services web), june
2006, Hammamet Tunisia.
[Foster99]
Foster and C. Kesselman (editors). The Grid Blueprint for a New
Computing Infrastructure. San Francisco, CA: Morgan Kaufmann Publishers Inc., 1999, 701
p. ISBN: 1558604758.
[Fox97]
A. Fox, S. D. Gribble, Y. Chawathe et al. Cluster-Based Scalable Network
Services. In Proc. of the 16th ACM Symposium on Operating System Principles, October
1997, Saint Malo, France, pp. 78–91.
[Fox96]
A. Fox, S. Gribble, E. Brewer, E. Amir. Adapting to Network and Client
Variation via On-Demand, Dynamic Distillation. In Proceedings ASPLOS-VII, October 1996,
Cambridge, MA, USA, pp. 160-170.
[Fox98a]
A. Fox, I. Goldberg, S. Gribble et al. Experience with TopGun Wingman:
A Proxy-Based Web Browser for the 3Com PalmPilot. In Proceedings of the IFIP
222
Bibliography.
International Conference on Distributed Systems Platforms and Open Distributed Processing
(Middleware ’98), Sept. 1998, Lake District, England, pp.407-426.
[Fox98b]
A. Fox, S. Gribble, Y. Chawathe et al. Adapting to Network and Client
Variation Using Infrastructural Proxies: Lessons and Perspectives. IEEE Personal
Communications, Special Issue on Adaptation, 1998, Vol.5, No. 4, pp. 10-19.
[Gaia]
Gaia (UIUC). [on line] Available on: <http://gaia.cs.uiuc.edu/>. (Accessed
on 21 May 2003).
[Gard01]
T. Gardner. An Introduction to Web Services [on line]. 02-October-2001.
Available on <http://www.ariadne.ac.uk/issue29/gardner/intro.html> (accessed on June 2004).
[Garl02]
D. Garlan, D. Siewiorek, A. Smailagic et al. Project Aura: Toward
Distraction-Free Pervasive Computing. IEEE Pervasive Computing, special Issue on
Integrated Computing Environments, April-June 2002, Vol.1, No.2, pp. 22-31. ISSN 15361268.
[Geld04]
M. Geldof. The Semantic Grid: will Semantic Web and Grid go hand in
hand? European Commission DG Information Society Unit “Grid technologies”, June 2004.
[Goeb01]
S. Goebel, S. Buchholz, T. Ziegert et al. Device Independent
Representation of Web-based Dialogs and Contents. In: Proceedings of the IEEE Youth
Forum in Computer Science and Engineering (YUFORIC'01), 2001, Valencia, Spain.
[Goff04]
M. K. Goff. The Scope of Network Distributed Computing [on line].
Chapter 3. In: Network Distributed Computing: Fitscapes and Fallacies, 27 February 2004.
Available
on:
<http://www.informit.com/content/images/0131001523/samplechapter/0131001523_ch03.pdf
>. (Accessed on December 7, 2005).
[Gribb01]
S. D. Gribble, M. Welsh, R. Behren et al. The Ninja architecture for robust
Internet-scale systems and services. Journal of Computer Networks, March 2001, Vol. 35, No.
4, pp. 473-497.
[Gross00]
H. Grosskreutz, G. Lakemeyer: cc-Golog: Towards More Realistic Logic-
Based Robot Controllers. In Proc. of the Seventeenth National Conference on Artificial
Intelligence and Twelfth Conference on Innovative Applications of Artificial Intelligence,
July 30 - August 3, 2000, Austin, Texas, USA. AAAI Press / The MIT Press, 2000, pp. p.476482. ISBN 0-262-51112-6.
[Grub95]
T. R. Gruber. Toward principles for the design of ontologies used for
knowledge sharing. International Journal of Human-Computer Studies, 1995, vol. 43, pp. 907928.
Bibliography
[Han98]
223
R. Han, P. Bhagwat, R. LaMaire et al. Dynamic adaptation in an image
transcoding proxy for mobile Web browsing. IEEE Personal communications, Dec. 1998,
Vol. 5, No. 6, pp. 8–17.
[Held02]
A. Held, S. Buchholz, A. Schill. Modeling of Context Information for
Pervasive Computing Applications. In Proceedings of the 6th World Multiconference on
Systemics, Cybernetics and Informatics (SCI), July 2002, Orlando, FL, USA.
[Hill03]
D. Hillmann. Using Dublin Core [on line]. 2003. available on
http://dublincore.org/documents/usageguide/ (Accessed on Dec. 7, 2004).
[Hori00]
M. Hori, G. Kondoh, and K. Ono. Annotation-based web content
transcoding. In Proceedings of the Ninth International World Wide Web Conference, May 1519 2000, Netherlands, Amsterdam, pp. 197—211.
[Horr02]
I. Horrocks. DAML+OIL: a reason-able web ontology language. Proc.
Intern. Conf. on Advances in Database Technology (EDBT), March 2002, Prague, Czech
Republic, pp. 2-13.
[Henr01]
K. Henricksen K, J. Indulska J, A. Rakotonirainy. Infrastructure for
Pervasive computing: Challenges. In Proceedings of Workshop on Pervasive Computing
INFORMATIK 01, Viena, September 2001.
[Hous96]
B. C. Housel and D. B. Lindquist. WebExpress: A System for Optimizing
Web Browsing in a Wireless Environment. In Proceedings of ACM/IEEE of the Second
Annual International Conference on Mobile Computing and Networking (MobiCom '96),
November 1996, Rye, New York, pp. 108—116.
[Huan04]
J.L. Huang, M.S. Chen and H.P. Hung. A QoS- Aware Transcoding Proxy
Using On-demand Data Broadcasting. In Proceedings IEEE INFOCOM 2004, The 23rd
Annual Joint Conference of the IEEE Computer and Communications Societies, 7-11 March
2004, Hong Kong, China.
[Hunt98]
J. Hunter, R. Iannella. The Application of Metadata Standards to Video
Indexing. In Proceedings of Second European Conference on Research and Advanced
Technology for Digital Libraries ECDL'98, 19 - 23 September, 1998, Crete, Greece.
[Hutter05]
A. Hutter, P. Amon, G. Panis et al. Automatic Adaptation of Streaming
Multimedia Content in A Dynamic and Distributed Environment. Proceedings of the IEEE
Int’l Conference on Image Processing (ICIP 2005 2005), 11-14 Sept. 2005, Genova, Italy, pp.
716-719.
[IBM00]
2000.
IBM Inc. Internet Transcoding for Universall Access [on line]. September
Available
on:
224
Bibliography.
<http://www.research.ibm.com/networked_data_systems/transcoding/index.html>. (Accessed
on 10 January 2004).
[IBMPC]
IBM Pervasive Computing [on line]. Available on: <http://www-
3.ibm.com/pvc/pervasive.shtml> (Accessed on 11 October 2003).
[IBMTR]
IBM Think Research: Pervasive Computing [on line]. Available on:
<http://www.research.ibm.com/thinkresearch/pervasive.shtml>. (Accessed on 10 October
2003).
[IBMWS]
IBM Web services tutorial [on line]. Available on: <http://www-
106.ibm.com/developerworks/webservices/> (Accessed on 10 October 2003).
[Ihde01]
S.C. Ihde, P.P. Maglio, J. Meyer et al. Intermediary-based transcoding
framework. IBM Systems Journal 2001, Vol. 40, No. 1, pp. 179-192.
[Inou97]
J. Inouye., S. Cen, C. Pu et al. System Support for mobile multimedia
Applications. In Proc. 7th Intl. Workshop on Network and Operating Systems Support for
Digital Audio and Video (NOSSDAV '97), May 1997, St. Louis, Missouri, USA, pp. 143154.
[Intel]
Intel
QuickWeb
[on
line].
Available
on:
<http://www.intel.com/quickweb>. (Accessed on 22 May 2003).
[Ipiqa01]
Diego Lspez de Ipiqa, Sai-Lai Lo. Sentient Computing for Everyone. In
Proceedings of the The Third IFIP WG 6.1 International Working Conference on Distributed
Applications and Interoperable Systems (DAIS'2001), 17-19 September, 2001, Krakow,
Poland, pp. 41-54.
[ISO03]
ISO 2003. Information and documentation — The Dublin Core metadata
element set. ISO 15836:2003 (E) [on line]. 23 February 2003. Available on:
<http://www.niso.org/international/SC4/n515.pdf>. (Accessed on 17 October 2004).
[ISO04]
ISO/IEC 21000-7:2004. Information technology — Multimedia framework
(MPEG-21) — Part 7: Digital Item Adaptation, First Edition, October 2004.
[ITMF03]
ISO/IEC 21000-2:2003. Information Technology-Multimedia Framework
(MPEG-21)-Part 2: Digital Item Declaration, March, 2003.
[Jaco97]
S.Jacobs and A.Eleftheriadis. Real-time dynamic rate shaping and control
for internet video applications. In Proceedings of Workshop on Multimedia Signal Processing,
June 23-25 1997, Princeton, NJ, pp. 23-25.
[Jaza02]
M. Jazayeri. On the way to pervasive computing. In Proceedings of XVI
Brazilian Symposium on Software Engineering (SBES'2002), 16-18 October, 2002, Rio
Grande do Sul – Brazil.
Bibliography
[Joshi00]
225
A. Joshi. On proxy agents, mobility and web access. ACM/Baltzer Journal
of Mobile Networks and Applications, 2000, Vol. 5, No. 4, pp. 233-241.
[Kazai03]
G. Kazai, M. Lalmas, M-L. Bourguet et al. Using Metadata to Provide
Scalable Broadcast and Internet Content and Services. Proceedings of the 4th European
Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS 2003), April
2003, London, pp. 487-492.
[Kim03]
J.-G. Kim, Y. Wang, and S.-F. Chang. Content-adaptive utility-based video
adaptation. In Proc. of IEEE Int'l Conference on Multimedia & Expo (ICME), July 6-9, 2003,
Baltimore, MD, pp. 281-284.
[Kist92]
J. Kistler and M.Satyanarayanan. Disconnected Operation in the Coda File
System. ACM Transcations on Computers, Vol.10, No.1, Feb. 1992, pp. 3-25.
[Klyne01]
G.
Klyne,
F.
Reynolds,
C.
Woodrow
et
al.
Composite
Capability/Preference Profiles (CC/PP): Structure and Vocabularies. W3C Working Draft [on
line]. Mar 15, 2001. Available on: <http://www.w3.org/TR/2001/ WD-CCPP-struct-vocab20010315/>. (Accessed on 18 October 2003).
[Klyne99]
G. Klyne. A Syntax for Describing Media Feature Sets. RFC 2533 [on
line]. Mar 1999. Available on <: http://www.faqs.org/rfcs/rfc2533.html>. (Accessed on 19
octobre 2003).
[Kosch03]
H. Kosch. Distributed Multimedia Database Technologies supported by
MPEG-7 and MPEG-21. CRC Press LLC, November 2003, 248 p. ISBN: 0849318548.
[Kumar93]
P. Kumar and M.Satyanarayanan. Supporting Application-Specific
Resolution in an Optimistically Replicated File System. In Proceedings of the 4th Workshop
on Workstation Operating systems, October 1993, Napa, CA, pp. 66-70.
[Kumar95]
P. Kumar and M.Satyanarayanan. Flexible and Safe Resolution of File
Conflicts. In Proceedings of the 1995 Winter Usenix Conference, January 1995, New Orleans,
LA, pp. 95-106.
[Kwan02]
W. M. Kwan. A Distributed Proxy System for Functionality Adaptation in
Pervasive Computing Environments. Master’s Thesis, Department of Computer Science and
Information Systems, The University of Hong Kong, August 2002.
[Lass99]
O. Lassila, R. Swick. Resource Description Framework (RDF) Model and
Syntax Specification. W3C Recommendation [on mine]. Feb 22, 1999. Available on:
<http://www.w3.org/TR/1999/REC-rdf-syntax- 19990222/>. (Accessed on 20 October 2003).
[Lee01]
S. J. Lee, W. Y. Ma, and B. Shen. Interactive video caching and delivery
using video abstraction and summarization. Proc. International Workshop on Web Caching
226
Bibliography.
and Content Distribution (WCW’01), 20-22 June 2001, Boston University, Boston,
Massachusetts, USA.
[Lee03]
D. G. Lee, D. Panigrahi, and S. Dey. Network-Aware Image Data Shaping
for Low-Latency and Energy-Efficient Data Services over the Palm Wireless Network. In
Proc. of World Wireless Congress (3G Wireless Conference), 27-30 May 2003, San Francisco
/ Silicon Valley, U.S.A.
[Lei01a]
Z. Lei and N.D. Georganas. Context-based Media Adaptation in Pervasive
Computing. In Proc. of Canadian Conference on Electrical and Computer Engineering
(CCECE), Toronto, May 2001.
[Lei01b]
Z. Lei. Media transcoding for pervasive computing. ACM Press: Vol. 9. In
Proceedings of the ninth ACM international conference on Multimedia, 2001, Ottawa,
Canada, pp. 459-460. ISBN: 1-58113-394-4.
[Lein04]
B.M.Leiner, V.G.Cerf, D.D.Clark, et al. A Brief History of the Internet, 10
Dec 2003 [on line]. Available on <http://www.isoc.org/internet/history/brief.shtml>
(Accessed on January 2004)
[Leml01]
T. Lemlouma and N. Layaïda. A Framework for Media Resource
Manipulation in an Adaptation and Negotiation Architecture. Research Report, Opéra Project,
INRIA Rhône Alpes, August 2001.
[Leml02]
T. Lemlouma, N. Layaïda. Universal Profiling for Content Negotiation and
Adaptation in Heterogeneous Environments. W3C Workshop on Delivery Context, 4-5 March
2002, W3C/INRIA Sophia-Antipolis, France.
[Leml03]
T. Lemlouma and N. Layaïda. SMIL Content Adaptation for Embedded
Devices. SMIL Europe 2003, Paris, February 12-14, 2003.
[Leml04]
T. Lemlouma. Architecture de Négociation et d'Adaptation de Services
MultiMedia dans des Environnements Hétérogènes. PhD Thesis defended on June 9, 2004,
INPG, Grenoble, France.
[Leve97]
H.J Levesque, R. Reiter, Y. Lesperance et al. GOLOG: A logic
programming language for dynamic domains. Journal of Logic Programming, 1997, Vol. 31,
No.1-3, pp. 59-83.
[Leym01]
F. Leymann. Web Services Flow Language (WSFL 1.0) [on line].
Technical report, 2001. Available on: <http://www.ibm.com/software/solutions/webservices/
pdf/WSFL.pdf> (Accessed on 11 June 2004).
[Libsie04]
M. Libsie. Metadata Supported Content Adaptation in Distributed
Multimedia Systems. PhD thesis, defended on June 2004, University Klagenfurt Austria.
Bibliography
[Lilj95]
227
M. Liljeberg, T. Alanko, M. Kojo et al. Optimizing World-Wide Web for
Weakly Connected Mobile Workstations: An Indirect Approach. In Proc. 2nd International
Workshop on Services in Distributed and Networked Environments (SDNE'95) 5-6 June
1995, Whistler, Canada, pp. 132-139. ISBN: 0-8186-7092-4.
[List02]
C.List. Information Research. 2nd edition, Dubuque, IA: Kendall/Hunt
Publishing Co.2002. Located in the Reserve Room at Moon Library - Call Number Reserve Z
711.2 .L57 2002.
[Lomo04]
G. Lomow and E. Newcomer. Introduction to SOA with Web Services. In:
E. Newcomer and G. Lomow. Understanding SOA with Web Services (Independent
Technology Guides). Addison-Wesley Professional Sample Chapter is provided courtesy of
Addison Wesley Professional, 2004, 480 p. ISBN: 0321180860.
[Lu99]
G. Lu. Multimedia Database Management Systems. Boston/London:
Artech House Publishers, 1999, 373 p. ISBN: 0-89006-342-7.
[Lum02a]
W.-Y. Lum and F.C.M. Lau. A context-aware decision engine for content
adaptation. IEEE Pervasive Computing, July-September 2002, Vol.1 No. 3, pp. 41-49.
[Lum02b]
W.-Y. Lum and F.C.M. Lau. On balancing between transcoding overhead
and spatial consumption in content adaptation. Proc. of ACM Mobicom 2002, Sept. 2002,
Atlanta, USA, pp. 239-250.
[Ma00]
W.-Y. Ma, I. Bedner, G. Chang et al. A Framework for Adaptive Content
Delivery in Heterogeneous Network Environments. In Proceedings of Multimedia Computing
and Networking (MMCN'00), 2000, San Jose, California, USA, pp.86-100.
[Maga04]
J. Magalhães, F. Pereira. Using MPEG standards for multimedia
customization. Signal Processing: Image Communication, 2004, Vol. 19, No. 5, pp. 437-456.
ISSN 0923-5965.
[Mahe02]
A. Maheshwari, A. Sharma, K. Ramamritham et al. TranSquid:
Transcoding and Caching Proxy for Heterogeneous E-Commerce Environments. In
Proceedings of the 12th IEEE Workshop on Research Issues in Data Engineering (RIDE ’02),
Feb. 2002, San Jose, CA. pp. 50-59.
[Manj02]
B. Manjunath, P. Salembier, and T. Sikora. Introduction to MPEG 7:
Multimedia Content Description Language. Ed. John Wiley and Sons, Ltd, 2002, West
Sussex, England, p. 352. ISBN: 0471486787.
[Manu04]
K. Manuel. Information Literacy Textbook [on line]. Printed 12/13/2004.
On
line
Textbook.
Available
on:
228
Bibliography.
<http://lib.nmsu.edu/instruction/lsc311/textbook/information.pdf>
(Accessed
on
10
September 2005).
[Marg01]
M. Margaritidis and G.C. Polyzos. Adaptation Techniques for Ubiquitous
Internet Multimedia. Wireless Communications and Mobile Computing, 2001, John Wiley &
Sons, Vol. 1, No. 2, pp. 141-164.
[Marr02]
K. Marriott, B. Meyer, and L. Tardif. Fast and Efficient client-Side
Adaptivity for SVG. ACM Press, 2002: In Proc. of WWW 2002, May 2002, Honolulu,
Hawaii, USA, pp. 496–507.
[Mart02]
J. M. Martínez, R. Koenen and F. Pereira. MPEG-7: the generic
multimedia content description standard. (part 1) IEEE Multimedia, 9 (2), April-June 2002,
pp. 78-87.
[Mart03]
J. M. Martínez. MPEG-7 Overview. ISO/IEC JTC1/SC29/WG11 N5525,
March 2003.
[McCan97]
S.McCanne, M. Vetterli, and V. Jacobson. Low-complexity video coding
for receiver-driven layered multicast. IEEE JSAC, August 1997, Vol.16, No.6, pp. 983-1001.
[McDe00a]
D.McDermott. The 1998 AI planning systems competition. AI Magazine,
2000, Vol. 21, No.2, pp. 35-55.
[McDe00b]
D. McDermott. Estimated-Regression Planning for Interactions with Web
Services. Proceedings of the AI Planning Systems Conference (AIPS'02), June 2002.
[McIl02]
S. McIlraith and T. C. Son. Adapting Golog for composition of Semantic
Web services. In Proceedings of the 8th International Conference on Knowledge
Representation and Reasoning (KR2002), April 2002, Toulouse, France, pp. 482-493.
[MDS03]
MPEG MDS Group, MPEG-21 Multimedia Framework-Part 7: Digital
Item Adaptation (final Committee Draft) ISO/MPEG N5845, July 2003.
[Medj03]
B. Medjahed, A. Bouguettaya, and A. K. Elmagarmid. Composing Web
services on the Semantic Web. The VLDB, November 2003, Journal Vol. 12, No.4, pp. 333351.
[MFTR05]
Media Feature Tags Registry (last updated November 2005) [on line].
Available on: http://www.iana.org/assignments/media-feature-tags (Accessed on 14 October
2004).
[Milan04]
N. Milanovic, M. Malek. Current Solutions for Web Service Composition.
IEEE Internet Computing, November 2004, Vol.8, No.6, pp.51-59.
Bibliography
[Mobi02]
229
MobiXtar Rich Media Service Center. [on line]. April 5, 2002. Available
on: <http://products.datamation.com/wireless/dt/1017134171.html>. (Accessed on 23 October
2003).
[Mogul01]
J.C. Mogul. Server-Directed Transcoding. Computer Communications,
Feb. 2001, Vol. 24, No.2, pp.155-162.
[Mohan99]
R. Mohan, J.R. Smith and C.S. Li. Adapting Multimedia Internet Content
for Universal Access. IEEE Trans. on Multimedia, 1999, Vol. 1, No. 1, pp.104–114.
[More84]
R. Moreau. The Computer Comes of Age: The People, the Hardware, and
the Software, translated by J. Howlett (Cambridge: MIT Press, 1984).
[Morphis]
Morphis Open Source Java Wireless Content Transcoder [on line].
Available on: <http://www.morphis.org/>. (Accessed on 26 October 2003).
[Mosch99]
C.J.P. Moschovitis, H. Poole, T.Schuyler et al. History of the Internet: A
Chronology, 1843 to the Present. ABC-Clio 1999. ISBN 1576071189.
[Motta03]
E. Motta, J. Domingue, L. Cabral, and M. Gaspari. IRS-II: A Framework
and Infrastructure for Semantic Web Services. In Lecture Notes in Computer Science: Vol
2870: In Proceedings of the International Semantic Web Conference, October 2003, Sanibel
Island, FL, USA, Springer Verlag, pp. 306-318.
[Mumm95]
L.
Mummert,
M.
Ebling,
M.Satyanarayanan.
Exploiting
Weak
Connectivity for Mobile File Access. Symposium on Operating System Principles, December
1995, Copper Mountain, Colorado, pp. 143-155.
[Nagao01]
K. Nagao, Y. Shirai, and K. Squire. Semantic annotation and transcoding:
Making web content more accessible. IEEE Multimedia, April-June 2001, pp. 69--81.
[Nand99]
T. Nandagopal, TE. Kim, P Sinha. Et al. Service Differentiation Through
End-to-End Rate Control in Low Bandwidth Wirless Packet Networks. Proceedings of the 6th
International Workshop on Mobile Multimedia Communications; November 1999, San
Diego, CA, USA.
[Nau99]
D. S. Nau, Y. Cao, A. Lotem et al. SHOP: Simple hierarchical ordered
planner. In Thomas Dean, editor, Proceedings of the International Joint Conference on
Artificial Intelligence (IJCAI). Morgan Kaufmann Publishers, July 31–August 6 1999, pp.
968–973.
[NISO04]
National Information Standards Organization (NISO). Understanding
Metadata. NISO Press 2004, Bethesda, MD, USA, p.1.
[Noble00]
B. Noble. System Support for Mobile, Adaptive Applications. IEEE
Personal Communications, February 2000.
230
Bibliography.
[Noble95]
B.D. Noble, M. Price, and M. Satyanarayanan. A programming Interface
for Application Aware adaptation in mobile computing. Proceedings of the Second USENIX
Symposium on Mobile and Location-Independent Computing, Apr 1995, Ann Arbor, MI,
USA.
[Noble97]
B.D. Noble, M. Satyanarayanan, D. Narayanana et al. Agile application-
aware adaptation for mobility. In Proceedings of the 16th ACM Symposium on Operating
Systems Principles, December 1997, Vol. 31, No. 5, pp. 276-287.
[Norm98]
D.A. Norman. The invisible computing. Cambrideg, MA, MIT Press, 1998.
[OMG02]
Object Management Group. The Common Object Request Broker
Architecture (CORBA): Core Specification. Revision 3.0.2. OMG Document Technical
Report formal/2002-12-02, Object Management Group, Dec. 2002, Framingham, MA, USA.
[OnLine]
OnLineAnywhere
[on
line].
Available
on:
<http://www.onlineanywhere.com>. (Accessed on 28 May 2003).
[Ooi01]
W. T. Ooi, R. v. Renesse. Distributing Media Transformation Over
Multiple Media Gateways. In Proceedings of the 9th ACM International Multimedia
Conference, September 2001, Ottawa, Canada, pp. 159-168.
[Opera04]
Opera press release. The Web adapted to every screen size: Opera solves
the problem of rendering Web content on medium-sized screens [on line]. Jan. 20, 2004.
Available on: <http://www.opera.com/pressreleases/en/2004/01/20/> (Accessed on 9 July
2004).
[Oracle03]
White
Oracle Application Server Wireless: Complete Mobile Platform. Technical
Paper
[on
line].
August
2003.
Available
on:
<http://www.otn.oracle.com/products/iaswe/OracleAS_Wireless_Enterprises_TWP.pdf>
(Accessed on 27 July 2004).
[Ort05]
E. Ort. Service-Oriented Architecture and Web Services: Concepts,
Technologies, and Tools. Sun Microsystems, April 2005.
[Orte97]
A. Ortega, F. Carignano, S. Ayer, et al. Soft Caching: Web cache
management techniques for image. In IEEE Signal Processing Society Workshop on
Multimedia Signal Processing, June 23-25, 1997, Princeton, New Jersey, USA, pp. 475-480.
[Oxygen]
Oxygen (MIT) [on line]. Available on: <http://oxygen.lcs.mit.edu>
(Accessed on 23 May 2003).
[Paol02]
M. Paolucci, T. Kawamura, T. R. Payne et al. Semantic matching of web
services capabilities. In First International Semantic Web Conference, June 2002, Sardinia,
Italy, pp. 333-347.
Bibliography
[Pascoe98]
231
J. Pascoe. Adding Generic Contextual Capabilities to Wearable Computers.
2nd International Symposium on Wearable Computers, 1998, Pittsburgh, PA, USA, pp. 9299. ISBN 0-8186-9074-7.
[Peak98]
S. Peak and J. R. Smith. Detecting Image Purpose in World-Wide Web
Documents. Symposium on Electronic Imaging: Science and Technology - Document
Recognition, IS&T/SPIE, 1998, p. 151-158.
[Pere05]
F. Pereira. Content and Context: Two worlds to bridge. In Int. Workshop
on Content-Based Multimedia Indexing, CBMI'2005, June 2005, Riga.
[Perk01]
A. Perkis, Y. Abdeljaoued, C. Christopoulos, et al. Universal multimedia
access from wired and wireless systems. Transactions on Circuits, Systems and Signal
Processing, Birkhauser Boston, 2001, Vol. 10, No. 3, pp. 387-402.
[Portolano]
Portolano
(University
of
Washington)
[on
line].
Available
on:
<http://www.cs.washington.edu/research/portolano> (Accessed on 27 May 2003).
[ProxiNet]
ProxiNet [on line]. Available on: <http://www.proxinet.com>. (Accessed
on 27 July 2004).
[Ramos05]
A. Carrillo Ramos, J. Gensel, M. Villanova-Oliver, H. Martin. PUMAS: a
Framework based on Ubiquitous Agents for Accessing Web Information Systems through
Mobile Devices. 20th Annual ACM Symposium on Applied Computing - Handheld
Computing (SAC2005), March 13 -17, 2005, Santa Fe, New Mexico. ACM Press, New York,
NY, 2005, Vol. 2, pp 1003-1008.
[Rao04]
Jinghai Rao, Xiaomeng Su: A Survey of Automated Web Service
Composition Methods. Semantic Web Services and Web Process Composition (SWSWPC),
First International Workshop, July 6, 2004, San Diego, CA, USA, pp. 43-54.
[Reim02]
M. Reimer. Design and Implementation of a mobile Personal Health
Monitor. Master thesis. Technische Universität Darmstadt, Dept of Computer Science, 2002.
[Rodd98]
T. Rodden, K. Cheverst, K. Davies et al. Exploiting Context in HCI Design
for Mobile Systems. In Proceedings of the First Workshop on Human Computer Interaction
for Mobile Devices, May 21-23, 1998, Staffordshire University, UK, pp.12-17.
[Rodr01]
J. Rodriguez, B. Dakar, F. L. Marras, et al. Transcoding in WebSphere
Everyplace Suite. Chapter 11. In: New Capabilities in IBM WebSphere Transcoding
Publisher Version 3.5 Extending Web Applications to the Pervasive World. IBM Redbooks
[on
line],
30
May
2001,
446
p.
Available
on:
http://www.redbooks.ibm.com/redbooks/pdfs/sg246233.pdf (Accessed on 16 May 2003).
ISBN 0738421545.
232
[Roure03]
Bibliography.
D. De Roure. Semantic Grid and Pervasive Computing. First GGF
Semantic Grid Workshop at 9th Global Grid Forum (GGF9), October 2003, Chicago, USA.
[Ryan97]
N. Ryan, J. Pascoe, D. Morse. Enhanced Reality Fieldwork: the Context-
Aware Archaeological Assistant. In Gaffney,V., van Leusen, M., Exxon, S. (eds.) Computer
Applications in Archaeology, 1997.
[Saha03]
D. Saha, A. Mukherjee. Pervasive Computing: A Paradigm for the 21st
Century, IEEE Computer, IEEE Computer Society Press, March 2003, pp. 25-31.
[Satya02]
M. Satyanarayanan. Evolution of the Coda File System. To appear in ACM
Trans. Computer Systems, ACM Press, New York, 2002.
[Satya96]
M. Satyanarayanan. Mobile Information Access. IEEE Personal Comm.,
Feb. 1996, Vol. 3, No. 1, pp. 26–33.
[Schi94a]
B. Schilit, M. Theimer. Disseminating Active Map Information to Mobile
Hosts. IEEE Network, 1994, Vol.8 No.5 pp.42-47.
[Schi94b]
B. Schilit, N. Adams, and R. Want. Context-aware computing applications.
In Proceedings of the 1st International Workshop on Mobile Computing Systems and
Applications, 1994.
[Schu00]
H. Schuster, D. Georgakopoulos, A. Cichocki, and D. Baker. Modeling and
composing service-based and reference process-based multi-enterprise processes. In
Proceeding of 12th International Conference on Advanced Information Systems Engineering
(CAiSE), Stockholm, Sweden, June 2000. Springer Verlag.
[Sentient]
Sentient
Computing
[on
line].
Available
on:
http://www.uk.research.att.com/spirit (Accessed on 19 March 2003).
[Shafer98]
Steve Shafer et al. The New EasyLiving Project at Microsoft Research:
Joint DARPA/NIST Smart Spaces Workshop, July 30-31, 1998, Gaithersburg, Marymand.
[Shaha01]
N. Shaha, A. Desai, M. Parashar. Multimedia Content Adaptation for QoS
Management over Heterogeneous Networks. In Proc of International Conference on Internet
Computing 2001, 2001, Nevada, USA, pp. 642-648.
[Shan00]
T. Shanableh and M. Ghanbari. Heterogeneous video transcoding to lower
spatio-temporal resolutions and different encoding formats. IEEE Trans. Multimedia, June
2000, Vol. 2, pp. 101-110.
[Shapiro01]
May 2001.
R. Shapiro. A comparison of XPDL, BPML and BPEL4WS, Cape Visions,
Bibliography
233
[Shen06]
H. Shen, X. Sun, F. Wu et al. Scalable Video Adaptation for IPTV. Internet
Protocol TeleVision (IPTV) workshop in conjunction with WWW '06, May 2006, Edinburgh,
Scotland, United Kingdom.
[Singh04]
A. Singh, A. Trivedi, K. Ramamritham et al. PTC: Proxies that Transcode
and Cache in Heterogeneous Web Client Environments. World Wide Web Journal, 2004, Vol.
7, No. 1, pp. 7-28.
[Sirin04a]
E. Sirin and B. Parsia. Planning for semantic web services. In Semantic
Web Services Workshop at 3rd International Semantic Web Conference (ISWC2004), 2004.
[Sirin04b]
E. Sirin, B. Parsia, D.Wu et al.. HTN planning for web service composition
using SHOP2. Journal of Web Semantics, 2004, Vol. 1, No. 4, pp.377-396.
[Smith00]
J. R. Smith. Interoperable Content-based Access of Multimedia in Digital
Libraries.In Proceedings of the First DELOS Network of Excellence Workshop on
Information Seeking, Searching and Querying in Digital Libraries, 11-12 December 2000
Zurich, Switzerland.
[Smith00]
J. R. Smith and B. Lugeon. A Visual Annotation Tool for Multimedia
Content Description. Proc. SPIE Photonics East, Internet Multimedia Management Systems,
November 2000, Boston, MA, USA, pp. 49-59.
[Smith04]
Language
M. K. Smith, C. Welty and D. L. McGuinness. OWL Web Ontology
Guide.
W3C
Recommendation,
2004
[on
line].
Available
on:
<http://www.w3.org/TR/2004/REC-owl-guide-20040210/> (Accessed on 20 September
2005).
[Smith98a]
J. R. Smith, R. Mohan, C.-S. Li. Content-based Transcoding of Images in
the Internet. In Proceedings of the 1998 IEEE International Conference on Image Processing
(ICIP-98), 4-7 October 1998, Chicago, Illinois. IEEE Computer Society, 1998, Vol. 3, pp. 711. ISBN 0-8186-8821-1.
[Smith98b]
J.R Smith, R.Mohan, C.S. Li. Transcoding Internet Content for
Heterogenouse Client Devices. Proceeedings IEEE International Symposium on Circuits and
Systems (ISCAS), June 1998, Monterey, California, Vol.3, pp. 599-602.
[Smith99]
J.R. Smith, R.Mohan, and C.S. Li. Scalable Multimedia Delivery for
Pervasive Computing. In Proceedings of ACM Multimedia Conference (Multimedia 99), Oct.
-Nov., 1999, Orlando, Fl, USA, pp. 131-140.
[Solla02]
T. Sollazzo, S. Handschuh, S. Staab et al. Semantic Web service
architecture-evolving Web service standards toward the Semantic Web. In Proceedings of the
234
Bibliography.
15th International FLAIRS Conference, 16-18 May 2002, Pensacola, Florida, USA, pp. 425429.
[Sow01]
D. M. Sow, G. Banavar, J. S. Davis II et al. Preparing the Edge of the
Network for Pervasive Content Delivery. Workshop on Middleware for Mobile Computing
(in association with IFIP/ACM Middleware 2001 Conference), 16 November 2001,
Heidelberg, Germany.
[Spgyglass]
Spgyglass-Prism [on line]. Available on: <http://www.sphyglass.com>
(Accessed on 21 July 2003).
[Stemm98]
M. Stemm and R.H. Katz. Verical handoffs in wireless overlay networks.
ACM mobile Networking (MONET), Special issue on Mobile Networking in the Internet,
1998, Vol. 3, No.4, pp. 335–350.
[Sun]
Sun Java System Portal Server Mobile Access [on line]. Available on:
<http://wwws.sun.com/software/products/portal_ma/index.html> (Accessed 19 June 2004).
[Sun03]
H. Sun, A. Vetro, K. Asai. Resource Adaptation Based on MPEG-21
Usage Environment Descriptions. IEEE International Symposium on Circuits and Systems
(ISCAS), 25-28 May 2003, Bangkok, Thailand, Vol. 2, pp. 536-539.
[Szyp02]
C.
Szyperski.
Component
Software:
Beyond
Object-Oriented
Programming. 2nd Edition. Harlow, UK: Addison-Wesley/ ACM Press, 2002, p. 589. ISBN 0201-74572-0.
[Taub94]
D.Taubman, A. Zakhor. Multi-rate 3D sub-band coding of video. IEEE
Trans. on Image Processing, Sept. 1994, Vol.3, No.5, pp.572-588.
[Tayl03]
C. Taylor. An Introduction to Metadata. University of Queensland Library,
July 2003 [on line]. Available on: <http://www.library.uq.edu.au/iad/ctmeta4.html>
[That01]
Corporation,
S. Thatte. XLANG, Web services for business process design. Microsoft
2001
[on
line].
Available
on:
<http://www.gotdotnet.com/team/xmlwsspecs/xlang-c/default.htm> (Accessed on 14 June
2004).
[Turn03]
M. Turner, D. Budgen, and P. Brereton. Turning software into a service.
New York: IEEE Computer Society, October 2003, Vol. 36, No. 10, pp. 38-44. ISSN 00189162.
[TV02]
TV-Anytime Provisional Specification 1.3 (Sept 2002) [on line]. Available
on: <http://www.tv-anytime.org> (Accessed on 18 May 2003).
Bibliography
[Usch96]
235
M. Uschold and M. Gruninger. Ontologies: Principles, Methods and
Applications. Knowledge Engineering Review, February 1996, Vol. 11, No. 2, pp. 93-155.
[Vela04]
C.A. Velasco, Y. Mohamad, A.S. Gilman et al. Universal Access to
Information Services - the Need for User Information and its Relationship to Device Profiles.
In: Gulliksen J, Harker S, Vanderheiden G (eds), Special issue on guidelines, methods and
processes for software accessibility. Universal Access in the Information Society, 2004, Vol.
3, No. 1, pp. 88-95. ISSN: 1615-5289
[Verd06]
C. Verdier. Health information systems: from local to pervasive medical
data. Santé et Systèmique, Hermès ed. 2006.
[Vetro02]
A. Vetro, Y. Wang, and H. Sun. Rate-distortion optimized video coding
considering frameskip. In Proc. IEEE ICIP, Oct. 2002, Rochester, NY.
[Ward97]
A. Ward, A. Jones, A. Hopper. A New Location Technique for the Active
Office. IEEE Personal Communications, 1997, Vol. 4, No. 5, pp. 42-47.
[Wee03]
S. Wee and J. Apostolopoulos. Secure scalable streaming and secure
transcoding with JPEG-2000. IEEE International Conference on Image Processing (ICIP), 1417 Sept. 2003, Barcelona, Spain, Vol.1, pp. 205-212. ISBN: 0-7803-7750-8.
[Weer02]
BPEL4WS,
S.Weerawarana,
Part
1,
Aug
C.
2002.
Francisco.
[on
Business
line].
Processes:
Availabile
on
Understanding
:
http://www-
128.ibm.com/developerworks/webservices/library/ws-bpelcol1/(Accessed on 16 October
2003)
[Weiser91]
M. Weiser. The Computer for the 21st Century. Scientific American,
September, 1991, Vol. 265, No. 3, pp.94-104.
[Wing01]
T. Winograd. Architecture for Context. Human Computer Interaction
Journal, 2001, Vol. 16, pp. 401-419.
[Wolb02]
D. Wolbers. Information Society: From Cave Walls to the Internet. Course
Material
2002
[on
line].
Available
on:
<http://smccd.net/accounts/skylib/lsci100/lesson1_2.htm> (Accessed on September 12, 2005).
[Wu03]
D. Wu, E. Sirin, B. Parsia et al. Automatic web services composition using
SHOP2. In Proceedings of Planning for Web Services Workshop in ICAPS 2003, 10 June
2003, Trento, Italy.
[Yarvis01]
M. Yarvis. Conductor: Distributed Adaptation for Heterogeneous
Networks. Ph.D. Dissertation, UCLA Department of Computer Science, November 2001.
[Yau01]
S.S. Yau and F. Karim. Context-Sensitive Distributed Software
Development for Ubiquitous Computing Environments. Proc. 25th IEEE International
236
Bibliography.
Computer Software and Applications Conference (COMPSAC 2001), 8-12 Oct. 2001,
Chicago, USA, pp. 263-268. ISBN: 0-7695-1372-7.
[Yosh97]
C. Yoshikawa, B. Chun, P. Eastham et al. Using smart clients to build
scalable services. In Proc. Winter 1997 USENIX Technical Conf., 6-10 January 1997,
Anaheim, California, USA, pp. 105-117.
[Zang00]
H. Zang. Adaptive content delivery: A new research area in media
computing. Proceedings of the 2000 International Workshop on Multimedia Data Storage,
Retrieval, Integration and Applications, 13-15 January 2000, Hong Kong, China
[Zenel97]
B. Zenel, D. Duchamp. A general purpose proxy filtering mechanism
applied to the mobile environment. Proceedings of MOBICOM’97, September 1997,
Budapest, Hungary, pp. 248-259.
Curriculum Vitae
Nom
Prénom
Date et lieu de naissance
Etat civil
Nationalité
Adresse
Girma
Berhe
le 06 Janvier 1974 à Adigrat (Ethiopie)
Célibataire
Ethiopienne
INSA de Lyon, LIRIS Bat.501 (Bat. Blaise Pascal)
20 Avenue Albert Einstein, 69621 Villeurbanne, France
Tel. +33 (0)4 72 43 63 48
Fax: +33 (0)4 72 43 87 13
Formation
• Doctorant en informatique au LIRIS, INSA de Lyon (Oct.2002-présent).
• Master en informatique, School of Information Study for Africa, Université d’AddisAbeba, Addis-Abeba, Ethiopie: (Sept. 1999- Juillet 2001).
• ‘Bachelor of Science (BSc)’ en informatique, Département de mathématiques, Faculté
des Sciences, Université d’Addis-Abeba, Addis-Abeba, Ethiopie : (Sept.1992-Juillet
1996).
• E.S.L.C.E (Diplôme de fin d’études secondaires), Addis-Abeba, Ethiopie: (Juin 1992).
Expérience professionnelle
• Enseignement de cours en informatique, Département de mathématiques et
Informatique, Université d’Addis-Abeba: (Sept. 1996- Juin 2002).
Expérience administrative
• Directeur adjoint des études, Faculté des Sciences Université d’Addis-Abeba:
(Sept.1999- Août 2000).
• Intervenant et responsable pour les cours de Bureautique et logiciels d’Application,
formation continue, Département de mathématiques et Informatique, Université d’
Addis-Abeba: (Sept. 2001-Juin 2002).
Connaissances linguistiques
• Anglais
• Français
• Amharique et Tigrigna (langues d’Ethiopie)
Liste des publications
I.
Revues nationales/internationales
•
•
Girma Berhe, Lionel Brunie, “Adaptation de contenus multimédias pour les systèmes
d’information pervasifs”, Journal d’Ingénierie des Systèmes d’Information, Revue des
Sciences et Technologies de l'Information (RSTI) série ISI, Volume 9, no 2/2004
pp.61-85.
Girma Berhe, Lionel Brunie, Jean-Marc Pierson, “Content adaptation in distributed
multimedia systems”, Journal of Digital Information Management, Special Issue on
Distributed Data Management, Volume 3, No 2, June 2005, pp.95-100.
II. Conférences Internationales
•
•
•
•
•
Surafel Lemma, Dawit Bekele, Girma Berhe. “Semantic Description of Multimedia
Content Adaptation Web services”, International Joint Conferences on Computer,
Information, and Systems Sciences, and Engineering (CIS2E 05), University of
Bridgeport, USA, December 10-20, 2005.
Girma Berhe, Lionel Brunie and Jean-Marc Pierson, “Composing Multimedia
Content Adaptation Services”, Workshop on Distributed Multimedia Databases and
Multimedia
Adaptation,
WDMDMA'05
(held
in
conjunction
with
EUROMEDIA'2005), Toulouse, France, 11-13 April 2005 pp.199-203. Proceedings
published in 11th Annual EUROMEDIA'05 Conference.
Girma Berhe, Lionel Brunie, Jean-Marc Pierson, “Distributed Content Adaptation for
Pervasive Systems”, In proceedings of IEEE International Conference on Information
Technology, ITCC 2005, April 4-6, 2005, Las Vegas, Nevada, USA, Vol.2 pp.234241.
Girma Berhe, Lionel Brunie, Jean-Marc Pierson, “Modeling Service-Based
Multimedia Content Adaptation in Pervasive Computing”, ACM Proceedings of the
first conference on computing frontiers (CF’04), Ischia , Italy, April 14 - 16, 2004 pp.
60 – 69
Girma Berhe, Lionel Brunie, Jean-Marc Pierson, “Service-Based Architectural
Framework of Multimedia Content Adaptation for Pervasive Computing
Environment”, 2004 Communication Networks and Distributed Systems Modeling
and Simulation Conference (CNDS'04), January 18-21, 2004.
FOLIO ADMINISTRATIF
THESE SOUTENUE DEVANT L'INSTITUT NATIONAL DES SCIENCES APPLIQUEES DE LYON
NOM : GIRMA
DATE de SOUTENANCE : 25 septembre, 2006
(avec précision du nom de jeune fille, le cas échéant)
Prénoms : BERHE
TITRE : Accès
et adaptation de contenus multimédia pour les systèmes pervasifs
NATURE : Doctorat
Numéro d'ordre : 06 ISAL 0056
Ecole doctorale : Informatique et Information pour la société (EDIIS)
Spécialité : Documents Multimédia, Images et D’Information Communicants (DISIC)
Cote B.I.U. - Lyon : T 50/210/19
/
et
bis
CLASSE :
RESUME :
La gestion des données en représente clairement l’une des composantes-clefs, recouvrant de nombreux aspects: extensibilité du système,
accès et intégration des données (en termes d’indexation et d’exécution de requêtes dans un environnement distribué fortement hétérogène),
optimisation de l’utilisation des ressources et de la qualité de service (bande passante réduite, discontinuité du service), cohérence de
l’information (l’utilisateur pouvant se trouver en mode déconnecté, comment assurer la cohérence et la pérennité des données qu'il manipule
?). Enfin, l'adaptation de contenus multimédias constitue certainement l'un des mécanismes centraux dans la mise en œuvre effective des
systèmes d'information pervasifs. La très grande hétérogénéité des moyens de connexion et des conditions de communication (bande
passante notamment), la très grande variabilité des préférences utilisateur imposent en effet de fournir des outils d'adaptation des contenus
transmis aux utilisateurs.
Dans ce contexte, beaucoup de travaux de recherche ont été effectués que l’on peut classer en trois approches principales selon le site
d’exécution des mécanismes d'adaptation : l’approche orientée client, l’approche orientée serveur et l’approche orientée proxy
Dans cette thèse, nous présentons une architecture logicielle d'adaptation de contenus multimédia (nommée DCAF: Distributed Content
Adaptation Framework). Cette architecture est fondée sur l’utilisation de services d'adaptation accessibles sur l'Internet. A l’inverse des
approches précédentes où le client, le serveur de contenu ou un proxy est responsable du processus d'adaptation, ce sont ici des services
développés extérieurement qui effectuent la transformation des contenus. DCAF fournit ainsi une solution générique d'adaptation de
contenus offrant flexibilité, extensibilité et interopérabilité.
La principale problématique introduite est la détermination du scénario optimal d’adaptation (également appelé chemin optimal
d'adaptation), plusieurs services d'adaptation pouvant fournir les mêmes fonctionnalités. Cette thèse propose d’adopter une technique issue
de l’intelligence artificielle pour optimiser la planification, la composition et l’exécution des services impliqués dans l’adaptation d’un
contenu. L’algorithme proposé prend en compte le profil des utilisateurs, les caractéristiques des dispositifs, la connectivité du réseau, le
format des contenus échangés et les spécifications (coût, qualité) des services d'adaptation disponibles pour construire un graphe d'adaptation
modélisant le scénario d’adaptation choisi.
Parallèlement, nous avons développé une ontologie dédiée à la description des services d'adaptation. Cette ontologie permet de décrire la
sémantique des services d’adaptation enfin de faciliter la découverte et la composition des services. Elle permet également de passer
facilement de la description d’un service à la représentation de l’opérateur d’adaptation.
Pour démontrer la faisabilité de l’architecture DCAF, un prototype a été développé et évalué. Les expériences conduites montrent que le
temps de construction du graphe d'adaptation et la durée moyenne d'adaptation sont compatibles avec les contraints des environnements
pervasifs.
MOTS-CLES : Systèmes Pervasifs, Adaptation de Contenu Multimédia, Composition de Services, Planification, Contexte
Laboratoire (s) de recherche : LIRIS (Laboratoire d’Informatique en Image et Systèmes d’Information)
Prof. Lionel Brunie
Co-directeur:
Prof. Jean-Marc Pierson
Président de jury :
Composition du jury :
Rapporteur:
Examinateur:
Co-directeur de thèse:
Harald Kosch, Professeur, Université de Passau, Allemagne
Hervé Martin, Professeur, LSR, Université Joseph Fourier
Günter Haring, Professeur, Université de Vienne, Autriche
Ahmed Mostefaoui, Maître de conférences, (LIFC) UFR-STGI, Montbéliard
Lionel Brunie, Professeur, INSA de Lyon
Jean-Marc, Pierson, Professeur, IRIT, Toulouse

Accès et adaptation de contenus multimédia pour les

Transcription

Documents pareils

Revue de presse ANGLAIS Semaine du 24 au 30 octobre 2011

Query-Oriented Summarization of RDF Graphs

4228 Holland Dr Des Moines, IA 50310 | Home for Sale | Real

Halton Hills Village, Georgetown - FEB 9-16

master de communication internationale - Univ

Term 2 – Years 4 Unit vocabulary Janvier January Un cahier An

mounto daballo – 2014

“What We Stand to Lose” Poster Lesson Plans

January 18, 2007

COUNSELORS IN THE ADOLESCENT DRUG SCENE