Design and development of a demonstration prototype for energy

Transcription

TEMPO Technical Report number ATP-3101:
Design and development of a demonstration prototype
for energy-aware distributed service delivery
algorithms
Philippe Robert1
NICTA
Locked Bag 9013
Alexandria NSW 1430
Australia
1
For further details please contact: [email protected]
Submitted: September 7, 2009
Summary:
This document present the internship report of Philippe Robert from the ISAE, Toulouse,
France. This report is in two parts. The first part is the summary in French, then the
report in English is provided.
Keywords: OMF, Virtualisation, Media Centre.
2
Internship Report
Networked Systems
National ICT of Australia
March - August 2009
To obtain the graduation of the
ISAE - ENSICA
Institut Supérieur de l’Aéronautique et de l’Espace
École Nationale Supérieure d’Ingénieurs en Constructions Aéronautiques
M. Philippe ROBERT
Design and development of a demonstration prototype for
energy-aware distributed service delivery algorithms
National ICT of Australia
Australian Technology Park Level 5
13 Garden Street
Eveleigh NSW 2015
Australia
Supervisor at NICTA
Thierry Rakotoarivelo
Supervisor at the ISAE
Patrick Senac
Acknowledgement
During this internship, I have been part of the TEMPO project. I would like to thank all the people
who have helped me to understand the purpose of this project, and to realize it. Thus, I would like
to give thanks to:
• Thierry Rakotoarivelo, my supervisor at NICTA, and Guillaume Jourjon, for their monitoring all along the project
• Olivier Mehani, for all his help with the virtualisation tool and with the configuration of
the virtual private network
• Sebastien Ardon, who has helped me to understand miscellaneous aspects concerning networking P2P protocols
• Emmanuel Lochin, for some networking advices
• Rodney Berriman, for all his help about everything concerning the wiki of NICTA and the
hardware issues
• And all the NICTA people for having integrated me in their team. This internship has
been very enjoyable thanks to the warm welcome inside NICTA, which let people work in
a pleasant frame.
iii
iv
Table of Contents
iv
Table of Contents
I
French Summary
1
1
Résumé du Rapport de Stage
3
1.1
Contexte de l’étude . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3
1.2
Implémentation d’un outil de virtualisation . . . . . . . . . . . . . . . . . . . .
6
1.2.1
Solutions de virtualisation . . . . . . . . . . . . . . . . . . . . . . . . .
7
1.2.2
Implémentation de la solution choisie . . . . . . . . . . . . . . . . . . .
7
1.2.3
Xen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
8
Vers un service multimédia décentralisé . . . . . . . . . . . . . . . . . . . . . .
9
1.3.1
Protocole P2P . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
9
1.3.2
Media-Center . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
11
Démonstration des solutions proposées . . . . . . . . . . . . . . . . . . . . . . .
12
1.4.1
OMF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
12
1.4.2
Démonstration NICTA . . . . . . . . . . . . . . . . . . . . . . . . . . .
13
1.4.3
Application P2P en réseau privé . . . . . . . . . . . . . . . . . . . . . .
13
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
14
1.3
1.4
1.5
II
2
3
Main Report
17
Introduction
19
2.1
Presentation of NICTA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
19
2.2
The TEMPO Project . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
20
2.3
Subject of my internship . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
20
2.4
Organisation of this report . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
21
Context of this internship
23
v
vi
4
Table of Contents
3.1
New services available through the Internet . . . . . . . . . . . . . . . . . . . .
23
3.2
Energy issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
24
3.3
Nano DataCenters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
26
3.4
Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
28
Integration of a virtualization tool
31
4.1
Introduction to Virtualization . . . . . . . . . . . . . . . . . . . . . . . . . . . .
31
4.2
Nada project and virtualization . . . . . . . . . . . . . . . . . . . . . . . . . . .
32
4.3
Comparison of the different kinds of virtualization . . . . . . . . . . . . . . . . .
33
4.3.1
Full-virtualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
34
4.3.2
Para-virtualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
36
4.3.3
OS-level virtualization . . . . . . . . . . . . . . . . . . . . . . . . . . .
37
4.3.4
Choice of OpenVZ . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
39
Installation/configuration of OpenVZ . . . . . . . . . . . . . . . . . . . . . . . .
39
4.4.1
Installation of OpenVZ . . . . . . . . . . . . . . . . . . . . . . . . . . .
39
4.4.2
Configuration of OpenVZ . . . . . . . . . . . . . . . . . . . . . . . . .
40
4.4.3
Network inside VPS . . . . . . . . . . . . . . . . . . . . . . . . . . . .
40
4.4.4
Graphic applications inside VPS . . . . . . . . . . . . . . . . . . . . . .
41
Xen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
42
4.5.1
Partitionning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
43
4.5.2
Installation and configuration . . . . . . . . . . . . . . . . . . . . . . .
44
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
45
4.4
4.5
4.6
5
Toward a decentralized multimedia service
47
5.1
P2P protocols and P2P applications . . . . . . . . . . . . . . . . . . . . . . . .
47
5.1.1
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
47
5.1.2
P2P protocols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
48
5.1.3
BitTorrent protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
48
5.1.4
Choice of a P2P application . . . . . . . . . . . . . . . . . . . . . . . .
50
5.1.5
Choice of the tracker application . . . . . . . . . . . . . . . . . . . . . .
52
Media-Centre Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
53
5.2.1
Requirements of the Media-Centre application and choice of XBMC . . .
53
5.2.2
Installation and configuration . . . . . . . . . . . . . . . . . . . . . . .
54
5.2
vi
Table of Contents
6
5.2.3
Providing a remote control . . . . . . . . . . . . . . . . . . . . . . . . .
54
5.2.4
Creation of a list of content for the graphic interface . . . . . . . . . . .
54
5.2.5
Skin and appearance . . . . . . . . . . . . . . . . . . . . . . . . . . . .
54
5.3
HTTP-proxy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
55
5.4
Issues ecountered . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
55
5.5
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
56
Demonstrations provided
57
6.1
First demonstration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
57
6.1.1
Structure of the demonstration . . . . . . . . . . . . . . . . . . . . . . .
57
6.1.2
Network configuration . . . . . . . . . . . . . . . . . . . . . . . . . . .
58
6.1.3
Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
59
Second demonstration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
60
6.2.1
Structure of the demonstration . . . . . . . . . . . . . . . . . . . . . . .
60
6.2.2
Network configuration . . . . . . . . . . . . . . . . . . . . . . . . . . .
60
OMF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
60
6.2
6.3
7
vii
Conclusion
63
7.1
Conclusion of this internship . . . . . . . . . . . . . . . . . . . . . . . . . . . .
63
7.2
Conclusion of this project . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
64
7.3
Possible evolutions of my achievements . . . . . . . . . . . . . . . . . . . . . .
64
Appendix A
67
List of Figures
71
List of Tables
71
vii
Part I
French Summary
1
C HAPTER 1
Ce rapport de stage rend compte de mon Projet de Fin d’Études réalisé au sein de NICTA [49],
National ICT (Information and Communications Technology) of Australia, du 5 mars au 7 août
2009. NICTA est une entreprise indépendante fondée et financée par le gouvernement australien
dont les domaines d’activités concernent la recherche informatique, la commercialisation de solutions et la formation. Elle emploie ainsi 426 personnes et participe à la formation de presque 300
étudiants.
Actuellement élève ingénieur de troisième année à l’ISAE-ENSICA, j’ai choisi de me spécialiser
en informatique, et après avoir suivi de nombreux modules informatiques, j’ai voulu réaliser un
Projet de Fin d’Études dans ce domaine. La réalisation de ce stage dans un environnement international m’est apparue comme capitale dans mon futur métier d’ingénieur, puisqu’elle offre la possibilité d’acquérir de nouvelles méthodes de travail. D’autre part, le choix d’un pays anglophone a
été dicté par le caractère incontournable de cette langue dans les échanges internationaux et également dans la documentation informatique, majoritairement écrite en Anglais. Enfin, l’occasion de
travailler dans un organisme de recherche et non en entreprise m’a permis de poursuivre ma formation dans le domaine informatique en découvrant des applications très récentes de technologies
existantes.
Travailler à NICTA m’a donc permis de remplir ces objectifs. J’ai été intégré dans le département
Networked Systems dans le projet Service Delivery and Testbed Framework, anciennement appelé
TEMPO [25], projet de recherche sur les services distribués et l’administration de testbeds.
1.1
CONTEXTE DE L’ÉTUDE
J’ai été intégré durant 5 mois dans le projet TEMPO. À travers ce projet, NICTA travaille principalement avec le projet européen Nado-Data-Center [47] dans le but de développer des méthodes
de transferts de données et de services internet plus efficaces et moins énergétivores.
3
4
Chapter 1
Le principal objectif du projet TEMPO consiste en la mise en place d’une structure universelle
de contrôle, de mesure et de gestion des expériences réseaux. Cette structure est appelée OMF
[16] En effet, si la simple simulation permet d’avoir une idée du résultat de certaines expériences
réseau, seule le déployement sur des testbeds dédiés permet de valider de manière rigoureuse et
scientifique les résultats obtenus. OMF apparait comme un testbed universel pouvant être utilisé
dans un grand nombre de projets différents pour réaliser et contrôler des expériences.
Le second objectif se propose d’explorer le domaine des applications Pair-à-Pair (Peer-to-Peer ou
P2P) et ses applications dans la distribution de données, dans le but de fournir des services internet
de façon plus efficace, en terme de rapidité de transfert comme en terme d’économie d’énergie.
J’ai pris part durant mon projet de fin d’études à ce second objectif.
L’approche traditionnelle des échanges internet est basée sur un modèle Client/Serveur. Le serveur
centralise toutes les données et se doit d’avoir une structure très robuste pour résister en permanence à des surcharges temporaires, et se trouve donc dans le cas général surdimensionné par
rapport au service rendu. Au niveau énergétique, les serveurs sont localisés dans des datacenters
et leur grande concentration nécessite des frais très élevés en terme de consommation d’énergie
pour le refroidissement [42] [43]. D’autre part, les capacités de calcul et de stockage des Modem, ou Set-Top-Box (STB) ont suffisamment progressé pour qu’il soit possible de transférer des
fonctionnalités traditionnellement réservées aux serveurs des datacenters vers les utilisateurs finaux, offrant ainsi un service de distribution décentralisé tout en allégeant le côté serveur. Cette
nouvelle architecture permet d’économiser les coûts liés au refroidissement, désormais réalisé de
manière passive. Enfin, le protocole TCP/IP, majoritairement utilisé dans les échanges internet,
n’est plus adéquat pour les nouvelles applications ?? qui doivent de plus en plus répondre à des
contraintes de temps-réel: du fait notamment de ses mécanismes de retransmission en cas de perte
de paquets, ce protocole utilise inutilement des ressources.
Certaines applications internet fonctionnent déjà en surmontant les obstacles de l’architecture
Client/Server et du protocole TPC/IP, comme le protocole Skype, permettant de relayer les données en les transmettant par les autres postes connectés qui bénéficient du même service et non
par un serveur localisé dans un datacenter [52]. Ce protocole applicatif utilise de plus à la fois les
protocoles de transport TCP et UDP [24].
Le projet NADA propose de délocaliser les services internet des serveurs vers les utilisateurs
finaux. Cela est rendu possible par l’utilisation d’outils de virtualisation, permettant de faire fonctionner sur une même machine physique différentes machines virtuelles isolées. Le fournisseur
d’accès internet peut alors louer des machines virtuelles à des fournisseurs de contenu internet
comme Google ou YouTube par exemple, pour leur permettre de faire fonctionner leurs applications directement sur la machine de l’utilisateur final. La virtualisation permet cette architecture
ainsi que la création d’un environnement de confiance totalement contrôlé au sein même de la
machine de l’utilisateur. Cette structure évite l’ajout de nouveau hardware, et réduit la charge
supportée par les datacenters. La figure 3.4 montre l’architecture proposée par cette solution.
Cette nouvelle architecture se doit pourtant de maintenir la machine utilisateur en permanence en
service. NICTA se propose d’étudier des algorithmes qui permettraient, en maintenant la même
disponibilité, d’éteindre les machines les moins solicitées dans un soucis d’économie d’énergie.
Le but de mon stage, dans le cadre du projet TEMPO, est de créer un modèle de Set-Top-Box,
qui se veut être un prototype de machine utilisateur intégrant l’architecture de NADA. Ainsi,
4
1.1 Contexte de l’étude
5
la première partie de mon stage a consisté en l’étude d’outils de virtualisation et son implémentation dans un modèle de Set-Top-Box. L’outil de virtualisation retenu permet de séparer
les applications utilisateurs (Media-Centre) des applications utilisées par le fournisseur d’accès
internet utilisées pour fournir du contenu ou des services (applications BitTorrent). Ensuite,
l’intégration d’applications P2P BitTorrent et d’un Media-Center au sein de cette architecture a
permis d’achever la construction de ce prototype.
La figure 1.1 montre la structure du prototype attendu par NICTA. En réalité, compte tenu de
la difficulté à faire fonctionner un Media-Centre dans un VPS sans accès direct au matériel, j’ai
installé XBMC dans le HardNode, ou HN, soit la machine physique. Cette nouvelle structure
répond à l’exigence d’isolement des différentes applications.
Figure 1.1 Structure du prototype créé
Les démonstrations mises en place au sein du testbed utilisé par le projet TEMPO a permis enfin de démontrer son efficacité. L’intégration complète dans ce testbed n’a pas pu être achevée,
notamment en raison de la difficulté à intégrer l’outil de virtualisation. Poursuivre cette intégration pourrait permettre de mesurer l’efficacité des transferts avec la nouvelle architecture. Les
applications P2P proposées, open-sources, permettront également aux chercheurs de NICTA de
proposer de nouveaux services, comme des services de Vidéo à la Demande (Video on Demand,
VoD) obtenus à partir de ce protocole P2P.
La figure 1.2 montre le principe de la démonstration réalisée à NICTA. L’utilisateur n’a accès qu’au fonctions du Media-Center. Une liste du contenu disponible est affichée. Lorsque
l’utilisateur choisi un contenu, il est téléchargé à partir d’un serveur HTTP situé dans un espace
isolé auquel l’utilisateur n’a pas accès. Si celui-ci n’est pas disponible, au lieu de renvoyer l’erreur
404, le proxy exécute des scripts qui permettront d’obtenir celui-ci à partir du réseau P2P (les
5
6
Chapter 1
machines du testbed d’OMF). L’utilisateur dans ce cas ne perçoit qu’un temps d’attente supplémentaire.
Figure 1.2 Diagrqmme de séquence suivi lors des démonstrations
1.2
IMPLÉMENTATION D’UN OUTIL DE VIRTUALISATION
La première partie de mon stage a consisté à l’étude et l’implémentation d’un outil de virtualisation. J’ai d’abord fait un travail de recherche sur les différentes solutions existantes afin d’en
sélectionner une qui réponde aux besoins de NICTA et aux exigeances de NADA. L’outil de virtualisation présente une structure légère avec peu de perte de performances par rapport à une
structure sans virtualisation. Il permet également une isolation des différentes machines virtuelles
fonctionnant sur la machine physique et un moyen de communication entre les différentes entités
pour le transfert de données notamment. l’architecture du prototype souhaitée est la suivante:
Dans une machine virtuelle fonctionnent les applications P2P et dans une autre les applications
utilisateur dont notamment un Media-Center, application graphiques. Cependant, l’intégration de
cette solution de virtualisation avec les applications graphiques a posé problème et une nouvelle
architecture a dûe être proposée pour le contourner.
6
1.2 Implémentation d’un outil de virtualisation
1.2.1
7
Solutions de virtualisation
Le terme virtualisation décrit la séparation d’un service avec le matériel qui rend ce service. Les
outils de virtualisation premettent de créer des machines virtuelles (VPS: Virtual Private Server
ou VE: Virtual Environment) sur un même support phyisque (HN: HardNode). un VMM (Virtual Machine Monitor) gère les accès au matériel qui doit dès lors être partagé par les différents
VE. Du fait de l’introduction de cette couche supplémentaire entre le matériel et les systèmes
d’exploitation, les instructions des systèmes invités ne peuvent plus être directement transmises
et exécutées par le matériel et différentes architectures ont été mises en place pour transmettre les
instructions des systèmes invités au matériel:
• Full-virtualisation Les instructions privilégiées des systèmes invités, c’est à dire les instructions qui ont un impact fort sur le matériel, sont interceptées par le VMM et traduite
dans un langage pouvant être interprété par le système hôte qui a les transmet directement
au matériel. On parle de traduction binaire. L’avantage est l’universalité de cette solution,
puisque tous les systèmes, libres ou non, peuvent être virtualisés. Cette universalité ne nous
intéresse pas puisqu’il s’agit de virtualiser des systèmes Linux libres. Elle a de plus un coût
conséquent en terme de performances. Je n’ai pas retenu cette solution pour cette raison.
• Para-virtualisation Le système invité doit être modifié pour être compatible avec le système hôte au travers du VMM. Les instructions privilégiées sont remplacées par des appels
au VMM qui se charge de transmettre au matériel. Le système invité à une structure plus
proche du matériel. Seules des systèmes libres peuvent dès lors être virtualisés, puisqu’il
faut les modifier. Le principal avantage est le faible coût au niveau des performances dûe à
la couche de virtualisation.
• Virtualisation niveau Système d’Exploitation Un seul système d’exploitation fonctionne
sur la machine. Celui-ci est un système capable d’exécuter des instructions dans des espaces isolés appelés containers. Généralement, un noyau Linux modifié rend disponible
cette fonctionnalité. l’architecture de ce type de virtualisation est comparable à un système sans virtualisation, puisque les ordres de la machines invité sont directement exécutés
compte tenu qu’il n’y a qu’un système d’exploitation. Pour autant, il y a un espace administrateur, appelé HardNode (HN), qui comporte toutes les fonctionnalités, et les VPS
invités n’ont que des fonctionnalités restreintes (pas d’accès direct au matériel). Compte
tenu des grandes performances de ce type de solution, le fonctionnement d’applications nécessitant de grandes ressources sera possible. Nous avons donc retenu ce type de solution
de virtualisation.
Linux VServer [11] et OpenVZ [18] sont deux implémentations de virtualisation de niveau système d’exploitation. Nous avons préféré OpenVZ, compte tenu notamment de la richesse de sa
documentation et de son wiki et de la grande activité de son forum.
1.2.2
Implémentation de la solution choisie
J’ai d’abord installé OpenVZ, qui à la base est un noyau Linux modifié pour rendre disponible le
containement. Cette installation a été réalisée en suivant les instructions de différents tutoriaux
7
8
Chapter 1
dont celui issu du wiki officiel d’OpenVZ [18] et de howtoforge [54]. Deux types de fichiers de
configuration sont nécessaires pour OpenVZ:
• /etc/vz/vz.conf Le fichier de configuration global servant notamment lors de la création de
nouveaux VPS. Il spécifie où sont situés les OS-templates à utiliser pour créer un VPS, et
où sont situés les systèmes de fichiers de chaque VPS.
• /etc/vz/conf/$CTID.conf où $CTID est le numéro identifiant le VPS. Ce fichier est le fichier
de configuration global du VPS, avec sa configuration réseau, son adresse IP, son nom
d’hôte, et toutes les limitations mémoires du VPS. Ce fichier est créer à partir du fichier
de configuration global lors de la création d’un nouveau VPS. Il peut être édité manuellement, mais les modifications ne seront prises en compte qu’à partir d’un redémarrage, ou
bien avec des outils de configuration mis en place par OpenVZ comme vzctl par exemple.
J’ai ensuite configuré son réseau pour connecter les VPS à Internet. Le HN se comporte comme
une gateway qu’il est possible de configurer pour rendre Internet accessible depuis les machines
virtuelles. Il y a deux types de connexion des VPS au HN:
• venet ou virtual internet. Il suffit de spécifier depuis le HN l’adresse IP et le nom d’hôte de
la machine virtuelle. Le reste de la configuration se réalise comme la configuration normale
d’un réseau privée derrière une passerelle. ce moyen rapide et efficasse présente l’avantage
de ne rien avoir à configurer côté VPS, ce qui est pertinent au niveau sécurité lorsque les VPS
ne sont pas des environnement de confiance (utilisateur quelconque). Il n’y a pas d’adresse
MAC, et par conséquent pas de possibilité de configurer le réseau de manière dynamique à
l’aide d’un serveur DHCP, mais pas non plus de possibilité de capturer des paquets. Cette
configuration a été retenue pour la construction du prototype.
• veth ou virtual ethernet. La configuration est plus complexe mais plus proche de la réalité
d’un réseau privé. Cette configuration n’interdit pas la capture de paquet, et permet la mise
en place d’un serveur DHCP dans un VPS. Cette configuration a été retenue pour tester les
applications P2P dans un réseau simulant un véritable réseau privé.
Au niveau du gateway (Le HN) il suffit de mettre en place l’IP forwarding et créer une interface
virtuelle pour connecter les VPS à Internet. Les VPS se comportent alors comme de véritables
serveurs, l’installation de nouvelles applications se fait de la même façon qu’avec un serveur
Debian classique.
1.2.3
Xen
Considérant des résultats de Thomson [4] lors de l’étude comparative des solutions de virtualisation de Xen et d’OpenVZ, montrant une meilleure stabilité de Xen et des performances non
régulières d’OpenVZ, nous avons décidé de privilégier cette solution pour un temps lors de la
construction de notre prototype.
Pour éviter la perte du travail déjà réalisé sur le noyau OpenVZ, j’ai décidé de partitionner le disque
de façon à installer Xen sur une partition vierge et placer OpenVZ sur une partition restreinte. Une
autre partition utilisée par le swap et une dernière pour partager le répertoire /home de OpenVZ
8
1.3 Vers un service multimédia décentralisé
9
à Xen ont été créées. Compte tenu que le nombre maximale de 4 partitions allait être atteint, j’ai
décidé de créer une partition étendue pour ne pas être limité par la suite s’il s’avérait nécessaire
d’en rajouter une. Ce partitionnement a été réalisé avec l’outil gparted.
L’installation de Xen a été réalisée à partir d’un tutoriel du site howtoforge [55]. J’ai déploré
la documentation de Xen, bien que très dense, a tout à envier à celle, concise et très structurée, d’OpenVZ. La configuration réseau adoptée est la même que pour OpenVZ, la machine
physique, ou dom0, joue le rôle de passerelle pour les machines virtuelles, domU. L’installation
d’applications dans les DomUs se fait alors de la même façon que pour un serveur physique.
L’expérience décrite dans la figure 6.1 a été réalisée également avec Xen. (Il n’était pas possible
d’installer un serveur graphique dans un DomU, j’ai installé XBMC dans Dom0, comme je l’ai
fait sur le HN plutôt que dans un VPS pour OpenVZ).
Par la suite, nous sommes revenus sur la solution OpenVZ, beaucoup plus légère, et privilégiée
par NADA.
1.3
VERS UN SERVICE MULTIMÉDIA DÉCENTRALISÉ
Une fois la solution de virtualisation implémentée, j’ai étudié les différentes applications P2P et
Media-Center pour poursuivre la construction du prototype. J’ai ensuite installé les applications
P2P choisies dans un VPS et le Media Center dans un autre.
1.3.1
Protocole P2P
Le protocole Pair-à-Pair ou Peer-to-Peer (P2P) est un protocole de communication dans lequel
chaque entité joue le même rôle par rapport aux autres. Chaque entités est ainsi égale aux autres,
par opposition au modèle de communication classique client/serveur. Ce protocole peut alors être
utilisé pour se substituer au modèle client/serveur et réaliser des transferts de fichiers par exemple,
mais il a aussi pour vocation le partage de flux multimédia ou le calcul distribué:
On peut par exemple citer le projet SETI@home (Search ExtraTerrestrial Intelligence, recherche
d’intelligence extraterrestre) [23]. SETI@home est une expérience scientifique utilisant la puissance de calcul des ordinateurs des particuliers pour analyser les données de radio-télescopes et
rechercher des preuves d’intelligence extraterrestre. Les opérations à effectuer sont téléchargées
à partir d’un serveur. Les calculs sont effectués sur les machines des particuliers et les résultats
renvoyés à ce serveur. Un autre exemple d’application P2P est le protocole Skype: Les données
multimédia (flux audio et vidéo) sont relayées par les utilisateurs de ce même service du destinataire à la destination sans passer par un serveur central.
Cette technologie désengorge les serveurs pour délivrer le contenu de pair à pair.
Les applications P2P peuvent être classées selon leur degrés de décentralisation:
• Architecture centralisée: les clients se connectent à un serveur qui gère la recherche de
contenu et de pairs possédant ce contenu. Les clients ensuite se connectent aux pairs ayant
le contenu recherché. Le serveur orchestre les échanges, mais le contenu n’est transféré que
de clients à clients.
9
10
Chapter 1
• Architecture décentralisée: de nombreux pairs privilégiés jouent le rôle de serveur (orchestrant les transferts, mais ne partageant pas les informations). La recherche de contenu est
beaucoup plus complexe, mais le réseau est plus léger (répartition de la charge sur différents
serveurs) et robuste (la perte d’un serveur n’interdit pas de nouveaux transferts). On distingue les architectures structurées, dans lesquelles les listes de pairs et de contenus sont
partagés par tous les serveurs, et les architectures non-structurées, dans lesquelles le pair
client doit contacter tous les serveurs pour obtenir une liste complète des pairs et du contenu
disponible.
• Architecture hybride: Certains noeuds sont choisis pour partager le contenu en fonction de
leur bande-passante et de leur position dans le réseau. Skype, développé par Kazaa, est un
exemple d’architecture hybride [52].
Dans le cadre du projet, une architecture centralisée a été retenue. Parmis les différentes implémentations de cette architecture, nous avons utilisé le protocole BitTorrent. À la différence
des autres protocoles centralisés, dans lesquels une file d’attente doit se mettre en place dès que
plusieurs clients tentent d’accéder au contenu, le protocole BitTorrent divise le fichier en pièces
et plusieurs utilisateurs peuvent télécharger des pièces différentes au même moment. Les pièces
téléchargées sont automatiquement mises à disposition des autres clients potentiels. Le fichier
étant recomposé par un nombre important de souces, (tous les clients qui le téléchargent à un instant donné, plus ceux qui continuent à le mettre à disposition des autres une fois le téléchargement
achevé), son téléchargement est d’autant plus rapide qu’un grand nombre de clients sont intéressés
par lui. Avec BitTorrent, plus les clients téléchargent un fichier, plus il est rapide de le télécharger
soi-même au même moment.
L’obtention de contenu avec cette implémentation se fait de la manière suivante:
Un pair se connecte à un serveur HTTP ou FTP et télécharge un fichier .torrent correspondant
au contenu recherché. Dans ce fichier sont spécifiés la taille du contenu, le nombre de pièces
le composant, et l’adresse IP du tracker, serveur qui organise le transfert. Le client se connecte
alors au tracker qui lui fournit la liste des paires recherchant le même contenu ou le possédant
et le mettant à disposition. Le client se connecte alors à chaque pair et télécharge les pièces
qui lui manquent. L’ordre de téléchargement de ces pièces est aléatoire, de sorte que plusieurs
clients ne possédant pas le contenu en totalité peuvent se compléter. Les clients se connectent
périodiquement au tracker pour mettre à jour sa liste de pairs disponibles. Le transfert des données
est illustré par la figure 5.1.
J’ai choisi transmission [26] comme application P2P. Cette application est open-source ce qui
permettra de modifier son code source et ajouter à terme de nouvelles fonctionnalités. Il est écrit
en C, ce qui est apprécié par l’équipe à NICTA travaillant sur ce projet. Sa documentation est
très complète et son forum très actif. Il est disponible en plusieurs versions, dont celle par défaut
installée avec Ubuntu, mais aussi d’autres plus légères sans interface graphique, permettant le
contôle à distance, ou bien fonctionnant comme un démon.
En ce qui concerne le tracker, j’ai choisi rivettracker [22]. Cette application doit être mise en ligne
comme un simple site internet et une plateforme LAMP (Linux Apache Mysql Phpmyadmin) a
été installée pour rendre le tracker disponible. Une fois installée, cette application est très simple d’utilisation, elle est open-source, si éventuellement de nouvelles fonctionnalités doivent être
ajoutées. Elle est écrite en PHP, langage dont je suis relativement familier. Son fonctionnement,
10
1.3 Vers un service multimédia décentralisé
11
avec l’interface graphique d’un web-browser, est un atout pour montrer les statistiques des transferts lors des démonstrations. Pour ces raisons, j’ai préféré ce logiciel plutôt que d’autres comme
opentracker [17] bien plus léger mais aussi moins évident à utiliser et sans interface graphique à
présenter lors de démonstrations.
1.3.2
Media-Center
Les applications utilisateurs, modélisées par un Media-Center, fonctionnent en avant-plan, la
couche de virtualisation lui est transparente. Le Media-Center a été choisi en fonction de ses
qualités graphiques, puisque dans le cadre des démonstrations, il fallait une application montrant
la solidité de l’architecture globale, et ne pas se contenter d’un simple visionneur de vidéos comme
VLC. Trois Media-Center ont un rendu graphique qui m’ont paru intéressant: BOXEE, ELISA et
XBMC. BOXEE n’a pas été retenu en raison de son interface graphique entièrement propriétaire.
XBMC propose clef en main les principales fonctions attendues, à savoir la possibilité de se connecter à un serveur http et d’éditer/configurer entièrement ses play-lists, tandis que ces fonctions
ne sont pas disponibles chez ELSIA, il faut les créer à partir de modules python. Nous avons donc
retenu dans un premier temps XBMC que j’ai installé dans un VPS.
Le fonctionnement d’XBMC doit passer par un accès direct au matériel, et j’ai pu simuler cet
accès direct, en interdisant les tests préliminaires fait à son démarrage, mais les performances,
alors très médiocres, m’ont fait abondonner ce type de solution. Les outils de virtualisation sont
en effet principalement utilisés pour l’hébergement de serveurs internet en permettant de fusionner plusieurs serveurs physiques en un seul, en conservant les avantages des serveurs dédiés.
L’utilisation de cet outil pose des soucis au niveau des applications graphiques, puisque rien ou
presque n’est prévu pour héberger un serveur graphique dans une machine virtuelle, ce qui était
pourtant nécessaire pour pouvoir faire fonctionner un Media Center correctement. J’ai donc décidé de faire fonctionner cette application directement sur le HN, considérant que l’isolation des
applications P2P dans un VPS étaient suffisantes.
Un outil de contrôle à distance, télécommande, a été également installé à l’aide de tutoriels internet
[12]. des listes de films ont été créés et leur rendu graphique a été configuré à partir des bases de
données SQLite de XBMC.
Pour la suite de la démonstration, j’ai utilisé la fonction d’HTTP-client disponible dans XBMC
pour contacter le proxy installé dans le VPS gérant les applications P2P. Ce proxy est un simple
serveur HTTP python que j’a modifié pour ajouté un comportement supplémentaire: lorsqu’un
fichier de taille nulle est demandé, il exécute des scripts bash qui démarrent les applications P2P
pour obtenir le fichier à partir du réseau P2P. Il est ensuite transféré du VPS au HN à l’aide du
protocole HTTP. L’utilisateur final ne remarque qu’un délai plus long dû à la recherche du contenu.
Un timeout côté XBMC provoque parfois la deconnexion du server lorsque la recherche du contenu prend trop de temps, je n’ai pas sû modifier ce comportement malgré l’édition d’un fichier
de configuration normalement censé pouvoir augmenter ce timeout. Plutôt qu’augmenter ce timeout, qui présente le défaut d’être atteint si décidément le contenu est trop long à récupérer, j’ai
également pensé à modifier le proxy pour qu’il envoie des réponses régulièrement pour remettre
le timeout à zéro, mais cette solution n’a pas pu être implémentée faute de temps.
11
12
1.4
Chapter 1
DÉMONSTRATION DES SOLUTIONS PROPOSÉES
La solution de virtualisation, et les applications utilisateurs et P2P, installées et configurées, le
prototype est désormais prêt à être testé. La structure de la première démonstration est schématisée figure: 6.1. C’est la démonstration demandée par NICTA. Une autre démonstration, utilisée
pour mieux appréhender les phénomènes réseaux mis en jeu dans les échanges de données, est
schématisée figure: 6.2.
Le testbed d’OMF a été utilisé pour mettre en place cette démonstration. Dans le prototype, fonctionne XBMC et en arrière-plan un proxy qui sert du contenu multimédia. L’intégration complète
dans le testbed n’a pas pu être achevé en raison des difficultés à intégrer OpenVZ.
J’ai donc utilisé les machines du testbed NICTA pour constituer un réseau P2P. Un tracker et des
applications P2P transmission ont été installées sur les postes du testbed. Le testbed n’étant pas
connecté à Internet, il a fallu installé les applications à partir des paquets .deb à copier depuis le
prototype de STB, dans le répertoire /var/cache/apt/archives puis utiliser la commande dpkg -i
nomDuPaquet.deb pour l’installer. L’installation est longue, puisque les paquets peuvent posséder de nombreuses dépendances. Une fois l’application installée, il est possible grace aux outils
mis à disposition par OMF de sauver une image de l’état du disque pour ensuite le déployer sur
les autres noeuds du testbed.
1.4.1
OMF
OMF (cOntrol andManagement Framework) est un projet ayant pour but le développement et le
déployement d’une structure universelle de contrôle et de gestion d’expériences réseau. L’objectif
à long terme est d’augmenter la rigueur scientifique dans le domaine des recherches réseaux.
En effet, si la simulation permet à faible coût de prévoire les comportements réseau comme les
performances ou la mise à échelle, du fait de la complexité des phénomènes mis en jeu, elle ne
permet la validation que de modèles simplifiés, qui ne prennent pas en compte la totalité d’un
environnement réél. La simulation est une alternative, qui utilise d’une part des prototypes et
d’une autre utilise des entités reproduisants le comportement réél. Cependant, du fait de ces entités
imitant le comportement réél, la complexité des phénomènes réseau peut encore nous échapper.
La solution est de concevoir et d’utiliser des testbeds, qui seuls permettent l’étude d’expériences à
grande échelle.
La complexité de ces testbeds et leur coût, interdisent leur utilisation à de nombreux développeurs.
De plus, un testbed est générallement conçu et développé pour répondre à un besoin particulier
formulé dans le cadre d’un projet, ce besoin disparait une fois le projet mené à son terme. Le
testbed n’est donc plus utilisé par la suite. Des solutions peuvent être obtenues par l’utilisation de
testbeds tels PlanetLab [20] et Orbit [19] déployant des noeuds dans le monde entier et pouvant être
réutilisés dans le cadre d’autres expériences. OMF apparait comme une amélioration de ces deux
technologies et offre un protocole universel de contôle et de gestion des expériences. L’avantage
est de réaliser et contrôler des expériences différentes, du moment qu’elles suivent le protocole
défini par OMF. OMF met ainsi à disposition des outils permettant:
• Pour l’utilisateur, de décrire l’expérience, l’exécuter et d’en obtenir les résultats
12
1.4 Démonstration des solutions proposées
13
• Pour l’administrateur, de gérer les ressources du testbed (redémarrer les noeuds, installer de
nouvelles images de système d’exploitation, installer des applications)
Le but est d’utiliser les outils d’OMF pour réaliser l’expérience et ainsi permettre d’en obtenir des
résultats quantitatifs. L’intégration des différentes applications P2P s’est correctement déroulée,
mais leur exécution, à l’aide de scipts ruby pour respecter les standards d’OMF, n’a pas pu être
achevée, de même que l’intégration du STB et des outils de virtualisation, principalement par
manque de temps.
1.4.2
Démonstration NICTA
le prototype fait office de STB. L’application XBMC permet de se connecter au proxy situé dans le
VPS hébergeant le client P2P. Le proxy met à disposition du contenu, directement téléchargeable
par protocole HTTP, et des fichiers de taille nulle. Lorsqu’un fichier de taille nulle est demandée,
le proxy exécute un script qui télécharge le fichier .torrent correspondant au contenu demandé à
partir d’un poste du testbed. Ensuite, le fichier .torrent est interprété par l’application P2P qui se
connecte au tracker. Le tracker lui fournit une liste de paires possédant le contenu demandé. Le
client se connecte à chacun de ces paires à partir d’un port par défaut. Le paire lui spécifie alors un
autre port à ouvrir. Le paire se déconnecte et ouvre ce nouveau port. Il attend ensuite la connexion
du paire partageant le contenu et le transfert commence. Une fois le transfert terminé, le contenu
est alors disponible dans le répertoire de partage du proxy et est téléchargé si le timeout d’XBMC
n’est pas atteint. Le contenu est alors affiché à l’écran d’XBMC.
Les principaux problèmes rencontrés concernent la configuration réseau, car si le paire recherchant
le contenu est dans un réseau privé (ce qui est le cas ici puisque le HN d’OpenVZ agit comme
une gateway) les paires fournissant ce contenu ne peuvent initier de connexion sur un nouveau
port. Il a fallu utiliser un logiciel de capture de paquets comme wireshark pour comprendre ce
phénomène. Le problème a été résolu en forçant les paires à utiliser un port particulier et à faire
du port-forwarding au niveau du HN pour relayer les paquets arrivant sur ce port vers le VPS.
Sur le testbed d’OMF, les résultats ont été satisfaisants, puisque le prototype est à même de
récupérer du contenu selon l’architecture demandée en utilisant le testbed d’OMF. Les applications situées dans les machines du testbed peuvent être lancées par les scripts utilisant la syntaxe
d’OMF ce qui permettra de récupérer des résultats des expériences de manière rigoureuse. Par
contre, le STB n’a pas pu être intégré dans cette framework, ce qui limitera la portée des résultats
obtenus.
1.4.3
Application P2P en réseau privé
L’application en réseau privé consiste à faire fonctionner le tracker et les applications P2P fournissant le contenu dans le même sous-réseau. Le problème posé est le fait que l’application P2P
doit contacter le tracker à partir de son addresse IP publique. Là encore, wireshark a été nécessaire pour comprendre les phénomènes en jeu et appliquer les règles adéquates d’iptables. Cette
seconde démonstration, durant une phase du stage où le testbed d’OMF n’était pas disponible,
m’a permis de tester cette configuration réseau particulière (normalement le tracker et les applications P2P fournissant le contenu ne sont pas dans le même sous-réseau alors que l’application P2P
recherchant ce contenu est à l’extérieur).
13
14
1.5
Chapter 1
CONCLUSION
Au cours de ce stage à NICTA, j’ai pu travailler dans un projet aux multiples facettes:
• L’étude de solution de virtualisation, pour commencer, m’a permis d’entrer dans le monde
de Linux dans lequel je n’avais aucune connaissance au début de mon stage. J’ai pu acquérir d’une part des connaissances dans l’administration de systèmes Linux, sur les distributions Debian et Ubuntu, et également des connaissances sur les outils de virtualisation
tels OpenVZ ou Xen. Ces solutions ont un avenir considérable dans la consolidation de
serveurs et apparaissent comme un moyen possible pour réduire la consommation énergétique des Data Centers en fusionnant plusieurs serveurs physiques dédiés en un unique tout
en conservant les avantages de l’isolation. L’application de ces solutions au projet a permis
de réaliser un prototype de STB avec une architecture semblable à celle exigée par le projet
NADA. Les points les plus difficiles à mettre en place concernent la configuration réseau:
La machine physique pouvait être assimilée à une passerelle internet, et le réseau se configure comme un réseau privé réel, mais la couche de virtualisation introduit une complexité
supplémentaire qui se résoud différemment selon la solution retenue. L’installation d’un
serveur physique dans un environnement virtuel ensuite n’a pas pu être réalisé, notamment
à cause du fait que la virtualisation n’est pas prévue pour ce cas de figure. La documentation
associée se révèle ainsi très pauvre à ce niveau. Enfin, l’intégration avec le testbed d’OMF
n’a pas pu être achevée du fait de la difficulté supplémentaire introduite par la virtualisation. Cette intégration pourrait donner lieu à un prolongement possible de ce stage, ce qui
permettrait d’obtenir des résultats plus concrets des expériences menées.
• L’étude du protocole Pair-à-Pair m’a permis de voir une alternative prometteuse à l’architecture
classique des échanges internet Client/Serveur. La mise en place d’un réseau BitTorrent et
l’utilisation d’applications BitTorrent telles Transmission, Rivettracker, Opentracker, m’a
permis de comprendre en détail le fonctionnement d’un type d’implémentation de ce protocole et de la mettre en place dans des configurations réseaux variées. L’utilisation d’un
logiciel de capture de paquets m’a permis de suivre les échanges entre les différentes entitées. j’ai pris soin à chaque fois d’utiliser des applications libres, afin de permettre de
modifier le code source et ajouter de nouvelles fonctionnalités aux différentes applications
par la suite. Une application intéressante possible en modifiant le code de transmission est
de faire de la vidéo à la demande supportée par ce protocole. Il faudrait modifier la partie du code qui s’intéresse à la priorité de recherche des pièces d’un contenu (le plus rare
en premier ou bien de manière aléatoire) pour que les premiers paquets téléchargés soient
également les premiers à être visionnés. En fonction de la vitesse de téléchargement, un
intervalle de confiance plus ou moins large est définie par la position de lecture en cours et
le premier paquet non encore disponible. Cette application est illustrée figure 1.3.
• L’étude d’application Media-Center comme XBMC puis ELISA et le configuration de nombreux paramètres différents comme l’intégration d’une commande à distance, le choix et
l’intégration d’une interface graphique, la création de play-listes et l’intégration avec un
serveur web m’a montré les possibilités immenses des applications open-sources en terme
d’ajout de nouvelles fonctionnalités. La question du timeout qui déconnecte XBMC du
proxy peut être réglée en modifiant le code du proxy pour qu’il puisse envoyer des réponses
14
1.5 Conclusion
15
Figure 1.3 Application du protocole BitTorrent à la Vidéo à la Demande
périodiquement et ainsi concerver la connexion. L’ajout de fenêtres avertissant l’utilisateur
de la fin du téléchargement d’un contenu quelconque ainsi que d’autres fonctionnalités rendant l’interface graphique plus conviviale et user-friendly peuvent également être réalisées.
Des messages avertissant l’utilisateur du déroulement du téléchargement peuvent être créés.
le manque de temps, et de connaissance sur la programmation en C/C++ m’ont empêché de
réaliser ces tâches.
D’autre part, l’ensemble du projet ainsi que toutes les étapes et recherches que j’ai menées pour le
réaliser ont été repportées dans le wiki de NICTA pour conserver des traces pour les applications
futures du prototype ou simplement pour comprendre son fonctionnement.
Ces différents aspects m’ont permis d’acquérir une certaine expérience dans le domaine de Linux
et des applications open-sources. Le fait d’avoir étudié et configuré différents outils de virtualisation a largement joué à l’obtention de mon premier emploi en tant qu’ingénieur système à
l’INRIA Nancy au sein du projet grid 5000. L’étude de configurations réseau m’a permis de mettre en pratique les connaissances théoriques vues en cours. La découverte du protocole P2P et
de ses récentes applications m’ont beaucoup appris dans les nouvelles architectures internet. Je
tiens enfin à remercier l’ensemble des ingénieurs et chercheurs de NICTA, grace auxquels j’ai pu
acquérir toutes ces connaissances et qui m’ont permis d’évoluer dans un cadre très convivial tout
au long du stage.
15
16
16
Chapter 1
Part II
Main Report
17
C HAPTER 2
Introduction
2.1
PRESENTATION OF NICTA
National ICT (Information and Communications Technology) of Australia [49] is an independant
company in the business of research, commercialisation and research training. It represents the
largest organisation in Australia dedicated to ICT research and employs 426 people and trains
298 students. It runs around 25 research projects and holds currently 93 active and current patent
applications.
This company was established in 2002 by the Australian Government, and is funded by the Australian Government through the Departement of Broadband, Communications and Digital Economy and the Australian Research Council. This company has also a large range of partners, including the University of New South Wales.
In order to maximise the impact of its research efforts, NICTA has chosen to focus its activities in
four domains:
• Embedded Systems: developing smart products of the future
• Networked Systems: connecting the smart products of embedded systems together to constitute smart network
• Making Sense of Data: finding ways to make sense and extract value from the growing
amounts of data created by ICT systems
• Managing Complexity: developing efficient and reliable tools and processes to solve complex ICT problems
These four domains are located in five laboratories across Australia: two in Sydney, one in Canbera, one in Melbourne and one on the Gold Coast. Furthermore, NICTA has developed collaborations with the University of Adelaide and the University of Western Australia. These different
locations can be represented by the Figure 2.1.
19
20
Introduction
Chapter 2
Figure 2.1 Map of the different NICTA’s laboratories
2.2
THE TEMPO PROJECT
I have done my End of Study Project at NICTA, Sydney, Australia, in the Networked Systems
Team. I have worked with the Service Delivery and Testbed Framework project (formerly known
as TEMPO project) [25].
The first goal of this project is to develop and deploy an unified framework to control, measure, and
manage networking experiences, in order to increase the scientific rigor in the networking fields,
and make easier the future Internet researches. Indeed, only experimentations in real testbeds can
provide evaluation results close to reality. But many testbeds are built as part of a specific project
and stay rarely operational once this one is finished. This reason explains the need for an unified
control and management framework which overcomes this limit of reusability.
The second objective of the TEMPO project is to investigate schemes that minimize the energy
cost of a distributed services delivery plateform, and thereby, reduce both the energy costs and the
environment footprint, and explore the Peer-to-Peer domain. Indeed, the traditionnal approach of
the Internet, based on a Client/Server architecture, reachs its limits to provide online applications
such as Video-on-Demand (VoD), Web-serving or Network-Gaming (Massively-Multiplayer Online: MMO). At NICTA, the aim of the researches on the P2P paradigm is to discover alternatives
technologies for the traditionnal network structure and try to solve the issues encountered with the
Client/Server structure, in order to provide services more efficiently and more economically, regarding of the energy consumption and the costs required by the Internet Services Providers (ISP).
I am taking part in this first goal.
During my internship at NICTA, I have been included to the TEMPO project. I have thus used
its framework to realize the different experiences described in the chapter 6. However, due to a
lack of time, I have not provided the integration of the STB in this framework. The BitTorrent
applications have been installed in the testbed’s computers, but their execution using the OMF
protocol has not been provided.
2.3
SUBJECT OF MY INTERNSHIP
In the context of the TEMPO project, my internship has three main goals:
20
2.4 Organisation of this report
21
• Design a prototype hosted in a Set-Top-Box (STB) which is able to provide Media-Centre
services using Peer-to-Peer applications at background. Both having to be strongly isolated
to avoid security or privacy issues. This prototype will implement in the P2P container
algotithms designed by NICTA researchers aiming at minimizing the energy consumption
and improving the efficience of the new P2P schema.
• The second objective of this internship is to integrate these algorithms in the STBs as a proof
of concept demonstration of energy aware Peer-to-Peer service delivery.
• To finish with, the progress and documents of this project has to be tracked, recorded and
explained in the NICTA’s wiki. In this way, a tutorial, provided as an appendix, has also
been written about the OpenVZ virtualization tool.
2.4
ORGANISATION OF THIS REPORT
This master thesis is organised as follows:
• Chapter 3 defines the context of this internship by giving an overview of the trends in energy
consumption and some preliminary results of a distributed architecture.
• Chapter 4 presents the different virtualisation tools. It also presents the reasons that lead us
to chose OpenVZ. It then gives a summary of the difficulties encountered to integrate this
tool on the NICTA Testbed.
• Chapter 5 describes the integration of P2P and Media-Centre applications with the prototype
of STB and the virtualization tool chosen.
• Chapter 6 summerizes the different demonstrations which have been realized and the network configurations. I will also explain the integration into the NICTA testbed OMF and
the reasons why this integration has not completly been achieved.
• Chapter 7 concludes this report and shows the possible evolutions of my achievements
21
22
Introduction
22
Chapter 2
C HAPTER 3
This chapter describes the context of this internship. During these five months at NICTA, I have
been integrated in the TEMPO project. Through this project, NICTA is working in collaboration
with Seventh Framework Programme and Nano Data Centers in order to develop new ways to
deliver content and web-services to users.
3.1
NEW SERVICES AVAILABLE THROUGH THE INTERNET
The current model of content delivery for Internet services, such as web-browsing, e-mails, or
files transfers, relies on a traditionnal approach of the Internet through the Client/Server paradigm
and the TCP/IP protocol. However, this approach reaches its limits since new Internet services
have appeared, like IPTV, VoD (Video on Demand), VoIP (Voice over IP), or MMO (Massively
Multiplayer Online), where the protocol TCP is not efficient enough (for instance, VoIP does not
use TCP protocol for the data delivery). Indeed, such rich media content can saturate the capacity
of a traditionnal network based on a Client/Server architecture. The server appears as a critical
point of failure requiring constant maintenance, because every requests have to be processed by
the Server. This aspect shows the low scalability of the Client/Server architecture. Furthermore,
the TCP/IP protocol is not efficient to process with Real-Time applications. For instance, retransmission mechanisms, typically used in this protocol when a loss of packets is detected, are not
needed for services such as VoD or VoIP, and can even be a drawback using the bandwidth unnecessarily. As following, it is necessary to implement an other way to deliver content. The figure 3.1
shows some recent applications available through the Internet:
In the meantime, some solutions have ever been found to overcome these disadvantages: as example, Skype [52], developped by Kazaa, uses a P2P approach to deliver the media content. Skype
uses both the TCP and the UDP protocols to improve this delivery [24] . This application shows the
efficiency of the P2P protocol for content delivery: indeed, Skype provides VoIP, instant messenging, and video-conferencing, to 42 million users connected every days, and 443 million registered
[60].
23
24
Chapter 3
Figure 3.1 Evolution of the Internet and examples of new services available
Furthermore, the storage capabilities and the efficience of the Set-Top-Boxes (STB) have improved
a lot. The purpose of NICTA, together with the NaDa partners [47], is to develop ways to take
advantage of the ever increasing capabilities of STB to move functionality from the Data-Centres
to the end-users STB.
The aim of the TEMPO project is to explore the P2P paradigm as an alternative technique to
the Client/Server architecture. Thus, the main idea of the NaDa project is to virtualize the STB,
granting a part to the end-user, and connecting the other part to a managed P2P-network, this
second part constituing a Nano Data Centre. Each user takes part in the content delivery network,
instead of a single server located in a data-centre.
3.2
ENERGY ISSUES
The Client/Server structure is based on Data-Centres, concentrating the whole content available.
The energy consumption represents 26% of their maintenance costs (20% for the electricity and
6% for the cooling equipments) [38].
Furthermore, each Watt to serve losses 0.7 Watt to power distribution losses and cooling. So, for
each Watt effectively used to serve, 1.7 Watt are needed: η ≈ 0.59, which means that there are
41% of energetical losses in data-centers.
These losses are constituted by power distribution losses and cooling. Power distribution losses
are simpler to track than cooling losses and are mostly due to power transmission and switching
losses and represent 8% of the total loss as illustrated in the following figure 3.2 [45]
24
3.2 Energy issues
25
Figure 3.2 Loss of electrical power during its distribution
As following, there are 41 − 8 = 33% of energetical losses due to data centres cooling. The
cooling is used to maintain the temperature in the data centres and to avoid computer overheats.
The energy consumption cost can rise to 2.3M $ per month to power high scale data centres and
33% of this cost is due to the high concentration of running computers in the same place.
Some solutions can be found to reduce this cost. Virtualization, as instance, merging several dedicated physical servers in a single one with the same level of security and efficiency, can contribute
to this purpose. But new alternative approaches for the structure of the Internet network can also
be explored: the P2P algorithms can attempt to solve the Client/Server issues and provide contents
or services more efficiently. This possibility leans on the increasing capabilities of the STB, which
can to some extend be compared to Nano Data-Centres. Through this protocol, ISP can lower their
operational energy costs and environmental footprint.
But NICTA wants to go further. Because people are increasibly concerned about the energy consumption of their electric equipments, (the part of consumer electronics in electricity usage is
always increasing and represents 11% of the US residential electricity in 2006, [48]), NICTA
specifically develops distributed schemes which power down individual STB when not in use by
their owners, maintaining the same level of efficiency. By using NICTA’s proposed P2P energy
aware schemes to share content, end-users can reduce their electronical costs and environmental
footprint.
25
26
3.3
Chapter 3
NANO DATACENTERS
The NanoDatacenter (NaDa) [47] project is funded by the European Union 7th Framework (EUFP7). It proposes an innovative and orthogonal approach to traditional data centres, by hosting
services in nano datacentres as illustrated in Figure 3.3 [53]. These nano datacentres are deployed
in the same “boxes” requiring those services, such as home gateways or a STBs. The role of the
network now shifts from connecting these STB devices to central data centers, to interconnecting
the nano datacentres. The P2P network then becomes a communication medium supporting the
end-to-end service of the datacentre. This shift allows for many innovative solutions in providing
services and is also expected to reduce the overall energy consumption, as many of these emerging
STB devices only require passive cooling.
STB
NADA
Enhanced STB
App
App
App
OS
Secure & Managed Kernel
NADA
Monitoring
Management
Functions
Monitoring
Management
Functions
Data Centre
with
services ( )
Figure 3.3 Current Set-Top-Box architecture compared to the NanoDatacentre approach.
NaDa proposes to develop a novel secure and managed peer-to-peer (P2P) communication infrastructure for the nano datacentre backbone. However, NaDa makes the assumption that all the STBs
are always On, even if not currently used by their owners. This assumption prevents the NaDa’s
approach from reaching its full potential in terms of energy saving. Indeed, an STB currently has
an average power consumption of 43W in active mode (depending on its functionality set) [51],
while many regulatory entities such as the Australian Ministerial Council on Energy push for a
target of < 1W in stand-by mode by 2012 [39]. This consumption difference (i.e. 42W) becomes
significant when multiplied by the large deployment of STBs (approximately 148M in the US in
2007 [51]).
The deployment of virtualization devices will enable the emergence of innovatives services. The
main idea of the Nano Data Centers project is to implement a new architecture in which the
Internet Services Provider (ISP) adds through virtualization Virtual Machines (VM) to the home
gateway located at the user’s permises. The deployment of such services will be done using the
architecture shown in Figure 3.4. In this architecture, every distributed application, which has
previously developed its own application tracker, will first contact the NaDa ID provider. This
26
3.3 Nano DataCenters
27
provider will then return available nodes where the application can be instantiated after checking
with the NaDa management block. The application tracker then contact the different nodes through
the Comm/API block and start it dedicated slide. For further details please refer to [58].
The ISP then can rent a VM, or slice, to a third part service providers such as Google, YouTube, or
Skype, so that they can run their applications on the users’ computers themsleves. Virtualization
allows the deployment of new services such as P2P services in a trusted and controlled environment, unlike the user’s PC. This will avoid the inclusion of additional Hardware and reduce the
load on traditionnal data-centres.
Figure 3.4 Proposed Nada Architecture
The NaDa project will in a first time deploy two kinds of service: a Video on Demand service and
a Second life-like application.
First results, by our partner Telefonica in Spain, show that this kind of architecture would allow
to save up to 60% compared to the Energy consumed by the datacentre architecture [56]. These
results have been found in the context of the deployment of the netflix and YouTube services.
Nevertheless, this new architecture supposed to maintain all the home gateways always on to
provide the service. This could be improved thanks to a distributed algorithm that dynamically
turn off gateways if not used and the service is maintained. Indeed, measurements showed [46]
that for the IPTV service only an average of 7% of user are concurrently using the service. This
order of magnitude has been proved to be equivalent for VoD service and YouTube [44]. We
therefore aim at using only 7% of the set of home gateways to provide the same service. This
figure has to be put in perspective with the ever increasing number of intelligent home gateways
(nearly 50 million in the US in 2006) and the their average consumption of 42 W according to the
Australian Ministerial Council on Energy.
27
28
3.4
Chapter 3
STUDY
The TEMPO project is currently working with the NADA project to push P2P applications over
Set-Top Boxes. Because of security considerations, and the fact that a part of the STB should
be shared in the P2P-network, a virtualization tool have to be found and configured to provide a
strong separation between the user applications (Media-Centre) and the P2P applications running
in the background. In each part, some applications have to run with an efficiency level comparable
to the one obtained without virtualization. This structure is required in order to build a TestBed
compliant with NADA requists. In this way, the choice of a virtualization tool, its installation and
its configuration, have been the first part of my internship. It has been necessary to configure a
way of communication between the separated entities.
Then, on this platform, I have installed both BitTorrent applications and a Media-Centre. This has
led me to reach informations about the P2P and the BitTorrent protocols and their implementations to provide this feature complying with the NICTA requirements. As instance, the BitTorrent
applications have to be open-source to let the TEMPO-team implements their new schemes. The
Media-Centre has to provide high quality display rendering and to let the configuration of a lot
of parameteres, such as http-connection, GUI aspect, possibility of control via a remote device.
The figure 3.5 shows the structure of the prototype of STB expected by NICTA. In fact, considering that the Media-Center needs direct access to the hardware, an other structure has finally been
created with the Media-Center in the Physical Node (HN) instead of a VPS. The isolation is still
provided with this new structure.
Figure 3.5 Structure of the STB-prototype created
28
3.4 Study
29
Demonstrations of this prototype, first using the network created with the virtualization tool, which
basically behaves like a private network, then using the NICTA OMF testbed, have been realized to
show the usability of the prototype of STB created. The figure 3.6 shows demonstration expected
by NICTA: The end-user has only access to the Media-Center. When he choses a content, it is
dowloaded from a HTTP-proxy located in a isolated VPS. If the requested content is not available,
instead of returning the 404-error, the proxy executes scripts which will reach the content from the
P2P network by the BitTorrent application. In this case, the end-user only notices a longer time
before the content is displayed.
Figure 3.6 Sequence diagram of the different steps followed during the demonstration
To finish with, during this internship, I have provided NICTA with a prototype of STB ready to implement the new algorithms developped by the TEMPO team. I have also learnt a lot about Linux
and about virtualization of servers, mostly with OpenVZ, but also with Xen. I have also learnt
about P2P protocol, setting up P2P clients and tracker, and about the configuration of open-source
software. Some networking aspects, such as iptables, routing, ssh connection and protocol, have
been used a lot to set up this demonstration, it has been a realy instructive part of this internship.
29
30
30
Chapter 3
C HAPTER 4
In the context of the NaDa and the TEMPO project, a solution has been implemented to provide the
separation between the end-user applications in the foreground and the background of the STB,
hosting content-delivery applications and taking part of the P2P managed network. The NaDa
project has decided to proceed by virtualization.
In computer science, virtualization consist in the different software or hardware techniques used to
let the running of several Operating-Systems in one physical server. These OS operate separatly,
and behave as if they were on different physical servers. Virtualization provides a logical rather
than physical view of the underlying hardware [57].
To provide the STB prototype with virtualization, I have studied its different implementations to
find an appropriate one, I have then installed and configured it in order to provide the required
features.
The main parameters to be configured were the network inside the Containers and the display
of the graphic applications. This last point has arosen troubles, because the display from inside
Containers can not be directly available. After a while, considering some results from Thomson
company, we decided to switch on an other virtualization tool, Xen, which is said to provide better
scalability.
4.1
INTRODUCTION TO VIRTUALIZATION
The term virtualization broadly describes the separation of a service request from the underlying
physical delivery of that service.
The virtualisation tools create Virtual Private Servers (VPS) or Virtual Environments (VE) on one
single HardNode (HN).
The main purpose of the virtualization is for web-hosting. It is very frequent for en enterprise to
own different servers, for instance a web server, a mail server, a DHCP server, and on each of them,
31
32
Chapter 4
the global load rarely exceeds 15%. A 15% loaded server does not consum much more electrical
energy than a 90% loaded server, and the idea, taking in account the inceasing performances of
modern Hardware, is to merge 3 or 4 servers in one HN. The traditionnal isolation provided by the
Hardware (one server on one computer) used to avoid that the crash of one server disables every
servers installed on the same machine, is replaced by the isolation provided by virtualization,
where the crash of a server only disables the VE where it is installed [40].
In this way, I have written a tutorial provided as an annexe to explain how web-hosting can be
performed via a virtualization-tool such as OpenVZ.
The main benefits provided by virtualization can be summarized as following [50] [41]:
• Improved security Consider a server hosting for instance a mail-server, a web-site, a
DHCP-server and a DNS-server. All these applications will handle network requests, and
some of them can contain security holes. The benefit of the virtualization is to provide isolation. In this way, if one of these applications is compromised, the others will still work
normally.
• Better use of the Hardware ressources Provide server-separation to improve security lead
to increased costs of ressources (power needed to make all these servers running, maintenance of a lot of them) and modern Hardware is underemployed in this way. Virtualization
tools provide the benefit of dedicated server without these drawbacks.
• Tests and development Virtualization tools provide the possibility of testing tricky and
armfull configurations without damage the HN. Indeed, if a configuration leads to a crash,
backups can be made before testing, and anyway, recreating a new VPS from scratch is quite
easy. Furthermore, developpers often need to try a lot of different configurations to test their
software. With virtualization tool, a VPS can be created for each Linux-distributions without
need to provide a partition for each of them.
4.2
NADA PROJECT AND VIRTUALIZATION
However, the purpose of the virtualization tool here is quite different. It consists on providing
separation to the user environment, where heavy graphic applications such as a media-centre can
be installed, and the ISP environment, trusted and controlled by a third party services provider. In
this second environment, P2P applications have been installed to provide a P2P network aiming at
reaching media-contents.
During the first demonstrations, I have decided to use the VPS and the virtual network provided
by OpenVZ as a private network, and install the Tracker, the P2P applications, some http-Servers
and the media-centre in the VPSs. Some network-configurations (routing tables, IP-forwarding)
have been performed to provide the communication between the VPSs, and from the ouside to the
private network.
In this way, and in order to follow the NaDa requirements, the virtualization tool provides:
• A strong isolation between the different services
32
4.3 Comparison of the different kinds of virtualization
33
• A simple way of communication between the separated entities: possibility to create and
configure a private internet network inside the virtual machines having the same behavior
as an usual internet network (LAN-like)
• Acceptable performances to let the working of heavy applications such as a Media-Centre
without freezing
4.3
COMPARISON OF THE DIFFERENT KINDS OF VIRTUALIZATION
All virtualization techniques rely on a low-level software layer called Virtual Machines Manager.
In the same way an OS manages multiple user-processes, the VMM manages multiple Virtual
Machines (VMs) and hide the physical hardware from the guest OSes. As following, there can
not be direct access from applications to Hardware anymore, because all IO and low-level CPU
instructions are handled by the VMM. These instructions are called privileged instructions and
are intercepted by VMM before execution. The VMM then emulates the requested behavior. The
other instructions, can be directly executed. The x86 architecture offers four levels of privilege
known as Ring 0 to Ring 3 to OS and applications to manage the access to the Hardware. While
user level applications (non-privileged instructions) typically run in Ring 3, OS needs direct access to the hardware and executes its privileged instructions in Ring 0. Here is the architecture of
a computer-working without vitualization features provided 4.1 [59]:
Figure 4.1 Architecture without virtualization
Without virtualisation, the instructions from the applications and the OS are directly sent and executed by the underlying Hardware. But because this structure can not be followed anymore when
virtualizing, some solutions have been implemented to let the VMM handle the privileged instructions: These different solutions have led to different kinds of virtualization [59]:
33
34
4.3.1
Chapter 4
Full-virtualization
This provides a complete simulation of the Hardware. All software or Operating-Systems capable of execution on Hardware can be run in Virtual Machines. The VE does not know being
virtualized, and behaves exactly as if it was installed directly on the Hardware [29].
The privileged instructions (intended to have strong effects on the Hardware) are emulated and
replaced by new sequences of instructions with have similar effects on the Hardware (but not the
same!). This operation is called binary-translation 4.2 [59]. The privileged instructions are intercepted by the VMM before execution. The VMM handles these instructions by binary translation
and send them to the computer hardware. The user-requests (non-privileged instructions) are directly executed.
Figure 4.2 Full-Virtualization architecture
Virtual Machines
The virtualization by Virtual Machines is an instance of full-virtualization. Qemu for example
is an implementation of this kind of virtualization [21]. This kind of virtualization consists on
a software running on the Host-OS. This software provides the installation and the working of
different Guest-OSes or Virtual Machines. This is a full-virtualization, the Virtual-Machines are
working as they were not virtualized. This solution provide isolation, but has low performances
because the kernels are installed one on the other. The figure 4.3 shows the structure of this
virtualization. This solution is very close to the emulation, but here, the processor and the RAM
(Random Access Memory) are directly accessible from the Virtual Machines which is not the case
for Virtual Machines running by emulation.
As following, the Virtual Machines virtualization has better performances than the emulation, because the instructions to the processor and the RAM are directly executed and there is no emulation
layer to provide this execution.
34
35
Figure 4.3 Virtual-Machines Virtualization Structure
User-space kernels
A Guest-OS is running as a classic application on a Host-OS. This solution has very low performances, because the OSes are installed one on the other and there is no independance towards
the Guest-OS. This solution is mostly used for kernel development purposes. This is a paravirtualization tool.
Figure 4.4 User-Space Kernel Virtualization Structure
Here are two solutions of virtualization by kernel in User-space:
• User Mode Linux [27], mostly used for kernels development and experimenting with new
kernels and distributions according to its official website.
• Cooperative Linux or co-Linux [3] is a open-source software used to let the running of a
Linux kernel in a Windows environment without significant loss of performances. It does
not simulate a complete machine instead of VMWare for instance, most of the computer’s
drivers are not accessible (video and sound chipset for instance) One the Hardware is in-
35
36
Chapter 4
stalled Windows, wich let the running of both a Linux and a second Windows kernel on top
of it.
4.3.2
Para-virtualization
This provides an interface to the Virtual Machines similar but not identical to the Hardware. The
OS has to be modified to provide the installation and the running on the virtualization-level [32].
The OS has to be modified to replace non-virtualizable instructions by hypercalls to the virtualization layer. The main asset of para-virtualization is low virtualization overhead (the Guest-OS is
modified to provide direct calls to the virtualization layer instead of binary translation, in this way,
the guest OS has a structure closer to the Hardware than with full-virtualization, where the OS is
unchanged) 4.5 [59]. The user-requests are still directly executed, while the privileged instructions
are provided from the paravirtualized Guest OS as hypercalls send to the Virtualization Layer (or
hypervisor). As the hypervisor stands in a level of privilege closer to the Hardware, it is said to
execute in a Ring -1 level.
Figure 4.5 Para-Virtualization architecture
Virtualization by Hypervisor
An hypervisor is a very-light Operating-System optimized to provide access from the Guest-OS
to the HardWare. Virtualization by Hypervisor is the main kind of para-virtualization. In the
preview technologies, the Host System is the only one to have direct access to the HardWare.
With Hypervisor, the Host-System share this access with the Guest-Systems. Every Systems have
to go through the Hypervisor to access the HardWare. The machine boots on the Hypervisor,
which start the Host-System and then, the user logins on this system. He can then have access
to the Guest-Systems. As the Hypervisor is a very-light OS, good performances can be expected,
good enough to run a heavy graphic application such as a Medi-Centre in it.
These virtualization tools can provide para and full-virtualization.
36
37
Figure 4.6 Hypervisor Virtualization Structure
The most famous virtualization tools using a hypervisor are Xen and VMWare:
• Xen [36] Is a Virtual Machines hypervisor developed by the Cambridge University in UK.
The Guest-OSes are running in a domain. Xen is directly working on the Hardware.
• VMWare [28] Actually, VMWare deals with a lot of different kinds of virtualization, VMWare
is a company specialized in the design of virtualization tools. But it is a non-free and noopen-source virtualization solution, and has not been retained because of this.
4.3.3
OS-level virtualization
Only a single kernel runs on the computer. This kind of virtualization provides a separation of
the applications in Containers, or VPS (Virtual Private Servers), or VE (Virtual Environment).
The main advantage is the performances, because VPS are kernel space structure. According to
the OpenVZ official website, [18], the overhead-time (time spent by the OS to manage itself) is
very low, and does not exceed 3% of over-time per VPS. The kernel usually provides ressources
management tools to limit the impact of one VPS’s activity to the other ones. In the cases of
Linux VServer and OpenVZ, this kernel is a Linux kernel modified to provide isolation. As a
matter of fact, this architecture of virtualization is still comparable with the architecture without
virtualization 4.1, so that we often prefer use the term of isolation instead of virtualisation to
describe OS-level virtualisation.
On one Hardware support, is running one Linux-like OS, modified to provide isolation between
VPS. As following, only the Hardnode (HN) has direct access to the real Hardware (Network and
Graphic Cards, for instance) and the VPS has only access to virtualized one, even if it is still
possible to enable direct access to some devices from inside a VPS.
An OS-template have to be installed on the Linux modified-kernel. With this OS-template are
created the different VPS. The figure 4.7 shows the architecture of a computer implementing a
OS-level virtualization. On the Hardware is installed the kernel and the HN which lets a user login
and execute the same applications as on a generic Linux kernel. Then, VPS are created from the
HN.
37
38
Chapter 4
Figure 4.7 Container Virtualization Structure
The main solutions of virtualization by Containers on Linux plateform are OpenVZ and Linux
VServer, both of them can be regared as improvements of the chroot mechanism.
• Chroot is a mechanism allowing the user to run applications in a different file-system. This
is many used for safety reason, to avoid for instance the malicious use of buffer overflow.
The OS-level virtualization solutions can be seen as improvements of this mechanism.
• Linux-VServer [11] has been firstly be named Virtual private servers and security contexts
and managed by the Solucorp society. This project is free and open-source. The web-site
LinuxFR.org uses this solution of virtualization for web-hosting and handles 100000 visitors
a day.
• OpenVZ [18] has been developped by the SWsoft company. The commercial model of
SWsoft consists on a non-open-source and no-free solution named Virtuozzo and a free and
open-source projet OpenVZ. Even if the better solution with all the features and support
is Virtuozzo, OpenVZ is still a very efficient virtualization solution. OpenVZ modifies
more deeply the Linux-kernel compared to Linux VServer, and then some more features are
available.
The table 4.1 shows the differences between the virtualization solutions presented:
Technique
Modified Guest-OS
Different Guest-OS
Performances
Full-virtualization
binary translation
No
Yes
+
Para-virtualization
VMM-aware Guest-OS
Yes
Yes
++
Table 4.1 Comparison of virtualization implementations
38
OS-level virtualization
Containers (jails)
Patch applied to the Host-OS
No
+++
4.4 Installation/configuration of OpenVZ
4.3.4
39
Choice of OpenVZ
The kind of virtualization provided by OpenVZ has been retained for the following reasons:
First, the isolation by Containers is the simplest to implement. There is a very small loss of
performances induced by the virtualization layer, so the running of heavy applications will not be
a issue with this virtualization solution. OS-level virtualization appears as a very light and efficient
solution to provide isolation between the VPS. Then, in this kind of virtualization, OpenVZ and
Linux VServer are the most popular ones, and they can both be integrated in a Linux plateform.
So, these two solutions have retained our attention.
About OpenVZ, the documentation is much better written, the forum is very active and the wiki
very well provided. To continue with the documentation, some other web-sites provide documentation about OpenVZ as well, more than for Linux VServer [1].
About the features provided, OpenVZ provides two kinds of network, an Internet-like and a
Ethernet-like Network. The use of the both of them could be an asset for the demonstration
purpose. Indeed, for the demonstration, venet seems to be more accurate (the isolation is better
provided) but to test network configurations and simulate private networks, eth0 seems to be more
relevant. Both of these solutions, according to their official websites [18] [11], are highly scalable
virtualization technologies with near-zero overhead, strong isolation between the different VPSs,
and ready for production use right now.
Finally, some more features are provided by OpenVZ, and not by Linux-VServer: we can quote
the live-migration, or a much better dynamic allocation of the memory ressources to a VPS. This
could be usefull if the ressources needs is changing during the experiments.
4.4
INSTALLATION/CONFIGURATION OF OPENVZ
This part describes the first operations made with OpenVZ: first the installation and the configuration of OpenVZ and then the creation and the configuration of VPSs. For more details, please
refer to the OpenVZ tutorial.
Then, I will focus on the networking and the two ways of providing the Internet access to the VPS.
To finish with, the display of graphic applications was capital for this project, I will thus present
the different ways of providing graphic applications from inside a VPS. However, the direct access
to the Hardware from the VPS, needed by the Media-Centre application to run, is not provided by
the OpenVZ developpers, after having tried some tricks to workaround this issue, we have decided
to install and run the Media-Centre in the Hardnode instead of in a VPS.
4.4.1
Installation of OpenVZ
The installation has firstly been made on an instance of the Set-Top-Boxes located in the locals of
NICTA and included into the OMF network. To provide OpenVZ on this computer, I have first
installed Debian 5.0 Lenny and then OpenVZ following both the tutorials from the official website
of OpenVZ and the howtoforge website [54]. I have chosen this distribution of Linux, because this
is one of the more user-friendly Linux distributions, and because, unlike Ubuntu, the integration
39
40
Chapter 4
of OpenVZ is much better provided, easier with more documentation in the Internet. OpenVZ is a
Linux-modified kernel. A patch has been added to the generic version of Linux 2.6.26 to provide
the isolation of the VPS and their management. It could be installed the same way the generic
version of Debian has been installed, via CD, or USB-stick, or by compiling the kernel, but I have
chosen to install first Debian, and then, download the pre-compiled OpenVZ-kernel.
4.4.2
Configuration of OpenVZ
OpenVZ provides a tool, vzctl, to control and manage the VPS. It is then possible to create and
make the whole configuration of the VPS by this tool.
For the internship purpose, two VPS at least have been created: One for the P2P-applications, and
the second one for the Media-Centre. Once the VPS created, virtual network have been set, to let
the VPS communicate together and with the HardNode (HN), and to provide Internet access to
them. Then, the different applications have to be installed from inside the VPS. For the MediaCentre application, a way has to be found for the displaying, because the direct access to the
Graphic-Server is not provided by default from inside the VPS.
The creation of VPS has been done using the vzctl command:
vzctl create 101 - -ostemplate debian-5.0-i386-minimal - -config vps.basic
We are using a Debian image, because this is the simplest way to install VPS on the HN, considering that the HN is also a Debian-like server. The OS-template is a compressed image containing
the minimal configuration of a Debian-like server. Two configuration files are used:
• /etc/vz/vz.conf the global configuration file with the main settings ruling every VPS, and
some security settings (iptables, as instance). In this file is specified where the OS-template
are stored, where are located by default the VPS file system. This configuration file is used
to create a new VPS.
• /etc/vz/conf/$CTID.conf where $CTID is the ID ot the VPS. This is the local configuration
file, ruling the $CTID VPS. This file contains informations about the network, the working
of this VPS (on HN-boot starting, which distribution of Linux, which hostname, IP-address,
and so on) and every quotas this VPS has to respect: memory, TCP-buffer-size, as instance.
This file is created when a VPS is created, accordingly to the informations provided from
the global configuation file. This file can be edited to provide some more features to the
VPS or to modify its behavior. Every modification can be realized either using the vzctl
command, which will basically modify this configuration file, or editing it with a text-editor
such as vim.
4.4.3
Network inside VPS
There are two ways of providing Internet access inside the VPS:
• venet or virtual network. It is a OpenVZ network, where the whole configuration is made
from the HN. There is no way to configure the Network from inside the VPS, and this,
for safety reasons. This configuration is mostly used in uncontrolled environment, where
40
4.4 Installation/configuration of OpenVZ
41
the VPS-administrator can not be trusted. This way does not let the VPS-administrator
configure the network, or sniff the virtual traffic. The IP-addresses are set permanently,
and this is almost the only parameter to set. There is no MAC-address, so no possibility of
setting up a DHCP-server: The configuration can only be static.
• veth or virtual ethernet. It is an ethernet-like network. each VPS has its own MAC-address.
It s possible to sniff the traffic, even if OpenVZ provides tools to forbid it from the HN. The
virtual network behaves like a real private network. A DHCP-server can be set, letting a dynamic allocation of the IP-addresses. The VPS-administrator can configure his connection
by himself.
The two ways of configuration have been tested. For the experience purpose, the VPS have to
be strongly separated and the access from one to the other have to be restricted: We do not want
traffic-sniffing as instance. Furthermore, we prefer prevent the VPS-administrator from setting his
own network configuration by himself. For these reasons, the venet network has been set.
To provide simulations of a virtual private network, we have firstly used veth, and then, because it
was much more easier, venet.
Here are summerized the stages followed to set up the Internet acces from inside a VPS with venet:
vzctl set $CTID - -ipadd 192.168.10.101 - -save to set up the IP-address.
vzctl set $CTID - -hostname try - -save to set up the hostname
vzctl set $CTID - -nameserver addr1 - -nameserver addr2 - -save to set up the DNS IPaddresses
vzctl set $CTID - -searchdomain name2 - -searchdomain name2 - -save For the names of the
DNS servers
These two last instructions will overwritte the /etc/resolv.conf file. Now, you can start your VPS
and enter inside:
vzctl start $CTID
vzctl enter $CTID
The installation of applications has been done as this VPS is a real server, using apt-get install
command.
4.4.4
Graphic applications inside VPS
There are two ways of providing the display of graphic applications from inside the VPS. However,
neither of them provide a direct access to the Graphic-Server Xorg installed on the HN.
• The X-forwarding: The connection from the VPS to the Graphic-Server is granted by the
xhost application which authenticate the authorized users. All you have to do is to add the IP
address to the authorized clients list in the server-side by the xhost+ @IPclient command,
and in the VPS, export the environment variable DISPLAY to the server IP-address: export
DISPLAY=@IPserver:0.0. Then, you can start the graphic applications inside the VPS
and the display will apprear on the server-screen.
41
42
Chapter 4
• SSH tunneling: The connection from the VPS to the Graphic-Server is provided by SSH
connection. This way let connection and authentication happen at the same time. A Client
having rights to connect by SSH to the Graphic-Server can then display graphic applications.
All you have to do is to ensure you can connect to the HN by SSH, and use the ssh -X
@IPhote.
These two ways are quite simple to set up, however, the direct-access to the Graphic-Server is not
provided. This direct-access is needed to let the working of heavy graphic applications such as a
Media-Centre from inside a VPS.
In fact, it is still possible to make a Media-Centre application run from inside a VPS with both of
the ways explained above, with some workarounds, but the application is very freezing and hardly
usable for the end-user.
I have tried to install a KDE desktop and a X-server like Xorg inside a VPS. But this still has
not resolved the problem, because the display then is provided by VNC, which does not grant
direct-access to the X-server.
Trying to start the Graphic-Server from inside the VPS, once the Graphic-Server in the HN was
shut-down, does not work either because the Graphic-Server needs direct-access to the Hardware
and the Graphic-Card.
So, considering that OpenVZ provides some tools to export real-devices from the HN to the VPS,
I have exported the devices I needed to provide direct-access to the Hardware, to the VPS. In fact,
the devices exported seem to be virtualized, the Graphic-Server still does not recognize them and
can not start.
The architecture of OpenVZ lets this possibility to the VPS, but this seems to have not been
implemented yet. The main purpose of OpenVZ is for Web-Hosting, where there are no graphic
applications, so this issue, however regularly required, has not been set yet.
The solution found was to let the working of the Media-Centre in the HN instead of in a VPS. The
P2P applications will run in VPS. So, the isolation between the two functions is still provided,
even if we would have prefered having the Media-Centre application inside a VPS.
4.5
XEN
Considering some results from Thomson [4], showing that Xen is more scalable and stable than
OpenVZ, we have decided to switch to Xen instead of OpenVZ and tried to provide the same
experiments with this virtualization tool.
The figure 4.8 [4] shows that for high CPU, the isolation is not very constant for OpenVZ Virtual
Machines. For Xen, the isolation remains the same and the behavior of the Virtual Machines is
much more constant.
This report shows a general better behavior of the Xen isolation compared to the OpenVZ isolation.
Isolation is a strong requirement for this project, and for this reason we have decided to switch to
Xen.
42
4.5 Xen
43
Figure 4.8 Effect of CPU load by another Virtual Machine
Basically, Xen provides full-virtualization (the Guest-OS is not aware of being virtualized, but
the performances are lower, and this feature is not compulsory for the demonstrations) and paravirtualization (the Guest-OS is created from images provided on the Xen website, the performances are better and it is simpler to administrate). We have chosen to use para-virtualization.
Because we did not want to lose the structure created with OpenVZ, I have decided to build an
other partition, where Xen has been installed. The advantage is to provide isolation between the
Debian-OpenVZ partition and the Debian-Xen partition. Then, the crash of one partition, would
not damage the second one.
The configuration and the installation of software on Xen Virtual Machines have also been made,
but because NaDa uses exclusively OpenVZ, we have decided to switch again to OpenVZ then.
4.5.1
Partitionning
There are 4 primary partitions maximum in a computer. But it is possible then to create from one
of them a partitions container, which means that in the extended one, some more partitions can be
created. These additionnal partitions are called logical partitions. In order to be usable, a partition
should be formated, meaning that a file format must be set for this partition. Only the extended
one should not be formated, because its prupose is only to contain logical partitions. Any primary
43
44
Chapter 4
partition can be changed into a extended partition, but it is better to select the last one in order to
have more flexibility to resize the space allocated to the logical partitions.
On Linux systems, the partitions are written from sda1 to sda4 for the primary partitions. The first
letter stands for the kind of disk, and can be h or s. The sda5 and following are allocated for the
logical ones. /dev is the folder used to communicate with these partitions, as instance, /dev/sda1
is used to interact with the content of the sda1 partition.
In this case, a partition has been used for the OpenVZ kernel, the second one for the /home directory, which will be shared between the OpenVZ and the Xen partition with the mount command
or by the edition of the /etc/fstab folder . Then, a partition has been created for the Xen kernel.
An other partition has been created for the swap, which is in some aspects a RAM extention. I
have created this one in a logical partition, to keep the possibility to add more partitions if needed
in the future.
The result can be shown via the command cat /proc/partitions
Figure 4.9 Partitions structure
4.5.2
Installation and configuration
Then, the installation of Xen has been realized following the instructions of the howtoforge website
[55].
The installation and the configuration of Virtual Machines with XEN have been made following
the instructions of this tutorial.
A configuration file /etc/xen-tools/xen-tools.conf containing the default values is used by the
xen-create-image script when creating a VM. In this file are specified here the VM images will be
stored in the host file-system, which distribution is used by the VM, if a password is needed or not
while login in a VM. The creation of a VM can be done with this command:
44
4.6 Conclusion
45
xen-create-image - - hostname=xen1.example.com - -size=4Gb - - swap=256Mb
- -ip=192.168.10.101 - -memory=128Mb - -arch=i386 - -role=udev
A configuration file for this VM is the created and located at /etc/xen/hostname.cfg
These following commands let the management of the VM:
xm create /etc/xen/try.cfg creates the VM from the configuration file created by the preview
command
xm console try enters in the VM. we can as well use ssh-client to connect to it. Then, the installation and the configuration of applications has been made as the VM were a real server.
The /etc/xen/xend-config.sxq configuration file has been edited to provide the Internet network
inside the Virtual Machines. This configuration file lets the choice between connecting the Virtual
Machines directly to the Internet or through the Dom0 which acts like a gateway. This second
configuration has been chosen.
A demonstration has been provided to validate the architecture using Xen instead of OpenVZ. I
have installed the same P2P and Media-Center applications. Because OpenVZ is used by NaDa,
we decided to switch again to OpenVZ. Thus, the further steps have been proceeded with OpenVZ.
4.6
CONCLUSION
The virtualization tool is now installed and configured. On the top of this one, we will install the
further appications to continue the construction of the STB.
This step has teached me a lot about Linux in general, and about different virtualization tools
more specifically. I have learnt the problematic around virtualization tools, and their main usecases. Virtualization tools are mainy used for web-hosting, reducing the maintenance costs by
merging different dedicated web-servers in one physical node. I have studied and configured two
implementations of para-virtualization, OpenVZ and Xen. During the tests stages, I have tried a
large number of operations to explore the possibilities of OpenVZ, such as onlive migration, VPScloning, network configuration via ethernet-like and internet-like devices. The installation of Xen
has been particulary interesting, because it has been necessary to provide a solution to avoid that
an issue on this kernel delete the components installed one the OpenVZ one. I have then learnt
about partitionning the disks to provide isolation between the both kernels.
45
46
46
Chapter 4
C HAPTER 5
5.1
P2P PROTOCOLS AND P2P APPLICATIONS
5.1.1
Overview
The Peer-to-Peer (P2P) protocol is a communication protocol where every nodes is equal to
the other ones, instead of the classic Client/Server protocol. This protocol can be used instead
of the Client/Server protocol and realizes files transfers for instance, but it can also be used
to share multimedia stream, to realize allocated computing (Folding@home, SETI@home [23],
genome@home), or to provide Internet services (Skype) [33] [34].
The main purpose of the P2P protocol is to decentralize services available usually only on a server.
An improvement of the P2P protocol is the BitTorrent protocol: A files provider makes its files
available to the network and let others, called peers, connect and download these files, first to him,
and then, to each others. Indeed, each peer downloading a part of the data makes it available to
other peers and the parts are never provided in the same order.
This protocol allows users to receive large amount of data without putting the level of strain on
their computers that would be necessary for standard Internet hosting. The principe is to put the
complexity from the Servers located in datacenters to the Clients or Peers. It works as an alternative data distribution method of the traditionnal approach through the Client/Server paradigm. As
a result, even small computers with low brandwidth can take part of the data sharing.
This way of transfer can be both more economic and ecologic than the usual transfers with a
Client/Server architecture:
• More economic if the P2P protocol used take in account a border noation between ISP, because the numbers of requests to the Internet Service Provider is reduced: Once the content
is downloaded by the traditionnal approach, it can then be put on the disposal of others by
the P2P protocol.
47
48
Chapter 5
• More ecologic, because the energy needed for the working of a data-center (over-dimensionned
servers to prevent ponctual over-loads, numbers of servers in case of crashes of one of them,
isolation, climatisation) is spared.
5.1.2
P2P protocols
The P2P applications can be split according to the decentralization level of their architecture:
Centralized architecture The clients connect to a server which handles the content and peers
research. The informations themselves pass through clients directly and not through the server. A
central server is needed, which does not avoid the risks of attack (judicial cases, pirates) but no
content goes through it.
We can quote Napster [15], used to share musical files, or the protocol eDonkey2000 , used by
eMule, which follow this architecture.
Decentralized architecture There are a lot of different servers, which constitutes a stronger
network. But the informations research is more complicated and depends on the kind of architecture followed:
GNUtella: [7] decentralized and unstructured architecture:
The number of messages send to search a content is proportionnal to the number of peers connected, and exponential to the deep of the research;
GNUnet: [6] decentralized and structured architecture:
There are distributed hashtables containing the informations about the content in every servers,
which reduces the number of requests to find a content: the number of messages evolves like the
logarithm of the number of peers.
Hybrid architecture The servers are replaced by super-nodes: basically, super-nodes are network components chosen for their computing power or their bandwidth for the realization of functions such as indexation of informations.
KaZaA, for instance, follows this architecture.
Here is a table to sum up these different structures:
5.1.3
BitTorrent protocol
For the demonstration purpose, a centralized protocol has been retained. Let’s details the working
of this protocol:
A peer which wants to download a content connects to a HTTP or FTP server, and downloads
file corresponding to this content. This file contains the IP-address of the tracker, which is the
48
5.1 P2P protocols and P2P applications
Technology
Client-Server
Napster
DirectConnect
KaZaA
Ressources location
centralized
decentralized
decentralized
decentralized
49
Content searching
centralized
centralized
decentralized
decentralized
Peers searching
centralized
centralized
centralized
semi-centralized
Multi-source
no
no
no
yes
Table 5.1 Content delivery protocols
computer responsible for the coordination of the transfer, some metadata, and the structure of the
content to be downloaded. This content is split in parts, or pieces.
Once having downloaded the .torrent file, the peer will connect to the tracker. The tracker will
give him the IP-address of some other peers downloading the same content, and which parts are
available from which peer. Then, the peer connects to the others and downloads the pieces from
them. He will begin by the rarest parts first. A peer looking for parts is called a leecher, and a peer
providing parts is a seeder. Each piece downloaded is available for the others peers, so basically,
a peer is at the same time a seeder and a leecher. periodically, the peer connects to the tracker to
set up his list of content parts and tells him which new parts are available. When the download is
finished, the peer can let the others continue to download from him.
The Tracker The tracker coordinates the transfers; Periodically, the P2P client connects to the
tracker to update the list of pieces he can upload and from which peer he can download the pieces
still missing. The P2P protocol can alsol be set without the intervention of the tracker: Each peer
acts also like a tracker. The system is called a tracker-less system. This difference has not been
taken in account.
Content of a .torrent file The torrent file has a announce section with the URL of the tracker
and the name of the file to be shared, the number of pieces constituting this file, their lengths. The
file to be shared is regarded as a number of identically sized pieces (typically with a size between
64kB and 4MB, the smaller size lets the more efficient transfers). There is a checksum for each
pieces. Pieces’ name and associated checksums are located in the .torrent file.
Optimisation of the protocol The main idea is to increase the opportunity to exchange data,
which is only possible of peers owns different pieces. This validates the random or rarest-first
approaches.
A possible issue is the peers who are only leeching and never downloading. A solution is to give to
a peer as much content as this peer is giving back (tit-for-tat scheme) to improve the reciprocity in
the exchange. The problem is that a new-comer, having never seeded yet, can not take part of the
transfer. Or, neither of the peers takes the initiative. A solution is to send some pieces to unknown
peers to let them taking part of the transfer. This solution provides the peer with the opportunity
to discover new and even better peers.
Limitations of the P2P protocol
• There is a lack of anonymity, because the IP-address of the participants is available.
49
50
Chapter 5
Figure 5.1 Peer-To-Peer Protocol
• It is impossible to prevent peers to leech only. More important, some P2P applications have
been created to download only and not upload.
• As this protocol is decentralized, because there is actually no server storing the data, it is
easy to share copyrighted content.
5.1.4
Choice of a P2P application
The requirement for the P2P application were the following:
• The P2P application has to be free and open-source, taking in account that the source-code
has to be edited to implement the innovations of the researchers.
50
5.1 P2P protocols and P2P applications
51
• It should be light-weight and the working can be made remotely by command line. Indeed,
this application will be installed in a OpenVZ-VPS.
• It should be Linux compatible, provinding NAT, and being written in C/C++ or python.
Four bitTorrent clients which satisfied these requirements have been hold:
BitTorrent clients
BitTyrant
http://bittyrant.cs.washington.edu
Deluge
http://deluge-torrent.org
Transmission
http://transmissionbt.com
kTorrent
http://ktorrent.org
Advantages &Drawbacks
Written in JAVA (we prefer C/C++)
Too many useless features
The one retained
Too many useless features
Table 5.2 List of BitTorrent clients respecting the requirements
Finally, Transmission, which is the P2P application installed by default on Ubuntu, has been chosen. Its performances are very high compared to the others [31], it is well documented, with a
wiki and an active forum in its official website [26]. The structure can be very light, as versions
are available:
• transmissioncli light and used by command line only.
• transmission-daemon more options and can also be used by command-line only.
• transmission-remote used to control transmission-daemon
• transmission transmission version with GUI (standard version)
There is even a way to control it from a web-browser, which avoids to install the transmission-GUI
to control transmission-daemon.
The installation has been done compiling the sources. Indeed, we want to modify them afterwards.
The installation in a VPS has been done with the same way without the graphic interface.
Create a .torrent file from the source-file transmissioncli -n source-file -a tracker-URL -c comment -r torrent-file-name
Seed/Leech a content from the .torrent file transmissioncli -w /folder/where/the/torrent/file/is
torrentfile.torrent If the source file and the torrent file are in the same folder, transmission will seed
it.
51
52
5.1.5
Chapter 5
Choice of the tracker application
I have decided to install a LAMP (Linux Apache MySQL Phpmyadmin) platform because basically a tracker is a web-server providing some additionnal features. This platform has been
installed and configured following tutorials from the Internet [5].
The configuration-file of Apache2 has been modified to let it working inside a VPS. Have a look
to the OpenVZ tutorial for further informations.
The working of apache then is quite simple: Apache serves the file contained in the /var/www
folder and can be started/stopped/restarted bu these commands:
/etc/init.d/apache2 start|stop|restart
Once the Apache server working, and PHP plugin enabled, the installation of the tracker can be
realized.
I have decided to install rivettracker because:
• It is a very user-friendly tracker
• It provides a web-interface displaying statistics, which can be usefull to control the working
of the P2P-applications
• It has been written in PHP language, which I am quite familiar with
• It is working like a standard web-server
The installation is quite straightforward; All you have to do is to extract the compressed file downloaded from the rivettracker website [22] in the /var/www directory and then restart Apache.
Some configurations have to be made, most of them through the rivettracker web-interface. For
instance, some databases have to be created to store the .torrent files. Once the torrent files created
by the P2P-applications, the user has to upload them to the tracker. This can be done via the
web-browser. Then, the content-transfer can start.
There is a configuration file located at /var/www/rivettracker/ named config.php. This configuration file is quite important, because it is used to retain some caracteristics which apply during the
content transfer, such as:
• report_interval time between two peer connections to the tracker
• maxpeers maximal number of peers connected to each others (the tracker select in its peers
database maxpeers peers maximum to connect to a new-comer peer)
• website_url this is the URL of the tracker, in this case, its IP-address. As the tracker is
installed in a machine with a dynamic configuration for the internet access, this value has to
be modified each time a new IP-address is offered by the DHCP server of NICTA’s network.
52
5.2 Media-Centre Applications
5.2
53
MEDIA-CENTRE APPLICATIONS
Once the P2P-applications installed, to provide the demonstration, a Media-Centre has to be chosen, installed and configurated inside a VPS. I have firstly read some documentations in order to
find an appropriate application, according to the team requirements. This has led me to the choice
of XBMC. Then, I have installed XBMC, first in the HN to provide a demonstration in the Cebit
meeting, with some extra-features, like a remote control, and a new skin, and then I have tried to
provide the installation in a VPS. This stage was not possible, due to the actual structure of most
of virtualization tools.
To finish with the configuration of XBMC, I have chosen some content and created lists to display
them on its graphic interface.
During a demonstration, a trouble with the configuration of some features has led me to try an
other Media-Centre application, ELISA.
5.2.1
Requirements of the Media-Centre application and choice of XBMC
In order to chose a suitable Media-Centre application, I have listed a lot of them [30] and compared
them with these criteria:
• This application has to be free and totally open-source
• The installation on Linux must be provided
• The programming language must be C/C++ python or eventully Java
• GUI: The user interface must be very elaborated, for the demonstration purpose
• It has to provide the possibility of using a remote control device, again for the demonstration
purpose
• It has to support HTTP or FTP protocol. Indeed, we will use a HTTP or FTP server to
provide contents to the Media-Centre.
Three Media-Centre applications have retain our attention:
• ELISA [14]: A lot of features, such as the support of the FTP/HTTP server protocol, have to
be created as python plugins. In fact, the whole project is build from plugins, this modularity
is said to improve the user configuration ability. Moreover, the community is not as active
as the XBMC one, and the documentation is rather poor.
• BOXEE [2]: The whole Graphic User Interface (GUI) is non-open-source.
• XBMC [35]: We have chosen this one, because the most of the requested features are
already provided. Furthermore, the wiki and the forum, very active and helpful, has been an
asset during the development. There are a lot of documentations web-pages available in the
forum as well, with instructions to set up a new skin, or a remote control device.
53
54
5.2.2
Chapter 5
Installation and configuration
The installation can be made from the source code available from a SVN-server.
Once subversion installed, the code-source can be downloaded from the
https://xbmc.svn.sourceforge.net/svnroot/xbmc/branches/linuxport/XBMC web-site.
Then, the source-code has to be compiled. I have used gcc, installed from the build-essential
package. Different additionnal packqges have to be installed before compiling the source-code.
XBMC provides tutorials to explain the process [9] and [10].
5.2.3
Providing a remote control
During the demonstration, a remote control device has to be installed. This remote control is a
LIRC (Linux Infrared Remote Control) [12]. Basically, LIRC is a package used to decode and
send infra-red signals from a lot of communly used remote control devices. Some tutorials had to
be found to provide the installation on Debian Lenny [8]. (It is working on Lenny as well).
5.2.4
Creation of a list of content for the graphic interface
To provide the demonstration, XBMC has been configured to provide a library with the contents
(mostly movies) and metadata such as title, duration, actors, genre, comments, and so on.
The databases ruling this configuration is located at $HOME_XBMC/userdata/database and has a
name such as MyVideos??.db where ? is a number.
I have modified this database to provide metadata of the content we want to display. These modifications have been made through the Firefox interface (Firefox provides a plugin to manage sqlite
databases). There are three tables used to provide the metadata to the Media-Centre:
• movie
• files
• path
5.2.5
Skin and appearance
To provide a better Graphic-User-Interface for the Media-Centre, we have chosen to set up the
Media-Stream skin for XBMC.
This skin comes in the form of a .zip file downloaded from [13].
Unzip it in the skin folder located at /$HOME_XBMC/skin where $HOME_XBMC is the path of
the XBMC installation folder (usually ∼/.xbmc).
The set-up of this skin is then to be made from the XBMC GUI.
54
5.3 HTTP-proxy
5.3
55
HTTP-PROXY
For the demonstration, a HTTP-proxy has to be created from scratch. This server has to provide
these features:
• If the content is present on the served folder, then it acts basically as a web-server and
returns this content
• If the content is not present, instead of returning a 404 - file not found error, a bash-script
has to be executed to let the working of the P2P-applications in ordre to reach the content
from the network. Once the content is downloaded, the proxy returns it (back to the preview
case).
The source code of the HTTP-proxy has been written in Python, using the standard library.
The HTTP-proxy source-code and the bash script are provided in the appendix.
By this way, the end-user has to tip the very title of the content he wants to reach. This way was
appropriate during the tests, which have been made through a web-browser, tiping a wrong URL
to let the working of the P2P applications, but for the demonstration purpose, a list of the content
available in the network has to be showed to the user to avoid him to tip the title of each content
he would like to display.
Thus, we have modified the behavior of the proxy: the content available in the network is represented by a empty file in the shared folder. This file appears as the others one in the XBMC list.
Then, the bash-script is executed if an empty-file is requested.
5.4
ISSUES ECOUNTERED
If the user tries to get content ever available on the HTTP-proxy, then everything is fine.
But when it takes too long for the proxy to reach the content from the P2P network, the timeout is
reached and XBMC stops waiting. Then, the connection is closed and the user is not aware that
something unusual has happened. Some workarounds have been tried to fix this issue:
• A configuration file has been created for overwritten the default XBMC timeout, following the instructions of the XBMC wiki [37]. Basically, this file has to be stored in the
∼/.xbmc/userdata directory. The log file ∼/.xbmc/temps/xbmc.log shows that this file has
been taken in account, but this does not solve the problem nor change actually the timeout.
• The kernel timeout has also been modified, changing the value of the /proc/sys/net/ipv4/tcp_keepalive
files, but without success.
• Modifying the proxy timeout instead of the XBMC timeout has been tried, but this solution
has not worked.
The timeout problem could be resolved by adding a new feature to the proxy: Periodically, it
sends a response to the XBMC-client to avoid the timeout to be reached. This is even better than
55
56
Chapter 5
modifying the timeout, because even a longer timeout can be reached if the content is very huge
or is difficult to reach. This solution has not been tried because of a lack of time.
Anyway, the content research through the P2P network works perfectly, and if this query is quick
enough, the download from the proxy is normally realized. Otherwise, the content will be located
in the shared folder and the user can try a second time to reach it.
5.5
CONCLUSION
In this stage, a P2P application and a tracker have been chosen, installed and configured to provide
the first content transfers.
These applications are crucial, because we will have to modified them to validate the innovations
of the NICTA’s researchers of the TEMPO project. Furthermore, they are the keystone of the
demonstration I have to set up. I have tried to chose the most efficient P2P client, transmission,
which is open-source, and written in C. The large amount of options available with transmission
(transmissioncli, transmission-daemon, transmission-remote) let its usage in a lot of different use
cases.
This stage has taught me a lot about the P2P protocol, and its main use-cases, for content delivery.
Then, the configuration of the Media-Centre, although less depending for the demonstration, has
been an occasion to work on a open source project, providing a lot of documentation, and to
provide new features, as the integration with a HTTP-proxy. I have had thus to learn some aspects
of the python programming language (OOP, http-serving, for instance). The XBMC GUI could
be modified to add some more features, like pop-ups to inform the user when the content is fully
available.
56
C HAPTER 6
6.1
FIRST DEMONSTRATION
Now the virtualization solution configured, and the P2P application installed on VPS, it is possible to provide a first demonstration showing the global working of a content download via the
P2P protocol. We want to check if the global configuration (Virtualization tool, communication
between the different applications through the Virtual Private Network) is accurate. We will use
the nodes of the OMF testbed to provide this demonstration. The demonstration provided follows
the steps described in the figure 1.2.
6.1.1
Structure of the demonstration
In the OpenVZ-VPS are running BitTorrent applications (transmission). Each of these applications
have to connect periodically to the tracker inside the virtual private network of the VPS. The
tracker is installed in a VPS. It has to be accesible both from the Internet and from the local VPS.
A VPS contains a HTTP-server containing some content to be directly downloaded. An other VPS
contains a HTTP-server with every .torrent files.
The demonstration has been made following two steps:
• The user tries to reach some content, and this content is ever available. So, he just downloads
it from the HTTP-Server.
• The user tries to reach some content, but this content is not available. The HTTP-server
returns a 404-error. Instead of this error, we want the BitTorrent applications run and get
this content. Once the content is downloaded, it is served by the HTTP-server and provided
to the user.
The first step was to create a simple HTTP-server. This server puts the content on-line. If a nonavailable content is requested, it will download the .torrent file from the torrent-server and executes
57
58
Chapter 6
P2P applications to get this content. This HTTP-server has been written in Python, and a class
heriting from the Python library SimpleHTTPServer has been created to modify the behavior of
the HTTP-server when a missing file is requested: executing a shell script instead of returning the
404 error. This shell script will first download the appropriate .torrent file and then start the P2Pclient to reach the content. It will finally stop the P2P-client once the content is fully available
and provide it to the HTTP-server which returns it to the Media-Centre. The Media-Centre is
modelised by a web-browser for the first demonstrations.
6.1.2
Network configuration
Here is the architecture used to provide a first demonstration using the YB (Yellow-Boxes) of the
OMF-testbed.
Figure 6.1 Architecture of the first demonstration
Two different kinds of connection have to be performed:
• The connection between the leeching-peer and the tracker. This connection is made via the
HTTP protocol, which means that the peer initiates the connection and then the packets from
the tracker go to the peer. The peer has a private address (192.168.10.101). To perform this
connection, the IP formwarding has to be set up.
This has been made by these commands:
echo 1 /proc/sys/net/ipv4/ip_forwarding
iptables -t nat -A POSTROUTING -s 192.168.10.0/24 -o eth0 -j MASQUERADE
58
6.1 First demonstration
59
Then, the connection can be established between the VPS and the tracker. This one gives
to the VPS the IP-address of the connected peers and their port of connection (51413 by
default). Then, the leeching-peer tries to connect to the seeding-peers. The same process
is used to establish the connection. But this time, the connection is not kept any longer.
The seeding-peers send to the leeching-peer an other port to connect and tranfer the content.
Then, the connection between the peers is closed.
• The leeching-peer has got the information about the new port of connection. He opens it
(LISTEN status) and waits for incoming peers. The seeding-peers tries to connect to the
leeching one at this port. The difference here is this port has never been open from inside
the private network and the connection can not be established because the ip forwarding is
not applicable here. The content is not transfered even if there is a connection between each
peer and the tracker, and between each peer together to share the new port of connection.
The use of a packets tracer catcher (wireshark) has been usefull to understand this. The
solution is to proceed by NAT (Network Address Translation): each packet incoming in the
public interface with the default connection port of the seeding-peer has to be send to the
leeching peer to the required port.
This has been made by this command:
iptables -t nat -A POSTROUTING -p tcp -i eth0 -d $ip –dport 51413
where $ip is the IP-address of the public interface of the gateway obtained by the command:
‘ip=ifconfig eth0 | grep inet | grep -v inet6 | cut -f2 -d: | awk ! {print $1 } ! ‘
6.1.3
Results
The download of the content works perfectly. However, some issues have been encountered:
• The download only works once, and then, it is not possible to download the same content.
This is due to the P2P applications, keeping statistics from the download. This feature is
provided in order to be able to restart a download at the point it has been stopped in case of
interruption (lost of network connectivity, computer powered down). A configuration file is
created at each new file seeded/leeched:
∼/.config/transmission-cli/stats.json.
If this file does not exist, a new one is created, otherwise, this one is taken in account. As
following, when a file is got, then it will be seeded, even if the user delete it. A solution is
to modify the bash-script to delete this configuration file before starting up the P2P applications.
• XBMC can lost the connection during the transfert if this one takes too much time. This
problem has been presented in the preview chapter, as an issue encountered.
59
60
6.2
Chapter 6
SECOND DEMONSTRATION
This second demonstration has been set up to check the content transfer from a subnetwork. Actually, it has not been necessary to provide it for the demonstration of the STB prototype, but it has
been interesting to set it up to understand miscellaneous networking aspects.
6.2.1
Structure of the demonstration
The tracker and th P2P application which is seeding the content are located in the same subnetwork, with private IP-addresses, the P2P application trying to get it is located outside, with a public
IP-address. The OpenVZ HN stands for the gateway and has both a public and a private interface.
6.2.2
Network configuration
Here is the structure of this demonstration. Only a computer running with OpenVZ and a second
one are needed.
Figure 6.2 Architecture of the second demonstration
The architecture is a little bit different, the port-forwarding has been used to provide access to the
tracker and the P2P application inside the private network.
6.3
OMF
OMF (cOntrol andManagement Framework) is a project aiming at developping and deploying a
unified framework to control, measure, and manage federations of networking testbeds. The final
purpose is to increase the scientific rigor in the networking fields and facilitating future Internet
researches.
60
6.3 OMF
61
Indeed, the simulation provides usually an affordable way to predict network behaviors such as
performances or scalability, but due to design and computational complexities, it only uses simplified models, which does not capture the behavior of real environments.
Emulation is an alternative, and uses prototypes and entities trying to reproduce the real-world
behavior. These entities fail to capture the complexity of a real-environment.
Only experimentations in real testbeds can provide evaluation close to the reality. The solution is
then to conceive and use testbeds, which only can let the study of high-scalability experiences.
The complexity of these testbeds and their costs forbides their use for most of developers. Moreover, a testbed is generally conceived and developped in the frame of a project, and is not used
once this project reaches to the end.
Some solutions can be found by the use of testbeds such as PlanetLab [20] and Orbit [19], which
deploy nodes around the world. These testbeds can be reuse for further experiences. OMF appears
like an improvement of these both technologies and provide a universal protocol to control and
manage the experiences. OMF provides then tools:
• For the user, to describe and execute the experience, and obtain its results
• For the administrator, to manage the ressources of the testbed (reboot the nodes, install new
OS-images or applications)
The purpose of the integration of the experience into OMF testbed is to get quantitative results.
The integration of the different applications with ruby scripts to respect the OMF standards and
protocols has been realized, but some issues have been encountered during the execution of the
applications. Moreover, the integration of the STB and of the virtualization tool has not been done
mostly because of a lack of time. This could be realized as an evolution of this demonstration.
To conclue with, the demonstration expected by NICTA have been provided and now NICTA has a
prototype which can be used both to show for the different custumers and parteners the efficiency
of this architecture, and to provide experiments to test and control the efficiency of the new P2P
algorithms created inside NICTA. The integration of the prototype in the OMF framework has not
been provided and is the next step to obtain from this demonstration quantitatives results.
61
62
62
Chapter 6
C HAPTER 7
Conclusion
7.1
CONCLUSION OF THIS INTERNSHIP
During this internship, I have worked in the TEMPO project. This has teached me a lot on new
P2P and BitTorrent technologies and their applications to multimedia delivery services. I have
then discovered a new application of BitTorrent services. The different goals of this project, the
energy sparing, the delivery of multimedia services via P2P protocol, have been very interesting
and challenging as well.
I have also worked on a lot of different fields including Linux administration, virtualization, network configuration. Previewsly a total beginner at Linux Operating-System, I now know different
ways of installation, partitionning, compilation, and network configuration. The configuration of
virtalization tools has teached me a lot about this technology and its more recent applications,
mostly for web-hosting but also to put the complexity of the Internet from the Data-Center to
the end-user computer, and has been a way to get a better understanding of the Linux OperatingSystem and of network configuration. The use of virtualization tools as well have been decisive
for the obtention of my first job in the INRIA-Nancy on the project Grid 5000.
The different P2P protocols, and more precisely the BitTorrent protocol, have been communication
protocols totaly new for me. I have learnt about their protocols, and their implementation. I have
set up P2P networks and by the use of packer tracker like wireshark, I have understood in details
the working of these applications.
The configuration of Media-Center has to some points been challenging, having to deal with new
features totally new like the integration of a remote-control device. It has teached me a lot about
how to find relevant informations in the Internet to provide these features.
63
64
7.2
Conclusion
Chapter 7
CONCLUSION OF THIS PROJECT
The main purpose of this project is to provide the TEMPO-team with a prototype of Set-TopBox to be used during the next experiences and to prove and measure the efficiency of the new
P2P algorithms of NICTA. This prototype can now be used by the NICTA team to test the new
algorithms or for the custumers to show them the design of the STB, which can deal with heavy
graphic applications such as a Media-Centre. This part have been correctly realized, even if the
Media-Center have been installed in the physical machine instead of a virtualized one.
The Media-Center, Open source, has been configured to comply with the NICTA requirements,
and to make it as user-friendly as possible, with a new skin, playlists, remote control device.
The BitTorrent application installed on a Virtual Machine automatically reaches missing content
on the user-request. The OMF testbed’s nodes constitutes a P2P network providing the missing
content.
The integration within the OMF framework has not been finished, mostly because of a lack of time,
and because of the difficulty of the integration of the virtualization tool. This could provide the
NICTA TEMPO’s team with quantitative results of the different experiments. The OMF testbed
although has been used to provide the different demonstrations of content delivery.
Every applications used to design the prototype are open-source, letting the opportunity for the
NICTA TEMPO’s team to modify them as their wish.
7.3
POSSIBLE EVOLUTIONS OF MY ACHIEVEMENTS
The first evolution is to finish the integration within the OMF framework. Then, the results of the
different experiences will be accessible. But an other possible evolution is to use the bitTorrent
protocol to provide Video On Demand (VoD). Currently, the end-user has to wait for the content
to be fully reached before watching it. Because the BitTorrent protocol download the rarest pieces
first, it is not always possible to watch the content before every pieces are downloaded. But it
is possible to modify the BitTorrent client side and download during a first period the pieces the
end-user will watch first. Then, he can start to watch the content while the bitTorrent client gets
the following pieces with the current model rarest first. If the download is fast enough, then the
application will returns progressively to the current protocol downloading the rarest pieces first
more often. In this way, the BitTorrent protocol is more respected and its efficiency is kept. Else,
the application will tries to get the following pieces more often, in order to let the end-user watch
the content. This situation does not keep the efficiency of the BitTorrent protocol as if the rarest
pieces would have been downloaded first, so when the confidence interval should not be too large,
to respect the BitTorrent protocol, and not to short, to let the end-user watch the content.
Every steps I have followed to design the prototype and to realize the demonstrations have been
recorded and explained in the NICTA’s wiki, so that nothing will be lost when the TEMPO team
will use the prototype. This wiki provides as well a bibliography with every websites and sources
used during this internship, to help finding informations more quickly and efficiency if needed. I
have saved every scripts, network configuration, and codes in this wiki to let the team working on
this project set up an other prototype easily when it will be needed. I have written this documen64
7.3 Possible evolutions of my achievements
65
Figure 7.1 Application of the BitTorrent protocol to Video on Demand services
tation as precisely as possible so that nothing will be lost in the future about this internship. If
anything seems to be incomplete or confusing, let me know and I will do my best to provide the
missing informations.
65
66
Conclusion
66
Chapter 7
Appendix A
Iptables
This is the bash script used to create the routing rules The start option is used during the demonstrations, the startBis option if only the IP Forwarding is necessary.
#!/bin/sh -e
### BEGIN NetworkInsideContainer INFO
# Provides: IP-Forwarding/NAT
# Required-Start: Netwoking
# Required-stop:
# Default start: 2
# Default stop:
# Short-description: Enable the network inside
# the virtual network/Ip_forwarding
### END NetworkInsideContainer INFO
PATH="/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin"
ip=‘ifconfig eth0|grep inet|grep -v inet6|cut -f2 -d:|awk ’{print $1}’‘
echo "IP address: $ip"
case "$1" in
start)
echo 1 >/proc/sys/net/ipv4/ip_forward
echo "Forwarding Enabled"
iptables -t nat -A POSTROUTING -s 192.168.10.0/24 -o eth0 -j
MASQUERADE
echo "NAT Enabled"
# For the HTTPServer.
iptables -t nat -A PREROUTING -p
-j DNAT
iptables -t nat -A PREROUTING -p
-j DNAT
67
tcp -i eth0 -d $ip --dport 80
--to 192.168.10.101:80
tcp -i venet0 -d $ip --dport 80
--to 192.168.10.101:80
68
Appendix A
echo "Port Forwarding for httpServer CTID 101 (port 80) Enabled"
# For the P2P connection on the port and 51413
iptables -t nat -A PREROUTING -p tcp -i venet0 -d $ip --dport 51413
-j DNAT --to 192.168.10.102:51413
echo "Port Forwarding for transmission CTID 102 (port 51413) Enabled"
# For the torrentServer
iptables -t nat -A PREROUTING -p tcp
-j DNAT --to
iptables -t nat -A PREROUTING -p tcp
-j DNAT --to
echo "Port Forwarding for httpServer
-i eth0 -d $ip --dport 8080
192.168.10.103:80
-i venet0 -d $ip --dport 8080
192.168.10.103:80
CTID 103 (port 8080) Enabled"
# For the tracker
iptables -t nat -A PREROUTING -p tcp -i eth0 -d $ip --dport 8181
-j DNAT --to 192.168.10.104:80
iptables -t nat -A PREROUTING -p tcp -i venet0 -d $ip --dport 8181
-j DNAT --to 192.168.10.104:80
echo "Port Forwarding for tracker CTID 104 (port 8181) Enabled"
iptables -t nat -A POSTROUTING -p tcp --dport 80 -o venet0
-s 192.168.10.102 -j SNAT --to $ip
iptables -t nat -A POSTROUTING -p tcp --dport 80 -o venet0
-s 192.168.10.101 -j SNAT --to $ip
echo "Port Forwarding for tracker from the local network (venet0)
Enabled"
;;
startBis)
echo 1 >/proc/sys/net/ipv4/ip_forward
echo "Forwarding enabled"
iptables -t nat -A POSTROUTING -s 192.168.10/24 -o eth0
-j MASQUERADE
echo "Nat Enabled"
;;
stop)
echo "Deactivating routing rules and IP forwarding"
echo 0 > /proc/sys/net/ipv4/ip_forward
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
;;
restart)
$0 stop
$0 start
68
Appendix A
69
;;
*)
echo "Usage: $0 {start|stop|restart}"
exit 1
;;
esac
exit 0
HTTP-Proxy
Proxy code source
#!/usr/bin/env python
import os
from BaseHTTPServer import HTTPServer
from MySimpleHTTPServer import MySimpleHTTPRequestHandler
# The path is the one of the directory served
os.chdir(os.path.expanduser("/home/httpServer/content"))
# IP-Address of the server and the port listened
httpd=HTTPServer((’192.168.10.101’, 80), MySimpleHTTPRequestHandler)
httpd.serve_forever()
Script bash
Here is the script executed when an empty file is requested:
#!/bin/sh
##torrent downloader
PATH="/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin"
# torrent file associated to the file requested
CONTENTDIR=/home/httpServer/content
echo "STARTED"
extention=".torrent"
torrent="$1$extention"
echo $torrent
#The torrent file is downloaded from a http server
wget http://192.168.10.103/$torrent -O ${CONTENTDIR}/${torrent}
69
70
Appendix A
echo ".torrent file downloaded"
# Deleting old configuration files
if [ -e /root/.config/transmission-cli/settings.json ]
then
rm -R /root/.config
echo "Deteting the configuration files"
else
echo "No config files found"
fi
# Content leeched
# When the torrent finishes, the terminated script is executed
echo "P2P application transmission in progress ..."
cd ${CONTENTDIR}
echo ‘pwd‘
transmissioncli -w ./ ${torrent} -f ../../terminated
echo "torrentDownloader finished"
Basically, the terminated script killall transmission processes. This has been necessary otherwise
the torrentDownloader.sh script never finishes (transmissioncli still continues seeding once the
whole content is reached).
70
List of Figures
1.1
1.2
1.3
Structure du prototype créé . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Diagrqmme de séquence suivi lors des démonstrations . . . . . . . . . . . . . .
Application du protocole BitTorrent à la Vidéo à la Demande . . . . . . . . . . .
5
6
15
2.1
Map of the different NICTA’s laboratories . . . . . . . . . . . . . . . . . . . . .
20
3.1
3.2
3.3
3.4
3.5
3.6
Evolution of the Internet and examples of new services available . . . . . .
Loss of electrical power during its distribution . . . . . . . . . . . . . . . .
Current Set-Top-Box architecture compared to the NanoDatacentre approach. . . .
Proposed Nada Architecture . . . . . . . . . . . . . . . . . . . . . . . . .
Structure of the STB-prototype created . . . . . . . . . . . . . . . . . . . .
Sequence diagram of the different steps followed during the demonstration .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
24
25
26
27
28
29
4.1
4.2
4.3
4.4
4.5
4.6
4.7
4.8
4.9
Architecture without virtualization . . . . . . .
Full-Virtualization architecture . . . . . . . . .
Virtual-Machines Virtualization Structure . . .
User-Space Kernel Virtualization Structure . . .
Para-Virtualization architecture . . . . . . . . .
Hypervisor Virtualization Structure . . . . . . .
Container Virtualization Structure . . . . . . .
Effect of CPU load by another Virtual Machine
Partitions structure . . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
33
34
35
35
36
37
38
43
44
5.1
Peer-To-Peer Protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
50
6.1
6.2
Architecture of the first demonstration . . . . . . . . . . . . . . . . . . . . . . .
Architecture of the second demonstration . . . . . . . . . . . . . . . . . . . . .
58
60
7.1
Application of the BitTorrent protocol to Video on Demand services . . . . . . .
65
71
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
72
List of Figures
72
List of Tables
4.1
Comparison of virtualization implementations . . . . . . . . . . . . . . . . . . .
38
5.1
5.2
Content delivery protocols . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
List of BitTorrent clients respecting the requirements . . . . . . . . . . . . . . .
49
51
73
74
List of Tables
74
Bibliography
[1] Beginlinux website. http://beginlinux.com/blog/tag/openvz.
[2] Boxee official website. http://www.boxee.tv.
[3] Co-linux official website. http://www.colinux.org/.
[4] Comparison openvz and xen virtualisation solutions
by thomson. http://www.thlab.net/∼asoule/publications/RR_CR_PRL_2009_03_0001.pdf.
[5] Documentation about the lamp platform on ubuntu french official website. http://doc.ubuntufr.org/lamp.
[6] Gnunet official website. http://gnunet.org.
[7] Gnutella forum. www.gnutellaforums.com.
[8] Installation of lirc on debian-etch. http://www.mythtv.org/wiki/LIRC_on_Debian_Etch.
[9] Installation of xbmc media-centre on debian. http://xbmc.org/forum/showthread.php?t=42582&highlight=inst
[10] Installation of xbmc media-centre on debian, compiling from the source code.
http://xbmc.org/wiki/?title=HOW-TO_compile_XBMC_for_Linux_from_source_code.
[11] Linux vserver official website. http://linux-vserver.org/Welcome_to_Linux-VServer.org.
[12] Lirc (linux infrared remote control) official website. http://www.lirc.org.
[13] Media stream skin on xbmc media centre. http://www.teamrazorfish.co.uk/mediastream.html.
[14] Moovida (former elisa) official website. http://www.moovida.com.
[15] Napster official website. http://free.napster.com/.
[16] Omf, control, measurement and resource management framework
for heterogeneous,mobile & wireless testbeds. http://omf.mytestbed.net.
[17] Opentracker official website. http://erdgeist.org/arts/software/opentracker/.
[18] Openvz official website. http://wiki.openvz.org/FAQ#How_scalable_is_OpenVZ.3F.
[19] Orbit official website. http://orbit-lab.org/.
[20] Planetlab official website. http://www.planet-lab.org/.
75
76
List of Tables
[21] Qemu official website. http://www.nongnu.org/qemu/.
[22] Rivettracker official website. http://www.rivetcode.com/software/rivettracker.
[23] Seti@home project official website. http://setiathome.ssl.berkeley.edu/.
[24] Skype protocol.
[25] Testbed management framework and its applications to peer-to-peer overlays.
http://www.nicta.com.au/research/projects/tempo.
[26] Transmission official website. http://www.transmissionbt.com.
[27] User-mode-linux official website. http://user-mode-linux.sourceforge.net/.
[28] Vmware official website. https://www.vmware.com.
[29] Wikipedia en full virtualization. http://en.wikipedia.org/wiki/Full_virtualization.
[30] Wikipedia en, list of media-centres applications. http://en.wikipedia.org/wiki/Comparison_of_media_players.
[31] Wikipedia en, list of p2p clients. http://en.wikipedia.org/wiki/BitTorrent_client.
[32] Wikipedia en para virtualization. http://en.wikipedia.org/wiki/Paravirtualization.
[33] Wikipedia en, protocol bittorrent. http://en.wikipedia.ord/wiki/BitTorrent_(protocol).
[34] Wikipedia fr, p2p file transfer. http://fr.wikipedia.org/wiki/Partage_de_fichiers_en_pair_à_pair.
[35] Xbmc official website. http://xbmc.org.
[36] Xen official website. http://www.xen.org.
[37] 19
July
2009.
Xbmc
configuration
file
http://xbmc.org/wiki/index.php?title=Advancedsettings.xml.
advancedsettings.xml.
[38] apcmedia. Determining total cost of ownership for data centers and network rooms infrastructure. http://www.apcmedia.com/salestools/CMRP-5T9PQG_R3_EN.pdf.
[39] Australian Ministerial Council on Energy. Standby Product Profile - Free-to-Air Digital Set
Top Boxes. Technical report, 2004.
[40] Nicolas Bonnet. tutorial. http://systeme.developpez.com/tutoriels/virtualisation/livre-blancnicolas-bonnet.
[41] eukhost. Use cases openvz. http://blog.eukhost.com/webhosting/openvz.
[42] FRDATACENTER.
Evaluation
energie
http://nauges.typepad.com/my_weblog/2008/04/web-20-on-the-cloud-maiso%C3%B9.html.
data-center.
[43] FROGDATACENTER.
Ordre de grandeurs de consommations des datacenters.
http://enterprise.amd.com/Downloads/svrpwrusecompletefinal.pdf.
76
List of Tables
77
[44] H. Yu, D. Zheng, B. Zhao and W. Zheng. Understanding User Behavior in Large-Scale
Video-on-Demand Systems. In Proc. of EuroSys’06, 2006.
[45] James Hamilton.
Talk paper about data centers power consumption.
http://mvdirona.com/jrh/TalksAndPapers/JamesHamilton_USENIX2009.pdf.
2009.
[46] M. Cha, P. Rodriguez, J. Crowcroft, S. Moon, and X. Amatriain. Watching Television Over
an IP Network. In Proc. of ACM SIGCOMM/USENIX IMC’08, 2008.
[47] NANODATACENTERS. Nano data center project. http://www.nanodatacenters.eu.
[48] NICTA.
Energy-efficientservice
delivery
nicta
http://nicta.com.au/research/research_themes/networked_systems/tempo/tempo-e.
pages.
[49] NICTA.
National information and communications technology of australia.
http://www.nicta.com.au.
[50] OpenVZ
official
website.
http://wiki.openvz.org/Use_cases.
Use
cases
openvz,
official
website.
[51] K.W. Roth and K. McKenney. Energy consumption by consumer electronics in U.S. residences. Final Report to the Consumer Electronics Association (CEA), TIAX Reference
D5525, January 2007.
[52] Skype. http://www.skype.com/.
[53] Max Ott Thierry Rakotoarivelo, Guillaume Jourjon. Models for an energy-efficient p2p
delivery service. Technical Report NICTA number ATP-2293. August 2008.
[54] Falko
Timme.
Installation
of
openvz
howtoforge
http://www.howtoforge.com/installing-and-unsing-openvz-on-debian-lenny-amd64.
[55] Falko
Timme.
Xen
installation
and
configuration
http://www.howtoforge.com/perfect_xen_setup_debian_ubuntu.
way.
tutorial.
[56] V. Valancius, N. Laoutaris, L. Massoulie, C. Diot and P. Rodriguez. Greening the Internet
with Nano Data Centers. Technical Report CR-PRL-2009-02-0003, Thomson, 2009.
[57] VGRID.
fr and en:
Laboratoire
http://www.lri.fr/∼quetier/vgrid/vgrid.
de
recherche
en
informatique.
[58] Vijay Erramilli, Nikolaos Laoutaris. System Design and Decomposition. Technical Report
D11-D31, NaDa, 2009.
[59] vmware. Main kinds of virtualization. http://www.vmware.com/files/pdf/VMware_paravirtualization.pdf.
[60] Wikipedia. Skype users. http://en.wikipedia.org/wiki/Skype.
77

Design and development of a demonstration prototype for energy

Transcription

Documents pareils

Volume 1, numéro 6

PRETERIT OU PRESENT PERFECT ?

Lire le discours - Ambassade de France au Canada

science.

SERVICE D`ÉDUCATION PERMANENTE ET À DISTANCE

Sujet, corrigé et rapport d`Anglais 2014

london 2012 olympics - Franco

Hi Dorothy - Oaklawn Farm Zoo

traduction française : David Weir

Pierre‐Henri Gouyon

Virtualisation avec openVZ