Le Routage Robuste Réactif Une combinaison de techniques d

Transcription

Le Routage Robuste Réactif
Une combinaison de techniques d’ingénierie de trafic proactives et
réactives pour traiter le trafic dynamique de réseau
Pedro Casas et Sandrine Vaton
Séminaire des Doctorants de TELECOM Bretagne
Brest, France, 27-28 mars 2008
TELECOM Bretagne
Département Informatique
Universidad de la República
Facultad de Ingenierı́a
Uruguay
Pedro CASAS
Outline
1
Introduction to the problem
2
A proactive approach: the Robust Routing
3
A reactive approach: Anomaly Detection/Localization
4
A combined approach: the Reactive Robust Routing
5
Conclusions and Perspectives
Pedro CASAS
Outline
1
2
3
4
5
Pedro CASAS
Routing Optimization (RO) in current network scenario
Current network scenario:
network convergence is a tangible reality
heterogeneous services make network traffic uncertain and highly
variable
new kinds of network anomalies increase this traffic uncertainty
Pedro CASAS
variable
Routing performance under all possible network situations:
expected traffic variations: routing optimization for expected traffic
unexpected traffic variations (Anomalies): minimize impact on
other QoS services between anomalies’ detection and resolution
Pedro CASAS
variable
Routing performance under all possible network situations:
expected traffic variations: routing optimization for expected traffic
unexpected traffic variations (Anomalies): minimize impact on
other QoS services between anomalies’ detection and resolution
Major challenge: ...how to optimize routing for an unknown traffic
demand?
Pedro CASAS
RO in current scenario: a challenging task
Sources of Demands
Variation
Unexpected
Events
Daily Periodic
Usage Patterns
Equipment
Failures
Network
Attacks
External
Routing
Changes
Flash
Crowds
Spontaneous
Services
(P2P)
Pedro CASAS
Sources of Demands
Variation
Unexpected
Events
Daily Periodic
Usage Patterns
Equipment
Failures
Network
Attacks
External
Routing
Changes
Flash
Crowds
Spontaneous
Services
(P2P)
Pedro CASAS
Sources of Demands
Variation
Unexpected
Events
Daily Periodic
Usage Patterns
Equipment
Failures
Network
Attacks
External
Routing
Changes
Flash
Crowds
Spontaneous
Services
(P2P)
Pedro CASAS
Sources of Demands
Variation
Unexpected
Events
Daily Periodic
Usage Patterns
Equipment
Failures
Network
Attacks
External
Routing
Changes
Flash
Crowds
Spontaneous
Services
(P2P)
Pedro CASAS
Sources of Demands
Variation
Unexpected
Events
Daily Periodic
Usage Patterns
Equipment
Failures
Network
Attacks
External
Routing
Changes
Flash
Crowds
Spontaneous
Services
(P2P)
Pedro CASAS
Sources of Demands
Variation
Unexpected
Events
Daily Periodic
Usage Patterns
Equipment
Failures
Network
Attacks
External
Routing
Changes
Flash
Crowds
Spontaneous
Services
(P2P)
Pedro CASAS
Sources of Demands
Variation
Unexpected
Events
Daily Periodic
Usage Patterns
Equipment
Failures
Network
Attacks
External
Routing
Changes
Flash
Crowds
Spontaneous
Services
(P2P)
Pedro CASAS
Sources of Demands
Variation
Unexpected
Events
Daily Periodic
Usage Patterns
Equipment
Failures
Network
Attacks
External
Routing
Changes
Flash
Crowds
Spontaneous
Services
(P2P)
Pedro CASAS
Anomalies
Sources of Demands
Variation
Unexpected
Events
Daily Periodic
Usage Patterns
Equipment
Failures
Network
Attacks
External
Routing
Changes
Flash
Crowds
Spontaneous
Services
(P2P)
Pedro CASAS
Anomalies
Additional source of uncertainty
traffic demands are rarely available:
direct traffic measurements (e.g. CISCO Netflow) seldom available
for every ingress/egress links.
direct traffic measurements causes router overloading.
measurements are generally conducted at a higher level of
aggregation, rendering the traffic process non-observable.
Pedro CASAS
Examples of Variations in Real Data
10000
Link 47
Link 58
Link 104
Link 126
Correlated volume changes
9000
Link Load (unknown unit)
8000
Unidentifiable variations
7000
6000
5000
4000
3000
2000
1000
0
0
1000
2000
3000
4000
5000
6000
Time (min)
7000
8000
(a) Traffic patterns in a large Tier-2 network.
Pedro CASAS
9000
10000
How to tackle the problem?
Proactive approach: Robust Routing Techniques
consider traffic uncertainty within the routing optimization.
Sources of Traffic
Variation
Expected variations
and small load changes
under normal operation
Daily Traffic Patterns
Unexpected
Events
Emerging Challenges
Multi Hour Robust Routing
Network Attacks
(DoS/DDoS)
Reactive Robust Routing
Pedro CASAS
External Routing
Modifications
Flash Crowd
Events
Equipment
Failures
Spontaneous Overlay
Services
(P2P)
Anomaly Detection
and Localization
Reactive approach: Anomaly Detection and Localization
anomaly detection and localization from simple measurements.
parsimonious traffic modeling to overcome the non-observability
problem.
Sources of Traffic
Variation
Expected variations
Unexpected
Events
Emerging Challenges
Network Attacks
(DoS/DDoS)
Pedro CASAS
External Routing
Modifications
Flash Crowd
Events
Equipment
Failures
Spontaneous Overlay
Services
(P2P)
Anomaly Detection
and Localization
Combined approach: Reactive Robust Routing
robust routing for usual network operation.
anomaly detection/localization for the unexpected events.
robust routing reconfiguration to minimize network congestion.
anomalies’ end detection (automation of the routing process).
Sources of Traffic
Variation
Expected variations
Unexpected
Events
Emerging Challenges
Network Attacks
(DoS/DDoS)
Pedro CASAS
External Routing
Modifications
Flash Crowd
Events
Equipment
Failures
Spontaneous Overlay
Services
(P2P)
Anomaly Detection
and Localization
Outline
1
2
3
4
5
Pedro CASAS
A Proactive Approach
The Boy Scout motto:
Pedro CASAS
A Proactive Approach
The Boy Scout motto:
Be Prepared!!!
Stable Robust Routing
robust Traffic Engineering techniques.
consider traffic uncertainty in advance (robustness).
Pedro CASAS
The Stable Robust Routing (SRR)
Problem formulation
Consider the following network scenario:
Network topology:
n nodes.
L = {1, . . . , r } links with capacities in C = (c1 , c2 , . . . , cr ).
N = {OD1 , .., ODm=n(n−1) } Origin-Destination traffic flows.
Routing matrix R = {rl,k ;l=1..r ,k =1..m }, 0 6 rl,k 6 1.
P(k ) = {set of paths p for ODk }, k = 1..m.
Traffic OD flows d = {di,j;i,j=1..n}; d = {dk , k =1..m }
Links traffic (aggregated ODs traffic) y = {yl, l=1..r }
y(t) = R × d(t) ∀t.
Pedro CASAS
The Stable Robust Routing
Multipath Routing Optimization
Given d, C, R and P(k), RO seeks to optimally balance d in P(k) to
minimize some performance criterion:
Pedro CASAS
umax (C, d, R) = max
l∈{1...r }
Pedro CASAS
X rl,k · dk
y
= max l
cl
l∈{1...r } cl
k
l∈{1...r }
X rl,k · dk
y
= max l
cl
l∈{1...r } cl
k
xpk , 0 6 xpk 6 1, fraction of
dk in p ∈ P(k )
xlk , 0 6 xlk 6 1, fraction of
dk in l ∈ p
Pedro CASAS
l∈{1...r }
xpk , 0 6 xpk 6 1, fraction of
dk in p ∈ P(k )
X rl,k · dk
y
= max l
cl
l∈{1...r } cl
k
minimize
umax
subject to:
P
xpk
>1
∀k ∈N
6 xlk
∀ (l, k ) ∈ (L, N)
p∈P(k )
xlk , 0 6 xlk
dk in l ∈ p
6 1, fraction of
xpk
P
p∈P(k ), l∈p
xlk .dk
6 umax · cl
xpk , xlk
>0
umax
61
P
k ∈N
Pedro CASAS
∀l∈L
∀ (l, k ) ∈ (L, N), ∀ p ∈ P(k )
Traffic uncertainty set and routing optimization
Traffic d is uncertain → belongs to a polyhedral uncertainty set D:
D =
minimize
subject to:
P k
xp
p∈P(k )
P
xpk
p∈P(k ), l∈p
P k
xl .dk
d ∈ Rm , R × d 6 ymax , d > 0
umax
>1
∀k ∈N
6 xlk
∀ k ∈ N, ∀ l ∈ L
6 umax · cl
∀ l ∈ L, ∀ d ∈ D
k ∈N
xpk , xlk
umax
>0
61
∀ l ∈ L, ∀ p ∈ P(k ), ∀ k ∈ N
Solved by a column and constraints generation method [BAK-05].
[BAK-05] W. Ben-Ameur and H. Kerivin, “Routing of Uncertain Traffic Demands”, Optimization and Engineering, 2005.
Pedro CASAS
Trade off in the size of the uncertainty set D
d ∈ Rm , R × d 6 y4:00−18:00
, d>0
max
=
d ∈ Rm , R × d 6 y18:00−4:00
, d>0
max
=
DA
DB
0.8
Maximum Link Utilization
0.7
Historical Routing
Robust Routing B
Robust Routing A
0.6
0.5
0.4
0.3
0.2
0.1
0
5:00 7:00 9:00 11:00 13:00 15:00 17:00 19:00 21:00 23:00 1:00 3:00
Time (hours)
Remark: a single stable routing scheme for long-time periods results in
sub-optimal performance.
Pedro CASAS
The Multi-Hour Robust Routing (MHRR)
D1
Dt
D2
β1
00:00
12:00
time
β2
24:00
time
β3
Idea: divide the uncertainty set to reduce cost (adapt the set) and consider a
SRR configuration for each sub-set.
Pedro CASAS
D1
Dt
D2
β1
00:00
12:00
time
β2
24:00
time
β3
Partitioning hyperplane α.d = β.
Pedro CASAS
D1
Dt
D2
β1
00:00
12:00
time
β2
24:00
time
β3
The optimal division is generally NP-complex.
Pedro CASAS
D1
Dt
D2
β1
00:00
12:00
time
β2
24:00
time
β3
The optimal division is generally NP-complex.
However, when the direction (α) is known, it can be approximately
solved [BA-07]: in our case, we take the time direction [CV-07].
[BA-07] W. Ben-Ameur, “Between Fully Dynamic Routing and Robust Stable Routing”, DRCN, 2007.
[CV-07] P. Casas and S. Vaton, “An Adaptive Multi-Temporal Approach for Robust Routing”, EuroFGI WIPQoS&TC, 2007.
Pedro CASAS
Some examples in Abilene
0.8
Historical Routing
Stable Robust Routing A
Stable Robust Routing B
M−H Robust Routing
Historical Routing
Stable Robust Routing A
Stable Robust Routing B
M−H Robust Routing
0.7
0.6
0.4
0.45
0.35
0.3
0.5
0.4
0.3
0.2
0.25
0.1
0.2
21:00
23:00 1:00
3:00
5:00
7:00 9:00
Time (hours)
11:00 13:00 17:00 19:00 21:00
(a) Expected daily behaviour
0
5:00
7:00
9:00 11:00 13:00 15:00 17:00 19:00 21:00 23:00 1:00
Time (hours)
(b) Anomalous unexpected event
Observation: the MHRR outperforms the SRR, but depends on a
rough knowledge of the daily uncertainty set (we use it to improve
routing for the expected traffic behavior in a robust fashion).
Pedro CASAS
3:00
Outline
1
2
3
4
5
Pedro CASAS
Anomaly Detection/Localization
The Problem
Objectives: detect and localize an abrupt change in traffic
demand d(t) from link load measurements y(t) = R × d(t).
d(t) is seldom available.
Conversely, link load measurements are highly spread.
Problem: d(t) is non-observable directly from y(t).
ill-posed problem r << m.
Pedro CASAS
Proposed methodology [FNS-07]
Stochastic parsimonious linear model for anomaly-free traffic
demand d(t)
The anomaly-free is considered as a nuisance parameter.
Hypothesis testing to detect/localize an anomaly in the traffic
residuals (after removing the modeled traffic)
[FNS-07] L. Fillatre, I. Nikiforov and S. Vaton, “Détection Localisation Séquentielle d’Anomalies Volumiques dans un Réseau”,
GRETSI, 2007.
Pedro CASAS
Stochastic Traffic model
The order of increasing anomaly-free OD flows remains constant
during long-time periods.
The values of the ordered OD flows can be decomposed over a
spline-basis with a small number of components.
d(t)
λk (t)
1200
1000
800
600
400
200
k
0
2000
140
120
1500
tim 1000
et
(mi 500
n)
100
80
60
40
0
20
0
Small flows
Large flows
Medium-size flows
Approximation of real OD flows by the spline-based model
Pedro CASAS
Model for the anomaly-free traffic:
d(t) = Sµ(t) + ξ(t)
ξ(t) is a white Gaussian noise with covariance matrix Σ(t).
S = (s1 s2 . . . sq ), is a splines basis that describes the traffic
spatial distribution.
µ(t) = (µ1 (t) . . . µq (t))T ∈ Rq , with q << m.
the coefficients µk (t) describe the anomaly-free intensity
variations.
Pedro CASAS
Anomaly-free model for y(t):
y(t) = Hµ(t) + ζ(t)
H = RS ∈ Rr ×q , small no columns → easy to retrieve µ(t) from
y(t).
ζ(t)’s covariance is estimated from samples.
the anomaly-free traffic is eliminated by projecting y(t) into the null
space of H.
Pedro CASAS
Detection/localization of volume anomaly at time t0 as a multiple
hypothesis testing:
H0 = {the OD flows are anomaly-free}
S
Hj = t0 =1..∞ Htj0 , with Htj0 = {the j-th OD flow presents an
anomalous traffic from an unknown change time t0 }
compute (T , ν), T is the alarm time at which a ν-type change
(ν ∈ {1, 2, . . . , m}) is detected and localized.
Pedro CASAS
Detection/localization is conducted on the residuals obtained
from link load observations, after filtering the anomaly-free traffic.
Optimal recursive algorithm with well-established optimality
properties in terms of detection delay and false alarm rate
[IN-00].
Optimal trade-off between the worst case of the average detection
delay (E(T )), the false alarm rate and the false localization
probabilities.
Multiple recursive detection and decision functions.
[IN-00] I. Nikiforov, “A Simple Recursive Algorithm for Diagnosis of Abrupt Changes in Random Signals”, IEEE Trans. on IT, 2000.
Pedro CASAS
50
15
45
10
40
5
Alarm on OD flow 87
Level of alarm
0
35
−5
30
st (i)
gt (i, 0)
Another example in Abilene
Anomaly begins
25
−10
−15
20
−20
15
−25
10
−30
5
0
−35
1020
1030
1040
1050
1060
1070
Time t (min)
(a) Recursive detection functions
−40
1020
1030
1040
1050
1060
1070
Time t (min)
(b) Decision functions
Figure: Typical realizations of decision functions in Abilene.
Pedro CASAS
Outline
1
2
3
4
5
Pedro CASAS
The Reactive Robust Routing
Combination of the reactive and the proactive approaches
[CFS-08].
MHRR to handle usual-traffic and small changes in traffic
demands in a robust and efficient way.
Anomaly Detection/Localization algorithm to deal with unexpected
events.
Exploits the localization ability to compute an adapted SRR after
the detection, based on an expansion of the uncertainty set
[PLS-08].
Detects the end of the anomaly (if applicable) and takes back the
MHRR configuration.
[CFV-08] P. Casas, L. Fillatre and S. Vaton, “Multi-Hour Robust Routing and Fast Load Change Detection for Traffic Engineering”,
IEEE ICC, 2008.
[PLS-08] P. Casas, L. Fillatre and S. Vaton, “Robust and Reactive Traffic Engineering for Dynamic Demands”, EuroNGI, 2008.
Pedro CASAS
Routing reconfiguration
yimax + θR i,k
D : Before the anomaly
yhmax + θR h,k
0
D : After the anomaly
yjmax + θR j,k
before the anomaly:
Pedro CASAS
D = {R . d 6 ymax }
yimax + θR i,k
yhmax + θR h,k
0
yjmax + θR j,k
D = {R . d 6 ymax }
θ = θ.δ k
anomaly in OD flow k : d∗ = d + θ,
δ k = (δ1,k , .., δm,k )T , δi,k = Ii=k
before the anomaly:
Pedro CASAS
yimax + θR i,k
yhmax + θR h,k
0
yjmax + θR j,k
D = {R . d 6 ymax }
θ = θ.δ k
δ k = (δ1,k , .., δm,k )T , δi,k = Ii=k
before the anomaly:
0
after the anomaly: D = {R . d 6 ymax + R . θ}
Pedro CASAS
yimax + θR i,k
yhmax + θR h,k
0
yjmax + θR j,k
D = {R . d 6 ymax }
θ = θ.δ k
δ k = (δ1,k , .., δm,k )T , δi,k = Ii=k
before the anomaly:
0
after the anomaly: D = {R . d 6 ymax + R . θ}
0
robust routing reconfiguration, using D as the uncertainty set
(expansion of the uncertainty set).
Pedro CASAS
Anomalies’ end detection
The detection algorithm focuses in the abnormal OD flow k after
the routing reconfiguration.
Two simple hypotheses:
Ha : {the OD flow k is abnormal}
Hb : {the OD flow k presents a usual behavior}
simple Neyman-Pearson’s test (most powerful test for 2 simple
hypotheses):
∆(z(t)) = log
f0 (z(t))
− h > 0 −→ Hb
fk (z(t))
The decision threshold h is fixed according to the prescribed false
alarm probability.
Pedro CASAS
Reactive Robust Routing evaluation
Multi-Hour Robust Routing, adapted to the expected traffic
demand.
Anomaly detection/localization in OD flow k = 63 at time t = 1135.
Robust routing reconfiguration.
Detection of the anomaly’s end at time t = 1353.
Return to the MHRR configuration.
Pedro CASAS
80
0.8
∆(z(t)), k = 63
HR
0.7
40
∆(z(t))
60
k -th anomaly ends
20
0
SRR A
SRR B
MHRR
0.6
RRR
0.5
0.4
0.3
replacements
−20
1200
1250
1300
1350
Time t (min)
1400
(a) Neyman-Pearson Test ∆(z(t))
Pedro CASAS
0.2
1020
1070
1120
1170
1220
1270
1320
1370
Time t (min)
(b) Performance evaluation
1420
0.8
HR
0.7
SRR A
SRR B
MHRR
0.6
RRR
0.5
0.4
0.3
0.2
1020
1070
1120
1170
1220
1270
1320
1370
1420
Time t (min)
between 20% and 50% of utilization improvement w.r.t. the stable
robust routing approach
Pedro CASAS
Outline
1
2
3
4
5
Pedro CASAS
Conclusions of this work
Robust TE techniques can not deal by themselves with traffic
uncertainty, but they should be used in current traffic scenario.
We have shown that reactive and proactive techniques can work
together to improve the treatment of large volume anomalies.
Decision theory offers powerful and promising techniques for the
automation of diagnosis in the field of network anomaly detection
(still in the very early stage).
Pedro CASAS
Future directions
exploit the linear parsimonious traffic model for further studies
(d(t) becomes observable, many problems can be solved).
use this traffic model for the traffic matrix estimation problem,
tracking of traffic demands from simple link measurements, etc.
consider other performance criterion for the optimization problem.
Pedro CASAS
Thank You for Your Attention!!
Remarks & Questions?
Pedro CASAS

Le Routage Robuste Réactif Une combinaison de techniques d

Transcription

Documents pareils

4228 Holland Dr Des Moines, IA 50310 | Home for Sale | Real

Niveau : 2AM School Year : 2011-2012 English

mounto daballo – 2014

master de communication internationale - Univ

DISTRIBUTION OF YOUNG TREES The Ministère des Forêts, de la

Year 4 Scheme of Work

Levée de fonds du temps des Fêtes Holiday Fundraising Campaign

Formule 20I Formule des reignements financiers

January 18, 2007

course summary - Department of Civil and Environmental Engineering

Robust Estimation Techniques for GNSS Positioning

Stateless Geocasting

world gravity map - Le service Cartographie

Inter-Domain Routing Under Scrutiny: Routing Models and

Telekom Slovenia Chooses Netgem`s Next Generation Hybrid IPTV