p. 1 Lecture: Remote Sensing Sommersemester - Burckhardt

Transcription

Lecture: Remote Sensing
Sommersemester
Prof. Dr. Christoph Kleinn
Institut für Waldinventur und Waldwachstum
Arbeitsbereich Fernerkundung und Waldinventur
Sources of error in remotely sensed data
Source: Lunetta, Congalton,
Fenstermaker, Jensen,
McGwire and Tinney 1981.
Lecture Introduction to Remote Sensing
Institut für Waldinventur und Waldwachstum, Georg-August-Universität Göttingen
Accuracy Assessment
Slide No 1
Even in high resultion imagery it is not always clear where
to define the boundary between forest and non-forest: It is
a question of a clear definition.
Four delineation options for forest boundary from aerial photographs:
Accuracy Assessment
Source:
Slide
No 2 Kenneweg. 2002.
Quantifying classification accuracy
Simple approach:
- We select a sample of points in the image which we
visit in the field (or interpret in another reference map that is
supposed to be giving the “true” land cover):
Ground control points.
- Then we compare for each point the true classes with our
classification result in the image.
- The result can be presented in a so-called confusion matrix,
which allows further analysis and quantification.
Mod. 1
Accuracy Assessment
Slide No 3
p. 1
Sommersemester
Accuracy assessment
Three basic components of
an accuracy assessment are
(after Stehman and
Czaplewski 1999):
1.
2.
3.
The sampling design
to select the reference
sample.
The design how to measure truth on the sampling units
selected.
The analysis procedure to apply once the data have been
collected.
Accuracy Assessment
Slide No 4
Guiding criteria for design
desicions for accuracy
assessment (from Stehman 1999)
1.
2.
3.
4.
5.
6.
Probability sampling
protocols.
Simple to implement
and analyse.
Result in low variability for estimates requiring the highest
accuracy levels.
Allow for reliable variance estimation.
Result in a well-distributed sample.
Cost effective.
Accuracy Assessment
Slide No 5
Quantifying
classification
accuracy
Per class:
Mod. 1
Accuracy Assessment
Slide No 6
Source: Congalton and Green 1999.
p. 2
Sommersemester
“Improving“ the accuracy measures
- The more classes there are,
the more are the confusion
possibilities.
- Separating only forest and
non-forest will usually yield
much higher accuracy
values than the distinction
of 10 further forest classes!
- If we include large areas of easily identifyable features (the
sea, for example), the overall accuracy is also considerably
improved (because there are more observations in the diagonal).
→ These features must be taken into account when evaluating
the accuracy assessment.
Accuracy Assessment
Slide No 7
Calculating statistics from an error matrix
In statistics this type of matrix
is called “contingency table”,
giving the joint frequency
distribution of two variables,
and allowing for the test of
numerous statistical
hypothesis.
For accuracy assessment
the kappa statistic is
frequently calculated, characterizing the degree of matching between
reference data set and classication. In fact, it is a statistic that compares two matrices: here, it describes the difference between the
agreement found in the actual matrix and the chance agreement given
the same marginal frequencies:
κˆ =
p0 − pc
1 − pc
Accuracy Assessment
Slide No 8
The Kappa statistic:
A measure of agreement
κˆ =
p0 − pc
1 − pc
∑
pii =
with p0 =
sum of relative frequency
in the diagonal of the
actual error matrix, and
∑
pc =
pi + p+ j = relative frequency of a random allocation
of observations to the cells = chance agreement.
The notation “i+” and “+j” is for the relative marginal frequencies.
Weights wij may be assigned to each po and pc terms, if the
accuracy of some classes is more important than for others.
However, weights would be subjective!
Mod. 1
Accuracy Assessment
Slide No 9
p. 3
Sommersemester
The Kappa statistic:
Interpretation
κˆ =
p0 − pc
1 − pc
κˆ = 0
indicates that
obtained agreement equals
chance agreement.
κˆ > 0 indicates that obtained agreement is greater than
chance agreement.
κˆ < 0 indicates that obtained agreement is smaller than
chance agreement.
κˆ = 1
is perfect agreement (theoretical situation for p0=1).
For kappa, a variance, and statistical significance can be given.
Accuracy Assessment
Slide No 10
Example
calculation
κˆ =
p0 − pc
1 − pc
p0 =
45 + 91 + 55 + 55
= 0.7321
336
pc =
(85*55) (110*110) (75*72) (69*96)
+
+
+
336
336
336
336 = 0.2551
336
κˆ =
0.7321 − 0.2551
= 0.64
1 − 0.2551
69
Source: Congalton and Green 1999.
Accuracy Assessment
Slide No 11
How to select “ground truth” data
This is a sampling exercise!
If we want to do further analysis with the ground truth data, i.e.
making inferences that go beyond a mere description of what we
found in those pixels only, then: principles of statistical sampling
must be employed.
In particular: the calculation of kappa and its variance must take
place with the estimators that correspond to the sampling design
applied! It has to be probability sampling – and not subjective
sampling.
- Stratification is frequently sensible, according to, for example
- cover classes, or
- assumed classification reliability.
- Would it be a good idea to use the training sites (that had been
observed for classification, anyway) also for accuracy
assessment?
Mod. 1
Accuracy Assessment
Slide No 12
p. 4

p. 1 Lecture: Remote Sensing Sommersemester - Burckhardt

Transcription

Documents pareils

changing family issues - Hu

Glass pipes for sale - Hookah Online Store

Die Children`s improvement organization setzt sich für Waisen und

seminar invitation - Lehrstuhl für Systemdynamik und Prozessführung

LC-310 - MS-Tech

Vom Praktiker - SCOPE ONLINE

Software design engineer (Ingenieur/in