Sparse Coding

1
DYNamiques de l'Information
2 PR, 4 MC, 1 postdoc, 2 IG, 2+2 doctorants fin 2015
11 PhD + 1 HDR soutenues sur 2010-2015
Axe : Signal → Apprentissage / Structuration → Information
Recherches théoriques : Optimisation convexe régularisée, Classification
semi/supervisée, Apprentissage génératif de données fonctionnelles,
Réseaux profonds (convolutionels), Modèles neuromimétiques, Sparse
Coding, Big-Data, Modélisation et manipulation de données
Valorisation : Fossé sémantique sur grande échelle, annotation SON,
image, vidéo, surveillance environnementale, RI Big Data, veille web.
2
Dyni = Interdisciplinary Research on Multimodal Signal
Methods for data structuration
• Graph Sparse Coding, Sparse Coding
• Bayesian Non Parametric clusterin
• Bayesian user model...
– Image classification
– Acoustic classification (nevus, web, handwritten, speech,
bioacoustics)
– Ontology / Data user Model
Interface entre les mathématiques appliquées (statistique) et
l'informatique avec un intérêt particulier pour le traitement du signal,
et l'apprentissage statistique
Depuis 2011 : 1) Segmentation de grandes données Temporelles ; 2)
Analyse de données fonctionnelle; 3) Régularisation Bayésienne ; 4)
Apprentissage Bayésien non-paramétrique; 5) Modélisation robuste
de données non-Gaussiennes et atypiques ; 6) Tracking de sources
concurrentes ; 7) Personnalisation RI
3
Projets principaux 2010-15
Projets ANR :
●
SPID (2015-16) : Surveillance acoustique passive, application à suivi de drone (67 K)
●
Cognilego CONTINT (2011-14) : Réseau profond pour la lecture (80 K) (PI)
●
ANCL BLANC (2009-12) : Modèle neuromimétiques de la lecture (100 K)
●
AVEIR MDCA (2008-11) : Annotation d'image du web (80 K)
Autres :
●
RAPID 'PHRASE' 2012-2015 en traitement image sous-marine (100 K)
●
FUI 'SYCIE' 2014-2017 en traitement donnés drone sous marin (100 K)
●
Ministère Environnement : Bioacoustique sur Parc National Port-Cros : VAMOS (2015-17), DECAN
(2015), DECAV (2013-15) (total 140 K)
- Animation nationale : 'Scaled Acoustic Biodiversity' sabiod.org, 5 UMR et coll. Internat,(180 K sur 201215), Mission Interdisciplinaire du CNRS ; Responsable de l'Action masse de données bioacoustiques du
GDR MADICS 2015-17.
- Animation internationale : organisation de plusieurs challenges et workshops à ICML13, 14, NIPS13,
ICDM15, LIFECLEF14,15,16...
- Pilotage interlabo sur UTLN de JASON “Joint Acoustic Survey” regroupant 3 UMRs sur UTLN (50 K)
- Projet indus. Sermicro (éq. IBM Espagne) (70 K)
- IUF (100 K)
-Thèses : CIFREs : COEXEL (110 K) en veille informationnelle ; Bio-id (100 K) dès fin 2015 ; 3 MRTs
Total : ~1 M euros en 5 ans
Results 2011-15
•
20 ACL (IEEE TIP, Pattern Recognition, IJPRAI, Applied acoustics, JASA, MTAP, Neurocomp.,
Meth. Ecology & Evolution, Science China IS,...)
40 ACTI A (ICPR, ICASSP, IJCNN...), 10% inter-équipe (IM, Escodi), 10% internat.
. 5 book chapters, 2 (+1) brevets (EU, USA)
•
. 6 Wkp Machine Learning (IEEE EADM15, ICML13, ICML14, NIPS13, IJCNN12)
. BDA 2015 (150 part.), Summer schools ERMITES (10 éd. 2007 : 10*30 part.)
. Coll. : Cornell univ NY, Victoria univ (Canada), Pavia univ., Heifi univ (CN)
: ENS Ulm DATA Team, INRIA Zenith, CIRAD, PNPC, MNH Paris...
•
•
•
LifeClef Lab creation with INRIA Zenith at Clef. 2013-15
Reviewers in PLOS one, IEEE TSI, Applied Acoustics, JASA...
Patent Licensing EU, USA / Price : Minist. Ecology 2014, 2015
2016...
. Axe information UTLN : JASON 10 (Pr + MC) Escodi & UMR IM2NP
. HPC Bioacoustics : Coll. ON Canada, Pavia univ., Cornell Univ.
. FUI LSIS GIPSA DCNS 2016... ?
. Project Eu H2020 MOBIDIC posé (Berlin, UK) ?
. Feder Alpes InterReg ?
5
9 UMR, 4 CNRS institutes (IN2SI, INSB, INEE, INSU) ;
Internat. Coll : New York , Cornell univ, Cibra...
Industry : Cyberio, Sermicro, Nortek, Dodotronics…
Parc National Port-Cros, Pelagos, réserves intégrales Italie, île de la Réunion, Vancouver...
Data.ENS GIPSA CPPM LIF Zenith LAMFA
SABIOD synopsis
Array sensors
7
SABIOD
Crowdsourcing
CNRS IN2SI & INSB & INEE (SABIOD)
CNRS (INSU) = 68 years Fe = 100 Hz from 2005 to 2015
0
1 recordings
1
1
2
0
1
2
Vancouver
BIRD
Madagascar
Réunion
Nvelle Calédonie
HPC system for Big Data
Detection Localisation of Marine Mammals
Natural sounds analyses needs
Scaled Learned Representation
 Feature learning for fast indexing & classifying
 Discovering new spatio temporal pattern for (un)known sources
 Scaling the methods
0
1
1
1
Infinite HMM for unsupervised segmentation
2
0
1
2
Units
1,
2,
,t
0
1
1
1
2
0
1
2
Megaptera Whale song
IHMM decomposition
THEMA
0
1
1
1
Sentence 1
p2 p3......
Sentence 2
p2
p1 p5 p2 ….
p5
2
0
1
2
Chamroukhi et al. 2014
Sparse Coding computation
Why Sparse Coding ?
●
More discriminatve
●
Beter generalizaton for new data
●
Reducton of the reconstructon error
●
Data compression
● Each data vector x is expressed as a c linear combinaton of a dictonary D of
i
i
size K (only one in usual K-means)
●
Formulaton :
●
introduces sparsity (regularizaton constraint : some contributon are non zero)
Iteratve learning of D and C untl convergence by LASSO and K-SVD algorithms
●
Complexity for projecton in ~O(K n nnz), n the number of vectors to project, nnz the
average non-zero coefcients
●
[ B. A. Olshausen and D. J. Field. Sparse coding with an overcomplete basis set: A strategy employed by V1? Vision Research, 37:3311–3325,1997 ]
1
4
Classification efficiency of SC vs state of the art
R
[ Razik Paris and Glotin, ICPRAM 2011 ]

Real data :broadcastnews

2h duration

At equivalentsize SC outperformsGMM

Overtrainingeffect for GMM

Still increasing recognition results for SC
Humpback whale song analysis by sparse coding :
exploring song components
• Humpback songs are structured
– Most decomposition algorithms use prior information
• Unsupervised determination of recurrent pattern in a data flow
– Usually clustering
– K-means clustering drawbacks:
• Each cluster may not cover all the space
• Each cluster not suit the data.
• This study:
– Validate the “subunit” component hypothesis
– Propose a method to automatically classify song species by the “subunit” components of the song
* Material : Songs recorded in Madagascar, Reunion & New Caledonia, Tonga, Hawaï...
48 kHz FS, from 2008 to 2014 (no 2010 neither 2011)
13 MFCCs, 10 ms frame shift, 32 ms frame length
N windows are concatenated to desired T scale (250 ms
Long term WHALE SONG EVOLUTION shown by Sparse Coding
Year evolution from A to B = log( P( xy, A ) / P ( xy, B) )
D = 1Byear
- A = 1 year
D = 4 years
yY
Y
X
X
Y
D = 5 years
D = 6 years
x
Our algorithm, by unsupervised dictionary learning of
a proto-lexicon of the song of the humpback whale
allows long term systemic units are efficiently extracted and
show the variation of the composition of the songs from one year to
another
=> WORLD SCALE BIOPOPULATION ANALYSIS
[ Glotin et al J. Acoust. Soc. Am. 133, 3311 (2013)
Doh 2014 Phd Thesis ]
Analysis and display at diferent tme scales to identfy acoustc structures of singing species
Methods in Passive Acoustic 3D tracking
Transitivity … to filter TOA
(2 patents, EU USA...) licensed
Time Delay of Arrival filtering by transitivity, and Dopler constraint :
I
T
Robust
to different species, chirp, voicing,...
EU, USA patent, Maturation (2013-14), licencied 2015...
Scaled Tracking by Sparse Coding
[Glotin, Razik et al in POMA ASA 2013]
B
A
Replace cross. corr
By cosine similarity measure :
Cos(A,B) = ( A .B ) / ( ||A|| . ||B||),
Higher the cosine is, the more the vectors are correlated,
FAST vectorial implementation / parallel processing :
allcosines( h1, h2 ) = ( H1 * H2’ ) / (norm(H1’) * norm(H2)) ,
where Hi is the matrix of the 1024 by 10 minutes frames,
* is the matrix product,
norm(Hi) is the L2 norm of each frame vector of Hi.
[Glotin, Razik et al in POMA ASA 2013]
Scaled Sparse Time Delay of Arrival estimations applied
to Voicing of Whales (Minke) on 10 sec. recordings
Direct CrossCorr(s1, s2)
Sparse code correlation :
Cosinus(SC(s1) ,SC(s2))
with dictionary learned on s1 U s2
[ Hervé GLOTIN - Joseph RAZIK - GIRAUDET Pascale - Sébastien PARIS - Frédéric BÉNARD Sparse coding for fast minke whale tracking with Hawaiian bottom
mounted hydrophones" , International Workshop on Detection, Classification, Localization & Density Estimation of Marine Mammals using Passive Acoustics,
Portland, USA, supported by ONR Dpt of the Navy & Acoustical Society of America (ASA) 2011
Time Delays Of Arrival Estmatons
We extract 14 TDOA over these 30 minutes,
between h1,h3,h4,h6
=> Coherent and regular variations
Large aperture hydrophone array : real time tracking system : demo. on real data:
[ Patent Glotin et al. Multiple whale tracking USA patent 2013
Glotin et al. Whale Cocktail Party, Canac Acoustics, 2008
Bénard Glotin, Neutrino whale tracking, Applied Acoustics 2011 ]
Online demo at http://sabiod.org/tv
RANGE [ 500 to 5000 m] prec :15m
Illustraton on large scale whale
monitoring : Marseille-Toulon-Nice
[ 2013 Abeille Phd, Glotin - Coll G. Pavan
2014 Doh Phd, Glotin - Coll Adam ]
Multimodal Platform
SABIOD BOMBYX Observatory – Univ. Toulon
http://glotin.univ-tln.fr/BOMBYX
South Port-Cros National Park – 2012...
Collaboration with UMR LSIS, IUF, PNPC, UMR MIO, PME Osean
Multimodal instrumentation : Hydrophones (stereo), ADCP, Video submarine camera
Objective : Inverse model by machine learning of audio, surface, and subsurface
sea-states, at short and long ranges.
Application transient analysis on Physeter (cachalot) recorded on BOMBYX 2014
Distance estimation using only one hydrophone[Doh et al 2014]
Physeter
Whale,
Hydro
Distance
Estim.
(meter)
2400m
12 lines
25 storeys / line
3 PMT / storey
40 km to
shore
450 m
Junction
box
60 m
Readout cables
Join us in the Large Scale Bird Id
Focus on Brazil area
Most populated country in XC dataset
1st country in terms of endangered species
14027 recordings from the top-501 species
minimally 15 recordings per species (max 91)
minimally 10 diferent recordists per species (max 42)
Metadata:
type of sound (call, song, alarm, etc)
day + tme + locaton + gps + elevaton
recordist & personal remarks, quality rate
Life Clef 2014
Large scale challenge BIRD SPECIES from Bresil
using Crowdsourcing and Crowdsolving : the largest species classifcaton system
# load
Giga
Oct