Web Usage Mining: Discovery and Applications of Usage Patterns

Transcription

Utilisation du Web Mining
en E-Commerce
Par Jianfeng NIE
Mars 29, 2006
Plan
Introduction
Taxonomie du Web Mining
Processus d’extraire des
connaissances dans Web Mining
Application du Web Mining en Ecommerce
Conclusion

29 Mars, 2006
Utilisation du Web Mining en E-commerce
2
Plan

Introduction
Définition du Web Mining
Problématique
Histoire
Processus d’extracter des connaissances dans
Web Mining
Application du Web Mining en E-commerce
Conclusion

29 Mars, 2006
3
Introduction (1/3)
Web Mining:
s’est développé à la fin des années 90. ce
domaine consiste à utiliser l’ensemble des
techniques du Data Mining afin de
développer des approches et des outils,
permettant d’extraire des informations
pertinentes à partir de données du web
(documents, traces d’interactions,
structure des pages, des liens……)
29 Mars, 2006
4
Introduction (2/3)
Problématique : Pourquoi Web Mining

Trouver l’information du web n’est plus facile.

Information est trop.
Structure du web est complexe.
Changement est fréquent

Data Mining vise souvent à traiter des données
déjà organisées dans une base de données.

E-Commerce is the killer of the informaion.

29 Mars, 2006
Stratégie, business intélligent
Sérvices personalisé
5
Introduction (3/3)
Histoire:

Terme créé par Etzioni en 1996
Taxonomie du Web Mining (Chang et al., 2001;
Chakrabarti, 2003)
29 Mars, 2006
6
Plan

Introduction

Web Content Mining
Web structure Mining
Web Usage Mining
Processus d’extraire des connaissances dans
Web Mining
Conclusion
29 Mars, 2006
7
Web
WebMining
Mining
Web
WebContent
Content
Mining
Mining
29 Mars, 2006
Web
Web
Structure
Structure
Mining
Mining
Web
WebUsage
Usage
Mining
Mining
8
Web Content Mining (3-1)
Web Content
Web
Content
Mining
Mining

Web Mining
Web Mining
Web
Web
Structure
Structure
Mining
Mining
Web Usage
Web
Usage
Mining
Mining
Recherche d’information (RI) (1/2)

Précision = documents pertinents retrouvés / documents
retrouvés

Rappel = documents pertinents retrouvés / documents
pertinents
29 Mars, 2006
9
Web Mining
Web Mining
Web Content Mining (3-1)
Web Content
Web
Content
Mining
Mining

Web
Web
Structure
Structure
Mining
Mining
Web Usage
Web
Usage
Mining
Mining
Classification du Contenu (2/2)
Yahoo home page
Education
College University ……
Business
Science
Companies Finance
News
Jobs
Exemple: Pour trouver le site web de l’U de M
Directory > Regional > Countries > Canada > Provinces and
Territories > Québec > Regions > Montréal > Cities >
Montréal > Education > College and University
29 Mars, 2006
10
Web Structure Mining (3-2)
Web Content
Web
Content
Mining
Mining
Web Mining
Web Mining
Web
Web
Structure
Structure
Mining
Mining
Web Usage
Web
Usage
Mining
Mining
Visualization du web (1/4)
Source: Grobelnik, Marko (2001) Web-Mining…searching for the knowledge on
the Internet…
29 Mars, 2006
11
Web Content
Web
Content
Mining
Mining
Web Mining
Web Mining
Web
Web
Structure
Structure
Mining
Mining
Web Usage
Web
Usage
Mining
Mining
Structure du Web (2/4)
Source:Linoff, Gordon S. et Berry, Michael J.A. (2001) Mining the Web: Transforming customer Data into Customer Value
29 Mars, 2006
12
Web Content
Web
Content
Mining
Mining
Web Mining
Web Mining
Web
Web
Structure
Structure
Mining
Mining
Web Usage
Web
Usage
Mining
Mining
Hubs et Autorités (3/4)
Hubs
29 Mars, 2006
Authorités
13
Web Mining
Web Mining
Web Content
Web
Content
Mining
Mining
Web
Web
Structure
Structure
Mining
Mining
Web Usage
Web
Usage
Mining
Mining
Algorithme de HITS (4/4)

Sommation
∑
a (u ) =
∑
a (u ) =
h (v )
u :( v , u )∈ E
a (v )
u :( v , u )∈ E
(v, u) est un lien de citation : la page v cite la page u

Normalisation
a (u ) =
a (u )
∑ [a ( q ) ]
2
et
a (u ) =
q
29 Mars, 2006
h (v)
∑ [h ( q ) ]2
q
14
Problème: Web Mining =RI ?
Non:
Il y a des différences
importantes:

29 Mars, 2006
Résultat et But
Tâche
Fonction
Etc.
15
Web Mining
Web Mining
Web Usage Mining (3-3)
Web Content
Web
Content
Mining
Mining
Web
Web
Structure
Structure
Mining
Mining
Web Usage
Web
Usage
Mining
Mining
Web Usage Mining
Analyse du comportement de l’utilisateur à
travers sa navigation et notamment
l’ensemble des clics effectués sur le site
(clickstream).
29 Mars, 2006
16
Web Mining
Web Mining
Web Usage Mining (3-3)
Web Content
Web
Content
Mining
Mining
Web
Web
Structure
Structure
Mining
Mining
Web Usage
Web
Usage
Mining
Mining
Serveur Log
Source: Sanjay Kumar Madria (2002) Web Mining: A Bird’s Eye View, University of Missouri-Rolla
29 Mars, 2006
17
Plan

Introduction

Préparation de données
OLAP
Analyse (Data Mining)
Conclusion

29 Mars, 2006
18
Source: Zaiane Osmar R. and Cai JiAn (2001), ‘Web Mining: From
Concepts to Practical Systems’, DASFAA’2001
29 Mars, 2006
19
connaissances dans Web Mining(3-1)
Préparation de données (1/2)

Capturer les données

Nettoyer les données

Transformer les données
29 Mars, 2006
20
Préparation de données (2/2)

Un exemple de la data transformé
Source:Zaiane Osmar R., Xin Man and Han Jiawei (1998)
29 Mars, 2006
21
OLAP (1/3)
Source: Zhang Zhongping (2001) Introduction of OLAP
29 Mars, 2006
22
Cube de données (2/3)
Salaire
B.C.
Province Prairies
Ontario
sum
0-20K20-40K 40-60K60K- sum
Agrigolture
Accounting
… ...
Industrie
sum
Source:Zaiane Osmar R. and Cai JiAn (2001), ‘Web Mining: From
Practical Systems’, DASFAA’2001
29 Mars, 2006
Concepts to
23
Cube de données (3/3)
Salaire
B.C.
Province Prairies
Ontario
sum
0-20K20-40K 40-60K60K- sum
Agrigolture
Accounting
… ...
Industrie
sum
29 Mars, 2006
24
Analyse (Data Mining)

Analyse de route

Association

Classification et prédiction

Clustering
29 Mars, 2006
25
Analyse (Data Mining)(1/4)

Analyse de route

29 Mars, 2006
80% clients visite le site par ‘compagnie/produit2 ’, mais
pas par la page d’accueil.
55% clients visite moins de 5 pages web avant sortir.
……
26
Analyse (Data Mining) (2/4)

Association
[N (ii ), right, N (it )]
I = {i1 , i 2 ,..., i n }
pour mesurer l’association entre deux items. ‘right’ est le poids
de transformer du nceud ‘i’ au nceud ‘t’. Si la valeur de ‘right’ >
0.5, on pense que les deux noeuds ont un lien fort. Ce lien
signifie qu’un client qui visite la page A va souvent visiter la
page B aussi.
29 Mars, 2006
27
Classification Bayesianne

Idea: assign to example X the class label C
such that P(C|X) is maximal
Computes the distribution of an input
associated with each class, for example,
given the variable X with a value at xi the
probability of it being in Class A is greater
than it being in Class B
Source: Auvil, Loreta et Searsmith Duane (2003)
29 Mars, 2006
Mathematically speaking — If one knows
how P(X | C), and the densities P(xi) and
P(cj) (prior probabilities) are known
then the classifier is one which assigns
class cj to datum xi if cj has the highest
posterior probability given the data.
28

Clustering
L=(ip, uid, url, time)
Ip, uid, url, time sont l’adresse du client,
l’identification du client, l’ URL qui est demandé par le client, le
temps de navigation du visiteur
M
29 Mars, 2006
m×n
6 4 4 44UID
74444
8
C 12 ... C 1 n ⎤ ⎫
⎡ C 11
⎥⎪
⎢C
C
C
...
21
22
2n ⎥⎪
⎢
=
⎬URL
⎥
⎢ ...
...
...
... ⎪
⎥⎪
⎢
C
C
C
...
m2
mn ⎦ ⎭
⎣ m1
29
Plan

Introduction
Processus d’extraire des connaissances
dans Web Mining
Système de recommandation
Conclusion

29 Mars, 2006
30
Application du Web Mining
Web Usage
Mining
Personalization
Site H elper
Letizia
Web Watcher
M obasher
Analog
Krishnapuram
System
Improvement
R exford
Schecter
Ag g arwal
WebSIFT
WU M
SpeedTracer
WebLog M iner
Shahabi
Site
M odification
Adaptive Sites
Business
Intellig ence
SurfAid
Buchner
Tuzhilin
U sag e
C haracterization
Pitkow
Arlitt
M anley
Almeida
Source: J. Srivastava, R. Cooley, M. Deshpande et PN Tan (2000) Web Usage Mining:
Discovery and Applications of Usage Patterns from Web Data, SIGKDD Explorations
29 Mars, 2006
31
Système de Recommandation
le magasin peut recommander le produit qui peut être
intéressant au client selon plusieurs critères :

29 Mars, 2006
la correspondance du produit au profil du client ;
la similarité du produit aux produits que le client a déjà
achetés auparavant ;
l’appréciation des autres clients similaires au client.
32
Système de Recommandation

Architecture
Source: Markellou, Mousourouli, Sirmakessis et Tsakalidis (2005)
29 Mars, 2006
33
Plan

Introduction
Processus d’extraire des connaissances
dans Web Mining
Conclusion
29 Mars, 2006
34
Conclusion

Il y a lien forte entre Web Mining et Ecommerce
Web Mining utilise des outils mûres, par
exemple, des méthodes de RI, Data Mining
etc.
Il y a des limitations du Web Mining

Dépendre très forte sur le ficher de log
Le bruite d’information
OLAP est complexe et cher
29 Mars, 2006
35
Références

Auvil, Loretta et Searsmith Duane (2003) Using Text Mining for Spam Filtering, University of
Illinois
Grobelnik, Marko (2001) Web-Mining…searching for the knowledge on the Internet… , Institut
Jožef Stefan http://www-ai.ijs.si/MarkoGrobelnik/
J. Srivastava, R. Cooley, M. Deshpande et PN Tan (2000) Web Usage Mining: Discovery and
Applications of Usage Patterns from Web Data, SIGKDD Explorations
Sanjay Kumar Madria (2002) Web Mining: A Bird’s Eye View, University of Missouri-Rolla
Vipin Kumar (2002) Army High Performance Computing Research Center,
University of Minnesota
Zhang Zhongping (2001), Introduction of OLAP
http://www.dmgroup.org.cn/pptdown050322/OLAPIntr.ppt
29 Mars, 2006
36
Référence

[1]Amazon.com (2004), Annual Report
[2]Abraham Ajith (2003), ‘Business Intelligence from Web Usage Mining’, Journal of Information & Knowledge Management, Vol. 2, No. 4, PP375-390
[3]Ansari Suhail, Kohavi Ron, Mason Llew and Zheng Zijian (2000), ‘Integrating E-commerce and Data Mining: Architecture and Challenges’, WEBKDD
‘2000 Workshop
[4]Baraglia Ranieri and Palmerini Paolo (2002), ‘Suggest: A Web Usage Mining System’, IEEE International Conference on Information Technology
(ITCC’02)
[5]Barnes & Noble.com (2004), Annual Report
[6]Buchner Alex G. and Mulvenna Maurice D. (1998), ‘Discovering Internet Marketing Intelligence through Online Analytical Web Usage Mining’,
SIGMOD Record, Vol. 27, No. 4
[7]Castellano Marcello, Pastore Nicola and Arcieri Francesco (2005), ‘A Flexible Mining Architecture for Providing New E-knowledge Services’, IEEE
38th Hawaii International Conference on System Sciences
[8]Chakrabarti Soumen (2000), ‘Data Mining for Hypertext: A Tutorial Survey’, ACM SIGKDD, Volume 1, Issue 2
[9]Chie Ed H., Rosien Adam and Heer Jeffrey (2000), ‘Lumber Jack: Intelligent Discovery and Analysis of Web User Traffic Composition’, Working Paper
[10]Cooley Robert, Mobasher Bamshad and Srivastava Jaideep (1999), ‘Data Preparation for Mining World Wide Web Browsing Patterns’, Working
Paper, University of Minnesota
[11]Ebay.com (2004), Annual Report
[12]Eirinaki, M. and Vazirgiannis, M. (2003), ‘Web Mining for Web Personalization’, ACM Transactions on Internet Technology, 2003, 3(1), PP1-27
[13]Gomory Stephen, Hoch Robert, Lee Juhnyoung, Podlaseck Mark and Schonberg Edith(1999), ‘E-commerce Intelligence: Measuring, Analyzing, and
Reporting on Merchandising Effectiveness of Online Stores’, Working Paper, IBM T. J. Watson Research Center
[14]Grcar Miha(2004), ‘User Profiling: Web Usage Mining’, SIKDD’2004 at Multiconference, IS 2004, PP12-15
[15]Hu Xiaohua and Cercone Nick (2002), ‘An OLAM Framework for Web Usage Mining and Business Intelligence Reporting’, IEEE Volume 2, PP 950 955
[16]Jespersen Soren E., Thorhauge Jesper and Pedersen Torben Bach (2002), ‘A Hybrid Approach to Web Usage Mining’, Technical Report 02-5002,
Aalborg University
[17]Ji Junzhong, Sha Zhiqiang, Liu Chunnian and Zhong Ning (2003), ‘Online Recommendation Based on Customer Shopping Model in E-commerce’,
IEEE/WIC, WI’03
[18]Joo Dongkwon and Moon Songchun (2001), ‘Scalable Web Mining Architecture for Backward Induction in Data Warehouse Environment’, IEEE
Catalogue No. 01CH37239
[19]Kawano Hiroyuki (2004), ‘Applications of Web Mining: From Web Search Engine to P2P Filtering’, IEEE 12th International conference on Informatics
Research for Development of Knowledge Society Infrastructure (ICKS’04)
[20]Kohavi Ron (2001), ‘Mining E-commerce Data: The Good, the Bad, and the Ugly’, 7th ACM SIGKDD Intl' Conf. on Knowledge Discovery and Data
Mining (KDD '01)
[21]Kohavi Ron, Provost Foster (2001), ‘Applications of Data Mining to Electronic Commerce’, Data Mining and Knowledge Discovery, 5, 5-10, Kluwer
Academic Publishers
29 Mars, 2006
37
Référence

[22]Kohavi Ron, Mason Llew, Parekh Rajesh and Zheng Zijian (2004), ‘Lessons and Challenges from Mining Retail E-commerce Data’, Machine
Learning Journal, Special Issue on Data Mining lessons learned
[23]Kosala Raymond and Blockeel Hendrik (2000), ‘Web Mining Research: A Survey’, SIGKDD Explorations, Volume 2, Issue 1
[24]Kwan Irene S. Y. (2005), ‘Towards Effective Web Resources Management: An Empirical Study and a Web Mining Approach’, IEEE the Fifth
International Conference on System Sciences
[25]Lee Chung-Hong and Yang Hsin-Chang (2001), ‘Developing an Adaptive Search Engine for E-commerce Using a Web Mining Approach’, IEEE
[26]Linoff Gordon S. and Berry Michael J.A. (2001), ‘Mining the Web: Transforming Customer Data into Customer Value’, John Wiley & Sons
[27]Liu Lizhen, Chen Junjie and Song Hantao (2001), ‘The Research of Web Mining’, IEEE 4th World Congress on Intelligent Control and Automation
[28]Liu Jian-guo, Huang Zheng-hong and Wu Wei-ping (2003), ‘Web Mining for Electronic Business Application’, IEEE
[29]Li Jia and Zaiane Osmar. R (2004), ‘Using Distinctive Information Channels for a Mission-based Web Recommender System’, 6th ACM SIGKDD
workshop on Webmining and Web Analysis (WebKDD 2004), PP 35-46
[30]Markellou Penelope, Mousourouli Ioanna, Sirmakessis Spiros and Tsakalidis Athanasios (2005), ‘Personalized E-commerce Recommendations’,
IEEE International Conference on E-business Engineering (ICEBE’05)
[31]Mena Jesus (1999), ‘Mining E-custor Behavior’, DB2 Magazine, Vol. 4, Issue 4, http://www.db2mag.com/db_area/archives/1999/q4/mena.shtml
[32]Mobasher Bamshad, Jain Namit, Han Eui-Hong and Srivastava Jaideep (1997), ‘Web Mining: Pattern Discovery from World Wide Web
Transactions’, Working paper, University of Minnesota
[33]Pal Sankar K, Talwar Varun and Mitra Pabitra (2002), ‘Web Mining in Soft Computing Framework: Relevance, State of the Art and Future
Directions’, IEEE Transactions on Neural Networds, Vol. 13, No. 5
[34]Perner Petra (2002), ‘Advances in Data Mining: Applications in E-commerce, Medicine, and Knowledge Management’, Springer, LNAI2394
[35]Punin John R., Krishnamoorthy Mukkai S. and Zaki Mohammed J. (2001), ‘Web Usage Mining – Languages and Algorithms’, In Studies in
Classification, Data Analysis and Knowledge Organization, Springer-Verlag, 2001. RPI Tech Report 01-3
[36]Roy Nivedita and Mphapaatra Tapas (2005), ‘Web Mining: A Key Enabler in E-business’, IEEE ICSSSM '05 International Conference
[37]Sarwar Badrul, Darypis George, Konstan Joseph and Riedl John (2000), ‘Analysis of Recommendation Algorithms for E-commerce’, Technical
Report, ACM, University of Minnesota
[38]Scime Anthony (2005), ‘Web Mining : Applications and Techniques’, Idea Group Publishing
[39]Stefani A., Vassiliadis B. and Xenos M. (2004), ‘Behavioral Patterns in Hypermedia Systems: A Short Study of E-commerce vs. E-learning
Practices’, EPEAEKII Iraklitos programme at HOU
[40]Wang Jicheng, Huang Yuan, Wu Gangshan and Zhang Fuyan (1999), ‘Web Mining: Knowledge Discovery on the Web’, IEEE, Volume. 2, PP 137 141 vol.2
[41]Web-datamining.net, ‘FAQ, web mining’
http://www.web-datamining.net/forum/faq_wm.asp
[42]Yun Chin-Huang and Chen Ming-Syan (2000), ‘Mining Web Transaction Patterns in an Electronic Commerce Environment’, Pacific-Asia Conference
on Knowledge Discovery and Data Mining
29 Mars, 2006
38
Référence

[43]Zaiane Osmar R., Xin Man and Han Jiawei (1998), ‘Discovering Web Access Patterns and Trends by Applying OLAP and Data Mining Technology
on Web logs’, Advances in Digital Libraries Conference, PP 19--29
[44]Zaiane Osmar R. (2001), ‘Web Usage Mining for a Better Web-Based Learning Environment’, Conference on Advanced Technology for Education,
pages 60--64
[45]Zaiane Osmar R. and Cai JiAn (2001), ‘Web Mining: From Concepts to Practical Systems’, DASFAA’2001
29 Mars, 2006
39
Merci à Votre Attention
29 Mars, 2006
40
Bienvenu votre
Conseil et Question
29 Mars, 2006
41