, . . . http://cvsp.cs.ntua.gr . , .

, . . . http://cvsp.cs.ntua.gr . , . . CVSP --

. () 3 7 . + 2-5 . + . : . , . , / ( )

- & / & : http://cvsp.cs.ntua.gr

(McGurk & MacDonald)) () : / -

/ , :

- (King et al., Deng) : N (Articulatory Gestures, Browman & Gold)stein) ... (.. Bell, 1867))

G. Papand)reou, A. Katsamanis, V. Pitsikalis, and) P. Maragos, Ad)aptive Multimod)al Fusion by Uncertainty Compensation with Application to Aud)io-Visual Speech Recognition, IEEE Trans. ASLP, 2009 : &

() :

1 2 :

. Face detector System Overview Adaboost-based, @5 fps Image Acquisition Firewire color camera, 640x480 @25 fps (Re)initialization

Face tracking & feature extraction Real-time AAM fitting algorithms GPU-accelerated processing OpenGL implementation HMM-based backend Transcription :

; (.. ) , , , ...

: , , (Knill & Richard)s) (.. Ernst et al.) // Maragos et al., Cross-Mod)al Integration, Springer 2008 : :

: Wiener Kalman ; : :

SNR= 20d)B SNR= 5d)B : o (Gaussian Mixture Mod)el - GMM) S

. : : C X : C X

Y !

GMM S p c | x1:s p (c)1 s ,c N xs ; s ,c , s ,c C X :

p ys | xs N ys ; xs e , s , e, s S M s ,c p c | y1:s p (c ) s ,c ,m N ys ; s ,c ,m e, s , s ,c ,m e , s s 1 m 1 C X Y GMM

1- (y1 y2), 2 S ws : b (c | y ) p ( c ) p( y | c)

1:s s 1 : S p c | y1:s p (c)1 N ys ; s ,c , s ,c e ,s PoG : w N x; , N x; , w 1

S b c | y1:s p (c)1 N ys ; s ,c , w s ,c : 1 s ,c ws ,c e ,s 1 1 s ,c

EM- C

Q( ,pXCX ) [log p( X ,{C}| ) | X , pXCX ] X C X Y Q( , pXCX ) [log p(Y ,{ X , C}| ) | Y , pXCX ]

Markov () & Viterbi () - () ( frame) C1 C2

C3 C4 X1 X2 X3 X4 C1

C2 C3 C4 X1

X2 X3 X4 Y1 Y2 Y3 Y4

Mel Frequency Cepstral Coefficients (MFCCs): Pre-emphasis STFT | . | Mel-scale log( . ) DCT (e.g. SPLICE, ALGONQUIN) MFCC (VTS) X noisy f ( X clean , N ) MFCC MFCC

+ X clean X E Deng, Droppo, Acero, IEEE Tr. SAP, 2005 - 1 2 3

C1 C2 C3 X1 X2 X3 Multistream-

Product- : Asynchronous-HMM, Coupled)-HMM, Dynamic Bayesian Networks, CUAVE

. : CUAVE: 36 (30 , 6 ) 5 10 : 1500 (30x5x10) : 300 (6x5x10) babble - NOISEX HMMs (- , 8 , 1 /, ) HTK (

) AV A

/ AV-W-UC vs. A-UC 28.7 %

AV-UC vs. AV AV-W-UC vs. AV-W 20 % Product-HMM Prod)uct-HMM vs.

Multistream-HMM 1.2 % : &

: MUSCLE (NoE) & HIWIRE (STREP) - A. Katsamanis, G. Papand)reou, and) P. Maragos, Face Active Appearance Mod)eling and) Speech Acoustic Information to Recover Articulation, IEEE Trans. ASLP, 2009 -

: : () : , , MOCHA CSTR, Univ. Edinburgh

(, 1 /1 ), 460 TIMIT (2- 9 ) 30 - phoneme

37 y, x : prior : Yehia, Rubin & Vatikiotis-Bateson, Speech Comm., 1998

CCA . (CCA) CCA : : . : 40

Viterbi Markov -> Hiroya & Honda, IEEE TSAP 2004 : /

: . : HMM / MS-HMM: () : / . : Visemes ( ) ( ) MOCHA

- ( ) (//) : :

: . 51 Katsamanis et al. EUSIPCO 2008 / CVSP (. )

: X-rays, (. . ) Audiovisual Speech Inversion Articulatory Parameter Extraction Articulatory Speech Synthesis Articulatory Model

Training - : : ()

: , , : ASPI (FET) & ()

!

: http://cvsp.cs.ntua.gr

Recently Viewed Presentations

  • Temporal Aspects of Visual Extinction

    Temporal Aspects of Visual Extinction

    X-Linked Inheritance Females have two X-chromosomes, males have one. All boys inherited their X-chromosome from their mother. Boys vulnerable to recessive mutations on the X-chromosome. Example: Red-green color blindness. Plunnett square For traits that are determined by a single gene,...
  • Health Safety and Environmental Management The Compliance Consultancy

    Health Safety and Environmental Management The Compliance Consultancy

    An innovative approach to Bridge Building - Build the unit on flat ground, once finished dig the "Roadway" under it We choose collective control measures whenever possible because it protects the greater number of people, it only requires that nobody...
  • Seizures in Koolen de Vries Syndrome

    Seizures in Koolen de Vries Syndrome

    Robin Clegg. Jeffrey Buchhalter. Austin Health/ University of Melbourne. Ingrid Scheffer. Royal Hospital for Sick Children/ University of Glasgow. Joe Symonds. Sameer Zuberi. Interval between EEG and SUDEP. EEG at 14 years, died at 16 years.
  • MLA Documentation Learning Module - megritchey

    MLA Documentation Learning Module - megritchey

    Web. 8 Feb. 2012 Article posted on a wiki - A wiki is a web site whose users can make additions or changes. "How to Write a Wikipedia Article." Wiki How. Media Wiki., 10 Sept. 2010. 11 February, 2012 E-mail...
  • Psychological perspectives for Health and Social Care

    Psychological perspectives for Health and Social Care

    Psychological perspectives for Health and Social Care ... Oxford University Press * * * * * * * Unit Outcomes On completion of this unit a learner should: 1 Understand psychological perspectives 2 Understand psychological approaches to health and social...
  • Managing Users and Their Resources

    Managing Users and Their Resources

    [email protected] ... Schoolbook MagnaLearn™Temp304 1_MagnaLearn™Temp304 2_MagnaLearn™Temp304 Microsoft Photo Editor 3.0 Photo Canvas Drawing The Role of Roles and Privileges The Role of Roles and Privileges Database Security Privileges and Roles System Privileges System Privileges ...
  • INTERNET NEWS - Trung tâm y khoa MEDIC

    INTERNET NEWS - Trung tâm y khoa MEDIC

    "Nasal polyps can lead to loss of smell and often patients require surgery to remove the polyps," Sally Seymour, MD, director, Division of Pulmonary, Allergy and Rheumatology Products in the FDA's Center for Drug Evaluation and Research, said in the...
  • OCEN 201 Introduction to Ocean & Coastal Engineering

    OCEN 201 Introduction to Ocean & Coastal Engineering

    OCEN 201 Introduction to Ocean & Coastal Engineering Introduction Jun Zhang [email protected] * * Definition of Ocean Engineering: can be defined as the application of engineering principles to the analysis, design, development and management of systems that must function in...