Joint Alignment and Modeling of Correlated Behavior Streams

Liliana Lo Presti, Liliana Lo Presti, Stan Sclaroff, Agata Rozga

Risultato della ricerca: Otherpeer review

15 Citazioni (Scopus)


The Variable Time-Shift Hidden Markov Model (VTS- HMM) is proposed for learning and modeling pairs of cor- related streams. Unlike previous coupled models for time series, the VTS-HMM accounts for varying time shifts be- tween correlated events in pairs of streams having different properties. The VTS-HMM is learned on a set of pairs of unaligned streams and, thus, learning entails simultaneous estimation of the varying time shifts and of the parameters of the model. The formulation is demonstrated in the analysis of videos of dyadic social interactions between children and adults in the Multimodal Dyadic Behavior Dataset (MMDB). In dyadic social interactions, an agent starts an interaction with one or more “initiating behaviors” that elicit one or more “responding behaviors” from the partner within a temporal window. The proposed VTS-HMM explicitly accounts for varying time shifts between initiating and responding behaviors in these behavior streams. The experiments confirm that modeling of these varying time shifts in the VTS-HMM can yield improved estimation of the level of engagement of the child and adult and more accurate dis- crimination among complex activities.
Lingua originaleEnglish
Numero di pagine8
Stato di pubblicazionePublished - 2013

All Science Journal Classification (ASJC) codes

  • Software
  • Computer Vision and Pattern Recognition

Fingerprint Entra nei temi di ricerca di 'Joint Alignment and Modeling of Correlated Behavior Streams'. Insieme formano una fingerprint unica.

Cita questo