Temporal Commonality Discovery


Temporal Commonality Discovery (TCD) ≡ Given two (or more) videos, discover common events in an unsupervised manner.


Code


  • Usage

    In Matlab

    >> cd tcd_v1.3
    >> make
    >> addpaths 
    >> demoTCD

    • Change the variable pickEx for more examples.

  • File Structure

    Files included in tcd_v1.3.zip

    • lib/: Folder of useful functions
    • src/: Folder of C++ implementation
    • addpaths.m: Add required paths to Matlab
    • demoTCD.m: Demo of TCD
    • make.m: Matlab makefile for C++ code

  • Change Log

    2014-08-08

    • Implemented the main TCD routine in C++
    • The speedup against v1.2 is fascinating

     

    2013-01-06

    • Extend TCD with soft clustering and hard clustering
    • Speedup TCD search using integral image
    • Speedup multiple TCD search using queue elimination

     

    2012-10-06

    • Add the main implementation of the TCD algorithm
    • Add comparisons to the naive SW approach
    • Add a demo for 8 synthetic examples, including detecting multiple commonalities and an application to video indexing

  • More info

    Check the readme

Introduction

  • Temporal Commonality Discovery (TCD)
    • Unsupervised commonality discovery in images has recently attracted much interest (see an example of co-segmentation).
    • In this study, we investigate a relatively unexplored problem to discover common semantic temporal patterns in videos.

  • Contributions
    • To the best of our knowledge, this is the first work that addresses unsupervised discovery of common events in videos.
    • By interpreting the problem into a 2D search space, we propose a branch and bound (B&B) algorithm to efficiently find the global optimum.
    • We derive bounding functions for classical distances between Temporal Bag of Words (TBoW) of two segments, including L1, intersection and Χ2.
    • The framework is general and can be applied to any feature that can be quantified into histograms.


C++ Implementation and Speedup

Check out the following figure for speedup with C++ against Matlab implementation. We compared the efficiency in terms of 8 synthetic examples provided in TCD_v1.2. Different colors indicate various distance functions as explained in the paper.




Publications

    Unsupervised Temporal Commonality Discovery
Wen-Sheng Chu, Feng Zhou and Fernando De la Torre
in ECCV 2012, Firenze, Italy.
[ pdf ][ code ][ poster (6.7MB) ][ slides (12MB) ]
  

Copyright

Thanks for using the software! This software is free for use in research projects. If you publish results obtained using this software, please use this citation:
@inproceedings{chu2012unsupervised,
    title={Unsupervised temporal commonality discovery},
    author={Wen-Sheng Chu and Feng Zhou and Fernando {De la Torre}},
    booktitle={ECCV},
    year={2012}
}
Contributing back bugfixes and improvements is polite and encouraged. If you have any questions, feel free to contact Wen-Sheng Chu. Contributions to my GitHub page is welcomed.