Image-based Tracking and Understanding

Perception is essential for humans to interact with each other and with the surrounding environment. Humans are able to build, match, and update stored reference object models on-line and follow them in time. Therefore, every artificial system or agent that aims to interact in a variable environment with humans in a native way must be able to interpret and react to common and visually measurable utterances of their human partners. Most importantly, it must be able to follow environmental changes and to track the most diverse range of objects in its visual field.

Visual tracking combines fast computer vision and image understanding techniques with sequential estimation methodologies (tracking) for localizing one or more moving objects in a sequence of images from at least one camera using information from the past to predict the expected state for the next time frame. Hence, visual tracking itself is a cognitive process. Results of visual tracking can be applied to a wide variety of research fields including cognitive robotics, man-machine interfaces, video surveillance, scene analysis and interpretation, non-invasive monitoring of reagents, navigation systems, and self-localization.

ITrackU aims to develop an object-oriented, unified software architecture for visual model-based tracking for the CoTeSys cluster with both, theoretical and practical goals:

provide advanced visual tracking methods and modules
create a common framework (software library) to design, describe, and specify a tracking system by setting only few system parameters
use the software modules as building blocks for more complex systems (e.g., gesture recognition, workspace surveillance)

integrate the software into smart sensors (e.g. intelligent microscopes)

CCRL demonstration scenario

Project developers

Robotics and Embedded Systems (Informatik VI)

Coordinator: Giorgio Panin, Ph.D.
Thorsten Röder, M.Sc.
Dipl.-Ing. Erwin Roth
Dipl.-Ing. Claus Lenz
Suraj Nair, M.Sc.
Dr. Martin Wojtczyk
Dipl.-Inf. Thomas Friedlhuber

OpenTL Framework

More information about the OpenTL framework can be found at the OpenTL website.

Project partners

Chair for Human-Machine Communication (MMK)

Dr.-Ing. Frank Wallhoff
Dipl.-Ing. Tobias Rehrl

Project Initiators

Prof. Dr.-Ing. Alois Knoll (Informatik VI)
Prof. Dr.-Ing. Gerhard Rigoll (MMK)

Acknowledgements

This ongoing work is supported by the DFG excellence initiative research cluster Cognition for Technical Systems CoTeSys, see www.cotesys.org for further details.

Publications

[1]	Claus Lenz, Alice Sotzek, Thorsten Röder, Helmuth Radrich, Alois Knoll, Markus Huber, and Stefan Glasauer. Human workflow analysis using 3d occupancy grid hand tracking in a human-robot collaboration scenario. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, San Francisco, USA, 2011. [ DOI \| .bib \| .pdf ]
[2]	Claus Lenz. Context-aware human-robot collaboration as a basis for future cognitive factories. Dissertation, Technische Universität München, München, 2011. [ .bib \| .pdf ]
[3]	Claus Lenz, Thorsten Röder, Martin Eggers, Sikandar Amin, Thomas Kisler, Bernd Radig, Giorgio Panin, and Alois Knoll. A distributed many-camera system for multi-person tracking. In Proceedings of the 1st International Joint Conference on Ambient Intelligence, Malaga, Spain, 2010. [ DOI \| .bib \| .pdf ]
[4]	Claus Lenz, Giorgio Panin, Thorsten Röder, Martin Wojtczyk, and Alois Knoll. Hardware-assisted multiple object tracking for human-robot-interaction. In Francois Michaud, Matthias Scheutz, Pamela Hinds, and Brian Scassellati, editors, HRI '09: Proceedings of the 4th ACM/IEEE international conference on Human robot interaction, pages 283-284, La Jolla, CA, USA, 2009. ACM. [ DOI \| .bib \| .pdf ]
[5]	Claus Lenz, Markus Rickert, Giorgio Panin, and Alois Knoll. Constraint task-based control in industrial settings. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 3058-3063, St. Louis, MO, USA, 2009. [ DOI \| .bib \| .pdf ]
[6]	Thomas Müller and Alois Knoll. Virtualization techniques for cross platform automated software builds, tests and deployment. In Proceedings of the Fourth International Conference on Software Engineering Advances, Porto, Portugal, 2009. [ .bib \| .pdf ]
[7]	Suraj Nair, Giorgio Panin, Thorsten Röder, Thomas Friedelhuber, and Alois Knoll. A distributed and scalable person tracking system for robotic visual servoing with 8 dof in virtual reality tv studio automation. In Proceedings of the 6th International Symposium on Mechatronics and its Applications (ISMA09). IEEE, 2009. [ .bib \| .pdf ]
[8]	Giorgio Panin, Sebastian Klose, and Alois Knoll. Real-time articulated hand detection and pose estimation. In International Symposium on Visual Computing (ISVC), Las Vegas, Nevada, USA, 2009. [ .bib \| .pdf ]
[9]	Giorgio Panin, Sebastian Klose, and Alois Knoll. Multi-target and multi-camera object detection with monte-carlo sampling. In International Symposium on Visual Computing (ISVC), Las Vegas, Nevada, USA, 2009. [ .bib \| .pdf ]
[10]	Martin Wojtczyk, Giorgio Panin, Claus Lenz, Thorsten Röder, Suraj Nair, Erwin Roth, Rüdiger Heidemann, Klaus Joeris, Chun Zhang, Mark Burnett, Tom Monica, and Alois Knoll. A vision based human robot interface for robotic walkthroughs in a biotech laboratory. In François Michaud, Matthias Scheutz, Pamela Hinds, and Brian Scassellati, editors, HRI '09: Proceedings of the 4th ACM/IEEE international conference on Human Robot Interaction, pages 309-310, La Jolla, CA, USA, 2009. ACM. [ DOI \| .bib \| .pdf ]
[11]	Claus Lenz, Giorgio Panin, and Alois Knoll. A GPU-accelerated particle filter with pixel-level likelihood. In International Workshop on Vision, Modeling and Visualization (VMV), Konstanz, Germany, 2008. [ .bib \| .pdf ]
[12]	Thomas Müller, Claus Lenz, Simon Barner, and Alois Knoll. Accelerating integral histograms using an adaptive approach. In Proceedings of the 3rd International Conference on Image and Signal Processing, Lecture Notes in Computer Science (LNCS), pages 209-217, Cherbourg-Octeville, France, 2008. Springer. [ DOI \| .bib \| .pdf ]
[13]	Suraj Nair, Giorgio Panin, Martin Wojtczyk, Claus Lenz, Thomas Friedelhuber, and Alois Knoll. A multi-camera person tracking system for robotic applications in virtual reality tv studio. In Proceedings of the 17th IEEE/RSJ International Conference on Intelligent Robots and Systems 2008. IEEE, 2008. [ .bib \| .pdf ]
[14]	Giorgio Panin, Claus Lenz, Suraj Nair, Erwin Roth, Martin Wojtczyk, Thomas Friedlhuber, and Alois Knoll. A unifying software architecture for model-based visual tracking. In IS&T/SPIE 20th Annual Symposium of Electronic Imaging, San Jose, CA, 2008. [ .bib \| .pdf ]
[15]	Giorgio Panin, Erwin Roth, and Alois Knoll. Robust contour-based object tracking integrating color and edge likelihoods. In International Workshop on Vision, Modeling and Visualization (VMV), Konstanz, Germany, 2008. [ .bib \| .pdf ]
[16]	Giorgio Panin, Thorsten Röder, and Alois Knoll. Integrating robust likelihoods with monte-carlo filters for multi-target tracking. In International Workshop on Vision, Modeling and Visualization (VMV), Konstanz, Germany, 2008. [ .bib \| .pdf ]
[17]	Giorgio Panin, Erwin Roth, Thorsten Röder, Suraj Nair, Claus Lenz, Martin Wojtczyk, Thomas Friedlhuber, and Alois Knoll. ITrackU: An integrated framework for image-based tracking and understanding. In Proceedings of the International Workshop on Cognition for Technical Systems, Munich, Germany, 2008. [ .bib \| .pdf ]
[18]	Martin Wojtczyk, Kushal Abhyankar, Suraj Nair, and Alois Knoll. A computer vision based approach for cell tracking to increase throughput in visual drug discovery. In Natural Product Discovery and Production II, Whistler B.C., Canada, 2008. [ .bib \| .pdf ]