ICCV 2019: 1st Workshop on Visual Perception for Navigation in Human Environments

  • Augest 20

    Submission Deadline

  • Augest 22

    Dataset Released

  • September 6

    Acceptance Notifications

  • September 22

    Camera Ready Data

  • October 22

    Challenge Deadline

  • October 28

    Date of Workshop

Workshop Goals

This workshop focuses on the unique perceptual problems related to autonomous navigation in indoor and outdoor human environments. These problems include human detection and tracking from 2D and/or 3D data, human posture detection and prediction, object detection, segmentation, trajectory forecasting and any other perceptual task that, when solved, provides valuable information to autonomous agents and robots that need to navigate safely in human crowded environments.

Last years the community has paid increasing attention to these perceptual problems and has provided annotated datasets to learn to solve them and evaluate them, yet restricted to the point of view of autonomous vehicles. Robots and other navigating agents, however, view the world from a different perspective, can go indoors, get closer to humans and need to perceive other types of information. In this workshop, we present the JackRabbot social navigation dataset , a novel annotated dataset with the signals from our mobile manipulator JackRabbot. For the workshop we launch the first set of visual challenges, to benchmark perceptual algorithms in this new type of data

The dataset contains 67 minutes of ground truth annotated sensor data acquired from the JackRabbot mobile manipulator and includes more than 50 indoor and outdoor sequences in a university campus environment. Sensor data includes a stereo RGB 360° cylindrical video stream, 3D point clouds from two LiDAR sensors, audio and GPS positions.

In the workshop we will present the winners of the first challenge. We have also as invited speakers world renowned expers in perception for autonomous navigation. We aim to foster discussion between the attendants to find useful synergies and applications of the solutions of perceptual tasks for navigation in human environments.


In this first phase, the annotations of the JackRobot Dataset include:

  • 2D bounding box annotations around pedestrians, cyclist and vehicles in five RGB streams and in their cylindrical composition.
  • 3D oriented cuboid annotations around pedestrians, cyclist, vehicles in two Velodyne-16 LiDAR streams
  • 2D-3D associations between bounding boxes and cuboids.

Open Challenge

We launch in this workshop a first challenge on our annotated data in the following categories:

  • 2D-3D person detection.
  • 2D-3D person tracking

The winner of each challenge will be given an opportunity to present their work as oral peresentation during the workshop.

Call for Papers

We invite researchers to submit their papers addressing topics related to autonomous (robot) navigation in human environments. Relevant topics include:

  • 2D or 3D object detection and pose estimation, tracking and forecasting
  • 2D or 3D pedestrian and vehicle detection, tracking and forecasting
  • 2D or 3D human skeleton pose estimation, tracking and forecasting
  • 2D or 3D human face detection
  • Human emotion recognition
  • Human walking behaviour analysis
  • 2D and 3D semantic, instance or panoptic segmentation
  • Motion prediction and social models
  • Individual, group and social activity recognition
  • Abnormal activity recognition
  • Visual and social navigation in crowded scenes
  • Dataset proposals and bias analysis
  • New metrics and performance measure for different visual perception problems related to autonomous navigation

Submisison format should match the ICCV format, e.g. 8 pages, double column format. By submitting to this workshop, authors agree to the review process and understand that we will do our best to match papers to the best possible reviewers. The reviewing process is double blind. Submission to the challenge is independent from the paper submission, but we encourage paper authors to submit to one of the challenges.

Submissions can be made here. If you have any questions about submitting, please contact us here.


Start Time End Time Description
8:30 AM 8:45 AM Introduction
8:45 AM 9:15 AM Invited Talk: Konrad Schindler, ETH Zurich - A Tale of Two Benchmarks
9:15 AM 9:45 AM Invited Talk: Raquel Urtasun, University of Toronto & Uber ATG - TBA
9:45 AM 10:15 AM The JackRabbot Dataset and Challenge
10:15 AM 10:45 AM Coffee Break
10:45 AM 11:15 AM Invited Talk: Michael Black, Max Planck Institute for Intelligent Systems & Amazon - Estimating 3D people interacting with 3D scenes
11:15 AM 11:45 AM Invited Talk: Bernt Schiele, Max Planck Institute for Informatics - 3D Human Pose Estimation and Model Stealing
11:45 AM 12:15 AM Invited Talk: Alexandre Alahi, EPFL - The 3P's of Mobility: Perceiving, Predicting, and Planning
12:15 PM 12:30 PM Discussion and Closing Remarks

Invited Speakers

Konrad Schindler

Raquel Urtasun

Michael Black

Bernt Schiele

Alexandre Alahi

Invited Speakers

Konrad Schindler

Raquel Urtasun

Michael Black

Bernt Schiele

Alexandre Alahi

Program Committee

Name Organization
Afshin Dehghan Apple, Inc.
Alexandre Alahi EPFL
Amir A. Sadeghian AiBee, Inc.
Anthony Dick University of Adelaide
Chunzhao Guo Toyota Research Institute
Ehsan Adeli Stanford University
Fatemeh Sadat Australian National University
Jingwei Ji Stanford University
Juan Carlos Niebles Stanford University
JunYoung Gwak Stanford University
Kiyosumi Kidono Toyota Central R&D Labs, Inc.
Laura Leal-Taixé Technical University of Munich
Niko Suenderhauf Queensland University of Technology
Noriaki Hirose Toyota Central R&D Labs, Inc.
Qinfeng Shi University of Adelaide
Shun Taguchi Toyota Central R&D Labs, Inc.
Shyamal Buch Stanford University
Noriaki Hirose Toyota Central
Stephen Gould Australian National University
Wongun Choi AiBee, Inc.


Hamid Rezatofighi

Roberto Martín-Martín

Ian Reid

Silvio Savarese