ICCV 2023: 4th Workshop on Visual Perception for Navigation in Human Environments.

The JackRabbot Human Motion Forecasting


Workshop Goals

Welcome to the fourth workshop in the JRDB series! Our workshops are designed to tackle perceptual problems that arise when autonomous robots operate, interact, and navigate in human environments including human detection, tracking, forecasting, and body pose estimation, as well as social grouping and activity recognition.

In this workshop, we are excited to explore the challenging task of human motion forecasting in dynamic and unpredictable environments. We will introduce a new competition that challenges participants to develop models capable of accurately predicting human trajectories using the JRDB dataset. We have also invited speakers from the field of visual perception and prediction to offer valuable insights into understanding human action and behavior.

Call for Papers


We invite researchers to submit their papers addressing topics related to autonomous (robot) navigation in human environments. Relevant topics include, but not limited to:

  • Human trajectory forecasting
  • Human motion prediction and safety
  • Visual scene prediction
  • Human-robot Interaction considering predictions
  • Visual and social navigation in crowded scenes
  • Human motion/body skeleton pose prediction
  • Predictive planning and control
  • 2D or 3D human detection and tracking
  • 2D or 3D skeleton pose estimation and tracking
  • Individual, group and social activity recognition
  • Human walking behaviour analysis
  • 2D and 3D semantic, instance or panoptic segmentation
  • Dataset proposals and bias analysis
  • New metrics and performance measure for different visual perception problems related to autonomous navigation


  • Submission deadline for the full papers:
    July 14 July 24 - 23:59 PT
  • Acceptance notification of full papers:
    August 9 - 23:59 PT
  • Camera-ready deadline for the full papers:
    August 19 - 23:59 PT
  • Submission deadline for the extended abstracts:
    September 8 - 23:59 PT
  • Acceptance notification of the extended abstracts:
    September 22 - 23:59 PT


Full-paper are limited to eight pages, including figures and tables, in the ICCV style. Additional pages containing only cited references are allowed. For more information, please refer to the guidelines provided by ICCV here. Extended abstracts should also adhere to the ICCV style and are restricted to a maximum of one page, excluding references (The submission for extended abstracts will be open on August 9). Accepted papers have the opportunity to be presented as a poster during the workshop. However, only full-papers will appear in the proceedings. By submitting to this workshop, the authors agree to the review process and understand that we will do our best to match papers to the best possible reviewers. The reviewing process is double-blind. Submission to the challenge is independent of the paper submission, but we encourage the authors to submit to one of the challenges.

Submissions can be made here. If you have any questions about submitting, please contact us here.

Submit to our Workshop

Accepted Papers

SegDA: Maximum Separable Segment Mask with Pseudo Labels for Domain Adaptive Semantic Segmentation

Anant Khandelwal

Cross-Dimensional Refined Learning for Real-Time 3D Visual Perception from Monocular Video

Ziyang Hong C., Patrick Yue

A Dual Perspective of Human Motion Analysis - 3D Pose Estimation and 2D Trajectory Prediction

Mayssa Zaier, Hazem Wannous, Hassen Drira, Jacques Boonaert

A Lightweight Skeleton-Based 3D-CNN for Real-Time Fall Detection and Action Recognition

Nadhira Noor, In Kyu Park

THOR-Magni: Comparative Analysis of Deep Learning Models for Role-conditioned Human Motion Prediction

Tiago Rodrigues de Almeida, Andrey Rudenko, Tim Schreiter, Yufei Zhu, Eduardo Gutierrez Maestro, Lucas Morillo-Mendez, Tomasz P. Kucner, Oscar Martinez Mozos, Martin Magnusson, Luigi Palmieri, Kai O. Arras, Achim J. Lilienthal

Open Challenge

🏆 Winner Announced! 🎉

Congratulations to the "Human Scene Transformer" team for securing the 1st place and winning the challenge! 🥇
For more details and to explore the performance of other teams, check out the leaderboard.

We are organizing a new challenge on end-to-end Human Trajectory Forecasting with a cash prize of $1,000 sponsored by VinAI.

In real-world scenarios, the ability to forecast trajectories is crucial, even in situations where tracking algorithms may not perform optimally. This poses an open challenge that requires end-to-end forecasting future trajectories given raw sensory data. In this task, given video clips and point clouds of jrdb dataset, the objective is to forecast the future trajectories of people (x,y coordinates of center of bounding boxes on the ground) in each scene for the next 4.8 seconds at a frame rate of 2.5 frames per second. The videos and point clouds for the 4.8 seconds part are hidden. The input is unrestricted, and we solely assess the forecast trajectories for the hidden part of the test. Feel free to use tracking leaderboards, off-the-shelf models, or even your own model to detect people from input data and forecast people's trajectories! For more information on the dataset, metric and benchmark, please refer to JRDB-Traj paper.

Each challenge submission should be followed by an extended abstract or full paper submission via our paper submission portal (see details above) or a link to an existing preprint/publication. The top 3 will have the opportunity to present their work as a spotlight (5 minutes) in the workshop, with the first-place winner receiving the prize. To qualify for the first-place prize ($1,000) and certificate, a minimum of 5 distinct groups must submit their solutions to the challenge before the deadline.

The challenge submission deadline is September 15 September 18 - 23:59 PT.

Guidelines for participation

Participants must strictly adhere to the submission policy provided on the main JRDB webpage, which can be found here. To differentiate challenge submissions from other regular submissions, each submission name should be followed by an ICCV23 tag, e.g., "submissionname_ICCV23". Submissions without this tag will not be considered for the challenge. Moreover, the name, abstract and the link to a paper (an arXiv preprint is acceptable) should be completed in due time. Please be aware that after the updating information deadline, incomplete submissions will be filtered out.

Evaluation & Toolkit

We use the first metric after "name" in the leaderboard as the main evaluation for ranking the entries. For each benchmark, we have also created toolkits to work with the dataset, perform evaluation, and create submissions. These toolkits are available at here.


Time Speaker Title
13:30 - 13:40 Introduction
13:40 - 14:10 Fatma Güney Multi-Agent Trajectory Prediction: Scene-Centric or Agent-Centric?
14:10 - 14:30 Saeed Saadatnejad and Hamid Rezatofighi Introducing the JRDB Dataset and the Challenge
14:30 - 15:00 Patrick Pérez Better Perception for Safer Driving
15:00 - 15:15 Khoi Nguyen VinAI, Data Annotation Services
15:15 - 15:45 Coffee break & poster session
15:45 - 16:15 Ehsan Adeli Compositional Activity Parsing
16:15 - 16:45 Georgios Pavlakos Perceiving Humans in the 4D World
16:45 - 17:00 Challenge winners presentations
17:00 - 17:30 Marco Pavone Scaling Up AV Simulation via Generative AI
17:30 - 17:40 Discussion, Closing Remarks and Awards

Please follow the ZOOM link for virtual attendance.

Invited Speakers

Marco Pavone

Associate Professor at Stanford University, USA

Fatma Güney

Assistant Professor at Koc University in Istanbul, Turkey

Patrick Pérez

Valeo VP of AI and Scientific Director of valeo.ai, France

Georgios Pavlakos

Assistant Professor at UT Austin, USA

Ehsan Adeli

Assistant Professor at Stanford University, USA

Video Recordings

Program Committee

Name Organization
Andrey Rudenko Bosch


Hamid Rezatofighi

Monash University

Alexandre Alahi


Ian Reid

The University of Adelaide

Silvio Savarese

Stanford University

Saeed Saadatnejad*


Simindokht Jahangard

Monash University

Duy Tho Le

Monash University

* Corresponding organizer. For inquiries, please contact.