JRDB Leaderboard

Social Group Submissions

Name

mAP1

mAP2

32.5%

27.4%

Simindokht Jahangard, Munawar Hayat, Hamid Rezatofighi Real-time Trajectory-based Social Group Detection in Proceedings of the 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

Details

Name	Trajectory_based
Submission Date	2022-11-01 15:46:59+00:00
Abstract	we propose a simple and efficient framework that performs reliably in challenging scenarios. Explicitly, we explore the effect of motion trajectory on social grouping detection and propose a novel, reliable and fast data-driven approach to detect social groups. To this end, we first formulate all the individuals in a scene as a graph with (i) the node features attained using an LSTM encoding the trajectories and (ii) an edge feature, calculated by a distance between each pair of the tracks. Next, our approach adopts a modified graph transformer module and graph clustering losses to detect social groups.
Publication title	Real-time Trajectory-based Social Group Detection	Publication authors:	Simindokht Jahangard, Munawar Hayat, Hamid Rezatofighi
Publication venue and year	Proceedings of the 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)	Publication URL:	https://arxiv.org/pdf/2304.05678.pdf
Code Language	N/A	Hardware	N/A
Code Website	N/A	Code URL	N/A

Visualisation

Name: hewlett-class-2019-01-23_1, Video Generating currently not available!

Best 3 By mAP1 ↓

Worst 3 By mAP1 ↓

Per-sequence Results

Sequence	mAP	G1	G2	G3	G4	G5+
cubberly-auditorium-2019-04-22_1	0.401	0.244	0.448	0.512	-	-
discovery-walk-2019-02-28_0	0.366	0.253	0.48	-	-	-
discovery-walk-2019-02-28_1	0.492	0.228	0.651	0.417	0.672	-
food-trucks-2019-02-12_0	0.423	0.217	0.521	0.77	-	0.184
gates-ai-lab-2019-04-17_0	0.435	0.225	0.374	0.446	0.623	0.505
gates-basement-elevators-2019-01-17_0	0.427	0.206	0.624	0.451	-	-
gates-foyer-2019-01-17_0	0.474	0.411	0.564	0.446	-	-
gates-to-clark-2019-02-28_0	0.456	0.342	0.569	-	-	-
hewlett-class-2019-01-23_0	0.317	0.224	0.411	-	-	-
hewlett-class-2019-01-23_1	0.568	0.591	0.544	-	-	-
huang-2-2019-01-25_1	0.196	0.315	0.482	0.0	0.125	0.06
huang-intersection-2019-01-22_0	0.256	0.22	0.559	0.23	0.023	0.249
indoor-coupa-cafe-2019-02-06_0	0.237	0.149	0.324	-	-	-
lomita-serra-intersection-2019-01-30_0	0.379	0.252	0.506	-	-	-
meyer-green-2019-03-16_1	0.268	0.171	0.47	0.164	-	-
nvidia-aud-2019-01-25_0	0.447	0.183	0.4	0.667	0.624	0.359
nvidia-aud-2019-04-18_1	0.441	0.23	0.779	0.313	-	-
nvidia-aud-2019-04-18_2	0.476	0.238	0.714	-	-	-
outdoor-coupa-cafe-2019-02-06_0	0.232	0.122	0.341	-	-	-
quarry-road-2019-02-28_0	0.288	0.195	0.381	-	-	-
serra-street-2019-01-30_0	0.166	0.234	0.177	0.184	0.028	0.207
stlc-111-2019-04-19_1	0.484	0.293	0.676	-	-	-
stlc-111-2019-04-19_2	0.469	0.369	0.57	-	-	-
tressider-2019-03-16_2	0.362	0.146	0.613	0.328	-	-
tressider-2019-04-26_0	0.179	0.117	0.221	0.237	0.057	0.261
tressider-2019-04-26_1	0.372	0.245	0.447	0.368	0.456	0.345
tressider-2019-04-26_3	0.184	0.152	0.322	0.094	0.169	-
overall	0.325	0.195	0.397	0.3	0.486	0.247

Sequence	AP
cubberly-auditorium-2019-04-22_1	0.297
discovery-walk-2019-02-28_0	0.343
discovery-walk-2019-02-28_1	0.442
food-trucks-2019-02-12_0	0.304
gates-ai-lab-2019-04-17_0	0.395
gates-basement-elevators-2019-01-17_0	0.359
gates-foyer-2019-01-17_0	0.47
gates-to-clark-2019-02-28_0	0.414
hewlett-class-2019-01-23_0	0.257
hewlett-class-2019-01-23_1	0.566
huang-2-2019-01-25_1	0.324
huang-intersection-2019-01-22_0	0.306
indoor-coupa-cafe-2019-02-06_0	0.225
lomita-serra-intersection-2019-01-30_0	0.293
meyer-green-2019-03-16_1	0.274
nvidia-aud-2019-01-25_0	0.277
nvidia-aud-2019-04-18_1	0.446
nvidia-aud-2019-04-18_2	0.433
outdoor-coupa-cafe-2019-02-06_0	0.207
quarry-road-2019-02-28_0	0.234
serra-street-2019-01-30_0	0.199
stlc-111-2019-04-19_1	0.423
stlc-111-2019-04-19_2	0.409
tressider-2019-03-16_2	0.255
tressider-2019-04-26_0	0.166
tressider-2019-04-26_1	0.339
tressider-2019-04-26_3	0.197
overall	0.274

JRDB-Act

31.6%

46.0%

Mahsa Ehsanpour, Fatemeh Saleh, Silvio Savarese, Ian Reid, Hamid Rezatofighi JRDB-Act: A Large-scale Dataset for Spatio-temporal Action, Social Group and Activity Detection in CVPR2022

Details

Name	JRDB-Act
Submission Date	2021-12-01 11:36:41+00:00
Abstract	The availability of large-scale video action understanding datasets has facilitated advances in the interpretation of visual scenes containing people. However, learning to recognise human actions and their social interactions in an unconstrained real-world environment comprising numerous people, with potentially highly unbalanced and long-tailed distributed action labels from a stream of sensory data captured from a mobile robot platform remains a significant challenge, not least owing to the lack of a reflective large-scale dataset. In this paper, we introduce JRDB-Act, as an extension of the existing JRDB, which is captured by a social mobile manipulator and reflects a real distribution of human daily-life actions in a university campus environment. JRDB-Act has been densely annotated with atomic actions, comprises over 2.8M action labels, constituting a large-scale spatio-temporal action detection dataset. Each human bounding box is labeled with one pose-based action label and multiple (optional) interaction-based action labels. Moreover JRDB-Act provides social group annotation, conducive to the task of grouping individuals based on their interactions in the scene to infer their social activities (common activities in each social group). Each annotated label in JRDB-Act is tagged with the annotators' confidence level which contributes to the development of reliable evaluation strategies. In order to demonstrate how one can effectively utilise such annotations, we develop an end-to-end trainable pipeline to learn and infer these tasks, i.e. individual action and social group detection.
Publication title	JRDB-Act: A Large-scale Dataset for Spatio-temporal Action, Social Group and Activity Detection	Publication authors:	Mahsa Ehsanpour, Fatemeh Saleh, Silvio Savarese, Ian Reid, Hamid Rezatofighi
Publication venue and year	CVPR2022	Publication URL:	https://openaccess.thecvf.com/content/CVPR2022/papers/Ehsanpour_JRDB-Act_A_Large-Scale_Dataset_for_S
Code Language	N/A	Hardware	N/A
Code Website	N/A	Code URL	N/A

Visualisation

Name: hewlett-class-2019-01-23_1, Video Generating currently not available!

Best 3 By mAP1 ↓

Worst 3 By mAP1 ↓

Per-sequence Results

Sequence	mAP	G1	G2	G3	G4	G5+
cubberly-auditorium-2019-04-22_1	0.415	0.575	0.401	0.27	-	-
discovery-walk-2019-02-28_0	0.642	0.75	0.534	-	-	-
discovery-walk-2019-02-28_1	0.426	0.72	0.462	0.22	0.303	-
food-trucks-2019-02-12_0	0.412	0.652	0.476	0.331	-	0.188
gates-ai-lab-2019-04-17_0	0.36	0.67	0.407	0.286	0.254	0.182
gates-basement-elevators-2019-01-17_0	0.489	0.664	0.493	0.311	-	-
gates-foyer-2019-01-17_0	0.498	0.719	0.514	0.261	-	-
gates-to-clark-2019-02-28_0	0.56	0.716	0.405	-	-	-
hewlett-class-2019-01-23_0	0.549	0.681	0.417	-	-	-
hewlett-class-2019-01-23_1	0.677	0.873	0.481	-	-	-
huang-2-2019-01-25_1	0.25	0.607	0.46	0.013	0.083	0.088
huang-intersection-2019-01-22_0	0.347	0.741	0.508	0.275	0.062	0.15
indoor-coupa-cafe-2019-02-06_0	0.401	0.438	0.363	-	-	-
lomita-serra-intersection-2019-01-30_0	0.62	0.711	0.529	-	-	-
meyer-green-2019-03-16_1	0.337	0.415	0.371	0.224	-	-
nvidia-aud-2019-01-25_0	0.302	0.558	0.34	0.222	0.25	0.138
nvidia-aud-2019-04-18_1	0.541	0.834	0.458	0.332	-	-
nvidia-aud-2019-04-18_2	0.608	0.731	0.485	-	-	-
outdoor-coupa-cafe-2019-02-06_0	0.405	0.446	0.365	-	-	-
quarry-road-2019-02-28_0	0.531	0.682	0.379	-	-	-
serra-street-2019-01-30_0	0.231	0.718	0.183	0.03	0.069	0.154
stlc-111-2019-04-19_1	0.642	0.766	0.517	-	-	-
stlc-111-2019-04-19_2	0.596	0.77	0.422	-	-	-
tressider-2019-03-16_2	0.491	0.731	0.451	0.291	-	-
tressider-2019-04-26_0	0.269	0.397	0.303	0.238	0.222	0.183
tressider-2019-04-26_1	0.327	0.604	0.428	0.296	0.223	0.084
tressider-2019-04-26_3	0.339	0.509	0.375	0.267	0.207	-
overall	0.316	0.563	0.394	0.243	0.229	0.153

Sequence	AP
cubberly-auditorium-2019-04-22_1	0.51
discovery-walk-2019-02-28_0	0.654
discovery-walk-2019-02-28_1	0.509
food-trucks-2019-02-12_0	0.519
gates-ai-lab-2019-04-17_0	0.372
gates-basement-elevators-2019-01-17_0	0.551
gates-foyer-2019-01-17_0	0.618
gates-to-clark-2019-02-28_0	0.535
hewlett-class-2019-01-23_0	0.632
hewlett-class-2019-01-23_1	0.719
huang-2-2019-01-25_1	0.461
huang-intersection-2019-01-22_0	0.5
indoor-coupa-cafe-2019-02-06_0	0.404
lomita-serra-intersection-2019-01-30_0	0.629
meyer-green-2019-03-16_1	0.363
nvidia-aud-2019-01-25_0	0.396
nvidia-aud-2019-04-18_1	0.606
nvidia-aud-2019-04-18_2	0.623
outdoor-coupa-cafe-2019-02-06_0	0.408
quarry-road-2019-02-28_0	0.584
serra-street-2019-01-30_0	0.502
stlc-111-2019-04-19_1	0.673
stlc-111-2019-04-19_2	0.656
tressider-2019-03-16_2	0.561
tressider-2019-04-26_0	0.331
tressider-2019-04-26_1	0.486
tressider-2019-04-26_3	0.446
overall	0.46

I3D-SA-GAT

24.5%

17.5%

Mahsa Ehsanpour, Alireza Abedin, Fatemeh Saleh, Javen Shi, Ian Reid, Hamid Rezatofighi Joint Learning of Social Groups, Individuals Action and Sub-group Activities in Videos in ECCV2020

Details

Name	I3D-SA-GAT
Submission Date	2021-12-01 11:13:13+00:00
Abstract	We use the Inflated 3D ConvNet (I3D) as the backbone to capture the spatio-temporal context of an input video clip. we conduct ROIAlign to project the coordinates on the frame’s feature map and slice out the corresponding features for each individual’s bounding box. we adopt self-attention mechanism to directly learn the interactions between any two feature positions of an individual’s feature representation and accordingly leverage this information to refine each individual’s feature map. We then adopt GAT which consumes the individuals’ feature map obtained from the self-attention component, encodes inter-node relations, and generates an updated representation for each individual. Finally, action classification is performed on the obtained individuals' feature representation.
Publication title	Joint Learning of Social Groups, Individuals Action and Sub-group Activities in Videos	Publication authors:	Mahsa Ehsanpour, Alireza Abedin, Fatemeh Saleh, Javen Shi, Ian Reid, Hamid Rezatofighi
Publication venue and year	ECCV2020	Publication URL:	https://arxiv.org/pdf/2007.02632.pdf
Code Language	N/A	Hardware	N/A
Code Website	N/A	Code URL	N/A

Visualisation

Name: lomita-serra-intersection-2019-01-30_0, Video Generating currently not available!

Best 3 By mAP1 ↓

Worst 3 By mAP1 ↓

Per-sequence Results

Sequence	mAP	G1	G2	G3	G4	G5+
cubberly-auditorium-2019-04-22_1	0.513	0.245	0.59	0.704	-	-
discovery-walk-2019-02-28_0	0.503	0.334	0.672	-	-	-
discovery-walk-2019-02-28_1	0.493	0.23	0.563	0.475	0.704	-
food-trucks-2019-02-12_0	0.266	0.109	0.211	0.51	-	0.232
gates-ai-lab-2019-04-17_0	0.343	0.126	0.181	0.35	0.543	0.518
gates-basement-elevators-2019-01-17_0	0.367	0.188	0.474	0.439	-	-
gates-foyer-2019-01-17_0	0.496	0.392	0.656	0.44	-	-
gates-to-clark-2019-02-28_0	0.487	0.304	0.671	-	-	-
hewlett-class-2019-01-23_0	0.213	0.12	0.307	-	-	-
hewlett-class-2019-01-23_1	0.465	0.303	0.626	-	-	-
huang-2-2019-01-25_1	0.246	0.224	0.614	0.111	0.193	0.087
huang-intersection-2019-01-22_0	0.354	0.29	0.566	0.394	0.031	0.488
indoor-coupa-cafe-2019-02-06_0	0.124	0.061	0.187	-	-	-
lomita-serra-intersection-2019-01-30_0	0.666	0.429	0.903	-	-	-
meyer-green-2019-03-16_1	0.187	0.135	0.315	0.109	-	-
nvidia-aud-2019-01-25_0	0.338	0.15	0.302	0.539	0.405	0.295
nvidia-aud-2019-04-18_1	0.485	0.394	0.571	0.489	-	-
nvidia-aud-2019-04-18_2	0.346	0.264	0.427	-	-	-
outdoor-coupa-cafe-2019-02-06_0	0.182	0.09	0.275	-	-	-
quarry-road-2019-02-28_0	0.416	0.295	0.536	-	-	-
serra-street-2019-01-30_0	0.182	0.273	0.35	0.079	0.009	0.201
stlc-111-2019-04-19_1	0.49	0.334	0.646	-	-	-
stlc-111-2019-04-19_2	0.532	0.347	0.716	-	-	-
tressider-2019-03-16_2	0.269	0.167	0.216	0.423	-	-
tressider-2019-04-26_0	0.103	0.018	0.081	0.125	0.064	0.225
tressider-2019-04-26_1	0.116	0.016	0.068	0.099	0.285	0.111
tressider-2019-04-26_3	0.092	0.025	0.113	0.062	0.167	-
overall	0.245	0.112	0.246	0.218	0.412	0.235

Sequence	AP
cubberly-auditorium-2019-04-22_1	0.343
discovery-walk-2019-02-28_0	0.482
discovery-walk-2019-02-28_1	0.464
food-trucks-2019-02-12_0	0.175
gates-ai-lab-2019-04-17_0	0.272
gates-basement-elevators-2019-01-17_0	0.311
gates-foyer-2019-01-17_0	0.499
gates-to-clark-2019-02-28_0	0.492
hewlett-class-2019-01-23_0	0.154
hewlett-class-2019-01-23_1	0.428
huang-2-2019-01-25_1	0.335
huang-intersection-2019-01-22_0	0.432
indoor-coupa-cafe-2019-02-06_0	0.118
lomita-serra-intersection-2019-01-30_0	0.554
meyer-green-2019-03-16_1	0.187
nvidia-aud-2019-01-25_0	0.23
nvidia-aud-2019-04-18_1	0.477
nvidia-aud-2019-04-18_2	0.332
outdoor-coupa-cafe-2019-02-06_0	0.163
quarry-road-2019-02-28_0	0.351
serra-street-2019-01-30_0	0.252
stlc-111-2019-04-19_1	0.434
stlc-111-2019-04-19_2	0.434
tressider-2019-03-16_2	0.231
tressider-2019-04-26_0	0.057
tressider-2019-04-26_1	0.046
tressider-2019-04-26_3	0.054
overall	0.175

Symbol	Description
Individual Image	Method uses individual images from each camera
Stitched Image	Method uses stitched images combined from the individual cameras
Pointcloud	Method uses 3D pointcloud data
Online Tracking	Method does frame-by-frame processing with no lookahead
Offline Tracking	Method does not do in-order frame processing
Public Detections	Method uses publicly available detections
Private Detections	Method uses its own private detections

Measure	Better	Perfect	Description
mAP1	higher	100%	mAP1 reports the average AP of groups with 1, 2, 3, 4 and more than 5 members and is used as the main metric for the social group detection challenge
mAP2	higher	100%	mAP2 reports the average AP of all groups.

Leaderboard

Instructions

Social Group Submissions

Additional Information Used

Evaluation Measures[1]

Reference