Intro

I am currently a Professor in Intelligent Media Analysis Group (IMAG), at School of Computer Science and Engineering, Nanjing University of Science and Technology, China. From Mar. 2023 to Apr. 2025, I was an Assistant Researcher of Department of Computer Science and Technology at Nanjing University and working with Tieniu Tan. I obtained Ph.D. degree from Nanjing University of Science and Technolog, under the supervision of Prof. Jinhui Tang, in Nov. 2022. From Jan. 2022 to Aug. 2022, I worked as a Research Intern (Part-time) at ByteDance. From Sep. 2021 to Dec. 2021, I worked as a Research Intern (Part-time) at Tencent with Yixiao Ge. From Dec. 2018 to Dec. 2019, I worked as a Research Intern at HUAWEI NOAH'S ARK LAB with Lingxi Xie and Prof. Qi Tian (IEEE Fellow). I am working closely with Mike Shou and Xiangbo Shu. My research mainly focus on Video Understanding and Multimodal Understanding.

Researh Interests

Video Understanding, Human Behavior Analysis, Embodied Intelligence and other related human-centric problems in Artificial Intelligence, Computer Vision and Multimedia.


*Positions for Interns/Master/PhD's Programme*
We are looking for students, who are self-motivated and have a solid foundation in mathematics and programming. If you are interested, please feel free to contact us!.

News

  • 2025.12: Two paper is accepted by IEEE TCSVT.
  • 2025.9: One paper is accepted by NeurIPS 2025.
  • 2025.7: One paper is accepted by ICCV 2025.
  • 2025.4: Two papers is accepted by IJCAI 2025.
  • 2025.1: One paper is accepted by ICLR 2025.
  • 2024.12: One paper is accepted by IEEE TMM.
  • 2024.7: Two papers is accepted by ACM MM 2024.
  • 2024.7: One paper is accepted by IEEE TMM.
  • 2024.4: One paper is accepted by IEEE TCSVT.
  • 2024.2: One paper is accepted by IJCAI 2024.
  • 2023.12: One paper is accepted by IEEE TIP.
  • 2023.6: Four paper is accepted by ACM MM 2023.
  • 2023.6: One paper is accepted by ICCV 2023.
  • 2023.5: One paper is accepted by IEEE TCSVT.
  • 2023.3: One paper is accepted by IEEE TPAMI.
  • 2023.2: One paper is accepted by CVPR 2023.
  • 2022.11: One paper is accepted by AAAI 2023.
  • 2022.09: One paper is accepted by NeurIPS 2022.
  • 2022.07: One paper is accepted by ACM MM 2022.
  • 2022.06: Our team achieves the First Place Award in Object State Change Classification Track, the Second Place Award in Natural Language Queries for Episodic Memory Track, and the Third Place Award in PNR Temporal Localization Track of EGO4D Challenge (CVPR 2022).
  • 2022.06: Our team achieves the First Place Award in Multi-Instance Action Retrieval Track of EPIC-Kitchens Dataset Challenges (CVPR 2022).
  • 2022.03: Two papers are accepted by CVPR 2022.
  • 2022.01: One paper is accepted by IEEE TCSVT.
  • 2021.12: I give a talk about Video-Language Pre-training at PCG, Tencent.
  • 2021.08: I will work with Prof. Mike Shou at Show Lab, National University of Singapore.
  • 2020.05.19: One paper is accepted by IEEE TNNLS.
  • 2020.12.08: One paper is accepted by ACM MM Asia 2020.
  • 2020.10.20: One paper is accepted by IEEE TPAMI.
  • 2020.07.03: One paper is accepted by ECCV 2020.
  • 2020.05: Selected as the Outstanding PhD of NJUST.
  • 2019.05: I give a talk about GAR at the Noah’s Ark Lab, Huawei Inc.

Selected Publications

Vision-centric Token Compression in Large Language Model
Ling Xing, Alex Jinpeng Wang, Rui Yan*, Xiangbo Shu, Jinhui Tang
NeurIPS (Spotlight), 2025 [PDF] [新智元]

V-MAGE: A Game Evaluation Framework for Assessing Visual-Centric Capabilities in Multimodal Large Language Models
Xiangxi Zheng, Linjie Li, Zhengyuan Yang, Ping Yu, Alex Jinpeng Wang, Rui Yan, Yuan Yao, Lijuan Wang
arxiv, 2025 [PDF]

TEST-V: TEst-time Support-set Tuning for Zero-shot Video Classification
Rui Yan, Jin Wang, Hongyu Qu, Xiaoyu Du, Dong Zhang, Jinhui Tang, Tieniu Tan
IJCAI, 2025 [PDF]

DTS-TPT: Dual Temporal-Sync Test-time Prompt Tuning for Zero-shot Activity Recognition
Rui Yan, Hongyu Qu, Xiangbo Shu, Wenbin Li, Jinhui Tang, Tieniu Tan
IJCAI, 2024 [PDF]

Progressive Instance-aware Feature Learning for Compositional Action Recognition
Rui Yan, Lingxi Xie, Xiangbo Shu, Liyan Zhang, and Jinhui Tang
TPAMI, 2023 [PDF][Code]

All in One: Exploring Unified Video-language Pre-training
Jinpeng Wang, Yixiao Ge, Rui Yan, Xudong Lin, Guanyu Cai, Jianping Wu, Ying Shan, Xiaohu Qie, Mike Zheng Shou
CVPR 2023 [PDF][Code]

Video-Text Pre-training with Learned Regions for Retrieval
Rui Yan, Mike Zheng Shou, Yixiao Ge, Alex Jinpeng Wang, Xudong Lin, Guanyu Cai, and Jinhui Tang
AAAI 2023 [PDF][Code]

Egocentric Video-Language Pretraining
Kevin Qinghong Lin, Alex Jinpeng Wang, Mattia Soldan, Michael Wray, Rui Yan, Eric Zhongcong Xu, Difei Gao, Rongcheng Tu, Wenzhe Zhao, Weijie Kong, Chengfei Cai, Hongfa Wang, Dima Damen, Bernard Ghanem, Wei Liu, Mike Zheng Shou
NeurIPS 2022 (Spotlight) [PDF][Code]

Look Less Think More: Rethinking Compositional Action Recognition
Rui Yan, Peng Huang, Xiangbo Shu, Junhao Zhang, Yonghua Pan, Jinhui Tang
ACM MM 2022 [PDF][Split]

Dual-AI: Dual-path Actor Interaction Learning for Group Activity Recognition
Mingfei Han, David Junhao Zhang, Yali Wang, Rui Yan, Lina Yao, Xiaojun Chang, and Yu Qiao
CVPR 2022 (Oral) [PDF]

HiGCIN: Hierarchical Graph-based Cross Inference Network for Group Activity Recognition
Rui Yan, Lingxi Xie, Jinhui Tang, Xiangbo Shu, and Qi Tian
TPAMI, 2020 [PDF][Code]

Adaptive Module for Weakly-supervised Group Activity Recognition
Rui Yan, Lingxi Xie, Jinhui Tang, Xiangbo Shu, and Qi Tian
ECCV 2020 [PDF][Project][Code]

Coherence Constrained Graph LSTM for Group Activity Recognition
Jinhui Tang, Xiangbo Shu, Rui Yan, and Liyan Zhang
TPAMI, 2019

Participation-Contributed Temporal Dynamic Model for Group Activity Recognition
Rui Yan, Jinhui Tang, Xiangbo Shu, Zechao Li and Qi Tian
ACM MM 2018 (Oral) ~8.5% (Journal version is accepted by TNNLS)
[PDF][Code][Slides]

For more papers, please kindly refer to my Google Scholar page

Honors and Awards

  • 江苏省科学技术一等奖(排4),江苏省政府,2024
  • 中国图象图形学学会(CSIG)-优博,中国图象图形学学会,2024
  • 江苏省青年科技人才托举工程, 江苏省科协,2024
  • 江苏省计算机学会-优博,江苏省计算机学会,2024
  • 校优秀博士论文,南京理工大学,2024
  • 国家资助博士后,中国博士后科学基金会,2023
  • 中国博士后科学基金特别资助,中国博士后科学基金会,2023
  • 江苏省卓越博士后,江苏省人社厅,2023
  • 新城市教育基金-后备学科带头人奖,南京理工大学,2026
  • 毓秀青年学者,南京大学,2023

Grants

  • 国家自然科学基金面上项目, 2025.1-2028.12
  • 国家自然科学基金青年科学基金项目, 2024.1-2024.12
  • 国家资助博士后项目, 2023-2025
  • 中国博士后科学基金第73批面上资助, 2023-2025
  • 南京理工大学科研启动经费, 2025
  • 中央高校基本科研业务费-揭榜挂帅, 2023/2024

Academic Service

  • 中国图像图形学会多媒体专业委员会委员(CSIG-MM)
  • 中国计算机学会计算机视觉专业委员会委员(CCF-CV)
  • TPC for CVPR/ICCV/ECCV/NeurIPS/AAAI/IJCAI/MM, TPAMI/TIP/TNNLS/TIFS/TMM/TCSVT/TOMM/PR/Neurocomputing/Information Sciences.