Open for Collaboration
More details

The Institute of Science Tokyo provides access to up to 128 NVIDIA H100 GPUs through TSUBAME4.0, with compute credits allocated by our lab for open-ended research and collaboration.

I am actively seeking close collaborations with both academia and industry. If this aligns with your interests, please feel free to reach out.

Hi, I’m Xuhan (盛 栩涵), a Ph.D. student in Artificial Intelligence in the Department of Computer Science at the Institute of Science Tokyo (formerly Tokyo Institute of Technology), advised by Prof. Rio Yokota. I received my Master’s degree from Peking University, advised by Prof. Jian ZHANG, and my Bachelor’s degree in Artificial Intelligence from Dalian University of Technology, advised by Prof. Xu JIA.

My research background includes diffusion models, memory-efficient training and inference, vision-language models, and video data pipelines. My previous research includes OmniSSR (ECCV 2024 Oral, co-first author) and panoramic image-to-video generation. I am currently a World Model Research Intern at Xiaomi Auto (Shanghai team).

I am exploring future research directions in video generation, AI agents, and large language models, and am seeking long-term research internships in these areas. I am available for on-site internships in China and Tokyo, Japan, and will also consider opportunities in the United States. Start date is flexible; working hours depend on local requirements and company policy. View my CV.

I am always open to academic and industry collaborations. Feel free to reach out.

🔥 News

  • 2026.10: 🎓 Started my Ph.D. in AI at Institute of Science Tokyo, advised by Prof. Rio Yokota.
  • 2026.08: 💼 Joined Xiaomi Auto as a research intern, developing video super-resolution for World Model 2.0.
  • 2026.03: 💼 Started my internship at Tencent IEG (Game AI Engine Department).
  • 2025.09: 🎉 Collaboration: 1 paper accepted in PRCV 2025.
  • 2025.08: 🎉 Collaboration: 1 paper accepted in IJCV 2025.
  • 2025.06: 🎉 Collaboration: 1 paper accepted in ICME 2025 for oral presentation.
  • 2025.05: 💼 Started my internship at OPPO Research Institute (AI Talent Program).
  • 2024.09: 🎉 First-author: 1 paper (“OmniSSR”) accepted in ECCV 2024 for oral presentation.
  • 2023.06: 🏆 Collaboration: 1 work won the “NTIRE 2023 Challenge on 360deg Omnidirectional Image Super-Resolution track” championship.

💼 Industry Experience

Xiaomi Auto, Shanghai Team

2026.08 - Present · World Model Research Intern

  • Contributing to Xiaomi Auto World Model 2.0 (forthcoming), with primary responsibility for video super-resolution.

Tencent, IEG, Game AI Engine Department

2026.03 - 2026.06 · Algorithm Intern

  • Qwen-Image-based image enhancement for game content and production-level visual assets.
  • Built data-pipeline operators for video generative models; developed a streaming shot-segmentation operator that splits long videos into single-shot clips.

OPPO Research Institute

2025.05 - 2025.09 · Imaging Algorithm Engineer Intern

Worked on vision-language models for assisting AI-based image enhancement. Advisor: Prof. Lei Zhang.

🔬 Research Interests

  • Interactive Long-Video Generation: Post-training and distillation for controllable video world models with long-horizon consistency.
  • 3D Spatial Memory for World Models: Explicit, updatable scene memory to preserve object identity, scene geometry, and occlusion consistency across camera motion and interactions.
  • World Models for Visual Reasoning: Modeling physical and causal dynamics to predict future states and evaluate action outcomes.

📑 Publications

(*: indicates equal contribution; #: indicates corresponding author)

Diffusion Models for Image & Video Generation and Restoration

Preprint
RealOSR overview
RealOSR: Latent Guidance Boosts Diffusion-based Real-world Omnidirectional Image Super-Resolutions
Xuhan Sheng*, Runyi Li*, Bin Chen, Weiqi Li, Xu Jiang, Jian Zhang#
Preprint
One-step diffusion for real-world 360° super-resolution with efficient latent-space condition guidance.
IJCV 2025
OmniDrag teaser
OmniDrag: Enabling Motion Control for Omnidirectional Image-to-Video Generation
Weiqi Li, Shijie Zhao, Chong Mou, Xuhan Sheng, et al.
IJCV 2025
First diffusion framework bringing precise drag-style motion control to omnidirectional image-to-video generation on the spherical domain.
ICME 2025 Oral
Re-Face overview
Label-guided Facial Retouching Reversion
Guanhua Zhao*, Yu Gu*, Xuhan Sheng, Yujie Hu, Jian Zhang#
ICME 2025 Oral Presentation
Label-guided diffusion that reverses skin smoothing, eye enlargement and face slimming, recovering authentic portraits with identity preserved.
ECCV 2024 Oral
OmniSSR overview
OmniSSR: Zero-shot Omnidirectional Image Super-Resolution using Stable Diffusion Model
Runyi Li*, Xuhan Sheng*, Weiqi Li, Jian Zhang#
ECCV 2024 Oral Presentation
First zero-shot 360° super-resolution leveraging Stable Diffusion priors via tangent-plane interaction and gradient-based decomposition, no training needed.
CVPRW 2023
NTIRE overview
OPDN: Omnidirectional Position-Aware Deformable Network for Omnidirectional Image Super-Resolution
Xiaopeng Sun*, Weiqi Li*, Zhenyu Zhang, Qiufang Ma, Xuhan Sheng, et al.
CVPR Workshops 2023 NTIRE 2023 360° Panoramic SR Challenge — Champion
Position-aware deformable network tackling ERP geometric distortions; champion of the NTIRE 2023 360° panoramic SR challenge.

Video Understanding

PRCV 2025
PBHL pipeline
Bridging the Point to Boundary Gap for Point-supervised Temporal Action Localization with Single-stage Inference
Junshi Yang, Shenglan Liu, Xuhan Sheng, et al.
PRCV 2025
Hierarchical point-to-boundary framework converting sparse point labels into boundary-accurate pseudo labels via Gaussian-prior boundary enhancement.

🎓 Educations

Institute of Science Tokyo, Department of Computer Science

2026.10 - Present · Ph.D. Student in Artificial Intelligence (earliest graduation: Sept. 2029; option to graduate in Apr. 2030)

Advisor: Prof. Rio Yokota

Peking University, VILLA Lab

2023.09 - 2026.06 · Master Student

Advisor: Prof. Jian ZHANG; GPA: 3.87/4.00

Dalian University of Technology

2019.09 - 2023.06 · Bachelor Student

Advisor: Prof. Xu JIA; GPA: 4.17/5.00

🏅 Honors and Awards

  • 2024, Peking University Academic Excellence Award.
  • 2023, Dalian University of Technology University-level Outstanding Graduate.
  • 2022, Dalian University of Technology First-class Academic Excellence Scholarship.
  • 2021, Dalian University of Technology Second-class Academic Excellence Scholarship.
  • 2020, Dalian University of Technology First-class Academic Excellence Scholarship.

😊 Hobbies

Click image to zoom.

I love playing flutes (Irish flute, Boehm flute, Shakuhachi). I love Hatsune Miku. I am learning Cantonese.