Seungjae (Jay) Lee

I am a Ph.D. student in the Department of Computer Science 💻 at UMD, where I am fortunate to be co-advised by Prof. Furong Huang and Prof. Jia-Bin Huang.

Prior to joining UMD, I earned my Master's degree in the Department of Aerospace Engineering ✈️ at SNU, where I was fortunate to be advised by Prof. H. Jin Kim. I also had the wonderful opportunity to work on robot learning 🤖 with Prof. Lerrel Pinto of NYU.

Before that, I received Bachelor's degrees in Mechanical and Aerospace Engineering at SNU ⚙️.

"💻 + ✈️ + 🤖 + ⚙️ = Me"

profile photo

News

Education & Experiences

Education

Ph.D. in Computer Science
University of Maryland

Advised by Professor Furong Huang and Professor Jia-Bin Huang.

Aug 2024 - Present | College Park, MD

M.S. in Aerospace Engineering
Seoul National University

Advised by Professor H. Jin Kim.

Mar 2021 - Feb 2024 | Seoul, Korea

B.S. in Mechanical & Aerospace Engineering
Seoul National University

Mar 2015 - Feb 2021 | Seoul, Korea

Experiences

NVIDIA

GRAR Team Research Intern (Upcoming)

May 2026 - | Santa Clara, CA

Toyota Research Institute

Large Behavior Model Team Research Intern

May 2025 - Aug 2025 | Cambridge, MA

New York University

Research collaborator (Remote), advised by Professor Lerrel Pinto.

Jul 2023 - Jun 2024 | Seoul, Korea (Remote)

Samsung Electronics

Deep Learning Algorithm Team Intern

Jul 2020 - Sep 2020 | Gyunggi-do, Korea

Research

My research interest is understanding the interaction between agents and environments, and devising data-efficient decision-making (or robot learning) algorithms, especially in the field of reinforcement learning (RL). Selected publications are marked with ★ below.

TraceGen: World Modeling in 3D Trace-Space Enables Learning from Cross-Embodiment Videos
Seungjae Lee*, Yoonkyo Jung*, Inkook Chun*, Yao-Chih Lee, Zikui Cai, Hongjia Huang, Aayush Talreja, Tan Dat Dao, Yongyuan Liang, Jia-Bin Huang, Furong Huang (*equal contribution)
CVPR, 2026
project website / arXiv / github

MomaGraph: State-Aware Unified Scene Graphs with Vision-Language Model for Embodied Task Planning
Yuanchen Ju, Yongyuan Liang, Yen-Jen Wang, Nandiraju Gireesh, Yuanliang Ju, Seungjae Lee, Qiao Gu, Elvis Hsieh, Furong Huang, Koushil Sreenath
ICLR, 2026
project website / arXiv

Dynamic Test-Time Compute Scaling in Control Policy: Difficulty-Aware Stochastic Interpolant Policy
Inkook Chun, Seungjae Lee, Michael Samuel Albergo, Saining Xie, Eric Vanden-Eijnden
NeurIPS, 2025
arXiv / github

Imagine, Verify, Execute: Memory-guided Agentic Exploration with Vision-Language Models
Seungjae Lee*, Daniel Ekpo*, Haowen Liu, Furong Huang†, Abhinav Shrivastava†, Jia-Bin Huang† (*equal contribution, †equal advising)
CoRL, 2025
project website / arXiv

Why Are Web AI Agents More Vulnerable Than Standalone LLMs? A Security Analysis
Jeffrey Yang Fan Chiang*, Seungjae Lee*, Jia-Bin Huang, Furong Huang, Yizheng Chen (*equal contribution)
ICLRw, 2025
project website / arXiv

Robot Utility Models: General Policies for Zero-Shot Deployment in New Environments
Haritheja Etukuru, Norihito Naka, Zijin Hu, Seungjae Lee, Julian Mehu, Aaron Edsinger, Chris Paxton, Soumith Chintala, Lerrel Pinto, Nur Muhammad Mahi Shafiullah
ICRA, 2025
project website/ paper/ github
CoRL 2024 Workshop on Language and Robot Learning (🏆 Oral)

Behavior Generation with Latent Actions
Seungjae Lee, Yibin Wang, Haritheja Etukuru, H. Jin Kim, Nur Muhammad Mahi Shafiullah, Lerrel Pinto
ICML, 2024 (🏆 Spotlight, Top 3.5%)
project website / arXiv / github / 🤗 Lerobot Library
RSS 2024 Workshop SemRob (🏆 Oral spotlights) ICML 2024 Workshop MFM-EAI (🏆️ Outstanding Paper Award - Winner)

CQM: Curriculum Reinforcement Learning with a Quantized World Model
Seungjae Lee, Daesol Cho, Jonghae Park, H Jin Kim
NeurIPS, 2023
arXiv

Diversify & Conquer: Outcome-directed Curriculum RL via Out-of-Distribution Disagreement
Daesol Cho, Seungjae Lee, H Jin Kim
NeurIPS, 2023
arXiv

SNeRL: Semantic-aware Neural Radiance Fields for Reinforcement Learning
Dongseok Shim*, Seungjae Lee*, H Jin Kim (*equal contribution)
ICML, 2023
arXiv / github

Outcome-directed Reinforcement Learning by Uncertainty & Temporal Distance-Aware Curriculum Goal Generation
Daesol Cho*, Seungjae Lee*, H Jin Kim (*equal contribution)
ICLR, 2023 (🏆 Spotlight, Top 5.65%)
arXiv / github

Deep End-to-End Imitation Learning for Missile Guidance with Infrared Images
Seungjae Lee, Jongho Shin, Hyeong-Geun Kim, Daesol Cho, H. Jin Kim
IJCAS, 2023

DHRL: A Graph-Based Approach for Long-Horizon and Sparse Hierarchical Reinforcement Learning
Seungjae Lee, Jigang Kim, Inkyu Jang, H. Jin Kim
NeurIPS, 2022 (🏆 Oral, Top 1.76%)
arXiv / github

Robust and Recursively Feasible Real-Time Trajectory Planning in Unknown Environments
Inkyu Jang, Dongjae Lee, Seungjae Lee, H Jin Kim

IROS, 2021 arXiv

Awards and Achievements

Academic Services

Teaching