Hohei Chan

Hohei Chan

MS student at SCUT and intern with the Kling Team at Kuaishou. Research: reinforcement learning, agents, and multimodal generation.

Ad Hoc Teamwork via Offline Goal-Based Decision Transformers

Published in Proceedings of the 42nd International Conference on Machine Learning, 2025

In offline multi-agent collaboration scenarios, we propose a transformer-based hierarchical sequence decision-making framework that enables rapid adaptation to unknown teammate agents.

Related links:

PDF 
@inproceedings{zhang2025ad,
  title={Ad Hoc Teamwork via Offline Goal-Based Decision Transformers},
  author={Zhang, Xinzhi and Chan, Hohei and Ye, Deheng and Cai, Yi and Zhao, Mengchen},
  booktitle={Forty-second International Conference on Machine Learning},
  year={2025}
}

The ability of agents to collaborate with previously unknown teammates on the fly, known as ad hoc teamwork (AHT), is crucial in many real-world applications. Existing approaches to AHT require online interactions with the environment and some carefully designed teammates. However, these prerequisites can be infeasible in practice. In this work, we extend the AHT problem to the offline setting, where the policy of the ego agent is directly learned from a multi-agent interaction dataset. We propose a hierarchical sequence modeling framework called TAGET that addresses critical challenges in the offline setting, including limited data, partial observability and online adaptation. The core idea of TAGET is to dynamically predict teammate-aware rewards-to-go and sub-goals, so that the ego agent can adapt to the changes of teammates’ behaviors in real time. Extensive experimental results show that TAGET significantly outperforms existing solutions to AHT in the offline setting.