Samuel Garcin

S[DOT]GARCIN[AT]ED.AC.UK

I am a PhD candidate and a member of the Autonomous Agents Research Group at the University of Edinburgh. I am fortunate to be co-advised by Stefano Albrecht and Chris Lucas. Before starting my PhD I completed an MEng in Aeronautics at Imperial College London, where I started the Imperial College Aerial Vehicle project, and then worked three years in the aerial robotics industry. I have been previously involved in multi-agent robotic navigation and autonomous vehicles research projects.

My PhD investigates how to understand the generalisation capabilities of deep reinforcement learning algorithms through the study of the flow of information passing through the model over the course of training. In reinforcement learning, the training data is generated online as the agent interacts with its environment, and we must consider this data generation process when studying this flow.

When the agent is trained in simulation, this process is not solely determined by the agent policy but also by the environment instance selected in the simulator. I’ve recently been investigating how novel forms of information bottlenecks may be induced by adaptively sampling from a set of environment instances or augmenting this set using a generative model. As such, my work has connections to the fields of generative modeling, environment design and autocurricula generation.

I am always interested in collaboration opportunities so feel free to reach out via email or through any of the channels listed at the bottom of this page! I am currently on the market for research internships.

I organise our group’s virtual Reinforcement Learning Reading Group, do reach out if you would like to come present your work. If you are in Edinburgh, I also organise an in-person discussion group on the topic of generalisation in machine learning.

news

Dec 8, 2023	I’ll be presenting my most recent project at the NeurIPS ALOE workshop in New Orleans this week.

selected publications

Preprint
How the level sampling process impacts zero-shot generalisation in deep reinforcement learning

Samuel Garcin, James Doran, Shangmin Guo, and 2 more authors

Under review, 2024

Abs arXiv Bib Poster

A key limitation preventing the wider adoption of autonomous agents trained via deep reinforcement learning (RL) is their limited ability to generalise to new environments, even when these share similar characteristics with environments encountered during training. In this work, we investigate how a non-uniform sampling strategy of individual environment instances, or levels, affects the zero-shot generalisation (ZSG) ability of RL agents, considering two failure modes: overfitting and over-generalisation. As a first step, we measure the mutual information (MI) between the agent’s internal representation and the set of training levels, which we find to be well-correlated to instance overfitting. In contrast to uniform sampling, adaptive sampling strategies prioritising levels based on their value loss are more effective at maintaining lower MI, which provides a novel theoretical justification for this class of techniques. We then turn our attention to unsupervised environment design (UED) methods, which adaptively generate new training levels and minimise MI more effectively than methods sampling from a fixed set. However, we find UED methods significantly shift the training distribution, resulting in over-generalisation and worse ZSG performance over the distribution of interest. To prevent both instance overfitting and over-generalisation, we introduce self-supervised environment design (SSED). SSED generates levels using a variational autoencoder, effectively reducing MI while minimising the shift with the distribution of interest, and leads to statistically significant improvements in ZSG over fixed-set level sampling strategies and UED methods.
@article{garcin2023level, title = {How the level sampling process impacts zero-shot generalisation in deep reinforcement learning}, author = {Garcin, Samuel and Doran, James and Guo, Shangmin and Lucas, Christopher G and Albrecht, Stefano V}, journal = {Under review}, year = {2024}, }