Alexandre Marthe

Alexandre Marthe

Recently defended my PhD on Risk-Sensitive RL. Interested in LLM post-training and RL.

About Me

I recently defended my PhD which I did under the supervision of Aurelien Garivier and Claire Vernade at ENS de Lyon, in France. You can find here my Phd Thesis and the slides for my Oral Defense.

My research focused on understanding the link between the Distributional Perspective in MDPs (see Distributional Reinforcement Learning, Bellemare et al. 2023), and Dynamic Programming with Risk Measures. Here is a wonderful talk from Claire Vernade presenting one of our papers.

I am currently looking for opportunities in the industry, feel free to contact me if you have offers. My CV can be found Here.

Selected Publications :

  • Beyond Average Return in Markov Decision Processes
    Alexandre Marthe, Aurélien Garivier, Claire Vernade. NeurIPS 2023 [Link]

  • Risk-Sensitive Planning via Entropic Risk Measures
    Alexandre Marthe, Samuel Bounan, Aurélien Garivier, Claire Vernade. EWRL 2025, (under review for publication) [Link]

PhD Thesis :

  • Risk-Sensitive Planning in Markov Decision Processes: Distributional Perspective and Entropic Front
    Defended before a thesis committee composed by Aurélien Garivier (supervisor), Claire Vernade (co-supervisor), Vianney Perchet (reviewer), Marc G. Bellemare (reviewer) and Rémi Munos (examiner) [link].

Personal Project: Implementing LLMs from Scratch

I am currently exploring the development of small-scale LLMs by implementing the entire process from scratch:

  • Training a small Language Model (~17M parameters) on the TinyStories dataset:
    • Implemented a BPE tokenizer, all Transformer modules, AdamW, and the full training pipeline from scratch in PyTorch.
    • Conducted ablation studies on several components to better understand their importance.
  • Post-training OLMo-2-0425-1B with RLVR on the GSM8K dataset:
    • Implemented the pipeline and several RL algorithm variants: GRPO (both online and offline), Dr. GRPO, RFT, MaxRL, and GSPO.
  • Currently implementing an RLHF pipeline.
  • Coming soon: implementing distillation pipelines and writing GPU kernels.

Resources used: Stanford’s CS336 lectures and handouts, Andrej Karpathy’s Youtube videos, Sebastian Raschka’s Youtube Playlist

Extracurricular

In my spare time, I like sports. I am involved in a competitive Cheerleading club (Reign, from Villeurbanne). With my team, I won the bronze medal at the 2025 French National Cheerleading Championship and we recently qualified for the 2026 European Championship. See here our latest performance.

I am also into games, mainly card games. I was deeply involved in the Competitive scene of the Trading Card Game Yu-Gi-Oh!. As part of the Judge Program, I supervised a dozen of competition at the highest level of international competitive play, with several thousand players from all around the world. I was even selected to Judge at the very selective 2025 World Championship in Paris.

Wanna talk about any of the subjects mentioned above? I’ll be more than pleased to discuss it with you.

Contact

Email : firstname.lastname@ens-lyon.fr