About
I am a Doctoral Fellow at the ETH AI Center. My research lies at the intersection of natural language processing and machine learning. I focus on using formal methods—such as formal logic, formal language theory, and circuit complexity—to both analyze and interpret modern deep learning models, and to guide the development of new architectures and algorithms that draw inspiration from these formal perspectives.
I am fortunate to be advised by Ryan Cotterell and co-advised by Mrinmaya Sachan.
Selected Publications
- Characterizing the Expressivity of Fixed-Precision Transformer Language Models
- Unique Hard Attention: A Tale of Two Sides
- What Do Language Models Learn in Context? The Structured Task Hypothesis
- A Transformer with Stack Attention
- Towards a Mechanistic Interpretation of Multi-Step Reasoning Capabilities of Language Models
- Probing via Prompting
- Differentiable Subset Pruning of Transformer Heads
- Vision Matters When It Should: Sanity Checking Multimodal Machine Translation Models
Education
-
Ph.D. in Computer Science, ETH Zürich2021 - 2025 (expected)
-
Master's in Data Science, ETH Zürich2018 - 2021
-
Bachelor's in Electronic and Communication Engineering, City University of Hong Kong2014 - 2018