MI Workshop Orals and Poster Sessions

Poster Session 1: 11:00 AM - 12:00 PM

Spotlighted papers assigned to poster session 1 will be presented between 10:30 AM and 11:00 AM, immediately before the poster session.

Comgra: A Tool for Analyzing and Debugging Neural Networks (Spotlight)
Identifying Functionally Important Features with End-to-End Sparse Dictionary Learning (Spotlight)
Crafting Large Language Models for Enhanced Interpretability
Learning Syntax Without Planting Trees: Understanding When and Why Transformers Generalize Hierarchically (Spotlight)
Mechanistic Interpretability of Binary and Ternary Transformer Networks
How Truncating Weights Improves Reasoning in Language Models
Does Editing Provide Evidence for Localization?
Modularity in Biologically Inspired Representations Depends on Task Variable Range Independence
Compact Proofs of Model Performance via Mechanistic Interpretability (Spotlight)
Dissecting Query-Key Interaction in Vision Transformers (Spotlight)
How Do Transformers Fill in the Blanks? A Case Study on Matrix Completion
Penzai + Treescope: A Toolkit for Interpreting, Visualizing, and Editing Models As Data (Spotlight)
Localizing Auditory Concepts in CNNs
TracrBench: Generating Interpretability Testbeds with Large Language Models
Transcoders find interpretable LLM feature circuits (Spotlight)
Learning to grok: Emergence of in-context learning and skill composition in modular arithmetic tasks (Spotlight)
Hypothesis Testing the Circuit Hypothesis in LLMs (Oral)
Iteration Head: A Mechanistic Study of Chain-of-Thought
How do Llamas process multilingual text? A latent exploration through activation patching (Spotlight)
Contrastive Sparse Autoencoders for Interpreting Planning of Chess-Playing Agents
Benchmarking Mental State Representations in Language Models
Investigating the Indirect Object Identification circuit in Mamba
Adversarial Circuit Evaluation
The Geometry of Categorical and Hierarchical Concepts in Large Language Models (Oral)
On the Similarity of Circuits across Languages: a Case Study on the Subject-verb Agreement Task
Segmentation CNNs are denoising models
Refusal in Language Models Is Mediated by a Single Direction
Why do recurrent neural networks suddenly learn? Bifurcation mechanisms in neuro-inspired short-term memory tasks
CoSy: Evaluating Textual Explanations of Neurons
Transformers on Markov data: Constant depth suffices
InversionView: A General-Purpose Method for Reading Information from Neural Activations (Oral)
Do LLMs dream of elephants (when told not to)? Latent concept association and associative memory in transformers
Relational Composition in Neural Networks: A Survey and Call to Action (Spotlight)
Missed Causes and Ambiguous Effects: Counterfactuals Pose Challenges for Interpreting Neural Networks (Oral)
Understanding Inhibition through Maximally Tense Images
Interpretability in Action: Exploratory Analysis of VPT, a Minecraft Agent
Loss in the Crowd: Hidden Breakthroughs in Language Model Training (Spotlight)
InterpBench: Semi-Synthetic Transformers for Evaluating Mechanistic Interpretability Techniques
Language Models Linearly Represent Sentiment (Spotlight)
Have Faith in Faithfulness: Going Beyond Circuit Overlap When Finding Model Mechanisms (Spotlight)
Improving Sparse Decomposition of Language Model Activations with Gated Sparse Autoencoders
Extracting Finite State Machines from Transformers
Planning behavior in a recurrent neural network that plays Sokoban
Exploring the Internal Mechanisms of Music LLMs: A Study of Root and Quality via Probing and Intervention Techniques
Grokking and the Geometry of Circuit Formation
Attention with Markov: A Curious Case of Single-layer Transformers

Mechanistic Interpretability Workshop 2024

Orals and Poster Sessions

July 27, 2024

Oral Presentations: 9:30 AM - 10:30 AM

Poster Session 1: 11:00 AM - 12:00 PM

Poster Session 2: 2:30 PM - 3:30 PM