Research Question: How do neural network weights evolve when neurons transition from monosemantic to polysemantic representations?

Hypotheses

1. Weight Distribution: As neurons transition to polysemantic representations, their incoming weights become more distributed (less sparse).

2. Success Metric: Higher weight entropy in polysemantic neurons compared to monosemantic neurons.

3. Weight Magnitude: The magnitude of weight vectors increases to accommodate multiple feature representations.

4. Success Metric: Larger L2 norm of weight vectors in polysemantic neurons.

5. Angular Distance: The angular distance between weight vectors of different neurons decreases as they begin to capture overlapping concepts.

6. Success Metric: Decreased mean angular distance between neuron weight vectors in Phase 2.

Work Done

Phase1 Plots

Phase2 Plots

Confusions

• The graphs for phase 2 are all over the place.

Next Steps

• Inspect the graphs that have the odd distance changes

Relevant Literature

• Polysemanticity vs Superposition
• Engineering Monosemanticity