Work Done
- Determined a way to measure feature granularity / overlap as we go down
- Separated and cleaned up embedding generation
- Claude Project Feature Granularity
Confusions
- What does each metric I made mean?
- Do they actually measure how broad each concept becomes
Next Steps
-
Find the most similar vectors from one layer to the other and see what happens when you apply them??
-
Separate out results as well because some datasets are overpowering others.
- Analyze results and see if I can come to a conclusion
- If so then try to invalidate those results
- Try other methods of validation as well