Expansion factor same on logic vs. math?
I'm curious and a bit suspicious how would it come to be both the logic and math SAE layers have the the same expansion factor in each their configs while simultaneosly math .pt file is 411MB and logic is 822MB, seems like the second would span twice the neurons/features..?
"expansion_factor": 4
https://huggingface.co/Goodfire/DeepSeek-R1-SAE-l37/blob/main/logic/config.json
k i suppose is the sparsity targeted so that would not influence the size, as I understand it.
hey! it is the same expansion factor - if you load them up, the dims should be the same for both. due to a slightly different training process, the logic SAE was trained and saved in float32, while the math SAE was trained and saved in bfloat16 - hence the 2x size difference. lmk if you have any issues with them!