Independent interpretability research
Sentient reads the internal structure of neural networks: the features, circuits, and directions that fix an output before a single token is produced. We treat a trained network as an object to be measured, not a product to be praised.
Everything here is a claim we can defend with an intervention. If ablating a component does not change the behaviour it is said to cause, we retract the claim.
Cloud, lattice, word. A 40,000-point activation field resolving under scroll.
Research / 01
Paper 01
Dictionary learning on residual stream activations yields a basis of 32,768 features, of which 71.2 percent are judged monosemantic by held-out human raters. Feature density falls off as a power law in layer depth. We release the trained dictionary, the activation cache, and the rater protocol so the fit can be reproduced without our compute.
Fig. 1 Fraction monosemantic against dictionary width
Paper 02
A small set of heads in the middle layers predicts the depth of a token in its dependency parse. The same heads carry depth information in five languages the model was not explicitly aligned on, with a mean rank correlation of 0.63. Ablating the three strongest heads removes 62 percent of subject verb agreement in a controlled probe.
Fig. 2 Depth correlation by layer
Paper 03
Whether the model follows or declines an instruction is well predicted by the projection of its activations onto a rank-four subspace. Steering along this subspace flips compliance in 88 percent of held-out cases without measurable loss on unrelated benchmarks. We report the failure cases in full, including four prompts where the intervention had no effect.
Fig. 3 Compliance against projection magnitude
Probe / 02
Illustrative. Weights are precomputed for this figure and do not read from a live model. Hover or focus a word to see where it looks.
Method / 03
Run a fixed probe set through the model and cache activations at every layer. The probe set is versioned and public, so a finding is tied to inputs anyone can rerun.
Fit a sparse dictionary to the cached activations. The dictionary turns a dense, entangled vector into a short list of named features that fire or stay silent.
Trace a feature back to the circuit that computes it using path patching. Attribution is a hypothesis about mechanism, not a heat map to admire.
Ablate and steer. If a component is claimed to cause a behaviour, removing it must change that behaviour and leave the rest intact. This is the step that can falsify us.
Publish weights, code, the probe set, and the negative results. A method that cannot be rerun by a stranger is an anecdote, and we do not shelve it as a finding.
People / 04