On this page
Many chat models say no to some requests, and that no may sit in one place. In a toy network of 100 lines of Python, one direction in its numbers carries the refusal. All 100 flagged requests, invented stand-ins for ones a model should decline, are refused. Take that direction out and none is, while 99 of 100 plain answers stay the same. A published study of 13 open chat models, up to 72 billion parameters, reports refusal mediated by a single direction across all of them. A model’s refusals are a thin layer, and an app that rests its safety on them alone rests on something that can be taken away.
In one minute
Picture a map where every request is a dot. Plain dots sit near the middle. Flagged dots sit far to the north. The model’s refusal is one small habit: when a dot is far enough north, say no. Find the north arrow by averaging the flagged dots, averaging the plain dots and subtracting. Then flatten the map along that arrow: every dot’s north-south position becomes 0, and its east-west position stays. The habit never fires again, so nothing is refused, and what the model knows from east and west is untouched.
The toy’s map has 32 directions, not 2, and a real model’s has many more. Two cautions. The toy has a refusal planted along one direction, so that we can check the arrow it finds; nobody plants one in a real model. And one arrow is not always the whole story, which the numbers below show.
Explore the numbers
Open the widget. It opens on the toy’s default: all of the direction taken out, read after layer 5 of 6. The sentence on top says 0 of 100 flagged requests are refused, down from 100, and 1 of 100 plain answers changes. Drag Share taken out to 20%: 44 flagged requests are still refused, so the refusals fall in a narrow band, between 10% and 30%. Drag Read after layer to 0: the direction is found 37.9 degrees off the planted one, 13 flagged requests are still refused and 8 plain answers change. Under More settings, Second route set to 30% leaves 29 refused, though all of the direction is out.
How it works
The picture, step by step
The network passes a vector of 32 numbers through 6 layers, and each layer adds its change to it. In every layer a refusal unit switches on when the vector holds enough of one planted direction, and adds more of it. The last step reads that part as “refuse”. Here are the six steps, as Figure 1 shows the result of steps 2 to 4. A flagged request is a made-up vector with a push along the planted direction; a plain one has none.
Figure 1. Held-out requests along the found direction, before and after it is taken out.
- Read the vectors. Run 64 flagged and 64 plain requests through the network and keep each one’s 32 numbers after layer 5.
- Subtract the averages. The average flagged vector minus the average plain vector, scaled to length 1, is the refusal direction. It lands 8.4 degrees from the planted one.
- Check it on new requests. Along it, 100 flagged requests the step never saw sit at 8.1 on average and 100 plain ones at -0.1. One cut at 4.0 sorts all 200.
- Take it out at every layer. Subtract each vector’s part along it, 7 times: once before the first layer and after each of the 6. This is directional ablation.
- Write the same edit into the weights. Cut that part from every matrix that writes into the vector, once (orthogonalization). The answers do not change.
- Count. Flagged requests refused: 100 before, 0 after. Plain requests whose answer changes: 1 of 100.
Each step in code
One real line of the program does each step. The files are in code/, and the line numbers are those of this lesson’s files.
| Step | The code line | What it does |
|---|---|---|
| 1. Keep the vectors | refusal_toy.py:45points.append(x) |
Each layer’s vector is kept, so the direction can be read at any layer. |
| 2. Subtract the averages | refusal_toy.py:51d = flagged_points.mean(0) - plain_points.mean(0) |
The next line scales it to length 1. |
| 3. Check along it | trace.py:14flagged_along = base["test_flagged"][1][LAYER] @ u |
One number per held-out request: how far along the direction it sits. |
| 4. Take it out | refusal_toy.py:57return lambda x: x - share * np.outer(x @ u, u) |
Subtract the part along u from each vector. |
| 5. Cut it from the weights | refusal_toy.py:62cut = lambda W: W - np.outer(u, u @ W) |
The same subtraction on a matrix: each column loses its part along u. |
| 6. Count refusals | refusal_toy.py:87refused = lambda z: int((z.argmax(1) == 0).sum()) |
A request is refused when answer 0 scores highest. |
A real run
trace.py runs the six steps and prints one line for each, then one line for the limit. Run it from the repository root:
cd lessons/can-a-models-refusals-be-removed/code && python3 trace.pyPhone readers: slide the box sideways to see every column.
step 1 read the vectors after layer 5: 64 flagged and 64 plain requests, 32 numbers each
step 2 difference of the two means, length 1: 8.4 degrees from the planted direction
step 3 held-out distance along it: flagged 8.1, plain -0.1; one cut at 4.0 sorts 200 of 200 right
step 4 taken out at all 7 points: last-layer length of flagged vectors 13.3 -> 5.6, plain 5.7 -> 5.7
step 5 same edit in the weights: largest difference in the answer scores 0.0000000000
step 6 flagged requests refused: 100 of 100 before, 0 of 100 after; plain answers changed: 1 of 100 (mean KL 0.006)
limit with 30 of 100 flagged requests on a second route, the same edit leaves 29 of 100 refusedRead it against the steps.
- Steps 1 and 2. The direction is found from 64 and 64 requests and lands 8.4 degrees from the planted one. The toy can say so only because it planted that direction. A real model offers no such check, so step 3, on requests the direction was not found from, is the test that matters.
- Step 3. Flagged requests sit at 8.1 along the direction and plain ones at -0.1, so one cut sorts all 200. Figure 1 draws 30 of each. After step 4 every dot sits at 0 along the direction and keeps its other positions.
- Step 4. The last-layer length of flagged vectors falls from 13.3 to 5.6 and plain ones stay at 5.7. What was taken out was most of what made flagged vectors differ.
- Step 5. The weight edit gives the same answer scores as the hook, to the last of ten decimals. The edited network needs no hook, and nothing in the code that runs it shows the change.
- Step 6. All 100 flagged requests were refused before, and 0 are after. One plain answer in 100 changes, with a mean KL of 0.006. KL measures how far the answer odds moved, and 0 means not at all. Figure 2 shows how much must come out: nothing happens at 10%, and by 30% no flagged request is refused. That sharp fall comes from the toy’s own switch (Go deeper).
Figure 2. Refusals fall between 10% and 30% taken out, and plain answers hardly move.
- The limit. Put 30 of the 100 flagged requests on a second route, and the same edit leaves 29 refused (Figure 3). The direction found then sits 25.0 degrees off the planted one. One direction removes the refusals that travel along it and few others.
Figure 3. One direction does not remove a refusal that travels along two.
Hands on
Your turn
The direction was read after layer 5. See what reading it before the first layer does.
- In
code/trace.py, change line 6 toLAYER = 0. - Run
python3 trace.pyagain in thecodefolder.
Look at steps 2 and 6. How far from the planted direction is the direction now, and how many flagged requests are still refused?
What you should see
Every line but step 5 changes:
step 1 read the vectors after layer 0: 64 flagged and 64 plain requests, 32 numbers each
step 2 difference of the two means, length 1: 37.9 degrees from the planted direction
step 3 held-out distance along it: flagged 1.2, plain -0.0; one cut at 0.6 sorts 166 of 200 right
step 4 taken out at all 7 points: last-layer length of flagged vectors 13.3 -> 5.9, plain 5.7 -> 5.7
step 5 same edit in the weights: largest difference in the answer scores 0.0000000000
step 6 flagged requests refused: 100 of 100 before, 13 of 100 after; plain answers changed: 8 of 100 (mean KL 0.060)
limit with 30 of 100 flagged requests on a second route, the same edit leaves 49 of 100 refusedThe direction is 37.9 degrees off, 13 flagged requests are still refused, and 8 plain answers change, where reading after layer 5 gave 8.4 degrees, 0 and 1. Before the first layer, flagged and plain requests differ only by the push that was planted, so 64 and 64 examples pin the direction down poorly. By layer 5 the sets are far apart along it, and the same examples find it well (Figure 4).
What this means when you build
- Treat a model’s refusals as a setting, not a guarantee. In the toy, one edit takes all of them away. In the study of real models, one edit to the weights did the same to many. A model whose files someone else can change can lose its refusals.
- Keep your own checks outside the model. A filter in your code, on what goes in and what comes out, works whatever the model does (lesson 15). The model’s refusals are one layer.
- Test the model you serve. Run your own test set (lesson 13) on the weights you run, with requests that should be declined, and check who published the file (lesson 17). A refusal that held in someone’s demo says little about a model that another party changed.
- Do not take good ordinary answers as proof. The edit left 99 of 100 plain answers the same while all 100 refusals went. A model that answers well still may have lost its refusals.
- Read the toy as a mechanism, not a measurement. It shows how a direction can carry a refusal, how it is found and why one edit is not always enough. It says nothing about how many refusals any real model keeps.
Optionalabout 17 min
Go deeper
Optional. This part is for readers who write Python. The code is in the code/ folder next to this lesson. From inside that folder, python3 show_numbers.py prints the numbers below, python3 trace.py prints the run above, and python3 -B -m pytest -q -p no:cacheprovider runs the tests.
refusal_toy.py is 100 lines, with its comments and blank lines: a network, a way to make requests, the three edits of the lesson and a way to count. Every number in it is made up inside the program from a fixed seed. It reads no file, opens no connection and loads no model, and nothing in it could be pointed at a real one.
The toy network
def build(seed=SEED):
"""The network: an embedding E, six blocks (W_in, bias, W_out) and a readout U."""
rng = np.random.default_rng(seed)
E, _ = np.linalg.qr(rng.normal(size=(D, D)))
routes = E[:, :ROUTES].T # the planted directions, one unit vector each
blocks = []
for _ in range(LAYERS):
W_in = np.vstack([routes, rng.normal(size=(HIDDEN - ROUTES, D)) / np.sqrt(D)])
W_out = np.hstack([GAIN * routes.T, 0.15 * rng.normal(size=(D, HIDDEN - ROUTES)) / np.sqrt(HIDDEN)])
bias = np.r_[np.full(ROUTES, TAU), np.zeros(HIDDEN - ROUTES)]
blocks.append((W_in, bias, W_out))
U = np.hstack([READ * routes.sum(0)[:, None], 2 / np.sqrt(D) * rng.normal(size=(D, ANSWERS - 1))])
return {"E": E, "routes": routes, "blocks": blocks, "U": U}E is a random rotation, and its first two rows are the two planted directions, routes. Each layer reads the vector with W_in, keeps what is above a bias, and writes back with W_out. The first two of its 24 hidden units are the refusal units. They read along a planted direction, switch on only above TAU = 0.5, and write GAIN = 0.5 times the excess back along the same direction. The other 22 are random and write small changes everywhere. The readout U scores answer 0, “refuse”, by READ = 2 times the part along the planted directions, and the other 7 answers with random numbers.
def requests(n, flagged, rng, noise=1.0, second=0.0):
"""n requests as feature vectors. A flagged one carries a push along route 0, or along route 1 for the first `second` share."""
f = rng.normal(size=(n, D)) * noise
f[:, :ROUTES] *= QUIET
if flagged:
f[np.arange(n), (np.arange(n) < round(second * n)).astype(int)] += FLAG
return fA request is noise in every feature, with the two route features cut to a tenth, plus a push of FLAG = 1.5 along route 0 if it is flagged. For the share second of flagged requests the push goes along route 1 instead. Once the refusal units are on, each layer multiplies a flagged request’s part along its route by 1.5 and takes 0.25 off. Without noise it would read 1.5, 2.0, 2.75, 3.9, 5.6, 8.1 and 11.9 at the 7 points, and the toy’s averages along the planted direction are within 0.2 of these.
def forward(model, f, hook=lambda x: x):
"""Run requests f. Returns the answer scores and the residual vectors at LAYERS + 1 points."""
x = hook(f @ model["E"].T)
points = [x]
for W_in, bias, W_out in model["blocks"]:
x = hook(x + np.maximum(x @ W_in.T - bias, 0) @ W_out.T)
points.append(x)
return x @ model["U"], pointsThe vector is called the residual stream in the papers, because each layer adds to it and passes it on. forward keeps its value at 7 points, which are what step 1 reads, and applies hook at each of them, which is how step 4 works.
Where the direction is found
def refusal_direction(flagged_points, plain_points):
"""Step 2: the difference of the two means, scaled to length 1."""
d = flagged_points.mean(0) - plain_points.mean(0)
return d / np.linalg.norm(d)
def remove(u, share=1.0):
"""Step 4: a hook that takes `share` of each vector's part along u out, at every point it is applied."""
return lambda x: x - share * np.outer(x @ u, u)
def orthogonalize(model, u):
"""Step 5: the same edit written into the weights that write into the residual vector."""
cut = lambda W: W - np.outer(u, u @ W)
return {**model, "E": cut(model["E"]), "blocks": [(W_in, bias, cut(W_out)) for W_in, bias, W_out in model["blocks"]]}Write a request’s vector as x = s·r + n, where r is the planted direction, s the push along it and n all the rest. The two sets differ in s and share the same kind of n. In the difference of the two averages, the s part adds up and the n part partly cancels. What is left of n is an error across r, whose length shrinks with the square root of the number of examples and grows with the square root of the number of other directions. The angle of the result is the arctangent of that error over the gap along r. The numbers are in show_numbers.py. After layer 5 the gap is 8.34 along r and 1.23 across it, which is 8.4 degrees. After layer 0 it is 1.49 along and 1.16 across, which is 37.9 degrees. The same 64 and 64 examples find the direction better when the sets sit farther apart, and Figure 4 shows what follows. After layer 0 the direction misses, so 13 flagged requests are refused anyway and 8 plain answers change, because the cut takes out some of what plain requests need. The paper builds one candidate for each layer and each position after the instruction and chooses among them with held-out sets (below). The toy has no such step: it reads the direction after layer 5, a late layer picked by hand, and layer 6 sits nearer the planted direction, at 6.5 degrees (Figure 4). Layers 4, 5 and 6 give the same held-out counts, so the counts did not pick it.
Figure 4. The later the direction is read, the nearer it is to the planted one and the less it disturbs.
Why the hook and the weight edit are the same edit
Let P = I − u uᵀ, the matrix that strips the part along u. Hooked, the vectors are x₀ = P E f, and then x₍k+1₎ = P (x₍k₎ + W_out h₍k₎), where h₍k₎ is the layer’s hidden values. If x₍k₎ already has no part along u, then P x₍k₎ = x₍k₎ and x₍k+1₎ = x₍k₎ + (P W_out) h₍k₎. By induction, running the unhooked network with E and every W_out replaced by P E and P W_out gives the same x at every point, so the same answer scores. P W_out is W - np.outer(u, u @ W): every column of the matrix loses its part along u. W_in and U only read the vector, so they stay. Step 5 prints 0.0000000000 for this reason. It is a rank-one change to each matrix, which is as large as the edit gets.
Figure 5. The weight edit changes the one column that carries the refusal, and the others hardly.
A real model has several matrices that write into its vector: the embedding, and the output matrix of every attention block and every MLP block. The paper applies the edit to each of them (Section 4.1 and Appendix E). The result is a model of the same shape and size whose running code has no extra step.
What the paper reports on real models
Arditi et al. (arXiv preprint, version 3; read 2026-10-20) study 13 open chat models from five families, with 1.8 to 72 billion parameters. They call their two sets harmful and harmless instructions; the toy’s flagged and plain requests are invented stand-ins, and this lesson reproduces no such instruction.
- Finding the direction. It is the difference between the mean activations on the two sets, for each layer and each token position after the instruction. One candidate is chosen with validation sets, by which one most reduces refusals when removed, among those that still induce a refusal when added to harmless requests and that change the model’s output on harmless requests by a KL under 0.1.
- Taking it out. Directional ablation at every layer and every position cuts the refusals on 100 test instructions, and adding the direction makes models refuse harmless ones. For one 8B model, the refusal score falls from 0.95 to 0.01 and the safety score from 0.97 to 0.15 (Table 10).
- Weights. Orthogonalizing the weights gives the same behaviour, and they call it an interpretable rank-one weight edit.
- What it costs. On MMLU, an edited 7B model scores 51.8 against 51.7 for the original and an edited 70B model 79.8 against 79.9. On TruthfulQA, the largest model of each of five families falls by 1.0 to 3.5 points.
- Cost of doing it. The authors write that their method “can yield a jailbroken version of a 70B parameter model using less than $5 of compute”. They weigh whether publishing the method lowers the bar for misuse and judge that it does so only marginally.
- Where it stops. The authors write that their findings “may not generalize to untested models, especially those at greater scale”, and name current proprietary models and future ones among them. They call their way of finding the direction “likely not optimal” and a heuristic, an “existence proof”, and say the meaning of the direction is unclear. In one model, with one adversarial suffix, they find that the suffix suppresses the direction (Section 5).
Their abstract says the findings “underscore the brittleness of current safety fine-tuning methods.”
What the toy leaves out
- A planted switch. The refusal is made by planting a direction and a switch on it. The sharp fall of Figure 2 comes from that switch: a unit adds to the direction only when the vector already holds more than 0.5 of it, and every layer pushes further, so a flagged vector either keeps growing along the direction or fades. Taking out part of the direction moves requests across that line. How refusals fall with the share taken out in a real model is not shown here.
- One route, or two. The toy has one planted route and, in the limit run, a second. A real model is whatever training made it. The study reports one direction for each of its 13 models.
- No tokens and no attention. A request is one vector, not a row of words passing through attention (lesson 6).
- Made-up requests. “Flagged” and “plain” are labels on noise, not content.
- A small measure of damage. The toy counts plain answers that change and the mean KL. The paper uses benchmarks and the loss on text.
- No defence. The toy only removes. It has nothing about training a model so that the direction cannot be removed, and the sources read for this lesson have nothing either beyond calling today’s safety fine-tuning brittle.
- No layer norms or output biases. The toy has no layer norms and no biases that write into its vector, which real models can have. The paper’s model description leaves out layer normalization (Section 2.1), and its weight edit covers every matrix and output bias that writes into the vector (Section 4.1 and Appendix E). In the toy the two match because every writer is edited; in a real model that match has to be checked, not assumed.
- No real weights. Nothing here loads, reads or writes a real model’s weights.
Proof it works
test_refusal_toy.py guards what a reader would see go wrong. The direction found must stay close to the planted one. The hooked network and the one with edited weights must give the same scores. The run must refuse 100 of 100 before and 0 after, change 1 of 100 plain answers, and leave 29 refused when 30 use a second route. The part along the route must follow the rule above. One test checks that every number the lesson, the trace and the five figures quote comes from the code, and that each code block and each file:line in the table matches the real file. Another runs the Your turn change, and a last one checks the figures. test_widget.py runs the widget’s JavaScript in Node against the Python on every setting. The widget is a table of the toy’s own results, written into the page by embed_widget.py, so it has no model of its own to drift.
Where this is used in practice
The paper above (Arditi et al., arXiv 2406.11717; read 2026-10-20) is the published account of the effect. It reports that a single direction mediates refusal across the 13 models and that the edit works on their weights. Removing the direction from a model’s weights is known as abliteration. This lesson shows the idea on a toy and names no tool that does it to a real model.
It rests on earlier work. Park, Choe and Veitch state the linear representation hypothesis as “the idea that high-level concepts are represented linearly as directions in some representation space” (arXiv 2311.03658; read 2026-10-20). Zou et al. propose representation engineering, which “places population-level representations, rather than neurons or circuits, at the center of analysis”, with methods to monitor and to manipulate high-level phenomena in deep networks (arXiv 2310.01405; read 2026-10-20). A refusal direction is one such direction. For a builder the lesson of all three is the one above: what a model does on a request can depend on a few directions, and whoever holds the weights can change them.
Skills this lesson builds
- Say where a refusal can live in a model and how one direction can carry it.
- Find a direction from two sets of examples and test it on requests it was not found from.
- Explain why taking a direction out of every vector and cutting it from the weights give the same model.
- Judge an edit by the refusals it removes and by the ordinary answers it leaves alone.
- Say what to check outside the model when its refusals may not hold.
Portfolio project: extend the toy with a third planted route and show how many flagged requests stay refused after one direction is taken out, found separately for each route. Chart refusals against the share taken out, and add a short note on which checks outside the model an app of your own has and what each would catch. Publish the code, the chart and the note.
Check yourself
-
After the edit, 0 of 100 flagged requests are refused and 1 of 100 plain answers changes. Do good answers on plain requests show that a model’s refusals are intact?
Answer
No. In the toy, all refusals went while 99 of 100 plain answers stayed the same. A model that answers ordinary requests well may have lost its refusals, so test the refusals with your own set.
-
Taking out 10% of the direction leaves 100 of 100 flagged requests refused, and 30% leaves none. What in the toy makes the fall sharp?
Answer
The refusal units switch on only when the vector holds more than 0.5 along the direction, and each layer pushes it further. A flagged vector either keeps growing along the direction or fades, and taking out 10% to 30% moves requests across that line. The toy does not say that a real model falls the same way.
-
Read before the first layer, the direction is 37.9 degrees off the planted one. After layer 5 it is 8.4. Why?
Answer
Before the first layer a flagged vector differs from a plain one only by the planted push of 1.5. The two averages differ by 1.49 along the planted direction but by 1.16 across it, so the difference points well off. After layer 5 the gap along it is 8.34 against 1.23 across it.
-
With 30 of 100 flagged requests on a second route, the edit leaves 29 refused. Why is that not a safety to rely on?
Answer
The edit cut the one direction it found, so the requests that use the second route still refuse. That is a fact about this toy and this edit, not something an app controls. Another edit or another model gives another count, so measure the refusals you actually have.
-
Why does cutting the part along u from every matrix that writes into the vector give the same answers as removing it from the vector after every layer?
Answer
If the vector has no part along u before a layer, the only way the layer can add one is through what it writes. Cutting that part from each column of the matrix means nothing it writes has a part along u, so the vector never gains one. The two networks agree at every point, and in step 5 their scores differ by 0.0000000000.
Further reading
Gentle
- How does a safety filter decide, and where does it fail? (lesson 15, the checks that sit outside the model)
- How do I check my AI’s answers with a small test set? (lesson 13, how to test the model you serve)
- Is it safe to download a model file? (lesson 17, who published the file)
- Refusal in Language Models Is Mediated by a Single Direction (Arditi et al., arXiv preprint; the abstract and the first figure; read 2026-10-20)
Deep
- Refusal in Language Models Is Mediated by a Single Direction (Arditi et al., arXiv preprint, version 3; the method in Section 2, weight orthogonalization in Section 4.1, the equivalence in Appendix E, limitations in Section 7; read 2026-10-20)
- The Linear Representation Hypothesis and the Geometry of Large Language Models (Park, Choe and Veitch; what it means for a concept to be a direction; read 2026-10-20)
- Representation Engineering: A Top-Down Approach to AI Transparency (Zou et al.; reading and steering a model through population-level representations; read 2026-10-20)
Optional
Code
Each file runs on its own. Download it, or read it here.
Show code
"""Keep the table of numbers inside ../widget.html equal to the one show.py computes from refusal_toy.py.
python3 embed_widget.py rewrite the data block in widget.html
python3 embed_widget.py --check exit 1 if widget.html differs (test_widget.py runs the same check)
"""
import json
import re
import sys
from pathlib import Path
import show
WIDGET = Path(__file__).resolve().parent.parent / "widget.html"
CONFIG_BLOCK = re.compile(r'(<script type="application/json" id="rf-config">)(.*?)(</script>)', re.S)
def embed(html):
"""html with the data block rewritten from show.py."""
settings = json.dumps(show.widget_config(), ensure_ascii=False, separators=(", ", ": "))
return CONFIG_BLOCK.sub(lambda m: m.group(1) + "\n" + settings + "\n" + m.group(3), html, count=1)
if __name__ == "__main__":
current = WIDGET.read_text(encoding="utf-8")
updated = embed(current)
if "--check" in sys.argv:
sys.exit(0 if updated == current else 1)
WIDGET.write_text(updated, encoding="utf-8")
print("widget.html data block now matches show.py")Show code
"""Draw the five figures of lesson 32 from refusal_toy.py, so that no number in a figure is typed by hand.
python3 make_figs.py rewrite ../figs/*.svg
python3 make_figs.py --check exit 1 if a figure on disk differs (test_refusal_toy.py runs the same check)
All five figures are 600 units wide. Text is at least 22 units (20 for axis ticks) in ink or grey on white or a light fill; orange and green are used on marks
only, and every mark that colour tells apart also differs in shape or dash and is named in words.
"""
import sys
from pathlib import Path
from xml.sax.saxutils import escape
import numpy as np
import show
from refusal_toy import LAYERS, direction_at, forward, orthogonalize, remove, study
FIGS = Path(__file__).resolve().parent.parent / "figs"
INK, GREY, ORANGE, GREEN, FILL, PANEL = "#0a1626", "#44566b", "#b36200", "#00855F", "#d2dae4", "#ebf0f6"
DASH = "10 8"
LAYER, SHOWN = 5, 30 # the layer the lesson reads; requests drawn per set in figure 1
def num(x: float) -> str:
return f"{x:.2f}".rstrip("0").rstrip(".")
# The SVG namespace is a name that every standalone .svg file must carry, not an address this program visits. It is written in two pieces so that a plain
# search of code/ for the start of a web address finds nothing (test_refusal_toy.py runs that search).
NAMESPACE = "ht" + "tp://www.w3.org/2000/svg"
def frame(width: int, height: int, title: str, desc: str, body: list[str], defs: str = "") -> str:
head = (f'<svg xmlns="{NAMESPACE}" viewBox="0 0 {width} {height}" width="{width}" height="{height}" role="img" '
f'font-family="system-ui, sans-serif" font-size="22" fill="{GREY}">')
return "\n".join([head, f" <title>{escape(title)}</title>", f" <desc>{escape(desc)}</desc>", *([f" {defs}"] if defs else []),
' <rect width="100%" height="100%" fill="#ffffff"/>', *(" " + line for line in body), "</svg>", ""])
def text(x: float, y: float, content: str, anchor: str = "start", color: str = INK, bold: bool = False, size: int = 22) -> str:
extra = (f' text-anchor="{anchor}"' if anchor != "start" else "") + (' font-weight="bold"' if bold else "") + (f' font-size="{size}"' if size != 22 else "")
return f'<text x="{num(x)}" y="{num(y)}"{extra} fill="{color}">{escape(content)}</text>'
def triangle(x: float, y: float) -> str:
return f'<path d="M{num(x)} {num(y - 8)}L{num(x + 8)} {num(y + 6)}L{num(x - 8)} {num(y + 6)}z" fill="{ORANGE}" stroke="#ffffff" stroke-width="1"/>'
def circle(x: float, y: float, r: float = 6, color: str = GREEN) -> str:
return f'<circle cx="{num(x)}" cy="{num(y)}" r="{num(r)}" fill="{color}" stroke="#ffffff" stroke-width="1"/>'
def square(x: float, y: float, half: float = 6) -> str:
return f'<rect x="{num(x - half)}" y="{num(y - half)}" width="{num(2 * half)}" height="{num(2 * half)}" fill="#ffffff" stroke="{GREEN}" stroke-width="3"/>'
def tick(x: float, y: float, label: str, anchor: str = "middle") -> str:
return text(x, y, label, anchor, GREY, size=20)
# --- figure 1: the map, before and after ------------------------------------------------------------------------------------------
def fig_the_map() -> str:
model, sets, base = study()
u = direction_at(base, LAYER)
points = {name: base[name][1][LAYER][:SHOWN] for name in ("test_flagged", "test_plain")}
both = np.vstack(list(points.values()))
across = both - np.outer(both @ u, u) # what is left of each vector once its part along u is taken away
v = np.linalg.svd(across - across.mean(0), full_matrices=False)[2][0] # the direction of the most spread that remains
v = v if v[np.abs(v).argmax()] > 0 else -v
along = {name: p @ u for name, p in points.items()}
height = {name: (p - np.outer(p @ u, u)) @ v for name, p in points.items()}
cut = (base["test_flagged"][1][LAYER] @ u).mean() / 2 + (base["test_plain"][1][LAYER] @ u).mean() / 2
scale, low, mid = 21.5, -1.5, (min(h.min() for h in height.values()) + max(h.max() for h in height.values())) / 2
top, floor = 76, 76 + 187
body = []
for k, (title, left) in enumerate((("Before", 20), ("After", 310))):
px = lambda x: left + 6 + (x - low) * scale
py = lambda y: top + 93.5 - (y - mid) * scale
body.append(text(left, 32, title, bold=True))
body.append(f'<path d="M{num(px(0))} {top}V{floor}" stroke="{FILL}" stroke-width="2"/>')
body.append(f'<path d="M{left} {floor}H{left + 270}" stroke="{GREY}" stroke-width="2"/>')
for t in (0, 4, 8):
body.append(tick(px(t), floor + 28, str(t)))
body.append(text(left + 135, floor + 62, "along the direction", "middle", GREY))
body.append(f'<text x="{left - 2}" y="170" text-anchor="middle" transform="rotate(-90 {left - 2} 170)" fill="{GREY}">other spread</text>')
if k == 0:
body.append(f'<path d="M{num(px(cut))} {top}V{floor}" stroke="{GREY}" stroke-width="3" stroke-dasharray="{DASH}"/>')
body.append(text(px(cut), 62, "cut", "middle"))
for x, y in zip(along["test_plain"], height["test_plain"]):
body.append(circle(px(x if k == 0 else 0.0), py(y)))
for x, y in zip(along["test_flagged"], height["test_flagged"]):
body.append(triangle(px(x if k == 0 else 0.0), py(y)))
body.append(text(20 + 6 + (0 - low) * scale, 62, "plain", "middle"))
body.append(text(20 + 6 + (float(np.mean(along["test_flagged"])) - low) * scale, 62, "flagged", "middle"))
body.append(text(316, 62, "both sets"))
flagged, plain = along["test_flagged"], along["test_plain"]
desc = (f"Two panels of {SHOWN} flagged requests (orange triangles) and {SHOWN} plain requests (green circles) at layer {LAYER}. Across: how far along the found direction a "
f"request sits. Up, labelled other spread: the biggest spread that remains. Before: flagged requests sit between {flagged.min():.1f} and {flagged.max():.1f}, plain requests between "
f"{plain.min():.1f} and {plain.max():.1f}, and a cut at {cut:.1f} sorts them. After the direction is taken out, every request sits at 0.0 along it, "
f"both sets in one stripe, and their up positions are unchanged.")
return frame(600, floor + 80, "Before the cut flagged and plain requests sit apart along one direction; after it they sit together", desc, body)
# --- figure 2: the same edit in the weights --------------------------------------------------------------------------------------
def fig_weight_edit() -> str:
model, sets, base = study()
u = direction_at(base, LAYER)
W = model["blocks"][LAYER][2] # the matrix that writes into the vector after the layer read
before = W.T @ u # how much of each of its 24 columns lies along u
after = orthogonalize(model, u)["blocks"][LAYER][2].T @ u
others = before[1:]
pitch, scale = 540 / len(before), 200.0
body = []
for title, y, values in (("Before: the part of each column along u", 170, before), ("After: the same, once that part is cut", 306, after)):
body.append(text(30, 34 if y == 170 else 272, title, bold=True))
body.append(f'<path d="M30 {y}H570" stroke="{GREY}" stroke-width="2"/>')
for k, value in enumerate(values):
if abs(value) >= 0.0005:
body.append(f'<rect x="{num(30 + k * pitch + 3)}" y="{num(y - max(value, 0) * scale)}" width="{num(pitch - 6)}" height="{num(abs(value) * scale)}" '
f'fill="{ORANGE if k == 0 else INK}"/>')
body.append(text(30 + pitch + 10, 100, f"refusal unit: {before[0]:.2f}"))
body.append(text(30, 212, f"the other {len(others)}: {others.min():.2f} to {others.max():.2f}", color=GREY))
body.append(text(30, 344, f"all {len(after)} columns: {np.abs(after).max():.2f}", color=GREY))
body.append(text(30, 386, "bars: one column of the matrix each", color=GREY))
desc = (f"Two rows of {len(before)} bars, one for each column of the matrix that writes into the vector after layer {LAYER}, giving how much of the column lies along u. "
f"Before, column 0, the refusal unit, has {before[0]:.2f} along u and the other {len(others)} columns between {others.min():.2f} and {others.max():.2f}. "
f"After the edit every column has {np.abs(after).max():.2f} along u.")
return frame(600, 406, "Cutting the part along u from each column that writes into the vector changes almost one column", desc, body)
# --- figure 3: how much to take out -----------------------------------------------------------------------------------------------
def fig_share_removed() -> str:
r = show.run()
refused, changed = r["refused"][LAYER], r["changed"][LAYER]
x0, x1, top, base = 84, 560, 60, 320
px = lambda share: x0 + share / 100 * (x1 - x0)
py = lambda count: base - count / 100 * (base - top)
body = [text(20, 36, "requests out of 100", color=GREY)]
for tick_value in (0, 50, 100):
body.append(f'<path d="M{x0} {num(py(tick_value))}H{x1}" stroke="{FILL}" stroke-width="2"/>')
body.append(tick(x0 - 8, py(tick_value) + 7, str(tick_value), "end"))
body.append(f'<path d="M{x0} {base}H{x1}" stroke="{GREY}" stroke-width="2"/>')
for share in (0, 25, 50, 75, 100):
body.append(tick(px(share), base + 30, f"{share}%"))
body.append(text((x0 + x1) / 2, base + 64, "share of the direction taken out", "middle", GREY))
body.append(f'<path d="M{" L".join(f"{num(px(s))} {num(py(c))}" for s, c in zip(show.SHARES, changed))}" fill="none" stroke="{GREEN}" stroke-width="4" stroke-dasharray="{DASH}"/>')
body.append(f'<path d="M{" L".join(f"{num(px(s))} {num(py(c))}" for s, c in zip(show.SHARES, refused))}" fill="none" stroke="{ORANGE}" stroke-width="4"/>')
for s, c in zip(show.SHARES, changed):
body.append(square(px(s), py(c)))
for s, c in zip(show.SHARES, refused):
body.append(circle(px(s), py(c), 7, ORANGE))
body.append(circle(250, py(75) - 7, 7, ORANGE))
body.append(text(266, py(75), "flagged requests still refused"))
body.append(square(250, py(40) - 7))
body.append(text(266, py(40), "plain answers changed"))
at = lambda series, share: series[show.SHARES.index(share)]
desc = (f"Requests out of 100 against the share of the direction taken out, read after layer {LAYER}. Flagged requests still refused (orange line, filled circles): "
f"{at(refused, 0)} at 0%, {at(refused, 10)} at 10%, {at(refused, 15)} at 15%, {at(refused, 20)} at 20%, {at(refused, 25)} at 25% and {at(refused, 30)} from 30% on. "
f"Plain answers changed (green dashed line, hollow squares): {at(changed, 0)} at 0%, {at(changed, 5)} from 5% on.")
return frame(600, base + 84, "Refusals fall in a narrow band of the share taken out, while ordinary answers hardly move", desc, body)
# --- figure 4: where to read the direction ----------------------------------------------------------------------------------------
def fig_where_to_read() -> str:
r = show.run()
top, step = 100, 44
body = [text(150, 34, "degrees off", bold=True), text(440, 34, "still", "middle", bold=True), text(540, 34, "answers", "middle", bold=True),
text(440, 60, "refused", "middle", bold=True), text(540, 60, "changed", "middle", bold=True)]
rows = []
for layer in range(LAYERS + 1):
y = top + layer * step
length = r["angle"][layer] * 4.5
refused, changed = r["refused"][layer][-1], r["changed"][layer][-1]
body.append(text(30, y + 8, f"layer {layer}"))
body.append(f'<rect x="150" y="{y - 16}" width="{num(length)}" height="30" fill="{PANEL}" stroke="{INK}" stroke-width="2"/>')
body.append(text(150 + length + 10, y + 8, f"{r['angle'][layer]:.1f}", bold=True))
body.append(text(440, y + 8, str(refused), "middle", bold=True))
body.append(text(540, y + 8, str(changed), "middle", bold=True))
rows.append(f"layer {layer}: {r['angle'][layer]:.1f} degrees off, {refused} refused, {changed} changed")
desc = (f"For each of the {LAYERS + 1} layers at which the direction can be read, how many degrees the direction found is from the planted one, how many of 100 flagged requests "
f"are still refused after all of it is taken out, and how many of 100 plain answers change. " + "; ".join(rows) + ".")
return frame(600, top + (LAYERS + 1) * step - 10, "The later the direction is read, the nearer it is to the planted one and the less it disturbs", desc, body)
# --- figure 5: a second route ---------------------------------------------------------------------------------------------------
def fig_second_route() -> str:
second = 0.3
model, sets, base = study(second=second)
u = direction_at(base, LAYER)
refused_after = set(np.flatnonzero(forward(model, sets["test_flagged"], remove(u))[0].argmax(1) == 0).tolist())
refused_before = set(np.flatnonzero(base["test_flagged"][0].argmax(1) == 0).tolist())
on_second = round(second * len(sets["test_flagged"]))
inside = sum(k < on_second for k in refused_after)
top, pitch = 76, 20
body = [f'<rect x="{left - 5}" y="{top - 5}" width="{10 * pitch + 5}" height="{3 * pitch + 5}" fill="{PANEL}"/>' for left in (30, 330)]
for title, left, refused in (("Before", 30, refused_before), ("After", 330, refused_after)):
body.append(text(left, 36, title, bold=True))
for k in range(len(sets["test_flagged"])):
x, y = left + (k % 10) * pitch, top + (k // 10) * pitch
fill = ORANGE if k in refused else "none"
body.append(f'<rect x="{x}" y="{y}" width="16" height="16" fill="{fill}" stroke="{ORANGE if k in refused else INK}" stroke-width="2"/>')
body.append(text(left, top + 10 * pitch + 32, f"{len(refused)} refused", bold=True))
body.append(text(300, top + 10 * pitch + 72, "shaded rows: the second route", "middle", GREY))
body.append(text(300, top + 10 * pitch + 102, "filled: refused. hollow: answered", "middle", GREY))
desc = (f"Two grids of the 100 held-out flagged requests when {on_second} of them use a second route (the shaded top three rows), each square one request. Before the edit "
f"{len(refused_before)} are refused (filled). After the direction read at layer {LAYER} is taken out, {len(refused_after)} are refused, {inside} of them among the {on_second} "
f"second-route requests.")
return frame(600, top + 10 * pitch + 122, "One direction does not remove a refusal that travels along two", desc, body)
FIGURES = {
"the-map.svg": fig_the_map,
"weight-edit.svg": fig_weight_edit,
"share-removed.svg": fig_share_removed,
"where-to-read.svg": fig_where_to_read,
"second-route.svg": fig_second_route,
}
def main() -> int:
stale = []
for name, draw in FIGURES.items():
path, svg = FIGS / name, draw()
if "--check" in sys.argv:
if not path.exists() or path.read_text(encoding="utf-8") != svg:
stale.append(name)
else:
FIGS.mkdir(exist_ok=True)
path.write_text(svg, encoding="utf-8")
if "--check" in sys.argv:
print("figures differ from refusal_toy.py: " + ", ".join(stale) if stale else "figures match refusal_toy.py")
return 1 if stale else 0
print("wrote " + ", ".join(FIGURES))
return 0
if __name__ == "__main__":
sys.exit(main())Show code
"""A toy network with a refusal planted along one direction, and the steps that find and remove it.
Everything is small and made up here: a residual vector of 32 numbers, six layers, eight possible
answers (answer 0 is "refuse"). No real model, no downloaded weights, only NumPy and a fixed seed.
"""
import numpy as np
D, HIDDEN, LAYERS, ANSWERS = 32, 24, 6, 8 # answer 0 is "refuse"
ROUTES = 2 # directions a refusal can travel along
TAU, GAIN, READ = 0.5, 0.5, 2.0 # a layer's refusal unit switches on above TAU and adds GAIN times the excess
FLAG, QUIET = 1.5, 0.1 # push a flagged request carries; how little plain requests use the routes
SEED, DATA_SEED = 7, 11
def build(seed=SEED):
"""The network: an embedding E, six blocks (W_in, bias, W_out) and a readout U."""
rng = np.random.default_rng(seed)
E, _ = np.linalg.qr(rng.normal(size=(D, D)))
routes = E[:, :ROUTES].T # the planted directions, one unit vector each
blocks = []
for _ in range(LAYERS):
W_in = np.vstack([routes, rng.normal(size=(HIDDEN - ROUTES, D)) / np.sqrt(D)])
W_out = np.hstack([GAIN * routes.T, 0.15 * rng.normal(size=(D, HIDDEN - ROUTES)) / np.sqrt(HIDDEN)])
bias = np.r_[np.full(ROUTES, TAU), np.zeros(HIDDEN - ROUTES)]
blocks.append((W_in, bias, W_out))
U = np.hstack([READ * routes.sum(0)[:, None], 2 / np.sqrt(D) * rng.normal(size=(D, ANSWERS - 1))])
return {"E": E, "routes": routes, "blocks": blocks, "U": U}
def requests(n, flagged, rng, noise=1.0, second=0.0):
"""n requests as feature vectors. A flagged one carries a push along route 0, or along route 1 for the first `second` share."""
f = rng.normal(size=(n, D)) * noise
f[:, :ROUTES] *= QUIET
if flagged:
f[np.arange(n), (np.arange(n) < round(second * n)).astype(int)] += FLAG
return f
def forward(model, f, hook=lambda x: x):
"""Run requests f. Returns the answer scores and the residual vectors at LAYERS + 1 points."""
x = hook(f @ model["E"].T)
points = [x]
for W_in, bias, W_out in model["blocks"]:
x = hook(x + np.maximum(x @ W_in.T - bias, 0) @ W_out.T)
points.append(x)
return x @ model["U"], points
def refusal_direction(flagged_points, plain_points):
"""Step 2: the difference of the two means, scaled to length 1."""
d = flagged_points.mean(0) - plain_points.mean(0)
return d / np.linalg.norm(d)
def remove(u, share=1.0):
"""Step 4: a hook that takes `share` of each vector's part along u out, at every point it is applied."""
return lambda x: x - share * np.outer(x @ u, u)
def orthogonalize(model, u):
"""Step 5: the same edit written into the weights that write into the residual vector."""
cut = lambda W: W - np.outer(u, u @ W)
return {**model, "E": cut(model["E"]), "blocks": [(W_in, bias, cut(W_out)) for W_in, bias, W_out in model["blocks"]]}
def softmax(z):
e = np.exp(z - z.max(1, keepdims=True))
return e / e.sum(1, keepdims=True)
def study(noise=1.0, second=0.0, n_train=64, n_test=100):
"""Build the toy, draw the four request sets and run each once unedited."""
model = build()
rng = np.random.default_rng(DATA_SEED)
sets = {name: requests(n, flagged, rng, noise, second) for name, n, flagged in
(("train_flagged", n_train, True), ("train_plain", n_train, False),
("test_flagged", n_test, True), ("test_plain", n_test, False))}
return model, sets, {name: forward(model, f) for name, f in sets.items()}
def direction_at(base, layer):
return refusal_direction(base["train_flagged"][1][layer], base["train_plain"][1][layer])
def measure(base, edited_scores):
"""Step 6: held-out flagged requests still refused, plain requests refused, plain answers changed, mean KL on plain."""
refused = lambda z: int((z.argmax(1) == 0).sum())
before, after = base["test_plain"][0], edited_scores["test_plain"]
p, q = softmax(before), softmax(after)
return {"flagged_refused": refused(edited_scores["test_flagged"]), "plain_refused": refused(after),
"changed": int((before.argmax(1) != after.argmax(1)).sum()), "kl": float((p * np.log(p / q)).sum(1).mean())}
def edit_and_measure(model, sets, base, u, share=1.0):
"""Remove `share` of direction u at every layer, run the held-out sets, measure."""
return measure(base, {name: forward(model, sets[name], remove(u, share))[0] for name in ("test_flagged", "test_plain")})
def angle(u, v):
return float(np.degrees(np.arccos(np.clip(abs(u @ v), 0, 1))))Show code
"""Prints the numbers the lesson text quotes beyond the trace. Run: python3 show_numbers.py"""
import numpy as np
from refusal_toy import study
from show import SHARES, run
default, spread, noisy, quiet = run(), run(second=0.3), run(noise=2.0), run(noise=0.5)
print("where the direction is read (default setting, all of it taken out)")
for layer in range(len(default["angle"])):
print(f" after layer {layer}: {default['angle'][layer]:5.1f} degrees off, refused {default['refused'][layer][-1]:3d} of 100, plain answers changed {default['changed'][layer][-1]:2d}, mean KL {default['kl'][layer][-1]:.3f}")
model, _, base = study()
for layer in (0, 5):
gap = base["train_flagged"][1][layer].mean(0) - base["train_plain"][1][layer].mean(0)
along = gap @ model["routes"][0]
print(f" after layer {layer}: the gap between the two averages is {along:.2f} along the planted direction and {np.linalg.norm(gap - along * model['routes'][0]):.2f} across it")
print("how much is taken out (read after layer 5)")
for i, share in enumerate(SHARES):
print(f" {share:3d}%: refused {default['refused'][5][i]:3d} of 100, plain answers changed {default['changed'][5][i]:2d}")
for name, r in (("a second route for 30 of 100", spread), ("high noise", noisy), ("low noise", quiet)):
print(f"{name} (read after layer 5, all taken out): {r['angle'][5]:.1f} degrees off, refused {r['refused'][5][-1]} of 100, plain answers changed {r['changed'][5][-1]}, mean KL {r['kl'][5][-1]:.3f}")Show code
"""Numbers shared by the widget, the figures and the lesson text. Nothing here changes the toy."""
from functools import lru_cache
from refusal_toy import LAYERS, angle, direction_at, edit_and_measure, study
SHARES = list(range(0, 101, 5)) # percent of the direction taken out
NOISE_WORDS, NOISE = ["low", "normal", "high"], [0.5, 1.0, 2.0]
SECOND = [0, 15, 30] # percent of flagged requests that use the second route
OPENING = {"share": 100, "layer": 5, "second": 0, "noise": 1} # the widget opens here: the default setting of trace.py, all of the direction taken out
def run(noise=1.0, second=0.0):
"""One setting of the toy: the direction found at every layer and what taking out each share does."""
model, sets, base = study(noise, second)
out = {"base": int((base["test_flagged"][0].argmax(1) == 0).sum()), "angle": [], "refused": [], "changed": [], "kl": []}
for layer in range(LAYERS + 1):
u = direction_at(base, layer)
results = [edit_and_measure(model, sets, base, u, share / 100) for share in SHARES]
out["angle"].append(round(angle(u, model["routes"][0]), 1))
out["refused"].append([r["flagged_refused"] for r in results])
out["changed"].append([r["changed"] for r in results])
out["kl"].append([r["kl"] for r in results])
return out
@lru_cache(maxsize=None)
def grid():
"""Every setting the widget offers: runs[noise][second]."""
return [[run(noise, second / 100) for second in SECOND] for noise in NOISE]
def headline(share, layer, base, refused, changed):
"""The one sentence the widget shows; widget.html builds the same words."""
if share == 0:
return f"Nothing taken out: {base} of 100 flagged requests are refused and no plain request changes its answer."
state = f"{refused} of 100 flagged requests are still refused" if refused == base else f"{refused} of 100 flagged requests are refused, down from {base}"
return f"Take out {share}% of the direction read after layer {layer} and {state}. Plain requests whose answer changes: {changed} of 100."
def describe(share, layer, second_index, noise_index):
"""What the widget shows for one setting; test_widget.py compares this with the page's own words for every setting."""
r = grid()[noise_index][second_index]
i = SHARES.index(share)
return {"head": headline(share, layer, r["base"], r["refused"][layer][i], r["changed"][layer][i]),
"angle": f"The direction found is {r['angle'][layer]:.1f} degrees from the planted one.",
"refused": r["refused"][layer], "changed": r["changed"][layer]}
def widget_config():
return {"shares": SHARES, "noise": NOISE_WORDS, "second": SECOND, "opening": OPENING,
"runs": [[{k: run_[k] for k in ("base", "angle", "refused", "changed")} for run_ in row] for row in grid()]}Show code
"""Prints the six steps of the lesson with real numbers. Run: python3 trace.py"""
import numpy as np
from refusal_toy import D, angle, direction_at, edit_and_measure, forward, measure, orthogonalize, remove, study
LAYER = 5 # read the direction after this layer (0 is before the first layer)
# Layer 5 is a late layer picked by hand; layers 4 to 6 tie on the held-out counts, and 6 reads nearer (6.5 against 8.4 degrees).
SECOND = 0.3 # share of flagged requests that use a second route, for the last line only
model, sets, base = study()
print(f"step 1 read the vectors after layer {LAYER}: {len(sets['train_flagged'])} flagged and {len(sets['train_plain'])} plain requests, {D} numbers each")
u = direction_at(base, LAYER)
print(f"step 2 difference of the two means, length 1: {angle(u, model['routes'][0]):.1f} degrees from the planted direction")
flagged_along = base["test_flagged"][1][LAYER] @ u
plain_along = base["test_plain"][1][LAYER] @ u
cut = (flagged_along.mean() + plain_along.mean()) / 2
sorted_right = int((flagged_along > cut).sum() + (plain_along <= cut).sum())
print(f"step 3 held-out distance along it: flagged {flagged_along.mean():.1f}, plain {plain_along.mean():.1f}; one cut at {cut:.1f} sorts {sorted_right} of 200 right")
hooked = {name: forward(model, sets[name], remove(u)) for name in ("test_flagged", "test_plain")}
length = lambda points: np.linalg.norm(points[-1], axis=1).mean()
print(f"step 4 taken out at all {len(hooked['test_plain'][1])} points: last-layer length of flagged vectors {length(base['test_flagged'][1]):.1f} -> {length(hooked['test_flagged'][1]):.1f}, plain {length(base['test_plain'][1]):.1f} -> {length(hooked['test_plain'][1]):.1f}")
edited_model = orthogonalize(model, u)
weights = {name: forward(edited_model, sets[name])[0] for name in ("test_flagged", "test_plain")}
print(f"step 5 same edit in the weights: largest difference in the answer scores {max(abs(weights[n] - hooked[n][0]).max() for n in weights):.10f}")
result = measure(base, weights)
print(f"step 6 flagged requests refused: {int((base['test_flagged'][0].argmax(1) == 0).sum())} of 100 before, {result['flagged_refused']} of 100 after; "
f"plain answers changed: {result['changed']} of 100 (mean KL {result['kl']:.3f})")
model2, sets2, base2 = study(second=SECOND)
result2 = edit_and_measure(model2, sets2, base2, direction_at(base2, LAYER))
print(f"limit with {round(SECOND * 100)} of 100 flagged requests on a second route, the same edit leaves {result2['flagged_refused']} of 100 refused")Show code
"""Guards for refusal_toy.py: the direction found against the planted one, the hook against the weight edit, the story the lesson tells, the limit case, and that every
number the lesson, its trace and its figures print is the one the code gives."""
import io
import re
import subprocess
import sys
import xml.etree.ElementTree as ET
from contextlib import redirect_stdout
from functools import lru_cache
from pathlib import Path
import numpy as np
import make_figs
import show
import refusal_toy as toy
from refusal_toy import angle, direction_at, edit_and_measure, forward, orthogonalize, remove, study
HERE = Path(__file__).resolve().parent
LESSON = HERE.parent
SOURCE = (HERE / "refusal_toy.py").read_text(encoding="utf-8")
LAYER = 5
@lru_cache(maxsize=None)
def toy_study(second=0.0):
model, sets, base = study(second=second)
return model, sets, base, direction_at(base, LAYER)
def refused_mask(model, f, hook=lambda x: x):
return forward(model, f, hook)[0].argmax(1) == 0
def test_direction_found_is_near_the_planted_one_and_sorts_new_requests():
"""Guards the method finding something other than the planted direction: the difference of means must land near it, and one cut along it must sort held-out requests."""
model, sets, base, u = toy_study()
assert angle(u, model["routes"][0]) < 10 and abs(np.linalg.norm(u) - 1) < 1e-12
flagged, plain = base["test_flagged"][1][LAYER] @ u, base["test_plain"][1][LAYER] @ u
cut = (flagged.mean() + plain.mean()) / 2
assert int((flagged > cut).sum() + (plain <= cut).sum()) == 200
def test_hook_and_weight_edit_agree_at_every_point_and_for_every_share_cut():
"""Guards the claim that cutting u from the weights is the same edit as taking it out of the vector: the vectors and the answer scores agree at all 7 points."""
model, sets, base, u = toy_study()
edited = orthogonalize(model, u)
for name in ("test_flagged", "test_plain"):
hooked_scores, hooked_points = forward(model, sets[name], remove(u))
scores, points = forward(edited, sets[name])
assert len(points) == toy.LAYERS + 1 and abs(scores - hooked_scores).max() < 1e-9
assert all(abs(a - b).max() < 1e-9 and abs(b @ u).max() < 1e-9 for a, b in zip(points, hooked_points))
W = model["blocks"][LAYER][2]
assert abs(orthogonalize(model, u)["blocks"][LAYER][2].T @ u).max() < 1e-12 and abs(W.T @ u).max() > 0.4 # the cut removes a part that was there
def test_the_story_reproduces():
"""Guards the lesson's claims: every flagged request refused before and none after, one plain answer in 100 changed, refusals falling in a narrow band of the share
taken out, a later layer finding the direction better, and a second route that one direction does not remove."""
model, sets, base, u = toy_study()
assert int(refused_mask(model, sets["test_flagged"]).sum()) == 100 and not refused_mask(model, sets["test_plain"]).any()
full = edit_and_measure(model, sets, base, u)
assert full["flagged_refused"] == 0 and full["changed"] == 1 and full["plain_refused"] == 0 and 0.005 < full["kl"] < 0.007
by_share = {share: edit_and_measure(model, sets, base, u, share / 100)["flagged_refused"] for share in (0, 10, 15, 20, 25, 30)}
assert by_share == {0: 100, 10: 100, 15: 90, 20: 44, 25: 5, 30: 0}
at0 = edit_and_measure(model, sets, base, direction_at(base, 0))
assert at0["flagged_refused"] == 13 and at0["changed"] == 8 and 37.9 == round(angle(direction_at(base, 0), model["routes"][0]), 1)
angles = [angle(direction_at(base, k), model["routes"][0]) for k in range(toy.LAYERS + 1)]
assert angles == sorted(angles, reverse=True)
model2, sets2, base2, u2 = toy_study(0.3)
left = refused_mask(model2, sets2["test_flagged"], remove(u2))
assert int(left.sum()) == 29 and int(left[:30].sum()) == 29 and not left[30:].any() # all that stay refused are on the second route
assert int(refused_mask(model2, sets2["test_flagged"]).sum()) == 100
def test_route_part_follows_the_layer_rule_in_the_lesson():
"""Guards the sentence in Go deeper that a refusal unit multiplies the part along the route by 1.5 and takes 0.25 off: the toy's averages must stay near it."""
model, sets, base, u = toy_study()
x, rule = toy.FLAG, []
for _ in range(toy.LAYERS + 1):
rule.append(x)
x = (1 + toy.GAIN) * x - toy.GAIN * toy.TAU
measured = [float((p @ model["routes"][0]).mean()) for p in base["test_flagged"][1]]
assert all(abs(a - b) < 0.2 for a, b in zip(rule, measured)) and [round(v, 2) for v in rule[:3]] == [1.5, 2.0, 2.75]
assert [round(v, 1) for v in rule[3:]] == [3.9, 5.6, 8.1, 11.9]
def reading():
prose = (LESSON / "lesson.md").read_text(encoding="utf-8")
return prose, {name: (LESSON / "figs" / name).read_text(encoding="utf-8") for name in make_figs.FIGURES}
def gaps(base, model, layer):
"""The gap between the two training averages along the planted direction and across it."""
d = base["train_flagged"][1][layer].mean(0) - base["train_plain"][1][layer].mean(0)
along = float(d @ model["routes"][0])
return along, float(np.sqrt(d @ d - along ** 2))
def test_every_number_the_lesson_and_its_figures_quote_comes_from_the_code():
"""Guards prose, trace and figure numbers drifting from the code: each fragment is built from a value computed here and must appear in the lesson or the figure."""
prose, figures = reading()
model, sets, base, u = toy_study()
grid = show.grid()[1] # normal noise: one run for each second-route share
run, second30 = grid[0], grid[2]
i = lambda share: show.SHARES.index(share)
ang = lambda k: run["angle"][k]
full = edit_and_measure(model, sets, base, u)
flagged, plain = base["test_flagged"][1][LAYER] @ u, base["test_plain"][1][LAYER] @ u
cut = (flagged.mean() + plain.mean()) / 2
length = lambda name: np.linalg.norm(base[name][1][-1], axis=1).mean()
hooked = {n: forward(model, sets[n], remove(u)) for n in ("test_flagged", "test_plain")}
hooked_length = lambda name: np.linalg.norm(hooked[name][1][-1], axis=1).mean()
a0, c0 = gaps(base, model, 0)
a5, c5 = gaps(base, model, LAYER)
lines = len(SOURCE.splitlines())
L5, L0 = run["refused"][LAYER][i(100)], run["refused"][0][i(100)]
wanted = {"lesson.md": [
"toy network of 100 lines of Python", f"`refusal_toy.py` is {lines} lines", f"All {run['base']} flagged requests",
f"and {full['changed']} of 100 plain answers changes", f"0 of 100 flagged requests are refused, down from {run['base']}", "none is, while 99 of 100 plain answers stay the same",
f"Share taken out to 20%: {run['refused'][LAYER][i(20)]} flagged requests are still refused", "between 10% and 30%",
f"the direction is found {ang(0)} degrees off the planted one, {L0} flagged requests are still refused and {run['changed'][0][i(100)]} plain answers change",
f"leaves {second30['refused'][LAYER][i(100)]} refused", f"Run {len(sets['train_flagged'])} flagged and {len(sets['train_plain'])} plain requests through the network",
f"keep each one's {toy.D} numbers after layer {LAYER}", f"It lands {ang(LAYER)} degrees from the planted one", f"{len(sets['test_flagged'])} flagged requests the step never saw sit at {flagged.mean():.1f} on average",
f"100 plain ones at {plain.mean():.1f}", f"One cut at {cut:.1f} sorts all {int((flagged > cut).sum() + (plain <= cut).sum())}", f"{len(hooked['test_plain'][1])} times",
f"Flagged requests refused: {run['base']} before, {full['flagged_refused']} after. Plain requests whose answer changes: {full['changed']} of 100",
f"The direction is found from {len(sets['train_flagged'])} and {len(sets['train_plain'])} requests and lands {ang(LAYER)} degrees", f"Flagged requests sit at {flagged.mean():.1f} along the direction and plain ones at {plain.mean():.1f}",
f"falls from {length('test_flagged'):.1f} to {hooked_length('test_flagged'):.1f} and plain ones stay at {hooked_length('test_plain'):.1f}", f"with a mean KL of {full['kl']:.3f}",
"nothing happens at 10%, and by 30% no flagged request is refused",
f"Put 30 of the 100 flagged requests on a second route, and the same edit leaves {second30['refused'][LAYER][i(100)]} refused",
f"{ang(0)} degrees off, {L0} flagged requests are still refused, and {run['changed'][0][i(100)]} plain answers change",
f"gave {ang(LAYER)} degrees, {L5} and {run['changed'][LAYER][i(100)]}", "while all 100 refusals went",
f"{toy.HIDDEN} hidden units", f"`TAU = {toy.TAU}`", f"`GAIN = {toy.GAIN}` times", f"`READ = {toy.READ:.0f}` times", f"`FLAG = {toy.FLAG}`",
"1.5, 2.0, 2.75, 3.9, 5.6, 8.1 and 11.9", f"After layer {LAYER} the gap is {a5:.2f} along r and {c5:.2f} across it, which is {ang(LAYER)} degrees",
f"After layer 0 it is {a0:.2f} along and {c0:.2f} across, which is {ang(0)} degrees", f"the 7 points",
f"gap along it is {a5:.2f} against {c5:.2f} across it",
f"differ by {a0:.2f} along the planted direction but by {c0:.2f} across it", f"their scores differ by {0:.10f}",
f"{full['flagged_refused']} of 100 flagged requests are refused and {full['changed']} of 100 plain answers changes",
f"Taking out 10% of the direction leaves {run['refused'][LAYER][i(10)]} of 100 flagged requests refused, and 30% leaves none",
f"{ang(0)} degrees off the planted one. After layer 5 it is {ang(LAYER)}",
f"the edit leaves {second30['refused'][LAYER][i(100)]} refused",
f"The direction found then sits {second30['angle'][LAYER]:.1f} degrees off the planted one", f"layer 6 sits nearer the planted direction, at {ang(6)} degrees",
]}
wanted.update({
"the-map.svg": [f"{len(sets['test_flagged'][:30])} flagged requests", f"a cut at {cut:.1f}", "along the direction"],
"weight-edit.svg": [f"{toy.HIDDEN} bars", f"all {toy.HIDDEN} columns: 0.00", f"the other {toy.HIDDEN - 1}"],
"share-removed.svg": [f"{run['refused'][LAYER][i(15)]} at 15%", f"{run['refused'][LAYER][i(20)]} at 20%", f"{run['refused'][LAYER][i(25)]} at 25%", f"{run['refused'][LAYER][i(10)]} at 10%", "0 from 30% on"],
"where-to-read.svg": [f"layer {k}: {ang(k)} degrees off, {run['refused'][k][i(100)]} refused, {run['changed'][k][i(100)]} changed" for k in range(toy.LAYERS + 1)],
"second-route.svg": [f"{second30['refused'][LAYER][i(100)]} are refused, {second30['refused'][LAYER][i(100)]} of them among the 30", f">{second30['refused'][LAYER][i(100)]} refused<"],
})
texts = {"lesson.md": prose, **figures}
missing = [f"{name}: {fragment!r}" for name, fragments in wanted.items() for fragment in fragments if fragment not in texts[name]]
assert not missing, "not found: " + "; ".join(missing)
assert lines < 150 and run["refused"][LAYER][i(30)] == 0
def test_the_figures_are_the_ones_the_code_draws_and_follow_the_house_style():
"""Guards a figure going stale or off-palette: the files on disk equal what make_figs draws, the lesson shows each one, and each parses, has a title and a description,
a white ground, no script, no web address, only the house colours and no text under 20 units."""
prose, figures = reading()
assert set(figures) == set(re.findall(r"\]\(figs/([\w-]+\.svg)\)", prose)) == set(make_figs.FIGURES)
allowed = {make_figs.INK, make_figs.GREY, make_figs.ORANGE, make_figs.GREEN, make_figs.FILL, make_figs.PANEL, "#ffffff"}
web_address = re.compile("ht" + r"tps?://")
for name, svg in figures.items():
assert svg == make_figs.FIGURES[name](), name
root = ET.fromstring(svg)
assert root.find("{%s}title" % make_figs.NAMESPACE) is not None and root.find("{%s}desc" % make_figs.NAMESPACE) is not None and 'fill="#ffffff"' in svg, name
assert "<script" not in svg and not web_address.search(svg.replace(make_figs.NAMESPACE, "")) and root.get("width") == "600", name
assert {c.lower() for c in re.findall(r"#[0-9a-fA-F]{6}\b", svg)} <= {c.lower() for c in allowed}, name
assert all(int(size) >= 20 for size in re.findall(r'font-size="(\d+)"', svg)), name
for path in HERE.glob("*.py"):
assert not web_address.search(path.read_text(encoding="utf-8").replace("ht" + "tp://www.w3.org", "")), path.name
def table_rows(prose):
return re.findall(r"^\| \d\. [^|]+\| `(\w+\.py):(\d+)`<br>`(.+?)` \|", prose, re.M)
def test_code_blocks_are_slices_of_the_real_file_and_the_table_points_at_its_lines():
"""Guards a `file:line` in the code table pointing at the wrong line, and a code block in Go deeper that no longer matches the file."""
prose, _ = reading()
blocks = re.findall(r"```python\n(.*?)\n```", prose, re.S)
assert len(blocks) == 4 and all(block in SOURCE for block in blocks)
rows = table_rows(prose)
assert [(name, int(n)) for name, n, _ in rows] == [("refusal_toy.py", 45), ("refusal_toy.py", 51), ("trace.py", 14), ("refusal_toy.py", 57), ("refusal_toy.py", 62), ("refusal_toy.py", 87)]
for name, number, code in rows:
line = (HERE / name).read_text(encoding="utf-8").splitlines()[int(number) - 1]
assert re.sub(r"\s+# .*$", "", line).strip() == code, (name, number)
def run_trace(source):
out = io.StringIO()
with redirect_stdout(out):
exec(compile(source, "trace.py", "exec"), {"__name__": "__main__"})
return out.getvalue()
def test_trace_is_the_block_in_the_lesson_and_the_your_turn_answer_is_right():
"""Guards the pasted run drifting from `python3 trace.py` (the real command, from the code folder) and the Your turn answer: changing line 6 to LAYER = 0 must give
the lines in the details block."""
prose, _ = reading()
real = subprocess.run([sys.executable, "-B", "trace.py"], cwd=HERE, capture_output=True, text=True, check=True).stdout
pasted = re.search(r"python3 trace\.py\n```\n\nPhone readers: slide the box sideways to see every column\.\n\n```text\n(.*?)\n```", prose, re.S).group(1)
assert real == pasted + "\n"
trace = (HERE / "trace.py").read_text(encoding="utf-8")
assert trace.splitlines()[5] == "LAYER = 5 # read the direction after this layer (0 is before the first layer)" and "change line 6 to `LAYER = 0`" in prose
changed = run_trace(trace.replace("LAYER = 5 ", "LAYER = 0 ", 1))
answer = re.search(r"<summary>What you should see</summary>.*?```text\n(.*?)\n```", prose, re.S).group(1)
assert answer + "\n" == changed and "13 of 100 after" in answer
assert [a == b for a, b in zip(real.splitlines(), changed.splitlines())] == [False, False, False, False, True, False, False]Show code
"""Guards for ../widget.html: its table of numbers, that its words and numbers equal show.py's for every setting, and what a reader sees and can press."""
import json
import re
import subprocess
import show
from embed_widget import CONFIG_BLOCK, WIDGET, embed
from refusal_toy import LAYERS
HTML = WIDGET.read_text(encoding="utf-8")
# Every setting the four sliders offer: share taken out x layer read x second route x request noise.
GRID = [(share, layer, second, noise) for share in show.SHARES for layer in range(LAYERS + 1) for second in range(len(show.SECOND)) for noise in range(len(show.NOISE))]
# Reads the model out of the widget between its marker comments, runs every setting with Node and compares what the page would show with what show.py gives, exactly.
PARITY = r"""
import fs from 'node:fs';
const [fixture, widget] = process.argv.slice(2);
const fx = JSON.parse(fs.readFileSync(fixture, 'utf8'));
const html = fs.readFileSync(widget, 'utf8');
const cfg = JSON.parse(html.match(/<script type="application\/json" id="rf-config">([\s\S]*?)<\/script>/)[1]);
const js = html.slice(html.indexOf('/* ---------- the model ('), html.indexOf('/* ---------- display'));
const api = new Function('cfg', "'use strict';" + js + '; return {describe};')(cfg);
const bad = [];
fx.cases.forEach((c) => {
const got = api.describe(...c.args);
if (JSON.stringify(got) !== JSON.stringify(c.want)) bad.push([c.args, JSON.stringify(got).slice(0, 300), JSON.stringify(c.want).slice(0, 300)]);
});
console.log(JSON.stringify({ checks: fx.cases.length, bad: bad.slice(0, 3) }));
process.exit(bad.length ? 1 : 0);
"""
# Runs the page script against a small stand-in for the DOM, moves the sliders and reads back what a reader sees.
PAGE = r"""
import fs from 'node:fs';
const [widget, fixture] = process.argv.slice(2);
const html = fs.readFileSync(widget, 'utf8');
const fx = JSON.parse(fs.readFileSync(fixture, 'utf8'));
const config = html.match(/<script type="application\/json" id="rf-config">([\s\S]*?)<\/script>/)[1];
const code = html.match(/<script>([\s\S]*?)<\/script>/)[1];
function el(tag) {
const e = { tag, children: [], listeners: {}, value: '', style: {}, attrs: {}, textContent: '', innerHTML: '' };
e.setAttribute = (k, v) => { e.attrs[k] = v; };
e.getAttribute = (k) => e.attrs[k];
e.addEventListener = (type, f) => (e.listeners[type] = e.listeners[type] || []).push(f);
return e;
}
const page = { root: el('html'), messages: [], parent: { other: true } };
[...html.matchAll(/id="(rf-[^"]+)"/g)].forEach((m) => { page[m[1]] = el('x'); });
page['rf-config'].textContent = config;
const doc = { getElementById: (id) => page[id], documentElement: page.root, createElement: el };
const win = { addEventListener: (type, f) => page.messages.push(f) };
new Function('document', 'window', 'parent', code)(doc, win, page.parent);
const fire = (e, type) => (e.listeners[type] || []).forEach((f) => f({ target: e }));
const bad = [];
const want = (what, got, expected) => { if (JSON.stringify(got) !== JSON.stringify(expected)) bad.push([what, JSON.stringify(got).slice(0, 300), JSON.stringify(expected).slice(0, 300)]); };
const shown = () => [page['rf-line'].textContent, page['rf-angle'].textContent];
const set = (share, layer, second, noise) => { page['rf-share'].value = String(share); page['rf-layer'].value = String(layer); page['rf-second'].value = String(second); page['rf-noise'].value = String(noise); };
const cfg = JSON.parse(config);
want('opening controls', [page['rf-share'].value, page['rf-layer'].value, page['rf-second'].value, page['rf-noise'].value], fx.opening.controls);
want('opening state', shown(), fx.opening.shown);
want('nothing spoken yet', page['rf-live'].textContent, '');
// The picture, read back from the SVG text: the current-share mark sits on the right edge of the plot at the opening, the big dot and square sit on the data point the
// line passes through, there is one small mark per share on each line, and nothing is drawn past the edge of the viewBox.
const picture = () => {
const svg = page['rf-chart'].innerHTML;
const [, w, h] = svg.match(/viewBox="0 0 (\d+) (\d+)"/).map(Number);
const line = (color) => svg.match(new RegExp('<path d="M([^"]+)" fill="none" stroke="var\\(--' + color + '\\)"'))[1].split('L').map((p) => p.split(' ').map(Number));
const mark = svg.match(/<path d="M([\d.]+) 80V190" stroke="var\(--ink\)"/);
const big = svg.match(/<circle cx="([\d.]+)" cy="([\d.]+)" r="6"/);
const bigSquare = svg.match(/<rect x="([\d.]+)" y="([\d.]+)" width="11.0"/);
const coords = [...svg.matchAll(/(?:cx|x)="([\d.]+)"[^>]*?(?:cy|y)="([\d.]+)"/g)].map((m) => [+m[1], +m[2]]);
return { w, h, orange: line('orange'), green: line('green'), markX: mark && +mark[1], big: big && [+big[1], +big[2]], bigSquare: bigSquare && [Math.round((+bigSquare[1] + 5.5) * 10) / 10, Math.round((+bigSquare[2] + 5.5) * 10) / 10],
smallDots: (svg.match(/r="3.5"/g) || []).length - 1, smallSquares: (svg.match(/width="6.0"/g) || []).length - 1, inside: coords.every(([x, y]) => x <= w && y <= h) };
};
const seen = (k) => { // the k-th share's point on both lines is where the big marks are
const p = picture();
return { size: [p.w, p.h], points: [p.orange.length, p.green.length], marksOnTheLines: JSON.stringify([p.big, p.bigSquare]) === JSON.stringify([p.orange[k], p.green[k]]) && p.markX === p.orange[k][0],
dots: [p.smallDots, p.smallSquares], inside: p.inside };
};
const all = cfg.shares.length;
want('opening chart', seen(all - 1), { size: [320, 242], points: [all, all], marksOnTheLines: true, dots: [all, all], inside: true });
set(20, 5, 0, 1);
fire(page['rf-share'], 'input');
want('dragging shows the new state', shown(), fx.dragged.shown);
want('chart follows the share', seen(4), { size: [320, 242], points: [all, all], marksOnTheLines: true, dots: [all, all], inside: true });
want('dragging stays silent', page['rf-live'].textContent, '');
fire(page['rf-share'], 'change');
want('release speaks', page['rf-live'].textContent, fx.dragged.shown[0]);
want('words on the controls', [page['rf-share-out'].textContent, page['rf-layer-out'].textContent, page['rf-second-out'].textContent, page['rf-noise-out'].textContent].map(String).concat([page['rf-share'].attrs['aria-valuetext'], page['rf-layer'].attrs['aria-valuetext'], page['rf-second'].attrs['aria-valuetext'], page['rf-noise'].attrs['aria-valuetext']]),
['20%', '5', '0%', 'normal', '20 percent of the direction taken out', 'read after layer 5', '0 percent of flagged requests use a second route', 'request noise normal']);
fx.others.forEach((c) => {
set(...c.args);
fire(page['rf-noise'], 'input');
want('state ' + c.args.join('/'), shown(), c.shown);
});
set(0, 5, 0, 1);
fire(page['rf-share'], 'input');
want('nothing taken out', shown()[0].startsWith('Nothing taken out: 100 of 100'), true);
const theme = (source, name) => page.messages.forEach((f) => f({ data: { type: 'spicyshark:theme', theme: name }, source }));
theme({ other: 'frame' }, 'dark');
want('theme from another sender is ignored', page.root.attrs['data-theme'], undefined);
theme(page.parent, 'dark');
want('theme from the host page', page.root.attrs['data-theme'], 'dark');
console.log(JSON.stringify({ bad: bad.slice(0, 3) }));
process.exit(bad.length ? 1 : 0);
"""
def run_node(tmp_path, script, *args):
path = tmp_path / "check.mjs"
path.write_text(script, encoding="utf-8")
done = subprocess.run(["node", str(path), *args], capture_output=True, text=True, timeout=120)
assert done.returncode == 0, done.stdout + done.stderr
return json.loads(done.stdout)
def shown(*setting):
"""What the page shows for one setting, in the form the PAGE script reads back."""
d = show.describe(*setting)
return [d["head"], d["angle"]]
def test_widget_data_block_matches_the_python_module():
assert embed(HTML) == HTML
assert json.loads(CONFIG_BLOCK.search(HTML).group(2)) == json.loads(json.dumps(show.widget_config()))
for name, count in (("share", len(show.SHARES)), ("layer", LAYERS + 1), ("second", len(show.SECOND)), ("noise", len(show.NOISE))): # each slider walks the stops the grid covers
top = {"share": show.SHARES[-1], "layer": LAYERS}.get(name, count - 1)
step = 5 if name == "share" else 1
assert re.search(r'id="rf-%s" min="0" max="%d" step="%d"' % (name, top, step), HTML), name
def test_widget_gives_the_same_numbers_and_words_as_python(tmp_path):
cases = [{"args": list(g), "want": show.describe(*g)} for g in GRID]
heads = " ".join(c["want"]["head"] for c in cases)
assert "Nothing taken out" in heads and "still refused" in heads and ", down from " in heads # the grid reaches every kind of sentence
(tmp_path / "fx.json").write_text(json.dumps({"cases": cases}), encoding="utf-8")
assert run_node(tmp_path, PARITY, str(tmp_path / "fx.json"), str(WIDGET))["checks"] == len(GRID)
def test_widget_page_opens_on_the_point_and_follows_the_controls(tmp_path):
opening = (show.OPENING["share"], show.OPENING["layer"], show.OPENING["second"], show.OPENING["noise"])
head = show.describe(*opening)["head"]
assert "0 of 100 flagged requests are refused, down from 100" in head and "Plain requests whose answer changes: 1 of 100" in head # the first screen makes the point
others = [(50, 5, 0, 1), (100, 0, 0, 1), (100, 5, 2, 1), (100, 5, 0, 2), (100, 5, 0, 0), (100, 6, 1, 1)]
fixture = {"opening": {"controls": [str(v) for v in opening], "shown": shown(*opening)}, "dragged": {"shown": shown(20, 5, 0, 1)},
"others": [{"args": list(o), "shown": shown(*o)} for o in others]}
(tmp_path / "fx.json").write_text(json.dumps(fixture), encoding="utf-8")
run_node(tmp_path, PAGE, str(WIDGET), str(tmp_path / "fx.json"))
def test_widget_is_small_themed_and_uses_only_the_site_palette():
body = HTML[HTML.index('<div class="rf-root">'):HTML.index("<script")]
visible = re.sub(r"<details.*?</details>", "", body, flags=re.S)
assert len(re.findall(r"<input|<select|<button", visible)) <= 5 and "More settings" in body and "A toy network and invented requests" in visible # at most five controls on show, caveat always on show
longest = {"share": f"{show.SHARES[-1]}%", "layer": str(LAYERS), "second": f"{max(show.SECOND)}%", "noise": max(show.NOISE_WORDS, key=len)}
labels = re.findall(r'<label class="rf-lab" for="rf-(\w+)">(.*?)</label>', body)
assert [name for name, _ in labels] == list(longest)
for name, label in labels: # a label and its longest value fit one line at 340 px
text = re.sub(r"<[^>]+>", "", label) + longest[name]
assert len(text) <= 22, text
assert len(HTML.splitlines()) < 300 and "light-dark(#ffffff, #0a0f17)" in HTML and "spicyshark:theme" in HTML and "prefers-reduced-motion" in HTML
assert "forced-colors: active" in HTML and 'role="status" aria-live="polite"' in HTML
assert not re.search(r"0072B2|56B4E9|D55E00|ht" r"tps?://|<img|@import", HTML, re.I)
legend_y = [int(y) for y in re.findall(r'<text class="rf-t" x="36" y="(\d+)">', HTML)]
assert len(legend_y) == 2 and max(legend_y) < 80 # the legend sits above the plot top (80), off the curves (round 1 review)
sizes = [int(px) for px in re.findall(r"\.rf-[kt] \{ font: (?:600 )?(\d+)px", HTML)]
assert len(sizes) == 2 and min(sizes) >= 15 # chart text 15 units or more, about 12.5 px in the 290 px frame at 340