--- title: "Is it safe to download a model file?" description: "Some model files are programs in disguise: opening an old PyTorch file can run code. Files that hold only numbers are built not to. Here is how to tell them apart." topic: "AI systems" chapter: "trust-the-answers" order: 17 minutes: { plain: 6, deep: 16 } prerequisites: ["Reading short Python functions", "Bytes and hexadecimal (one byte is two hex digits)"] seo_title: "Is it safe to download an AI model file?" date: 2026-10-19 --- # Is it safe to download a model file? A model file is either a list of numbers or a list of steps to follow, and that decides what opening it can do. In a demo built for this lesson, the same six numbers are saved two ways. Opening the 158-byte pickle file (Python's way of saving objects) makes Python call 1 function: the file has 28 steps and the call is step 25. Opening the 100-byte safetensors file (a format that holds a label and numbers) calls 0 functions, in 3 steps of reading. The demo's function only writes down a note, but a real pickle can name any function that Python can import (find and make ready). ## In one minute A file that holds only numbers has nothing to run, but a pickle file can run code on your computer the moment you open it. A model is a big set of numbers (its weights, the values it learned). Think of a box with a packing list: you read the list and take the numbers out, and nothing in the box can act. Safetensors files work like that. A pickle file is closer to assembly instructions: you follow each step, and one step can say "now go and do this other job". Many older PyTorch files are pickles, often named .bin, .pt or .pth (the PyTorch and Hugging Face pages below use those names), though a name does not prove the format. So prefer files that hold only numbers, load PyTorch files in weights-only mode (a setting that limits what the file may build while it opens), leave remote code (code a repository ships with its model) off unless you have read it, and use a publisher you trust. ## Explore the numbers [Open the widget](https://spicyshark.co/learn/is-it-safe-to-download-a-model-file/widget.html). It starts with both files fully opened. Drag the step slider back to 0 and then forward: two pickle steps turn orange, the import and the call, and the safetensors file finishes in 3 steps. Then tick "Restricted loader" and see where the pickle stops. (A loader is the program that opens a file and puts its contents in memory.) The widget shows what Python's loader would do with the demo's bytes. It only reads them and writes words; it runs nothing itself. ## How it works ### Two ways to open a file The demo saves one table of six numbers, named `layer.weight`, with 2 rows and 3 columns. ![Two lists. A pickle file is five groups of steps to follow, and two of them, an import and a call, run a function. A safetensors file is three parts to read: a length, a header and the numbers.](https://spicyshark.co/learn/is-it-safe-to-download-a-model-file/figs/two-ways-to-open.svg) A safetensors file is read in 3 steps, and each only reads bytes. A pickle is read as 28 steps, which Python calls opcodes; the picture groups them into five. Most build things, such as a dictionary, a list or a number. Two do more. Step 19 names a function and asks the loader to import it (find it and make it ready). Step 25 calls it (makes it run). ### What a pickle can do while it opens Python's `pickle` module saves an object as the steps that rebuild it. An object can say how: "to rebuild me, call this function with these inputs". The demo does that once, with a function that only adds a note to a list. Opening the file adds 1 note. The Python documentation (read 2026-10-19) says the module "is not secure" and "Only unpickle data you trust". The file holds the function's name, so whoever writes the file chooses it: any function Python can import, including ones that start programs or read your files. It is the same pattern as [prompt injection](https://spicyshark.co/learn/prompt-injection/) (lesson 14): text from a stranger that a program takes for orders, here a loader instead of a model. ### Files that hold only numbers A safetensors file holds its header length, a header, then the numbers. The header is 68 bytes of text that says the table's name, its kind of number, its shape and where its bytes sit. The loader reads it and takes the 24 bytes it points to. No step names a function, so the file cannot ask for one. The format's README (read 2026-10-19) defines "safe" as being able to use a file "randomly downloaded" and "expect not to run arbitrary code". GGUF, a second format that holds only numbers, has the same shape: a header, a list of tables, then the numbers. A reader of either format is still a program that reads bytes from a stranger, so it must check them. The demo's reader refuses a header longer than the file, and tables that overlap or run past the end. ### Closing the door in the loader You can keep pickle and limit it. A restricted loader checks each import against a short list and stops at any name not on it (the Python documentation calls this "Restricting Globals"). PyTorch's weights-only mode follows a similar idea with its own, separate unpickler. Its notes (read 2026-10-19) say that unpickler "is not allowed to dynamically import anything during unpickling" and let you add more globals to its allowed list. The class in Go deeper is the Python documentation's "Restricting Globals" example, not PyTorch's code. ![A flow chart. The loader meets an import and asks whether the name is on a short list. If yes, it imports it and carries on. If no, it stops with an error and nothing is called.](https://spicyshark.co/learn/is-it-safe-to-download-a-model-file/figs/restricted-loader.svg) In the demo the list has 5 names. The loader stops at step 19, before the call at step 25, and the note list stays empty. PyTorch's notes say the mode "narrows the surface of remote code execution attacks" and list limits, so it narrows the risk and is not a wall. ### The demo and the real thing The demo is small and its one function is harmless. Real files are much larger and hold many tables, and a real bad file would not be harmless. ## What this means when you build - **Prefer safetensors or GGUF files.** They hold a label and numbers, so the loader has nothing to run. Hugging Face's Transformers library, which loads models in code, uses safetensors weights when a repository has them. - **Load PyTorch files in weights-only mode.** Pass `weights_only=True` to `torch.load`. It is the default since version 2.6 unless you pass your own pickle module, and PyTorch's page still says never to load data from an untrusted source. - **Leave `trust_remote_code` off unless you have read the code.** It lets a repository bring its own model code. Pin a specific revision (a fixed version of the repository) so that code cannot change under you. - **Check who published the file.** Hugging Face says its own pickle scan "is not 100% foolproof", so a scan does not replace trust. - **Give the files around the weights the same care.** Configuration, tokenizer and code files come with the weights, and number-only weights do not make them safe. ## Go deeper *Optional, for readers who write Python. The code is in the `code/` folder: `modelfile.py` writes and reads both files, `harmless.py` holds the one function the demo calls, and `test_modelfile.py` and `test_widget.py` are the tests. `python3 show_numbers.py` prints every number above.* ### The one function the demo calls The demo's pickle calls exactly one function, kept in its own file so you can see it has nothing else in it: ```python def ran_on_open(note): """Called by the loader while it opens the demo pickle; keeps the note in NOTES and hands it back.""" NOTES.append(note) return note ``` `harmless.py` imports nothing, holds one list called `NOTES` and this function. A test reads the file's syntax tree and fails if that stops being true. No file in the lesson folder holds a pickle: `demo_pickle()` builds the bytes when the code runs. ### Why one object makes a call ```python class Opens: """An object whose pickle says: to rebuild me, call ran_on_open with this note.""" def __reduce__(self): return (harmless.ran_on_open, (NOTE,)) def demo_pickle(): """The six numbers plus one object that calls ran_on_open when the file is opened.""" return pickle.dumps({NAME: list(VALUES), "note": Opens()}, protocol=PICKLE_PROTOCOL) ``` When Python saves an object it asks for `__reduce__`, which returns a function and a tuple of inputs for it (the Python documentation, read 2026-10-19, describes it as a callable and a tuple of arguments). Pickle writes down the function's module and name and the inputs. It does not write the class. So the file says "import `harmless.ran_on_open`, call it with this text", and the loader does what the file says. Whoever writes the bytes picks the function. That is the whole mechanism, and the demo uses it for the smallest harmless job there is. ### The 28 steps of the demo pickle `opcodes` reads the steps with `pickletools.genops`, which lists them without running anything, and `kind` marks which ones import and which ones call: ```python def opcodes(data): """[(position, name, argument)] for every opcode, read with pickletools.genops. Nothing is executed.""" return [(pos, op.name, arg) for op, arg, pos in pickletools.genops(data)] def kind(name): return "import" if name in IMPORTS else "call" if name in CALLS else "build" ``` The demo uses pickle protocol 3, which writes the import as one `GLOBAL` step with the two names as plain text. These are the steps that matter: | Step | Action | Opcode | |---|---|---| | 5 | push the first key | BINUNICODE | | 10 | push the six numbers (steps 10 to 15) | BINFLOAT | | 16 | put them in the list | APPENDS | | 17 | push the second key | BINUNICODE | | 19 | import the demo's function | GLOBAL | | 21 | push the note's text | BINUNICODE | | 23 | wrap it as the inputs | TUPLE1 | | 25 | call the function | REDUCE | | 27 | fill the dictionary | SETITEMS | | 28 | hand the dictionary back | STOP | The other 13 steps set up the dictionary and the list, and remember items; 8 of them are `BINPUT`. Position in the file runs from byte 0 to byte 157, which with the last step's 1 byte makes the 158 in the headline. The picture shows the stack, the loader's pile of items, at the import and at the call: ![Three snapshots of the loader's stack. After the import step the stack holds the function ran_on_open. After the note text is pushed and wrapped, it holds the function and a one-item tuple of inputs. After the call step it holds the result, and one note has been recorded. Only the top of the stack is shown.](https://spicyshark.co/learn/is-it-safe-to-download-a-model-file/figs/pickle-stack.svg) Hugging Face's page on pickle scanning (read 2026-10-19) calls a pickle "basically a stack of instructions or opcodes" and says "The instructions that pose a threat are `STACK_GLOBAL`, `GLOBAL` and `REDUCE`". Protocol 4 writes the import as `STACK_GLOBAL`, which takes its two names from items pushed earlier. The call is `REDUCE` in the demo; the code also lists `OBJ`, `NEWOBJ` and `NEWOBJ_EX` as steps that call. ### The safetensors file, byte by byte ```python def write_safetensors(tensors, metadata=None): """tensors is {name: (dtype, shape, values)}. Row-major, little-endian, no padding.""" header, body = {}, b"" if metadata: header["__metadata__"] = dict(metadata) for name, (dtype, shape, values) in tensors.items(): code, _ = DTYPES[dtype] assert len(values) == _count(shape) raw = struct.pack(f"<{len(values)}{code}", *values) header[name] = {"dtype": dtype, "shape": list(shape), "data_offsets": [len(body), len(body) + len(raw)]} body += raw text = json.dumps(header, separators=(",", ":")).encode("utf-8") return struct.pack(" MAX_HEADER or 8 + n > len(data): raise FormatError("header length is larger than the file or the 100 MB cap") raw = data[8:8 + n] if not raw.startswith(b"{"): raise FormatError("header does not begin with {") ``` Later checks refuse repeated names, a byte range that does not fit its shape, and (after sorting the ranges) a table that overlaps another, leaves a hole, or leaves bytes after the last one unaccounted for: ```python expected = 0 for begin, end in sorted(spans): if begin != expected: raise FormatError("tensors overlap or leave a hole") expected = end if expected != len(body): raise FormatError("bytes after the last tensor are not indexed") return tensors, metadata ``` The README requires the numbers to be fully indexed, with no holes, and names polyglot files (one file valid as two formats) as what that prevents. A test feeds the reader 26 broken files and checks that each one is refused. ### The restricted loader and the scanner ```python class Restricted(pickle.Unpickler): """The 'Restricting Globals' pattern from the Python pickle documentation: only the allowed names may be imported.""" def find_class(self, module, name): if (module, name) in ALLOWED: return super().find_class(module, name) raise pickle.UnpicklingError(f"global '{module}.{name}' is forbidden") def load_restricted(data): return Restricted(io.BytesIO(data)).load() ``` Python calls `find_class` whenever a file asks for a name, so this is the one place to say no. `ALLOWED` holds five pairs: `range`, `set`, `frozenset`, `complex` and `slice`, from the Python documentation's example. Run on the demo pickle it raises `global 'harmless.ran_on_open' is forbidden` at step 19, and the note list is empty. Run on a pickle that holds only the six numbers it returns them, with no import at all. The scanner reads the same steps without loading anything and reports each import that is not on the list: ```python def scan(data): """What a name-based scanner flags: every import not on the allowed list, as 'module.name', or 'unreadable'.""" return [f"{m}.{n}" if m is not None else "unreadable" for _, m, n in imports(data) if m is None or (m, n) not in ALLOWED] ``` On the demo it returns `['harmless.ran_on_open']`, and on the numbers-only pickle `[]`. It checks names, not intent: it flags this function though the function is harmless. Where the names do not sit as plain text just before an import, it reports "unreadable" and does not pass it. Hugging Face says its scan is "not 100% foolproof" and that its lists of safe and unsafe names are kept "in a best-effort manner". A name list is a way to warn a person, and it does not replace a trusted source. ### Proof it works ```bash cd lessons/is-it-safe-to-download-a-model-file/code python3 -m pytest -q ``` The tests cover: opening the demo pickle records 1 note and gives back the six numbers; opening the safetensors file records 0 notes, even when a name in its header reads like a function call; the safetensors reader refuses the pickle's bytes and 26 broken files; the restricted loader refuses the demo pickle at step 19 with 0 notes, in protocols 3 and 4, yet loads a pickle of `range(3)`; the opcode list has 28 steps with 1 import and 1 call; the scanner flags the demo by name and flags an unreadable import; and the numbers 158, 100, 68, 28, 19 and 25. One more test reads `harmless.py` and fails if it imports anything or does more than the one job, and one reads every file in the folder for words that start programs or run text and for stored model files. The lesson's code blocks are checked as verbatim excerpts of the source. A second file, `test_widget.py`, runs the widget's JavaScript reader in Node.js (a way to run JavaScript outside a browser) on five pickles and two safetensors files, and checks that its steps and its answers equal Python's. That file is the only one in the folder that starts another program, and it starts only Node.js, on the widget's own script. ### What the demo leaves out - **One small table.** Real checkpoints hold many tables and are much larger. - **No zip archive.** Since PyTorch 1.6.0, `torch.save` writes a ZIP64 archive: a `data.pkl` file for the pickle, and separate files for the numbers (PyTorch's serialization notes, read 2026-10-19). Weights-only loading restricts the unpickler (the notes' words); the lesson's reading is that it reads the `data.pkl` part, which the notes do not say outright. - **A list of 5 names.** The demo's rule is an allowlist of 5 imports. PyTorch's documented rule differs: its weights-only unpickler "is not allowed to dynamically import anything during unpickling" and builds only what state dicts of "plain torch.Tensors" and "some other primitive types" need, and `add_safe_globals` "Marks the given globals as safe for weights_only load". This lesson does not read PyTorch's code. - **A mode with limits.** The notes say `weights_only=True` "narrows the surface of remote code execution attacks" and does not guard against denial of service (a file that makes the loader use up memory or time). They also say memory corruption, which PyTorch tries to prevent, "might still be possible". - **GGUF.** Its specification (read 2026-10-19) puts a header first (a magic word "GGUF", a version, counts, key-value metadata), then a description of each table, padding to an alignment (32 by default), then the numbers. The demo does not write it. - **Bugs in readers.** Any reader of any format can have bugs. This lesson cites no security audit of safetensors or GGUF. - **The attack itself.** The demo calls a function that writes a note. How a bad file is built is outside this lesson. ### Where this is used in practice The Python documentation (read 2026-10-19) says that unpickling imports any class or function a file asks for by default, which "permits the unpickler to import and invoke arbitrary code", and shows `find_class` as the place to restrict it. It suggests formats such as JSON for untrusted data and signing data with `hmac`. PyTorch's `torch.load` page (version 2.14 docs, read 2026-10-19) says it "uses an unpickler under the hood" and carries the warning "Never load data from an untrusted source". With `weights_only` on, the unpickler is restricted to "tensors, primitive types, dictionaries and any types added via torch.serialization.add_safe_globals()". Its serialization notes say that from version 2.6 `weights_only=True` is used "if the pickle_module argument is not passed". The safetensors README (read 2026-10-19) says the format exists to stop a file from running code, and lists pickle as "Unsafe, runs arbitrary code". It also claims that it is "almost impossible to use malicious files to DOS attack a user". That is the project's own claim, and no independent audit is cited here. The Transformers documentation (v5.20.2 in the page, read 2026-10-19) says that when a repository has safetensors weights, `from_pretrained()` loads them, and that "Traditionally, PyTorch model weights are serialized with the pickle utility which is known to be insecure. Safetensor files are more secure and faster to load." For custom model code it asks for `trust_remote_code=True` and suggests a pinned revision as an extra layer. The common thread: pick a format with nothing to run, narrow the loader when you cannot, and know who published the file. This lesson's demo matches that reading. ## Skills this lesson builds - Explaining why opening a pickle can run a function and opening a safetensors file cannot, in terms of steps to follow and data to read. - Listing a pickle's opcodes with `pickletools.genops` and marking the import and call steps without loading the file. - Writing a restricted unpickler with a short list of names and testing that it refuses anything else. - Writing a reader for a binary format that checks its input (lengths, offsets, overlaps) before it trusts it. - Choosing a model file format and loader settings by what each can do when it opens. **Portfolio project:** write a small tool that takes the bytes of a pickle, lists its opcodes without loading it, and prints each import step with a verdict from your own short list. Test it on pickles you build yourself from one harmless function, publish a table of what it flags and what it cannot read, and say in the README what a check on names cannot tell you. ## Check yourself 1. Why can opening a pickle run a function when opening a safetensors file cannot?
Answer A pickle is a list of steps for the loader to follow, and two kinds of step import a function and call it. A safetensors file is a length, a header and numbers: the loader unpacks them, and no step in the layout names a function.
2. The restricted loader stops the demo pickle at step 19. Why is that early enough, and why would stopping at step 25 be too late?
Answer Step 19 is the import that names the function, and step 25 is the call. Stopping at the import means the call never happens and the note list stays empty. Stopping at the call is too late: the function would already be running. Importing a module can itself run that module's code, so the import step is the doorway.
3. A file is named `model.safetensors`. Name two things you should still check.
Answer Any two of: that its bytes really are that format (a name does not prove it, and a reader that checks its input will refuse a bad header); who published it; and the configuration, tokenizer and code files in the same repository, especially if `trust_remote_code` is on.
4. What does `weights_only=True` do in `torch.load`, and why is it not a guarantee?
Answer This describes PyTorch's documented rule, not the demo's list of 5 imports. The unpickler is restricted to tensors, primitive types and dictionaries, plus types the caller adds, and is not allowed to import anything dynamically. PyTorch's own notes say it narrows the surface of remote code execution attacks and list limits: no guard against denial of service, and memory corruption "might still be possible". The `torch.load` page still says never to load data from an untrusted source.
5. The scanner flags `harmless.ran_on_open`, though that function is harmless. Is that a bug?
Answer No. The scanner checks names against a list, not what a function does, so it flags anything that is not on the list and leaves the decision to a person. It also flags an import whose names it cannot read, and Hugging Face says such scans are "not 100% foolproof".
## Further reading Web pages were read on 2026-10-19. **Gentle** - [Loading models](https://huggingface.co/docs/transformers/en/models) (Hugging Face) - [Pickle Scanning](https://huggingface.co/docs/hub/security-pickle) (Hugging Face) - [Serialization semantics](https://docs.pytorch.org/docs/2.14/notes/serialization.html) (PyTorch) **Deep** - [pickle: Python object serialization](https://docs.python.org/3/library/pickle.html) (Python Software Foundation) - [torch.load](https://docs.pytorch.org/docs/2.14/generated/torch.load.html) (PyTorch) - [safetensors README](https://github.com/safetensors/safetensors) (safetensors, GitHub) - [GGUF file format specification](https://github.com/ggml-org/ggml/blob/master/docs/gguf.md) (ggml-org, GitHub)