Trust the answers

Know how an AI app goes wrong, leaks or is tricked, and the plain fix.

  1. Why does AI make things up?

    A language model writes the likely next words, not checked facts. Where its training text says little, a fluent answer can be wrong, and it rarely says so.

    6 min+11 min deeper

  2. How do I check my AI's answers with a small test set?

    Keep about 20 questions with known answers and run all of them after every change. A fix for one answer can break others that the score alone hides.

    6 min+15 min deeper

  3. What is prompt injection, and can a prompt stop it?

    A web page, email or file can hide text that an AI app takes for orders. Words in a prompt do not reliably stop it; limits in code cap the harm.

    6 min+14 min deeper

  4. How does a safety filter decide, and where does it fail?

    A safety filter scores each message and blocks it at a threshold. Every threshold misses some harm or blocks some harmless messages, and rare harm changes the mix.

    6 min+15 min deeper

  5. Can one user's chat leak into another's?

    It can, but rarely through the model. Leaks come from a shared cache whose key omits who asked, from logs, or mixed-up requests. Put the user in the key.

    6 min+18 min deeper

  6. Is it safe to download a model file?

    Some model files are programs in disguise: opening an old PyTorch file can run code. Files that hold only numbers are built not to. Here is how to tell them apart.

    6 min+16 min deeper