Skip to content
AI Governance Commonsby Synthetic Labs
Courses›AI literacy essentials

Module 1 of 5 · about 8 minutes

How generative AI works, and fails

A generative AI tool such as ChatGPT, Copilot, Gemini, or Claude produces text by predicting what words are likely to come next, based on patterns in an enormous amount of text it was trained on. It does not look facts up in a database, and it does not know whether what it writes is true. That is why it can write a confident, fluent paragraph that is wrong, invent a court case or a statistic, or produce a plausible-looking citation that does not exist.

The same is true of image, audio, and video generators: they produce what is statistically likely to look or sound right. They are very useful for drafting, summarizing, translating, brainstorming, and writing code, and very dangerous when the output is used without a person checking it.

Three things to remember. First, the tool does not know your company, your customers, or today's law unless you give it that information, and then it only has what you gave it. Second, the output is a draft, never a decision. Third, what you type in may be stored by the vendor, and on some plans may be used to train future models.

  • Generative AI predicts likely words or pixels; it does not verify facts.
  • Confident tone is not evidence of accuracy.
  • Every output is a draft until a person has checked it.

Modules

1. How generative AI works, and fails
2. What may go into which tools
3. Spotting hallucinations and bias
4. Disclosure and labelling duties
5. When a human must decide
6. Check your understanding

You can choose which cookies to allow below.

Privacy Policy