All glossary terms
AI Fundamentals & AI for Proposals

Token

An individual unit of text, such as a word or piece of punctuation, that an AI language model processes; tokenization is the act of breaking text into these units.

Related terms