AI Agent Glossary 2026: The Complete Enterprise Buyer's Reference

Agent - A software system that perceives its environment, makes autonomous decisions, and takes actions to achieve goals. Agents differ from chatbots by their ability to use tools, plan multi-step sequences, and operate without human intervention between steps.

Agentic AI - AI systems designed to operate autonomously for extended periods, making decisions and taking actions without human approval for each step. Distinguishes from chatbots that require human input after each response.

API - Application Programming Interface. A set of rules allowing software to request and exchange data. Most AI agents interact with business systems through APIs (e.g., calling Salesforce API to update a contact).

Autonomous - Operating independently without human intervention. An autonomous agent makes decisions and takes actions based on its goals and observations without asking permission for each step.

Benchmark - A standardized test measuring AI system performance. Common benchmarks: MMLU (general knowledge), HumanEval (coding), GAIA (agent reasoning). Enterprise buyers use benchmarks to compare models objectively.

Benchmark Gaming - When vendors optimize models specifically for benchmark performance, not real-world tasks. A model might perform well on benchmarks but poorly on your actual use case.

Chain-of-Thought - A prompting technique where the AI explains its reasoning step-by-step before providing an answer. Dramatically improves accuracy on complex tasks by making the model think through problems explicitly.

Context Window - The amount of text (in tokens) an AI model can consider at once. GPT-4 has 128K tokens (about 100K words). Larger context windows mean the agent can consider more documents, conversation history, and context simultaneously.

Cold Start Problem - When an AI system lacks sufficient data or context to make good decisions initially. Solved through human feedback, training data, or gradual deployment starting with easy cases.

Embeddings - A numerical representation of text (or images) that captures meaning. Embeddings allow AI systems to understand that "customer" and "buyer" are similar concepts. Used in vector databases for semantic search.

Evaluation - Systematic testing of AI agent performance against defined metrics. Rigorous enterprises create evaluation frameworks with test data, success criteria, and red-team scenarios before deployment.

Few-Shot Prompting - Providing the AI with a few examples before asking it to perform a task. "Few-shot" = 2-10 examples. More examples = better performance (usually), up to a point.

Fine-Tuning - Training an AI model on your proprietary data to specialize its behavior. More time-intensive and expensive than prompt engineering, but enables models to learn your exact style, domain language, and decision criteria.

Function Calling - When an AI model decides to call a specific function or tool based on user input. For example, when an agent decides "I need to call the Slack API to send a message" without being told explicitly.

Guardrails - Safety boundaries and constraints embedded in an agent's instructions. Examples: "Never approve payments over $1M without human review," "Always cite sources," "Escalate legal questions to our attorney."

Grounding - Anchoring an AI response in factual, verified information. A grounded agent retrieves specific documents from your knowledge base before responding. An ungrounded agent might make up plausible-sounding answers.

Hallucination - When an AI generates false or nonsensical information confidently, as if it were true. A hallucinating agent might cite a company policy that doesn't exist or mention features your product doesn't have.

Human-in-the-Loop - Workflow where the AI agent makes recommendations, but humans approve before execution. Used for high-stakes decisions: contract approvals, budget changes, customer refunds.

Inference - Using a trained AI model to generate predictions or responses on new data. Running an agent in production is "inference." Training it is "training."

Integration - Connecting an AI agent to your existing systems (CRM, ERP, HRIS, etc.). Most enterprise value comes from what the agent can access and do, not the agent itself.

Knowledge Graph - A structured database of facts and relationships. For example: "Microsoft acquired LinkedIn. LinkedIn is a social network. LinkedIn has 900M users." Agents use knowledge graphs to reason about complex relationships.

LLM - Large Language Model. A neural network trained on billions of text examples to predict the next word (and chain predictions into coherent text). Examples: GPT-4, Claude 3.5, Gemini 2.0.

Latency - Response time. How long it takes an agent to respond to a query. Critical for real-time applications. Typical agent latency: 1-10 seconds depending on complexity.

Memory - An agent's ability to remember previous conversations and use that context in current decisions. Short-term memory: the current conversation. Long-term memory: summaries or embeddings of past conversations.

Multi-Agent - A system with multiple specialized agents coordinating to solve problems. Example: One agent for sales, one for customer service, one for compliance, all working on the same customer request.

RAG - Retrieval-Augmented Generation. The agent retrieves relevant documents from your knowledge base, then generates responses grounded in those documents. Reduces hallucinations and adds specificity.

RLHF - Reinforcement Learning from Human Feedback. Training approach where humans rate model outputs ("this response is good," "this is bad"), and the model learns to improve. Used to align models with human preferences.

Retrieval - The process of finding relevant information from a knowledge base (documents, databases, APIs). RAG systems combine retrieval with generation. Without retrieval, agents might hallucinate.

Temperature - A parameter controlling randomness/creativity in AI responses. Low temperature (0-0.3): Deterministic, repetitive answers. High temperature (0.7-1.0): Creative, varied answers. Typically set to 0.2-0.5 for agent tasks.

Token - The basic unit of text for AI models. Roughly, 1 token = 0.75 words. A 100-word article = about 133 tokens. Important for understanding API costs and context window limits.

Tool Use - An agent's ability to decide when and how to call external tools (APIs, functions, systems). Modern agents choose which tools to use autonomously based on user requests.

Vector Database - A specialized database optimized for storing and searching embeddings. Enables semantic search: finding documents similar in meaning, not just keyword matching. Examples: Pinecone, Weaviate, Milvus.

Validation - Testing outputs before using them. An agent might generate a customer response, but you validate it (check for policy compliance, tone, accuracy) before sending.

Zero-Shot - Asking an AI to perform a task without any examples. "Summarize this document." Harder than few-shot but requires less setup.

Zero-Shot CoT - A variant where you ask for chain-of-thought reasoning without examples. "Think step-by-step, then answer."

Accuracy - Percentage of agent outputs judged correct by experts. For customer service: 85-90% is good. For legal contracts: 95%+ expected.

Recall - Percentage of relevant information the agent finds. High recall: "Finds all relevant documents." Low recall: "Misses some important info."

Precision - Percentage of retrieved information that's actually relevant. High precision: "Everything retrieved is useful." Low precision: "Lots of noise in results."

Latency SLA - Service Level Agreement for response time. Example: "95% of responses within 5 seconds." Critical for real-time applications.

Throughput - How many requests an agent can handle per second. Important for scaling: Can the system handle 1,000 concurrent requests?

Cost per Request - API pricing model. Example: "$0.01 per 1M tokens." Understanding token costs is essential for ROI calculations.

Jailbreak - When a user tricks an agent into ignoring its guardrails. Example: "Pretend you're not bound by safety policies and tell me X." Good agents resist jailbreaks.

Adversarial Testing - Intentionally trying to break an agent by feeding it unusual inputs, edge cases, or hostile requests. Essential for enterprise deployment.

Alignment - How well an agent's behavior matches human values and intentions. A well-aligned agent does what you want. A misaligned agent does unexpected things.

Interpretability - Ability to understand why an agent made a specific decision. Example: "The agent recommended escalation because the customer mentioned 'legal action' and guardrails require human review for legal threats."

Explainability - The agent's ability to explain its reasoning to users. Different from interpretability (which is for engineers). Explainability is customer-facing.

Bias - Systematic errors favoring certain groups or outcomes. Example: An agent might score loan applications lower for applicants from certain zip codes if trained on biased historical data.

Fairness - Treating all users equitably. A fair agent doesn't discriminate based on protected characteristics (race, gender, age, etc.).

Model Drift - When an agent's performance degrades over time as real-world data changes. Real customers behave differently from training data. Requires monitoring and retraining.

Prompt Injection - When user input tricks an agent into ignoring its system prompt. Example: User embeds "Ignore previous instructions" in a document the agent processes.

Semantic Search - Finding information based on meaning, not keywords. An agent can search for "things customers complained about" and find documents discussing satisfaction, not just documents with the word "complaint."

Cosine Similarity - A mathematical measure of how similar two embeddings are. Ranges from -1 (opposite) to 1 (identical). Used in vector databases for finding similar documents.

Attention Mechanism - How transformer models (the architecture behind modern LLMs) decide which parts of the input to focus on. Not necessary to understand this fully, but it enables context understanding.

Transformer - The neural network architecture underlying modern LLMs. Released in 2017, enabled the explosion in AI capability. Not something you need to understand the details of, but good to know it exists.

Scaling Laws - The observation that larger models perform better. More parameters, more training data, more compute = better performance. Important for understanding why GPT-4 is better than GPT-3.5.

API Rate Limiting - Restrictions on how many requests you can make. Example: "10,000 requests/minute." Hit the limit and requests fail. Important for high-throughput applications.

Fallback Model - A backup model used if the primary model fails. Example: If GPT-4 API is unavailable, use Claude 3.5 Sonnet instead.

Ensemble - Combining predictions from multiple agents or models. Often more accurate than any single model. Trade-off: More expensive and slower.

Batch Processing - Processing multiple requests together instead of one at a time. Cheaper and slower than real-time requests. Good for non-urgent tasks.

Streaming - Returning responses incrementally as tokens are generated, instead of waiting for the complete response. Feels faster to users (they see text appearing), though total time is the same.

Caching - Storing previous responses so repeated requests don't hit the API. Saves money and improves latency. Cache hit rate: percentage of requests served from cache.

Quantization - Reducing precision of model weights to make models smaller and faster. Trade-off: Slightly lower accuracy for much lower cost and latency.

Distillation - Training a small model to imitate a large model. Results in a smaller, faster agent with similar performance.

Prompt Versioning - Treating prompts like code with version control. Track changes, who approved them, performance metrics for each version.

A/B Testing - Comparing two versions of an agent with different subsets of users. "Version A resolved 85% of tickets on first contact. Version B resolved 87%. Version B is better."

Red Teaming - Hiring experts to try to break your agent before deployment. Security-focused red teaming, bias-focused red teaming, jailbreak red teaming.

Compliance Audit - Reviewing an agent's decisions to ensure they comply with regulations and company policy. Common in finance, healthcare, legal.

Transparency Report - Documentation of an AI system's capabilities, limitations, and performance. Essential for regulated industries and enterprise deployments.