AI Agent Architecture: Single-Agent vs Multi-Agent Systems

Large Language Models (LLMs) like ChatGPT are trained to predict the next word in a sequence. The training process involves taking text (like a Wikipedia article), using the first 100 words as input, and the next word as the correct label. This is repeated by shifting the window one word at a time across all text on the internet, creating tons of training data. To make ChatGPT more helpful, OpenAI used reinforcement learning with human feedback (RLHF), where human annotators rated responses as good or bad, and the neural network's weights were updated to promote good answers and demote bad ones. System prompts are hidden instructions given to AI models at the beginning of a conversation to guide their behavior, such as 'You're a helpful assistant that answers questions. Don't say any bad words or teach the user how to do harmful things.' A context window is the maximum number of words that can be fed into a neural network at once, and when a conversation history exceeds this limit, the oldest parts may be cut off or summarized.

Large language models (LLMs) are AI systems that generate human-like text. ChatGPT, deployed by OpenAI in 2022, was the first widely accessible interface for interacting with LLMs through text. The ecosystem has expanded with major providers including OpenAI (ChatGPT), Google (Gemini), Meta (Llama), Microsoft (Copilot), Anthropic (Claude), and XAI (Grok). LLMs process text by breaking it into tokens—small text chunks from a vocabulary of roughly 200,000. The context window is the model's working memory, a one-dimensional token sequence containing all conversation history. When you start a new chat, the window resets. Tokens accumulate as you interact, and anything inside is directly accessible to the model. Users should treat tokens as precious resources and avoid overloading the window with irrelevant information.

Large Language Models (LLMs) are neural network-based models that can parse and generate text, accessible via APIs. The fundamental unit is the token (approximately 4 characters). A prompt is the text provided to a model before generating output. System prompts contain hidden instructions to the LLM, including model identity, tools, and behavioral constraints. Prompt templates inject user questions with system prompts containing instructions, restrictions, and style guidelines. The context window is the amount of text an LLM can consider at once, including input prompt and output, with newer models like Gemini Pro having 1.5 million tokens.

LLM (Large Language Model) is a computer program that predicts and generates the next word in a sequence. A prompt is the input request given to an LLM, which can be a single sentence or set of sentences. Zero-shot prompts provide no examples, while few-shot prompts include examples to guide the LLM. System prompts are standard instructions that define how the LLM should behave, such as formatting preferences or tone. Skills are sets of instructions that enable LLMs to perform specific tasks according to requirements. The context window stores all conversation history during a session, but LLMs tend to forget early information as conversations continue. Hallucination occurs when LLMs generate incorrect information, often due to context window limitations. Token limits restrict how much information an LLM can process, with free plans having smaller limits than paid plans. Temperature controls randomness in responses—lower values produce more predictable outputs, while higher values increase creativity and variation.

The context window is the working memory of an LLM, representing the maximum text tokens the model can consider at once. It differs from parameterized knowledge (vague recollection from pre-training) which is less reliable. LLMs have no persistent memory between sessions, so information must be included in the context window. During inference, the model uses finalized weights to make predictions without updating weights, involving forward pass through the neural network and prediction of next tokens one at a time. The context window acts as a short-term memory buffer, including the latest inquiry, chat history, and system prompts. LLMs can use tools like internet search for current information, code interpreters (especially Python) for accurate calculations, and document handling for domain-specific information. Retrieval-Augmented Generation (RAG) allows LLMs to recall facts from custom knowledge bases. Agentic tools enable LLMs to break complex goals into steps and execute them autonomously.

Tool calling (function calling) enables AI models to interact with external APIs. Tools serve two main purposes: (1) Information retrieval - getting real-time data like date, time, weather, or database information that LLMs cannot provide; (2) Taking actions - performing operations like saving data to databases, updating records, or deleting information. This extends AI capabilities beyond static text generation to dynamic, interactive applications.

Function calling enables AI models to access external data and perform actions beyond their training knowledge. It emerged with chat GPT, allowing models to act as assistants that call tools. Two related concepts exist: function calling (strict, forcing tool use) and tool use (flexible, model chooses when to use tools). The OpenAI JSON schema standardizes function definitions with names, parameters, and descriptions. Models return which function to call; applications execute these calls. When no relevant functions exist, models provide direct answers. This two-step process ensures seamless user experiences regardless of tool availability.

Tool use, also referred to as function calling, enables AI models to interact with their environment by utilizing external tools. This capability allows AI agents to perform tasks such as retrieving order status from a database or sending emails on behalf of users. With tool use, developers can supply a list of available tools to the model, and the model will select the appropriate tool to complete its task. The model can handle complex tasks requiring several tools to be called in sequence.

Tool calling (also called function calling) is the mechanism by which language models generate calls to external functions rather than just text responses. The model identifies which tool to use and provides the necessary arguments. This enables developers to connect models with APIs, databases, and other logic systems. For example, when asking about weather in Paris, the model might suggest using a 'get_weather' function with the argument 'Paris'.

This section explains function calling (tool calling), which enables AI models to access and execute application functions. The speaker demonstrates defining tools with names, descriptions, and parameters (like product ID for stock queries). When users ask questions, the AI identifies which tool to use and passes parameters to execute the function. The section covers handling JSON parameters in the onExecute event, executing database queries, calling external APIs (like CEP lookup), and returning results to users. This capability transforms AI from a passive responder into an active system integrator.

Microservices architecture decomposes applications into small, independent services that communicate via HTTP, requiring patterns like API Gateway for routing, Service Discovery for locating services, Load Balancer for distributing traffic, Centralized Configuration for managing settings, Event Driven for asynchronous communication, Event Sourcing for storing state changes as events, CQRS for separating read/write operations, and Saga for managing distributed transactions across services.

Microservices communication requires robust patterns for reliability and scalability. OpenFeign provides declarative HTTP clients with automatic load balancing through strategies like Round Robin and Response Time based selection. Circuit Breaker patterns with Hystrix protect against cascading failures by detecting service failures and returning fallback responses. API Gateways like Spring Cloud Gateway serve as centralized entry points, routing requests based on predicates and integrating with service discovery. Alternative solutions include Spring Gateway (reactive) and Kong (Lua-based). These patterns collectively enable resilient, scalable microservices architectures.

Microservices communicate primarily through REST APIs using HTTP/HTTPS protocols. Key communication patterns include: (1) Stateless design - each API call contains all necessary information, eliminating server-side session state storage; (2) HTTP methods - use GET for retrieving data, POST for creating resources, PUT/PATCH for updating, and DELETE for removing resources; (3) API versioning - include version numbers in endpoints to manage backward compatibility during evolution; (4) Rate limiting and throttling - protect services from abuse by limiting request rates; (5) API gateways - use gateways to handle cross-cutting concerns like authentication, routing, and load balancing before routing requests to appropriate microservices.

Microservices architecture breaks applications into smaller, independent services, each owning its own database. Data sharing uses REST APIs or event-driven messaging. Polly is the primary library for resilience patterns (retry, circuit breakers). HTTP clients are registered via AddHttpClient(). API gateways handle authentication, routing, and response aggregation. Services discover each other dynamically using tools like Consul. The most common communication method is REST APIs over HTTP/HTTPS. Communication security uses HTTPS with SSL/TLS encryption, mutual TLS for verification, and JWT tokens for authentication. Revealing internal server details in error responses is a bad security practice. Controller routes use [Route] attributes with controller names as prefixes. Global exception handling middleware is more scalable than try-catch in every action. Model validation fails return HTTP 400 Bad Request. DTOs control data exposure by separating API responses from database entities. [ApiController] attribute enables automatic validation and appropriate status codes. AddScoped creates new instances per HTTP request. Custom action filters apply logic before/after controller execution.

Microservices communication involves three main patterns: synchronous point-to-point (direct service-to-service calls), asynchronous message brokers (using queues for decoupled communication), and publish-subscribe systems (event-driven architectures). Key components include message brokers like Apache Kafka for buffering and routing, API gateways for centralized traffic management, and circuit breakers for fault tolerance. Effective microservices communication requires handling message deduplication, implementing rate limiting, and ensuring proper authentication/authorization across service boundaries.