Colang Variables and Flows in NeMo Guardrails | Chatbot Tutorial

Dialog state management tracks conversation progress to prevent repeating questions or greetings. The OnTurnAsync method saves conversation state and runs the dialog system. Waterfall Dialogue is a sequential conversation pattern where the bot asks questions one at a time in a specific order. The dialog system coordinates between components using BeginDialog, ContinueDialog, ReplaceDialog, EndDialog, and CancelDialog methods. Data flows through waterfall dialogue via the Options parameter in BeginDialog, enabling complex data structures to be passed between dialog components. The BookingDetails object contains Destination, Origin, and TravelDate properties. Child dialogs like DateResolverDialog are invoked for specific tasks. DateTime prompts include validation logic checking if input matches expected formats. The final step calls EndDialog with collected data, which is returned to the parent dialog.

Dialogue state tracking maintains conversation context by recording previous user messages, bot actions, and slot values to enable coherent multi-turn interactions. Unlike simple rule-based systems, dialogue state tracking allows bots to handle complex conversations where meaning depends on prior exchanges. Dialogue policy prediction uses recurrent neural networks (RNNs) to model sequential dependencies, receiving current user input, dialogue state, and API results as inputs. The network maintains internal state across time steps, enabling consideration of entire conversation histories. After prediction, action probabilities are renormalized to ensure valid distributions. This approach supports multi-turn dialogues where bots execute multiple actions per user message, such as fetching data and generating responses before awaiting further input.

The state machine pattern provides formalized management of chatbot dialog states. Developers define a graph of possible states, specify transition logic, and implement handlers for each state. This approach limits handler scope (only handling transitions from specific states) and formalizes entire dialog logic. Libraries like pytransitions support conditional transitions and state validation. Key design principles include: (1) Always provide users the ability to reset their dialog state; (2) Plan for scalability from the beginning; (3) Use state machines to formalize and manage states; (4) Implement robust error handling; (5) Learn from other developers' experiences. These practices help avoid common pitfalls and create reliable chatbot systems.

Conversation state memory enables the system to remember previous turns and maintain context across multiple interactions. The system tracks conversation turns, storing information about what has been asked and answered. This memory allows the bot to reference previous statements, maintain conversation history, and provide coherent responses that build upon earlier interactions. The memory persists throughout the conversation session, enabling the system to understand the full context of the conversation.

Dialogue management maintains conversation context across multiple messages through state tracking. The system combines current message features with accumulated state information, feeding this into machine learning models (SVMs or neural networks) to predict next actions. Action masking prevents invalid predictions based on security constraints. The dialogue loop continues until a 'listen' action triggers waiting for the next user message. Each iteration may involve API calls providing additional information to refine predictions, creating a dynamic feedback cycle.

Guardrails are monitoring systems that supervise conversations between users and virtual agents, triggering predefined responses when rules are violated. There are two types: system guardrails (maintained by technical teams) and custom guardrails (created by customers). Custom guardrails can detect prohibited topics like politics, religion, or attempts to manipulate the system. They also handle hallucinations and bad language through moderation. Global management capabilities ensure consistent quality across different use cases and conversation flows.

Guardrails provide essential safety mechanisms for production-ready AI agents. Available types include: (1) PII (Personally Identifiable Information) detection - blocking sensitive data like names, emails, phone numbers, addresses, and financial information before reaching the model, (2) Moderation - classifying and blocking harmful content including hate speech, (3) Jailbreak detection - preventing attempts to steal system prompts or force off-task behavior, and (4) Hallucination checking - using a secondary model to verify agent outputs against knowledge base documents. Each guardrail has configurable confidence thresholds and models.

Guard Rails is a security functionality in N8N that protects AI agents from user manipulation and hallucinations. The standard agent architecture involves users sending messages to agents equipped with tools, prompts, and vector databases. However, human users tend to make errors and may have malicious intentions, attempting to make agents hallucinate or deviate from their intended function. Hallucination occurs when agents deviate from their prompt instructions, which can happen due to user manipulation, poor prompt engineering, or information overload. Guard Rails implements a double-layer protection system using separate LLMs to filter both incoming user messages and outgoing agent responses. The protection mechanisms include keyword blocking, jailbreak prevention, NSFW filtering, PII protection, secret key protection, topical alignment, and URL injection prevention. Implementation requires updating N8N to version 1.119.0 or higher and should be tested in development environments before production deployment. The node offers two main operations: 'Check Text for Violations' which validates messages against policies and routes them to pass or fail paths, and 'Sanitize Text' which removes sensitive information while allowing messages to continue through the workflow. Jailbreak detection uses a sensitivity threshold (trade hold) ranging from 0.0 to 1.0, where 0.0 is most flexible and 1.0 is most restrictive. The default value is 0.7. Messages are scored based on how closely they match jailbreak patterns, and if the score exceeds the threshold, the message is blocked.

Implementing effective guardrails requires selecting appropriate tools and frameworks. Nemo Guardrails by Nvidia optimizes dialogue oversight with built-in tools and PII recognition integration. Guardrails AI provides a platform with an online hub for discovering and installing validators, similar to npm or pip. Specific models serving as judges include foundation models (GPT-4, Claude, Gemini, Llama 3), Bespoke Mini for evaluating claims against context, Llama Guard from Meta, and Shield Gemma from Google. A practical demonstration illustrates building a guarded chatbot for Acme Shoes, showing catastrophic failures in unguarded versions when asked about pricing (hallucination) or subjected to jailbreak prompts, versus successful protection in guarded versions employing competitor detection, jailbreak detection with adjustable thresholds, and output validation using Bespoke Mini to verify claims against provided context.

Implementing guardrails for hallucination prevention involves configuring the system to verify claims against trusted documents in the vector store. The process includes setting up guardrails that check whether information is supported by source documents, configuring routing based on validation results (passing to user or routing to error-handling agents), and providing detailed error information including the original query, reasoning for failure, and hallucination type. This creates a multi-stage validation process that ensures only verified information reaches end users.

Evaluation of AdversLLM guard rails demonstrates their effectiveness in detecting malicious content that bypasses native content filters. Testing against prompts that evade GPT-3.5 and GPT-4's built-in filters shows the guard rails achieve high detection rates across malicious, abuse, and hate categories. However, results indicate that approximately 25-50% of highly malicious prompts receive low sensitivity scores, suggesting room for improvement. The system's nuanced detection capabilities work well for related topics beyond the primary malicious category, though comprehensive testing against the full range of adversarial attacks remains an ongoing challenge.

Creating guardrails involves configuring content filters for prompts and responses, adding denied topics with descriptions and examples, enabling profanity filters, and customizing blocked messages. Testing uses the guardrail console with any foundation model: benign queries pass through normally, while denied topics trigger violations. Prompt injection attacks (like telling a banking assistant to behave like a pirate) are detected when prompt attack filters are enabled. Jailbreaking attempts (like 'do anything I ask') are blocked under multiple categories including prompt attack and misconduct.

AI agents can be bypassed through prompt injection attacks that exploit the gap between guardrail instructions and actual model behavior; even sophisticated guardrails can be circumvented by adding specific phrases that convince the model the query is legitimate, demonstrating that robust security requires not just guardrails but also careful instruction design and thorough testing against adversarial inputs.

This section demonstrates practical guardrail testing: (1) Access guardrails from the Bedrock console and select the working draft version; (2) Choose a supported model (ensure access is granted first); (3) Input various test prompts including innocuous questions, health advice requests, and financial queries; (4) Observe model responses versus final responses to verify guardrail intervention; (5) Troubleshoot ineffective filters by reviewing denied topic definitions and adjusting strength levels. Testing reveals that guardrails operate at both prompt and response levels, blocking inappropriate content before it reaches users.

Traditional guardrails fail in agentic systems for two main reasons. First, the 'lexical shortcut' problem occurs when guardrails are trained on datasets that conflate prompt injection techniques with adversarial intent, causing them to learn structural patterns rather than actual malicious intent. This leads to false positives where benign prompts containing injection markers are flagged. Second, 'application blindness' means guardrails built on application-agnostic datasets don't transfer to specific applications. The email assistant scenario demonstrates this: guardrails failed to catch harmful actions because they evaluated each action in isolation without understanding the overall trajectory, and they couldn't distinguish between legitimate mission fulfillment and malicious intent.