Chapter 10: YAML Prompts and Serialization

Theoretical Foundations

The fundamental challenge in constructing sophisticated AI agents is not merely invoking a Large Language Model (LLM) with a static string of text. It is the orchestration of complex, stateful, and multi-step workflows where the structure of the prompt, the metadata governing the model's behavior, and the integration of external tools must be managed with the same rigor as database schemas or API contracts. Raw strings, while flexible, are brittle; they lack structure, are difficult to validate, and make versioning a nightmare. This is the problem that YAML Prompts and Serialization addresses within the Microsoft Semantic Kernel.

To understand this concept, we must first anchor it in the architectural evolution of the Semantic Kernel. In Book 7, Chapter 9, we established the Kernel Function as the atomic unit of execution. We learned that a function is defined by its FunctionView—a metadata structure containing parameters, return types, and descriptions. However, we largely constructed these functions using imperative C# code or simple string interpolation. As we transition to Book 8, where the focus shifts to Agentic Patterns, we realize that agents often need to compose these functions dynamically. They need to generate prompts based on context, inject tools, and manage conversation history. Hardcoding these structures in C# strings is inefficient and violates the principle of separation of concerns.

The "What": YAML as a Structured Prompt Definition Language

YAML (YAML Ain't Markup Language) is a human-readable data serialization standard. In the context of Semantic Kernel, it is not merely a configuration file format; it is a domain-specific language (DSL) for defining the lifecycle of an AI function.

When we use a YAML prompt, we are moving away from:

// The "Old" Way: Fragile, unstructured string manipulation
string prompt = $"You are a helpful assistant. Please analyze the following text: {userInput} and summarize it.";

To a structured definition:

name: SummarizeText
template: |
  You are a helpful assistant.
  Please analyze the following text:
  {{$input}}
  and summarize it.
description: Summarizes a block of text into a concise paragraph.
parameters:
  input:
    type: string
    description: The text to summarize.
    default_value: ""

The Anatomy of a YAML Prompt

The Configuration Layer (config): This section defines the non-functional requirements of the prompt. It specifies the model to use (e.g., gpt-4o), the temperature (creativity vs. determinism), and the maximum tokens. Crucially, this allows for model agnosticism. You can swap the underlying LLM provider (OpenAI, Azure OpenAI, Hugging Face) without changing the logic of the prompt itself. 2. The Input Schema (input): Unlike raw strings where parameters are concatenated, YAML defines a strict schema. It declares variables (e.g., {{$user_name}}, {{$query}}) and enforces types. This enables static validation before the prompt is ever sent to the model, catching errors like missing required variables at load time rather than runtime. 3. The Template Logic (template): This is the actual prompt text. However, it is treated as a template engine. It supports control flow (conditionals, loops) and variable substitution. This allows for dynamic prompt generation where the structure of the prompt changes based on the input data.

The "Why": Serialization, Versioning, and Agentic Scalability

Why introduce the complexity of YAML parsing and serialization when C# strings work? The answer lies in the lifecycle of Agentic Workflows.

1. Serialization and Deserialization as a Boundary

Serialization (C# Object -> YAML): This is the authoring phase. Developers (or even non-developers via UI tools) define the agent's behavior in YAML. The Semantic Kernel serializer converts this into an internal PromptTemplateConfig object. * Deserialization (YAML -> Executable Kernel Function): This is the runtime phase. The Kernel reads the YAML, validates it against a schema, binds the C# parameters to the YAML variables, and compiles it into an IFunction that can be invoked.

This separation allows us to store prompts in a Git repository (version control), a database, or a remote file system, decoupling the agent's logic from the application's binary code.

2. Versioning Strategies

Semantic Versioning of Prompts: With YAML, we can tag versions. * classifier_v1.0.yaml: The baseline. * classifier_v1.1.yaml: Added handling for sarcasm. * classifier_v2.0.yaml: Switched to a reasoning model requiring different chain-of-thought instructions. * Rollback Capability: If v2.0 hallucinates, the agentic system can dynamically load v1.1 without recompiling the C# application. This is impossible with hardcoded strings.

3. Modular Prompt Templates (Composition)

system_instructions.yaml: Defines the persona. * safety_filters.yaml: Defines content moderation rules. * task_specific.yaml: Defines the actual task.

The deserialization process merges these contexts. This modularity is vital for Agentic Patterns like the Planner, where the agent must dynamically select and compose sub-tasks defined in separate YAML files.

The "How": The Internal Mechanics of YAML Processing

To visualize the flow of data and control within the Semantic Kernel when processing a YAML prompt, consider the following pipeline. It moves from a static file to a dynamic execution context.

This diagram illustrates the pipeline that transforms a static YAML prompt file into a dynamic execution context within the Semantic Kernel.

Deep Dive: The Serialization Process

Lexical Analysis: The YAML parser reads the file. YAML's significant whitespace is used to define hierarchy. This is parsed into a Dictionary-like structure (JSON equivalent). 2. Mapping to PromptTemplateConfig: The internal PromptTemplateConfig class is a C# representation of the prompt. The deserializer maps YAML keys to C# properties. ``csharp // Conceptual representation of the internal model public class PromptTemplateConfig { public string Name { get; set; } public string Template { get; set; } public string Description { get; set; } public InputConfig Input { get; set; } public ExecutionSettings ExecutionSettings { get; set; } } **Schema Validation:** Before the object is fully realized, the Kernel validates theInputsection. If a parameter is marked asrequired: true` but lacks a description or type, the deserialization fails immediately. This prevents "silent failures" where a prompt is sent to a model with missing context.

Deep Dive: The Execution Context (Handlebars vs. Liquid)

Handlebars Logic: Semantic Kernel often uses Handlebars syntax within the YAML template block. * {{$variable}}: Direct variable injection. * {{#if condition}}...{{/if}}: Conditional logic. * {{#each list}}...{{/each}}: Iteration.

Analogy: The Restaurant Kitchen Ingredients (Parameters): The YAML input section lists the ingredients (e.g., "2 cups of flour", "User Input"). If you don't have the flour, you can't cook. * Instructions (Template): The template section is the step-by-step guide. It tells you when to mix, when to bake, and how to plate. * Chef's Notes (Config): The config section specifies the oven temperature (Temperature) and the cooking time (Max Tokens). * The Cook (Kernel): The Kernel is the cook who reads the recipe, gathers the ingredients (serialization/binding), and executes the steps (rendering the prompt).

If you want to change the recipe (versioning), you don't rewrite the kitchen; you just swap the recipe card. If you want to cook for a different number of people (scaling), you adjust the input quantities without changing the instructions.

Real-World Analogy: The XML Configuration vs. YAML Prompts

In legacy enterprise software, we used XML configuration files (web.config) to define connection strings and logging levels. We moved away from XML because it was verbose and hard to read.

XML (Old Way): <Prompt><Name>Summarize</Name><Text>...</Text></Prompt> (Too verbose, hard to nest). * Raw Strings (Current Pain): "Summarize this: " + input (No structure, hard to version). * YAML (The Solution): It provides the structure of XML with the readability of JSON, specifically tailored for natural language processing.

Architectural Implications for Agentic Patterns

Dynamic Function Creation: In Book 7, Chapter 9, we registered C# methods as functions. Now, we can register YAML files as functions.

// Conceptual API usage
var promptConfig = await File.ReadAllTextAsync("summarize.yaml");
var kernelFunction = kernel.CreateFunctionFromPromptYaml(promptConfig);
kernel.Plugins.Add(new KernelPlugin("MyPrompts", new[] { kernelFunction }));

Cross-Platform Compatibility: Because YAML is a standard text format, the same prompt definition used in a C# backend can be parsed by a Python frontend or a JavaScript client. This ensures that the "Prompt as Code" is consistent across the entire stack.

Testing and CI/CD: We can write unit tests specifically for our YAML prompts. We can assert that:
- The prompt renders correctly given specific inputs.
- The output adheres to a specific schema (using JSON validation).
- The prompt does not exceed token limits (estimated via serialization length).

Escaping Special Characters: YAML uses special characters like :, {, }. When these appear in the prompt text (e.g., writing code examples in the prompt), they must be properly escaped or enclosed in quotes. The serializer handles this automatically, but it is a common source of error in manual editing. * Multi-line Templates: YAML's block scalar (|) is essential for preserving newlines in the prompt. Without it, the LLM might receive a flattened string, altering its behavior. * Variable Interpolation vs. Tool Calls: A subtle but critical distinction. In YAML, {{$input}} is a variable interpolation. However, Semantic Kernel also supports syntax for function calling within prompts (e.g., {{fetch_web_data $url}}). The deserializer must distinguish between data variables and function invocations, mapping them to the appropriate KernelFunction during execution.

Summary

The shift to YAML Prompts and Serialization represents the industrialization of prompt engineering. It moves the definition of AI behavior from the realm of "magic strings" into the realm of structured, version-controlled, and composable software artifacts. By treating prompts as serializable objects, we enable the scalability required for complex Agentic Patterns, allowing systems to dynamically load, validate, and execute AI workflows with the same reliability we expect from traditional software components.

Basic Code Example

Here is a simple, self-contained 'Hello World' level code example demonstrating the YAML prompt format and serialization in Microsoft Semantic Kernel.

The Scenario

Imagine you are building a simple internal tool for a customer support team. The team frequently needs to summarize customer feedback emails. Instead of hard-coding a raw string prompt, we will define the prompt using a YAML file. This allows non-developers to tweak the prompt's tone and structure without touching the C# code, and it allows version control to track changes to the AI's behavior over time.

The Code Example

using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.PromptTemplates.Yaml;
using System.ComponentModel;
using System.Text.Json;
using System.Threading.Tasks;

// 1. Define a simple plugin class with a native function.
// In a real scenario, this might be a database lookup or an API call.
public class EmailPlugin
{
    [KernelFunction, Description("Returns the raw text of the latest customer feedback email.")]
    public string GetLatestFeedback() => "The app is great, but the login button is hard to see on mobile.";
}

class Program
{
    static async Task Main(string[] args)
    {
        // 2. Initialize the Kernel. 
        // Note: We use a dummy model ID for demonstration; real usage requires Azure/OpenAI keys.
        var builder = Kernel.CreateBuilder();
        builder.AddAzureOpenAIChatCompletion(
            deploymentName: "gpt-4o-mini", 
            endpoint: "https://dummy.openai.azure.com/", 
            apiKey: "dummy"); 
        var kernel = builder.Build();

        // 3. Register the native plugin.
        kernel.ImportPluginFromObject(new EmailPlugin(), "email");

        // 4. Define the YAML prompt template.
        // This is the core of the example. We use a raw string here for self-containment,
        // but typically this would be loaded from a .yaml file.
        string yamlPrompt = """
            name: SummarizeFeedback
            template: |
                <message role="system">You are a helpful assistant summarizing customer feedback.</message>
                <message role="user">
                    Please summarize the following feedback concisely:
                    {{$feedback}}
                </message>
            template_format: semantic-kernel
name: feedback
                  description: The raw feedback text to summarize.
                  default: "No feedback provided."
            execution_settings:
                default:
                    max_tokens: 100
                    temperature: 0.7
            """;

        // 5. Create the Prompt Template Configuration from the YAML string.
        var promptConfig = PromptTemplateConfig.FromYaml(yamlPrompt);

        // 6. Create the Kernel Function using the configuration.
        // This step performs the serialization/deserialization of the YAML structure.
        var summarizeFunction = kernel.CreateFunctionFromPrompt(promptConfig);

        // 7. Prepare the arguments.
        // We retrieve the feedback using the native function we registered earlier.
        var feedback = kernel.Plugins["email"]["GetLatestFeedback"].InvokeAsync<string>();
        var arguments = new KernelArguments
        {
            ["feedback"] = await feedback
        };

        // 8. Invoke the function.
        // The Kernel handles the LLM call using the structured prompt defined in YAML.
        var result = await kernel.InvokeAsync(summarizeFunction, arguments);

        // 9. Output the result.
        Console.WriteLine("--- YAML Prompt Execution Result ---");
        Console.WriteLine(result);
    }
}

using Directives: We import the necessary Semantic Kernel namespaces. Microsoft.SemanticKernel.PromptTemplates.Yaml is specifically required to parse the YAML format into a PromptTemplateConfig.

EmailPlugin Class:
- We define a standard C# class decorated with [KernelFunction]. This makes the method callable by the Kernel.
- GetLatestFeedback simulates retrieving data. In a real application, this would connect to a database or email server. It returns a simple string representing the customer's raw text.
Kernel Initialization:
- We use Kernel.CreateBuilder() to set up the dependency injection container.
- AddAzureOpenAIChatCompletion is added. Note: The credentials provided are dummy values. To run this for real, you must replace them with valid Azure OpenAI or OpenAI credentials.
- builder.Build() assembles the Kernel.
Plugin Registration:
- kernel.ImportPluginFromObject(new EmailPlugin(), "email") registers our C# class. This makes the function available to the AI model if we choose to use it as a tool, or to the developer to retrieve data for prompt arguments (as we do here).
The YAML Prompt Definition:
- We define yamlPrompt using a C# raw string literal ("""...""").
- name: Identifies the function.
- template: The actual prompt text sent to the LLM. Note the use of <message> tags (Semantic Kernel's XML-like syntax for chat roles) and the variable placeholder {{$feedback}}.
- template_format: Specifies that this uses Semantic Kernel's standard format.
- input_variables: Defines the schema for inputs. Here, we define feedback, providing a description and a default value. This is crucial for validation and UI generation.
- execution_settings: Maps to LLM parameters like max_tokens and temperature.
Deserialization (PromptTemplateConfig.FromYaml):
- This is the critical serialization step. The static method FromYaml parses the string, validates the schema against Semantic Kernel's expected structure, and returns a PromptTemplateConfig object. This object holds the structured definition of the prompt.
Function Creation:
- kernel.CreateFunctionFromPrompt(promptConfig) converts the configuration object into an executable KernelFunction. During this process, the Kernel prepares the prompt template engine.
Argument Preparation:
- We create a KernelArguments dictionary.
- We retrieve the feedback text by invoking the native function GetLatestFeedback via the plugin collection.
- We assign this value to the key "feedback", which matches the {{$feedback}} placeholder in the YAML template.
Invocation:
- kernel.InvokeAsync(summarizeFunction, arguments) triggers the execution.
- The Kernel performs the following internal steps:
  1. Renders the template by replacing {{$feedback}} with the actual text.
  2. Sends the rendered prompt to the LLM.
  3. Receives the response.
- The result is stored in the result variable.
Output:
- The summarized text is printed to the console.

Visualizing the Flow

The following diagram illustrates how the YAML definition flows through the Semantic Kernel to produce a result.

Common Pitfalls

1. Mismatched Variable Placeholders Incorrect: Defining name: feedback but using {{$text}} in the prompt. * Result: The template engine will fail to replace the placeholder, often sending the literal string {{$text}} to the LLM, or throwing a runtime exception depending on the strictness of the configuration. * Fix: Ensure the name in input_variables matches the placeholder name in the template exactly (case-sensitive).

2. YAML Indentation Errors Result: YamlException or InvalidOperationException during deserialization. * Fix: Use an IDE with YAML support (like VS Code with the Red Hat YAML extension) to validate indentation before running the code.

3. Missing template_format Result: The prompt might not render correctly, or functions might not be invoked properly. * Fix: Always explicitly set template_format: semantic-kernel for standard Semantic Kernel prompts.

4. Forgetting to Register Native Plugins Result: The kernel throws an exception indicating the function or plugin was not found. * Fix: Call kernel.ImportPluginFromObject or ImportPluginFromPromptDirectory before creating or invoking functions that depend on those plugins.

The chapter continues with advanced code, exercises and solutions with analysis, you can find them on the ebook on Leanpub.com or Amazon

Loading knowledge check...

Code License: All code examples are released under the MIT License. Github repo.

All textual explanations, original diagrams, and illustrations are the intellectual property of the author. To support the maintenance of this site via AdSense, please read this content exclusively online. Copying, redistribution, or reproduction is strictly prohibited.