Skip to content

Chapter 19: Persistent Intelligence: Managing Model Weights with Kubernetes Storage

Theoretical Foundations

The theoretical foundation of cloud-native AI agents rests on a fundamental paradox: the most sophisticated computational models, born from abstract mathematics and distributed across ephemeral, stateless containers, require a rigid, stateful, and predictable orchestration layer to function reliably. This section deconstructs the architectural principles required to bridge this gap, focusing on the interplay between containerization, persistent storage, and event-driven scaling within the Kubernetes ecosystem.

The Ephemeral Nature of Inference and the Imperative of State

In traditional microservices, statelessness is a virtue. A web server handling HTTP requests can be scaled horizontally with near-zero friction; any instance can handle any request because the data resides in an external database or cache. Inference-heavy AI agents, however, possess a dual nature. The runtime—the Python interpreter, the PyTorch/TensorFlow libraries, and the inference engine—is ephemeral and can be containerized. However, the model weights—the gigabytes or terabytes of learned parameters—are inherently stateful and immutable during inference.

The "Library" Analogy: Imagine a university library (Kubernetes Cluster). The books (Model Weights) are heavy, expensive, and rarely change. The readers (Inference Pods) are transient; students come and go. If every student had to bring every book they might need, the campus would collapse under the weight of redundancy. Instead, the library maintains a central repository (Persistent Volume). When a student (Pod) needs to read "Advanced Quantum Mechanics" (Load Model Weights), they go to the reference desk (Init Container), check out the book (Mount Persistent Volume), and read it at a desk (GPU Memory). When they leave, the book returns to the shelf (Volume Detach), ready for the next student.

In C# architecture, this distinction is managed through Dependency Injection (DI) and Interfaces, a concept explored in Book 3: Architecting Resilient Microservices. Just as we used interfaces to decouple business logic from data access layers, we must now decouple the inference logic from the weight management.

Consider the IModelLoader interface. It abstracts the mechanism of retrieving weights, whether from a local file system, an S3 bucket, or a distributed file system like CephFS.

using System.Threading.Tasks;

namespace CloudNativeAI.Agents.Core
{
    // Concept from Book 3: Dependency Inversion Principle
    // High-level modules (Inference Service) should not depend on low-level modules (File I/O).
    // Both should depend on abstractions.
    public interface IModelLoader<TModel>
    {
        Task<TModel> LoadAsync(ModelMetadata metadata);
    }

    public class ModelMetadata
    {
        public string ModelName { get; set; }
        public string Version { get; set; }
        public string StoragePath { get; set; } // Maps to a Kubernetes Persistent Volume Claim
    }
}

Containerizing the Inference Runtime: The Immutable Artifact

Containerization of AI agents differs significantly from standard web apps. A standard .NET container might be 200MB. An AI inference container, containing CUDA drivers, cuDNN libraries, Python runtimes, and the model itself, can exceed 10GB.

The "Shipping Container" Analogy: Think of a standard shipping container. It doesn't matter what is inside—electronics, clothes, or machinery—the container has a standard size, locking mechanisms, and handling instructions (Dockerfile). The crane (Kubernetes Kubelet) lifts it without knowing the contents. For AI, we build a "heavy" container. However, to optimize for rapid scaling (cold starts), we separate the runtime environment from the weights.

We create a "Base Image" containing the heavy dependencies (CUDA, Python, Torch). The actual model weights are injected at runtime via a Volume Mount. This is analogous to a truck chassis (Base Image) arriving at a loading dock, where a specific trailer (Persistent Volume with weights) is attached.

In C#, we utilize Multi-Stage Builds and Minimal Runtime Images (like mcr.microsoft.com/dotnet/aspnet:8.0-noble-amd64) to ensure the container footprint is as small as possible, while the heavy lifting is offloaded to the GPU drivers on the host node.

// The entry point of the containerized agent.
// This class represents the "Chassis" of our truck.
using Microsoft.Extensions.Hosting;

namespace CloudNativeAI.Agents.Runtime
{
    public class InferenceHostedService : IHostedService
    {
        private readonly IModelLoader<InferenceEngine> _modelLoader;

        public InferenceHostedService(IModelLoader<InferenceEngine> modelLoader)
        {
            _modelLoader = modelLoader;
        }

        public async Task StartAsync(CancellationToken cancellationToken)
        {
            // On startup, the container attaches the "Trailer" (Volume)
            // and loads the weights into GPU memory.
            var metadata = new ModelMetadata 
            { 
                StoragePath = "/mnt/models/llama-3-70b" // Kubernetes Volume Mount Path
            };
            await _modelLoader.LoadAsync(metadata);
        }

        public Task StopAsync(CancellationToken cancellationToken) => Task.CompletedTask;
    }
}

Kubernetes Storage Patterns: The Data Gravity Problem

When scaling inference agents, we face "Data Gravity." Model weights are massive. Moving them across the network is slow and expensive. Kubernetes offers several volume types, but for AI, two are critical: PersistentVolumeClaims (PVC) and CSI (Container Storage Interface) Drivers.

  1. ReadWriteOnce (RWO): The volume can be mounted by a single node. This is suitable for training jobs where data locality is paramount.
  2. ReadOnlyMany (ROX): The volume can be mounted by multiple nodes simultaneously. This is the gold standard for serving inference. One "master" copy of the weights exists on a high-performance storage system (e.g., NFS, CephFS, or a specialized AI/ML dataset like Azure ML Datastore), and all inference Pods mount it read-only.

The "Valet Key" Analogy: Imagine a luxury car (The Model Weights). You want multiple valets (Inference Pods) to drive it, but you don't want them to crash it or modify it. You give them a "Valet Key" (Read-Only Mount). They can start the car and drive it (perform inference), but they cannot open the glove box or modify the engine (write to the disk). This ensures data integrity while maximizing utilization.

In C#, we handle file system access with System.IO.Abstractions, allowing us to mock these interactions for testing, but in production, these paths map directly to the Kubernetes PVC.

using System.IO.Abstractions;

namespace CloudNativeAI.Agents.Storage
{
    public class PersistentModelReader
    {
        private readonly IFileSystem _fileSystem;

        public PersistentModelReader(IFileSystem fileSystem)
        {
            _fileSystem = fileSystem;
        }

        public byte[] ReadWeights(string mountPath)
        {
            // In Kubernetes, mountPath is /mnt/models
            // This abstraction allows us to verify permissions and existence
            // before the heavy GPU loading begins.
            if (!_fileSystem.Directory.Exists(mountPath))
            {
                throw new DirectoryNotFoundException($"Volume not mounted at {mountPath}");
            }

            return _fileSystem.File.ReadAllBytes(_fileSystem.Path.Combine(mountPath, "model.bin"));
        }
    }
}

GPU Resource Management and Virtualization

Kubernetes treats GPUs as discrete resources. However, a single physical GPU (e.g., NVIDIA H100) is often too powerful for a single lightweight model instance. Conversely, a massive model might require multiple GPUs.

The "Office Space" Analogy: Imagine a GPU is a large office floor. If you assign one employee (Model Instance) to the entire floor, you waste space. If you assign too many employees, they trip over each other (OOM errors). Kubernetes allows GPU Time-Slicing and MIG (Multi-Instance GPU).

  • MIG: Physically partitions the GPU into isolated instances (like building walls in the office).
  • Time-Slicing: Allows multiple pods to share the GPU time (like hot-desking).

In C#, we don't directly control GPU partitioning; that is a node-level configuration. However, we must be aware of Memory Management. .NET's garbage collector (GC) is optimized for heap memory, but AI models reside in GPU VRAM. When we bridge .NET and CUDA (via libraries like TorchSharp or ONNX Runtime), we must manage the lifecycle of tensors explicitly to prevent VRAM fragmentation.

using TorchSharp;

namespace CloudNativeAI.Agents.Compute
{
    public class GpuInferenceEngine
    {
        public void PerformInference()
        {
            // Check if CUDA is available (mapped to Kubernetes resource limits)
            if (!torch.cuda.is_available())
            {
                throw new InvalidOperationException("GPU resource not available in this container.");
            }

            // Creating a tensor allocates VRAM.
            // In a long-running agent, failing to dispose of tensors leads to OOM (Out of Memory).
            using (var tensor = torch.rand(1000, 1000, device: torch.CUDA))
            {
                // Perform matrix multiplication
                var result = tensor.matmul(tensor);

                // Explicit disposal is crucial in containerized environments
                // where memory is constrained and shared.
                result.Dispose();
            }
        }
    }
}

Event-Driven Autoscaling: The KEDA Integration

Static scaling (Horizontal Pod Autoscaler - HPA) based on CPU/RAM is insufficient for AI. A model might sit idle consuming 0% CPU but holding 50GB of VRAM. Scaling based on "requests per second" requires an event-driven approach. This is where KEDA (Kubernetes Event-driven Autoscaling) comes in.

KEDA acts as an adapter between an event source (e.g., RabbitMQ, Kafka, Azure Service Bus) and Kubernetes. It monitors the "lag" or "queue depth" and scales the Deployment accordingly.

The "Restaurant Kitchen" Analogy: A traditional auto-scaler (HPA) is like a manager hiring more chefs because the kitchen is hot (CPU usage). This is reactive and often irrelevant to the actual workload. KEDA is like a manager watching the order ticket rail (The Queue). If 50 tickets pile up, they immediately call in extra chefs. If the rail is empty, they send chefs home. This is precise and cost-effective.

In C#, we utilize Background Services to consume these events. The IHostedService pattern is perfect for this. We connect to a message broker, and for every message received, we trigger the inference logic.

using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Logging;
using RabbitMQ.Client;
using RabbitMQ.Client.Events;

namespace CloudNativeAI.Agents.Messaging
{
    public class RabbitMQInferenceConsumer : BackgroundService
    {
        private readonly IConnection _connection;
        private readonly IModel _channel;
        private readonly ILogger<RabbitMQInferenceConsumer> _logger;

        public RabbitMQInferenceConsumer(ILogger<RabbitMQInferenceConsumer> logger)
        {
            _logger = logger;
            // Connection setup (omitted for brevity)
            var factory = new ConnectionFactory() { HostName = "rabbitmq-service" };
            _connection = factory.CreateConnection();
            _channel = _connection.CreateModel();
        }

        protected override Task ExecuteAsync(CancellationToken stoppingToken)
        {
            // KEDA monitors this queue. 
            // As messages accumulate, KEDA scales up replicas of this service.
            var consumer = new EventingBasicConsumer(_channel);
            consumer.Received += (model, ea) =>
            {
                var body = ea.Body.ToArray();
                var message = System.Text.Encoding.UTF8.GetString(body);

                _logger.LogInformation($"Processing inference request: {message}");
                // Trigger the AI inference logic here

                _channel.BasicAck(deliveryTag: ea.DeliveryTag, multiple: false);
            };

            _channel.BasicConsume(queue: "inference-requests", autoAck: false, consumer: consumer);

            return Task.CompletedTask;
        }
    }
}

The Synergy: Distributed Model Serving

Finally, we must address the scenario where a model is too large to fit on a single GPU. We employ Model Parallelism or Pipeline Parallelism.

The "Assembly Line" Analogy: If a car is too complex for one person to build, we use an assembly line. Person A installs the chassis, passes it to Person B for the engine, and Person C for the wheels. In AI, we split the model layers across multiple GPUs (or nodes).

  • Pipeline Parallelism: Layer 1-10 on GPU 0, Layer 11-20 on GPU 1.
  • Tensor Parallelism: Split the matrix multiplication itself across GPUs.

In C#, managing this requires a Orchestrator Pattern. We don't just run a model; we run a graph of models. We might use gRPC for inter-pod communication to pass intermediate tensors between layers.

using Grpc.Core;
using DistributedInference;

namespace CloudNativeAI.Agents.Distributed
{
    // This service acts as a node in a distributed inference graph.
    public class DistributedInferenceService : DistributedInference.DistributedInferenceBase
    {
        private readonly ILayerExecutor _layerExecutor;

        public DistributedInferenceService(ILayerExecutor layerExecutor)
        {
            _layerExecutor = layerExecutor;
        }

        public override async Task<TensorResponse> ForwardPass(TensorRequest request, ServerCallContext context)
        {
            // 1. Receive tensor data from previous node (or client)
            // 2. Execute specific layers (e.g., Layers 11-20)
            // 3. Pass result to next node
            var inputTensor = Deserialize(request);
            var outputTensor = await _layerExecutor.ExecuteAsync(inputTensor);

            return new TensorResponse 
            { 
                Data = ByteString.CopyFrom(Serialize(outputTensor)) 
            };
        }
    }
}

Theoretical Foundations

The theoretical foundation of cloud-native AI agents is a convergence of immutable infrastructure (Containers), stateful abstraction (Persistent Volumes), and reactive scaling (KEDA).

  1. Decoupling: We separate the heavy, immutable weights from the transient compute containers.
  2. Resource Awareness: We treat GPUs as first-class citizens, managing VRAM with the same rigor as system RAM.
  3. Event-Driven Logic: We scale based on queue depth (demand) rather than system load (utilization), ensuring cost efficiency.
  4. Distributed Coordination: We design agents not as monolithic binaries, but as composable microservices capable of passing tensors across the network.

This architecture allows us to treat AI not as a static monolith, but as a fluid, elastic fabric of compute that expands and contracts based on the needs of the data.

Basic Code Example

using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;

namespace CloudNativeAiMicroservices.Example
{
    /// <summary>
    /// Represents the core inference engine for our AI agent.
    /// In a real-world scenario, this would wrap a heavy ML library (e.g., PyTorch, TensorFlow, ONNX Runtime).
    /// For this "Hello World" example, we simulate the computational load and memory management.
    /// </summary>
    public class InferenceEngine
    {
        private readonly Random _rng = new Random();

        /// <summary>
        /// Simulates a heavy inference operation.
        /// In a containerized GPU environment, this would involve:
        /// 1. Loading input data into GPU VRAM.
        /// 2. Executing matrix multiplications on the GPU.
        /// 3. Retrieving results from VRAM to system RAM.
        /// </summary>
        /// <param name="inputData">The raw input data (e.g., text, image bytes).</param>
        /// <returns>A task representing the inference result with a confidence score.</returns>
        public async Task<InferenceResult> PredictAsync(string inputData)
        {
            // Simulate the latency of GPU computation and data transfer.
            // In a real GPU-bound workload, the duration depends on model size and batch size.
            // Here, we randomize it to mimic variable load.
            int processingTimeMs = _rng.Next(50, 200); 
            await Task.Delay(processingTimeMs);

            // Simulate a result. 
            // In a real scenario, this would be a tensor or structured object.
            double confidence = _rng.NextDouble(); 

            return new InferenceResult
            {
                Prediction = $"Processed: {inputData}",
                Confidence = confidence,
                ProcessingTimeMs = processingTimeMs
            };
        }
    }

    /// <summary>
    /// Data Transfer Object (DTO) for the inference result.
    /// </summary>
    public record InferenceResult
    {
        public string Prediction { get; init; } = string.Empty;
        public double Confidence { get; init; }
        public int ProcessingTimeMs { get; init; }
    }

    /// <summary>
    /// Represents the Kubernetes Pod lifecycle and resource management.
    /// In a real deployment, this class would interface with the Kubernetes C# Client 
    /// to report metrics (Prometheus exporter) or handle termination signals.
    /// </summary>
    public class PodContext
    {
        private readonly CancellationTokenSource _cts = new CancellationTokenSource();

        /// <summary>
        /// Simulates the Kubernetes Pod readiness probe.
        /// A Pod is ready only when its internal services (e.g., model loaded into GPU) are initialized.
        /// </summary>
        public bool IsReady { get; private set; } = false;

        public async Task InitializeAsync()
        {
            Console.WriteLine("[PodContext] Initializing model weights from Persistent Volume...");
            // Simulate loading gigabytes of model weights from a mounted PVC (Persistent Volume Claim).
            await Task.Delay(1000); 
            IsReady = true;
            Console.WriteLine("[PodContext] Model loaded. Ready to serve traffic.");
        }

        /// <summary>
        /// Simulates handling the SIGTERM signal sent by Kubernetes during scale-down or rolling updates.
        /// </summary>
        public void RegisterShutdownHandler()
        {
            Console.CancelKeyPress += (s, e) =>
            {
                e.Cancel = true;
                Console.WriteLine("[PodContext] SIGTERM received. Draining connections...");
                _cts.Cancel();
            };
        }

        public CancellationToken GetCancellationToken() => _cts.Token;
    }

    /// <summary>
    /// The main entry point simulating the containerized agent.
    /// </summary>
    class Program
    {
        static async Task Main(string[] args)
        {
            // 1. Setup Infrastructure
            var podContext = new PodContext();
            podContext.RegisterShutdownHandler();

            // 2. Initialize Inference Engine (Load Model)
            var engine = new InferenceEngine();
            await podContext.InitializeAsync();

            // 3. Simulate Request Processing Loop
            // In a real K8s environment, this would be an HTTP server (e.g., ASP.NET Core) 
            // listening on port 8080.
            Console.WriteLine("[Agent] Starting request processing loop...");

            var tasks = new List<Task>();

            // Simulate concurrent requests (e.g., from a Load Balancer)
            for (int i = 0; i < 5; i++)
            {
                if (podContext.GetCancellationToken().IsCancellationRequested) break;

                var requestTask = Task.Run(async () =>
                {
                    var result = await engine.PredictAsync($"Image_{Guid.NewGuid()}");
                    Console.WriteLine($"[Agent] Result: {result.Prediction} | Confidence: {result.Confidence:F2} | Time: {result.ProcessingTimeMs}ms");
                });

                tasks.Add(requestTask);
                await Task.Delay(50); // Simulate staggered incoming requests
            }

            try
            {
                await Task.WhenAll(tasks);
            }
            catch (OperationCanceledException)
            {
                Console.WriteLine("[Agent] Processing halted due to shutdown signal.");
            }

            Console.WriteLine("[Agent] Simulation complete. Container exiting.");
        }
    }
}

Line-by-Line Explanation

  1. using System...: Imports standard .NET libraries for threading, tasks, and collections. In a real Kubernetes pod, the .NET runtime is pre-installed in the container image.
  2. namespace CloudNativeAiMicroservices.Example: Organizes the code. In a microservices architecture, namespaces correspond to logical service boundaries.
  3. public class InferenceEngine: This encapsulates the core business logic. In the context of Book 7, this is the component that consumes the most resources (GPU/CPU).
    • private readonly Random _rng: Used to simulate variability in inference time. Real-world inference times vary based on input complexity (e.g., image resolution).
    • PredictAsync:
      • Task.Delay: Simulates the "blocking" nature of GPU computation. In a real scenario, this is where the C# code would P/Invoke into CUDA libraries or call a Python process via inter-process communication.
      • InferenceResult: A record type (modern C# feature) used to return structured data. This is crucial for serialization when exposing this via an HTTP API later.
  4. public class PodContext: Simulates the Kubernetes environment wrapper.
    • InitializeAsync: Represents the Init Container pattern or startup logic. In K8s, you often mount model weights via PVCs (Persistent Volume Claims). This delay simulates the I/O latency of loading large files (e.g., 5GB model weights) into memory/GPU.
    • RegisterShutdownHandler: Critical for Graceful Shutdown. Kubernetes sends a SIGTERM signal before killing a pod. Handling Console.CancelKeyPress allows the application to finish processing current requests before terminating, preventing data loss.
  5. Main Method:
    • podContext.InitializeAsync(): Ensures the model is loaded before accepting traffic. This aligns with K8s readinessProbe logic.
    • Task.Run & Task.Delay: Simulates an asynchronous web server handling concurrent requests. In a real app, this would be replaced by app.Run() in ASP.NET Core.
    • Task.WhenAll: Waits for all simulated requests to complete, mimicking a batch processing job or concurrent API handling.

Common Pitfalls

  1. Blocking Synchronous Calls:

    • Mistake: Using Thread.Sleep or synchronous I/O inside the inference loop.
    • Impact: In a containerized environment, this blocks the main thread. Since K8s relies on the application responding to health checks (HTTP endpoints) on the main thread, blocking it will cause the container to fail liveness probes and get restarted in a crash loop.
    • Fix: Always use async/await (as shown in PredictAsync) to keep the thread pool free to handle health checks and incoming requests.

  2. Ignoring GPU Memory Management:

    • Mistake: Assuming the garbage collector (GC) handles GPU memory if using wrappers like TorchSharp or TensorFlow.NET.
    • Impact: GPU memory is not managed by the .NET GC. If you allocate tensors in a loop without disposal, you will get Out Of Memory (OOM) errors from the GPU driver, crashing the container.
    • Fix: Use IDisposable patterns strictly. Wrap tensor allocations in using blocks or explicitly call .Dispose().

  3. Hardcoding Resource Limits:

    • Mistake: Assuming the container always has access to the full GPU.
    • Impact: In K8s, you request nvidia.com/gpu: 1. If the code assumes it has 40GB VRAM but the node only allocates 16GB (e.g., A10G), the model loading will fail.
    • Fix: Dynamically query available VRAM at startup (via NVML or environment variables injected by the NVIDIA device plugin) and adjust batch sizes accordingly.

Real-World Context: The "Inference-Heavy" Agent

Imagine you are deploying a Visual Question Answering (VQA) agent for an e-commerce mobile app. Users upload photos of products and ask questions like "Is this shirt available in blue?"

  1. The Challenge: Traffic is spiky. At 9:00 AM, traffic is low (0.5 req/sec). At 8:00 PM (peak shopping hours), traffic spikes to 50 req/sec.
  2. The Resource Constraint: GPUs are expensive. You cannot keep 10 GPU pods running at 9:00 AM just for the peak.
  3. The Solution (Concept):
    • Containerization: The C# code above is Dockerized. The image includes the .NET runtime and the necessary CUDA drivers.
    • Orchestration: Kubernetes manages these containers.
    • Scaling: We use KEDA (Kubernetes Event-driven Autoscaling). It monitors the "Request Queue Length" (e.g., an Azure Service Bus queue or RabbitMQ). When the queue depth > 10, KEDA triggers Kubernetes to scale the Deployment (replicas) from 1 to 10.
    • Persistence: The Model weights (the 5GB file) are stored on a Persistent Volume (e.g., Azure Blob mounted via CSI driver) so they don't bloat the Docker image.

Visualization of the Architecture

This diagram illustrates how a containerized application running in Kubernetes mounts an external Azure Blob storage volume via the CSI driver, keeping the application image lightweight by storing large data files outside the image.
Hold "Ctrl" to enable pan & zoom

This diagram illustrates how a containerized application running in Kubernetes mounts an external Azure Blob storage volume via the CSI driver, keeping the application image lightweight by storing large data files outside the image.

Deep Dive: Why This Pattern Matters

The code provided is simple, but it sits at the intersection of several complex architectural decisions:

1. The "Cold Start" Problem When KEDA scales from 0 to 1 replica, the PodContext.InitializeAsync method runs. Loading a 5GB model from a Persistent Volume into GPU memory might take 30-60 seconds. During this time, the pod is not ready.

  • Kubernetes Config: You must configure readinessProbe to fail until IsReady is true.
  • User Impact: If you don't handle this, the first users get 503 errors. Mitigation involves "Pre-warming" or keeping a minimum of 1 replica always running.

2. GPU Sharing and Time-Slicing In the code, we assume 1 Pod = 1 GPU. However, a modern NVIDIA A100 GPU is powerful enough to handle inference for multiple requests simultaneously.

  • Optimization: Instead of scaling Pods (Horizontal Pod Autoscaler), you can use NVIDIA MPS (Multi-Process Service) or Time-Slicing.
  • Code Implication: The InferenceEngine would need to support batch processing (processing multiple inputs in one PredictAsync call) to maximize GPU utilization.

3. Distributed Model Serving If the model is too large (e.g., 70B parameters) to fit in a single GPU's VRAM, we must use Model Parallelism.

  • Concept: Split the model layers across multiple GPUs (Tensor Parallelism) or multiple Pods (Pipeline Parallelism).
  • Code Implication: The InferenceEngine would become a distributed system. The Main method would initialize multiple clients, each connecting to a worker holding a shard of the model. The InferenceResult would need to be aggregated across these workers.

4. Event-Driven Autoscaling (KEDA) Standard CPU-based autoscaling is inefficient for AI inference. GPU utilization is non-linear; a GPU can be 0% busy waiting for data, then 100% busy for 50ms.

  • Why KEDA?: We scale based on Queue Depth (the Queue in the diagram). If there are 100 pending messages, we know we need more pods. This is more responsive than waiting for CPU usage to spike.
  • Implementation: KEDA queries the queue metric. If MessageCount > Threshold, it updates the replicas count in the Kubernetes Deployment. The C# code doesn't need to change; it just pulls messages from the queue faster.

5. Persistent Volumes vs. ConfigMaps

  • ConfigMaps: For small configuration files (e.g., appsettings.json).
  • Persistent Volumes (PVC): For model weights.
    • Why?: Model weights are binary blobs, often gigabytes in size. Storing them in a ConfigMap would exceed etcd limits (usually 1MB).
    • Mounting: In the Kubernetes YAML, you would define a volume mount at /models. The C# code reads from /models/v1/model.bin. When the model updates, you update the PVC content (or snapshot) without rebuilding the Docker image.

The chapter continues with advanced code, exercises and solutions with analysis, you can find them on the ebook on Leanpub.com or Amazon

Loading knowledge check...

Code License: All code examples are released under the MIT License. Github repo.

Content Copyright: Copyright © 2026 Edgar Milvus | Privacy & Cookie Policy. All rights reserved.

All textual explanations, original diagrams, and illustrations are the intellectual property of the author. To support the maintenance of this site via AdSense, please read this content exclusively online. Copying, redistribution, or reproduction is strictly prohibited.