Skip to content

Chapter 19: Garbage Collection & IDisposable - Cleaning up GPU/Memory Resources

Theoretical Foundations

In the realm of high-performance AI development within .NET, memory management is not merely a background process—it is the bedrock of system stability and performance. The .NET Garbage Collector (GC) is a non-deterministic, mark-and-sweep algorithm designed primarily for managed memory (objects on the heap). While efficient for general-purpose applications, it introduces latency and unpredictability that are detrimental to real-time AI inference and training loops.

When working with unmanaged resources—specifically GPU memory (VRAM) allocated via CUDA, DirectX, or Vulkan APIs for tensor operations—the GC is blind. It cannot track the memory footprint of pointers outside the managed heap. Relying on the GC to eventually trigger a finalizer (a mechanism for cleanup) is risky; it may lead to OutOfMemory (OOM) errors on the GPU or system RAM before the GC decides to run. To bridge this gap, C# provides the IDisposable interface, enabling deterministic, manual resource management.

The Analogy: The Library vs. The Workshop

To understand the distinction between GC-managed memory and unmanaged resources, consider the difference between a public library and a high-security chemical workshop.

The Library (Managed Memory): In a library, you check out a book (allocate an object). You don't need to worry about shelving it when you are done; the librarian (the Garbage Collector) periodically walks through the aisles, checks which books are not being referenced (no one is holding a card for them), and returns them to the shelves (frees the memory). This is non-deterministic. You don't know exactly when the librarian will walk by, but you trust they will eventually. This is fine for books (standard objects).

The Workshop (Unmanaged Resources): In a chemical workshop, you mix volatile compounds in a beaker (allocate a GPU tensor buffer). This beaker occupies physical space and has safety implications. If you leave the beaker on the table, it blocks other work and might explode if left too long. You cannot wait for a janitor (the GC) to wander by whenever they feel like it. You must clean up the beaker immediately after you finish your experiment. This is deterministic disposal. If you rely on the janitor, the workshop will run out of space, or an accident will occur.

In AI development, the "Workshop" is the GPU. VRAM is finite. Large language models or deep neural networks allocate massive buffers for weights and activations. If you rely on the library rules for the workshop, your application will crash.

The IDisposable Interface

The IDisposable interface is the standard mechanism in .NET for releasing unmanaged resources immediately. It defines a single method:

public interface IDisposable
{
    void Dispose();
}

When a class holds unmanaged resources (like a pointer to GPU memory) or manages other IDisposable objects, it should implement this interface. The consumer of the class is responsible for calling Dispose() as soon as the resource is no longer needed.

Implementing the Disposable Pattern

Implementing IDisposable requires a specific pattern to ensure safety in both deterministic and non-deterministic scenarios. We must handle two paths:

  1. Explicit Cleanup: The developer calls Dispose().
  2. Finalizer Fallback: The developer forgets to call Dispose(), and the GC eventually collects the object.

Here is a robust implementation for a hypothetical GpuTensor class, which manages a pointer to VRAM.

using System;

public class GpuTensor : IDisposable
{
    // Pointer to unmanaged GPU memory
    private IntPtr _devicePointer;
    private bool _disposed = false;

    // Constructor allocates unmanaged memory
    public GpuTensor(int sizeInBytes)
    {
        // Simulate allocation (e.g., cudaMalloc)
        _devicePointer = Marshal.AllocHGlobal(sizeInBytes); 
        Console.WriteLine($"Allocated {sizeInBytes} bytes at {_devicePointer}");
    }

    // Public Dispose method - the deterministic cleanup path
    public void Dispose()
    {
        Dispose(true);
        // We are explicitly disposing, so we don't need the finalizer to run
        GC.SuppressFinalize(this);
    }

    // Protected virtual method containing the actual cleanup logic
    protected virtual void Dispose(bool disposing)
    {
        if (_disposed) return;

        if (disposing)
        {
            // Managed resources cleanup:
            // If this class held references to other IDisposable objects (e.g., a file stream),
            // we would call their Dispose() methods here.
            // Example: _logStream?.Dispose();
        }

        // Unmanaged resources cleanup:
        // This runs regardless of whether Dispose() or the finalizer called it.
        if (_devicePointer != IntPtr.Zero)
        {
            Marshal.FreeHGlobal(_devicePointer); // Simulate cudaFree
            _devicePointer = IntPtr.Zero;
            Console.WriteLine($"Freed GPU memory at {_devicePointer}");
        }

        _disposed = true;
    }

    // Finalizer (Destructor syntax)
    // This acts as a safety net if Dispose() is never called.
    ~GpuTensor()
    {
        // The GC calls this, but we cannot access managed objects here safely.
        Dispose(false);
    }
}

Architectural Implications:

  • The _disposed Flag: Prevents double-free errors, which are critical in unmanaged memory contexts and can lead to segmentation faults or undefined behavior in GPU drivers.
  • GC.SuppressFinalize(this): When Dispose() is called explicitly, we tell the GC that cleanup has already occurred. This prevents the finalizer from running later, removing the object from the finalization queue and improving performance.
  • The disposing Parameter: This distinguishes between the two cleanup paths.
    • If true (called from Dispose()), we can safely access other managed objects (like _logStream) and dispose of them.
    • If false (called from the finalizer), we cannot touch other managed objects because they may have already been collected. We only clean up unmanaged resources.

Integrating Delegates and Lambda Expressions for Cleanup

In Book 1, we established the foundation of Delegates as type-safe function pointers. In Book 2, we expand this by using Lambda Expressions to create flexible cleanup logic and resource wrappers.

In complex AI pipelines, resource allocation is often dynamic. We might want to execute specific logic immediately after a resource is disposed, or we might want to wrap a block of code in an automatic disposal scope. Delegates allow us to pass this cleanup behavior as a parameter.

Consider a scenario where we want to ensure that a GPU tensor is disposed of immediately after a matrix operation, but we also want to log the duration of the operation. We can create a helper method that accepts a delegate for the work to be done and a delegate for the cleanup notification.

public class ResourceManager
{
    // A delegate type for cleanup notifications
    public delegate void CleanupCallback(string resourceName);

    // Generic method that manages the lifecycle of a resource
    public static void UseResource<T>(T resource, Action<T> work, CleanupCallback onCleanup) where T : IDisposable
    {
        try
        {
            // Perform the intensive AI work (e.g., matrix multiplication)
            work(resource);
        }
        finally
        {
            // Deterministic cleanup
            onCleanup?.Invoke(resource.GetType().Name);
            resource.Dispose();
        }
    }
}

Now, we can use Lambda Expressions to define the work and the cleanup logic inline. Lambdas are anonymous functions that capture variables from their surrounding scope, making them ideal for short, localized logic blocks.

public void RunInference()
{
    // Allocate a tensor
    var inputTensor = new GpuTensor(1024);

    // Use the ResourceManager with Lambda expressions
    ResourceManager.UseResource(
        resource: inputTensor,

        // Lambda for the work: Defines the matrix operation
        work: (tensor) => 
        {
            // Simulate heavy computation
            Console.WriteLine("Performing matrix multiplication on GPU...");
        },

        // Lambda for the cleanup callback
        onCleanup: (name) => 
        {
            Console.WriteLine($"Cleaning up {name} immediately after use.");
        }
    );

    // inputTensor is guaranteed to be disposed here
}

This pattern is powerful because it decouples the resource lifecycle management from the business logic. The delegate Action<T> allows us to inject any operation into the UseResource method, while the CleanupCallback delegate ensures that post-processing logic (like logging or releasing related semaphore locks) happens deterministically alongside the Dispose() call.

The using Statement: Syntactic Sugar for Delegates

The most common way to enforce deterministic disposal in C# is the using statement. It is essentially syntactic sugar for a try-finally block that calls Dispose().

public void ProcessBatch()
{
    // The 'using' statement guarantees Dispose() is called when the scope exits
    using (var tensor = new GpuTensor(2048))
    {
        // Perform operations
        Console.WriteLine("Processing batch data...");
    } // tensor.Dispose() is called automatically here, even if an exception occurs
}

C# 8.0+ Enhancement (Using Declarations): In newer versions of C#, we can use a using declaration, which disposes of the resource at the end of the current scope (usually the method).

public void ProcessBatchModern()
{
    using var tensor = new GpuTensor(2048);

    // Perform operations

    // tensor.Dispose() is called automatically when the method returns
}

Visualizing the Resource Lifecycle

The following diagram illustrates the flow of control and resource state when using the IDisposable pattern in an AI application. It contrasts the deterministic path (developer intervention) with the non-deterministic fallback (GC intervention).

The diagram contrasts the developer-controlled deterministic disposal path (Dispose) with the non-deterministic garbage collection fallback to visualize the resource lifecycle in an AI application.
Hold "Ctrl" to enable pan & zoom

The diagram contrasts the developer-controlled deterministic disposal path (Dispose) with the non-deterministic garbage collection fallback to visualize the resource lifecycle in an AI application.

Architectural Implications for AI Systems

When building large-scale modeling frameworks, the choice of when and how to dispose of resources dictates the scalability of the system.

  1. Memory Fragmentation: Frequent allocation and deallocation of GPU memory without proper disposal can lead to fragmentation. The GPU driver may have plenty of free memory, but not in contiguous blocks large enough for a new tensor. The IDisposable pattern allows developers to pool tensors (reuse memory) rather than constantly allocating new buffers.
  2. Exception Safety: In AI training loops, exceptions can occur (e.g., NaN values, hardware errors). The using statement ensures that even if an exception is thrown inside the matrix operation block, the Dispose() method is called. Without this, a crash could leave gigabytes of VRAM locked until the process terminates.
  3. Interop with C++/CUDA: As discussed in previous chapters regarding interoperability, .NET manages the managed heap, while CUDA manages the device heap. The IDisposable pattern is the bridge. It ensures that the .NET reference to the GPU buffer is removed from the managed heap (via the finalizer) and the corresponding pointer is removed from the device heap (via Dispose).

By mastering IDisposable and integrating it with delegates and lambda expressions, developers move from "hoping" the GC cleans up to "guaranteeing" that resources are released exactly when the computational graph no longer requires them. This deterministic control is what makes .NET a viable platform for high-performance AI.

Basic Code Example

In the high-stakes world of AI development, memory management is not just an academic exercise; it is the difference between a model that trains in minutes and one that crashes after hours due to VRAM exhaustion. The .NET Garbage Collector (GC) is a marvel of engineering for general-purpose applications, but it operates on a non-deterministic schedule. It cleans up managed memory when it decides it's best, not when you need it. For AI tensors backed by unmanaged GPU memory, this latency is fatal. A tensor holding a 4K image gradient might linger in GPU memory long after it's used, blocking the allocation for the next frame or batch. We need a mechanism to say, "I am done with this resource. Delete it now."

This is the problem the IDisposable interface solves. It allows us to create a contract for immediate, deterministic cleanup. Let's model a simplified scenario: a GpuTensorBuffer that simulates allocating a block of VRAM. We will use a using statement to ensure this "VRAM" is released the instant we are done with it.

using System;

// A mock class to simulate holding a valuable, limited resource like GPU memory.
// We implement IDisposable to signal that this class manages unmanaged resources.
public class GpuTensorBuffer : IDisposable
{
    private bool _isDisposed = false; // Tracks if Dispose() has been called
    private readonly int _bufferId;   // Simulates a handle to a GPU memory block

    public GpuTensorBuffer(int sizeInBytes)
    {
        _bufferId = new Random().Next(1000, 9999); // Simulate a unique GPU memory handle
        Console.WriteLine($"[Alloc] VRAM Block #{_bufferId} allocated ({sizeInBytes} bytes).");
    }

    public void PerformCalculation()
    {
        if (_isDisposed)
        {
            throw new ObjectDisposedException("GpuTensorBuffer", "Cannot perform calculation on disposed buffer.");
        }
        Console.WriteLine($"[Compute] Using VRAM Block #{_bufferId} for matrix multiplication...");
    }

    // The core of deterministic cleanup.
    public void Dispose()
    {
        // Call the internal cleanup method with 'true' to indicate explicit disposal.
        Dispose(true);

        // Tell the GC that we don't need the Finalizer anymore; we've already cleaned up.
        GC.SuppressFinalize(this);
    }

    // Protected virtual method allowing derived classes to clean up their own resources.
    protected virtual void Dispose(bool isDisposing)
    {
        if (_isDisposed) return; // Idempotency: multiple calls are safe.

        if (isDisposing)
        {
            // IMPORTANT: Only clean up managed resources here (other IDisposables).
            // We don't have any in this simple example, but if we held a Logger or Stream, we'd close it here.
            Console.WriteLine($"[Free] VRAM Block #{_bufferId} released immediately via Dispose().");
        }

        // Always clean up unmanaged resources (simulated here by the _bufferId).
        // This block runs whether Dispose() is called explicitly or by the Finalizer.
        // In a real scenario, this is where you call cudaFree() or clReleaseMemObject().

        _isDisposed = true;
    }

    // The Finalizer (Destructor) acts as a safety net.
    // If the user forgets to call Dispose(), this will eventually run.
    ~GpuTensorBuffer()
    {
        // We pass 'false' because we are in the GC thread; we cannot safely touch other managed objects.
        Dispose(false);
        Console.WriteLine($"[Warning] VRAM Block #{_bufferId} released via Finalizer. You forgot to use 'using'!");
    }
}

public class Program
{
    public static void Main()
    {
        Console.WriteLine("--- Simulation Start ---");

        // The 'using' statement is syntactic sugar for a try/finally block.
        // It guarantees that .Dispose() is called when the scope is exited.
        using (var tensor = new GpuTensorBuffer(1024 * 1024)) // 1 MB
        {
            tensor.PerformCalculation();
        } // <--- tensor.Dispose() is called automatically here.

        Console.WriteLine("\n--- Simulation End: Scope Exited ---");

        // Now, let's demonstrate what happens if we DON'T use 'using'.
        Console.WriteLine("\n--- Negligence Simulation ---");
        CreateAndForget();

        // We force a Garbage Collection to prove the Finalizer runs eventually.
        // Note: In production, you should never rely on GC.Collect()!
        Console.WriteLine("\n[System] Forcing Garbage Collection...");
        GC.Collect();
        GC.WaitForPendingFinalizers();

        Console.WriteLine("--- Final State ---");
    }

    // This helper method simulates a common coding mistake.
    public static void CreateAndForget()
    {
        var lostTensor = new GpuTensorBuffer(2048);
        lostTensor.PerformCalculation();
        // We exit the method here.
        // 'lostTensor' goes out of scope, but we never called Dispose().
        // It becomes "Garbage" waiting for the GC to find it.
    }
}

Visualizing the Lifecycle

The following diagram illustrates the control flow. Note the distinct paths for the "Safe Path" (using IDisposable) versus the "Risky Path" (relying on the Finalizer).

The diagram contrasts the deterministic Dispose() method, which immediately releases resources and bypasses the finalizer, with the non-deterministic finalizer path, where the Garbage Collector eventually reclaims memory but leaves resources unmanaged for an indefinite period.
Hold "Ctrl" to enable pan & zoom

The diagram contrasts the deterministic `Dispose()` method, which immediately releases resources and bypasses the finalizer, with the non-deterministic finalizer path, where the Garbage Collector eventually reclaims memory but leaves resources unmanaged for an indefinite period.

Step-by-Step Explanation

  1. Resource Allocation (GpuTensorBuffer Constructor): When new GpuTensorBuffer(...) is called, the constructor executes. In a real-world AI scenario, this is where the "heavy lifting" occurs. We would call a native C++ function (via P/Invoke) to allocate memory on the GPU. In our example, we simulate this by generating a random _bufferId and printing a log message. This establishes the "cost" of the object.

  2. The IDisposable Contract: The class implements IDisposable. This interface contains only one method: Dispose(). This is a standardized signal to the consumer of the class that they are responsible for cleanup.

  3. The using Statement (The "Safe" Path): In Main(), we wrap the instantiation in using (var tensor = ...). This is the most critical concept for an AI Engineer. It compiles down into a try / finally block.

    • Try: The code inside the braces runs.
    • Finally: Regardless of whether an exception occurs inside the try block (e.g., a matrix dimension mismatch error), the finally block guarantees that tensor.Dispose() is called.

  4. Explicit Cleanup (Dispose method): When Dispose() is called (manually or by using), we set isDisposing to true. This allows us to execute our cleanup logic immediately. We release the VRAM handle, ensuring the GPU is ready for the next operation instantly. We also call GC.SuppressFinalize(this). This is an optimization: it tells the Garbage Collector, "This object has already been cleaned up; you don't need to run the Finalizer later, so don't bother putting it in the finalization queue."

  5. The Safety Net (The ~GpuTensorBuffer Finalizer): In CreateAndForget(), we instantiate a buffer but exit the scope without calling Dispose(). The variable lostTensor vanishes, but the memory it allocated on the GPU is still occupied. Eventually, the .NET Garbage Collector will notice that the managed object has no references. It will then schedule the Finalizer to run.

    • Why Dispose(false)? Inside the Finalizer, we are already in a precarious state. The Garbage Collector is tearing down objects. We cannot safely access other managed objects (they might have already been finalized). Therefore, the Dispose(bool disposing) method checks the flag. If false, it skips the managed resource cleanup and only attempts to release the raw unmanaged GPU memory.

  6. The Consequence of Negligence: If you look at the output of the simulation, the "Negligence" path triggers the Finalizer. In a real AI training loop, this delay is disastrous. The GPU memory might not be reclaimed until the end of the epoch, causing an "Out of Memory" error during the next batch. The using statement ensures this never happens.

Common Pitfalls

1. Forgetting the using Statement The most frequent mistake in C# resource management is instantiating an IDisposable object without wrapping it in a using block.

  • The Mistake: var tensor = new GpuTensorBuffer(100);
  • The Consequence: If the method exits normally, Dispose() is never called. The object is orphaned. The Finalizer might clean it up eventually, but in the context of GPU programming, "eventually" is too late. You will leak VRAM until the application crashes.
  • The Fix: Always wrap in using or explicitly call .Dispose() in a finally block.

2. Implementing the Finalizer Incorrectly Developers sometimes put critical logic only inside the Dispose() method and forget the Finalizer.

  • The Mistake: Not implementing a Finalizer at all, or implementing it to call Dispose(true).
  • The Consequence: If a user forgets using, the unmanaged resource (GPU memory) is never released. The memory leak persists until the process terminates.
  • The Fix: Always implement the Full Dispose Pattern (as shown in the example) if your class holds unmanaged resources. The Finalizer is your insurance policy against user error.

3. Accessing Managed Members in the Finalizer

  • The Mistake: Inside ~GpuTensorBuffer(), trying to log to a file using a StreamWriter or access a managed object.
  • The Consequence: The Garbage Collector runs on a separate thread and does not guarantee the order in which objects are finalized. The StreamWriter you are trying to use might have already been finalized itself, leading to a NullReferenceException or undefined behavior.
  • The Fix: Only touch raw unmanaged pointers/handles inside the if (!isDisposing) block.

The chapter continues with advanced code, exercises and solutions with analysis, you can find them on the ebook on Leanpub.com or Amazon

Code License: All code examples are released under the MIT License. Github repo.

Content Copyright: Copyright © 2026 Edgar Milvus | Privacy & Cookie Policy. All rights reserved.

All textual explanations, original diagrams, and illustrations are the intellectual property of the author. To support the maintenance of this site via AdSense, please read this content exclusively online. Copying, redistribution, or reproduction is strictly prohibited.