Skip to content

Chapter 14: Avoiding Defensive Copies - 'in' Parameters and 'readonly' Members

Theoretical Foundations

In performance-critical AI pipelines, particularly those handling high-throughput token processing or tensor manipulations, memory allocation and CPU cycles are precious resources. When working with value types (struct), the default behavior in C# is to copy the entire value when passing it to a method or accessing it from a read-only context. In a standard business application, this overhead is negligible. However, in the context of AI inference—where a single request might process thousands of tokens, each represented by a struct containing metadata, probability distributions, or embedding vectors—this copying becomes a significant bottleneck. It manifests as increased pressure on the Garbage Collector (GC), higher latency, and wasted CPU cycles that could otherwise be utilized for model computation.

To understand the necessity of in parameters and readonly members, we must first visualize the lifecycle of data within an AI model's inference loop. Consider a tokenizer that breaks a prompt into tokens. Each token is a small struct containing an ID, a logit score, and perhaps a reference to a vocabulary string. If we pass this struct by value to a method that calculates the next token's probability, the runtime creates a full copy of that struct on the stack. If the struct is large (e.g., containing a small fixed-size buffer for embedding data), this copy operation is expensive.

The following diagram illustrates the flow of data in a standard pass-by-value scenario versus a pass-by-reference (in) scenario:

The Mechanics of Defensive Copying

In C#, the CLR (Common Language Runtime) performs "defensive copying" when a struct is passed to a method without the in modifier, particularly if the method is defined in a different assembly or if the struct is accessed in a context that requires immutability guarantees. This ensures that the callee cannot modify the caller's data unexpectedly. However, in the context of AI development—specifically when building layers of a neural network or optimizing token filters—this safety mechanism introduces a performance penalty.

Imagine a struct representing a Token in a Large Language Model (LLM) inference engine:

public struct Token
{
    public int Id;
    public float Logit;
    public fixed char Text[64]; // A fixed-size buffer for the token text
}

If we pass this struct to a method float CalculateProbability(Token token), the runtime must copy the entire 64-character buffer plus the integer and float to the stack. If this method is called inside a loop processing 10,000 tokens, we are effectively allocating and copying 640,000 bytes of memory on the stack (or heap, depending on context) just to read the data. This is the "defensive copy" we aim to avoid.

The in Parameter Modifier: Passing by Reference Safely

The in parameter modifier was introduced to allow pass-by-reference semantics without the ability to modify the data. It is a promise from the compiler: "I need the efficiency of a pointer, but I guarantee I will not change the data." When you use in, you are passing a reference to the original struct, often residing on the stack or in a register, rather than a copy.

Analogy: The Museum Artifact Think of a museum curator (the in parameter) examining a rare artifact (the struct). Instead of shipping a perfect replica of the artifact to the curator (pass-by-value), the curator is invited to the vault to view the original artifact. They can look at it, measure it, and analyze it, but they are not allowed to touch it or modify it. This saves the cost and time of creating and shipping the replica.

In the context of AI, this is crucial when passing tensor metadata or configuration structs. For instance, a ModelConfig struct containing hyperparameters (learning rate, batch size, etc.) is often read-only during inference. Passing it by value would copy the entire configuration object for every layer invocation. Using in ensures that the configuration is read directly from its original memory location.

Safe Usage and the readonly Modifier To fully leverage in, the struct itself should ideally be immutable. If a struct is mutable, passing it by in prevents modification within the method, but the struct's internal state might still be mutable by design, which can lead to confusion. This is where the readonly struct and readonly member modifiers come into play.

  1. readonly struct: This declares that the struct is immutable. The compiler enforces that all fields are readonly and that no method can modify the state of the struct. This guarantees that when the struct is passed by in, the data is truly immutable.
  2. readonly method modifier: This indicates that a method does not modify the state of the struct. It allows the method to be called on in parameters without triggering a defensive copy.

Analogy: The Glass Display Case If the museum artifact is placed inside a sealed glass display case (readonly struct), the curator (in parameter) can view it from all angles. Even if the curator wanted to modify the artifact, the glass case prevents it. This allows the museum to safely grant access to anyone without fear of accidental damage.

Interaction with Span and SIMD

The true power of in and readonly emerges when combined with Span<T> and SIMD (Single Instruction, Multiple Data) operations, which are foundational for high-performance AI in C#.

Span and Memory Safety As discussed in previous chapters, Span<T> provides a type-safe view over contiguous memory. When processing a tensor (a multi-dimensional array of numbers), we often use Span<float> to iterate over the data. If we have a helper method that processes a slice of the tensor, passing the Span by value would copy the reference (which is cheap), but if the method needs metadata about the slice (e.g., TensorSlice struct containing dimensions and strides), passing that struct by value is expensive.

Using in with a TensorSlice struct allows us to pass metadata efficiently while operating on the Span data.

SIMD and Value Types SIMD operations (via System.Numerics or hardware intrinsics) work best with vectors of data. Often, we define structs to hold SIMD vectors (e.g., Vector256<float>). These structs can be large (32 bytes for AVX). Passing them by value incurs a copy cost that negates the performance benefit of SIMD. By using in, we ensure that the SIMD vector is passed by reference, allowing the CPU to operate directly on the registers or memory location without unnecessary copying.

Architectural Implication in AI In an AI inference engine, a "Layer" might be defined as a struct containing weights (as a Span<float>) and configuration (as a LayerConfig struct). When the forward pass is calculated, the LayerConfig is read repeatedly. If we use in LayerConfig, we avoid copying the configuration for every neuron's calculation. This reduces the instruction count and memory bandwidth usage, allowing the CPU to focus on the floating-point arithmetic required for the neural network.

Edge Cases and the Stack vs. Heap

A critical nuance of in parameters is the lifetime of the data. Since in passes a reference, the caller must ensure that the data being referenced lives longer than the method call.

  • Stack Allocations: If a struct is created on the stack (e.g., var token = new Token();), passing it by in is safe as long as the method call is synchronous and the struct is not returned or stored in a field that outlives the stack frame.
  • Heap Allocations: If the struct is a field in a class (heap-allocated), passing it by in is safe because the class instance keeps the data alive.

The readonly Ref Returns C# allows returning references to struct members using ref return. When combined with readonly, this allows efficient access to internal data structures without copying. For example, a Matrix struct might expose a specific row via public ref readonly float GetRow(int index). This returns a reference to the row data (wrapped in a Span or ref), allowing the caller to read the data without copying the entire matrix.

Theoretical Foundations

The theoretical foundation of avoiding defensive copies rests on the principle of zero-cost abstractions. In high-performance C# for AI, we want the safety and expressiveness of the language without paying a runtime penalty for features we don't use.

  1. Defensive Copying is the runtime's mechanism to ensure safety when passing value types, but it incurs a memory and CPU cost proportional to the size of the struct.
  2. in Parameters provide a safe mechanism to pass large structs by reference, eliminating the copy cost while maintaining immutability guarantees.
  3. readonly Structs and Members enforce immutability at the compiler level, making in parameters safe and predictable.
  4. Integration with AI Pipelines: By using these features, we reduce GC pressure (fewer allocations/copies), improve cache locality (data is accessed directly from its original location), and enable efficient SIMD operations.

This approach is essential for building competitive AI applications in C#, where the difference between 10ms and 100ms per inference can be the difference between a viable product and a sluggish prototype.

Basic Code Example

Let's consider a scenario in a high-performance AI pipeline where we need to calculate the dot product of two vectors (representing token embeddings) repeatedly. In naive C#, passing a struct representing a vector to a method causes a defensive copy of the entire struct to be made on the stack. If the struct is large (e.g., holding 1024 floats for an embedding), this copying overhead accumulates significantly, causing CPU cache misses and slowing down token processing. The following example demonstrates how to eliminate these copies using in parameters and readonly struct modifiers.

using System;
using System.Numerics; // Required for Vector<T> (SIMD)

namespace HighPerformanceAI
{
    // Represents a vector of floats for AI embeddings.
    // We mark it as 'readonly struct' to guarantee immutability.
    // This allows the compiler to safely pass references to this struct
    // instead of copying it, provided we also use 'in' parameters.
    public readonly struct EmbeddingVector
    {
        private readonly float[] _values;

        public EmbeddingVector(float[] values)
        {
            // Defensive copy is necessary here during construction, 
            // but we want to avoid copies during processing.
            _values = values;
        }

        public int Length => _values.Length;

        // Indexer to access elements.
        // 'readonly' modifier ensures this method does not modify struct state.
        public readonly float this[int index] => _values[index];

        // Calculates the dot product with another vector.
        // 'in' parameter avoids copying the 'other' struct.
        // 'readonly' modifier on the method ensures we don't modify 'this'.
        public readonly float DotProduct(in EmbeddingVector other)
        {
            if (Length != other.Length)
                throw new ArgumentException("Vectors must be of the same length.");

            float sum = 0f;

            // In a real high-performance scenario, we would use Vector<T> (SIMD) here.
            // For this basic example, we use a simple loop to demonstrate the mechanics.
            for (int i = 0; i < Length; i++)
            {
                sum += this[i] * other[i];
            }

            return sum;
        }
    }

    class Program
    {
        static void Main()
        {
            // 1. Setup data
            float[] dataA = new float[1000];
            float[] dataB = new float[1000];

            // Fill with dummy data
            var random = new Random();
            for(int i = 0; i < 1000; i++)
            {
                dataA[i] = (float)random.NextDouble();
                dataB[i] = (float)random.NextDouble();
            }

            // 2. Create struct instances
            // Note: Structs are value types. These are allocated on the stack (if local)
            // or inline inside the containing type.
            var vectorA = new EmbeddingVector(dataA);
            var vectorB = new EmbeddingVector(dataB);

            // 3. Perform calculation
            // Without 'in', 'vectorB' would be copied entirely into the DotProduct method.
            // With 'in', we pass a reference (effectively a pointer) to vectorB.
            float similarity = vectorA.DotProduct(in vectorB);

            Console.WriteLine($"Dot Product Result: {similarity:F4}");
        }
    }
}

Detailed Explanation

  1. Struct Definition (EmbeddingVector):

    • We define a struct named EmbeddingVector. In C#, structs are value types. By default, when you pass a value type to a method, the runtime creates a complete copy of that data on the stack. For a struct containing an array reference and potentially other fields, this is cheap. However, if the struct contained the float data directly (inline) rather than a reference to an array, copying a large vector (e.g., 1024 floats) would be extremely expensive.
    • readonly struct: We apply the readonly struct modifier. This enforces that all fields of the struct are immutable. The compiler uses this guarantee to optimize memory. Crucially, it allows the compiler to pass the struct by reference (using in) without worrying that the callee will modify the original data.

  2. Fields and Constructor:

    • private readonly float[] _values: The actual data is stored in an array. Since arrays are reference types, the struct itself is small (just holding the reference pointer).
    • Constructor: We initialize the array. Note that we are creating a defensive copy here by passing float[] values by value. However, we are copying the reference to the array, not the array data itself. This is a standard and acceptable pattern.

  3. The DotProduct Method:

    • public readonly float DotProduct(in EmbeddingVector other):
      • in EmbeddingVector other: This is the key optimization. The in keyword passes the parameter by reference (similar to a pointer) rather than by value. This means no copy of other is created. If EmbeddingVector were a large struct (containing data inline), this would save significant CPU cycles and memory bandwidth.
      • readonly method: The readonly modifier on the method signature ensures that we cannot modify any fields of this inside the method. This is required because we are using in on other; the compiler needs to ensure that the method doesn't violate the immutability contract.
    • Logic: The loop iterates through the arrays. Because the arrays are reference types, accessing this[i] and other[i] retrieves data from the heap arrays referenced by the structs. The in parameter ensures we access the other struct's array reference directly from the original memory location.

  4. Execution in Main:

    • We create two instances of EmbeddingVector.
    • We call vectorA.DotProduct(in vectorB). The in keyword here explicitly tells the compiler to pass a reference to vectorB. Without it, the compiler would default to passing a copy.
    • The result is printed.

Visualizing Memory Layout

The following diagram illustrates the difference between passing by value (causing a defensive copy) and passing by in reference.

This diagram visually contrasts passing by value, which creates a defensive copy of the data, with passing by in reference, which allows direct access to the original memory address without duplication.
Hold "Ctrl" to enable pan & zoom

This diagram visually contrasts passing by value, which creates a defensive copy of the data, with passing by `in` reference, which allows direct access to the original memory address without duplication.

Common Pitfalls

  1. in with Heap-Allocated Structs Containing References: While in avoids copying the struct itself, it does not freeze the memory of the objects the struct references. If the struct contains a reference to a mutable class (like float[]), the in parameter prevents you from reassigning that reference, but it does not prevent the method from modifying the contents of the array (e.g., other._values[0] = 0;). If you need to guarantee the data is immutable, the array itself must be treated as immutable or wrapped in an immutable collection.

  2. readonly Method Requirements: If you define a method with an in parameter, the compiler may require the method to be marked readonly if the struct itself is readonly. Failing to do so will result in a compilation error because the compiler assumes you might modify this state, which conflicts with the safety guarantees of in parameters.

  3. Using in with Primitive Types: Do not use in for primitive types like int, float, or double. These types are smaller than or equal to a reference size (4-8 bytes). Passing them by value is faster than passing them by reference because passing a reference requires an extra indirection (dereferencing the pointer), whereas passing by value uses a CPU register.

  4. Ref Returns and in: You cannot return an in parameter as a ref return. The in parameter is a temporary reference with a limited lifetime (usually the duration of the method call). Attempting to return it as a ref would expose a reference to a stack location that might be invalid after the method returns, leading to undefined behavior.

The chapter continues with advanced code, exercises and solutions with analysis, you can find them on the ebook on Leanpub.com or Amazon

Loading knowledge check...

Code License: All code examples are released under the MIT License. Github repo.

Content Copyright: Copyright © 2026 Edgar Milvus | Privacy & Cookie Policy. All rights reserved.

All textual explanations, original diagrams, and illustrations are the intellectual property of the author. To support the maintenance of this site via AdSense, please read this content exclusively online. Copying, redistribution, or reproduction is strictly prohibited.