I (SRE) Need to Shift Left Too

As a reliability engineer, I have fallen into a curious hypocrisy. I read a couple books and began to preach shifting left to everyone else while remaining comfortably anchored on the far right of the lifecycle.

Telemetry Doesn’t Mean You Can Overlook Design

For over a decade, the software industry has rallied around a single operational imperative: Shift Left. We tell teams to embed security scanners into feature branches, run unit tests before pull requests merge, and embed operational telemetry directly into story point creation. We demand that reliability and performance stop being an afterthought.

Sitting downstream on the telemetry bubble, we build the dashboard with our cute little golden signals, wait for the latency spike trends, analyze heap dumps, and ring bells on the pod that is spiraling into a crash loop. We bring our cool techie skills to post-mortems and incident response, pointing to the exact log where a service started faltering. We can even hand a hot (pun intended) flame graph to a dev lead showing precisely which method saturated the CPU.

Delivering the diagnosis, reporting the symptom, and walking away is too far right of an approach for a team so dedicated to everyone else “Shifting Left”.

flowchart TB
    subgraph Title ["THE TELEMETRY GAP"]
        direction LR
        
        subgraph Upstream ["Upstream Architecture & Design Phase"]
            direction TB
            U_Goal["<b>Shift-Left Goal</b><br/><i>Where Engineering Thinking Should Happen</i>"]
            
            subgraph Focus ["Core Engineering Foundations"]
                F1["System Design & Trade-offs"]
                F2["Memory Models & Heap Allocation"]
                F3["Threading & Async Mechanics"]
            end
        end

        subgraph Downstream ["Downstream Production & SRE Engagement"]
            direction TB
            D_Goal["<b>Legacy SRE Domain</b><br/><i>Reactive Telemetry & Observability</i>"]
            
            subgraph Symptoms ["Symptom Management"]
                S1["Alerts & OOM Thresholds"]
                S2["CPU Spikes & Latency"]
                S3["Pod Evictions & Log Costs"]
            end
        end

        %% Left-Shift Bridge
        Upstream == "Shift-Left Engineering Integrity" ==> Downstream
        Downstream -. "Symptom Feedback Loop (Telemetry)" .-> Upstream
    end

    %% Styling
    classDef header fill:#1e293b,color:#f8fafc,stroke:#475569,stroke-width:2px;
    classDef leftShift fill:#0f766e,color:#f0fdf4,stroke:#14b8a6,stroke-width:1px;
    classDef rightShift fill:#334155,color:#f8fafc,stroke:#64748b,stroke-width:1px;
    
    class U_Goal leftShift;
    class D_Goal rightShift;

Telling a software engineer that their memory keeps climbing until Kubernetes evicts the pod lacks the RCA. It tells them what happened, but leaves them entirely alone with the why. Understanding memory behavior, concurrency models, and architectural trade-offs creates reliability, not just better dashboards.

Reliability in Architectural Design

When we treat services as black boxes and focus only on the metrics they emit, reliability work becomes reactive pattern matching.

Some right-wing polcies I have succumbed to: A pod runs out of memory –> I recommend we increase the container limits. A thread pool starves –> I build some autoremediation to scale horizontally.

It took me way too long to grasp that scaling may just be hiding an architectural flaw.

I recently looked at a service that was repeatedly getting OOMKilled in Kubernetes. My recommendation as an SRE focused purely on the “right side” of the pipeline was to double the memory limit. It worked briefly, but the service eventually started spiking again. The real issue was large payload buffering that created pressure on the Large Object Heap.

To shift SRE thinking left, we have to understand the runtime engine underneath. The concepts are basic, but essential to understanding the why when we are building our monitoring for a .NET app.

.NET memory mechanics in runtime

When memory saturates in a containerized environment, my instinctive reaction was to blame a memory leak. But in managed runtimes like .NET, memory behavior is intimately tied to the Garbage Collection (GC) mode selected at build or deployment time. And it’s not my job to recommend which mode to use, but it is my job to understand why it was used. Making sure I have a grasp on the design patterns help me understand how to autoremediate if inevitable incidents occur.

flowchart TB
    subgraph Title["CLR GARBAGE COLLECTION"]
        direction TB

        subgraph Modes[" Execution Modes "]
            direction LR

            subgraph Workstation["Workstation GC"]
                W1["- 1 Heap / GC Thread"]
                W2["- Low Latency, UI Focus"]
            end

            subgraph Server["Server GC"]
                S1["- 1 Heap & 1 GC Thread PER Core"]
                S2["- High Throughput, Higher Memory Footprint"]
            end
        end

        subgraph Heaps[" Heap Segments "]
            direction LR

            subgraph SOH["Small Object Heap (SOH)"]
                SOH1["- Compacted automatically"]
                SOH2["- Low fragmentation risk"]
            end

            subgraph LOH["Large Object Heap (LOH)"]
                LOH1["- Objects >= 85,000 bytes"]
                LOH2["- NOT compacted by default -> Fragmentation"]
            end
        end
    end

    %% Styling
    classDef box fill:#1e293b,stroke:#475569,stroke-width:1px,color:#f8fafc;
    class W1,W2,S1,S2,SOH1,SOH2,LOH1,LOH2 box;

The CLR offers two distinct modes of execution: Server GC and Workstation GC.

• Server GC creates dedicated heaps and GC threads for each logical CPU core. It maximizes throughput by allowing parallel garbage collection, but it does so at the cost of a significantly larger memory footprint.

• Workstation GC, by contrast, uses a single heap optimized for low latency and UI responsiveness.

If a microservice is deployed inside a Kubernetes pod with limited memory boundaries while running Server GC, the runtime will aggressively allocate memory heaps across every available core. The team will observe OOM (Out Of Memory) kills and assume the application has a leak, when in reality, the operational environment is in direct conflict with the runtime’s memory allocation strategy.

Furthermore, how the application handles data structures directly impacts GC health. Objects larger than or equal to 85,000 bytes are assigned to the Large Object Heap (LOH). Unlike standard object heaps, the LOH is not compacted by default because moving massive memory blocks is computationally expensive. Over time, sporadic large allocations cause severe LOH fragmentation—leading to memory pressure spikes even when active object counts remain low.

SRE Perspective Shift: Instead of simply alerting on memory consumption, track dotnet_gc_allocation_size_bytes and LOH fragmentation metrics. When high-volume data streams cause LOH fragmentation, help the development team evaluate targeted compaction strategies, such as setting GCSettings.LargeObjectHeapCompactorMode = GCLargeObjectHeapCompactorMode.CompactOnce; during controlled maintenance routines or streaming large payloads rather than buffering them entirely in memory.

Async Patterns and Thread Pool Exhaustion

A similar disconnect occurs around concurrency. High latency under load is often misdiagnosed as downstream network slow-downs or database bottlenecking, when the root cause is self-inflicted thread starvation.

In asynchronous programming, calling synchronous blocking methods on asynchronous tasks—commonly known as sync-over-async (such as invoking .Result or .Wait() on a Task)—creates an operational deadlock risk. When a worker thread blocks waiting for an async task to complete, it is unavailable to process other incoming work.

flowchart TD
    Req["Incoming Request"] --> ThA["Thread Pool Thread A"]
    ThA -- "Calls .Result" --> Blk["Blocks waiting for Task"]
    ThA --> Starv["Thread Pool Starvation"]
    Blk --> Starv
    Starv --> Limit["CLR Growth Rate Limit<br/>(~1-2 threads/sec/core)"]
    Limit --> Latency["Latency Spike"]

    %% Styling
    classDef default fill:#1e293b,stroke:#475569,stroke-width:1px,color:#f8fafc;
    classDef alert fill:#881337,stroke:#f43f5e,stroke-width:1px,color:#ffffff;
    class Starv,Limit,Latency alert;

Under heavy traffic, the CLR thread pool quickly depletes its available worker threads. While the thread pool will scale up to handle demand, its growth rate is intentionally throttled (often injecting only 1 to 2 new threads per second per core to prevent CPU thrashing). The resulting queue backup looks like system-wide network degradation, but it is actually the direct consequence of a fundamental anti-pattern in code structure.

The Upstream Mindset

To truly shift our thinking, SREs should be involved earlier in architecture reviews, not just incident reviews.

When we understand the runtime concepts like:

• memory allocation
• thread scheduling
• telemetry configuration

our conversations with development leaders transform. Shifting left is not just a practice for developers. It is a challenge for SREs as well. The goal is not to become architects. The goal is to understand architectural intent well enough that reliability is designed into systems before telemetry has to explain why they failed.