Backend Engineer TypeScript & Node.js: Singleflight Cache Stampede, Stale-While-Revalidate & Circuit Breaker Upstream

Dua penyebab paling umum yang menjatuhkan layanan API TypeScript & Node.js bertrafik tinggi adalah: (1) kedaluwarsa-nya kunci cache populer (misalnya leaderboard global) yang memicu 20+ kueri identik menghantam PostgreSQL secara serentak (cache stampede / thundering herd), serta (2) panggilan HTTP ke layanan upstream tanpa batas timeout yang memblokir koneksi hingga probe /healthz mengalami timeout berantai.

22 Request Bersamaan Data Stale Valid (SWR) Respon Instan < 2ms Peta In-Flight Singleflight 1 Promise Bersama per Kunci 1 Kueri PostgreSQL Saja Connection Pool Tetap Aman
Mekanisme Singleflight dan Stale-While-Revalidate (SWR): puluhan cache miss serentak digabung menjadi satu Promise ke PostgreSQL.

1. Menghentikan Cache Stampede di TypeScript dengan SingleflightCache<T> & SWR

Ketika kunci cache ber-TTL 300 detik kedaluwarsa saat jam sibuk, pola naif if (!cached) await loadFromDb() langsung mengirim N kueri berat ke database sekaligus. Dengan menyimpan Promise<T> yang sedang berjalan di peta inflight serta memisahkan batas segar (fresh TTL) dan masa tenggang stale-while-revalidate, beban PostgreSQL turun drastis hingga di atas 95%:

interface CacheRecord<T> {
  value: T;
  freshUntil: number;
  staleUntil: number;
}

export class SingleflightCache<T> {
  private readonly store = new Map<string, CacheRecord<T>>();
  private readonly inflight = new Map<string, Promise<T>>();

  constructor(
    private readonly freshTtlMs: number = 300_000,
    private readonly staleGraceMs: number = 120_000
  ) {}

  async getOrLoad(key: string, loader: () => Promise<T>): Promise<T> {
    const now = Date.now();
    const hit = this.store.get(key);

    if (hit && now < hit.freshUntil) {
      return hit.value;
    }

    if (hit && now < hit.staleUntil) {
      void this.refreshOnce(key, loader).catch(() => undefined);
      return hit.value;
    }

    return this.refreshOnce(key, loader);
  }

  private refreshOnce(key: string, loader: () => Promise<T>): Promise<T> {
    const existing = this.inflight.get(key);
    if (existing) {
      return existing;
    }

    const job = (async () => {
      try {
        const value = await loader();
        const jitter = Math.floor(Math.random() * this.freshTtlMs * 0.1);
        const now = Date.now();
        this.store.set(key, {
          value,
          freshUntil: now + this.freshTtlMs + jitter,
          staleUntil: now + this.freshTtlMs + jitter + this.staleGraceMs,
        });
        return value;
      } finally {
        this.inflight.delete(key);
      }
    })();

    this.inflight.set(key, job);
    return job;
  }
}

2. Menjaga /healthz Tetap Responsif Saat Layanan Upstream Melambat

Di layanan Express, Fastify, maupun NestJS, panggilan fetch() tanpa AbortSignal.timeout() dapat menggantung bermenit-menit saat host tujuan bermasalah. Lengkapi klien upstream dengan Circuit Breaker dan jaga agar endpoint /healthz tidak ikut memblokir event loop Node.js:

export class UpstreamCircuitBreaker {
  private failures = 0;
  private openUntil = 0;

  constructor(
    private readonly failureThreshold = 5,
    private readonly cooldownMs = 15_000,
    private readonly timeoutMs = 450
  ) {}

  async callInference<T>(url: string, body: unknown, fallback: () => T): Promise<T> {
    if (Date.now() < this.openUntil) {
      return fallback();
    }

    try {
      const res = await fetch(url, {
        method: "POST",
        headers: { "Content-Type": "application/json" },
        body: JSON.stringify(body),
        signal: AbortSignal.timeout(this.timeoutMs),
      });

      if (!res.ok) {
        throw new Error(`Upstream HTTP ${res.status}`);
      }

      this.failures = 0;
      return (await res.json()) as T;
    } catch (err) {
      this.failures += 1;
      if (this.failures >= this.failureThreshold) {
        this.openUntil = Date.now() + this.cooldownMs;
      }
      throw err;
    }
  }
}

3. Panduan Pilar Backend & Caching Terkait

Percepat Backend TypeScript & Node.js Anda dalam 1 Hari Kerja

Mengalami lonjakan p99 latency, cache stampede PostgreSQL, atau timeout upstream API? Pesan Layanan Pemrograman 1 Hari seharga Rp 1.000.000 (QRIS / Virtual Account Bank Indonesia) atau $100 USD (Kartu Kredit/Debit) melalui /id/1-day-service/ atau rekrut langsung di Upwork.


About the Author — Hire Aldian Fazrihady (Cloud Engineer, GCP Professional Cloud Architect, GCP Professional Data Engineer, Backend, AI & Full Stack Engineer)

Aldian Fazrihady is a Top-Rated Senior Cloud Engineer, GCP Professional Cloud Architect (Google Cloud Certified), GCP Professional Data Engineer (Google Cloud Certified), Backend Engineer (Python, Django, FastAPI, Go, PostgreSQL), AI Application Engineer (LLMs, RAG, MCP, Google ADK, LangGraph), and Full Stack Engineer (React, Next.js, TypeScript) with 15+ years of production software engineering experience.

Hire Aldian Fazrihady on Upwork | Book 1-Day Programming Service ($100 USD / Rp 1.000.000) | View Live AI Portfolio | Send a Direct Message

Related Engineering Articles

Engineering Role Pillar Guides & Category Hubs