Backend Engineer TypeScript & Node.js: Singleflight Cache Stampede, Stale-While-Revalidate & Circuit Breaker Upstream
Dua penyebab paling umum yang menjatuhkan layanan API TypeScript & Node.js bertrafik tinggi adalah: (1) kedaluwarsa-nya kunci cache populer (misalnya leaderboard global) yang memicu 20+ kueri identik menghantam PostgreSQL secara serentak (cache stampede / thundering herd), serta (2) panggilan HTTP ke layanan upstream tanpa batas timeout yang memblokir koneksi hingga probe /healthz mengalami timeout berantai.
1. Menghentikan Cache Stampede di TypeScript dengan SingleflightCache<T> & SWR
Ketika kunci cache ber-TTL 300 detik kedaluwarsa saat jam sibuk, pola naif if (!cached) await loadFromDb() langsung mengirim N kueri berat ke database sekaligus. Dengan menyimpan Promise<T> yang sedang berjalan di peta inflight serta memisahkan batas segar (fresh TTL) dan masa tenggang stale-while-revalidate, beban PostgreSQL turun drastis hingga di atas 95%:
interface CacheRecord<T> {
value: T;
freshUntil: number;
staleUntil: number;
}
export class SingleflightCache<T> {
private readonly store = new Map<string, CacheRecord<T>>();
private readonly inflight = new Map<string, Promise<T>>();
constructor(
private readonly freshTtlMs: number = 300_000,
private readonly staleGraceMs: number = 120_000
) {}
async getOrLoad(key: string, loader: () => Promise<T>): Promise<T> {
const now = Date.now();
const hit = this.store.get(key);
if (hit && now < hit.freshUntil) {
return hit.value;
}
if (hit && now < hit.staleUntil) {
void this.refreshOnce(key, loader).catch(() => undefined);
return hit.value;
}
return this.refreshOnce(key, loader);
}
private refreshOnce(key: string, loader: () => Promise<T>): Promise<T> {
const existing = this.inflight.get(key);
if (existing) {
return existing;
}
const job = (async () => {
try {
const value = await loader();
const jitter = Math.floor(Math.random() * this.freshTtlMs * 0.1);
const now = Date.now();
this.store.set(key, {
value,
freshUntil: now + this.freshTtlMs + jitter,
staleUntil: now + this.freshTtlMs + jitter + this.staleGraceMs,
});
return value;
} finally {
this.inflight.delete(key);
}
})();
this.inflight.set(key, job);
return job;
}
}
2. Menjaga /healthz Tetap Responsif Saat Layanan Upstream Melambat
Di layanan Express, Fastify, maupun NestJS, panggilan fetch() tanpa AbortSignal.timeout() dapat menggantung bermenit-menit saat host tujuan bermasalah. Lengkapi klien upstream dengan Circuit Breaker dan jaga agar endpoint /healthz tidak ikut memblokir event loop Node.js:
export class UpstreamCircuitBreaker {
private failures = 0;
private openUntil = 0;
constructor(
private readonly failureThreshold = 5,
private readonly cooldownMs = 15_000,
private readonly timeoutMs = 450
) {}
async callInference<T>(url: string, body: unknown, fallback: () => T): Promise<T> {
if (Date.now() < this.openUntil) {
return fallback();
}
try {
const res = await fetch(url, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify(body),
signal: AbortSignal.timeout(this.timeoutMs),
});
if (!res.ok) {
throw new Error(`Upstream HTTP ${res.status}`);
}
this.failures = 0;
return (await res.json()) as T;
} catch (err) {
this.failures += 1;
if (this.failures >= this.failureThreshold) {
this.openUntil = Date.now() + this.cooldownMs;
}
throw err;
}
}
}
3. Panduan Pilar Backend & Caching Terkait
- Arsitektur Downstream (CDN / RFC 7234) vs Upstream (Redis) Caching
- Pola Senior Backend Engineer — Transactional Outbox, Idempotensi Webhook & Kunci PostgreSQL
- Panduan Pilar Backend Engineer — Python, Go, TypeScript & Sistem Terdistribusi
- Panduan Pilar Full Stack Engineer — React, Next.js 15, TypeScript & WebSocket
Percepat Backend TypeScript & Node.js Anda dalam 1 Hari Kerja
Mengalami lonjakan p99 latency, cache stampede PostgreSQL, atau timeout upstream API? Pesan Layanan Pemrograman 1 Hari seharga Rp 1.000.000 (QRIS / Virtual Account Bank Indonesia) atau $100 USD (Kartu Kredit/Debit) melalui /id/1-day-service/ atau rekrut langsung di Upwork.