Node.js Concurrency & Worker Threads — Study Guide¶

A comprehensive guide to understanding Node.js performance, concurrency models, and when (and when not) to use Worker Threads.

Table of Contents¶

1. The Three Execution Systems
2. Event Loop
3. libuv Thread Pool
4. Worker Threads
5. Data Transfer & Structured Clone
6. Worker Pool Pattern
7. Queue Behaviour Under Load
8. When Workers Help vs. Hurt
9. Common Mistakes
10. Production Failure Modes
11. Real-World Architecture
12. Code Examples
13. Debugging Workers
14. Key Rules to Memorise
15. Practice Scenarios

1. The Three Execution Systems¶

Node.js has three distinct execution layers. Understanding these removes most confusion.

Node.js Process
      │
 ┌────┼─────────────┐
 ▼    ▼              ▼
Event Loop   libuv Pool   Worker Threads
 (JS logic)  (OS tasks)     (CPU JS)

2. Event Loop¶

The Event Loop runs all JavaScript on a single thread using non-blocking I/O and cooperative multitasking.

How I/O Works Efficiently¶

app.get("/user", async (req, res) => {
  const user = await db.query("SELECT * FROM users"); // non-blocking wait
  res.json(user);
});

While waiting for the database, the event loop handles other requests — no thread wasted.

What Blocks the Event Loop¶

These synchronous operations freeze the entire server while running:

Example of blocking impact:

This is called Event Loop Blocking — the entire API stalls during heavy computation.

3. libuv Thread Pool¶

Node uses libuv internally to run blocking OS tasks on background threads.

• Default pool size: 4 threads
• Max pool size: 128

APIs That Use the libuv Pool¶

fs.readFile · fs.writeFile · crypto.pbkdf2 · zlib.gzip · dns.lookup

Saturation Example¶

10 crypto.pbkdf2 calls arrive:
  → 4 execute immediately (one per thread)
  → 6 wait in queue

Increase the pool size via environment variable:

UV_THREADPOOL_SIZE=8 node app.js

Diagnosing Pool Saturation¶

4. Worker Threads¶

Introduced in Node.js v10.5.0, Worker Threads allow parallel execution of CPU-heavy JavaScript inside the same process.

Why They Exist¶

Before Worker Threads, developers had two poor options:

1. child_process / cluster — heavy memory, slow startup, complex IPC
2. Separate service (e.g. Python worker) — two systems to maintain

What Each Worker Gets¶

Workers run a separate JavaScript runtime, enabling true parallelism. Memory is isolated by default to prevent race conditions and data corruption.

Worker Lifecycle Costs¶

Key insight: If your task takes < 100 ms, the overhead of creating a worker can exceed the task itself.

The Tradeoffs¶

1. Thread creation overhead — workers are expensive to spin up
2. Memory duplication — each worker loads its own runtime
3. Data transfer cost — large payloads are copied, not shared
4. Complexity — multi-threaded debugging is harder

5. Data Transfer & Structured Clone¶

Workers communicate via message passing using the Structured Clone Algorithm.

// Main thread sends
worker.postMessage(data);

// Worker receives
parentPort.on("message", (data) => { /* ... */ });

The Hidden Cost¶

Objects are copied, not shared:

Send 150 MB object to a worker:
  Main thread  → 150 MB
  Worker       → 150 MB
  Total memory → 300 MB

Additional costs: serialisation CPU time, deserialisation, and GC pressure from large temporary objects.

Best Practice¶

• Send small messages (IDs, parameters)
• Let the worker fetch its own data if needed
• Avoid sending large JSON blobs

6. Worker Pool Pattern¶

Never create a worker per request. Instead, pre-create a pool of long-lived workers.

Incoming Requests
       │
       ▼
    Job Queue
       │
 ┌─────┼─────┐
 ▼     ▼     ▼
Worker Worker Worker  (long-lived, reused)

Sizing¶

workers ≈ CPU cores (or cores − 1 to leave room for the event loop)

Example: 8-core machine → 7–8 workers

Recommended Libraries¶

• Piscina
• workerpool

7. Queue Behaviour Under Load¶

When jobs arrive faster than workers can process them, the queue grows.

Throughput Calculation¶

Workers:     8
Task time:   500 ms each
Throughput:  8 jobs / 500 ms = 16 jobs/sec

Latency Under Spike¶

Memory Risk with Unbounded Queues¶

20,000 queued jobs × 2 MB each = 40 GB → server crash

Mitigation Strategies¶

• Queue size limits — reject when full
• Rate limiting — throttle incoming requests
• Backpressure — return 503 to clients
• Streaming — avoid buffering full payloads

8. When Workers Help vs. Hurt¶

✅ Use Workers When¶

• Task is CPU-bound (not I/O)
• Task takes > 100–200 ms
• Input/output is small
• Tasks are independent of each other

Good candidates: image resizing, password hashing, data compression, ML inference, large JSON parsing

❌ Do NOT Use Workers When¶

• Task is < 100 ms — overhead exceeds benefit
• Task is I/O-bound — databases and HTTP are already async
• Data transfer is large — clone cost dominates
• You'd create a new worker per request — memory explosion

Example of workers making things worse:

Task:    20 ms CPU
Worker:  ~30 ms startup + 20 ms task + ~10 ms teardown = ~60 ms
Direct:  20 ms

9. Common Mistakes¶

10. Production Failure Modes¶

Event Loop Blocking¶

Thread Pool Saturation¶

Queue Memory Explosion¶

11. Real-World Architecture¶

For heavy workloads, production teams often separate compute from the API:

HTTP API (Node.js)
      │
      ▼
   Job Queue (Redis / Kafka)
      │
 ┌────┼────────┐
 ▼    ▼        ▼
Worker Services (separate processes or containers)

Benefits:

• Isolated failures — worker crash doesn't affect the API
• Horizontal scaling — add workers independently
• No API latency spikes
• Separate memory budgets

Senior engineer mindset: Don't just ask "should I use a worker thread?" Ask: "Should this computation run in the API process at all?"

12. Code Examples¶

Main Thread¶

// main.js
const { Worker } = require("worker_threads");

function runWorker(input) {
  return new Promise((resolve, reject) => {
    const worker = new Worker("./worker.js", {
      workerData: input,
    });

    worker.on("message", resolve);
    worker.on("error", reject);
    worker.on("exit", (code) => {
      if (code !== 0) {
        reject(new Error(`Worker stopped with exit code ${code}`));
      }
    });
  });
}

runWorker(100_000_000).then((result) => {
  console.log("Result:", result);
});

Worker File¶

// worker.js
const { parentPort, workerData } = require("worker_threads");

function heavyCalculation(n) {
  let sum = 0;
  for (let i = 0; i < n; i++) {
    sum += i;
  }
  return sum;
}

const result = heavyCalculation(workerData);
parentPort.postMessage(result);

⚠️ Common Bug: workerData vs postMessage¶

Wrong — worker reads workerData, but main sends via postMessage:

// main.js — sends via postMessage
worker.postMessage(1000000);

// worker.js — reads workerData (will be undefined!)
const result = calc(workerData);

Fix — either pass data as workerData in the constructor, or listen with parentPort.on("message") in the worker:

// Option A: Use workerData
new Worker("./worker.js", { workerData: 1000000 });

// Option B: Use message passing
parentPort.on("message", (n) => {
  parentPort.postMessage(calc(n));
});

13. Debugging Workers¶

Strategy¶

• Use structured logging with worker IDs
• Implement message tracing between main and worker
• Add timeout protection — kill workers that run too long
• Always handle errors:

worker.on("error", (err) => console.error(`Worker error: ${err.message}`));
worker.on("exit", (code) => {
  if (code !== 0) console.error(`Worker exited with code ${code}`);
});

14. Key Rules to Memorise¶

15. Practice Scenarios¶

Scenario A: No Workers vs. 4 Workers¶

10 requests arrive simultaneously, each with a 2-second CPU task.

Scenario B: Video Upload API¶

200 uploads arrive; each needs 10 s of CPU compression. Server has 8 cores.

• Workers: 7–8
• Batches: ceil(200 / 8) = 25
• Total time: 25 × 10 s = 250 s
• Why not 200 workers? 200 × 20 MB = 4 GB memory + context-switching overhead

Scenario C: Tiny Task¶

Task takes 20 ms. Worker creation takes 30 ms.

• With worker: ~60 ms total
• Without worker: 20 ms
• Verdict: Workers make it worse

Scenario D: Large Queue + Small Payload¶

100 requests, 8 workers, 500 ms per task.

• Batches: ceil(100 / 8) = 13
• Total: 13 × 500 ms = 6.5 s

Scenario E: Low CPU but Slow System¶

8 workers, 20,000 queued jobs, 2 MB each, CPU at 40%.

• Queue memory: 20,000 × 2 MB = 40 GB
• Bottleneck: Memory pressure, GC pauses, and swap — not CPU

Practice Exercises¶

1. Write a worker that calculates factorial
2. Build a worker pool with 4 workers
3. Benchmark JSON.parse of a 50 MB string on the event loop vs. a worker thread
4. Simulate 1,000 jobs with a queue and measure per-job latency

Final thought: The goal isn't to use worker threads everywhere — it's to understand the execution model so you choose the right tool for each problem.