Part 2: Node.js Architecture Deep Dive

Understanding the Event Loop, libuv, and Async I/O Internals

The Problem with Understanding Node.js

• What the event loop really is and its phases
• How libuv manages async I/O
• When Node.js uses threads (yes, it does)
• The difference between microtasks and macrotasks
• Why setImmediate vs setTimeout(fn, 0) matters

Node.js Architecture Overview

┌─────────────────────────────────────────────────────────────────────────────┐
│                           Node.js Process                                    │
│                                                                              │
│  ┌────────────────────────────────────────────────────────────────────────┐ │
│  │                        JavaScript Code                                  │ │
│  │                    (Your application code)                              │ │
│  └────────────────────────────────────────────────────────────────────────┘ │
│                                    │                                         │
│                                    ▼                                         │
│  ┌────────────────────────────────────────────────────────────────────────┐ │
│  │                      Node.js Core Bindings                              │ │
│  │         (C++ code that bridges JavaScript and system calls)             │ │
│  │                                                                          │ │
│  │  fs bindings │ http bindings │ crypto bindings │ net bindings │ etc.    │ │
│  └────────────────────────────────────────────────────────────────────────┘ │
│                     │                              │                         │
│                     ▼                              ▼                         │
│  ┌──────────────────────────────┐  ┌────────────────────────────────────┐   │
│  │           V8 Engine          │  │              libuv                  │   │
│  │                              │  │                                     │   │
│  │  - JavaScript compilation    │  │  - Event loop                      │   │
│  │  - Execution                 │  │  - Thread pool (4 threads default) │   │
│  │  - Memory management         │  │  - Async I/O                       │   │
│  │  - Garbage collection        │  │  - Timers                          │   │
│  │                              │  │  - Network I/O                     │   │
│  └──────────────────────────────┘  └────────────────────────────────────┘   │
│                                                                              │
└─────────────────────────────────────────────────────────────────────────────┘

The Event Loop: Not What You Think

Event Loop Phases

   ┌───────────────────────────┐
┌─>│         timers            │  setTimeout, setInterval callbacks
│  └─────────────┬─────────────┘
│  ┌─────────────▼─────────────┐
│  │     pending callbacks     │  I/O callbacks deferred from previous iteration
│  └─────────────┬─────────────┘
│  ┌─────────────▼─────────────┐
│  │       idle, prepare       │  Internal use only
│  └─────────────┬─────────────┘
│  ┌─────────────▼─────────────┐
│  │           poll            │  Retrieve new I/O events, execute I/O callbacks
│  └─────────────┬─────────────┘
│  ┌─────────────▼─────────────┐
│  │          check            │  setImmediate callbacks
│  └─────────────┬─────────────┘
│  ┌─────────────▼─────────────┐
│  │      close callbacks      │  socket.on('close'), etc.
│  └─────────────┬─────────────┘
│                │
└────────────────┘

1. Timers Phase

javascript
// timer-phase.js
console.log('Script start');

setTimeout(() => {
    console.log('setTimeout 1');
}, 0);

setTimeout(() => {
    console.log('setTimeout 2');
}, 0);

console.log('Script end');

// Output:
// Script start
// Script end
// setTimeout 1
// setTimeout 2

javascript
// timer-accuracy.js
const start = Date.now();

setTimeout(() => {
    console.log(`Executed after ${Date.now() - start}ms`);
}, 100);

// Simulate blocking for 200ms
const blockUntil = Date.now() + 200;
while (Date.now() < blockUntil) {
    // Blocking the event loop
}

// Output: "Executed after ~200ms" (not 100ms)
// The timer was ready at 100ms, but we blocked the loop

2. Pending Callbacks Phase

3. Poll Phase

1. Calculates how long to block and poll for I/O
2. Processes events in the poll queue

javascript
// poll-phase.js
const fs = require('fs');

console.log('Start');

// This callback will be processed in the poll phase
fs.readFile('./package.json', () => {
    console.log('File read complete');
});

console.log('End');

// Output:
// Start
// End
// File read complete

• If poll queue is not empty, iterate through callbacks synchronously
• If poll queue is empty:
  • If setImmediate() is scheduled, end poll phase and go to check phase
  • If no setImmediate(), wait for callbacks to be added, then execute immediately

4. Check Phase

javascript
// check-phase.js
const fs = require('fs');

fs.readFile('./package.json', () => {
    setTimeout(() => {
        console.log('setTimeout');
    }, 0);

    setImmediate(() => {
        console.log('setImmediate');
    });
});

// Output (guaranteed order):
// setImmediate
// setTimeout

1. After I/O callback (poll phase), we go to check phase
2. setImmediate() runs in check phase
3. setTimeout() runs in the next iteration's timers phase

5. Close Callbacks Phase

javascript
// close-phase.js
const net = require('net');

const server = net.createServer((socket) => {
    socket.on('close', () => {
        console.log('Socket closed'); // Runs in close callbacks phase
    });
});

Microtasks: process.nextTick and Promises

The Microtask Queues

1. process.nextTick() queue (higher priority)
2. Promise callbacks queue

javascript
// microtasks.js
console.log('1. Script start');

setTimeout(() => {
    console.log('6. setTimeout');
}, 0);

Promise.resolve().then(() => {
    console.log('4. Promise 1');
}).then(() => {
    console.log('5. Promise 2');
});

process.nextTick(() => {
    console.log('3. nextTick');
});

console.log('2. Script end');

// Output:
// 1. Script start
// 2. Script end
// 3. nextTick
// 4. Promise 1
// 5. Promise 2
// 6. setTimeout

1. Synchronous code runs first
2. process.nextTick() callbacks run (all of them, until queue is empty)
3. Promise callbacks run (all of them, until queue is empty)
4. Event loop phases begin (timers phase executes setTimeout)

process.nextTick() vs setImmediate()

javascript
// nextTick-vs-setImmediate.js
setImmediate(() => {
    console.log('setImmediate');
});

process.nextTick(() => {
    console.log('nextTick');
});

// Output:
// nextTick
// setImmediate

javascript
// WARNING: This starves I/O!
function badRecursion() {
    process.nextTick(badRecursion);
}
badRecursion();

// I/O callbacks will never execute because nextTick queue never empties

javascript
// Good: This allows I/O between iterations
function goodRecursion() {
    setImmediate(goodRecursion);
}
goodRecursion();

The Thread Pool (libuv)

Operations That Use the Thread Pool

javascript
// Thread pool operations:

// 1. File System operations
const fs = require('fs');
fs.readFile('./file.txt', callback); // Uses thread pool

// 2. DNS lookups (dns.lookup, NOT dns.resolve)
const dns = require('dns');
dns.lookup('google.com', callback); // Uses thread pool

// 3. Crypto operations
const crypto = require('crypto');
crypto.pbkdf2('password', 'salt', 100000, 64, 'sha512', callback); // Uses thread pool

// 4. Zlib compression
const zlib = require('zlib');
zlib.gzip(buffer, callback); // Uses thread pool

Demonstrating the Thread Pool

javascript
// thread-pool-demo.js
const crypto = require('crypto');

const start = Date.now();

// Run 4 hash operations (default thread pool size)
for (let i = 0; i < 4; i++) {
    crypto.pbkdf2('password', 'salt', 100000, 64, 'sha512', () => {
        console.log(`Hash ${i + 1}: ${Date.now() - start}ms`);
    });
}

// Output (on 4+ core machine):
// Hash 1: ~50ms
// Hash 2: ~50ms
// Hash 3: ~50ms
// Hash 4: ~50ms
// All complete around the same time (parallel execution)

// Now run 8 operations
for (let i = 0; i < 8; i++) {
    crypto.pbkdf2('password', 'salt', 100000, 64, 'sha512', () => {
        console.log(`Hash ${i + 1}: ${Date.now() - start}ms`);
    });
}

// Output:
// Hash 1-4: ~50ms (first batch, parallel)
// Hash 5-8: ~100ms (second batch, waited for threads)

Configuring the Thread Pool

BashJAVASCRIPT
# Increase thread pool size (useful for I/O-heavy apps)
UV_THREADPOOL_SIZE=8 node app.js

# Or in code (must be set before any async operation)
process.env.UV_THREADPOOL_SIZE = 8;

Operations That Don't Use the Thread Pool

javascript
// network-io.js
const https = require('https');

const start = Date.now();

// These don't use the thread pool
// They use OS-level async I/O (epoll on Linux, kqueue on macOS, IOCP on Windows)
for (let i = 0; i < 100; i++) {
    https.get('https://api.github.com', (res) => {
        console.log(`Request ${i + 1}: ${Date.now() - start}ms`);
    });
}

// All 100 requests run concurrently without thread pool limitation

Understanding Async Stack Traces

javascript
// async-stack.js
function a() {
    setTimeout(() => {
        b();
    }, 0);
}

function b() {
    setImmediate(() => {
        c();
    });
}

function c() {
    throw new Error('Something went wrong');
}

a();

// Stack trace only shows c()
// It doesn't show a() or b() because they're in different event loop iterations

bash
node --async-stack-traces async-stack.js

Practical Example: Understanding Execution Order

javascript
// execution-order.js
const fs = require('fs');

console.log('1');

setTimeout(() => console.log('2'), 0);

setImmediate(() => console.log('3'));

Promise.resolve().then(() => console.log('4'));

process.nextTick(() => console.log('5'));

fs.readFile('./package.json', () => {
    console.log('6');

    setTimeout(() => console.log('7'), 0);

    setImmediate(() => console.log('8'));

    Promise.resolve().then(() => console.log('9'));

    process.nextTick(() => console.log('10'));
});

console.log('11');

// Output:
// 1
// 11
// 5 (nextTick, highest priority microtask)
// 4 (promise, microtask)
// 2 (setTimeout, timers phase) - order with 3 can vary at startup
// 3 (setImmediate, check phase) - order with 2 can vary at startup
// 6 (file read complete, poll phase)
// 10 (nextTick within I/O callback)
// 9 (promise within I/O callback)
// 8 (setImmediate, check phase - after I/O)
// 7 (setTimeout, next iteration's timers phase)

Blocking the Event Loop

javascript
// blocking-example.js
const http = require('http');

const server = http.createServer((req, res) => {
    if (req.url === '/slow') {
        // BAD: Blocks the event loop
        const start = Date.now();
        while (Date.now() - start < 5000) {
            // Synchronous blocking for 5 seconds
        }
        res.end('Slow response');
    } else {
        res.end('Fast response');
    }
});

server.listen(3000);

// While /slow is processing, ALL other requests wait
// Even /fast will wait until /slow completes

javascript
// non-blocking-solution.js
const { Worker } = require('worker_threads');
const http = require('http');

const server = http.createServer((req, res) => {
    if (req.url === '/slow') {
        const worker = new Worker('./heavy-computation.js');

        worker.on('message', (result) => {
            res.end(`Result: ${result}`);
        });

        worker.on('error', (err) => {
            res.statusCode = 500;
            res.end('Error');
        });
    } else {
        res.end('Fast response');
    }
});

server.listen(3000);
// Now /fast responses are not blocked by /slow

Summary

1. The event loop has 6 phases: timers, pending callbacks, idle/prepare, poll, check, close callbacks
2. Microtasks run between phases: process.nextTick() before Promises
3. JavaScript is single-threaded, but Node.js uses threads for file system, DNS lookups, crypto, and compression
4. Network I/O is truly async using OS-level mechanisms, not the thread pool
5. process.nextTick() is not "next tick": it runs immediately after current operation
6. setImmediate() vs setTimeout(fn, 0): Different phases, different guarantees
7. Blocking the event loop blocks everything: Use Worker Threads for CPU-intensive work

Questions to Think About

1. Why does setImmediate() always run before setTimeout(fn, 0) inside an I/O callback?

2. When would you use process.nextTick() vs setImmediate()?

3. Why is the default thread pool size 4?

4. How can you tell if your code is blocking the event loop?

5. What happens if the thread pool is exhausted?