Part 3: Modules and Package Management

CommonJS, ES Modules, npm, and the Node.js Module System

Why Modules Matter

Before module systems, JavaScript relied on global variables and script concatenation. This led to naming collisions, unclear dependencies, and maintenance nightmares.

html
<!-- Old way: Everything is global -->
<script src="jquery.js"></script>
<script src="utils.js"></script>
<script src="app.js"></script>
<!-- utils.js might override something from jquery.js -->
<!-- Who knows what depends on what? -->

Node.js introduced CommonJS modules, bringing encapsulation and explicit dependencies to JavaScript.

CommonJS Modules

CommonJS is the original module system in Node.js. Every file is a module with its own scope.

How CommonJS Works

When you require a file, Node.js wraps it in a function:

javascript
// What you write in math.js:
const PI = 3.14159;

function add(a, b) {
    return a + b;
}

module.exports = { PI, add };

// What Node.js actually executes:
(function(exports, require, module, __filename, __dirname) {
    const PI = 3.14159;

    function add(a, b) {
        return a + b;
    }

    module.exports = { PI, add };
});

This wrapper provides:

exports: A reference to module.exports
require: Function to import other modules
module: Object representing the current module
__filename: Absolute path to this file
__dirname: Absolute path to the directory containing this file

Exporting from Modules

There are multiple ways to export:

javascript
// Method 1: module.exports object
// math.js
module.exports = {
    add: (a, b) => a + b,
    subtract: (a, b) => a - b
};

// Method 2: module.exports function
// logger.js
module.exports = function(message) {
    console.log(`[${new Date().toISOString()}] ${message}`);
};

// Method 3: exports shorthand (adds properties to module.exports)
// utils.js
exports.formatDate = (date) => date.toISOString();
exports.capitalize = (str) => str.charAt(0).toUpperCase() + str.slice(1);

// WARNING: This doesn't work!
exports = { foo: 'bar' }; // This breaks the reference to module.exports

The exports gotcha explained:

javascript
// exports is initially a reference to module.exports
console.log(exports === module.exports); // true

// Adding properties works
exports.foo = 'bar';
console.log(module.exports.foo); // 'bar'

// Reassigning exports breaks the reference
exports = { baz: 'qux' };
console.log(module.exports.baz); // undefined

// Always use module.exports for complete replacement
module.exports = { baz: 'qux' }; // This works

Importing Modules

javascript
// Import entire module
const math = require('./math');
console.log(math.add(2, 3));

// Destructure what you need
const { add, subtract } = require('./math');
console.log(add(2, 3));

// Import a function
const log = require('./logger');
log('Hello');

// Import core modules (no path needed)
const fs = require('fs');
const path = require('path');

// Import from node_modules
const express = require('express');

Module Resolution Algorithm

When you call require('something'), Node.js follows this algorithm:

javascript
require('./local')      // 1. Relative path: look for ./local.js, ./local.json, ./local/index.js
require('/absolute')    // 2. Absolute path: look at /absolute.js, etc.
require('module')       // 3. Core module? Return it
                        // 4. Not core? Look in node_modules

For node_modules, Node.js walks up the directory tree:

/home/user/project/src/utils/helper.js calls require('lodash')

Search order:
1. /home/user/project/src/utils/node_modules/lodash
2. /home/user/project/src/node_modules/lodash
3. /home/user/project/node_modules/lodash
4. /home/user/node_modules/lodash
5. /home/node_modules/lodash
6. /node_modules/lodash

File Extension Resolution

When no extension is provided, Node.js tries in order:

javascript
require('./config');

// Looks for:
// 1. ./config.js
// 2. ./config.json
// 3. ./config.node (native addon)
// 4. ./config/package.json (main field)
// 5. ./config/index.js
// 6. ./config/index.json
// 7. ./config/index.node

Module Caching

Modules are cached after the first load:

javascript
// counter.js
let count = 0;
module.exports = {
    increment: () => ++count,
    getCount: () => count
};

// app.js
const counter1 = require('./counter');
const counter2 = require('./counter');

counter1.increment();
console.log(counter2.getCount()); // 1 - same instance!

console.log(counter1 === counter2); // true - identical object

This is important for singleton patterns and has implications for testing:

javascript
// To clear cache for testing:
delete require.cache[require.resolve('./counter')];

// Now a fresh import
const freshCounter = require('./counter');
console.log(freshCounter.getCount()); // 0

ES Modules (ESM)

ES Modules are the official JavaScript standard, supported in Node.js 12+.

Enabling ES Modules

There are two ways:

Use .mjs extension:

javascript
// math.mjs
export const PI = 3.14159;
export function add(a, b) {
    return a + b;
}

Set "type": "module" in package.json:

json
{
    "name": "my-app",
    "type": "module"
}

Then all .js files are treated as ES modules.

ES Module Syntax

javascript
// Named exports
// math.js
export const PI = 3.14159;
export function add(a, b) {
    return a + b;
}

// Default export
// logger.js
export default function log(message) {
    console.log(message);
}

// Both in one file
// utils.js
export const VERSION = '1.0.0';
export function helper() { }
export default class Utils { }

Importing ES Modules

javascript
// Import named exports
import { PI, add } from './math.js';

// Import with alias
import { add as addNumbers } from './math.js';

// Import all as namespace
import * as math from './math.js';
console.log(math.PI, math.add(1, 2));

// Import default export
import log from './logger.js';

// Import both default and named
import Utils, { VERSION, helper } from './utils.js';

// Import for side effects only
import './polyfills.js';

Key Differences: CommonJS vs ES Modules

Aspect	CommonJS	ES Modules
Syntax	`require()`, `module.exports`	`import`, `export`
Loading	Synchronous	Asynchronous
Parsing	Runtime	Static (at parse time)
Top-level await	No	Yes
`__dirname`, `__filename`	Available	Not available
Tree shaking	Limited	Full support
File extension	Optional	Required (usually)

Getting dirname and filename in ES Modules

javascript
// ES modules don't have __dirname and __filename
// Use import.meta.url instead

import { fileURLToPath } from 'url';
import { dirname } from 'path';

const __filename = fileURLToPath(import.meta.url);
const __dirname = dirname(__filename);

console.log(__filename); // /home/user/project/app.js
console.log(__dirname);  // /home/user/project

Top-Level Await

ES modules support await at the top level:

javascript
// config.js (ES module)
const response = await fetch('https://api.example.com/config');
export const config = await response.json();

// app.js
import { config } from './config.js';
// config is already resolved

Dynamic Imports

Both CommonJS and ES modules support dynamic imports:

javascript
// Works in both
const module = await import('./dynamic-module.js');

// Useful for conditional loading
let adapter;
if (process.env.DB === 'postgres') {
    adapter = await import('./postgres-adapter.js');
} else {
    adapter = await import('./mysql-adapter.js');
}

Interoperability

javascript
// ES module importing CommonJS
import pkg from 'commonjs-package'; // default import gets module.exports
import { named } from 'commonjs-package'; // might work, depending on export style

// CommonJS importing ES module (async only)
const esModule = await import('./es-module.mjs');

npm: Node Package Manager

npm is the default package manager for Node.js, hosting over 2 million packages.

package.json Fundamentals

json
{
    "name": "my-application",
    "version": "1.0.0",
    "description": "A sample application",
    "main": "index.js",
    "type": "module",
    "scripts": {
        "start": "node index.js",
        "dev": "node --watch index.js",
        "test": "jest",
        "lint": "eslint ."
    },
    "dependencies": {
        "express": "^4.18.2",
        "lodash": "~4.17.21"
    },
    "devDependencies": {
        "jest": "^29.0.0",
        "eslint": "^8.0.0"
    },
    "engines": {
        "node": ">=18.0.0"
    },
    "repository": {
        "type": "git",
        "url": "https://github.com/user/repo.git"
    },
    "keywords": ["node", "example"],
    "author": "Your Name <email@example.com>",
    "license": "MIT"
}

Semantic Versioning

npm uses semver: MAJOR.MINOR.PATCH

1.2.3
│ │ │
│ │ └── PATCH: Bug fixes, no API changes
│ └──── MINOR: New features, backward compatible
└────── MAJOR: Breaking changes

Version ranges in package.json:

json
{
    "dependencies": {
        "exact": "1.2.3",           // Exactly 1.2.3
        "caret": "^1.2.3",          // >=1.2.3 <2.0.0 (same major)
        "tilde": "~1.2.3",          // >=1.2.3 <1.3.0 (same minor)
        "range": ">=1.2.3 <2.0.0",  // Explicit range
        "or": "1.2.3 || 2.0.0",     // Either version
        "any": "*",                  // Any version (dangerous!)
        "latest": "latest"          // Always latest (dangerous!)
    }
}

Essential npm Commands

bash
# Initialize a new project
npm init
npm init -y  # Accept all defaults

# Install dependencies
npm install                    # Install all from package.json
npm install express            # Install and save to dependencies
npm install -D jest            # Install to devDependencies
npm install -g typescript      # Install globally

# Shorthand
npm i express
npm i -D jest

# Update packages
npm update                     # Update all packages
npm update lodash              # Update specific package

# Remove packages
npm uninstall express
npm rm express                 # Shorthand

# View information
npm list                       # List installed packages
npm list --depth=0             # Top-level only
npm outdated                   # Check for updates
npm view express               # Package information
npm view express versions      # All versions

# Run scripts
npm start                      # Run "start" script
npm test                       # Run "test" script
npm run dev                    # Run custom script

# Security
npm audit                      # Check for vulnerabilities
npm audit fix                  # Auto-fix vulnerabilities

# Cache
npm cache clean --force        # Clear cache

package-lock.json

The lockfile ensures reproducible builds by locking exact versions:

json
{
    "name": "my-app",
    "lockfileVersion": 3,
    "packages": {
        "node_modules/express": {
            "version": "4.18.2",
            "resolved": "https://registry.npmjs.org/express/-/express-4.18.2.tgz",
            "integrity": "sha512-...",
            "dependencies": {
                "accepts": "~1.3.8"
            }
        }
    }
}

Always commit package-lock.json to ensure everyone uses identical dependencies.

npm Scripts

Scripts in package.json can run any command:

json
{
    "scripts": {
        "start": "node server.js",
        "dev": "nodemon server.js",
        "build": "tsc",
        "test": "jest",
        "test:watch": "jest --watch",
        "test:coverage": "jest --coverage",
        "lint": "eslint .",
        "lint:fix": "eslint . --fix",
        "pretest": "npm run lint",
        "posttest": "echo 'Tests completed'",
        "prepare": "husky install"
    }
}

Pre and post hooks run automatically:

pretest runs before test
posttest runs after test

npx: Package Runner

npx runs packages without installing globally:

bash
# Instead of:
npm install -g create-react-app
create-react-app my-app

# Use:
npx create-react-app my-app

# Run a specific version
npx typescript@4.9.5 --version

# Run from GitHub
npx github:user/repo

Creating Your Own Packages

Basic Package Structure

my-package/
├── package.json
├── index.js           # Main entry point
├── lib/               # Library code
│   ├── utils.js
│   └── helpers.js
├── README.md
├── LICENSE
└── .gitignore

package.json for Publishing

json
{
    "name": "@username/my-package",
    "version": "1.0.0",
    "description": "A useful package",
    "main": "index.js",
    "module": "esm/index.js",
    "types": "types/index.d.ts",
    "exports": {
        ".": {
            "require": "./index.js",
            "import": "./esm/index.js",
            "types": "./types/index.d.ts"
        },
        "./utils": {
            "require": "./lib/utils.js",
            "import": "./esm/utils.js"
        }
    },
    "files": [
        "index.js",
        "lib/",
        "esm/",
        "types/"
    ],
    "keywords": ["utility", "helper"],
    "author": "Your Name",
    "license": "MIT",
    "repository": {
        "type": "git",
        "url": "https://github.com/username/my-package.git"
    },
    "bugs": {
        "url": "https://github.com/username/my-package/issues"
    },
    "homepage": "https://github.com/username/my-package#readme"
}

The exports Field

The exports field provides fine-grained control over entry points:

json
{
    "exports": {
        ".": "./index.js",
        "./utils": "./lib/utils.js",
        "./helpers/*": "./lib/helpers/*.js",
        "./package.json": "./package.json"
    }
}

This enables:

javascript
import pkg from 'my-package';           // loads ./index.js
import utils from 'my-package/utils';   // loads ./lib/utils.js
import a from 'my-package/helpers/a';   // loads ./lib/helpers/a.js

And prevents:

javascript
import internal from 'my-package/internal'; // Error! Not in exports

Publishing to npm

bash
# Login to npm
npm login

# Publish (public package)
npm publish

# Publish scoped package publicly
npm publish --access public

# Publish specific version
npm version patch  # 1.0.0 -> 1.0.1
npm version minor  # 1.0.1 -> 1.1.0
npm version major  # 1.1.0 -> 2.0.0
npm publish

Workspaces (Monorepos)

npm 7+ supports workspaces for managing multiple packages:

my-monorepo/
├── package.json
├── packages/
│   ├── core/
│   │   ├── package.json
│   │   └── index.js
│   ├── utils/
│   │   ├── package.json
│   │   └── index.js
│   └── app/
│       ├── package.json
│       └── index.js

Root package.json:

json
{
    "name": "my-monorepo",
    "workspaces": [
        "packages/*"
    ]
}

Working with workspaces:

bash
# Install dependencies for all workspaces
npm install

# Run script in specific workspace
npm run build -w packages/core

# Run script in all workspaces
npm run test --workspaces

# Add dependency to specific workspace
npm install lodash -w packages/utils

Summary

Key takeaways:

CommonJS is synchronous, uses require/module.exports, and caches modules
ES Modules are the standard, static, support top-level await, but require extensions
Module resolution walks up directory tree for node_modules
package.json defines metadata, dependencies, and scripts
Semantic versioning (semver) manages compatibility expectations
package-lock.json ensures reproducible builds
exports field controls package entry points

Questions to Think About

Why does CommonJS use synchronous loading?

When Node.js was created, it was for servers where all code is local. Synchronous loading simplified the mental model and was fast enough for local files. ES Modules needed async for browser compatibility where network loading is slow.

When should you use dependencies vs devDependencies?

Use dependencies for packages needed at runtime (express, lodash). Use devDependencies for development tools (jest, eslint). Production deployments can skip devDependencies with npm install --production.

Why use caret (^) vs tilde (~) vs exact versions?

Caret allows minor updates (new features, hopefully safe). Tilde allows only patches (bug fixes, safest). Exact versions ensure identical builds but require manual updates. Use caret for most packages, exact for critical dependencies.

How does module caching affect your application?

Modules are singletons after first load. This is useful for shared state (database connections) but can cause issues in tests. Clear cache when you need fresh instances.

Should you use CommonJS or ES Modules?

For new projects, prefer ES Modules. They're the standard, support tree shaking, and have better static analysis. Use CommonJS when you need synchronous require or maximum compatibility.

Next: Part 4 - Core Modules Essentials, where we explore fs, path, os, util, events, buffer, and other built-in modules.