TechSetupGuides
Intermediatesqlitedatabasenodejsjavascriptsynchronousembeddedpersistence

Better-SQLite3: Fast SQLite3 Bindings for Node.js

Fast and simple synchronous SQLite3 bindings for Node.js. Thread-safe, zero-copy, and fully featured SQLite wrapper with excellent performance.

  1. Step 1

    What is better-sqlite3?

    better-sqlite3 is a Node.js package that provides fast, simple, and synchronous bindings to SQLite3. Unlike other SQLite libraries for Node.js, it uses a synchronous API that makes database operations more straightforward and predictable.

    Key Philosophy: Synchronous, simple, and safe. The library maintains a single database handle per database connection, making it easy to reason about and thread-safe.

    Core Features:

    • Synchronous API: No callbacks or Promises - operations block the event loop (like traditional SQLite)
    • Thread-safe: All database operations are wrapped in worker threads internally
    • Zero-copy: Efficient memory usage with minimal overhead
    • Full SQLite3 feature support: Prepared statements, transactions, triggers, virtual tables
    • Developer-friendly: Exception-based error handling, no hidden async complexity
    • High performance: ~10x faster than sqlite3 npm package

    Use Cases:

    • Local-first applications and desktop apps (Electron, Tauri)
    • Mobile app backends (React Native with SQLite)
    • Data processing pipelines and ETL tools
    • Caching layers and session stores
    • Configuration management and settings
    • Lightweight web applications
    Key concepts:
    ├── Database - Synchronous connection to SQLite file
    ├── Statement - Prepared SQL statements for efficient execution
    ├── Prepared - Pre-compiled queries (.prepare() method)
    ├── Transactions - Atomic blocks (.transaction())
    ├── Hooks - Callbacks for database events (rollback, commit)
    └── Extensions - Load SQLite extensions (JSON1, FTS5, etc.)
    
    Unlike promises/callbacks:
      db.prepare('SELECT * FROM users WHERE id = ?').get(1) // sync!
      → { id: 1, name: 'Alice' } or null
      → No await, no then, no callbacks
  2. Step 2

    Technology Stack

    better-sqlite3 is a native Node.js addon written in C++ that wraps the official SQLite3 C library.

    Core Implementation:

    • Language: C++ (Node-API / N-API bindings)
    • SQLite Version: Uses embedded SQLite3 (defaults to latest stable)
    • Build System: node-pre-gyp for pre-built binary distributions
    • Compiler: Requires C++17 compatible compiler (GCC, Clang, MSVC)
    • Threading: Uses libuv thread pool for async operations under the hood

    Architecture:

    • Single-process, single-threaded database handle (thread-safe by design)
    • Each Database instance holds one SQLite connection
    • Prepared statements cache compiled SQL for performance
    • Memory-mapped I/O for large database files
    • WAL (Write-Ahead Logging) mode by default

    Dependencies:

    • Node.js: 16.13.0+ (LTS versions recommended)
    • napi: Node-API (version-agnostic native API)
    • Optional: libffi for SQLite extensions

    Platform Support:

    • Linux (x64, arm64, armv7l)
    • macOS (x64, arm64)
    • Windows (x64)
    • Pre-built binaries for common platforms
    • Fallback to C++ compilation if no pre-built available
    Architecture Overview:
    
    ┌───────────────────────────────────┐
    │         Node.js Application      │
    │  (JavaScript / TypeScript)       │
    └───────┬───────────────────────────┘
            │ better-sqlite3 API
    ┌───────▼───────────────────────────┐
    │     C++ Native Addon (Node-API)  │
    │  - Synchronous wrapper           │
    │  - Memory management             │
    │  - Error handling                │
    └───────┬───────────────────────────┘
            │ C API
    ┌───────▼───────────────────────────┐
    │        SQLite3 Library           │
    │  - Embedded SQL engine           │
    │  - B-tree storage                │
    │  - ACID transactions             │
    └───────┬───────────────────────────┘
            │
          [database.db file]
    
    All in one process, no external server needed!
  3. Step 3

    Installation

    Install better-sqlite3 using npm or yarn. The package includes pre-built binaries for most platforms, but you may need build tools to compile from source on unsupported platforms.

    Standard Installation:

    • npm: npm install better-sqlite3
    • yarn: yarn add better-sqlite3
    • pnpm: pnpm add better-sqlite3

    Build Prerequisites (only needed if pre-built binaries aren't available):

    • Linux: build-essential, Python 3
    • macOS: Xcode Command Line Tools
    • Windows: Windows SDK + Python (or use pre-built)

    Verify Installation: Check that the native module loaded correctly and create a test database.

    # Standard installation
    npm install better-sqlite3
    
    # Build prerequisites (if needed)
    # Linux (Ubuntu/Debian)
    sudo apt-get install -y build-essential python3
    
    # macOS
    xcode-select --install
    
    # Verify installation
    node -e "const Database = require('better-sqlite3'); const db = new Database(':memory:'); console.log('better-sqlite3 loaded successfully'); db.close();"
    
    # Test with a real database file
    node -e "const Database = require('better-sqlite3'); const db = new Database('test.db'); db.exec('CREATE TABLE test (id INTEGER PRIMARY KEY)'); console.log('Database created'); db.close();"
  4. Step 4

    Basic Usage: Creating a Database

    Start by creating a Database instance. You can use an in-memory database (for testing/temporary data) or a file-based database for persistence.

    Connection Modes:

    • File path: Creates/opens a database file on disk
    • :memory:: Creates a temporary in-memory database (lost when closed)
    • Read-only: Open existing database without writing (URI mode)

    Important Notes:

    • Database objects are synchronous and must be closed when done
    • Each Database instance is a separate connection
    • Closing a database with open transactions rolls them back
    • Use :memory:?cache=shared for shared in-memory databases across threads
    const Database = require('better-sqlite3');
    
    // Create a file-based database (persistent)
    const db = new Database('mydatabase.db');
    console.log('Database created/opened: mydatabase.db');
    
    // Create an in-memory database (temporary, lost on close)
    const memDb = new Database(':memory:');
    
    // Close the database when done
    db.close();
    
    // In TypeScript:
    // import Database from 'better-sqlite3';
    // const db = new Database('mydatabase.db');
  5. Step 5

    Executing SQL Queries

    better-sqlite3 provides several methods to execute SQL queries. Use exec() for statements that don't return rows, prepare() for parameterized queries that return data, and transaction() for atomic operations.

    Query Methods:

    • exec(): Execute SQL without returning rows (CREATE, INSERT, UPDATE, DELETE)
    • prepare(): Compile a parameterized statement for repeated use
    • statement.run(): Execute statement, returns row count/changes
    • statement.get(): Get single row or null
    • statement.all(): Get all rows as array
    • statement.each(): Iterate rows with callback
    • statement.iterator(): Get ES6 iterator for for...of loops

    Parameter Binding: All prepared statements support parameterized queries using ? placeholders (positional) or :name (named).

    Error Handling: Exceptions are thrown on SQL errors. Use try/catch blocks.

    const Database = require('better-sqlite3');
    const db = new Database(':memory:');
    
    // Create a table
    db.exec(`
      CREATE TABLE users (
        id INTEGER PRIMARY KEY,
        name TEXT NOT NULL,
        email TEXT UNIQUE,
        created_at DATETIME DEFAULT CURRENT_TIMESTAMP
      )
    `);
    
    // INSERT using parameterized statement
    const insertUser = db.prepare('INSERT INTO users (name, email) VALUES (?, ?)');
    insertUser.run('Alice', 'alice@example.com');
    insertUser.run('Bob', 'bob@example.com');
    
    console.log('Inserted users:', insertUser.changes, 'rows');
    
    // SELECT single row
    const getUser = db.prepare('SELECT * FROM users WHERE id = ?');
    const user = getUser.get(1);
    console.log('User 1:', user);
    
    // SELECT all rows
    const getAllUsers = db.prepare('SELECT * FROM users');
    const users = getAllUsers.all();
    console.log('All users:', users);
    
    db.close();
    
    // Named parameters
    const findByEmail = db.prepare('SELECT * FROM users WHERE email = :email');
    const found = findByEmail.get({ email: 'alice@example.com' });
    console.log('Found:', found);
  6. Step 6

    Transactions

    Transactions are critical for maintaining data integrity. better-sqlite3 wraps multiple statements in a single atomic transaction using the .transaction() method. All statements succeed or all fail - no partial updates.

    Transaction Behavior:

    • Atomic: All statements commit together or rollback together
    • Isolated: Changes invisible to other connections until committed
    • Automatic rollback: Exception during transaction triggers rollback
    • SAVEPOINT-based: Nested transactions use SAVEPOINT internally

    Transaction Types:

    • DEFERRABLE: Lock deferred until first query
    • IMMEDIATE: Lock immediately, exclusive write
    • EXCLUSIVE: Lock for the entire duration

    Best Practices:

    • Keep transactions short to reduce lock time
    • Avoid network I/O or long operations inside transactions
    • Use IMMEDIATE for write-heavy transactions
    const Database = require('better-sqlite3');
    const db = new Database(':memory:');
    
    db.exec('CREATE TABLE accounts (id INTEGER PRIMARY KEY, balance INTEGER)');
    
    // Insert initial accounts
    const insert = db.prepare('INSERT INTO accounts (id, balance) VALUES (?, ?)');
    insert.run(1, 1000);
    insert.run(2, 500);
    
    // Define a transaction function
    const transfer = db.transaction(function(fromId, toId, amount) {
      this.run('UPDATE accounts SET balance = balance - ? WHERE id = ?', amount, fromId);
      this.run('UPDATE accounts SET balance = balance + ? WHERE id = ?', amount, toId);
    });
    
    // Execute transaction
    transfer(1, 2, 100);
    
    // Check balances
    const getBalance = db.prepare('SELECT * FROM accounts WHERE id = ?');
    console.log('Account 1:', getBalance.get(1)); // { id: 1, balance: 900 }
    console.log('Account 2:', getBalance.get(2)); // { id: 2, balance: 600 }
    
    // Transaction with rollback on error
    const riskyTransfer = db.transaction(function(amount) {
      this.run('UPDATE accounts SET balance = balance - ? WHERE id = 1', amount);
      this.run('UPDATE accounts SET nonexistent = 1 WHERE id = 1'); // Throws
    });
    
    try {
      riskyTransfer(500);
    } catch (err) {
      console.log('Transaction rolled back:', err.message);
    }
    
    db.close();
  7. Step 7

    Advanced Features

    better-sqlite3 provides advanced features for optimization and extended functionality.

    Feature Highlights:

    1. Row Hooks: Callback functions on INSERT/UPDATE/DELETE
    2. Extensions: Load SQLite extensions (JSON1, FTS5, math functions)
    3. PRAGMA Control: Fine-tune SQLite behavior
    4. Custom Functions: Add JavaScript functions to SQL
    5. Custom Aggregates: Create custom aggregate functions
    6. Paging & Limits: Use LIMIT/OFFSET for pagination
    7. JSON Functions: Built-in JSON support via JSON1 extension
    8. Full-Text Search: FTS5 for advanced text search

    PRAGMA Settings:

    • journal_mode = WAL: Better concurrency
    • synchronous = NORMAL: Balance safety/performance
    • cache_size: Memory for page cache
    • temp_store = MEMORY: Use memory for temp files
    const Database = require('better-sqlite3');
    const db = new Database(':memory:');
    
    // PRAGMA settings for performance
    db.pragma('journal_mode = WAL');
    db.pragma('synchronous = NORMAL');
    db.pragma('cache_size = -64000');
    db.pragma('temp_store = MEMORY');
    
    // Row hooks
    db.hook('commit', () => console.log('Transaction committed'));
    db.hook('rollback', () => console.log('Transaction rolled back'));
    
    // Custom JavaScript function in SQL
    db.function('uppercase', (str) => str.toUpperCase());
    const result = db.prepare('SELECT uppercase(?)').get('hello');
    console.log(result);
    
    // Custom aggregate function
    db.aggregate('sumsq', class {
      constructor() { this.sum = 0; }
      step(value) { this.sum += value * value; }
      finalize() { return this.sum; }
    });
    
    db.exec('CREATE TABLE numbers (value INTEGER)');
    const insert = db.prepare('INSERT INTO numbers VALUES (?)');
    insert.run(1); insert.run(2); insert.run(3);
    const sumSq = db.prepare('SELECT sumsq(value) FROM numbers').get();
    console.log('Sum of squares:', sumSq);
    
    db.close();
  8. Step 8

    Common Use Cases

    better-sqlite3 excels in several scenarios where a lightweight, embedded database is needed.

    Desktop Applications (Electron) Perfect for local-first apps: settings, offline data storage, caches.

    Session Storage & Caching Fast read/write for temporary data, session tokens, API caches.

    Data Processing & ETL Batch processing, data transformation, intermediate storage.

    Configuration Management Storing app configs, user preferences, feature flags.

    Lightweight APIs (Express + SQLite) Simple REST APIs without a full database server.

    Development & Testing In-memory databases for fast, isolated test cases.

    // Express + better-sqlite3 REST API
    const express = require('express');
    const Database = require('better-sqlite3');
    const app = express();
    app.use(express.json());
    
    const db = new Database('api.db');
    db.exec(`
      CREATE TABLE IF NOT EXISTS posts (
        id INTEGER PRIMARY KEY AUTOINCREMENT,
        title TEXT NOT NULL,
        content TEXT,
        created_at DATETIME DEFAULT CURRENT_TIMESTAMP
      )
    `);
    
    // GET /posts
    app.get('/posts', (req, res) => {
      const posts = db.prepare('SELECT * FROM posts ORDER BY created_at DESC').all();
      res.json(posts);
    });
    
    // POST /posts
    app.post('/posts', (req, res) => {
      const { title, content } = req.body;
      const insert = db.prepare('INSERT INTO posts (title, content) VALUES (?, ?)');
      const result = insert.run(title, content);
      res.json({ id: result.lastInsertRowid, title, content });
    });
    
    // GET /posts/:id
    app.get('/posts/:id', (req, res) => {
      const post = db.prepare('SELECT * FROM posts WHERE id = ?').get(req.params.id);
      if (!post) return res.status(404).json({ error: 'Not found' });
      res.json(post);
    });
    
    app.listen(3000, () => console.log('API server on http://localhost:3000'));
    
    process.on('SIGINT', () => {
      db.close();
      process.exit(0);
    });
  9. Step 9

    Best Practices & Gotchas

    Using better-sqlite3 effectively requires understanding its synchronous nature and limitations.

    Key Considerations:

    1. Blocking the Event Loop: Synchronous operations block Node.js's event loop. For production servers, consider async alternatives or use worker threads.

    2. Memory Usage: Large queries or datasets consume memory - use streaming with iterator() for large results.

    3. File Locking: Only one process can write to a database file. Use read-only mode for concurrent readers.

    4. Prepared Statements: Always use prepared statements to prevent SQL injection and improve performance.

    5. Database Persistence: Close databases properly with .close() to avoid data corruption.

    6. Connection Pooling: Not needed - better-sqlite3 is single-connection by design.

    7. Error Handling: Wrap database operations in try/catch to handle SQL errors.

    8. Migration Management: Use tools like db-migrate or write custom migration scripts.

    Common Gotchas:
    
    DON'T:
      - Block event loop in production HTTP servers
      - Forget to close databases (leaks)
      - Use string concatenation in queries (SQL injection!)
      - Open multiple writable connections to same file
      - Run long operations in transactions
    
    DO:
      - Use prepared statements with parameter binding
      - Close databases with db.close() when done
      - Use worker threads for blocking I/O in servers
      - Implement proper error handling with try/catch
      - Enable WAL mode for better concurrency
      - Use IMMEDIATE transactions for writes
      - Backup databases (just copy the .db file)
    
    Comparison:
      sqlite3 (npm):      Async/callback-based, slower
      sql.js:            WASM-based, browser compatible
      better-sqlite3:    Sync, fast, native, recommended
  10. Step 10

    TypeScript Support

    better-sqlite3 includes TypeScript definitions, making it easy to use with TypeScript projects.

    Type Definitions:

    • Full TypeScript support built-in
    • IntelliSense and autocomplete support
    • Type-safe query results
    • Return types inferred from query
    // TypeScript usage
    import Database from 'better-sqlite3';
    
    interface User {
      id: number;
      name: string;
      email: string;
      created_at: string;
    }
    
    const db = new Database('example.db');
    
    const getUserById = db.prepare<User>('SELECT * FROM users WHERE id = ?');
    const user: User | undefined = getUserById.get(1);
    
    const getAllUsers = db.prepare<User[]>('SELECT * FROM users');
    const users: User[] = getAllUsers.all();
    
    process.on('SIGINT', () => {
      db.close();
      process.exit(0);
    });
  11. Step 11

    Resources & Further Reading

    Official Resources:

    • GitHub: https://github.com/WiseLibs/better-sqlite3
    • npm: https://www.npmjs.com/package/better-sqlite3

    SQLite Documentation:

    • SQLite Official: https://www.sqlite.org/docs.html
    • SQLite Language Reference: https://www.sqlite.org/lang.html

    Alternatives:

    • sql.js: WebAssembly-based SQLite (browser compatible)
    • sqlite3 npm: Async callback-based SQLite bindings
    • Kysely: SQL query builder on top of better-sqlite3
    • Prisma: ORM that can use SQLite

    Learning Resources:

    • SQLite tutorial: https://www.sqlite.org/tutorial.html
    • Node.js worker threads: https://nodejs.org/api/worker_threads.html
    Quick Links:
    
    GitHub:       https://github.com/WiseLibs/better-sqlite3
    npm:          https://npm.io/package/better-sqlite3
    SQLite Docs:  https://www.sqlite.org/docs.html
    
    Star Count: 33K+ stars
    Maintained by: Jameson Miller (@WiseLibs)
    Created: 2016

Feature requests

Sign in to suggest features or vote on existing ones.

No feature requests yet.

Discussion

0 people marked this as worked·Sign in to mark your own.

Sign in to join the discussion.

No comments yet.