Search Index Generator

The search index generator (scripts/generate_search_index.ts) produces per-locale JSON index files from the rendered dist/ HTML. Unlike a third-party search service, the index is built from exactly what the static site rendered — no separate crawl, no external dependency, no ongoing cost. The client-side search widget (src/public/js/site-search.js) fetches the index on first use of the Cmd+K dialog and performs fuzzy matching locally.

bun ssg:search

This runs as the last step in bun ssg. It can also be run standalone to rebuild the index without re-rendering the whole site.

CLI Options

bun scripts/generate_search_index.ts [options]
OptionDefaultDescription
--public <dir>./src/publicSource directory with page templates
--dist <dir>./distRendered site directory; indexes are written under it
--help-Print usage and exit

Output

For each configured source and each active locale, the generator writes one JSON file:

dist/search-index.json           ← default locale, whole-site source
dist/en-us/search-index.json     ← English locale
dist/sl-si/search-index.json     ← Slovenian locale

Multiple sources (see Configuration) produce multiple files per locale:

dist/search-index.json           ← default locale, "" source (whole site)
dist/docs/search-index.json      ← default locale, "/docs" source
dist/en-us/docs/search-index.json

Index Format

Each index is a flat JSON object:

{
  "site": "/docs",
  "records": [
    {
      "url": "/docs/install",
      "anchor": "prerequisites",
      "title": "Installation",
      "heading": "Prerequisites",
      "text": "ReeWeb requires Bun 1.2 or later. Install it from..."
    }
  ]
}
FieldDescription
siteURL prefix this index covers (matches the source prefix). Used for result grouping.
urlCanonical page URL, e.g. "/docs/translations".
anchorHeading id for deep-linking to the matching section. Empty for lead content.
titlePage title (every record from the same page shares this).
headingSection heading text. Equals title for the lead section before the first heading.
textPlain-text section body, capped at 1500 characters for compactness.

Each rendered page is split into one record per h1–h3 heading, so a search match deep-links directly to the relevant section rather than just the page top.

Configuration

Search behaviour is configured in config/search.ts:

import type { SearchConfig } from "$config/search";

export const search: SearchConfig = {
  enabled: true,
  sources: [{ prefix: "", brand: "" }],
  strip: [],
};
FieldTypeDescription
enabledbooleanWhen false, bun ssg:search exits without writing anything.
sourcesSearchSource[]One emitted index per entry. The default indexes the whole site into dist/search-index.json.
stripstring[]Exact strings removed from indexed text — repeated page chrome (banners, footers) that would otherwise match every query.

SearchSource

FieldTypeDescription
prefixstringURL prefix for this index. "" indexes the entire site. /docs indexes only pages under /docs.
brandstringGroup heading above this source's results in the search dialog. "" renders no heading.
rootstringSource folder under src/public/. Defaults to prefix without its leading slash.

Multi-Source Example

Split documentation from blog so the client fetches the active section's index first and the rest in the background:

export const search: SearchConfig = {
  enabled: true,
  sources: [
    { prefix: "",     brand: "" },
    { prefix: "/docs", brand: "Documentation" },
    { prefix: "/blog", brand: "Blog" },
  ],
  strip: ["★ New: Reepolee Studio is now available"],
};

The client-side search widget fetches the current section's index on first keystroke, then lazy-loads remaining indexes in the background so the full site is searchable as the visitor continues typing.

Visibility Rules

The generator honours the same visibility policy as the sitemap. Pages are excluded from the index when:

  • The page is a draft (draft: true frontmatter)
  • The page has a future published_at date
  • The page has noindex: true or sitemap: false frontmatter

Pages excluded from search are still rendered and served — they just don't appear in results.

How It Works

  1. Loads translations via load_all_translations() to build the route map for localized URLs.
  2. Collects page files via collect_page_files(), excluding drafts.
  3. Builds the route map for per-locale URL resolution (mirrors the sitemap).
  4. For each (locale × source) pair, filters pages belonging to that source.
  5. Reads each rendered HTML page from dist/ — the index reflects exactly what was built, not the raw templates.
  6. Extracts the article body — prefers <article class="article-body"> (markdown pages), falls back to <main>.
  7. Splits at headings — each h1–h3 with an id becomes a section record with its own anchor and heading.
  8. Strips repeated chrome — configurable via strip in config/search.ts.
  9. Writes the JSON index to dist/<locale-prefix><source-prefix>/search-index.json.

The dev server (scripts/dev/static_files.ts) serves previously-built indexes, so search works locally after a bun ssg run — no separate dev-time index rebuild.

Section Splitting

Content before the first heading becomes the lead section (anchor "", heading = page title). Each subsequent h1–h3 with an id attribute opens a new section. h4–h6 text stays inside the enclosing section.

Text extraction strips all HTML tags, decodes entities, collapses whitespace, removes <script> and <style> blocks, and applies the strip list.

Client Integration

The generated indexes are consumed by the client-side search widget, which:

  1. Hooks the Cmd+K / Ctrl+K keyboard shortcut.
  2. Fetches the index for the current URL prefix on first keystroke.
  3. Lazy-loads remaining indexes in the background.
  4. Performs fuzzy matching locally against the preloaded records.
  5. Renders results grouped by brand, with each result deep-linking to its section anchor.

The widget lives at src/public/js/site-search.js and requires no build step beyond the index generation itself.