Architecture

Understand how KuraDB separates lifecycle management, read-only queries, indexing writes, durable storage, and rebuildable caches.

System overview

graph TB
    CLI["kura CLI"] --> Registry["Registry<br/>db.json"]
    CLI --> Daemon["Background daemon"]
    CLI --> Stdio["kura mcp<br/>stdio MCP server"]
    Daemon --> API["Gin read-only API"]
    Daemon --> MCP["MCP over HTTP<br/>/mcp"]
    Daemon --> Watcher["Filesystem watcher"]
    Watcher --> Parser["Document parsers"]
    Parser --> PerDB["Per-database SQLite<br/>file_data"]
    PerDB --> Embedder["Embedding scheduler"]
    Embedder --> OpenAI["OpenAI embeddings"]
    Embedder --> Vector["In-memory vector cache"]
    API --> Core["Search core<br/>internal/search"]
    MCP --> Core
    Stdio --> Core
    Core --> Keyword["Keyword search"]
    Core --> Semantic["Semantic search"]
    Keyword --> PerDB
    Semantic --> QueryCache["Memory + global SQLite<br/>query_cache"]
    Semantic --> Vector
    Vector --> PerDB
    Registry --> Daemon

Runtime composition

Layer	Primary code	Responsibility
Entry and lifecycle	`cmd/app`	Dispatch CLI commands, daemonize, initialize subsystems, handle shutdown
HTTP API	`internal/api`	Expose local read-only routes and validate database selection
MCP server	`internal/mcp`	Serve `list_rag` and `search_rag` over streamable HTTP and stdio
Search core	`internal/search`	Run keyword and semantic branches, group hits, own argument validation
Persistence	`internal/database`	Open SQLite stores, maintain registry, and execute all content writes
File ingestion	`internal/filesystem`	Poll inboxes, detect changes, parse supported files, dismiss removals
Embeddings	`internal/openai`	Read credentials, call OpenAI, encode vectors, cache query embeddings
Vector retrieval	`internal/vector`	Maintain rebuildable buckets and perform two-stage cosine search
Tokenization	`internal/segmenter`	Tokenize and deduplicate keyword queries with gse
Process state	`internal/runtime`	Write PID metadata, detect liveness, and stop the daemon

Startup sequence

runServer launches the current executable with --daemon, redirects output to daemon.log, and waits for the endpoint file. runServerDaemon then performs the authoritative initialization sequence:

Create or refresh runtime.uid.
Initialize the KuraDB keychain context.
Load the database registry.
Construct the OpenAI embedder.
Open global.db and preload query embeddings of the expected dimension.
Initialize the tokenizer and process-wide vector cache.
Open every registered per-database SQLite file.
Rebuild each vector bucket from embedded, non-dismissed rows.
Start one watcher and one embedding scheduler per loaded database.
Start the local HTTP server and wait for SIGINT or SIGTERM.

Transports

Three surfaces answer queries, and all three call search.Search in-process. None of them proxies another:

Surface	Entry point	Process
REST	`/api/*` routes on the daemon	Daemon
MCP over HTTP	`/mcp` on the same listener, mounted only when `remote` is enabled	Daemon
MCP over stdio	`kura mcp`	Separate short-lived process spawned by the MCP client

kura mcp opens the registered SQLite databases and loads its own vector cache instead of calling the daemon's HTTP API. It registers no query-cache write hook and starts no watcher or embedding loop, so the daemon remains the only writer. Its vector cache is a snapshot taken at spawn time.

Storage topology

KuraDB uses several local state files under ~/.config/kuradb/:

Store	Authority	Contents
`db.json`	Durable registry	Database names and creation timestamps
`config.json`	Durable configuration	Optional pinned HTTP `port` and `remote` MCP flag
`global.db`	Durable cache store	Query text to embedding blobs
`{db}/data.db`	Source of truth	Parsed chunks, soft-delete state, and embeddings
`{db}/record.json`	Watcher snapshot	File size, mtime, type, and child snapshots
`runtime.uid`	Ephemeral process state	UID, PID, and start time
`endpoint`	Ephemeral discovery	Current local HTTP base URL
`daemon.log`	Operational log	Daemon stdout and stderr

vector.Cache and openai.Cache are accelerators, not authoritative stores. They must remain reconstructible from SQLite.

Concurrency model

Each loaded database owns independent watcher and embedder goroutines. HTTP keyword and semantic branches run concurrently when /api/search has no target. SQLite provides a read pool of eight connections and a separate write connection through go-sqlkit. Vector cache maps use process, database bucket, and query-cache mutexes to isolate concurrent access.

Trust boundaries and invariants

The HTTP surface contains GET query routes only; no mutation endpoint may bypass ingestion.
MCP exposes exactly two read-only tools; no tool writes to SQLite or the caches.
Content writes flow through watcher → parser → databaseHandler.Upsert → SQLite.
Keyword and semantic reads exclude rows where dismiss = TRUE.
All loaded embedding blobs must match the process-wide openai.Dim() expectation.
API responses expose source, chunk, and content, but not internal IDs, scores, hit counts, or totals.
Images and other skipped binary formats do not enter the text embedding pipeline.
Newly registered databases become available only after restart.

Shutdown

A termination signal cancels the shared context. The HTTP server removes endpoint and performs a five-second graceful shutdown. Watchers and embedders leave their select loops, runtime.uid is cleared, and all per-database plus global SQLite connectors are closed.