pardnchiu/KuraDB
Last updated
NoteThis README was generated by SKILL, get the ZH version from here.
DROP FILES IN, LET YOUR AGENT SEARCH THEM OUT
A Go read-only file database with automatic Markdown conversion, English keyword indexing, and agent retrieval tools
WarningRedesign in progress: Download a release build instead of compiling from source for now. See the Roadmap for progress.
Table of Contents
Positioning
NoteWhy no vector search
- Every document has to be split into chunks and embedded, and semantic matching still falls short on precision.
- Vectors have to stay resident in memory, which does not scale in the long run.
- "Kura" (蔵) means storehouse; vector search belongs to ToriiDB. KuraDB returns to its name and focuses on storage and retrieval built for agents.
NoteWhy no tokenizer
- The language of incoming data is unknown in advance, and no tokenizer segments every language well, so gse tokenization has been removed.
- An LLM generates English keywords for each section while the original text stays intact; queries then use English keywords, so data in any language is found the same way.
- Drop-In Conversion — Files dropped into
~/Kura_{name}are converted to Markdown and stored in SQLite with their original-language content intact. - Cross-Language English Keywords — An LLM generates English keywords for each section, indexed by SQLite FTS5, so retrieval depends on neither per-language tokenization nor vectors.
- Agent Retrieval Tools —
list,search, andreadlet an agent browse the data, locate sections by keyword, and expand context by line range. - One-Way Read-Only Boundary — The API serves queries only; the sole write path is watcher → conversion → SQLite.
Roadmap
| Phase | Scope | Status |
|---|---|---|
| Phase 1: Remove vector dependencies | Remove OpenAI embedding, the vector cache, semantic search, and the query cache; remove gse tokenization and match queries split on whitespace against the original text for now | Done |
| Phase 2: New storage | Store each file as one Markdown document, split it into sections by heading, page, slide, or table rows with line numbers, attach English keywords, index them in FTS5, and add HTML conversion | Planned |
| Phase 3: Retrieval tools | Replace the MCP and HTTP interfaces with list, search, and read, rank search with bm25 and return line ranges | Planned |
Supported formats: plain text, Markdown, PDF, DOCX, PPTX, CSV/TSV, XLSX, and HTML (added in Phase 2).
License
This project is licensed under the MIT LICENSE.
Author
Just open an issue to share an idea.
©️ 2026 邱敬幃 Pardn Chiu