Supported File Formats
Last updated
Which files the watcher indexes, which it skips, and how each format is split into searchable chunks.
Routing by Extension
Extensions are compared in lower case (
internal/filesystem/walkFiles.go).| Extension | Parser | Chunk unit | Log kind |
|---|---|---|---|
.pdf | pdftotext -layout -enc UTF-8, form feeds removed | Paragraph | pdf |
.docx | Body text, then headers, footers, footnotes, and endnotes | Paragraph | docx |
.pptx | Slide XML text, slides in numeric order | One chunk per non-empty slide | pptx |
.csv, .tsv | CSV reader (tab separator for .tsv), UTF-8 BOM stripped | 32 data rows | csv |
.xlsx | First worksheet only | 32 data rows | xlsx |
| Any other extension | Read as plain text when it passes the text check | Paragraph | text |
Paragraph Chunks
Text from PDF, DOCX, and plain-text files is split on blank lines; a line that contains only spaces or tabs also counts as blank. Each paragraph is trimmed, empty paragraphs are dropped, and the remaining paragraphs are numbered from 1 with the total count stored alongside. Lines inside a paragraph keep their original line breaks.
- Alpha Go notes
- line two
- Second paragraph about SQLite
This file produces two chunks: chunk 1 holds the first two lines with their line break kept, and chunk 2 holds the last line.
Table Chunks
CSV, TSV, and XLSX files are read with the first row as the header. Data rows are grouped 32 per chunk, and each row is rendered on its own line as
[row] header=value, ... with 1-based row numbers:- [1] name=ann, role=dev
- [2] name=bob, role=ops
| Case | Behavior |
|---|---|
| Header cell is empty | The key becomes col<N> |
| Row has more cells than the header | Extra cells are dropped |
| Only a header row, or the file is empty | No chunks; any earlier rows for the file are dismissed |
| CSV or TSV larger than 1 MiB | Parser error; the file is not stored |
Skipped Files
| Rule | Examples |
|---|---|
| Excluded name | .DS_Store |
| Binary extension | Images (.png, .jpg, .heic, .svg), video (.mp4, .mov), audio (.mp3, .wav), archives (.zip, .tar, .gz), executables and libraries (.exe, .so, .dylib), installers (.dmg, .deb), bytecode (.class, .jar, .pyc, .wasm), databases (.db, .sqlite), design files (.psd, .fig), fonts (.ttf, .woff2) |
| Fails the text check | The first 8 KiB contains a NUL byte or is not valid UTF-8, or the file is empty |
Skipped files are still recorded in the snapshot, so they are not re-examined until they change.