Jun 9, 2026, 3:25 PM

brain/GEMINI.md

@@ -6,13 +6,30 @@ This document defines the structure, conventions, and workflows for maintaining
 This wiki is an LLM-maintained knowledge base focused on **AI, Databases, Programming, and Knowledge Management**, integrated with **Feynman's 12 Favorite Problems** framework. 
 The LLM writes and maintains all files under `wiki/`. The human curates raw sources and directs queries.
 
-**The Core Idea**: Unlike traditional RAG systems that re-derive answers from scratch on every query, this wiki is a **persistent, compounding artifact**. The LLM incrementally builds a structured, interlinked collection of markdown files. When new sources are added, the LLM integrates the knowledge, updates cross-references, and flags contradictions. You (the human) are in charge of exploration; the LLM does the bookkeeping. *Obsidian is the IDE; the LLM is the programmer; the wiki is the codebase.*
+### The Core Idea
+Most people's experience with LLMs and documents looks like RAG: you upload a collection of files, the LLM retrieves relevant chunks at query time, and generates an answer. This works, but the LLM is rediscovering knowledge from scratch on every question. There's no accumulation. Ask a subtle question that requires synthesizing five documents, and the LLM has to find and piece together the relevant fragments every time. Nothing is built up.
+
+The idea here is different. Instead of just retrieving from raw documents at query time, the LLM **incrementally builds and maintains a persistent wiki** — a structured, interlinked collection of markdown files that sits between you and the raw sources. When you add a new source, the LLM doesn't just index it for later retrieval. It reads it, extracts the key information, and integrates it into the existing wiki — updating entity pages, revising topic summaries, noting where new data contradicts old claims, strengthening or challenging the evolving synthesis. The knowledge is compiled once and then *kept current*, not re-derived on every query.
+
+This is the key difference: **the wiki is a persistent, compounding artifact.** The cross-references are already there. The contradictions have already been flagged. The synthesis already reflects everything you've read. The wiki keeps getting richer with every source you add and every question you ask.
+
+You never (or rarely) write the wiki yourself — the LLM writes and maintains all of it. You're in charge of sourcing, exploration, and asking the right questions. The LLM does all the grunt work — the summarizing, cross-referencing, filing, and bookkeeping that makes a knowledge base actually useful over time. In practice, I have the LLM agent open on one side and Obsidian open on the other. The LLM makes edits based on our conversation, and I browse the results in real time. Obsidian is the IDE; the LLM is the programmer; the wiki is the codebase.
+
+### Example Contexts
+This pattern applies to a lot of different contexts:
+- **Personal**: tracking your own goals, health, psychology, self-improvement — filing journal entries, articles, podcast notes, and building up a structured picture of yourself over time.
+- **Research**: going deep on a topic over weeks or months — reading papers, articles, reports, and incrementally building a comprehensive wiki with an evolving thesis.
+- **Reading a book**: filing each chapter as you go, building out pages for characters, themes, plot threads, and how they connect.
+- **Business/team**: an internal wiki maintained by LLMs, fed by Slack threads, meeting transcripts, project documents, customer calls. The wiki stays current because the LLM does the maintenance that no one on the team wants to do.
+- **Competitive analysis, due diligence, trip planning, course notes, hobby deep-dives** — anything where you're accumulating knowledge over time and want it organized rather than scattered.
 
 ## Architecture
-There are three distinct layers to this setup:
-1. **Raw sources (`raw/`)** — Your curated collection of source documents. These are immutable.
-2. **The wiki (`wiki/`)** — A directory of LLM-generated markdown files (summaries, concepts, etc.). The LLM owns this layer entirely.
-3. **The schema (this `GEMINI.md` file)** — The configuration file that tells the LLM how the wiki is structured and what workflows to follow.
+
+There are three layers:
+
+1. **Raw sources (`raw/`)** — your curated collection of source documents. Articles, papers, images, data files. These are immutable — the LLM reads from them but never modifies them. This is your source of truth.
+2. **The wiki (`wiki/`)** — a directory of LLM-generated markdown files. Summaries, entity pages, concept pages, comparisons, an overview, a synthesis. The LLM owns this layer entirely. It creates pages, updates them when new sources arrive, maintains cross-references, and keeps everything consistent. You read it; the LLM writes it.
+3. **The schema (this `GEMINI.md` file)** — a document that tells the LLM how the wiki is structured, what the conventions are, and what workflows to follow when ingesting sources, answering questions, or maintaining the wiki. This is the key configuration file — it's what makes the LLM a disciplined wiki maintainer rather than a generic chatbot.
 
 ## Directory Layout
 - `raw/` — Immutable source documents (transcripts, articles, notes). Never modify these.
@@ -71,39 +88,50 @@ We maintain a list of ~12 "Favorite Problems" in `wiki/feynman_problems.md`. Eve
 ## Operations / Workflows
 
 ### 1. Ingest
-*You drop a new source into the raw collection and tell the LLM to process it.*
-1. Read the raw source completely. Discuss key takeaways if prompted.
-2. Create `wiki/summaries/<source-slug>.md` with a full summary.
+You drop a new source into the raw collection and tell the LLM to process it. An example flow: 
+1. The LLM reads the raw source completely.
+2. The LLM discusses key takeaways with you, and writes a summary page in the wiki (`wiki/summaries/<source-slug>.md`).
 3. Identify all concepts, entities, and strategies mentioned.
 4. **Feynman Check**: Compare against `wiki/feynman_problems.md` and update it if connections are found.
-5. Create/Update concept and entity pages using the required sections.
+5. Create/Update concept and entity pages across the wiki using the required sections. A single source might touch 10-15 wiki pages.
 6. Add cross-links in both directions between all touched pages.
 7. Update `wiki/log.md` and relevant indices in `wiki/indices/`.
 8. Update `wiki/index.md` if new categories are needed.
 9. Flag any contradictions with existing content.
-*(A single source might touch 10-15 wiki pages. The LLM handles the cascading updates.)*
+
+*Personally I prefer to ingest sources one at a time and stay involved — I read the summaries, check the updates, and guide the LLM on what to emphasize. But you could also batch-ingest many sources at once with less supervision.*
 
 ### 2. Query
-*You ask questions against the wiki. The LLM searches for relevant pages, reads them, and synthesizes an answer.*
+You ask questions against the wiki. The LLM searches for relevant pages, reads them, and synthesizes an answer with citations. 
 1. Read `wiki/index.md` and relevant thematic indices to find pages.
-2. Synthesize an answer with explicit citations (wiki links). Answers can take different forms (markdown page, comparison table, slide deck, chart).
-3. **Crucial insight**: Good answers should be filed back into the wiki as new pages (e.g., in `wiki/syntheses/`). A comparison you asked for or a connection you discovered shouldn't disappear into chat history.
+2. Synthesize an answer citing specific pages with wiki links.
+3. Answers can take different forms depending on the question — a markdown page, a comparison table, a slide deck (Marp), a chart, or a synthesis.
+4. **Crucial insight**: good answers can be filed back into the wiki as new pages. A comparison you asked for, an analysis, a connection you discovered — these are valuable and shouldn't disappear into chat history. This way your explorations compound in the knowledge base just like ingested sources do. Create a synthesis page in `wiki/syntheses/`.
 
 ### 3. Lint
-*Periodically, ask the LLM to health-check the wiki.*
-1. Look for orphan pages with no inbound links.
-2. Check for contradictions between pages or stale claims that newer sources have superseded.
-3. Find important concepts mentioned but lacking their own page.
-4. Identify missing cross-references or data gaps that could be filled with a web search.
-5. Suggest new questions to investigate and new sources to look for to keep the wiki healthy as it grows.
+Periodically, ask the LLM to health-check the wiki. Look for: 
+1. Contradictions between pages.
+2. Stale claims that newer sources have superseded.
+3. Orphan pages with no inbound links.
+4. Important concepts mentioned but lacking their own page.
+5. Missing cross-references.
+6. Data gaps that could be filled with a web search.
+
+*The LLM is good at suggesting new questions to investigate and new sources to look for. This keeps the wiki healthy as it grows.*
 
 ## Indexing and Logging
-Two special files help navigate the wiki as it grows:
-- **`wiki/index.md` (Content-oriented)**: A catalog of everything in the wiki. Each page listed with a link, a one-line summary, and optionally metadata (date, source count). Organized by category. The LLM updates it on every ingest. Read this first when answering a query.
-- **`wiki/log.md` (Chronological)**: An append-only record of what happened and when (ingests, queries, lint passes). Prefix entries consistently (e.g., `## [YYYY-MM-DD] ingest | Title`) so it's parseable via simple grep commands.
+Two special files help the LLM (and you) navigate the wiki as it grows. They serve different purposes:
+
+**index.md** is content-oriented. It's a catalog of everything in the wiki — each page listed with a link, a one-line summary, and optionally metadata like date or source count. Organized by category (entities, concepts, sources, etc.). The LLM updates it on every ingest. When answering a query, the LLM reads the index first to find relevant pages, then drills into them. This works surprisingly well at moderate scale (~100 sources, ~hundreds of pages) and avoids the need for embedding-based RAG infrastructure.
+
+**log.md** is chronological. It's an append-only record of what happened and when — ingests, queries, lint passes. A useful tip: if each entry starts with a consistent prefix (e.g. `## [YYYY-MM-DD] ingest | Article Title`), the log becomes parseable with simple unix tools. The log gives you a timeline of the wiki's evolution and helps the LLM understand what's been done recently.
 
 ## Why this works
-The tedious part of maintaining a knowledge base is not the reading or the thinking — it's the bookkeeping: updating cross-references, keeping summaries current, noting contradictions, and maintaining consistency. Humans abandon wikis because the maintenance burden grows faster than the value. LLMs don't get bored, don't forget to update cross-references, and can touch 15 files in one pass. The human curates sources and directs the analysis; the LLM does everything else. This relates to Vannevar Bush's *Memex* (1945) — a curated knowledge store with associative trails — but solves the problem of "who does the maintenance".
+The tedious part of maintaining a knowledge base is not the reading or the thinking — it's the bookkeeping. Updating cross-references, keeping summaries current, noting when new data contradicts old claims, maintaining consistency across dozens of pages. Humans abandon wikis because the maintenance burden grows faster than the value. LLMs don't get bored, don't forget to update a cross-reference, and can touch 15 files in one pass. The wiki stays maintained because the cost of maintenance is near zero.
+
+The human's job is to curate sources, direct the analysis, ask good questions, and think about what it all means. The LLM's job is everything else.
+
+The idea is related in spirit to Vannevar Bush's Memex (1945) — a personal, curated knowledge store with associative trails between documents. Bush's vision was closer to this than to what the web became: private, actively curated, with the connections between documents as valuable as the documents themselves. The part he couldn't solve was who does the maintenance. The LLM handles that.
 
 ## Rules
 - Never modify files in `raw/`.
@@ -112,10 +140,11 @@ The tedious part of maintaining a knowledge base is not the reading or the think
 - Confidence levels: **high** (multiple sources), **medium** (single source), **low** (speculative).
 - Polish language for content, English/Technical slugs for filenames.
 
-## Optional Tools & Tips
-- **Obsidian Web Clipper**: Browser extension for converting web articles to markdown to quickly get sources into your `raw/` collection.
-- **Download Images Locally**: Configure Obsidian to download attachments locally (e.g., to `raw/assets/`). This allows the LLM to view and reference images directly without relying on URLs that may break. (LLMs may need to read text first, then process images separately).
-- **Obsidian Graph View**: The best way to see the shape of the wiki, identify hubs, and spot orphan pages.
-- **Dataview Plugin**: Use this Obsidian plugin to generate dynamic tables and lists by querying the YAML frontmatter the LLM adds to pages.
-- **Marp**: Markdown-based slide format. Used in `wiki/presentations/` to generate decks directly from content.
-- **qmd (Search Tool)**: A local search engine for markdown files (hybrid BM25/vector search + LLM re-ranking). Very useful as the wiki grows beyond what `index.md` can efficiently handle. Can be used as a CLI tool or MCP server.
+## Optional: CLI tools, Tips and tricks
+- **Obsidian Web Clipper** is a browser extension that converts web articles to markdown. Very useful for quickly getting sources into your raw collection.
+- **Download images locally.** In Obsidian Settings → Files and links, set "Attachment folder path" to a fixed directory (e.g. `raw/assets/`). Then in Settings → Hotkeys, search for "Download" to find "Download attachments for current file" and bind it to a hotkey. After clipping an article, hit the hotkey and all images get downloaded to local disk. This lets the LLM view and reference images directly instead of relying on URLs that may break. 
+- **Obsidian's graph view** is the best way to see the shape of your wiki — what's connected to what, which pages are hubs, which are orphans.
+- **Marp** is a markdown-based slide deck format. Obsidian has a plugin for it. Useful for generating presentations directly from wiki content.
+- **Dataview** is an Obsidian plugin that runs queries over page frontmatter. If your LLM adds YAML frontmatter to wiki pages (tags, dates, source counts), Dataview can generate dynamic tables and lists.
+- **qmd** is a good option for proper search: it's a local search engine for markdown files with hybrid BM25/vector search and LLM re-ranking, all on-device. It has both a CLI and an MCP server.
+- The wiki is just a git repo of markdown files. You get version history, branching, and collaboration for free.