DBAdmin/brain/raw/articles/5_menthal models.md

title, source, author, published, created, description, tags

title	source	author	published	created	description	tags
The 5 Core Mental Models for AI Agents: Harness & Memory (Deep Dive + Action Plan) - BPMS Team				2026-05-14		clippings

Overview

This page distills the five consensus mental models expert teams use to take agents from demos to production, then maps them to a concrete fraud-analysis use case built with Gemini CLI, MCP tools, and BigQuery. Use it as a reference for design reviews, implementation planning, and capability audits.

Table of Contents

Mental Model 1 — The Model Is the CPU. The Harness Is the OS.

"Strip away the harness and you have a raw language model guessing its way through your codebase. Add the right harness and you have a system that ships production code."

Core Idea

LLMs are powerful but general-purpose reasoning engines. Quality emerges from the harness (the “OS”) that shapes inputs/outputs, tools, policies, and runtime controls. Changing the harness often changes outcomes more than swapping models.

Analogy Mapping

• Model → CPU: raw intelligence (e.g., Gemini 2.5 Pro)
• Context Window → RAM: working memory (MCP context, GEMINI.md)
• Harness → OS: tools, routers, policies, safety gates
• Agent → Application: your fraud pipeline (prompts + tools + flow)
• Memory → Disk/SSD: persistent state across sessions

Why It Matters

• Different harnesses around the same model produce large score deltas on realistic tasks.
• Slimmer, better-curated toolboxes often outperform maximalist sets due to lower decision entropy.

Design Implications

• Make the “OS layer” explicit: state store, checkpoints, retries, idempotency, and traceability.
• Bias for simple, tight tool catalogs with strong schemas and pre/post-conditions.
• Invest early in observability of agent steps, not just final replies (see Observability references below).

Your Gemini CLI fraud agent already has a proto-harness (MCP tools, GEMINI.md, SQL generation). The missing “OS” capabilities are: state persistence, checkpoint/resume, and run-level tracing.

Mental Model 2 — Context Is the Product

"Every harness artifact answers the same question: What does the agent need to know before it writes a single line of code?"

The Context Stack

5. Procedural Memory ("How we do things here")

4. Semantic Memory ("What is true right now")

3. Episodic Memory ("What happened before")

2. Session State ("Earlier this session")

1. Working Context ("Right now: last N turns + active tools")

The Core Tension

Trade off in-context stuffing vs. out-of-context retrieval:

• Full context: zero latency to recall, but costly and “lost-in-the-middle” risks.
• Sliding window: cheap, but causes session amnesia.
• RAG (vector): good semantic recall; must tune chunking, reranking, and filters.
• Hybrid vector + graph: best for entities/relations; requires ontology upkeep.

Design Implications

• Separate “always-in” system context from “retrieve-when-relevant” memory.
• Layer vector search (semantic) with graph traversal (relational correctness).
• Use LLM-managed memory policies: what to store, how to compress, when to forget.

For the query “877-417-4551,” persist: (a) semantic fact “shared phone links to 2,097 entities,” (b) episodic trace “investigation run with outcomes,” (c) procedural heuristic “for shared-phone queries use 3-hop traversal.” Your existing ArangoDB graph is a natural semantic + episodic backend.

Mental Model 3 — Separate the Doer from the Judge

Agents are unreliable self-graders. External verification (computational tests, an evaluator model, or humans) is essential.

Verification Spectrum

• Level 1: Self-check (cheap; catches formatting/syntax; misses subtle issues)
• Level 2: Computational checks (schema/type/linters/tests; low cost, medium power)
• Level 3: Inferential LLM judge (semantic/design review; medium cost)
• Level 4: Adversarial evaluator agent (high cost; highest defect catch rate)
• Level 5: Human-in-the-loop (HITL) (doesn’t scale; gold standard)

Design Implications

• Insert feedforward constraints (contracts, schemas, tool preconditions) and feedback sensors (tests, monitors, audits) in the harness.
• Never allow the authoring agent to “approve” high-impact actions; split duties.

Add for your BigQuery SQL agent: (1) pre-exec schema validation, (2) guardrails on result size/runtime; auto-ask for confirmation if estimated rows > threshold, (3) HITL gate for destructive or high-impact ops. Aligns to D&B oversight requirements and internal harness patterns observed in production agents.

Mental Model 4 — Memory Is Three Cognitive Tiers, Not One Bucket

Three Tiers

• Episodic: past events/traces/outcomes; chronological, bitemporal annotations; ideal in temporal graphs or run logs.
• Semantic: facts/entities/relationships; hybrid structured rows + embeddings; conflict detection and merging are key.
• Procedural: instructions, playbooks, heuristics; static files (AGENT.md) or runtime prompt updates with guardrails.

Design Implications

• Pick backends by tier, not “one DB for all.” Graph stores fit semantic and episodic; files or prompt-update APIs fit procedural.
• Implement conflict resolution for semantic memory (compare new facts to graph entries, merge/update/flag).
• Track “memory provenance” so the harness can justify recalls and updates.

Mapping for your fraud agent: “2,097 entities linked to 877‑417‑4551 in Florida” → Semantic (ArangoDB). “Last investigation on 23 kwi 2026 found shell factory” → Episodic. “Shared-phone queries → 3-hop traversal first” → Procedural (GEMINI.md or runtime system-prompt rule).

Mental Model 5 — Graduated Autonomy: Earn Trust Through Verification

Gradient of Autonomy

• Level 0 Read-only
• Level 1 Suggest-only
• Level 2 Approve-then-act
• Level 3 Act-then-review (audit)
• Level 4 Autonomous (bounded)
• Level 5 Fully autonomous

Design Implications

• Start lower, promote autonomy based on proven reliability (evals + memory of track record).
• Constrain the environment: tight tools, strict contracts, and yardsticks for promotion/demotion.

D&B’s internal governance aligns to 6 autonomy tiers. Use episodic memory for track record (“47 runs, zero false positives”) to graduate; use semantic memory for sensitivity flags (“touches PII”) to require HITL; use procedural memory to encode newly learned safe behaviors.

---

Applying the 5 Models to Your Gemini CLI + BigQuery Fraud Agent

Target Architecture Additions (“OS Layer”)

• State & Checkpointing: Persist per-run state and decisions (e.g., Postgres/Arango + run_id) so long tasks can resume after failure. See internal examples using LangGraph state + Postgres checkpointing in production agents.
• Observability: Trace prompts, tool calls, inputs/outputs, token/cost, and branch decisions. Capture row estimates for SQL, retrieval stats for memory calls, and validation outcomes. Align to AI observability guidance: infra + model/agent + quality/safety telemetry with correlation.
• Guardrails: Schema-aware SQL builder; preflight “EXPLAIN” or dry-run checks; result-size and latency thresholds → evaluator/HITL gates; redaction and policy filters for sensitive fields.

Memory Plan (Three Tiers)

• Episodic: Investigation traces with inputs, traversals (e.g., phone → entities → addresses), evidence URIs, outcomes, and evaluator judgments. Store bitemporally to enable regression and “what changed since last run?” analyses.
• Semantic: Entity facts in ArangoDB: phones, persons, businesses, addresses, edges with weights and recency. Add conflict detection rules when new facts disagree with old; record source and confidence.
• Procedural: GEMINI.md rules + runtime “policy injects” (e.g., “On shared identifiers: 3-hop traversal; for row estimates > 1M, ask confirmation; for PII tables, require HITL”). Maintain a changelog of procedural updates with who/what justified the change.

Harness Hardening (Doer vs. Judge)

• Feedforward: Strict tool schemas, SQL builder with typed columns and table contracts; entity resolver with disambiguation prompts; scope checks before any heavy query.
• Feedback: Computational checks (lint/validation), evaluator LLM for semantic soundness on traces, targeted HITL for high-impact actions or low-confidence decisions.

Graduated Autonomy Rollout

• Phase A (L1): Suggest-only SQL + rationale; mandatory evaluator review for all queries.
• Phase B (L2): Approve-then-act for read queries under thresholds; HITL for anything exceeding table sensitivity or row/latency caps.
• Phase C (L3): Act-then-review for non-sensitive reads with strong historical precision; auto-rollback heuristic if evaluator flags anomalies.

Context Strategy

• Working context: last N tool calls + current hypothesis + active graph nodes.
• Session state: rolling compressed summary of this investigation run.
• Episodic/semantic recall: hybrid retrieval—vector similarity on notes/explanations plus graph traversal for facts; re-rank by provenance and freshness.

---

Implementation Blueprint

Milestone 1 — OS Layer Foundations

• Add run_id and checkpoint tables; store step-level inputs/outputs and decisions.
• Integrate AI tracing (prompts, tool calls, latencies, cost) and expose searchable logs.
• Harden SQL tool: typed schema, table allowlist, safe templates, EXPLAIN pre-checks.

Milestone 2 — Memory (ArangoDB-first)

• Design graph schema for phone↔entity↔address relations with edge attrs: weight, recency, source, confidence.
• Implement semantic write path with conflict detection/merge rules; provenance is mandatory.
• Add episodic store for run traces linked to graph nodes/edges; enable “delta since last run.”

Milestone 3 — Doer/Judge Split

• Introduce computational validators (schema, row-estimate caps, sensitive-table guards).
• Add evaluator LLM for semantic quality on investigations (fact sufficiency, alternative explanations, leakage risks).
• Wire HITL gates for flagged conditions; log reviewer decisions back into episodic memory.

Milestone 4 — Graduated Autonomy

• Define promotion criteria (precision/recall on eval sets, incident-free runs, operator feedback).
• Automate demotion on evaluator regressions or anomaly spikes.
• Track autonomy tier in agent state; show in UI with audit trail.

---

Risk Controls and Operational Readiness

• Data governance: Align table access to sensitivity tiers; mask/redact PII in context; retain audit logs.
• Cost control: Token usage and query-cost meters; context compression; hybrid retrieval to avoid over-stuffing prompts.
• Reliability: Timeouts, retries with backoff; idempotent tool calls; partial-progress resumes via checkpoints.
• Evaluation: Seed test sets and continuous evals for accuracy, grounding, and safety; track scorecards per model/tool revision.

Quick-Start Artifacts

• AGENT.md (procedural seed): investigation steps, guardrails, escalation criteria.
• Tool contracts: JSON Schemas for SQL generation, graph traversal, entity resolution.
• Evaluator prompts: structured critique rubric (soundness, sufficiency, safety, performance impact) with pass/block decision.
• Runbook views: dashboard panels for per-run trace, thresholds triggered, autonomy tier, and memory writes.

FAQ

No. Start with a single agent and strong harness (tools, validators, evaluator). Add a separate evaluator agent later if quality gaps remain or scale requires parallel critics.

Use both. Vector for semantic recall of notes/past narratives; graph for factual correctness and relationship traversal. Your ArangoDB gives you a strong graph spine; add vectors for unstructured artifacts with provenance links.

Define quantitative gates (precision/recall on evals, zero critical incidents over last N runs, latency/cost SLO adherence) plus qualitative operator feedback. Store outcomes in episodic memory as evidence.

Follow-ups

• Create a 1-page “Mental Models Notebook” PDF with diagrams for team onboarding.
• Deliver a gap analysis specifically for your Gemini CLI + BigQuery fraud agent with a 30–60–90 day plan.
• Deep-dive design for “ArangoDB as memory backend” (schema, indices, traversal templates, conflict-resolution rules, and retrieval orchestration).

References