Open Knowledge Format (OKF)
Open Knowledge Format
(OKF) is an open, human- and agent-friendly format for representing knowledge:
a directory tree of markdown files with YAML frontmatter. There is no schema
registry and no required tooling — if you can git clone a repo, you can ship
an OKF bundle.
GrafitoDB can import an OKF bundle into a queryable property graph and export a graph back to an OKF bundle. This makes OKF a durable, diffable, human-readable storage layer while GrafitoDB provides Cypher, full-text, and semantic search on top — a natural fit for agent memory: persist knowledge as markdown in git, index and query it at runtime.
Start with OKFBundle
OKFBundle is the recommended high-level entry
point: load a bundle, navigate concepts/links/citations, search by meaning,
assemble grounded context() for an
agent prompt, and write memory back — with the raw graph one attribute away
via bundle.db. The functions below are the low-level layer it delegates to.
Core concepts
A quick glossary before diving in — each term links to where it's covered in depth.
- Bundle — the directory tree of markdown files itself: what you import from and export back to (see How concepts map to the graph).
- Concept — one markdown file, with YAML frontmatter (
type,title, ...) plus a body; becomes one node on import. - Layer — the directory a concept lives in (
decisions/,glossary/,runbooks/, ...), used for progressive disclosure (kb.layers(),kb.index()). - Link — a markdown link from one concept to another; becomes a typed
relationship (
LINKS_TOby default, or a custom type derived from headings viatyped_links). - Source — an entry of the
sourcesfrontmatter: a material the concept derives from, named by aresource(an external URL, another concept, a followable artifact, or a scope descriptor) plus optional credibility signals. Becomes aCITESrelationship. Called a citation throughout this page, since that is the relationship type and the accessor name. OKF v0.1 wrote the same thing as a list under a# Citationsheading; both forms are imported. - Stub — a placeholder concept created because something links to it
before it exists (or it has no
type); promoted in place once the real file shows up (see How concepts map to the graph). - Reference — the node a citation points to when the target is an external URL rather than a concept.
- Proposal — a concept held in a review queue instead of merged directly, pending a human or auto-approve decision (see Review queue).
Prerequisites
How concepts map to the graph
| OKF | GrafitoDB |
|---|---|
Concept (a .md file) |
Node |
type (frontmatter, required) |
Node label |
title, description, resource, tags, extra keys |
Node properties |
generated, verified, status, stale_after |
Node properties, read through trust/lifecycle accessors |
Concept ID (e.g. tables/orders) |
Node uri (okf:tables/orders) |
| Markdown body | body property (feeds full-text search) |
Markdown link [x](/tables/y.md) |
Relationship (LINKS_TO by default) |
sources frontmatter entry |
CITES relationship (to a concept or a Reference node), carrying the entry's id and credibility signals |
Link under a legacy # Citations heading |
The same CITES relationship (OKF v0.1 form) |
computation, executor.resource, attester.resource |
HAS_COMPUTATION / EXECUTED_BY / ATTESTED_BY relationship |
index.md / log.md |
Skipped (reserved, derivable) |
Concepts without a type and links to not-yet-written concepts fall back to the
generic Concept label (permissive consumption — broken links are tolerated,
not errors). A file whose frontmatter is not valid YAML does not abort the
import either: its full text is kept as the body and its ID is reported under
the summary's malformed key. The whole import runs in a single transaction,
so large bundles load fast and a hard failure leaves the database untouched.
Importing a bundle
from grafito import GrafitoDatabase
db = GrafitoDatabase(":memory:")
summary = db.import_okf_bundle("path/to/bundle")
print(summary) # {'nodes': 8, 'relationships': 9, 'stubs': 3, 'skipped': 6}
Once imported, the knowledge is fully queryable:
# Cypher: what does the Orders table link to?
db.execute("""
MATCH (a {title: 'Orders'})-[:LINKS_TO]->(b)
RETURN b.title AS target
""")
# Full-text search over titles, descriptions, and bodies
db.text_search("customer", k=5)
Semantic search
Pass an embedding function to embed each concept into a vector index at import
time. Concepts are embedded from their title, description, and body, so
you can query the bundle by meaning rather than keywords:
from grafito.embedding_functions import SentenceTransformerEmbeddingFunction
embedder = SentenceTransformerEmbeddingFunction("all-MiniLM-L6-v2")
db.import_okf_bundle("path/to/bundle", embed=embedder)
# Query by meaning; the index already knows how to embed the query text
db.semantic_search("how do customers pay for orders", index="okf", k=5)
Relevant import options:
| Argument | Default | Description |
|---|---|---|
embed |
None |
An EmbeddingFunction; when set, concepts are embedded for semantic search. |
embed_index |
"okf" |
Name of the vector index created for concept embeddings. |
embed_fields |
("title", "description", "body") |
Concept fields concatenated into the embedded document. |
embed_backend |
"bruteforce" |
Vector index backend (the default needs no extra dependencies). |
embed_options |
None |
Extra options for the vector index — {"store_embeddings": True} persists the vectors in the database for reuse across sessions; {"index_path": ...} places a file-backed index (faiss/hnswlib/...). |
The summary dict reports the number of embedded concepts. This pairs full-text
(text_search) and vector (semantic_search) retrieval over the same imported
bundle — useful for hybrid agent-memory workflows.
Options
| Argument | Default | Description |
|---|---|---|
link_type |
"LINKS_TO" |
Relationship type created for intra-bundle markdown links. |
typed_links |
False |
Derive the relationship type from the heading a link sits under — a link under # Joins with becomes a JOINS_WITH relationship. Links before any heading, under # Links, or under headings that don't normalize to a valid type keep link_type. |
wikilinks |
False |
Also resolve Obsidian-style [[Note]] links (see Obsidian wikilinks). |
configure_fts |
True |
Configure full-text search over title/description/body (best-effort; skipped if SQLite lacks FTS5). |
uri_prefix |
"okf:" |
Prefix prepended to each concept ID to form the node uri. |
progress_every |
None |
Print a progress line every N concept files (and per phase) — for large bundles. |
progress |
None |
Callback (phase, count) invoked instead of printing; phases are concepts, links, citations, embedded, done. |
The import runs in a single transaction and creates an expression index on
concept_id, so concept lookups stay fast as bundles grow.
The returned summary dict reports nodes, relationships, stubs (nodes
created for links whose target is not in the bundle), skipped
(index.md/log.md files), and malformed (concept IDs whose frontmatter
was not valid YAML and was imported as plain body text).
Obsidian wikilinks
An Obsidian vault is already an OKF bundle without a plugin — directories of
markdown files with YAML frontmatter. What Obsidian adds is its own link
syntax: [[Note]], [[Note|Alias]], [[Note#Heading]]. Pass
wikilinks=True to resolve those alongside plain markdown links:
Obsidian links by note title, not by path, so resolution is vault-wide rather than relative to the linking file:
- An exact concept-ID match first —
[[decisions/0001-use-sqlite]]works like a normal absolute markdown link. - Otherwise, a case-insensitive match against every concept's basename
(filename without the directory) —
[[0001-use-sqlite]]resolves the same way, from any file in the vault. - A basename shared by more than one concept is ambiguous and is skipped rather than guessed (no relationship, no stub).
- A target matching nothing becomes a stub keyed by the literal link text — Obsidian's own "red link" (not-yet-written note) convention. If a note with that exact title is added later, the stub is promoted just like a broken markdown link's would be (see Incremental import).
[[Note|Alias]] uses Alias as the relationship's anchor (falling back to
Note without one); the #Heading fragment is dropped, same as for markdown
links. Wikilinks participate in typed_links the same way markdown links do.
Only the main body is scanned — a [[Note]] under # Citations is not
picked up (citations resolve by URL or markdown link, not by title).
Provenance: the sources frontmatter
OKF v0.2 keeps a concept's provenance in frontmatter (SPEC §5.1) instead of a
# Citations list in the body. Each entry names a resource and may carry a
stable id (the key a body footnote cites), a title, and the credibility
signals author, usage_count, and last_modified — with usage_window
written once as a sibling to frame every count:
sources:
- id: rev-policy
resource: https://wiki.acme/finance/revenue-recognition
title: Revenue recognition policy
author: team:finance-fpa
last_modified: 2026-04-02
- id: exec-rev-dash
resource: dashboards/exec-revenue
usage_count: 5000
usage_window: { from: 2026-06-01, to: 2026-06-30 }
Every entry becomes a CITES relationship, so provenance is graph-queryable
however it was authored. What the target is depends on the resource:
resource |
Becomes |
|---|---|
| An external URL | A Reference node (deduplicated across the bundle) |
A path to a concept (/metrics/x.md, ../x.md) |
An edge to that concept — a lineage edge you can recurse into |
A .md path with no file yet |
A stub concept, like a broken markdown link |
Any other path (references/attesters/x.py) |
A Reference node — source paths never create stub concepts for files that are not concepts |
A scope descriptor (all queries in project X) |
A Reference node flagged scope_descriptor |
A descriptor is told apart from a path by containing whitespace; the SPEC gives no other marker.
The credibility signals live on the edge, not the Reference node, because
they describe this concept's use of the source: two concepts can cite one
dashboard over different usage windows. The shared usage_window is applied to
each edge as its default, so a query never has to look up the sibling key:
kb.db.execute(
"MATCH (c)-[r:CITES]->(t) WHERE r.usage_count > 1000 RETURN c.concept_id, t.title"
)
kb.concept("computations/revenue").cites()
# [{'url': 'https://wiki.acme/...', 'anchor': 'Revenue recognition policy',
# 'id': 'rev-policy', 'author': 'team:finance-fpa', ...}, ...]
The import summary reports sources (edges from frontmatter) alongside
citations (every citation edge, whatever form it was authored in).
A legacy # Citations body list is still imported — v0.1 bundles keep working
(SPEC §13.1) — and each edge records which form it came from in its via
property, so save() writes it back the way it was authored instead of
emitting both forms and doubling the edges on the next import.
Trust, freshness, and lifecycle
OKF v0.2 makes four questions answerable from frontmatter (SPEC §5.2–5.5): who wrote this, who confirmed it, is it the current version, and is it still true.
status: stable # draft | stable | deprecated
generated: { by: reference_agent/gemini-2.5-pro, at: 2026-06-20T22:53:05Z }
verified:
- { by: human:ahormati, at: 2026-06-25T09:00:00Z }
- { by: process:finance-nightly, at: 2026-06-26T02:00:00Z }
stale_after: 2026-09-23
generated and verified stay distinct because whoever wrote a concept need
not be whoever confirmed it, and content can be re-confirmed without being
regenerated. Read them through Concept:
c = kb.concept("computations/revenue")
c.generated_by # 'reference_agent/gemini-2.5-pro'
c.generated_at # '2026-06-20T22:53:05Z' — falls back to a v0.1 `timestamp`
c.verified # [{'by': 'human:ahormati', 'at': ...}, ...] — always a list
c.verified_at # the most recent check
c.trust_tier # 'unverified' | 'machine-confirmed' | 'human-reviewed'
c.stale_after # '2026-09-23'
c.is_stale # today >= stale_after
Two details the SPEC is strict about:
verifiedis always a list here, even when the concept writes a single verifier as a bare mapping without the list dash. Reading that as one element is required of consumers (§11), so the accessor normalizes it — whilesave()writes it back in the form it was authored.- The trust tier is derived, never stored.
human-reviewedwhen any verifier is ahuman:actor (§7),machine-confirmedwhen there are only non-human ones,unverifiedwith noverifiedkey. OKF stores the signals, not a verdict, so a tier can never go stale against the list it came from.
To record a check, use verify() — the writer counterpart, symmetric with
supersede():
kb.verify("computations/revenue", by="human:jp") # `at` defaults to now (UTC)
kb.verify("computations/revenue", by="process:nightly")
Events accumulate rather than overwrite, so a human sign-off and a nightly process are two independent checks.
Filtering retrieval by trust and freshness
search() and context() both take min_trust and include_stale:
kb.search(q, min_trust="human-reviewed") # only concepts a person confirmed
kb.search(q, include_stale=False) # drop anything past its stale_after
kb.context(q, min_trust="human-reviewed") # also governs graph expansion
Like where/tag, these govern graph-expanded neighbours in context(),
not just the seeds — a pack assembled under min_trust="human-reviewed" cannot
pick up an unreviewed concept through a link. Anything dropped is recorded in
pack.omitted with reason "low_trust" or "stale".
Note the asymmetry with include_superseded, which defaults to excluding:
retraction is the author asserting a claim is wrong, while staleness only says
"re-check me". Hiding stale knowledge by default would quietly empty a bundle
whose author set conservative dates, so include_stale defaults to True.
Attested computations
An Attested Computation concept (SPEC §10) carries not just what a value
means but the sanctioned way to compute it, plus the means to confirm a run
produced it that way. Three of its fields name paths pointing outside the
concept, and grafito turns each into an edge:
| Frontmatter | Relationship |
|---|---|
computation |
HAS_COMPUTATION |
executor.resource |
EXECUTED_BY |
attester.resource |
ATTESTED_BY |
kb.db.execute("""
MATCH (c)-[:ATTESTED_BY]->(a)
RETURN a.resource AS attester, count(c) AS guards ORDER BY guards DESC
""")
That makes "what runs this" and "what checks it" traversable instead of merely
readable — you can ask which attester guards the most computations, or let
context() pull an executor's skill document in alongside the computation it
runs. grafito never executes any of it: §10.5 puts running and attesting
on the consumer, and the format only records the contract.
Targets resolve exactly like a sources path: a concept the bundle has, a
stub for a .md document not written yet, or a Reference for anything else —
an attester is usually a .py file, which is followable but is not a concept.
A value that is not a path at all (an inline computation written as prose)
creates no edge and stays an ordinary property.
One resolution detail worth knowing: a plain relative path is tried against the
citing concept and against the bundle root. The SPEC writes
references/skills/run-on-bq.md from a concept inside computations/ while
placing that tree at the root (§6.3), so only the second reading finds it.
validate_okf_bundle warns about a contract a consumer could not honour: an
Attested Computation with no runtime, or with neither a computation path nor
a # Computation body section; a malformed executor/attester; a parameter
with no name. All warnings — §10 is a SHOULD, not one of the hard conformance
rules.
Incremental import
Re-importing a bundle normally reparses and re-embeds every file. For a large
or frequently-updated bundle, pass incremental=True to make re-imports cheap:
db.import_okf_bundle("path/to/bundle", embed=embedder, incremental=True)
# ... edit a couple of files in the bundle directory ...
summary = db.import_okf_bundle("path/to/bundle", embed=embedder, incremental=True)
print(summary["unchanged"], summary["updated"], summary["nodes"])
Each concept's raw file content is hashed (okf_hash, stored on the node and
excluded from exported frontmatter) and compared against the hash recorded on
its last import:
- Unchanged files are skipped entirely — no re-parsing, no re-embedding, no relationship churn. This is the main cost saved on a re-import.
- Changed files are updated in place: the same node ID is kept (so
relationships from unchanged concepts pointing at it stay valid), its own
outgoing links/citations are regenerated, and it is re-embedded. Edges added
by the trust model (
OKFBundle.supersede/conflicts_with—SUPERSEDES/CONFLICTS_WITH) are left untouched. - New files are created as usual. A link that previously created a stub (SPEC §6.1) is promoted in place when the target file is later added, rather than creating a duplicate node.
- Pass
prune=True(requiresincremental=True) to also delete nodes whose concept file was removed from the bundle since the last import — mirrors the exporter'spruneoption.
The summary dict gains unchanged, updated, and pruned counts, and
references/stubs count only genuinely new nodes (existing Reference/stub
nodes are reused, not duplicated).
directory_nodes/import_log are not incremental-aware: combining them with
incremental=True duplicates directory/log nodes on every re-import.
Validating a bundle
validate_okf_bundle is the linter counterpart to the importer's permissive
consumption: it checks a bundle against the OKF v0.2 conformance rules
(SPEC §11) without importing anything and without stopping at the first bad
file:
from grafito.okf import validate_okf_bundle
report = validate_okf_bundle("path/to/bundle")
report["conformant"] # True when there are no errors
report["errors"] # [{'path', 'error'}] — missing frontmatter block,
# unparseable YAML, missing/empty required `type`
report["warnings"] # [{'path', 'warning'}] — broken intra-bundle links,
# malformed `sources` entries, unreadable trust or
# lifecycle fields, frontmatter in a non-root index.md
Errors are conformance failures; warnings are soft guidance a consumer must tolerate (a broken link may simply be not-yet-written knowledge).
Layered linting: lint_okf_bundle
validate_okf_bundle only checks hard SPEC conformance. lint_okf_bundle
wraps it and adds two more layers — a three-tier model (Core / Profile /
Hygiene):
from grafito.okf import lint_okf_bundle
report = lint_okf_bundle("path/to/bundle", profile="profile.yaml")
report["conformant"] # Core has no errors AND no Profile rule severity="error" fired
report["core"] # {"errors", "warnings"} — identical to validate_okf_bundle
report["profile"] # [{"path", "rule", "message", "severity"}, ...]
report["hygiene"] # [{"path", "rule", "message"}, ...] — always advisory
Profile rules are bundle-specific and come from a manifest — a dict, or a path to a YAML file:
rules:
- id: adr-requires-status
applies_to: ADR # a type name, a list of types, or "*" (default)
require_field: status # missing, or empty ("" / [] / {}) unless non_empty: false
severity: error # "error" blocks `conformant`; "warning" (default) doesn't
- id: title-max-length
field: title
max_length: 80
severity: warning
A rule may combine more than one check: require_field (+ non_empty),
forbid_field, max_length (+ field), allowed_values (+ field),
pattern (a regex, + field).
Hygiene is a fixed set of best-practice checks for a knowledge graph
specifically — not customizable, and never blocks conformant:
missing-title, missing-description, short-body (main body under
short_body_chars, default 40, excluding citations), orphan-concept (no
intra-bundle links in or out — disconnected from the graph), and
duplicate-title (two concepts sharing a title).
mode="audit" (default) is the human-facing report — all three layers.
mode="validate" drops Hygiene, for a CI gate that only cares about
conformance:
report = lint_okf_bundle("path/to/bundle", profile="profile.yaml", mode="validate")
assert report["conformant"], report["core"]["errors"] + report["profile"]
Previewing a replacement: diff_okf_bundles
Before swapping a live bundle for a freshly built one, diff_okf_bundles
previews exactly what would change — a pure, read-only diff between two OKF
trees on disk. Neither side is imported (no graph, no LLM, no network), so it is
safe to run against a candidate you haven't accepted yet:
from grafito.okf import diff_okf_bundles
diff = diff_okf_bundles("kb/", "kb_staging/") # base, candidate
diff.added # [rel_path] — concepts only in the candidate
diff.removed # [rel_path] — concepts only in the base
diff.changed # {rel_path: ConceptDelta}
diff.invalid # {rel_path: error} — candidate conformance errors
diff.broken_links # [(rel_path, target)] — links to concepts absent in the candidate
diff.summary() # {"added", "removed", "changed", "invalid", "broken_links"} counts
diff.has_changes # bool — added or removed or changed
diff.conformant # bool — not invalid (broken links are warnings, never blocking)
Each changed entry is a ConceptDelta describing what moved — field-level
frontmatter changes (type included, so a retype is visible) and whether the
markdown body changed:
delta = diff.changed["decisions/0001-use-sqlite.md"]
delta.frontmatter_added # {key: candidate_value}
delta.frontmatter_removed # {key: base_value}
delta.frontmatter_changed # {key: (base_value, candidate_value)}
delta.body_changed # bool
Two properties make the preview trustworthy:
invalidandbroken_linksreusevalidate_okf_bundleon the candidate, so the conformance rules live in exactly one place.- The
changedset matches what an import would do. The content hash it compares is byte-for-byte theokf_hashthe incremental importer stores and re-checks, sochangedis exactly the set of concepts a subsequentimport_okf_bundlewould re-process — the preview never disagrees with the real import. Reserved files (index.md,log.md) are excluded, matching the importer and the validator.
The intended flow: build a candidate into a staging directory, diff_okf_bundles
it against the live one, show a human the added/removed/changed concepts
plus any invalid files or broken_links, and only then swap the directories.
Exporting a bundle
The inverse operation serializes the graph back to OKF markdown:
This writes:
- one markdown file per node (label →
type, properties → frontmatter,body→ markdown body); - per-directory
index.mdfiles for progressive disclosure — the root index lists child directories under# Subdirectories, and each directory groups its concepts bytype; - an optional self-contained
viz.htmlgraph viewer (write_viz=True).
Stub nodes (created for broken links during import) are not exported. Nodes
created programmatically without a stored body get synthesized link sections
from their outgoing relationships: LINKS_TO edges under # Links, and every
other type under a heading derived from it (JOINS_WITH → # Joins with), so
typed relationships round-trip through markdown when re-imported with
typed_links=True.
Options
| Argument | Default | Description |
|---|---|---|
uri_prefix |
"okf:" |
Prefix used to recover concept IDs from node URIs. Should match the import value. |
write_index |
True |
Generate per-directory index.md files. |
okf_version |
None |
Declare the format version as okf_version frontmatter in the root index.md — the only index file allowed to carry frontmatter (SPEC §12). OKFBundle.save() defaults to whatever the bundle declared when it was loaded, so a declaration survives the round-trip; pass None there to drop it. |
write_viz |
False |
Also emit a self-contained viz.html at the bundle root. |
write_log |
True |
Regenerate per-scope log.md files from the graph's LogEntry nodes (imported history plus log_entry/autolog additions). Scopes without entries are left alone — an existing log.md is never blanked. |
prune |
False |
Delete concept .md files that no longer correspond to a node (directories left empty are removed). log.md and non-markdown files are never touched. OKFBundle.save() prunes by default so removals round-trip. |
Round-trip and agent memory
Import → query/enrich → export is lossless for the graph structure and preserves unknown frontmatter keys:
db = GrafitoDatabase(":memory:")
db.import_okf_bundle("bundle")
# ... query, traverse, or add knowledge via Cypher / the programmatic API ...
db.execute("CREATE (n:Playbook {title: 'New runbook', body: 'Steps...'})")
# Persist the enriched knowledge back to markdown (commit it to git)
db.export_okf_bundle("bundle")
Multi-label nodes
OKF concepts have a single type. When a node has several labels, the
exporter uses the first label as type; representing multi-label nodes is
an open design question with no OKF-side convention yet.
High-level API: OKFBundle
The functions above are the low-level layer. grafito.okf.OKFBundle is an
OKF-flavored façade over them: it speaks concepts/links/citations/layers instead
of nodes/relationships, while exposing the full graph via bundle.db.
from grafito.okf import OKFBundle
kb = OKFBundle.load("examples/okf/okf_knowledge_base", embed=embedder)
kb.layers() # {'decisions': 3, 'glossary': 3, 'runbooks': 1}
kb.index() # root index.md, in memory (subdirs)
kb.index("decisions") # a directory's listing: title+description, no bodies
c = kb.concept("decisions/0003-vector-search")
c.title # 'Add optional vector search'
c.links() # [Concept, ...] any outgoing link type
c.links(type="JOINS_WITH") # restrict to one type (typed_links bundles)
c.cites() # [{'url'|'concept', 'anchor', 'id'?, signals...}, ...]
c.trust_tier # 'unverified' | 'machine-confirmed' | 'human-reviewed'
c.is_stale # today >= stale_after
kb.search("how do I make a query run faster", k=3) # semantic / text / hybrid
kb.search("make it faster", layer="decisions") # scoped to a layer
kb.search("vector similarity", mode="hybrid") # RRF fusion of FTS + vector
kb.search("storage", where={"owner": "data-team"}) # filter on frontmatter
kb.search("storage", min_trust="human-reviewed") # filter on trust tier
# hybrid degrades to text-only when the bundle was loaded without embed=
kb.db.execute("MATCH (n) RETURN count(n)") # escape hatch: full graph power
kb.save("out/bundle", write_viz=True) # round-trip back to markdown
Filtering on frontmatter: where= and tag=
OKF keeps every producer-defined frontmatter key as a node property, so
status, owner or confidentiality are all queryable. search()
and context() accept where= to filter retrieval on them — without dropping
to Cypher and losing ranking, graph expansion, and budgeting. Nested keys work
too, via a dotted path: where={"generated.at": PropertyFilter.gte(...)}
filters on the trust family without any special support.
from grafito.okf import PropertyFilter, PropertyFilterGroup
kb.search(q, where={"owner": "data-team"})
kb.search(q, where={"owner": "data-team", "confidentiality": "public"}) # AND
kb.search(q, where={"generated.at": PropertyFilter.gte("2026-01-01")}) # nested key
kb.search(q, where=PropertyFilterGroup.or_({"status": "draft"},
{"status": "stable"}))
where= speaks the same dialect as match_nodes, so there is nothing new to
learn: multiple keys in one dict combine with AND, PropertyFilter adds
operators (gte, between, contains, regex, ...), and PropertyFilterGroup
adds OR and nesting. concepts(where=...) takes the same argument for listing.
tag= is a separate parameter rather than a where key, because tags is a
list: where={"tags": "draft"} compares against the whole list and matches
nothing, while tag="draft" tests membership.
kb.search(q, tag="draft") # concepts tagged draft
kb.concepts(tag="draft", where={"owner": "data-team"})
Two behaviours worth knowing:
- The filter also governs graph expansion in
context(). A concept the filter excludes cannot slip into the pack through aLINKS_TOedge either; it is recorded inpack.omittedwith reason"filtered". That makeswhere={"confidentiality": "public"}a real guarantee about what reaches the prompt, not a hint expansion can route around.type/layerremain seed-only — they scope retrieval, whereaswhere/tagexpress what may be shown. include_supersededis itself astatusfilter. Sowhere={"status": "superseded"}returns nothing unless you also passinclude_superseded=True.
On the retrieval path, where= is pushed into the vector index as a true
pre-filter, so a highly selective filter still returns a full k. Full-text
search (FTS5) has no property filtering, so there the filter runs after BM25 and
the search window widens automatically until it yields k survivors or the
corpus is exhausted — a selective filter never silently returns an empty result.
That widening is not free: it re-runs BM25 over a growing window, so a very
selective filter in text mode costs several times an unfiltered search (~270 ms
vs ~35 ms over 20k concepts; unmeasurable on small bundles, and unaffected when
no filter is passed). Semantic and hybrid modes do not pay it — they pre-filter.
Prefer loading with embed= if you filter aggressively over a large bundle.
Grounded context for agents: context()
search() returns ranked hits; context() turns them into a prompt. It is the
framework-agnostic bridge to any agent loop — no LangChain, LlamaIndex, or SDK
required. Given a question it:
- seeds retrieval with
search()(semantic / text / hybrid); - graph-expands — follows each hit's outgoing links (any relationship
type except
CITES, including typed links) withinexpand_hops, so the pack carries linked context the embedding alone would miss (the GraphRAG edge over a flat vector store); - packs the concepts into a token budget as titled, cited blocks, greedily
in priority order (the top hit is never dropped — it is truncated if it alone
exceeds the budget). A graph-expanded block's header names the relationship
that pulled it in (e.g.
### Semantic search · Term · glossary/semantic-search · via JOINS_WITH) — explicit provenance the LLM can cite, not just a block that happens to sit nearby. Seed hits (found directly bysearch()) carry novia.
pack = kb.context("how do I make a query run faster", budget_tokens=2000)
str(pack) # prompt-ready text (same as pack.text)
pack.citations # [{'url'|'concept', 'anchor', 'cited_by': [...]}, ...] — deduped
pack.concepts # the Concepts that made it into the budget, in order
pack.hits # the seed search Hits (scores/provenance)
pack.tokens # estimated token count of the packed text
pack.truncated # True if anything was dropped/cut to fit
pack.omitted # [{'concept_id', 'title', 'reason', 'via'}, ...] — what was left out
pack.trace # step log when include_trace=True, else None
prompt = f"Answer using only this context:\n\n{pack}" # drops straight into a prompt
The pack is auditable: pack.omitted spells out what retrieval reached but
left out, so nothing is dropped silently. Each entry has a reason:
"budget"— a candidate that didn't fit the token budget;"superseded"— a retracted claim reached via graph expansion (see Trust model);"stale"— past itsstale_after, withinclude_stale=False;"low_trust"— below the requestedmin_trusttier (see Trust, freshness, and lifecycle);"filtered"— a graph-expanded neighbour excluded bywhere/tag(see Filtering on frontmatter);"reranked_out"— a candidate a reranker'stop_ndiscarded.
context(..., include_trace=True) additionally fills pack.trace with a compact,
deterministic step log — one step each for search (the index actually used, hit
count, plus filtered_to when where/tag narrowed the corpus), expand (hops
and neighbours added), rerank (pool in/out, only when a reranker runs), and
pack (budget, included/omitted counts, final tokens, truncated) — so an agent
can explain why the context is what it is:
pack = kb.context(question, include_trace=True)
pack.trace
# [{'step': 'search', 'mode': 'semantic', 'hits': 8},
# {'step': 'expand', 'hops': 1, 'added': 5},
# {'step': 'pack', 'budget_tokens': 2000, 'included': 6,
# 'omitted': 2, 'tokens': 1974, 'truncated': True}]
| Argument | Default | Description |
|---|---|---|
budget_tokens |
2000 |
Token budget for the packed text. |
k |
8 |
Seed hits to retrieve before expansion. |
mode |
"auto" |
"semantic" / "text" / "hybrid" / "auto". |
type, layer |
None |
Restrict retrieval to a concept type / directory layer (seeds only). |
where, tag |
None |
Filter on frontmatter — also applied to graph-expanded neighbours (see Filtering on frontmatter). |
expand_hops |
1 |
Outgoing link hops to graph-expand (0 disables); follows any relationship type except CITES. |
include_citations |
True |
Render Sources: lines and collect pack.citations. |
token_counter |
heuristic | Callable str -> int; default ≈ 4 chars/token. Pass your model's tokenizer for exact budgeting. |
rerank |
None |
An optional reranker (see below). |
Reranking
A bi-encoder (embedding) retrieves cheaply but coarsely. A reranker re-scores
candidates against the query text — the standard RAG precision step. In
context() it matters most: graph expansion deliberately pulls in loosely
related neighbours, and the reranker decides which of them deserve the token
budget. It runs over the seed + expanded pool before packing.
A Reranker is any callable (query, candidates) -> [(concept, score), ...]
(most relevant first) — inject your own, or use one of the bundled ones:
from grafito.okf import (
LexicalReranker, # dependency-free (query-term overlap); offline default
CrossEncoderReranker, # local HuggingFace cross-encoder (sentence-transformers)
CohereReranker, # Cohere rerank API
VoyageReranker, # Voyage AI rerank API
JinaReranker, # Jina AI rerank API
)
kb.context(question, rerank=LexicalReranker()) # offline, no deps
kb.context(question, rerank=CrossEncoderReranker("BAAI/bge-reranker-base")) # local HF
kb.context(question, rerank=CohereReranker()) # needs COHERE_API_KEY
# Custom: any matching callable works — no subclassing required.
def my_reranker(query, candidates):
return sorted(((c, score(query, c)) for c in candidates), key=lambda p: -p[1])
kb.context(question, rerank=my_reranker)
The API rerankers (Cohere/Voyage/Jina) need httpx and read their API key
from the matching environment variable (e.g. COHERE_API_KEY) or an explicit
api_key=. Requests use a 30s timeout (timeout=), and the instances are
context managers (close() releases the HTTP client). CrossEncoderReranker
needs sentence-transformers but runs offline. A reranker may return a subset
(e.g. its own top_n); context() packs exactly the order and subset it
returns.
Mutating a bundle (agent-memory write path):
kb.add_concept("notes/idea", type="Note", title="An idea",
body="# Notes\n...", tags=["draft"]) # embedded + FTS-indexed
kb.update_concept("notes/idea", body="# Notes\nRevised…",
status="reviewed") # partial update; re-embeds
kb.update_concept("notes/idea", description=None) # None removes a field
kb.link("notes/idea", "decisions/0001-use-sqlite", anchor="builds on")
kb.cite("notes/idea", "https://example.com/paper", anchor="source")
kb.remove_concept("notes/old")
kb.save() # persist to markdown
update_concept changes only the fields you pass (including type, which
relabels the node, and any producer-defined frontmatter key); the FTS index and
the vector embedding follow automatically. save() mirrors the graph to disk:
files for removed concepts are pruned so remove_concept round-trips (pass
prune=False to only add/overwrite).
Trust model: supersede() and conflicts_with()
An agent writing memory unattended can silently overwrite a correct claim with
a hallucinated one if edits always land in place. supersede() and
conflicts_with() give the write path the append-only-on-meaning discipline
from the OKF trust model: corrections create a new concept and link it to the
old one, rather than rewriting the old one's meaning.
new = kb.add_concept("decisions/0002-use-hnswlib", type="Decision",
title="Use hnswlib for ANN", body="# Context\n...")
kb.supersede("decisions/0001-use-bruteforce", new, note="scaled past 50k vectors")
kb.conflicts_with("glossary/latency", "glossary/throughput",
note="one source defines these interchangeably")
supersede(old, new) sets status="deprecated" / superseded_by on old,
appends to supersedes on new, and links new -[:SUPERSEDES]-> old (typed,
so it round-trips with typed_links=True). It does not delete or rewrite
old — the retracted claim stays inspectable via kb.concept(old_id), git
blame, and kb.log(). conflicts_with(a, b) is the softer, symmetric
sibling for when new information contradicts an existing concept without
strong enough evidence to supersede it outright: it links both concepts via
CONFLICTS_WITH (both directions — a conflict has no natural direction)
without changing either.
deprecated is OKF's own lifecycle value for "kept for links and history, no
longer current" (SPEC sec. 5.4), so retraction stays inside the spec's
vocabulary; which concept replaced it is the producer-defined superseded_by
key, which lifecycle alone cannot express.
search() and context() exclude status="deprecated" concepts by default
(include_superseded=True opts back in), so retrieval never hands an agent a
retracted claim as if it were current truth — while concepts()/concept()
still surface them for provenance and history browsing. This applies to any
deprecated concept, including one the bundle's author deprecated by hand rather
than through supersede().
kb.concept("decisions/0001-use-bruteforce").is_superseded # True
kb.concept("decisions/0001-use-bruteforce").superseded_by # 'decisions/0002-use-hnswlib'
new.supersedes # ['decisions/0001-use-bruteforce']
kb.concept("glossary/latency").conflicts() # [Concept('glossary/throughput')]
Review queue: propose(), approve(), reject()
add_concept() always writes immediately — right for a human curating the
bundle, or a pipeline you trust. An autonomous agent proposing new facts is
a different trust level: it might be duplicating something that already
exists, or contradicting it under a different id. propose() is the
agent-facing entry point that gates on that:
result = kb.propose("decisions/0004-use-hnswlib", type="Decision",
title="Use hnswlib for ANN", body="# Context\n...")
if isinstance(result, Proposal):
result.similar # [{'concept_id', 'title', 'score', 'via'}, ...]
kb.approve(result) # materializes it (same node id) — or:
kb.reject(result, note="duplicate of 0003")
Three modes, controlled by auto_approve:
None(default, conditional): searches the bundle for concepts similar to the proposal. With a vector index it auto-approves unless a hit scores at or abovesimilarity_threshold(cosine similarity, default0.85); without one (text-only), there's no comparable numeric scale, so any FTS hit at all triggers review. Nothing to compare against (no title/description/body, or no hits) auto-approves.True: always writes immediately, likeadd_concept— returns aConcept.False: always stages it, regardless of similarity — returns aProposal.
A staged proposal is a real graph node (pending_reviews() survives process
restarts) but is invisible to concept(), concepts(), search(),
iteration, and save() until approve()d — it can't leak into retrieval or
round-trip to markdown while undecided. It carries no links/citations of its
own; wire those up with link()/cite() after approval. An id collision
with an existing concept always raises — that's a hard conflict, not a
similarity judgment call, so it isn't staged for review either.
kb.pending_reviews() # [Proposal, ...], ordered by id
kb.approve("decisions/0004-use-hnswlib") # or an id string
kb.reject("decisions/0004-use-hnswlib", note="duplicate of 0003")
Changelog: log_entry() and autolog
An agent that writes memory should also leave a history. log_entry() appends
a changelog entry (a LogEntry node, SPEC §9) that save() serializes to the
scope's log.md — and autolog=True at load time does it automatically for
every add_concept / update_concept / remove_concept:
kb = OKFBundle.load("bundle", import_log=True, autolog=True)
kb.add_concept("notes/idea", type="Note", title="An idea", body="...")
kb.update_concept("notes/idea", description="Refined.")
kb.log_entry("Consolidated duplicate notes.", kind="Update",
concepts=["notes/idea"]) # manual entry, MENTIONS the concept
kb.log() # entries newest first, including the imported history
kb.save() # regenerates log.md per scope (git-diffable history)
Autolog entries embed a markdown link to the concept
(**Creation**: Created [An idea](/notes/idea.md).), so MENTIONS edges
survive markdown round-trips. Load with import_log=True when the bundle
already has a log.md so new entries extend the history instead of replacing
it on save().
Round-trip note: save() writes each concept's stored body verbatim. For a
concept created without a body, link edges are synthesized into a
# Links section on export (so they round-trip); for a concept with a
body, include the links in that body if you want them in the markdown — the
edges remain queryable in the graph regardless.
Citations are different: because v0.2 keeps provenance in frontmatter rather
than the body, cite() edges are always written into the concept's sources
block, body or no body. An authored entry naming the same resource wins (it is
round-tripped verbatim, so re-exporting an imported bundle is byte-stable);
edges no entry covers are appended.
Persistent reuse across sessions
load() parses markdown and (optionally) embeds every concept — work you only
want to do once. Back the bundle with a database file and persist the
embeddings, then later sessions open() the file directly: no markdown
parsing, no re-embedding.
# Session 1 — import once, persist graph + embeddings.
kb = OKFBundle.load(
"path/to/bundle",
db=GrafitoDatabase("kb.db"),
embed=embedder,
embed_options={"store_embeddings": True},
)
# Session 2+ — open the database file; the vector index rehydrates from it.
kb = OKFBundle.open(GrafitoDatabase("kb.db"), source_path="path/to/bundle")
kb.search("how do I make a query run faster") # semantic, no re-embedding
Pass embed= to open() only when the embedding function is a custom one the
registry cannot rebuild by name (built-ins such as the SentenceTransformer
function are rehydrated automatically from the index metadata). source_path
is optional; it sets the default save() target. For very large indexes,
prefer a file-backed ANN backend via embed_backend="hnswlib" (or faiss)
plus embed_options={"index_path": "kb.hnswlib"}.
If the bundle's markdown keeps changing across sessions, load() forwards
unknown keyword arguments to import_okf_bundle, so incremental=True (see
Incremental import) works the same way: OKFBundle.load("bundle",
db=GrafitoDatabase("kb.db"), embed=embedder, incremental=True) re-parses and
re-embeds only what changed since the last load().
Materializing the directory tree and history (opt-in) lets you traverse the hierarchy as a graph and query the changelog:
kb = OKFBundle.load("bundle", directory_nodes=True, import_log=True)
kb.children() # {'subdirs': ['decisions', ...], 'concepts': [...]}
kb.children("decisions") # one level down, via CONTAINS edges
kb.log() # all log.md entries, newest first
kb.log("decisions/0001-use-sqlite") # entries that mention this concept
directory_nodes=True adds Directory nodes + CONTAINS edges (root → subdir →
concept); import_log=True adds LogEntry nodes linked to mentioned concepts via
MENTIONS. Both are synthesized/derived and are skipped on export.
Design notes:
- Delegates, never duplicates —
load/savecallimport_okf_bundle/export_okf_bundle; the low-level API stays the canonical implementation. search()unifies grafito's text and vector results into a singleHit(hit.concept,hit.score,hit.via);mode="auto"uses vectors when the bundle was loaded withembed=, else full-text.context()is framework-agnostic — it returns prompt-ready text plus citations, not a framework-specific object; thererank=hook is any callable.Conceptis a thin view;concept.nodeis the raw grafito node.- Lookups scale —
concept(),concepts(),layers(),index()andlen()filter in SQL (backed by an expression index onconcept_id); only matching nodes are hydrated.layer=accepts nested paths ("references/joins"), matching any concept below that directory. - Captures
okf_versionfrom the rootindex.md(lost by the low-level import).
Examples
Both runnable examples use the OKFBundle façade. examples/okf/okf_import.py is a
short intro (load → concept/links → search → save with viz.html) over the
tabular sample bundle in examples/okf/okf_bundle/:
OKF shines on narrative, cross-linked knowledge rather than tabular data.
examples/okf/okf_knowledge_base/ is a small engineering knowledge base —
architecture decision records, an on-call runbook, and glossary terms, all
cross-linked with citations. The script walks the full façade — index/traversal,
the directory tree, aggregation via the kb.execute escape hatch, semantic
search, grounded context() assembly with a reranker, the agent-memory write
path, and visualization (it retrieves a "slow query" runbook for the query
"how do I make a query run faster", which never uses those words):
Agentic GraphRAG: grafito.okf.agent
Where context() is one-shot GraphRAG, grafito.okf.agent lets the model
drive the exploration itself through OpenAI-style tool calls:
BundleTools— the bundle façade as function tools:browse(progressive disclosure),search(hybrid),open(full concept + typed edges),follow(graph traversal by relationship type),history(changelog), andremember(write a linked, embedded, autologged note back into the bundle). Schemas + dispatch, framework-free:tools.schemasplustools.call(name, args)is the pair PydanticAI, CrewAI, LangGraph and MCP all ask for — see Other agent frameworks. Tool errors come back as{"error": ...}for the model to react to instead of killing the loop (raise_errors=Truepropagates them instead).run_agent(kb, question, chat=...)— a minimal tool-calling loop. One-shot by default; passmessages=to thread a multi-turn conversation,tools=to inject a scopedBundleTools, orextra_tools=to add app-specific tools. Returns anAgentRun(below), not a bare string.AgentRun— the answer plus what the run cost:turns, everyToolCall(name, args, result size, error), aggregated tokenusage, and the finalmessages.str(run)is the answer;run.summary()gives the efficiency numbers. See Measuring a run.Chat— the model contract: any callable(messages, tools) -> assistant messagein OpenAI chat format. Grafito never imports an LLM SDK — the client is injected, likererank=.ToolSet— the tool contract: any object withschemas(OpenAI function-tool schemas) and a matchingcall(name, args) -> str.BundleToolsis one; write your own the same shape to plug in viaextra_tools=.ThreadConfinedTools— the same toolset, reachable from any thread. A bundle belongs to the thread that opened it; this wrapper gives one dedicated thread ownership and queues every call onto it, for frameworks that run tools off the main thread. See Threading.OpenAIChat— the bundled convenience for any OpenAI-compatible endpoint (OpenAI, Ollama, vLLM, LM Studio, OpenRouter, ...); needshttpx, readsOPENAI_BASE_URL/OPENAI_API_KEY/OPENAI_MODEL.AnthropicChat— Claude via the officialanthropicSDK (pip install grafito[anthropic]). Translates the loop's OpenAI format to the Anthropic Messages API (system prompt,input_schematools,tool_use/tool_resultblocks), runs with adaptive thinking and preserves thinking blocks across turns. Defaults toclaude-opus-4-8; credentials resolve fromANTHROPIC_API_KEY(or anant auth loginprofile), andANTHROPIC_MODELoverrides the model.
from grafito.okf import OKFBundle, OpenAIChat, run_agent
kb = OKFBundle.load("bundle", embed=embedder, autolog=True)
run = run_agent(kb, "why did we pick SQLite?", chat=OpenAIChat())
print(run) # str(run) is the answer
kb.save() # the agent's remember()ed notes + changelog land in git
Multi-turn conversations
run_agent is one-shot by default: each call builds a fresh messages list
and discards it on return. Pass a list via messages= to thread a
conversation across calls instead — run_agent extends it in place with this
turn's question, tool calls, and answer, so the next call (same list) can
refer back to earlier turns without re-exploring the bundle:
history: list[dict] = []
run_agent(kb, "why did we pick SQLite?", chat=chat, messages=history)
run_agent(kb, "and what did we rule out?", chat=chat, messages=history)
Tool results (e.g. full concept bodies from open) accumulate in history
turn over turn, so a long-running conversation costs more tokens each turn —
fine for a modest back-and-forth, but a very long session will eventually
need trimming or summarization of older turns (not handled automatically).
Measuring a run
run_agent returns an AgentRun, so the cost of agentic exploration is
observable rather than inferred. str(run) is the answer; the rest is the
receipt:
run = run_agent(kb, "why did we pick SQLite?", chat=chat)
run.answer # the model's final text ("" if stopped_early)
run.stopped_early # True when max_turns ran out before an answer
run.turns # model calls made
run.tool_calls # [ToolCall(turn, name, args, result_bytes, error), ...]
run.usage # aggregated tokens, {} if the Chat reports none
run.turn_usage # the same counts per model call, in order
run.messages # the conversation (the same list you passed as messages=)
run.summary()
# {'turns': 3, 'tool_calls': 4, 'errors': 1, 'repeated_calls': 0,
# 'result_bytes': 8214,
# 'by_tool': {'search': {'calls': 2, 'errors': 0, 'bytes': 731},
# 'open': {'calls': 2, 'errors': 1, 'bytes': 7483}},
# 'usage': {'input_tokens': 18400, 'cached_input_tokens': 12000,
# 'cache_write_tokens': 0, 'output_tokens': 430, 'requests': 3},
# 'input_per_turn': [1121, 5482, 11797],
# 'resent_input_tokens': 6603}
What each number is actually for:
repeated_calls— invocations with a(name, args)the model already issued in this run. Non-zero means it is re-reading what is already in its context: the most actionable inefficiency signal here.errors/by_tool[...]['errors']— with a scopedBundleTools, this is how manyUnknown conceptresponses the filter fed the model.result_bytes— the size of tool output handed back. This is the real driver of per-turn cost, since results stay inmessagesfor the rest of the conversation, andopenis usually where it concentrates. Grafito computes it itself, so it is available even with aChatthat reports no tokens.usage— token counts, normalized across providers:input_tokensis the whole prompt sent that turn (cached parts included), withcached_input_tokens(≈0.1× price) andcache_write_tokens(≈1.25×, Anthropic only) broken out. Summinginput_tokensacross turns is the real billed cost, not the size of the context — a tool loop re-sends the full history every turn. Read the cached slice before concluding a long conversation was expensive.input_per_turn/resent_input_tokens— the growth curve, and how much of it was a repeat. The loop only appends tomessages, so the last turn's prompt already contains every distinct token the run sent; everything billed before it was re-sent. That figure is what prompt caching bills at roughly a tenth, so the gap between it andcached_input_tokensis the saving your endpoint is leaving on the table — on an OpenAI-compatible gateway with no cache support, a measured run spent 61% of its input tokens re-sending. See Which one should you use? for what that costs against a one-shotcontext()call.
usage stays {} for an injected Chat that reports nothing; the loop never
invents numbers. OpenAIChat and AnthropicChat both report.
verbose=True prints the same summary as a closing line after the trace.
Scoping an agent to part of the bundle
where=/tag= also apply to the agentic path, by scoping the whole toolset:
from grafito.okf import BundleTools, run_agent
tools = BundleTools(kb, where={"confidentiality": "public"})
run_agent(kb, question, chat=chat, tools=tools)
The filter is fixed by the application, not chosen by the model — it appears
in no tool schema, so the agent cannot widen or disable it. Every read path
honours it (browse, search, open, follow, history), which is the whole
point: a filter covering only search would be theatre, because the model reads
a concept id out of a link and opens it directly. Excluded concepts are also
stripped from the edge lists open returns and from follow results, and the
system prompt is built from the filtered layer counts.
A hidden concept is reported exactly like a nonexistent one
({"error": "Unknown concept: ..."}), so the agent cannot probe for what exists
behind the filter.
The filter is an access boundary, not redaction
It governs structure and retrieval — what can be listed, matched, opened,
and traversed. It cannot redact prose inside a concept you chose to show: a
visible body linking to a hidden concept still contains that markdown, and a
log.md line may name one. The model may learn a hidden id exists, though
every tool refuses to open it. If a body must not mention something, that
belongs in the bundle.
Likewise, remember writes plain notes that do not inherit the filter's
fields — so with a filter active the agent may not read back what it just wrote.
Grafito deliberately does not stamp the filter's values onto new notes, since
that would let an agent mark its own output status: approved.
Scoping the tool surface
where=/tag= scope what the tools can reach. include=/exclude= scope
which tools exist — the two are independent:
BundleTools(kb, exclude=["remember"]) # read-only
BundleTools(kb, include=["search", "open"]) # retrieval only
BundleTools(kb, where={"status": "approved"}, exclude=["remember"])
schemas reflects the choice, and call() enforces it: a disabled name is
refused exactly like one that never existed, so a framework routing a stale or
invented name cannot reach a tool you removed. Passing an unknown name to
include/exclude — or both arguments at once — raises ValueError at
construction, not at call time. BundleTools.ALL_SCHEMAS is always the full
list, independent of any instance.
This is the natural way to hand a toolset to a framework that owns its own
write path, or to give one agent in a crew read access and another the ability
to remember.
Other agent frameworks
run_agent is Grafito's own loop, but the toolset — not the loop — is the
integration point. BundleTools exposes exactly the pair every framework asks
for: tools.schemas (OpenAI-style function schemas) and
tools.call(name, args) -> str. Adapting is a schema translation plus a
closure. examples/okf/okf_frameworks.py is runnable (no framework needs to be
installed) and carries the adapter code for each target:
| Framework | What it takes |
|---|---|
| LangChain / LangGraph | Nothing to translate — bind_tools(tools.schemas) accepts the OpenAI format as-is; wire tools.call() into the ToolMessage. |
| MCP | Peel the "function" wrapper and rename parameters → inputSchema. A server is ~20 lines. |
| PydanticAI | Tool.from_schema(fn, name=..., json_schema=...) — no pydantic model to generate. Declare the tools async def (see threading below). |
| CrewAI | A BaseTool subclass per tool, with args_schema synthesized from the JSON Schema via pydantic.create_model, plus a ThreadConfinedTools wrapper. |
The PydanticAI and CrewAI adapters were verified end-to-end against a live
model (pydantic-ai 2.13, crewai 1.15): the agent runs search → open and
answers with a concept-id citation, exclude=["remember"] is honoured in the
tool list it is offered, and the where= boundary still refuses hidden
concepts through the framework.
One thing worth knowing before you write one: build the toolset with
raise_errors=True. PydanticAI turns exceptions into retries and CrewAI has
its own error handling; the default {"error": ...} string would read as a
successful call and defeat both. Keep the default for a raw tool-calling
loop, where an error the model can read beats a crash.
Always route through tools.call()
Reimplementing the tools against kb.search() / kb[concept_id] directly
is the tempting shortcut — it looks more native in a typed framework — and
it silently drops the where=/tag= access boundary along with the edge-
list stripping in open/follow. The filter lives in BundleTools, not
in the bundle.
Threading
A bundle belongs to the thread that opened it. sqlite3 connections are
created with check_same_thread=True, so reaching the bundle from another
thread raises sqlite3.ProgrammingError — and agent frameworks routinely run
tools off the main thread. This is the one part of an integration that is not a
pure schema translation.
It is not solved by asyncio.to_thread: that moves the call off the
owning thread and causes the very error it looks like it would prevent.
- PydanticAI — declare the tools
async def, so they run on the event loop's own thread. Its sync tools go to a worker thread and fail. - CrewAI, and anything else without an async tool path — wrap the toolset
in
ThreadConfinedTools.
ThreadConfinedTools gives one dedicated thread ownership of the bundle and
queues every call onto it. The factory runs on that thread and must open
the bundle itself — opening it in the caller and closing over it defeats the
purpose:
from grafito.okf import BundleTools, OKFBundle, ThreadConfinedTools
with ThreadConfinedTools(
lambda: BundleTools(OKFBundle.load(path, autolog=True), raise_errors=True)
) as tools:
agent = Agent(role="KB analyst", goal="...", tools=crewai_tools(tools))
...
tools.run(lambda t: t.kb.save()) # anything that is not a tool call
It is a ToolSet like any other, so it also works as run_agent(..., tools=)
or in extra_tools=. run() is the way to reach the bundle for everything
that is not a tool call — kb.save() included: calling it from the main
thread after an agent's remember raises the same ProgrammingError.
Calls are serialized, so this is a correctness device, not a scaling one:
concurrent agents queue behind each other. That is the honest tradeoff while
the bundle is single-threaded — the alternative, check_same_thread=False,
would share mutable state (transaction flag, in-memory vector and text
indexes) with no locking at all.
Custom tools
run_agent always includes BundleTools(kb) — browse/search/open/follow/
history/remember. Pass extra_tools= with your own ToolSet\ s to add
tools that have nothing to do with the bundle, e.g. sending a message or
calling an internal API. No base class required, just the same
schemas/call shape as BundleTools:
class SlackTools:
schemas = [{
"type": "function",
"function": {
"name": "notify_channel",
"description": "Post a message to the team Slack channel.",
"parameters": {
"type": "object",
"properties": {"text": {"type": "string"}},
"required": ["text"],
},
},
}]
def call(self, name: str, args: dict) -> str:
post_to_slack(args["text"]) # your own integration
return '{"posted": true}'
run_agent(kb, question, chat=chat, extra_tools=[SlackTools()])
Tool names must be unique across BundleTools and every extra_tools
entry — a collision (e.g. defining your own search) raises ValueError
before the model is ever called, rather than one tool silently shadowing
another. A tool call for a name no toolset owns comes back as
{"error": ...}, same as any other tool error the model can react to.
For a non-OpenAI-format provider, the adapter is a few lines — e.g. litellm:
import litellm
def chat(messages, tools):
response = litellm.completion(model="anthropic/claude-sonnet-5",
messages=messages, tools=tools)
return response.choices[0].message.model_dump()
run_agent(kb, question, chat=chat)
examples/okf/okf_agent.py is the runnable end-to-end walkthrough (explore →
answer with concept citations → remember a note → save bundle + log.md):
export OPENAI_BASE_URL=http://localhost:11434/v1 # e.g. Ollama
export OPENAI_MODEL=llama3.1
python examples/okf/okf_agent.py
Model Context Protocol server: grafito-mcp
run_agent drives a model you inject. The MCP server turns the same
tools the other way round: it exposes them over the Model Context
Protocol so a client that already has a model
— Claude Desktop, Claude Code, any MCP host — can use a bundle (or a plain
graph) with no integration code on your side. One install, no build:
uvx --from 'grafitodb[mcp]' grafito-mcp --bundle ./okf_bundle # a bundle
uvx --from 'grafitodb[mcp]' grafito-mcp --db ./graph.db # a plain graph
The server speaks stdio JSON-RPC; point a client's config at it, e.g. Claude
Desktop's claude_desktop_config.json:
{
"mcpServers": {
"grafito": {
"command": "uvx",
"args": ["--from", "grafitodb[mcp]", "grafito-mcp",
"--bundle", "/abs/path/to/okf_bundle",
"--embed", "sentence_transformer"]
}
}
}
Read-only by default — a client can explore and ground answers, but not
write, until you opt in with --enable-writes.
The tools, by altitude
The server offers tools at rising altitude; a client picks the highest one that answers its question. Which appear depends on the flags:
| Tier | Tools | When |
|---|---|---|
| Grounded (escalón 1) | context — a question in, prompt-ready cited context out, in one call |
--bundle |
| Explore (escalón 1) | browse, search, open, follow, history |
--bundle |
| Write (escalón 1) | remember — save a note, persisted back to the bundle |
--bundle --enable-writes |
| Structured (escalón 2) | graph_schema, text_search, vector_search, graph_neighbors |
--db, or --bundle --enable-graph |
| Cypher (escalón 3) | graph_query — read-only Cypher, row-capped |
--db, or --bundle --enable-graph |
context is the star: it is the one-shot context()
path, cheaper than a client running its own exploration loop, and the tool that
sets this apart from a raw graph server. The graph tiers hang on the underlying
GrafitoDatabase, not on the bundle — which is why --db can serve a graph
that is not an OKF bundle at all, exposing only those tiers with no OKF in play.
Writes persist to the bundle
With --enable-writes, remember is exposed and each saved note is written
back to the bundle directory (markdown + regenerated log.md) as it happens —
so the note survives the process and shows up in git diff. The bundle is the
memory; a crash cannot swallow what the client was told was saved.
Graph mode is read-only
graph_query runs arbitrary Cypher, but a query containing a mutating clause
(CREATE/MERGE/DELETE/DETACH/SET/REMOVE/DROP) is refused, and at
most --max-rows rows come back. Escalón 4 (writes over the graph) is not
implemented: these tiers cannot mutate the graph even when asked.
Flags
| Flag | Applies to | Default | Effect |
|---|---|---|---|
--bundle PATH / --db PATH |
— | required (one) | Serve an OKF bundle, or a plain .db graph |
--name NAME |
both | grafito |
Server name reported to the client |
--embed NAME / --embed-config JSON |
--bundle |
none | Embedder for semantic search/context (else text mode) |
--rerank NAME / --rerank-config JSON |
--bundle |
none | Reranker for context grounding (lexical is dep-free) |
--tag TAG |
--bundle |
none | Scope every tool to concepts with this tag |
--budget-tokens N |
--bundle |
2000 |
Token budget for context output |
--enable-graph |
--bundle |
off | Add the escalón 2-3 graph tiers over the bundle's graph |
--enable-writes |
--bundle |
off | Expose remember and persist writes to the bundle |
--max-rows N |
--db, --enable-graph |
100 |
Cap on graph_query rows |
The retrieval-quality and write flags apply to a bundle; passing them with
--db is an error, not a silent no-op.
How it stays generic
The server itself knows nothing about OKF. It hangs on a ToolRegistry
(grafito.ToolRegistry) — a bag of ToolSets — and serves whatever tools it
holds. --bundle loads the OKF toolsets
(ContextTools + BundleTools), --db loads the graph toolsets
(GraphTools + CypherTools), optionally both. Adding a tier is adding a
ToolSet to the registry, never touching the server — the same seam
run_agent uses. Building your own server over a custom mix of toolsets is
therefore a few lines: