Back to Blog
LoreConvoLoreDocs

Multi-Agent Coordination Using Shared Memory

Ten scheduled AI agents, one shared memory layer: how LoreConvo and LoreDocs eliminated agent drift, and the practical patterns that made it work.

Abstract illustration of five colored agent nodes connected by glowing data streams to a central database hub on a deep purple background, representing autonomous agents coordinating through shared memory

When a fleet of autonomous agents has to work together, the hardest part is keeping everyone on the same page. In a recent fleet deployment we scaled from three agents to ten (see what we learned scaling that workforce), each pulling data, transforming it, and publishing results for downstream pipelines on an hourly schedule. The agents never spoke directly to each other; instead they read and wrote a shared memory layer built on LoreConvo and LoreDocs. The result was a system that stayed consistent across runs, recovered gracefully from failures, and let new agents join without a single line of integration code.

Why shared memory matters for autonomous agents

Agents that operate in isolation quickly diverge. One agent may decide to rename a column, another may assume the original name still exists, and a third may generate a report that mixes the two schemas. By giving every agent a single source of truth, we eliminated that drift.

LoreConvo's cross-surface session memory lets an agent store the outcome of a run as a session, tag it with the project name, and link it to any prior sessions that contributed to the result. When the next agent starts, the auto-load hook automatically pulls the most relevant prior context, so the new run begins with a concise digest of what has already been decided. Full-text search powered by SQLite's FTS5 makes it easy to locate a session by keyword -- an agent can ask "how did we handle missing values in the last month?" and receive a list of matching sessions, even when the exact phrase never appeared in any of them (we go deeper into why FTS5 held up at this scale in our search engine benchmark).

LoreDocs stores knowledge in named vaults, each searchable with the same FTS5 engine. The Pro semantic index bridges the gap between terminology and intent, so when an agent queries a data quality rule that was written under a different name, the right document still surfaces.

How LoreConvo and LoreDocs keep the memory coherent

The first step was to give each scheduled job a project tag. Tags let us group sessions by the data pipeline they belong to, and the usage stats dashboard gave immediate visibility into how many sessions each project generated per day. Skill history tracking recorded which tools each session used, so we could later audit whether a particular transformation relied on a deprecated library.

When a session finished, the auto-save hook ran without any user interaction. It extracted a heuristic summary, captured tool calls, recorded tech-stack facts, and surfaced open questions when the log contained enough signal. Because the data lives in a local SQLite file that the user owns, we never had to worry about cloud-side latency or compliance concerns.

For sessions that needed tighter coordination, we used session linking and related-session discovery. Linking creates an explicit chain so an agent can follow the lineage of a transformation from raw ingest to final report. The related-session discovery feature looks for keyword co-occurrence and embedding similarity, then suggests sessions that are likely to be useful -- in the ten-agent fleet this cut the time spent searching for precedent by more than half.

LoreDocs complemented session memory by providing a structured vault for documentation (see the vault architecture blueprint for how vaults are organized). Each vault is scoped to a workspace directory, so when an agent started it could open the appropriate vault and instantly access all relevant design notes, schema diagrams, and versioned documents. Document versioning let us roll back a change to a data dictionary without hunting through git history. The Pro semantic index chunked each document at the paragraph level, allowing agents to retrieve the exact paragraph describing a data quality rule even when the rule was phrased differently in the code.

LoreConvo exposes a CLI for bulk operations -- its export and import commands moved a month's worth of sessions between development and production, and because the export format preserves UUIDs, re-importing was idempotent and safe. LoreDocs offers its own CLI for vault and document management; since a vault is just a portable SQLite file, keeping documents in sync across environments was just as straightforward.

External tools such as managed agents sometimes needed to run in the same environment. By marking those sessions as external, LoreConvo automatically excluded them from auto-load and search, preventing accidental contamination of the core memory. When we did need to include them, a single environment variable override made the sessions visible again.

Practical lessons from a ten-agent deployment

Starting with clear project boundaries proved to be the single most important decision. Tagging sessions and vaults by project gave immediate visibility into who was writing what, and the usage stats dashboard turned that visibility into actionable data about which pipelines were generating the most memory churn.

Letting the system handle persistence removed a class of problems entirely. The auto-load and auto-save hooks meant no agent had to implement its own persistence logic. Because the hooks run on every session end, we never missed a summary, and the heuristic extraction captured decisions that would otherwise have been lost in log files.

Session linking and related-session discovery work best when you treat them as a workflow primitive rather than a search utility. When an agent needed to recompute a metric, it followed the session links back to the original data source, then used discovery to find any recent experiments that touched the same columns. That pattern replaced ad-hoc log searching with a repeatable workflow.

Separating documentation from code -- while keeping them connected -- turned out to be the right abstraction. Storing design notes in LoreDocs vaults meant that agents could query the same knowledge base that engineers used for onboarding. The workspace-scoped vault behavior reduced friction: a new agent starting in a fresh directory automatically opened the appropriate vault.

Guarding against external noise matters more than it looks. Marking sessions that originated from third-party agents as external prevented those sessions from polluting the auto-load context. When we later needed to audit those runs, we could include them explicitly with a single flag.

Local-first portability proved its value when we spun up a fresh test environment. Because all data lives in a single SQLite file, moving the memory layer to a new machine was as simple as copying the file. The agents immediately had access to the full history without any network configuration.

Pro features became essential as the session count grew beyond a few hundred. The hybrid semantic search improved relevance significantly over FTS5 alone, and the LLM-based async summarizer upgraded heuristic summaries to higher-quality text that in turn improved the relevance of auto-load digests. The consolidation log gave us transparency into when and why a memory digest was injected, which made it straightforward to tune the TTL setting and reduce token overhead in downstream prompts.

The result

The shared memory approach turned a loosely coupled set of scheduled jobs into a coordinated team of agents that could reason about each other's work. The combination of LoreConvo's session management and LoreDocs' vault system provided a single, portable source of truth that was easy to query, easy to extend, and easy to back up.

If you are building pipelines that rely on multiple AI components, or you want a reliable way to keep your agents on the same page, you can explore both tools at /tools. If you are working through a larger deployment, reach out at /contact.

The torchlight, delivered.

One email when a new post is published — agentic AI, data engineering, and memory tools. No spam, no upsell, no AI summaries. Unsubscribe anytime.

Subscribe

Labyrinth Analytics Consulting helps organizations navigate the dark corners of their data. Learn more at labyrinthanalyticsconsulting.com.

More from the blog