You Are Six Months Into a Data Pipeline Project
And the knowledge is everywhere.
Schema design documents live in three versions across Confluence -- you are not sure which is current. A runbook for deployment exists in a README, but it was updated twice and you cannot find the changelog. Architecture decision records are scattered: one in git, one in Notion, one pinned in a Slack thread from two months ago. When a new engineer asks "did we decide on event sourcing or CDC?" you have to dig through documents and hope you remember which version to trust.
That is the reference knowledge problem. Your project has become a labyrinth -- every corridor looks like the right one, but there is no map. It is different from session memory (LoreConvo solves that). This is about organized, discoverable, canonical knowledge that you need to retrieve by topic, not by time.
The Hidden Cost of Scattered Knowledge
On the surface it seems like a minor inconvenience -- you spend five or ten minutes finding the right document. But the real cost is invisible.
Every time you and your team search for "what did we decide on partitioning?" you are burning cognitive overhead: remembering where to look, guessing which tool to search, hoping you find the current version and not an obsolete one. Multiple engineers searching for the same piece of knowledge means the same friction multiplied. Across a six-month project with a three-person team, that adds up to dozens of hours lost to context hunting.
There is a second cost that shows up in code: decisions made from outdated documents. A teammate reads an old version of the schema design and implements against stale assumptions. The implementation works locally but conflicts with decisions made two months later. Code review catches it, but now you have wasted a day debugging and rewriting. That cost compounds across a team.
And there is a third cost: onboarding overhead. When you bring someone new into the project, they ask "what is the current architecture?" and you do not have a single place to point them. You end up doing a one-on-one explanation that you have already given to the last three people who joined.
Why Sessions Start With Stale Knowledge
The problem exists because knowledge lives in many tools, each designed for a different purpose. Confluence is great for collaborative editing but weak at versioning. Notion is intuitive but does not integrate with AI workflows. Git is precise but scattered across files. Slack is where decisions get made, but the threads disappear into history. So you use all of them, and the "canonical source of truth" is whichever one you remember to check first.
Most general-purpose documentation tools try to solve this by adding search and organization features. But search that finds keywords is not the same as search that understands what you mean. And organization that relies on folders forces you to guess the structure upfront -- is a schema design decision a "technical design" or an "architecture decision" or a "database pattern"? You categorize it wrong and it becomes invisible.
What you actually need is a system designed specifically for reference knowledge: documents that need to stay current, be instantly discoverable, and integrate with the way you actually work -- inside Claude.
LoreDocs: Vault-Based Knowledge Organization
LoreDocs organizes knowledge into vaults -- semantic containers for related documents.
A vault is not a folder. It is a labeled collection around a topic. Here is what a typical vault structure looks like for a data engineering project:
[Architecture Vault]
- Partition Strategy (v2, current)
- Incremental Load Patterns (v1)
- Date Handling Standards (v3)
- Schema Evolution Policy (v2)
[Operations Vault]
- Deployment Runbook (v3, latest)
- Troubleshooting Guide (v2)
- Monitoring Setup (v2)
[SQL Patterns Vault]
- Window Function Examples (v4)
- Partition Pruning Techniques (v2)
Each vault is independent. Architecture decisions live in one place. Operational procedures in another. SQL patterns in a third. When you search or retrieve, you get current documents without wading through deprecated versions. The structure is semantic (organized by meaning), not hierarchical (organized by guessed folder names).
Current Status: Alpha
LoreDocs is production-ready for storing and organizing reference knowledge, but team collaboration features are still being built. A solo engineer can get started this week: create vaults, add documents, and immediately search across them by topic -- no upfront schema, no guessing folder names. When you ask Claude to inject the Architecture vault at the start of a session, Claude pulls your actual current documents into context, not a summary or paraphrase. You can also tag documents with custom labels like current, deprecated, or needs-review, which makes it straightforward to surface what is authoritative at a glance.
The road ahead is about scaling that same clarity to teams. We are building shared vault access so multiple engineers read the same canonical knowledge, web publishing so a vault can become a public-facing knowledge base, and integration with external sources like Git and Confluence so the pipeline between where decisions are made and where Claude retrieves them gets even shorter.
Free tier covers the practical workload for a solo project or small team. Pro tier (coming soon) unlocks team features and larger knowledge bases.
Why Not Just Use Confluence/Notion/Google Drive?
You open Confluence on a Tuesday morning to answer a teammate's question about your partitioning strategy. There are four pages with similar titles. You click the most recent one, but the "last edited" date is seven months ago. You check a second page -- this one has a note at the top: "see v2 for updates." The v2 link is broken. You eventually find the right document buried in a subfolder labeled "Archive (maybe?)". By the time you paste the answer into Slack, twenty minutes are gone.
These tools were designed for collaborative writing, not for knowledge that needs to stay current and machine-retrievable. The folder hierarchy requires guessing where things belong upfront. Full-text search returns keyword matches, not intent -- "incremental" finds fifty results and you have to read them to know which one is authoritative. Version management defaults to "make a copy" culture: version 1, version 1 final, version 1 final REAL. There is no systematic indicator that a document has been superseded.
LoreDocs is designed for the opposite case: technical knowledge that changes over time, where "current" must be unambiguous, and where Claude needs to retrieve the right document -- not the most-recently-touched one.
A Concrete Example: Onboarding with Vaults
Here is how it works in practice.
Without LoreDocs: A new engineer joins the project. You point them to a Confluence space with forty pages. They find a document about partition strategy but it is v1 from January. You did not realize it was stale. They ask "should we use event sourcing or CDC?" and you realize they read the wrong version. You spend thirty minutes explaining what is actually current, repeating context you have already given to three previous engineers.
With LoreDocs: A new engineer joins the project. You give them access to the project vaults. They ask Claude "what is our current partition strategy?" Claude retrieves the Partition Strategy document from the Architecture vault. They see the version is v3 (March 2026) and know it is current. They also see which documents are related (Schema Evolution Policy, Rate Limiting & Retry Logic) because the vault keeps related knowledge connected. No stale information. No wasted explanation.
Integration with Your AI Workflow
LoreDocs is built for AI-native development. It works as an MCP server, which means Claude can access your vaults directly.
Here is what that looks like in a real session. You are about to build a new ingestion endpoint and you start Claude Code with: "Load the Architecture vault -- I am designing the incremental load path." Claude pulls the Incremental Load Patterns doc (v2, current), the Schema Evolution Policy, and your Date Handling Standards directly into context. Not a summary you wrote last month. Not a paraphrase. The actual documents, versioned and tagged as current.
When you ask "should this use CDC or event sourcing?", Claude reasons from your real architecture decisions -- the ones your team actually made -- and flags the constraints documented in the Schema Evolution Policy before you write a single line of code. The downstream effect: you build the right thing the first time. No "wait, I think we decided against that" mid-review. If you run autonomous agents, each agent has access to the same vaults. The QA agent reads the same architecture docs as the builder. The deployment agent reads the same runbook. The knowledge stays synchronized whether a human or an agent is the one asking.
Getting Started
If you build data pipelines, ship ML systems, or manage any ongoing technical work, you need a reference library. Right now, most organizations improvise -- scattered documents in multiple tools, stale information mixed with current decisions, no way to know what to trust.
LoreDocs handles the structure so you do not have to. Think of it as building a map of your system -- so Claude is never wandering blind in the maze again.
Get started with LoreDocs -- alpha access available
Labyrinth Analytics Consulting helps organizations navigate the dark corners of their data. Learn more at labyrinthanalyticsconsulting.com.