Converts them to Markdown (HTML, Confluence storage format, etc.)
Extracts cross-document references — URLs to GitHub PRs, GitLab MRs, Jira tickets, Confluence pages, and other linked resources are identified, classified, and stored as a reference graph in PostgreSQL
Chunks them using heading-aware splitting (preserves semantic coherence)
Embeds each chunk into vectors using your configured embedding provider
Indexes the vectors in OpenSearch for hybrid search (k-NN + BM25), with referenced document IDs attached to each chunk for enrichment at query time
After ingestion, you can immediately start asking questions. DocBrain cites sources in every answer, linking back to the original document.
Configure sources in config/local.yaml (gitignored) under the top-level
sources: block. Put only infrastructure secrets in .env. A sub-source is
enabled simply by being present in YAML — there is no separate
INGEST_SOURCES env var, and every list of targets must be non-empty.
Connect DocBrain to your Atlassian Confluence instance. DocBrain fetches pages from the spaces you specify, converts Confluence storage format to Markdown, and indexes everything.
# Restart the server to pick up the new configdockercomposerestartserver
# Run ingestiondockercomposeexecserverdocbrain-ingest
You'll see output like:
Fetching pages from space ENG... 47 pages found
Fetching pages from space DOCS... 123 pages found
Converting 170 pages to Markdown...
Chunking... 892 chunks created
Generating embeddings... done
Indexing in OpenSearch... done
Ingestion complete: 170 pages, 892 chunks
The API token inherits the Confluence permissions of the user account. DocBrain can only access pages that user can read. For broad access, use a service account with read permissions across your target spaces.
Ingest documentation from one or more GitHub repositories. DocBrain clones each
repo, finds Markdown and text files, and indexes them. Each repo may optionally
pin a specific branch via the :branch suffix — otherwise the default branch
is used.
If your docs are in a subdirectory of a larger repo, DocBrain still ingests the whole repo but filters for documentation files. Future versions will support path filtering.
When ingesting from Confluence, DocBrain automatically downloads images (diagrams, screenshots, flowcharts) from each page and uses a vision-capable LLM to generate detailed descriptions. These descriptions are injected into the document content and indexed alongside the text — making image content searchable and available for Q&A.
This is enabled by default. No extra configuration needed if your LLM provider supports vision.
Vision models (llava, llama3.2-vision, moondream) work. Text-only models (llama3.1) are auto-detected on first call — images are skipped with a warning, no failures.
When disabled, images get a [Image: filename.png] placeholder in the text (the pre-existing behavior). You can re-enable later and re-ingest to pick up image descriptions.
Image descriptions use the HAIKU_MODEL_ID model if set (recommended for cost efficiency), otherwise falls back to LLM_MODEL_ID. With Claude Haiku, expect ~$0.001 per image. A full ingestion of 1000 pages with ~3 images each costs roughly $3.
By default, DocBrain ingests documents when you run docbrain-ingest manually or on a cron schedule. But if you want pages to sync automatically the moment they're created, updated, or deleted in Confluence, enable webhook integration.
If you see CONFLUENCE_WEBHOOK_SECRET set but missing CONFLUENCE_BASE_URL/API_TOKEN — webhook sync disabled, check that both CONFLUENCE_BASE_URL and CONFLUENCE_API_TOKEN are set.
Important: The URL must be HTTPS and publicly reachable from Atlassian's servers. If DocBrain runs behind a firewall, you'll need an ingress or tunnel (e.g., ngrok for testing, or a proper reverse proxy in production).
Recommendation: Use both. Run scheduled ingest as a daily safety net (catches anything webhooks might miss — network blips, downtime), and use webhooks for real-time updates.
Real-Time Capture: @docbrain capture and /docbrain capture¶
DocBrain supports on-demand capture from GitHub PRs/issues, GitLab MRs, and Slack threads. Capture only ingests the thread into the knowledge base — it does not generate a Q&A reply. After capture, the content is immediately searchable via /docbrain ask (Slack) or the API.
Message shortcut, @DocBrain capture mention, or /docbrain capture
All thread messages, user names resolved
Posts a message in the thread confirming capture
Capture is separate from the Q&A commands. To ask a question, just mention DocBrain — @DocBrain <your question> in Slack, or @docbrain <your question> on a GitHub PR / GitLab MR. The bot replies with an answer drawn from your indexed knowledge base. (You can also still type @DocBrain ask <question> in Slack — both forms work.)
GITHUB_CAPTURE_WEBHOOK_SECRET=your-webhook-secret # generate with: openssl rand -hex 32
GITHUB_CAPTURE_TOKEN=ghp_... # Personal access token with repo:read scope
Optional access control (recommended for shared installations):
GITHUB_CAPTURE_ALLOWED_REPOS=myorg/backend,myorg/frontend # Only these repos can trigger capture
GITHUB_CAPTURE_ALLOWED_USERS=alice,bob # Only these users can trigger capture
GITLAB_CAPTURE_WEBHOOK_SECRET=your-webhook-secret # generate with: openssl rand -hex 32
GITLAB_CAPTURE_TOKEN=glpat-... # Personal access token with api scope
GITLAB_CAPTURE_BASE_URL=https://gitlab.com # Default; set for self-hosted GitLab
Optional allowlists (recommended for shared instances):
GITLAB_CAPTURE_ALLOWED_USERS=alice,bob # Only these users can trigger capture
GITLAB_CAPTURE_ALLOWED_PROJECTS=myorg/myrepo # Only these projects can trigger capture
Ensure the Slack bot is installed and SLACK_BOT_TOKEN is configured. The bot needs channels:history and users:read OAuth scopes. For @mention support, also enable Event Subscriptions and add the app_mentions:read scope. See the Slack integration guide for full setup steps.
Use the message shortcut (right-click → Shortcuts → Capture to DocBrain), type @DocBrain capture in the thread, or run /docbrain capture (outside threads only)
DocBrain fetches all messages, resolves user names, and indexes the conversation
Within ~15 seconds, DocBrain posts back in the thread:
✅ Thread from #platform-incidents captured into DocBrain (8 chunks indexed).
It's now searchable and will be used by Autopilot's next gap analysis.
By default, any user in any channel can trigger capture (via shortcut, @mention, or slash command). Restrict access with:
SLACK_CAPTURE_ALLOWED_CHANNELS=platform-team,infra-review # channel names (no #) or IDs
SLACK_CAPTURE_ALLOWED_USERS=alice,U01234567 # usernames or user IDs
Channel check matches against both channel_name and channel_id
User check matches against both user_name and user_id
If rejected, DocBrain responds with an ephemeral message: ⚠️ You don't have permission to use /docbrain capture in this channel.
A capture is served the moment it is written; nothing waits for an admin. Every capture says who wrote
its words and what checks it, and the console's Captures page shows it:
Two writers. A person (from the IDE through the skill, the CLI, the API or a /capture command):
Recorded by . DocBrain itself, summarising a thread, a pull request or a CI run:
Summarised by DocBrain from , with the person who asked for it credited as its capturer.
Three labels, computed when it is read: Checked against code (a premise it declares holds),
Code it rests on changed (a premise is broken: answers that cite it carry a warning and its author
is told once), Not checked against code (nothing checks it, and every reader can see that).
Its author manages it. Edit (a person-written capture, while it is served), Withdraw (at any
time, with a reason shown with it; it leaves answers at once), Delete (only until an answer has used
it). An admin can Purge it everywhere, or withdraw every capture of one author with a reason.
Limits. One author may write fragments.max_per_author_per_hour captures an hour (default 60).
The same text from the same author within 24 hours is refused with the existing capture's id, so the
agent revises it (docbrain_revise) instead of writing it twice.
Where it helped — the people it helped this month, the open questions it closed, and what the
questions were about — is shown to its author only, on the capture's page. Counts are of people,
never of answers, and never name who asked.
Captured content is stored under a meaningful space name derived from the source:
Source
Space assigned
GitHub PR/issue
owner/repo (e.g., myorg/backend)
GitLab MR
group/project (e.g., platform/api)
Slack thread
Channel name (e.g., platform-incidents)
This means allowed_spaces ACL filtering works as expected — setting allowed_spaces: ["platform-incidents"] on an API key will correctly scope answers to Slack captures from that channel, GitHub captures from a matching repo, etc.
Unlike incident records (Jira, PagerDuty, Zendesk), which are permanent historical events, captured content decays with age. A GitHub PR discussing an architecture from 5 years ago, or a Slack thread about a since-replaced system, should score low in freshness — not be treated as always-current.
The freshness scorer uses the original content creation date (when the PR/MR was opened, when the Slack thread started) as the age baseline — not the time DocBrain captured it.
Captures age through the standard time-decay curve: a 2-year-old architectural discussion will score significantly lower freshness than a recent one. What that score does depends on whether a reranker is configured:
No reranker (RAG_RERANK_PROVIDER=none, the default): freshness also scales each result's retrieval score, so an older capture ranks below a newer one of similar relevance.
With a reranker (any other provider): retrieval ranks on relevance alone, so an old capture that answers the question is still picked. Its age is not hidden: a source whose freshness has fallen into the stale or needs-review band is labelled that way to the model writing the answer.
Autopilot reads the same score: it looks at the stalest documents first when it searches for pages to fix, and breaks a tie between two candidate pages in favour of the fresher one.
Re-capturing the same thread (via message shortcut, @DocBrain capture, or @docbrain capture on a PR) updates the content but preserves the original creation date as the age baseline.
Either way, outdated design decisions, replaced architectures and deprecated processes are never deleted — the historical record stays available to explicit search — and the model writing an answer from a stale one is told that it is stale.
If you change your embedding provider (e.g., from Ollama to OpenAI), you need a full re-index because embedding dimensions differ between providers. Delete the OpenSearch index and re-ingest:
# Delete the existing indexcurl-XDELETEhttp://localhost:9200/docbrain_chunks
# Re-ingest everythingdockercomposeexecserverdocbrain-ingest
DocBrain supports ingesting from multiple providers simultaneously. Simply
declare each under sources: — enablement is structural (a sub-source runs
if its block is present in YAML). There is no INGEST_SOURCES env var.