The Feedback Loop

Why Feedback Matters More Than Publishing

Most knowledge bases die not because they lack content, but because they lack a correction mechanism. Someone notices an article is wrong — outdated API, changed process, incorrect claim — and there’s no low-friction way to say so. They mention it in Slack, maybe. The author is busy. The article stays wrong. Trust erodes. This is the core failure mode of wikis: the effort to correct is high (find the page, edit, save, hope you don’t break formatting), the reward is invisible, and the original author may never know it happened.

Cairns addresses this with a two-part strategy: an inline annotation system that makes feedback effortless for readers, and an agent workflow that makes corrections fast for the maintainer.

graph LR
    R[Reader spots error] --> A[Highlights text]
    A --> C[Adds comment]
    C --> I[Creates GitHub Issue]
    I --> AG[Agent triages]
    AG --> V{Verified?}
    V -->|Yes| F[Agent fixes article]
    V -->|No| D[Agent replies with context]

How Inline Annotations Work

The annotation system is opt-in — it loads only when annotations.repo is configured in src/_data/site.json. When enabled, every article gets an unobtrusive annotation layer with no extra UI visible until the reader takes action.

The Reader Experience

1. Select text — highlight any passage in the article body
2. Click “Add Note” — a floating toolbar appears near the selection
3. Write a comment — “This API was deprecated in v3” or “The diagram shows the old architecture”
4. Repeat — annotations accumulate in a tray at the bottom of the page, stored in localStorage
5. Submit — click “Create Issue” to bundle all annotations into a single, well-formatted GitHub Issue

The result is a GitHub Issue that looks like this:

Each annotation includes the selected text (as a blockquote), the section heading (as a deep link), and the reader’s comment. The content-feedback label is applied automatically so the agent can filter for these issues.

Configuration

Enabling annotations requires two fields in src/_data/site.json:

{
  "title": "Cairns",
  "description": "Agent-powered knowledge trail system",
  "url": "https://your-cairns-site.example.com",
  "annotations": {
    "repo": "your-org/your-cairns-repo"
  }
}

That’s it. The annotation CSS and JavaScript load conditionally — when site.annotations is not set, zero annotation code is included in the build. No extra dependencies, no external services, no database. Annotations are stored in localStorage, scoped to the article’s permalink. They persist across page reloads but are private to the reader’s browser until explicitly submitted as a GitHub Issue. No data is sent anywhere until the reader clicks “Create Issue.”

The Agent Triage Workflow

Feedback without triage is just noise. The annotation system is designed to pair with an agent that monitors the content-feedback label on GitHub Issues and processes them autonomously.

What the Agent Does

When a content-feedback issue appears:

1. Parse the annotations — extract the selected text, section anchors, and comments
2. Verify against sources — check the claim against the actual codebase, docs, or external systems the article references
3. Decide the action:
   • Correction confirmed — update the article, commit, close the issue with a diff link
   • Already accurate — reply with evidence, close as “not applicable”
   • Needs human judgment — add a comment with findings, assign a reviewer

The key insight is that the agent doesn’t just trust the feedback — it verifies against the same sources it used to write the article. This prevents well-meaning but incorrect corrections from degrading content quality.

Scheduling Triage

The triage workflow can run on different triggers:

For most teams, a daily cron is sufficient. The agent queries for open issues with the content-feedback label, processes them in order, and reports a summary to the team channel.

The Full Correction Cycle

Putting it all together, the feedback loop creates a continuous improvement cycle:

graph TD
    P[Agent publishes article] --> R[Team reads]
    R --> F[Reader annotates]
    F --> I[GitHub Issue created]
    I --> T[Agent triages]
    T --> V{Verified?}
    V -->|Yes| U[Agent updates article]
    V -->|Ambiguous| H[Escalate to human]
    V -->|Incorrect| C[Agent replies, closes]
    U --> P
    H --> U

This cycle means articles get better over time without anyone owning a “documentation review” process. The readers are the reviewers. The agent is the editor. The GitHub Issue is the paper trail.

What Makes This Different

Traditional feedback mechanisms (wiki edit buttons, Google Docs comments, Slack threads) share a common failure: they put the burden of making the correction on the person who noticed the problem. Cairns inverts this:

1. Reader effort is minimal — highlight, comment, submit. No editing, no formatting, no finding the right file.
2. Context is preserved — the exact text, the section anchor, and the reader's interpretation travel together in a structured format.
3. Verification is automatic — the agent checks claims against sources before making changes, preventing well-intentioned but wrong corrections.
4. The paper trail is built-in — every correction is a GitHub Issue with a commit link. Auditable, searchable, attributable.

Trust Boundaries Are Yours to Define

Cairns is a framework, not a managed service. Every deployment makes its own decisions about what the agent can access, who can direct it, and what it’s allowed to publish. This applies to the feedback loop, on-demand content requests, and the source material the agent reads — internal repos, Notion workspaces, Slack history, external APIs, all of it.

Cairns does not prescribe trust boundaries. The deployer is responsible for:

• Source access — which repos, docs, and systems the agent can read from
• Input authority — who can request new content or corrections, and through what channels
• Publication controls — whether the agent commits directly to main or goes through PR review
• Content scope — what topics are in bounds and what material is too sensitive to synthesize

There are good practices — scoped API keys, read-only defaults, human review gates, content sandboxing — and we’ve developed in-depth guidance on LLM security for our own deployments. But the right configuration depends entirely on your threat model, your data sensitivity, and your team’s risk tolerance. Cairns gives you the levers. Pulling them is your job.

Discussion Prompts

• What's the current path for someone on your team to correct a wrong piece of documentation? How many steps does it take?
• Would your team be more likely to give feedback through inline annotations or through a Slack command like "@agent this article is wrong about X"?
• What are the trust boundaries for your deployment? Which sources should the agent read, and who should be allowed to direct it?

References & Further Reading

1. GitHub Issues — The underlying mechanism for annotation submission. Issues support labels, assignees, and deep linking.
2. localStorage API — Client-side storage used for annotations. Private to the reader's browser, no server required.
3. Selection API — The browser API that powers text selection and range extraction for the annotation toolbar.
4. Eleventy Data Cascade — How site.json configuration flows into templates, enabling conditional annotation loading.