zotero-kb/docs/superpowers/specs/2026-04-15-zotero-kb-design.md

Zotero KB Design

Overview

This project builds a local Zotero-to-card pipeline for agent-friendly research workflows. It reads a local Zotero data directory, converts selected literature into lightweight knowledge cards, supports project-scoped writing assistance, and prepares a file-based integration path for Claude Code / Codex via SKILL.

The first implementation pass prioritizes a local API + Web service, not a Zotero-native product. Zotero remains the source of truth for bibliographic data and attachments. The generated card library becomes the reusable asset layer for humans and agents.

Goals

• Read Zotero data directly from a local data directory such as /mnt/c/Users/WSX/Zotero
• Support importing the literature currently selected in Zotero into the local system
• Preserve Zotero collection hierarchy in the generated library structure
• Convert literature into lightweight Markdown knowledge cards with a JSON index
• Support project creation and project-scoped literature management
• Support adding and removing literature from a project at any time
• Support two independent writing-mode actions:
  • citation candidate recommendation
  • first-pass citation plan generation
• Prepare a file-based SKILL integration so Claude Code / Codex can read only the content of a specified project

Non-Goals For V1

• Full knowledge-base mode UI or workflows across the entire library
• General-purpose chat over all literature
• Final paper text generation
• Editing Zotero data in place
• Replacing Zotero as the bibliographic source of truth

Knowledge-base mode is intentionally deferred. V1 only reserves the underlying global library structure and API surface needed for future implementation.

Context And Constraints

Local Zotero Access

The user confirmed that the Zotero data directory is available locally and contains both zotero.sqlite and storage/.

V1 uses a mixed ingestion model:

• Read bibliographic data, notes, tags, collections, and attachments from the local Zotero data directory
• Use a minimal Zotero bridge only to retrieve the keys of the items currently selected in the Zotero UI

This split is required because direct filesystem access can read the library contents but cannot reliably observe Zotero's current UI selection state.

Reference Projects

• zotero-rag is the main implementation reference for local web service structure, attachment extraction, and search-oriented pipeline composition
• zotcard is the main reference for the Zotero-side bridge pattern and selected-item access inside the Zotero desktop runtime

The new system is intentionally not identical to either:

• unlike zotero-rag, the durable asset is a card library rather than only a retrieval index
• unlike zotcard, the primary product surface is a local service and file-based workspace rather than a Zotero-first note plugin

Product Model

The system has two storage scopes with different purposes.

1. Global Library Scope

This stores the full set of imported literature and generated cards across all imports. It is the foundation for the future knowledge-base mode.

Responsibilities:

• maintain the canonical local card library
• preserve collection hierarchy across all imported items
• store normalized item metadata and card indexes
• avoid regenerating duplicate cards for the same Zotero item unless the source content changes

2. Project Scope

Projects are scoped working sets built on top of the global library. A project contains references to selected literature rather than owning the canonical copy of every card.

Responsibilities:

• define which literature belongs to the project
• support adding and removing selected items at any time
• provide a project-scoped view for writing assistance
• constrain Claude Code / Codex to read only the project's content

This split avoids redundant card copies while preserving future flexibility for project-specific views and exports.

Architecture

The service is divided into five units.

1. Zotero Bridge

Purpose:

• ask Zotero for the currently selected item keys

Characteristics:

• minimal surface area
• no card logic
• no library parsing
• no knowledge of projects

The bridge can be a small Zotero plugin or local bridge script embedded into a plugin package. Its only required output is a list of selected item_key values.

2. Zotero Reader

Purpose:

• read local Zotero data from zotero.sqlite and storage/
• load item metadata, creators, abstract, tags, notes, collections, and attachments
• extract attachment text for supported files

Characteristics:

• local-data-first
• deterministic
• independent from UI

3. Card Builder

Purpose:

• normalize raw Zotero material into a source bundle
• call a configurable LLM to transform source material into an agent-readable card
• write Markdown cards and update JSON indexes

Characteristics:

• one card per Zotero regular item
• source-aware hash for rebuild detection
• card schema stable enough for direct SKILL consumption

4. Project Manager

Purpose:

• create projects
• track which Zotero items belong to which project
• expose project-scoped listings and stats
• allow project item removal without affecting the global library

5. Writing Service

Purpose:

• run project-scoped writing assistance over project cards

V1 includes two independent actions:

• recommend citation candidates
• generate a first-pass citation plan

These actions consume project-scoped card views and never read outside the target project.

Directory Layout

The workspace uses a shared library plus project views.

workspace/
  library/
    collections/
      Theory/
        Subtopic/
          Paper A [ABCD1234].md
    index/
      items.json
      cards.json
      collections.json
    cache/
      source-bundles/
        ABCD1234.json
  projects/
    thesis-ch2/
      project.json
      selected-items.json
      project-index.json

Library Scope Files

• library/collections/: canonical Markdown cards arranged to mirror Zotero collection hierarchy
• library/index/items.json: normalized item metadata keyed by Zotero item key
• library/index/cards.json: card metadata used for filtering, retrieval, and writing-mode preparation
• library/index/collections.json: normalized collection tree and item membership
• library/cache/source-bundles/: raw normalized source material captured before LLM card generation

Project Scope Files

• projects/<project-id>/project.json: project metadata and model configuration
• projects/<project-id>/selected-items.json: ordered set of Zotero item keys assigned to the project
• projects/<project-id>/project-index.json: precomputed project view for fast loading and SKILL consumption

Projects do not need to duplicate every canonical card. The project index maps selected items to canonical card paths in the global library.

Data Model

Project Metadata

Example:

{
  "id": "thesis-ch2",
  "name": "Thesis Chapter 2",
  "zotero_data_dir": "/mnt/c/Users/WSX/Zotero",
  "selection_mode": "zotero-bridge",
  "llm": {
    "provider": "openai",
    "model": "gpt-5-mini",
    "base_url": null
  },
  "created_at": "2026-04-15T00:00:00Z"
}

Item Index Entry

library/index/items.json stores one normalized entry per Zotero item.

Required fields:

• item_key
• title
• creators
• year
• item_type
• abstract
• tags
• collection_paths
• note_ids
• attachment_keys
• attachment_status
• card_path
• source_hash
• updated_at

Card Index Entry

library/index/cards.json stores fields optimized for reading and writing assistance.

Required fields:

• item_key
• card_path
• title
• summary
• keywords
• claims
• quotable_spans
• writing_hints
• updated_at

Collection Index Entry

library/index/collections.json stores normalized collection information.

Required fields:

• collection_key
• name
• parent_key
• path
• item_keys

Card Format

Each regular Zotero item generates one Markdown card in the canonical library.

Example:

---
item_key: ABCD1234
title: Retrieval-Augmented Writing in Humanities Research
year: 2024
collections:
  - Theory/Subtopic
authors:
  - Alice Smith
  - Bob Li
tags:
  - llm
  - retrieval
attachment_status: ok
source_hash: "sha256:6f6f5d9de1b5a9b98d7ce1c76d7a8d5c441d72e3"
---

# Summary
This paper argues that card-oriented retrieval pipelines improve citation-grounded drafting when notes, metadata, and attachment text are merged before synthesis.

# Core Claims
- Combined metadata, notes, and full-text evidence produce more reliable citation suggestions than metadata-only indexing.
- Project-scoped reading reduces irrelevant retrieval during drafting.

# Methods
- Compares card-generation inputs across three corpus preparation strategies.
- Evaluates writing-support quality with citation recommendation tasks.

# Evidence
- Reports higher citation precision for the merged-input pipeline.
- Includes examples where note-only systems miss relevant methodological context.

# Quotable Passages
- p.12: "Project-scoped card retrieval improves citation precision during drafting."
- Note: Author stresses that project-bounded retrieval lowers topical drift.

# Writing Hints
- Useful for defining the problem
- Useful as a supporting citation for claims about scoped retrieval during drafting

The card format is intentionally designed for:

• direct human reading
• direct SKILL consumption
• predictable parsing by the Web service

Import Flow

Selected-Items Import

1. The user selects multiple items in Zotero
2. The Zotero bridge returns the selected item_key values
3. The service reads the local Zotero database and storage directory
4. The reader collects metadata, notes, tags, collection paths, and attachment text
5. The builder creates or updates source bundles
6. The builder generates or refreshes Markdown cards and indexes
7. The project manager adds the selected item keys to the target project
8. The project view is rebuilt so Web and SKILL consumers can read the updated subset

Add / Remove Semantics

• Adding is upsert by item_key
• A Zotero item can belong to multiple projects
• Removing an item from a project only updates that project
• Removing an item from a project never deletes it from Zotero
• The canonical card remains in the global library unless explicit future garbage collection is added

LLM Configuration

Card generation is model-driven and configurable via API at the project level.

V1 requirements:

• support configurable provider and model fields
• support prompt templates for card generation
• keep the source bundle on disk before generation
• treat LLM output as structured content that is converted into canonical Markdown sections

The service should not hardcode a single provider design. It should expose a provider abstraction with one stable card-generation contract.

Modes

Writing Mode

Writing mode is project-scoped and exposes two independent actions, each implemented as a dedicated skill.

Skill 1: Citation Recommendation

Skill name: zotero-citation-recommender

Purpose: Given a writing intent or draft paragraph, recommend relevant literature from the project scope.

Input:

• writing intent text or draft paragraph

Output:

• structured list of recommended literature
• why each item is relevant
• card-derived claims or quotable spans
• suggested rhetorical role such as:
  • definition
  • supporting evidence
  • contrast
  • limitation

API endpoint: POST /api/projects/{project_id}/writing/recommend-citations

---

Skill 2: Citation Plan Generation

Skill name: zotero-citation-planner

Purpose: Generate a first-pass citation plan for a writing intent, suggesting paragraph structure, citation order, and the role of each cited work.

Input:

• writing intent text
• optional stance
• optional paragraph goal
• optional length expectation

Output:

• paragraph-level citation plan
• suggested sub-structure
• citation order
• recommended role for each cited work
• notes about which claims should be grounded by which items

This skill generates a plan, not polished final prose.

API endpoint: POST /api/projects/{project_id}/writing/generate-plan

Knowledge-Base Mode

Knowledge-base mode is reserved for future implementation and is not part of V1 delivery.

V1 only reserves:

• the global library storage model
• future-facing API stubs
• indexes suitable for full-library browsing and retrieval later

API Surface

The initial service should expose the following HTTP endpoints.

Project Endpoints

• POST /api/projects
  • create a project
• GET /api/projects
  • list projects
• GET /api/projects/{project_id}
  • get project detail and counts

Import Endpoints

• POST /api/projects/{project_id}/imports/selected-items
  • read selected item keys from the Zotero bridge
  • ingest source data from the local Zotero directory
  • generate or update cards
  • add the imported items to the project

Project Item Endpoints

• GET /api/projects/{project_id}/items
  • list project items
• DELETE /api/projects/{project_id}/items/{item_key}
  • remove an item from the project

Card Endpoints

• GET /api/projects/{project_id}/cards
  • list cards visible to the project
• POST /api/projects/{project_id}/cards/rebuild
  • rebuild all cards in the project scope
• POST /api/projects/{project_id}/cards/{item_key}/rebuild
  • rebuild one card

Writing Endpoints

• POST /api/projects/{project_id}/writing/recommend-citations
  • return project-scoped citation candidates
• POST /api/projects/{project_id}/writing/generate-plan
  • return a project-scoped first-pass citation plan

Future Knowledge-Base Endpoints

These routes are reserved in V1:

• GET /api/library/cards
• GET /api/library/collections
• POST /api/library/search

Web UI

The Web UI is an operational console, not the primary agent interface.

Left Panel

• project switcher
• create project action
• import selected Zotero items action
• project collection tree

Center Panel

• project card list
• filters by title, author, tag, year, and keyword
• card detail viewer
• remove-from-project action
• rebuild-card action

Right Panel

• writing mode tabs
  • recommendation
  • citation plan
• structured input form
• structured output display

The UI should remain intentionally small. It exists to manage and inspect the workspace rather than replace the card files as the main knowledge surface.

Error Handling

The system should degrade gracefully and preserve intermediate assets where possible.

Bridge Failure

• if the Zotero bridge cannot read current selection, return an explicit import error
• do not affect existing projects or cards

Attachment Failure

• if attachment text extraction fails, still build a partial card from available metadata, abstract, notes, and tags
• mark attachment_status as missing or partial

LLM Failure

• keep the source bundle on disk
• mark the item as card generation failed
• allow single-item or project-level retry

Source Drift

• detect changes through source_hash
• mark cards stale when Zotero source content changes
• surface rebuild-needed state in project views

Testing Strategy

V1 should focus on deterministic tests around boundaries rather than broad UI snapshots.

SQLite Reader Tests

• parse items, creators, notes, tags, and collection relationships from fixture data
• resolve attachment mappings from the Zotero schema used by the local directory

Project Management Tests

• create projects
• add selected items
• deduplicate repeated imports
• remove project items without mutating the canonical library

Card Builder Tests

• normalize source bundles
• verify prompt input assembly
• stub model output
• verify Markdown card writing and JSON index updates
• verify stale detection using source_hash

Writing Service Tests

• citation recommendation returns the expected structured shape
• citation plan generation returns the expected structured plan shape
• both actions remain project-scoped

API Tests

• validate project creation, import, removal, rebuild, and writing endpoints

SKILL Integration

The agent integration is file-based. Claude Code / Codex should read only the selected project's content.

Two writing-mode skills are provided:

• skills/zotero-citation-recommender/SKILL.md — handles citation candidate recommendation
• skills/zotero-citation-planner/SKILL.md — handles first-pass citation plan generation

A project reader skill is also provided:

• skills/zotero-project-reader/SKILL.md — core project and card reading

Responsibilities of each skill:

zotero-project-reader:

• read projects/<project-id>/project.json
• read selected-items.json and project-index.json
• load only the canonical cards referenced by the project

zotero-citation-recommender:

• read project-scoped cards
• accept writing intent or draft paragraph as input
• return structured citation recommendations with relevance rationale, claims, and rhetorical roles

zotero-citation-planner:

• read project-scoped cards
• accept writing intent and optional stance/goal/length as input
• return paragraph-level citation plan with structure, order, and per-work roles

Each skill must not access the entire global library unless the target project explicitly references those items.

Implementation Priorities For V1

1. Build the local workspace structure and project model
2. Implement Zotero local-data reading from zotero.sqlite and storage/
3. Implement the minimal Zotero selected-items bridge
4. Build source bundle normalization and card generation
5. Expose project import and item removal APIs
6. Expose project-scoped writing endpoints
7. Build the minimal Web console
8. Add file-based SKILL integration

Open Decisions Resolved

The following decisions were explicitly settled during brainstorming:

• service entrypoint is API + Web
• Claude Code / Codex integration is file-based, not API-driven
• card output format is Markdown cards + JSON index
• writing mode contains two independent actions
• knowledge-base mode is deferred, but its storage model is reserved now
• the data source is the local Zotero data directory
• current-selection import uses a small Zotero bridge for selection keys only
• collection hierarchy must mirror Zotero

Delivery Boundary

V1 is complete when the user can:

• create a project
• select multiple items in Zotero
• import the selected items into that project
• generate canonical Markdown cards from local Zotero data and attachments
• remove items from the project without affecting Zotero
• request citation candidates for a project-scoped writing intent
• request a first-pass citation plan for a project-scoped writing intent
• point Claude Code / Codex at a project and have it read only that project's cards