zotero-kb/docs/superpowers/plans/2026-04-15-zotero-collection-import.md

Zotero Collection Import Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Replace the current left-panel import widgets with a Zotero collection tree importer that supports descendant-aware browsing, remembered checkbox state, and batch import into the active project.

Architecture: Extend the SQLite-backed ZoteroReader with collection-tree and collection-item queries, expose those through dedicated API endpoints, and replace the current left-panel import UI with a tree-plus-item-list importer that keeps selection state in the browser session. Reuse the existing POST /api/projects/{project_id}/imports/item-keys endpoint as the import execution path.

Tech Stack: Python 3.10, FastAPI, sqlite3, vanilla HTML/CSS/JavaScript, pytest, uv

---

File Structure

• Modify: src/zotero_kb/zotero_reader.py Purpose: build collection tree metadata and collection-scoped item listings with descendant inclusion.
• Modify: src/zotero_kb/api.py Purpose: expose collection tree and collection item endpoints.
• Modify: src/zotero_kb/templates/index.html Purpose: replace the left-panel import UI with a collection tree importer and batch selection workflow.
• Modify: tests/test_zotero_reader.py Purpose: cover collection tree generation and descendant-aware item listing.
• Modify: tests/test_api.py Purpose: cover collection tree and collection items endpoints plus batch import path.
• Modify: tests/test_ui.py Purpose: ensure the index template includes the new collection importer controls.
• Modify: README.md Purpose: document the new left-panel import workflow.

Task 1: Add Collection Tree Reader Support

Files:

• Modify: src/zotero_kb/zotero_reader.py

• Modify: tests/test_zotero_reader.py

• Step 1: Write the failing reader tests

def test_build_collection_tree_returns_descendant_counts(tmp_path: Path) -> None:
    fixture_dir = tmp_path / "zotero"
    fixture_dir.mkdir()
    build_fixture_zotero_dir(fixture_dir)
    reader = ZoteroReader(fixture_dir)

    tree = reader.get_collection_tree()

    assert len(tree) == 1
    root = tree[0]
    assert root["name"] == "Theory"
    assert root["direct_item_count"] == 0
    assert root["descendant_item_count"] == 1
    assert root["children"][0]["name"] == "Drafting"


def test_get_collection_items_includes_descendants(tmp_path: Path) -> None:
    fixture_dir = tmp_path / "zotero"
    fixture_dir.mkdir()
    build_fixture_zotero_dir(fixture_dir)
    reader = ZoteroReader(fixture_dir)

    items = reader.get_collection_items("COLL0001", include_descendants=True)

    assert [item["item_key"] for item in items] == ["PAPER0001"]
    assert items[0]["collection_paths"] == [["Theory", "Drafting"]]

• Step 2: Run test to verify it fails

Run: UV_CACHE_DIR=/tmp/uv-cache uv run pytest tests/test_zotero_reader.py::test_build_collection_tree_returns_descendant_counts tests/test_zotero_reader.py::test_get_collection_items_includes_descendants -q Expected: FAIL with AttributeError because get_collection_tree and get_collection_items do not exist.

• Step 3: Write minimal implementation

def get_collection_tree(self) -> list[dict[str, object]]:
    conn = sqlite3.connect(self.db_path)
    conn.row_factory = sqlite3.Row
    try:
        collections = self._read_collection_lookup(conn)
        direct_counts = self._read_direct_item_counts(conn)
        children_lookup = self._build_children_lookup(collections)
        return [
            self._build_collection_node(collection_id, collections, children_lookup, direct_counts)
            for collection_id, row in collections.items()
            if row["parentCollectionID"] is None
        ]
    finally:
        conn.close()


def get_collection_items(self, collection_key: str, include_descendants: bool = True) -> list[dict[str, object]]:
    conn = sqlite3.connect(self.db_path)
    conn.row_factory = sqlite3.Row
    try:
        field_lookup = self._read_field_lookup(conn)
        collections = self._read_collection_lookup(conn)
        children_lookup = self._build_children_lookup(collections)
        collection_ids = self._resolve_collection_ids(collection_key, collections, children_lookup, include_descendants)
        item_ids = self._read_item_ids_for_collections(conn, collection_ids)
        return self._read_item_summaries(conn, item_ids, field_lookup, collections)
    finally:
        conn.close()

def _build_collection_node(
    self,
    collection_id: int,
    collections: dict[int, sqlite3.Row],
    children_lookup: dict[int, list[int]],
    direct_counts: dict[int, int],
) -> dict[str, object]:
    children = [
        self._build_collection_node(child_id, collections, children_lookup, direct_counts)
        for child_id in children_lookup.get(collection_id, [])
    ]
    descendant_count = direct_counts.get(collection_id, 0) + sum(
        int(child["descendant_item_count"]) for child in children
    )
    row = collections[collection_id]
    return {
        "collection_key": str(row["key"]),
        "name": str(row["collectionName"]),
        "parent_key": self._parent_key(row["parentCollectionID"], collections),
        "children": children,
        "direct_item_count": direct_counts.get(collection_id, 0),
        "descendant_item_count": descendant_count,
    }

• Step 4: Run test to verify it passes

Run: UV_CACHE_DIR=/tmp/uv-cache uv run pytest tests/test_zotero_reader.py::test_build_collection_tree_returns_descendant_counts tests/test_zotero_reader.py::test_get_collection_items_includes_descendants -q Expected: PASS

• Step 5: Commit

git add src/zotero_kb/zotero_reader.py tests/test_zotero_reader.py
git commit -m "feat: add zotero collection tree reader support"

Task 2: Expose Collection Import API

Files:

• Modify: src/zotero_kb/api.py

• Modify: tests/test_api.py

• Step 1: Write the failing API tests

def test_collection_tree_endpoint(tmp_path: Path) -> None:
    app = create_app(make_test_config(tmp_path), llm_client=FakeLlmClient())
    endpoint = _route(app, "/api/zotero/collections/tree", "GET")

    payload = endpoint()

    assert payload["collections"][0]["collection_key"] == "COLL0001"
    assert payload["collections"][0]["children"][0]["collection_key"] == "COLL0002"


def test_collection_items_endpoint_includes_descendants(tmp_path: Path) -> None:
    app = create_app(make_test_config(tmp_path), llm_client=FakeLlmClient())
    endpoint = _route(app, "/api/zotero/collections/{collection_key}/items", "GET")

    payload = endpoint("COLL0001", True)

    assert payload["items"][0]["item_key"] == "PAPER0001"

• Step 2: Run test to verify it fails

Run: UV_CACHE_DIR=/tmp/uv-cache uv run pytest tests/test_api.py::test_collection_tree_endpoint tests/test_api.py::test_collection_items_endpoint_includes_descendants -q Expected: FAIL because the routes are missing.

• Step 3: Write minimal implementation

@app.get("/api/zotero/collections/tree")
def zotero_collection_tree() -> dict[str, object]:
    return {"collections": reader.get_collection_tree()}


@app.get("/api/zotero/collections/{collection_key}/items")
def zotero_collection_items(collection_key: str, include_descendants: bool = True) -> dict[str, object]:
    return {
        "collection_key": collection_key,
        "items": reader.get_collection_items(collection_key, include_descendants=include_descendants),
    }

• Step 4: Run test to verify it passes

Run: UV_CACHE_DIR=/tmp/uv-cache uv run pytest tests/test_api.py::test_collection_tree_endpoint tests/test_api.py::test_collection_items_endpoint_includes_descendants -q Expected: PASS

• Step 5: Commit

git add src/zotero_kb/api.py tests/test_api.py
git commit -m "feat: expose zotero collection import api"

Task 3: Replace Left-Panel Import UI

Files:

• Modify: src/zotero_kb/templates/index.html

• Modify: tests/test_ui.py

• Step 1: Write the failing UI test

def test_index_contains_collection_import_controls(tmp_path) -> None:
    app = create_app(
        AppConfig(
            workspace_dir=tmp_path / "workspace",
            zotero_data_dir=tmp_path / "zotero",
            bridge_file=tmp_path / "bridge.json",
        )
    )

    endpoint = _route(app, "/", "GET")
    html = endpoint()

    assert 'id="collection-tree"' in html
    assert 'id="collection-items"' in html
    assert 'id="select-descendants-button"' in html
    assert 'id="clear-selection-button"' in html
    assert 'id="import-selected-items-button"' in html

• Step 2: Run test to verify it fails

Run: UV_CACHE_DIR=/tmp/uv-cache uv run pytest tests/test_ui.py::test_index_contains_collection_import_controls -q Expected: FAIL because the template still contains import-selected-button and zotero-search-form instead of the new collection importer ids.

• Step 3: Write minimal implementation

<div class="section surface">
  <h3>从 Zotero 导入</h3>
  <div id="collection-tree" class="tree"></div>
  <div id="collection-items" class="result-list"></div>
  <div class="action-bar">
    <span id="selected-count">已选 0 篇</span>
    <button id="select-descendants-button" type="button" class="secondary">全选当前目录及子目录</button>
    <button id="clear-selection-button" type="button" class="danger">清空选择</button>
    <button id="import-selected-items-button" type="button">导入所选到当前项目</button>
  </div>
</div>

state.collectionTree = [];
state.selectedCollectionKey = null;
state.selectedItemKeys = new Set();
state.expandedCollectionKeys = new Set();
state.visibleCollectionItems = [];

async function loadCollectionTree() {
  const payload = await api("/api/zotero/collections/tree");
  state.collectionTree = payload.collections || [];
  renderCollectionTree();
}

async function selectCollection(collectionKey) {
  state.selectedCollectionKey = collectionKey;
  const payload = await api(`/api/zotero/collections/${collectionKey}/items?include_descendants=true`);
  state.visibleCollectionItems = payload.items || [];
  renderCollectionItems();
}

function toggleItemSelection(itemKey, checked) {
  if (checked) state.selectedItemKeys.add(itemKey);
  else state.selectedItemKeys.delete(itemKey);
  renderSelectedCount();
}

• Step 4: Run test to verify it passes

Run: UV_CACHE_DIR=/tmp/uv-cache uv run pytest tests/test_ui.py::test_index_contains_collection_import_controls -q Expected: PASS

• Step 5: Commit

git add src/zotero_kb/templates/index.html tests/test_ui.py
git commit -m "feat: add zotero collection tree importer ui"

Task 4: Wire Batch Import And Refresh

Files:

• Modify: src/zotero_kb/templates/index.html

• Modify: tests/test_api.py

• Modify: README.md

• Step 1: Write the failing behavior test

def test_import_item_keys_endpoint_returns_project_view(tmp_path: Path) -> None:
    config = make_test_config(tmp_path)
    app = create_app(config, llm_client=FakeLlmClient())
    create_project = _route(app, "/api/projects", "POST")
    import_keys = _route(app, "/api/projects/{project_id}/imports/item-keys", "POST")

    create_project(
        CreateProjectRequest(
            project_id="thesis-ch2",
            name="Thesis Chapter 2",
            llm_provider="deepseek",
            llm_model="deepseek-chat",
        )
    )

    payload = import_keys("thesis-ch2", {"item_keys": ["PAPER0001"]})

    assert payload["project_view"]["cards"][0]["item_key"] == "PAPER0001"

• Step 2: Run test to verify it fails

Run: UV_CACHE_DIR=/tmp/uv-cache uv run pytest tests/test_api.py::test_import_item_keys_endpoint_returns_project_view -q Expected: FAIL only if the endpoint response shape or refreshed project view is wrong after the UI refactor.

• Step 3: Write minimal implementation

document.getElementById("select-descendants-button").addEventListener("click", () => {
  for (const item of state.visibleCollectionItems) {
    state.selectedItemKeys.add(item.item_key);
  }
  renderCollectionItems();
});

document.getElementById("clear-selection-button").addEventListener("click", () => {
  state.selectedItemKeys.clear();
  renderCollectionItems();
});

document.getElementById("import-selected-items-button").addEventListener("click", async () => {
  const payload = await api(`/api/projects/${state.currentProjectId}/imports/item-keys`, {
    method: "POST",
    body: JSON.stringify({ item_keys: Array.from(state.selectedItemKeys) }),
  });
  state.selectedItemKeys.clear();
  renderCards(payload.project_view.cards || []);
  renderCollectionItems();
});

## 从 Zotero 导入

1. 选择一个项目
2. 在左栏 `从 Zotero 导入` 中展开 collection 树
3. 点击一个目录，系统会加载该目录及其子目录的文献
4. 勾选单篇文献，或点击 `全选当前目录及子目录`
5. 点击 `导入所选到当前项目`

• Step 4: Run test to verify it passes

Run: UV_CACHE_DIR=/tmp/uv-cache uv run pytest tests/test_api.py::test_import_item_keys_endpoint_returns_project_view -q Expected: PASS

• Step 5: Run full verification

Run: UV_CACHE_DIR=/tmp/uv-cache uv run pytest -q Expected: PASS with all tests green

• Step 6: Commit

git add src/zotero_kb/templates/index.html tests/test_api.py README.md
git commit -m "feat: support batch import from zotero collections"