# Zotero Collection Import Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Replace the current left-panel import widgets with a Zotero collection tree importer that supports descendant-aware browsing, remembered checkbox state, and batch import into the active project.

**Architecture:** Extend the SQLite-backed `ZoteroReader` with collection-tree and collection-item queries, expose those through dedicated API endpoints, and replace the current left-panel import UI with a tree-plus-item-list importer that keeps selection state in the browser session. Reuse the existing `POST /api/projects/{project_id}/imports/item-keys` endpoint as the import execution path.

**Tech Stack:** Python 3.10, FastAPI, sqlite3, vanilla HTML/CSS/JavaScript, pytest, uv

---

## File Structure

- Modify: `src/zotero_kb/zotero_reader.py`
  Purpose: build collection tree metadata and collection-scoped item listings with descendant inclusion.
- Modify: `src/zotero_kb/api.py`
  Purpose: expose collection tree and collection item endpoints.
- Modify: `src/zotero_kb/templates/index.html`
  Purpose: replace the left-panel import UI with a collection tree importer and batch selection workflow.
- Modify: `tests/test_zotero_reader.py`
  Purpose: cover collection tree generation and descendant-aware item listing.
- Modify: `tests/test_api.py`
  Purpose: cover collection tree and collection items endpoints plus batch import path.
- Modify: `tests/test_ui.py`
  Purpose: ensure the index template includes the new collection importer controls.
- Modify: `README.md`
  Purpose: document the new left-panel import workflow.

### Task 1: Add Collection Tree Reader Support

**Files:**
- Modify: `src/zotero_kb/zotero_reader.py`
- Modify: `tests/test_zotero_reader.py`

- [ ] **Step 1: Write the failing reader tests**

```python
def test_build_collection_tree_returns_descendant_counts(tmp_path: Path) -> None:
    fixture_dir = tmp_path / "zotero"
    fixture_dir.mkdir()
    build_fixture_zotero_dir(fixture_dir)
    reader = ZoteroReader(fixture_dir)

    tree = reader.get_collection_tree()

    assert len(tree) == 1
    root = tree[0]
    assert root["name"] == "Theory"
    assert root["direct_item_count"] == 0
    assert root["descendant_item_count"] == 1
    assert root["children"][0]["name"] == "Drafting"


def test_get_collection_items_includes_descendants(tmp_path: Path) -> None:
    fixture_dir = tmp_path / "zotero"
    fixture_dir.mkdir()
    build_fixture_zotero_dir(fixture_dir)
    reader = ZoteroReader(fixture_dir)

    items = reader.get_collection_items("COLL0001", include_descendants=True)

    assert [item["item_key"] for item in items] == ["PAPER0001"]
    assert items[0]["collection_paths"] == [["Theory", "Drafting"]]
```

- [ ] **Step 2: Run test to verify it fails**

Run: `UV_CACHE_DIR=/tmp/uv-cache uv run pytest tests/test_zotero_reader.py::test_build_collection_tree_returns_descendant_counts tests/test_zotero_reader.py::test_get_collection_items_includes_descendants -q`
Expected: FAIL with `AttributeError` because `get_collection_tree` and `get_collection_items` do not exist.

- [ ] **Step 3: Write minimal implementation**

```python
def get_collection_tree(self) -> list[dict[str, object]]:
    conn = sqlite3.connect(self.db_path)
    conn.row_factory = sqlite3.Row
    try:
        collections = self._read_collection_lookup(conn)
        direct_counts = self._read_direct_item_counts(conn)
        children_lookup = self._build_children_lookup(collections)
        return [
            self._build_collection_node(collection_id, collections, children_lookup, direct_counts)
            for collection_id, row in collections.items()
            if row["parentCollectionID"] is None
        ]
    finally:
        conn.close()


def get_collection_items(self, collection_key: str, include_descendants: bool = True) -> list[dict[str, object]]:
    conn = sqlite3.connect(self.db_path)
    conn.row_factory = sqlite3.Row
    try:
        field_lookup = self._read_field_lookup(conn)
        collections = self._read_collection_lookup(conn)
        children_lookup = self._build_children_lookup(collections)
        collection_ids = self._resolve_collection_ids(collection_key, collections, children_lookup, include_descendants)
        item_ids = self._read_item_ids_for_collections(conn, collection_ids)
        return self._read_item_summaries(conn, item_ids, field_lookup, collections)
    finally:
        conn.close()
```

```python
def _build_collection_node(
    self,
    collection_id: int,
    collections: dict[int, sqlite3.Row],
    children_lookup: dict[int, list[int]],
    direct_counts: dict[int, int],
) -> dict[str, object]:
    children = [
        self._build_collection_node(child_id, collections, children_lookup, direct_counts)
        for child_id in children_lookup.get(collection_id, [])
    ]
    descendant_count = direct_counts.get(collection_id, 0) + sum(
        int(child["descendant_item_count"]) for child in children
    )
    row = collections[collection_id]
    return {
        "collection_key": str(row["key"]),
        "name": str(row["collectionName"]),
        "parent_key": self._parent_key(row["parentCollectionID"], collections),
        "children": children,
        "direct_item_count": direct_counts.get(collection_id, 0),
        "descendant_item_count": descendant_count,
    }
```

- [ ] **Step 4: Run test to verify it passes**

Run: `UV_CACHE_DIR=/tmp/uv-cache uv run pytest tests/test_zotero_reader.py::test_build_collection_tree_returns_descendant_counts tests/test_zotero_reader.py::test_get_collection_items_includes_descendants -q`
Expected: PASS

- [ ] **Step 5: Commit**

```bash
git add src/zotero_kb/zotero_reader.py tests/test_zotero_reader.py
git commit -m "feat: add zotero collection tree reader support"
```

### Task 2: Expose Collection Import API

**Files:**
- Modify: `src/zotero_kb/api.py`
- Modify: `tests/test_api.py`

- [ ] **Step 1: Write the failing API tests**

```python
def test_collection_tree_endpoint(tmp_path: Path) -> None:
    app = create_app(make_test_config(tmp_path), llm_client=FakeLlmClient())
    endpoint = _route(app, "/api/zotero/collections/tree", "GET")

    payload = endpoint()

    assert payload["collections"][0]["collection_key"] == "COLL0001"
    assert payload["collections"][0]["children"][0]["collection_key"] == "COLL0002"


def test_collection_items_endpoint_includes_descendants(tmp_path: Path) -> None:
    app = create_app(make_test_config(tmp_path), llm_client=FakeLlmClient())
    endpoint = _route(app, "/api/zotero/collections/{collection_key}/items", "GET")

    payload = endpoint("COLL0001", True)

    assert payload["items"][0]["item_key"] == "PAPER0001"
```

- [ ] **Step 2: Run test to verify it fails**

Run: `UV_CACHE_DIR=/tmp/uv-cache uv run pytest tests/test_api.py::test_collection_tree_endpoint tests/test_api.py::test_collection_items_endpoint_includes_descendants -q`
Expected: FAIL because the routes are missing.

- [ ] **Step 3: Write minimal implementation**

```python
@app.get("/api/zotero/collections/tree")
def zotero_collection_tree() -> dict[str, object]:
    return {"collections": reader.get_collection_tree()}


@app.get("/api/zotero/collections/{collection_key}/items")
def zotero_collection_items(collection_key: str, include_descendants: bool = True) -> dict[str, object]:
    return {
        "collection_key": collection_key,
        "items": reader.get_collection_items(collection_key, include_descendants=include_descendants),
    }
```

- [ ] **Step 4: Run test to verify it passes**

Run: `UV_CACHE_DIR=/tmp/uv-cache uv run pytest tests/test_api.py::test_collection_tree_endpoint tests/test_api.py::test_collection_items_endpoint_includes_descendants -q`
Expected: PASS

- [ ] **Step 5: Commit**

```bash
git add src/zotero_kb/api.py tests/test_api.py
git commit -m "feat: expose zotero collection import api"
```

### Task 3: Replace Left-Panel Import UI

**Files:**
- Modify: `src/zotero_kb/templates/index.html`
- Modify: `tests/test_ui.py`

- [ ] **Step 1: Write the failing UI test**

```python
def test_index_contains_collection_import_controls(tmp_path) -> None:
    app = create_app(
        AppConfig(
            workspace_dir=tmp_path / "workspace",
            zotero_data_dir=tmp_path / "zotero",
            bridge_file=tmp_path / "bridge.json",
        )
    )

    endpoint = _route(app, "/", "GET")
    html = endpoint()

    assert 'id="collection-tree"' in html
    assert 'id="collection-items"' in html
    assert 'id="select-descendants-button"' in html
    assert 'id="clear-selection-button"' in html
    assert 'id="import-selected-items-button"' in html
```

- [ ] **Step 2: Run test to verify it fails**

Run: `UV_CACHE_DIR=/tmp/uv-cache uv run pytest tests/test_ui.py::test_index_contains_collection_import_controls -q`
Expected: FAIL because the template still contains `import-selected-button` and `zotero-search-form` instead of the new collection importer ids.

- [ ] **Step 3: Write minimal implementation**

```html
<div class="section surface">
  <h3>从 Zotero 导入</h3>
  <div id="collection-tree" class="tree"></div>
  <div id="collection-items" class="result-list"></div>
  <div class="action-bar">
    <span id="selected-count">已选 0 篇</span>
    <button id="select-descendants-button" type="button" class="secondary">全选当前目录及子目录</button>
    <button id="clear-selection-button" type="button" class="danger">清空选择</button>
    <button id="import-selected-items-button" type="button">导入所选到当前项目</button>
  </div>
</div>
```

```javascript
state.collectionTree = [];
state.selectedCollectionKey = null;
state.selectedItemKeys = new Set();
state.expandedCollectionKeys = new Set();
state.visibleCollectionItems = [];

async function loadCollectionTree() {
  const payload = await api("/api/zotero/collections/tree");
  state.collectionTree = payload.collections || [];
  renderCollectionTree();
}

async function selectCollection(collectionKey) {
  state.selectedCollectionKey = collectionKey;
  const payload = await api(`/api/zotero/collections/${collectionKey}/items?include_descendants=true`);
  state.visibleCollectionItems = payload.items || [];
  renderCollectionItems();
}

function toggleItemSelection(itemKey, checked) {
  if (checked) state.selectedItemKeys.add(itemKey);
  else state.selectedItemKeys.delete(itemKey);
  renderSelectedCount();
}
```

- [ ] **Step 4: Run test to verify it passes**

Run: `UV_CACHE_DIR=/tmp/uv-cache uv run pytest tests/test_ui.py::test_index_contains_collection_import_controls -q`
Expected: PASS

- [ ] **Step 5: Commit**

```bash
git add src/zotero_kb/templates/index.html tests/test_ui.py
git commit -m "feat: add zotero collection tree importer ui"
```

### Task 4: Wire Batch Import And Refresh

**Files:**
- Modify: `src/zotero_kb/templates/index.html`
- Modify: `tests/test_api.py`
- Modify: `README.md`

- [ ] **Step 1: Write the failing behavior test**

```python
def test_import_item_keys_endpoint_returns_project_view(tmp_path: Path) -> None:
    config = make_test_config(tmp_path)
    app = create_app(config, llm_client=FakeLlmClient())
    create_project = _route(app, "/api/projects", "POST")
    import_keys = _route(app, "/api/projects/{project_id}/imports/item-keys", "POST")

    create_project(
        CreateProjectRequest(
            project_id="thesis-ch2",
            name="Thesis Chapter 2",
            llm_provider="deepseek",
            llm_model="deepseek-chat",
        )
    )

    payload = import_keys("thesis-ch2", {"item_keys": ["PAPER0001"]})

    assert payload["project_view"]["cards"][0]["item_key"] == "PAPER0001"
```

- [ ] **Step 2: Run test to verify it fails**

Run: `UV_CACHE_DIR=/tmp/uv-cache uv run pytest tests/test_api.py::test_import_item_keys_endpoint_returns_project_view -q`
Expected: FAIL only if the endpoint response shape or refreshed project view is wrong after the UI refactor.

- [ ] **Step 3: Write minimal implementation**

```javascript
document.getElementById("select-descendants-button").addEventListener("click", () => {
  for (const item of state.visibleCollectionItems) {
    state.selectedItemKeys.add(item.item_key);
  }
  renderCollectionItems();
});

document.getElementById("clear-selection-button").addEventListener("click", () => {
  state.selectedItemKeys.clear();
  renderCollectionItems();
});

document.getElementById("import-selected-items-button").addEventListener("click", async () => {
  const payload = await api(`/api/projects/${state.currentProjectId}/imports/item-keys`, {
    method: "POST",
    body: JSON.stringify({ item_keys: Array.from(state.selectedItemKeys) }),
  });
  state.selectedItemKeys.clear();
  renderCards(payload.project_view.cards || []);
  renderCollectionItems();
});
```

```markdown
## 从 Zotero 导入

1. 选择一个项目
2. 在左栏 `从 Zotero 导入` 中展开 collection 树
3. 点击一个目录，系统会加载该目录及其子目录的文献
4. 勾选单篇文献，或点击 `全选当前目录及子目录`
5. 点击 `导入所选到当前项目`
```

- [ ] **Step 4: Run test to verify it passes**

Run: `UV_CACHE_DIR=/tmp/uv-cache uv run pytest tests/test_api.py::test_import_item_keys_endpoint_returns_project_view -q`
Expected: PASS

- [ ] **Step 5: Run full verification**

Run: `UV_CACHE_DIR=/tmp/uv-cache uv run pytest -q`
Expected: PASS with all tests green

- [ ] **Step 6: Commit**

```bash
git add src/zotero_kb/templates/index.html tests/test_api.py README.md
git commit -m "feat: support batch import from zotero collections"
```