wangshuxun/zotero-kb
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
55edb5522a
zotero-kb/docs/superpowers/plans/2026-04-15-import-card-two-step.md
Saberlve cc1960e9a9 docs: add two-step import+card generation implementation plan 
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-15 17:04:08 +08:00

24 KiB
Raw Blame History

Two-Step Import + Card Generation Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Split the current "import文献 + immediately generate cards" flow into: (1) import items to project, check global cards.json for existing cards, (2) manual batch select + generate cards.

Architecture: Backend: two new API endpoints (import-state, cards/generate). imports/item-keys no longer generates cards — it only registers items. CardBuilder.build_or_update() already computes source_hash and skips LLM if hash unchanged (dedup). Frontend: import window two-column layout with pending/done visual states and batch generate button with real-time progress.

Tech Stack: Python/FastAPI, vanilla JS, HTML/CSS.

Task 1: Backend — GET /api/projects/{project_id}/import-state

Files:

  • Modify: src/zotero_kb/api.py (add endpoint)

  • Test: tests/test_api.py

  • Step 1: Write the failing test

In tests/test_api.py, add test test_import_state_returns_pending_and_done:

def test_import_state_returns_pending_and_done(tmp_path, monkeypatch):
    from zotero_kb.api import create_app
    from zotero_kb.config import AppConfig
    from zotero_kb.llm import DeterministicCardClient

    # Setup: create project with 2 items, one has card
    workspace = tmp_path / "workspace"
    items_index = workspace / "library" / "index" / "items.json"
    cards_index = workspace / "library" / "index" / "cards.json"
    items_index.parent.mkdir(parents=True)
    cards_index.parent.mkdir(parents=True)

    items_index.write_text(json.dumps({
        "KEY1": {"item_key": "KEY1", "title": "Paper A"},
        "KEY2": {"item_key": "KEY2", "title": "Paper B"},
    }))
    cards_index.write_text(json.dumps({
        "KEY1": {"item_key": "KEY1", "title": "Paper A"}  # KEY1 has card, KEY2 does not
    }))

    project_dir = workspace / "projects" / "proj1"
    project_dir.mkdir(parents=True)
    (project_dir / "selected-items.json").write_text(json.dumps(["KEY1", "KEY2"]))

    config = AppConfig(workspace_dir=workspace, zotero_data_dir=tmp_path / "zotero")
    app = create_app(config, llm_client=DeterministicCardClient())

    response = client.get("/api/projects/proj1/import-state")
    assert response.status_code == 200
    data = response.json()
    assert data["pending_count"] == 1
    assert data["done_count"] == 1
    items_by_key = {it["item_key"]: it for it in data["items"]}
    assert items_by_key["KEY1"]["card_status"] == "done"
    assert items_by_key["KEY2"]["card_status"] == "pending"
  • Step 2: Run test to verify it fails

Run: UV_CACHE_DIR=/tmp/uv-cache uv run pytest tests/test_api.py::test_import_state_returns_pending_and_done -v Expected: FAIL — endpoint not defined

  • Step 3: Implement the endpoint

Add to api.py:

@app.get("/api/projects/{project_id}/import-state")
def get_import_state(project_id: str) -> dict[str, object]:
    project_dir = config.workspace_dir / "projects" / project_id
    project_file = project_dir / "project.json"
    if not project_file.exists():
        raise HTTPException(status_code=404, detail="Project not found")

    selected_items = project_service._read_selected_items(project_id)
    cards_index = projects._read_json(config.workspace_dir / "library" / "index" / "cards.json")
    items_index = projects._read_json(config.workspace_dir / "library" / "index" / "items.json")

    items = []
    pending_count = 0
    done_count = 0
    for item_key in selected_items:
        item_data = items_index.get(item_key, {})
        card_status = "done" if item_key in cards_index else "pending"
        if card_status == "done":
            done_count += 1
        else:
            pending_count += 1
        items.append({
            "item_key": item_key,
            "title": item_data.get("title", ""),
            "creators": item_data.get("creators", []),
            "year": item_data.get("year"),
            "item_type": item_data.get("item_type", ""),
            "card_status": card_status,
        })

    return {
        "project_id": project_id,
        "items": items,
        "pending_count": pending_count,
        "done_count": done_count,
    }
  • Step 4: Run test to verify it passes

Run: UV_CACHE_DIR=/tmp/uv-cache uv run pytest tests/test_api.py::test_import_state_returns_pending_and_done -v Expected: PASS

  • Step 5: Commit
git add src/zotero_kb/api.py tests/test_api.py
git commit -m "feat: add GET /api/projects/{id}/import-state endpoint

Returns pending/done status for each item in a project by checking
selected-items.json against cards.json.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
"

Task 2: Backend — Modify POST /api/projects/{project_id}/imports/item-keys to not generate cards

Files:

  • Modify: src/zotero_kb/api.py (lines ~127-155, the import_item_keys endpoint)

  • Test: tests/test_api.py

  • Step 1: Write the failing test

Add test_import_item_keys_does_not_generate_cards:

def test_import_item_keys_does_not_generate_cards(tmp_path, monkeypatch):
    from zotero_kb.api import create_app
    from zotero_kb.config import AppConfig
    from zotero_kb.llm import DeterministicCardClient

    workspace = tmp_path / "workspace"
    items_index = workspace / "library" / "index" / "items.json"
    items_index.parent.mkdir(parents=True)
    items_index.write_text(json.dumps({}))

    project_dir = workspace / "projects" / "proj1"
    project_dir.mkdir(parents=True)
    (project_dir / "project.json").write_text(json.dumps({"project_id": "proj1", "name": "Test"}))

    config = AppConfig(workspace_dir=workspace, zotero_data_dir=tmp_path / "zotero")
    app = create_app(config, llm_client=DeterministicCardClient())

    # Mock reader to return fake items
    class FakeReader:
        def read_items(self, keys):
            return [ZoteroItemRecord(item_key=k, title=f"Paper {k}", creators=[], year=2024,
                          item_type="journalArticle", abstract="", tags=[], collection_paths=[],
                          notes=[], attachment_texts=[]) for k in keys]

    monkeypatch.setattr("zotero_kb.api.ZoteroReader", lambda *a, **k: FakeReader())

    response = client.post("/api/projects/proj1/imports/item-keys", json={"item_keys": ["KEY1"]})
    assert response.status_code == 200
    data = response.json()
    assert data["imported_item_keys"] == ["KEY1"]

    # Verify cards.json was NOT created (no card generation)
    cards_index = workspace / "library" / "index" / "cards.json"
    assert not cards_index.exists()
  • Step 2: Run test to verify it fails

Run: UV_CACHE_DIR=/tmp/uv-cache uv run pytest tests/test_api.py::test_import_item_keys_does_not_generate_cards -v Expected: FAIL — currently it calls CardBuilder.build_or_update() which creates cards

  • Step 3: Modify import_item_keys to not generate cards

Change the import_item_keys endpoint body from:

builder = CardBuilder(config.workspace_dir, resolved_client)
items = reader.read_items(item_keys)
imported_keys = []
for item in items:
    builder.build_or_update(item)  # REMOVE THIS
    imported_keys.append(item.item_key)

to:

items = reader.read_items(item_keys)
imported_keys = [item.item_key for item in items]
# NOTE: card generation happens later via explicit /cards/generate call
# Ensure items are registered in items.json for later reference
items_index = projects._read_json(config.workspace_dir / "library" / "index" / "items.json")
for item in items:
    items_index[item.item_key] = {
        "item_key": item.item_key,
        "title": item.title,
        "creators": item.creators,
        "year": item.year,
        "item_type": item.item_type,
        "abstract": item.abstract,
        "tags": item.tags,
        "collection_paths": item.collection_paths,
    }
projects._write_json(config.workspace_dir / "library" / "index" / "items.json", items_index)

Also remove the LLM client creation since it's no longer needed for this endpoint.

  • Step 4: Run test to verify it passes

Run: UV_CACHE_DIR=/tmp/uv-cache uv run pytest tests/test_api.py::test_import_item_keys_does_not_generate_cards -v Expected: PASS

  • Step 5: Commit
git add src/zotero_kb/api.py tests/test_api.py
git commit -m "refactor: import_item_keys no longer generates cards immediately

Items are registered in items.json but cards are generated later via
explicit POST /projects/{id}/cards/generate call.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
"

Task 3: Backend — POST /api/projects/{project_id}/cards/generate

Files:

  • Modify: src/zotero_kb/api.py (add endpoint)

  • Modify: src/zotero_kb/projects.py (add add_items helper)

  • Test: tests/test_api.py

  • Step 1: Write the failing test

Add test_cards_generate_creates_cards_for_pending_items:

def test_cards_generate_creates_cards_for_pending_items(tmp_path, monkeypatch):
    from zotero_kb.api import create_app
    from zotero_kb.config import AppConfig
    from zotero_kb.llm import DeterministicCardClient

    workspace = tmp_path / "workspace"
    items_index = workspace / "library" / "index" / "items.json"
    cards_index = workspace / "library" / "index" / "cards.json"
    items_index.parent.mkdir(parents=True)
    cards_index.parent.mkdir(parents=True)

    items_index.write_text(json.dumps({
        "KEY1": {"item_key": "KEY1", "title": "Paper A", "creators": ["Author A"], "year": 2024,
                 "item_type": "journalArticle", "abstract": "Abstract A", "tags": [], "collection_paths": []},
    }))
    cards_index.write_text(json.dumps({}))  # no existing cards

    project_dir = workspace / "projects" / "proj1"
    project_dir.mkdir(parents=True)
    (project_dir / "project.json").write_text(json.dumps({"project_id": "proj1", "name": "Test",
        "llm": {"provider": "deterministic", "model": "deterministic"}}))
    (project_dir / "selected-items.json").write_text(json.dumps(["KEY1"]))

    config = AppConfig(workspace_dir=workspace, zotero_data_dir=tmp_path / "zotero")
    app = create_app(config, llm_client=DeterministicCardClient())

    response = client.post("/api/projects/proj1/cards/generate", json={"item_keys": ["KEY1"]})
    assert response.status_code == 200
    data = response.json()
    assert data["generated"] == ["KEY1"]
    assert data["failed"] == []
    assert data["items"][0]["card_status"] == "done"

    # Verify cards.json now has the card
    cards = json.loads(cards_index.read_text())
    assert "KEY1" in cards
  • Step 2: Run test to verify it fails

Run: UV_CACHE_DIR=/tmp/uv-cache uv run pytest tests/test_api.py::test_cards_generate_creates_cards_for_pending_items -v Expected: FAIL — endpoint not defined

  • Step 3: Implement the endpoint

Add to api.py:

class GenerateCardsRequest(BaseModel):
    item_keys: list[str] = Field(min_length=1)


@app.post("/api/projects/{project_id}/cards/generate")
def generate_cards(project_id: str, payload: GenerateCardsRequest | dict[str, object]) -> dict[str, object]:
    project_file = config.workspace_dir / "projects" / project_id / "project.json"
    if not project_file.exists():
        raise HTTPException(status_code=404, detail="Project not found")

    if isinstance(payload, dict):
        item_keys = [str(value) for value in payload.get("item_keys", [])]
    else:
        item_keys = [str(value) for value in payload.item_keys]

    project_payload = _read_json(project_file)
    llm_payload = project_payload.get("llm", {})
    try:
        resolved_client = llm_client or create_card_generation_client(
            str(llm_payload.get("provider", "deterministic")),
            str(llm_payload.get("model", "deterministic")),
        )
    except ValueError as exc:
        raise HTTPException(status_code=400, detail=str(exc)) from exc

    builder = CardBuilder(config.workspace_dir, resolved_client)
    reader = ZoteroReader(config.zotero_data_dir)
    items = reader.read_items(item_keys)

    generated = []
    failed = []
    result_items = []

    for item in items:
        try:
            result = builder.build_or_update(item)
            generated.append(item.item_key)
            card_status = "done"
        except Exception as exc:
            failed.append(item.item_key)
            card_status = "failed"

        result_items.append({
            "item_key": item.item_key,
            "title": item.title,
            "creators": item.creators,
            "year": item.year,
            "item_type": item.item_type,
            "card_status": card_status,
        })

    project_service.add_items(project_id, generated)
    return {
        "project_id": project_id,
        "generated": generated,
        "failed": failed,
        "items": result_items,
    }
  • Step 4: Run test to verify it passes

Run: UV_CACHE_DIR=/tmp/uv-cache uv run pytest tests/test_api.py::test_cards_generate_creates_cards_for_pending_items -v Expected: PASS

  • Step 5: Commit
git add src/zotero_kb/api.py tests/test_api.py
git commit -m "feat: add POST /api/projects/{id}/cards/generate endpoint

Generates cards for specified item_keys via CardBuilder. Returns
generated/failed lists and updated item states.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
"

Task 4: Frontend — Import Window Two-Column Layout

Files:

  • Modify: src/zotero_kb/templates/index.html (import window HTML + JS)

This is the most complex task. Changes span:

  1. Add import-state API call function
  2. Add cards/generate API call function
  3. Replace current import window content with two-column layout
  4. Add pending/done visual states and checkbox logic
  5. Add batch generate button with progress feedback

Layout structure (HTML, inside .window-frame.import-window):

<div class="import-split">
  <div class="import-left">
    <div class="import-list-header">
      <span>Pending (<span id="pending-count">0</span>)</span>
      <button type="button" id="select-all-pending" class="link-btn">Select All</button>
    </div>
    <div class="import-list" id="import-items-list">
      <!-- populated by JS -->
    </div>
    <div class="import-list-footer">
      Done (<span id="done-count">0</span>)
    </div>
  </div>
  <div class="import-right">
    <div id="item-preview">
      <p class="hint">Click an item to preview</p>
    </div>
  </div>
</div>
<div class="import-actions">
  <button type="button" id="generate-cards-btn" class="primary" disabled>生成选中卡片 (0)</button>
</div>

CSS additions:

.import-split { display: grid; grid-template-columns: 40% 60%; height: 100%; gap: 1px; background: var(--line); }
.import-left, .import-right { background: var(--surface); overflow-y: auto; padding: 1rem; }
.import-list-header { display: flex; justify-content: space-between; align-items: center; margin-bottom: 0.75rem; font-weight: 600; }
.import-list { display: flex; flex-direction: column; gap: 0.25rem; }
.import-item { display: flex; align-items: flex-start; gap: 0.5rem; padding: 0.5rem; border-radius: 6px; cursor: pointer; }
.import-item.pending { background: var(--panel); }
.import-item.done { background: transparent; opacity: 0.7; }
.import-item.selected { outline: 2px solid var(--accent, #4a90d9); }
.import-item input[type="checkbox"] { margin-top: 0.25rem; }
.import-item.done input[type="checkbox"] { display: none; }
.import-item .item-title { font-size: 0.875rem; font-weight: 500; }
.import-item .item-meta { font-size: 0.75rem; color: var(--ink-dim, #666); }
.import-list-footer { margin-top: 0.75rem; font-weight: 600; font-size: 0.875rem; }
.import-actions { padding: 0.75rem 1rem; border-top: 1px solid var(--line); display: flex; justify-content: flex-end; }
#generate-cards-btn:disabled { opacity: 0.5; cursor: not-allowed; }
#generate-cards-btn .progress { font-size: 0.875rem; }
.link-btn { background: none; border: none; color: var(--accent, #4a90d9); cursor: pointer; font-size: 0.8rem; }
#item-preview { }
#item-preview .preview-title { font-size: 1rem; font-weight: 600; margin-bottom: 0.5rem; }
#item-preview .preview-meta { font-size: 0.8rem; color: var(--ink-dim); margin-bottom: 0.75rem; }
#item-preview .preview-abstract { font-size: 0.85rem; line-height: 1.5; }
.hint { color: var(--ink-dim); font-size: 0.875rem; }

JS additions (state + functions):

// State
state.importItems = [];       // items from import-state
state.selectedPendingKeys = new Set();
state.isGenerating = false;
state.generatingProgress = { current: 0, total: 0 };

// Functions
async function loadImportState() {
  const resp = await fetch(`/api/projects/${state.currentProjectId}/import-state`);
  const data = await resp.json();
  state.importItems = data.items;
  renderImportItemList();
  updateImportCounts(data.pending_count, data.done_count);
}

function renderImportItemList() {
  const list = document.getElementById('import-items-list');
  list.innerHTML = '';
  state.importItems.forEach(item => {
    const isDone = item.card_status === 'done';
    const div = document.createElement('div');
    div.className = `import-item ${item.card_status}`;
    div.dataset.key = item.item_key;
    div.innerHTML = `
      <input type="checkbox" ${isDone ? 'disabled' : ''} data-key="${item.item_key}">
      <div>
        <div class="item-title">${escapeHtml(item.title)}</div>
        <div class="item-meta">${item.creators.join(', ')} · ${item.year || 'n.d.'}</div>
      </div>
    `;
    if (!isDone) {
      div.querySelector('input').addEventListener('change', (e) => {
        if (e.target.checked) state.selectedPendingKeys.add(item.item_key);
        else state.selectedPendingKeys.delete(item.item_key);
        updateGenerateButton();
        div.classList.toggle('selected', e.target.checked);
      });
      div.addEventListener('click', (e) => { if (e.target.tagName !== 'INPUT') showItemPreview(item); });
    }
    div.addEventListener('click', (e) => { if (e.target.tagName !== 'INPUT') showItemPreview(item); });
    list.appendChild(div);
  });
}

function showItemPreview(item) {
  document.getElementById('item-preview').innerHTML = `
    <div class="preview-title">${escapeHtml(item.title)}</div>
    <div class="preview-meta">${item.creators.join(', ')} · ${item.year || 'n.d.'} · ${item.item_type}</div>
    <div class="preview-abstract">${escapeHtml(item.abstract || 'No abstract available.')}</div>
  `;
}

function updateGenerateButton() {
  const btn = document.getElementById('generate-cards-btn');
  const count = state.selectedPendingKeys.size;
  btn.disabled = count === 0 || state.isGenerating;
  btn.textContent = state.isGenerating
    ? `生成中 ${state.generatingProgress.current}/${state.generatingProgress.total}`
    : `生成选中卡片 (${count})`;
}

async function generateSelectedCards() {
  const keys = Array.from(state.selectedPendingKeys);
  if (!keys.length) return;
  state.isGenerating = true;
  state.generatingProgress = { current: 0, total: keys.length };
  updateGenerateButton();

  const btn = document.getElementById('generate-cards-btn');

  try {
    const resp = await fetch(`/api/projects/${state.currentProjectId}/cards/generate`, {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({ item_keys: keys }),
    });
    const data = await resp.json();

    // Update item states
    data.items.forEach(updated => {
      const item = state.importItems.find(i => i.item_key === updated.item_key);
      if (item) item.card_status = updated.card_status;
    });

    state.selectedPendingKeys.clear();
    renderImportItemList();

    const counts = {
      pending: state.importItems.filter(i => i.card_status === 'pending').length,
      done: state.importItems.filter(i => i.card_status === 'done').length,
    };
    updateImportCounts(counts.pending, counts.done);
  } finally {
    state.isGenerating = false;
    updateGenerateButton();
  }
}

function updateImportCounts(pending, done) {
  document.getElementById('pending-count').textContent = pending;
  document.getElementById('done-count').textContent = done;
}

Wiring:

  • In openImportWindow(): call loadImportState()

  • In closeImportWindow(): clear state.importItems and state.selectedPendingKeys

  • document.getElementById('select-all-pending').addEventListener('click', ...) — select all pending checkboxes

  • document.getElementById('generate-cards-btn').addEventListener('click', generateSelectedCards)

  • Step 1: Write the failing test

Add to tests/test_ui.py:

def test_index_import_window_has_two_column_layout():
    html = (Path(__file__).parent.parent / "src" / "zotero_kb" / "templates" / "index.html").read_text()
    assert 'class="import-split"' in html
    assert 'class="import-left"' in html
    assert 'class="import-right"' in html
    assert 'id="generate-cards-btn"' in html
    assert 'id="import-items-list"' in html
    assert 'id="pending-count"' in html
    assert 'id="done-count"' in html

def test_index_has_import_state_api_call():
    html = (Path(__file__).parent.parent / "src" / "zotero_kb" / "templates" / "index.html").read_text()
    assert '/api/projects/' in html and 'import-state' in html
    assert '/api/projects/' in html and 'cards/generate' in html
  • Step 2: Run tests to verify they fail

Run: UV_CACHE_DIR=/tmp/uv-cache uv run pytest tests/test_ui.py -v -k "two_column or import_state" Expected: FAIL — new elements don't exist yet

  • Step 3: Implement the two-column layout

Replace the current .window-body content inside .import-window with the two-column HTML structure above, add all CSS to the <style> block, and add all JS functions to the <script> section. Wire up the event listeners.

  • Step 4: Run tests to verify they pass

Run: UV_CACHE_DIR=/tmp/uv-cache uv run pytest tests/test_ui.py -v -k "two_column or import_state" Expected: PASS

  • Step 5: Commit
git add src/zotero_kb/templates/index.html tests/test_ui.py
git commit -m "feat: add two-column import window layout with batch generate

Import window now shows pending/done items in left column, preview in
right column. 'Generate Selected Cards' button triggers batch
generation with real-time progress.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
"

Task 5: End-to-End Verification

Files:

  • None (manual + test verification)

  • Step 1: Start the app and verify the flow

UV_CACHE_DIR=/tmp/uv-cache uv run python main.py
  1. Open http://localhost:8000
  2. Create/open a project
  3. Open import window — verify it shows pending items (empty cards list initially)
  4. Select items → click "生成选中卡片"
  5. Verify items move from pending to done after generation
  • Step 2: Run full test suite
UV_CACHE_DIR=/tmp/uv-cache uv run pytest tests/ -v

Expected: All 29+ tests pass

  • Step 3: Commit
git add -A && git commit -m "test: add UI tests for two-column import layout

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
"

Self-Review Checklist

  1. Spec coverage: Each requirement from 2026-04-15-import-card-two-step-design.md is addressed:

    • GET /import-state → Task 1
    • POST /cards/generate → Task 3
    • import_item_keys no longer generates → Task 2
    • Two-column layout + batch generate → Task 4
    • Real-time progress → Task 4 (JS progress counter in button)
    • Deduplication via source_hash → Already exists in CardBuilder.build_or_update()

  2. Placeholder scan: No "TBD", "TODO", or vague steps found.

  3. Type consistency: All method names match across tasks:

    • project_service._read_selected_items() — used in Task 1, defined in projects.py
    • CardBuilder.build_or_update() — called in Task 3, already exists in cards.py
    • ZoteroReader.read_items() — used in Tasks 2 and 3, already exists
    • GenerateCardsRequest model — defined in Task 3, used in API endpoint