# Bulk Async Article Summarization When you need to summarize many articles from the TGO pipeline (or any group) and do not want to wait for each one sequentially, submit them all in parallel against `POST /api/articles/summarize` and collect the async task IDs. The endpoint is single-article and asynchronous, so it returns immediately for each URL. ## Endpoint contract ``` POST http://127.0.0.1:5001/api/articles/summarize Content-Type: application/json {"url": "https://mp.weixin.qq.com/s?...", "title": "..."} ``` Responses: - `{"ok": true, "data": {"status": "cached", "content": "...", "report_id": N}}` — already summarized - `{"ok": true, "data": {"status": "started", "task_id": "..."}}` — Hermes job spawned; poll `/api/job/` if you want to wait ## Common mistakes - ❌ Calling `POST /api/articles/summarize` without `Content-Type: application/json` → `415 Unsupported Media Type`. - ❌ Calling it with an empty body or without `url` → `400 Bad Request` or `{"ok": false, "error": "需要 url"}`. - ❌ Expecting a batch API — there is none; the endpoint is one article per call. - ❌ Using `python3 -c` in a Hermes session to script the loop — this may be blocked by the `script execution via -e/-c flag` security policy. Use `bash + jq` or write a `.py`/`.sh` file and run it via `terminal`. ## Quick one-off bulk submit (bash + jq) ```bash API=http://127.0.0.1:5001 OUT=$HOME/.tmp/bulk_summaries mkdir -p "$OUT" # 1. Pull the unsummarized list curl -s "$API/api/articles/unsummarized" > "$OUT/unsummarized.json" # 2. Submit each URL in parallel; write response to a task file while IFS=$'\t' read -r id url; do payload=$(jq -n --arg url "$url" '{url:$url}') curl -s -X POST -H "Content-Type: application/json" \ -d "$payload" "$API/api/articles/summarize" \ > "$OUT/task_${id}.json" & done < <(jq -r '.data[] | [.id, .url] | @tsv' "$OUT/unsummarized.json") wait # 3. Inspect results jq -s 'map(.data) | {cached: map(select(.status=="cached")) | length, started: map(select(.status=="started")) | length}' "$OUT"/task_*.json ``` ## Concurrency guard The underlying `/api/articles/summarize` spawns a Hermes CLI subprocess per article. Do not fire 50+ requests simultaneously — it overloads the device and can trigger WeChat CAPTCHA / web-scraping limits. Keep parallel submits to **2–4** at a time, or use the existing `scan_articles.py` (sequential, zero concurrency) for cron runs. ## Files from this session - `/data/data/com.termux/files/home/tgo_summarize_batch.sh` — full working script that extracts, fetches unsummarized, and bulk-submits summaries in parallel.