wechat-web-viewer.md 19.2 KB

Edit Raw Blame History Permalink



name: wechat-web-viewer
description: Flask web app for browsing WeChat messages with article cards, image/voice media, video号 finder cards, daily report generation (skill-driven via Hermes CLI subprocess), and sending report images to WeChat contacts.
category: wechat
WeChat Web Viewer
When to Use

Building, modifying, or debugging the WeChat message viewer web app at ~/wechat-viewer/
Adding/changing message type handling, media serving, daily report generation
Fixing WeChat content parsing (XML, CDATA, sender prefixes, image paths)
Understanding the skill-driven subprocess architecture pattern

Design Principle: Skill-driven, not hardcoded Python
User explicit requirement: "应该调用 Hermes 子任务来完成。不是写死 Python 代码，未来 skill 变化时没法迭代。"

Two operations delegate to Hermes CLI subprocesses with skills:


Report generation: hermes chat -s wechat-group-digest --yolo -t terminal,file,web
Image generation: hermes chat -s gpt-image-gen --yolo -t terminal,file,web


Image sending does NOT use hermes subprocess — send_wechat_image is an MCP tool inaccessible via -t terminal. Instead, Flask directly imports send_image_via_hook() from ~/.hermes/event_bus/wechat_hook_send.py.

Article summarize/learn also use Hermes CLI subprocesses:


POST /api/articles/summarize → hermes chat -q "fetch + extract key points" --yolo -t terminal,web
POST /api/articles/learn → hermes chat -s learn-and-build --yolo -t terminal,web,file
Results saved to daily_reports table with chatroom_id="article"


Flask only does: task file creation, subprocess spawning in daemon thread, result reading, SQLite persistence. All intelligence lives in skills.
Architecture
~/wechat-viewer/
├── config.py        # DB paths, SQLCipher key, Flask config (DEBUG=False)
├── wechat_db.py     # SQLCipher read layer (su + sqlcipher CLI)
├── local_db.py      # Local SQLite: daily_reports, report_images, article_queue, image_analysis, chat_summaries
├── app.py           # Flask routes + message parser + media proxy + Hermes subprocess spawn
├── restart.sh       # Kill old + fuser -k 5000 + start fresh
├── templates/
│   ├── base.html    # Nav tabs (消息浏览/数据中心/日报列表/日报图片)
│   ├── index.html   # Session list with 📋 button on groups → opens chat page
│   ├── chat.html    # Chat: newest-first infinite scroll, 📋 generate-report, 📝/🔧/🖼 on article cards
│   ├── circle.html  # Data center
│   ├── reports.html # Report list + 生成图片 + delete buttons
│   ├── report_detail.html  # Markdown viewer
│   └── report_images.html  # Image gallery with 发送微信 + delete buttons

Navigation Tabs


Tab
Route
Function


消息浏览
/
Session list → /chat/<talker>


文章列表
/articles
All articles deduplicated, search/filter/sort, 📝摘要 + 🔧学习 buttons


日报列表
/reports
Three-type report list (群日报/文章摘要/学习记录), filterable, 生成图片 for ALL types


日报图片
/report-images
Image gallery with modal preview + 发送微信 button


Article Actions (2026-07-08)
Each article card in /articles and /chat/<talker> has three buttons:


Button
API
Hermes CLI
Skill
Timeout


📝 摘要
POST /api/articles/summarize
hermes chat -q "fetch + extract" --yolo -t terminal,web,file
none
180s


🔧 学习
POST /api/articles/learn
hermes chat -s learn-and-build --yolo -t terminal,web,file
learn-and-build
300s


🖼 图片
POST /api/articles/generate-image
hermes chat -s gpt-image-gen --yolo -t terminal,web,file
gpt-image-gen
300s


Summary and learn results → daily_reports (chatroom_name=article title, report_date=summary/learn).
Image results → report_images (report_id=0 for article images).
Three Button Mobile Layout Pattern
CRITICAL for mobile usability: On all list pages (reports, report-images, articles), buttons must be:


Placed at the bottom of each card, not cramped on the right side
flex:1 with padding: 10-12px and border-radius: 8px for thumb-friendly tapping
Each card's body content scrolls independently
Page must override base.html's body { overflow: hidden }: <style>body{overflow-y:auto!important;height:auto!important;}</style>

Article Actions (2026-07-08)
Each article in /articles has two buttons that spawn Hermes CLI subprocesses:


Button
API
Hermes CLI
Skill
Timeout


📝 摘要
POST /api/articles/summarize
hermes chat -q "fetch + extract" --yolo -t terminal,web,file
none
180s


🔧 学习
POST /api/articles/learn
hermes chat -s learn-and-build --yolo -t terminal,web,file
learn-and-build
300s


Both save results to daily_reports with chatroom_id="article" and report_date="summary" or "learn". Results appear in 日报列表.
Article Summary Dedup (2026-07-08)
Before spawning a Hermes subprocess, both routes check daily_reports for an existing record with the same dedup_key + report_date. If found, returns {status: "cached", content: "..."} immediately without spawning. Dedup key is extracted from the WeChat URL via _extract_dedup_key(url) — same biz|mid|idx|sn algorithm as article list dedup.

daily_reports table gained two columns:


dedup_key TEXT — the biz|mid|idx|sn key for fast lookup
article_url TEXT — original URL for reference


Index: CREATE INDEX idx_daily_reports_dedup ON daily_reports(dedup_key, report_date)

Batch check API (POST /api/articles/check-summaries):


Accepts {urls: [{url, title}, ...], type: "summary"|"learn"|""}
Returns {results: {dedup_key: {id, content, ...}}, missing: [{url, title, dedup_key}]}
Used by: group digest workflow (pre-checks before re-summarizing), frontend (grays out cached buttons)


Frontend: On /articles page load, fetches /api/articles/check-summaries twice (summary + learn) for all articles. Renders cached articles with grayed-out ✅ 已有摘要 / ✅ 已学习 buttons. Info bar shows 共 N 篇（已有摘要 X / 已学习 Y）.

_extract_dedup_key(url) helper (in app.py): extracts biz|mid|idx|sn from mp.weixin.qq.com URLs, returns raw URL for non-WeChat URLs.

local_db.py additions:


find_article_summary(dedup_key, report_type) — single lookup
find_article_summaries_batch(dedup_keys, report_type) — batch lookup, returns {dedup_key: {...}}


Critical: Both subprocesses need -t terminal,web,file — without file, the hermes subprocess cannot write the result file and the operation silently fails.
Article Deduplication
WeChat URLs have per-share tracking params (&chksm=..., &sharer_shareinfo=...) that make identical articles look different. Full-URL dedup fails → same article appears 3-7 times.

Fix (2026-07-08): Extract __biz, mid, idx, sn from the URL and use f"{biz}|{mid}|{idx}|{sn}" as the dedup key. These four params uniquely identify a WeChat article regardless of tracking garbage.
if "mp.weixin.qq.com" in url:
    biz = re.search(r"__biz=([^&]+)", url)
    mid = re.search(r"mid=(\d+)", url)
    idx = re.search(r"idx=(\d+)", url)
    sn = re.search(r"sn=([a-f0-9]+)", url)
    if biz and mid and idx and sn:
        dedup_key = f"{biz.group(1)}|{mid.group(1)}|{idx.group(1)}|{sn.group(1)}"


Store canonical URL separately: url_groups[dedup_key]["url"] = url (first-seen URL wins).
Article List (文章列表)
/articles — 2026-07-08 added, replacing 数据中心 tab.

API (/api/articles): Two-pass: first groups by dedup key (see above), collecting talkers per article; second resolves talker → display name via wechat_db.get_talker_info(). Returns {title, url, description, createTime, shareCount, groups: [names...]}.

Frontend: Search by title, filter by group (dropdown), sort by time↕ or hotness↕. All client-side after initial load.

Deduplication pattern: Two-pass algorithm — first pass builds {dedup_key: {title, desc, createTime, url, talkers: set()}}, second pass resolves talker → display name, sorts by createTime DESC.
Report List: Three-Type System (2026-07-08)
The /reports page now shows three report types with color-coded badges:


Type
badge
filter button
report_date value
generate-image button


群日报
群日报 (#2c3e50)
群日报
date like 2026-07-07
✅


文章摘要
文章摘要 (#3498db)
文章摘要
"summary"
✅


学习记录
学习记录 (#9b59b6)
学习记录
"learn"
✅


All three types have the 生成图片 button — user explicitly required this (2026-07-08).

POST /api/reports/generate {chatroom_id, date} spawns Hermes subprocess with wechat-group-digest skill. Returns immediately with {status: "started", task_id}. Frontend polls /api/reports/generate/status every 3s until content appears.

Sub-agent does: MCP query → parse messages → fetch article URLs → LLM topic summarization → Markdown output.
Report Image Generation
POST /api/reports/<id>/generate-image spawns Hermes subprocess with gpt-image-gen skill.

Sub-agent does: read report Markdown → compose vintage copperplate-style Chinese prompt → GPT Image 2 API (wanjiedata.com) → base64 decode → save PNG.

Image style (2026-07-08, user-specified): 19世纪黑白铜版画蚀刻风，泛黄做旧牛皮纸折痕毛边污渍撕裂，顶部粗黑体刊头=群名仅标日期不写版号ISSN期号，中间大幅复古工业插画，右侧竖栏市场简报文字框，底部分栏文字板块，油墨印刷颗粒质感细腻素描排线，严肃商务科技AI主题，黑白单色调做旧滤镜无多余色彩，高细节8K。

GPT Image 2 API details: model "GPT Image 2" (with spaces), never include responseFormat, timeout >= 120s.
Sending Images to WeChat
POST /api/report-image/<id>/send {to: wxid} copies image to /sdcard/Pictures/ then calls send_image_via_hook(wxid=to, image_path=dest) directly — NOT via hermes subprocess.

Why direct call, not hermes subprocess: Hermes CLI spawned with -t terminal has no access to MCP tools. send_wechat_image is an MCP tool, not a terminal tool. Spawning hermes for this creates 60s+ overhead for a sub-second operation. The Flask app imports from wechat_hook_send import send_image_via_hook from ~/.hermes/event_bus/ and calls it in a daemon thread.

Frontend: "发送微信" button → modal with contact list from /api/sessions (sorted by latest message, searchable) → select → send.
JS Bug: closeSendModal() nulls currentImageId
If sendToEl() calls closeSendModal() (which sets currentImageId = null) before using currentImageId in the fetch() URL, the URL becomes /api/report-image/null/send → 404 HTML → r.json() throws "is not valid json". Always save to local var first: var id = currentImageId; closeSendModal(); fetch('/path/' + id).
Image viewing: modal, not new tab
Use onclick="showImg(url)" with a full-screen overlay modal. Avoid <a target="_blank"> — users need a back button equivalent (click-anywhere-to-close modal).
Message Type Reference


type
meaning
parsed as


1
text
Strip sender:\n prefix


3
image
[图片] + thumbnail


34
voice
[语音] + audio file


43
video
[视频]


47
emoji
[表情]


49
article
Card (title, desc, url, thumbnail)


754974769
finder
Finder card (nickname, avatar, cover, video url, duration)


822083633
refermsg
Extract title from <refermsg>


Media Path Resolution
Images (2026-07-08, corrected): WeChat stores BOTH thumbnails AND original images in image2/:


Thumbnail: image2/{hash[:2]}/{hash[2:4]}/th_<hash> (2-12KB, no extension)
Original: image2/{hash[:2]}/{hash[2:4]}/<hash>.jpg (10-123KB, with .jpg extension)
Same hash is used for both — the .jpg file is the cached full-resolution version
Not all images have cached originals; some only have thumbnails


_resolve_image_path() search priority (in app.py):


<hash>.jpg — cached original
<hash>.png — PNG variant
<hash> — no-extension fallback
th_<hash> — thumbnail (lowest priority)


The old claim "full-size images are CDN-only" was wrong — WeChat caches originals locally when you view them in the app. See references/wechat-image-cdn-investigation.md for the discovery notes.

Chat template (chat.html): inline images use /media/image (auto-selects best available). Clicking opens modal with /media/image/full (explicit full-resolution attempt, falls back to thumbnail).

Voice: imgPath = raw hash → use su -c "find voice2 -name '*<hash>*'" to locate msg_<hash>.amr (directory structure is NOT hash-prefix based).
Key Pitfalls
SQLCipher output
sqlcipher .mode json prepends ok\n before JSON. Strip with output[2:].strip().
WeChat file permissions
Termux user cannot stat WeChat /data/data/ files. All file ops need su -c. Copy to /tmp before serving via send_file().
Group names
chatroom.displayname = member list (e.g. "丁鹏、火星人..."). Use COALESCE(NULLIF(r.nickname, ''), c.displayname) to get actual custom group name.
CDATA wrapping
Strip at parse entry: re.sub(r"<!\\[CDATA\\[(.*?)\\]\\]>", r"\\1", content, flags=re.DOTALL)
CRITICAL: Flask route placement
@app.route() decorators MUST appear BEFORE if __name__ == "__main__": block. When running python3 app.py, app.run() blocks at import time — any @app.route() decorators defined after app.run() are NEVER evaluated, returning 404. Verify: python3 -c "from app import app; [print(r) for r in app.url_map.iter_rules()]"
Multi-line f-string SyntaxError
Hermes subprocess prompts use f-strings. Literal newlines inside f-strings cause SyntaxError. Always use \\n escape: f"line1\\nline2" NOT f"line1\nline2".
JavaScript template strings
Literal newlines inside JS single-quoted strings in Jinja templates break the entire page — the page spins forever. Always use \\n escape.

CRITICAL: Avoid JS template literals (backticks) nested inside Jinja template rendering with ${} interpolation. The escaping cascade is fragile and debug-resistant. Prefer plain string concatenation: '<div onclick="fn(' + val + ')">' over backtick templates.
closeSendModal() nulls currentImageId
If a function calls closeSendModal() (which sets currentImageId = null) before using currentImageId in a fetch() URL, the URL becomes /api/report-image/null/send → 404 HTML → r.json() throws "is not valid json". Always save to local var first: var id = currentImageId; closeSendModal(); fetch('/path/' + id).
body { overflow: hidden } breaks scrollable pages
base.html sets body { overflow: hidden; height: 100vh } for the sidebar layout. Pages that need scrolling (articles, reports, report-images) must override: <style>body { overflow-y: auto !important; height: auto !important; }</style> in the template body block.
NEVER blame browser cache first
When user reports "page not working", "spinning forever", or "404": verify backend FIRST. Check curl the API, python3 -c "import app" for syntax errors, grep rendered HTML. Only after server-side passes should client caching be considered. User was frustrated by repeated cache-blame for real code bugs.
Article dedup race condition
When two summarize/learn requests for the same article arrive simultaneously, both pass the dedup check before either hermes finishes and writes to daily_reports. This means the second request spawns a redundant hermes subprocess. The dedup only prevents re-processing after the first result is fully persisted. Acceptable for now — practical dedup savings come from cross-session reuse (group digests, next-day page loads), not from same-second double-clicks.
Server management

DEBUG = False (auto-reload creates notification storms from killed processes)
Template changes require restart.sh
restart.sh flow: pkill -f 'python3 app.py' → fuser -k 5000/tcp → python3 app.py

CRITICAL: Never delete large blocks of app.py without verifying boundaries
When deleting code blocks from app.py, ALWAYS read the deletion range first with read_file(offset, limit) to confirm exactly what will be removed. A single misaligned deletion can remove hundreds of lines including ALL main routes (/reports, /articles, /api/sessions, etc.). If file becomes corrupted, check: python3 -c "import app; print(sum(1 for _ in app.app.url_map.iter_rules()))" — a healthy app.py has 29-30 routes. Anything less than 20 means routes were lost.
DB Tables
-- daily_reports: generated Markdown reports + article summaries/learn
daily_reports (
    id, chatroom_id, chatroom_name, report_date, content,
    msg_count, image_count, article_count,
    dedup_key TEXT,        -- 2026-07-08: biz|mid|idx|sn for article dedup
    article_url TEXT,      -- 2026-07-08: original URL for reference
    created_at
)
-- Index: idx_daily_reports_dedup ON (dedup_key, report_date)

-- report_images: generated WSJ-style infographics  
report_images (id, report_id, image_path, style, created_at)

-- existing tables
article_queue, image_analysis, chat_summaries, processed_messages

Hermes CLI Subprocess Pattern
See references/hermes-cli-subprocess-pattern.md for the complete reference on spawning Hermes from Flask.

Key flags:
| flag | purpose |
|------|---------|
| -q "prompt" | Non-interactive single query |
| -s skill | Preload skill(s) |
| --yolo | Skip confirmations |
| -t groups | Enable tool groups (terminal,file,web) |
| capture_output=True | Capture stdout/stderr |
| timeout=300 | 5-min timeout for long operations |

Do NOT use --quiet — it suppresses output generation. Use stdout as fallback when result files aren't written.
References

references/hermes-cli-subprocess-pattern.md — Complete guide for spawning Hermes CLI subprocesses from Flask
references/message-xml-patterns.md — WeChat message XML formats and parsing

references/hermes-subprocess-pitfalls.md — Common failures and fixes
references/frontend-js-pitfalls.md — JS bugs: closeSendModal before fetch, template literal escaping, image modals, route ordering
references/article-test-script.md — Automated test: URL validity → summarize → learn-build
references/report-list-ui-patterns.md — Three report types, filtering, mobile button layout
scripts/test_articles.py — Rerunnable test script for article links + summarize + learn pipeline
templates/restart.sh — Server restart script

LAN Access
Set HOST = "0.0.0.0" in config.py to allow Mac/other devices on same WiFi to access. Phone IP: ifconfig | grep wlan0 -A1 | grep inet. Access from Mac: http://<phone-ip>:5000。


references/wechat-image-cdn-investigation.md — Full-resolution image CDN investigation (2026-07-08): why only thumbnails are local, hex-encoded CDN params, viable download approaches
references/article-test-script.md — Automated test script: URL validity → summarize → learn-build