Commit e531655555f744af15f2a2f770c855a90cabafbc
1 parent
ca116630
feat(phoneuse): 修通 Frida Mac 桥 + 新增通用 Intent/Bitmap 检视 MCP 工具
frida_call 此前完全不可达(TASKS.md S3-03):Mac 端没装 frida/frida-tools pip 包。新建 phoneuse/.venv,锁定 frida==17.9.10 + frida-tools==14.8.0 (与设备上跑的 frida-server 17.9.10 匹配,更新版本的 frida-tools 对 frida 有严格 range 依赖会直接报错),新增 start_frida_bridge.sh 一键 adb reverse + 起桥。 新增 frida_inspect_tools.py:inspect_foreground_intent(package) / dump_live_bitmaps(package, max_export) 两个通用 MCP 工具,分别用 Java.choose 枚举目标进程里存活的 Activity(按 hasWindowFocus 识别前台) 和 Bitmap(按 allocationByteCount 排序导出最大的 N 张为 base64 PNG)。 不需要逆向任何具体 App,跟 per-app hook(如微信 sendMessage,已被证实 重度混淆卡死过)明确区分开。 已知限制:frida_bridge.py 现有的进程名子串匹配按 frida 报告的人类可读 标签匹配,不是包名,微信类多进程 App 会匹配到无 UI 的子进程导致静默空 结果——两个新工具已对空结果加诊断提示,根因不在本次改动范围。真机端到 端验证因设备从 adb devices 掉线未完成,TASKS.md 按既有审计格式如实标 记为 [!] 阻塞,未来设备在线后用 Settings/Calculator 之类无隐私 App 补测。 CLAUDE.md 补上 phoneuse 子项目说明——2026-06-23 拍平合并进本仓库后一直 没同步进根文档。 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Showing
6 changed files
with
319 additions
and
0 deletions
CLAUDE.md
| @@ -9,6 +9,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co | @@ -9,6 +9,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co | ||
| 9 | | 子项目 | 路径 | 形态 | 状态 | | 9 | | 子项目 | 路径 | 形态 | 状态 | |
| 10 | |---|---|---|---| | 10 | |---|---|---|---| |
| 11 | | **lifelog-recorder** | `lifelog-recorder/` | Pixel 原生 Android 应用(Kotlin + Compose) | **主线 v0.8** | | 11 | | **lifelog-recorder** | `lifelog-recorder/` | Pixel 原生 Android 应用(Kotlin + Compose) | **主线 v0.8** | |
| 12 | +| **phoneuse** | `phoneuse/` | Hermes Agent 的 Android MCP 工具层(Python/Termux + Kotlin APK 双实现) | 并行实验,多数工具未验证 | | ||
| 12 | | **wechat_plain** | `wechat_plain/` | Pixel 上微信 bot 的 Xposed/A11y 模块(端上) | 辅助 | | 13 | | **wechat_plain** | `wechat_plain/` | Pixel 上微信 bot 的 Xposed/A11y 模块(端上) | 辅助 | |
| 13 | | **lemonPCRM** | `server/` + `frontend/` + `packages/` | Node.js + Express 后端 + 静态前端 | **已废弃** | | 14 | | **lemonPCRM** | `server/` + `frontend/` + `packages/` | Node.js + Express 后端 + 静态前端 | **已废弃** | |
| 14 | 15 | ||
| @@ -34,6 +35,44 @@ Pixel 上微信发送/跟踪的落地,纯设备内运行(不依赖 Mac): | @@ -34,6 +35,44 @@ Pixel 上微信发送/跟踪的落地,纯设备内运行(不依赖 Mac): | ||
| 34 | 35 | ||
| 35 | 详见 `wechat_plain/` 内文档。 | 36 | 详见 `wechat_plain/` 内文档。 |
| 36 | 37 | ||
| 38 | +## phoneuse(Hermes Agent 的 Android MCP 工具层,并行实验) | ||
| 39 | + | ||
| 40 | +> 完整架构见 `phoneuse/ARCHITECTURE.md`,Sprint 任务 + 真机审计记录见 `phoneuse/TASKS.md`,产品 vision 原始稿见 `phoneuse/phoneuse.md`。本节只给定位 + 别踩的坑,细节进文件看。 | ||
| 41 | + | ||
| 42 | +### 定位 | ||
| 43 | + | ||
| 44 | +把 Pixel 上**所有 App** 虚拟化成 **Hermes Agent**(独立 LLM agent runtime,跑在设备 Termux 里)可调用的数据源/能力源:App Runtime Access → Semantic Extraction → Memory Graph → Agent Planning,目标是替代"截图 + OCR + 点击"那种第一代 phone use。跟 lifelog-recorder 是**不同的项目脉络**,2026-06-23 才拍平合并进本仓库,目前两边没有打通。 | ||
| 45 | + | ||
| 46 | +> [!IMPORTANT] 不是零 Mac | ||
| 47 | +> lifelog-recorder 的"完全本地零云端"约束**不适用于 phoneuse 的 Frida 层**:`frida_bridge.py` 跑在 Mac 上,设备端 MCP 工具通过 `adb reverse tcp:8767 tcp:8767` 调用 Mac 上的 frida Python 库去 attach 设备进程。判断这部分能力时不要套用 lifelog-recorder 那套"零网络"假设。 | ||
| 48 | + | ||
| 49 | +### 架构(两套并行实现,未合并) | ||
| 50 | + | ||
| 51 | +- **实现 A(Python/Termux,主链路已跑通)**:`notification_monitor.py`(root dumpsys 轮询) → `event_server.py`(FastAPI+SQLite 队列,:8765) → `android_mcp.py`(FastMCP stdio,Hermes 直连),独立工具模块各自 `register_xxx_tools(mcp)`:`ui_control_tools`/`device_tools`/`telephony_tools`/`ocr_tools`/`frida_inspect_tools`。 | ||
| 52 | +- **实现 B(Kotlin APK,大量 stub)**:`AgentAccessibilityService`/`AgentNotificationListenerService` 实时事件 → Unix socket → MCP HTTP/SSE(:8080);`frida-bridge` 模块是 Frida REST client 骨架。 | ||
| 53 | + | ||
| 54 | +### MCP 工具现状:不要假设"写了就能用" | ||
| 55 | + | ||
| 56 | +`android_mcp.py` 注册 ~60 个工具。`TASKS.md` 2026-06-16 真机逐个审计结论:**真可用 11 / 降级 1 / 失效 23**,且明确写了"之前版本所有任务全标 `[x]` 是错的"。典型坑:`EVENT_BUS_DIR` 未定义连带 connector/persona/cloud 11 个工具 `NameError`;`_run_sqlcipher` 不支持参数化导致部分微信查询工具直接 `datatype mismatch`。改这块前先读 `TASKS.md` 的审计记录,别被勾选状态骗。 | ||
| 57 | + | ||
| 58 | +### Frida 深度运行时层(Layer 5,"特种部队",只打高价值 App) | ||
| 59 | + | ||
| 60 | +通用能力(**不需要逆向任何具体 App**): | ||
| 61 | +- `frida_call(package, script)` — 注入任意调用方自写的 JS。 | ||
| 62 | +- `inspect_foreground_intent(package)` — 枚举目标进程里所有存活 Activity,按 `hasWindowFocus()` 识别前台的那个,dump 它的 Intent(action/data/component/extras)。 | ||
| 63 | +- `dump_live_bitmaps(package, max_export)` — 枚举目标进程堆里所有存活 Bitmap,导出最大的 N 张为 base64 PNG。 | ||
| 64 | + | ||
| 65 | +字符串解密 / `sendMessage()` 之类具体方法 hook 是**per-app 逆向**,不通用——微信 8.x 重度混淆已经卡死过一次(`TASKS.md` S3-05/S3-06),明确排除在这层通用能力之外。 | ||
| 66 | + | ||
| 67 | +> [!IMPORTANT] `frida_bridge.py` 版本锁定 + 已知匹配 bug | ||
| 68 | +> Mac 端跑 `frida_bridge.py` 必须用 **`frida==17.9.10` + `frida-tools==14.8.0`**(`phoneuse/.venv`,`start_frida_bridge.sh` 一键起 `adb reverse` + 起桥)——更新版本的 `frida-tools` 对 `frida` 有严格版本 range 依赖,配不上设备上跑的 `frida-server 17.9.10` 会直接报错。 | ||
| 69 | +> 现有进程匹配(`req.package in p.name`)按子串匹配 frida 报告的人类可读进程标签,不是包名——微信主进程标签是 `"WeChat"` 不含包名子串,会回退匹配到无 UI 的 `:push` 子进程,导致**静默空结果**(不报错,看起来像目标 App 没有 Activity/Bitmap)。`inspect_foreground_intent`/`dump_live_bitmaps` 已对空结果加诊断提示,根因(`frida_bridge.py` 本身)未修——影响面会牵连 `frida_call` 其它调用方,留作后续单独修。 | ||
| 70 | + | ||
| 71 | +### 已知阻塞 | ||
| 72 | + | ||
| 73 | +- 真机连续性差:开发时常掉 USB,涉及真机的验证要先确认 `adb devices` 能看到设备。 | ||
| 74 | +- 微信相关的 Frida/A11y 经验跟 `wechat_plain` 是分开积累的——`wechat_plain` 已验证过 Frida daemon 注入微信大 bundle 会触发反调试崩溃(见上文 §wechat_plain);`phoneuse` 走的是 USB attach 而非常驻 daemon,崩溃模式不一定一样,但动手前先看那段记录,别重新踩同一个坑。 | ||
| 75 | + | ||
| 37 | ## lifelog-recorder(Pixel 8 Pro 24/7 录音 + 本地中文 ASR + 本地 LLM,v0.8) | 76 | ## lifelog-recorder(Pixel 8 Pro 24/7 录音 + 本地中文 ASR + 本地 LLM,v0.8) |
| 38 | 77 | ||
| 39 | > 完整接手地图、踩坑清单、命令速查见 `lifelog-recorder/HANDOFF.md`。本节只勒住整个项目的**架构骨架 + 关键决策来源**,不重复细节。 | 78 | > 完整接手地图、踩坑清单、命令速查见 `lifelog-recorder/HANDOFF.md`。本节只勒住整个项目的**架构骨架 + 关键决策来源**,不重复细节。 |
phoneuse/TASKS.md
| @@ -170,6 +170,27 @@ | @@ -170,6 +170,27 @@ | ||
| 170 | - 修两处真机暴露的缺陷:① 发送前 `_wake_and_unlock()` 唤醒+解非安全锁屏(否则设备息屏时全部 tap 落空——已验证息屏起跑仍 `sent` msgId 4464); | 170 | - 修两处真机暴露的缺陷:① 发送前 `_wake_and_unlock()` 唤醒+解非安全锁屏(否则设备息屏时全部 tap 落空——已验证息屏起跑仍 `sent` msgId 4464); |
| 171 | ② `_capture_restore_ime()` 持久化用户真实 IME,防止上一轮中断把 ADBKeyboard 残留为默认输入法后被错误"还原"(修前实测残留过,已修复并回归)。 | 171 | ② `_capture_restore_ime()` 持久化用户真实 IME,防止上一轮中断把 ADBKeyboard 残留为默认输入法后被错误"还原"(修前实测残留过,已修复并回归)。 |
| 172 | 172 | ||
| 173 | +## 通用 Frida 深度检视(Intent / Bitmap,2026-06-24) | ||
| 174 | + | ||
| 175 | +> 目标:把"注入目标进程拿完整 Java 对象图"封装成两个**通用**(不需要逆向任何具体 App)MCP 工具, | ||
| 176 | +> 跟 S3-05/S3-06 的 per-app hook 路线区分开。字符串解密能力明确排除(per-app 逆向,超出本次范围)。 | ||
| 177 | + | ||
| 178 | +- [x] **MAC-01** 修通 Mac 端 Frida 链路(此前 S3-03 因 Mac 缺 frida pip 包而完全不可达) | ||
| 179 | + - 工作:`phoneuse/.venv` + `pip install frida==17.9.10 frida-tools==14.8.0 fastapi uvicorn`;新增 `start_frida_bridge.sh`(`adb reverse tcp:8767 tcp:8767` + 起 `frida_bridge.py`) | ||
| 180 | + - 🔬验证 2026-06-24:版本锁定经实测确认——`frida-tools` 最新版对 `frida` 有严格版本范围依赖,与设备上跑的 `frida-server 17.9.10` 不匹配会直接报错;`frida==17.9.10`+`frida-tools==14.8.0` 这组干净安装且匹配。venv 内 `import frida, frida_tools, fastapi, uvicorn` 通过。 | ||
| 181 | + | ||
| 182 | +- [!] **S3-08** 新增 `inspect_foreground_intent(package, timeout)` 工具 | ||
| 183 | + - 工作:新模块 `frida_inspect_tools.py`,`Java.choose('android.app.Activity', ...)` 枚举目标进程内所有存活 Activity,按 `hasWindowFocus()` 识别真正前台的那个,dump `getIntent()` 的 action/dataString/component/extras。 | ||
| 184 | + - 🔬JS 脚本本身的正确性已对一个真实在跑进程验证过(`hasWindowFocus` 准确识别出唯一前台 Activity,`extras` 正确提取出非空字段,Parcelable 类型 extras 按预期落到 Java 默认 `toString()`)。 | ||
| 185 | + - [!] 阻塞:本次会话设备从 `adb devices` 掉线,**MCP 工具本身(经 `deploy.sh` 推送 + 真实 `adb reverse` 隧道 + `android_mcp.py` 注册)的端到端验证未完成**,待设备重新连接后用 `com.android.settings`/Calculator 之类无隐私数据的单进程 App 补测。本地 `_build_server()` 已确认能正常注册此工具(60 个工具总数,无注册期异常)。 | ||
| 186 | + - 已知限制:`frida_bridge.py` 现有的进程名子串匹配对微信这类多进程 App 会匹配错进程(主进程标签是 `"WeChat"` 不含包名子串,回退逻辑会捡到无 UI 的 `:push` 子进程),导致静默空结果——工具已对空结果附加 `note` 诊断提示,但根因留作后续修复(不在本次范围内,怕影响 `frida_call` 其它调用方)。 | ||
| 187 | + | ||
| 188 | +- [!] **S3-09** 新增 `dump_live_bitmaps(package, max_export, timeout)` 工具 | ||
| 189 | + - 工作:同模块,`Java.choose('android.graphics.Bitmap', ...)` 枚举目标进程堆里所有存活 Bitmap,单次 `Java.choose` 内直接持有引用、按 `allocationByteCount` 排序,对最大的 `max_export` 张调 `compress(CompressFormat.PNG.value, 100, baos)` + `Base64.encodeToString(..., NO_WRAP)` 经 `send()` 回传。 | ||
| 190 | + - 🔬JS 脚本本身的正确性已对一个真实在跑进程验证过(导出的 base64 解码后是合法 PNG,magic bytes 正确,尺寸跟 `width`/`height` 字段一致);同时验证了两处易错点:`CompressFormat.PNG.value`(去掉 `.value` 会报 overload 不匹配)、回收过的 Bitmap 要整段跳过元数据读取(而非只在 `compress()` 处兜底,否则会有垂悬 0 值污染排序)。 | ||
| 191 | + - [!] 阻塞:同 S3-08,设备掉线,MCP 工具端到端验证待补。 | ||
| 192 | + - 已知限制:同 S3-08 的多进程匹配问题;另外单次调用只导出"当下堆里恰好有什么",不保证是调用者想要的那一张。 | ||
| 193 | + | ||
| 173 | --- | 194 | --- |
| 174 | 195 | ||
| 175 | # Sprint 4 — Memory Graph(中优先级,3-4 周) | 196 | # Sprint 4 — Memory Graph(中优先级,3-4 周) |
phoneuse/android_mcp.py
| @@ -1572,6 +1572,13 @@ def _build_server(): | @@ -1572,6 +1572,13 @@ def _build_server(): | ||
| 1572 | except Exception as exc: # pragma: no cover - keep server up if module/models missing | 1572 | except Exception as exc: # pragma: no cover - keep server up if module/models missing |
| 1573 | logger.warning("ocr_tools not registered: %s", exc) | 1573 | logger.warning("ocr_tools not registered: %s", exc) |
| 1574 | 1574 | ||
| 1575 | + # ── E6: Frida deep-runtime inspection (Intent/Bitmap) — independent module ── | ||
| 1576 | + try: | ||
| 1577 | + from frida_inspect_tools import register_frida_inspect_tools | ||
| 1578 | + register_frida_inspect_tools(mcp) | ||
| 1579 | + except Exception as exc: # pragma: no cover - keep server up if module missing | ||
| 1580 | + logger.warning("frida_inspect_tools not registered: %s", exc) | ||
| 1581 | + | ||
| 1575 | # ── E4: Code deploy tool (Mac → Pixel sync) ───────────────────────────────── | 1582 | # ── E4: Code deploy tool (Mac → Pixel sync) ───────────────────────────────── |
| 1576 | @mcp.tool() | 1583 | @mcp.tool() |
| 1577 | def deploy_phoneuse_code(restart: bool = True) -> str: | 1584 | def deploy_phoneuse_code(restart: bool = True) -> str: |
phoneuse/deploy.sh
| @@ -50,6 +50,7 @@ push_files() { | @@ -50,6 +50,7 @@ push_files() { | ||
| 50 | mitm_proxy.py | 50 | mitm_proxy.py |
| 51 | notification_monitor.py | 51 | notification_monitor.py |
| 52 | frida_bridge.py | 52 | frida_bridge.py |
| 53 | + frida_inspect_tools.py | ||
| 53 | ) | 54 | ) |
| 54 | 55 | ||
| 55 | log_info "Pushing ${#files[@]} files to Pixel..." | 56 | log_info "Pushing ${#files[@]} files to Pixel..." |
phoneuse/frida_inspect_tools.py
0 → 100644
| 1 | +""" | ||
| 2 | +Frida deep-runtime inspection MCP tools — generic, no per-app reverse engineering. | ||
| 3 | + | ||
| 4 | +Independent module (E6 / AgentPhone parity). Registered into the existing | ||
| 5 | +FastMCP instance by android_mcp._build_server() via: | ||
| 6 | + | ||
| 7 | + from frida_inspect_tools import register_frida_inspect_tools | ||
| 8 | + register_frida_inspect_tools(mcp) | ||
| 9 | + | ||
| 10 | +This is phoneuse.md's Layer 5 "Deep Runtime" (Frida), scoped deliberately | ||
| 11 | +narrow: only the two capabilities that work on ANY app without first | ||
| 12 | +reverse-engineering it. String-decryption hooking (the third capability | ||
| 13 | +discussed alongside these two) is NOT here — it requires locating a specific | ||
| 14 | +method in a specific app version, which is per-app work (see TASKS.md S3-05/ | ||
| 15 | +S3-06: WeChat's obfuscation already defeated that approach once). | ||
| 16 | + | ||
| 17 | + inspect_foreground_intent — dump the focused Activity's Intent (action/ | ||
| 18 | + data/component/extras) in a target process. | ||
| 19 | + dump_live_bitmaps — enumerate every live Bitmap in a target | ||
| 20 | + process's heap; export the largest N as PNG. | ||
| 21 | + | ||
| 22 | +Both inject a fixed, audited JS payload (not caller-supplied script) into the | ||
| 23 | +target process via the same Mac-side bridge the existing frida_call MCP tool | ||
| 24 | +uses (frida_bridge.py, POST http://127.0.0.1:8767/frida_call). On-device this | ||
| 25 | +reaches the Mac through `adb reverse tcp:8767 tcp:8767` (see | ||
| 26 | +start_frida_bridge.sh); requires frida-server running as root on the device | ||
| 27 | +and frida_bridge.py running on the Mac. | ||
| 28 | + | ||
| 29 | +Known limitation (in frida_bridge.py, not fixed here — out of scope for this | ||
| 30 | +module): its process-name matching does substring search against the | ||
| 31 | +human-readable process label frida reports, not the package name, so | ||
| 32 | +multi-process apps whose main process is labeled differently (e.g. WeChat's | ||
| 33 | +main process shows as "WeChat", not "com.tencent.mm") can resolve to a | ||
| 34 | +sub-process with no UI (e.g. "com.tencent.mm:push") instead of the real | ||
| 35 | +foreground process. Both tools below detect the resulting empty result and | ||
| 36 | +attach a `note` explaining this, instead of silently implying the target app | ||
| 37 | +has no Activities/Bitmaps. | ||
| 38 | +""" | ||
| 39 | + | ||
| 40 | +from __future__ import annotations | ||
| 41 | + | ||
| 42 | +import json | ||
| 43 | +import urllib.request | ||
| 44 | +from typing import Any | ||
| 45 | + | ||
| 46 | +BRIDGE_URL = "http://127.0.0.1:8767/frida_call" | ||
| 47 | + | ||
| 48 | +_INTENT_SCRIPT = """ | ||
| 49 | +Java.perform(function () { | ||
| 50 | + var results = []; | ||
| 51 | + Java.choose('android.app.Activity', { | ||
| 52 | + onMatch: function (instance) { | ||
| 53 | + try { | ||
| 54 | + var info = {}; | ||
| 55 | + info.className = instance.getClass().getName(); | ||
| 56 | + info.hasWindowFocus = instance.hasWindowFocus(); | ||
| 57 | + info.isFinishing = instance.isFinishing(); | ||
| 58 | + var intent = instance.getIntent(); | ||
| 59 | + if (intent !== null) { | ||
| 60 | + info.action = intent.getAction(); | ||
| 61 | + info.dataString = intent.getDataString(); | ||
| 62 | + var comp = intent.getComponent(); | ||
| 63 | + info.component = comp !== null ? comp.toString() : null; | ||
| 64 | + var extrasObj = {}; | ||
| 65 | + var bundle = intent.getExtras(); | ||
| 66 | + if (bundle !== null) { | ||
| 67 | + var iterator = bundle.keySet().iterator(); | ||
| 68 | + while (iterator.hasNext()) { | ||
| 69 | + var key = iterator.next(); | ||
| 70 | + try { | ||
| 71 | + var val = bundle.get(key); | ||
| 72 | + extrasObj[key] = val !== null ? val.toString() : null; | ||
| 73 | + } catch (innerErr) { | ||
| 74 | + extrasObj[key] = "<error: " + innerErr + ">"; | ||
| 75 | + } | ||
| 76 | + } | ||
| 77 | + } | ||
| 78 | + info.extras = extrasObj; | ||
| 79 | + } else { | ||
| 80 | + info.action = null; | ||
| 81 | + } | ||
| 82 | + results.push(info); | ||
| 83 | + } catch (e) { | ||
| 84 | + results.push({ error: e.toString() }); | ||
| 85 | + } | ||
| 86 | + }, | ||
| 87 | + onComplete: function () { | ||
| 88 | + send({ activities: results, count: results.length }); | ||
| 89 | + } | ||
| 90 | + }); | ||
| 91 | +}); | ||
| 92 | +""" | ||
| 93 | + | ||
| 94 | +_BITMAP_SCRIPT = """ | ||
| 95 | +Java.perform(function () { | ||
| 96 | + var MAX_EXPORT = __MAX_EXPORT__; | ||
| 97 | + var CompressFormat = Java.use('android.graphics.Bitmap$CompressFormat'); | ||
| 98 | + var ByteArrayOutputStream = Java.use('java.io.ByteArrayOutputStream'); | ||
| 99 | + var Base64 = Java.use('android.util.Base64'); | ||
| 100 | + var BASE64_NO_WRAP = 2; | ||
| 101 | + var candidates = []; | ||
| 102 | + | ||
| 103 | + Java.choose('android.graphics.Bitmap', { | ||
| 104 | + onMatch: function (bmp) { | ||
| 105 | + try { | ||
| 106 | + if (bmp.isRecycled()) return; | ||
| 107 | + candidates.push({ | ||
| 108 | + bmp: bmp, | ||
| 109 | + width: bmp.getWidth(), | ||
| 110 | + height: bmp.getHeight(), | ||
| 111 | + config: bmp.getConfig() !== null ? bmp.getConfig().toString() : null, | ||
| 112 | + allocationByteCount: bmp.getAllocationByteCount() | ||
| 113 | + }); | ||
| 114 | + } catch (e) {} | ||
| 115 | + }, | ||
| 116 | + onComplete: function () { | ||
| 117 | + candidates.sort(function (a, b) { return b.allocationByteCount - a.allocationByteCount; }); | ||
| 118 | + var top = candidates.slice(0, MAX_EXPORT); | ||
| 119 | + var exported = []; | ||
| 120 | + for (var i = 0; i < top.length; i++) { | ||
| 121 | + var c = top[i]; | ||
| 122 | + var item = { width: c.width, height: c.height, config: c.config, | ||
| 123 | + allocationByteCount: c.allocationByteCount }; | ||
| 124 | + try { | ||
| 125 | + var baos = ByteArrayOutputStream.$new(); | ||
| 126 | + item.compressOk = c.bmp.compress(CompressFormat.PNG.value, 100, baos); | ||
| 127 | + var bytes = baos.toByteArray(); | ||
| 128 | + item.pngByteLength = bytes.length; | ||
| 129 | + item.png_base64 = Base64.encodeToString(bytes, BASE64_NO_WRAP); | ||
| 130 | + } catch (e) { | ||
| 131 | + item.exportError = e.toString(); | ||
| 132 | + } | ||
| 133 | + exported.push(item); | ||
| 134 | + } | ||
| 135 | + send({ totalLiveBitmaps: candidates.length, exportedCount: exported.length, bitmaps: exported }); | ||
| 136 | + } | ||
| 137 | + }); | ||
| 138 | +}); | ||
| 139 | +""" | ||
| 140 | + | ||
| 141 | +_EMPTY_RESULT_NOTE = ( | ||
| 142 | + "Empty result. The target process may be a no-UI helper process rather " | ||
| 143 | + "than the app's main process — frida_bridge.py matches by substring " | ||
| 144 | + "against frida's process label (not the package name), which can pick " | ||
| 145 | + "the wrong process for multi-process apps (e.g. WeChat's main process is " | ||
| 146 | + "labeled 'WeChat', not 'com.tencent.mm', so the substring match falls " | ||
| 147 | + "back to a sub-process like 'com.tencent.mm:push'). This does not " | ||
| 148 | + "necessarily mean the app has no foreground Activity/Bitmaps." | ||
| 149 | +) | ||
| 150 | + | ||
| 151 | + | ||
| 152 | +def _frida_call(package: str, script: str, timeout: int) -> dict[str, Any]: | ||
| 153 | + """POST a JS payload to frida_bridge.py (Mac) via the adb-reverse tunnel.""" | ||
| 154 | + payload = json.dumps({"package": package, "script": script, "timeout": timeout}).encode() | ||
| 155 | + try: | ||
| 156 | + req = urllib.request.Request( | ||
| 157 | + BRIDGE_URL, data=payload, headers={"Content-Type": "application/json"} | ||
| 158 | + ) | ||
| 159 | + with urllib.request.urlopen(req, timeout=timeout + 10) as resp: | ||
| 160 | + return json.loads(resp.read().decode()) | ||
| 161 | + except Exception as e: | ||
| 162 | + return { | ||
| 163 | + "error": f"frida_bridge unreachable: {e}. Ensure start_frida_bridge.sh is " | ||
| 164 | + f"running on the Mac and adb reverse tcp:8767 tcp:8767 is set." | ||
| 165 | + } | ||
| 166 | + | ||
| 167 | + | ||
| 168 | +def register_frida_inspect_tools(mcp) -> None: | ||
| 169 | + """Register the generic Frida deep-runtime inspection tools onto an existing FastMCP.""" | ||
| 170 | + | ||
| 171 | + @mcp.tool() | ||
| 172 | + def inspect_foreground_intent(package: str, timeout: int = 10) -> str: | ||
| 173 | + """ | ||
| 174 | + Inspect the focused Activity's Intent in a running app via Frida. | ||
| 175 | + | ||
| 176 | + Injects a fixed, generic script (no per-app reverse engineering) that | ||
| 177 | + enumerates every live android.app.Activity instance in the target | ||
| 178 | + process and returns each one's class name, hasWindowFocus/isFinishing | ||
| 179 | + state, and Intent (action, data, component, all Bundle extras | ||
| 180 | + stringified). The entry with hasWindowFocus=true is the true | ||
| 181 | + foreground Activity; others are backgrounded-but-alive. | ||
| 182 | + | ||
| 183 | + Requires root + frida-server on device, and frida_bridge.py running | ||
| 184 | + on the Mac (start_frida_bridge.sh) reachable via adb reverse | ||
| 185 | + tcp:8767 tcp:8767. | ||
| 186 | + | ||
| 187 | + Note: complex (Parcelable/Serializable) extras only show Java's | ||
| 188 | + default toString() (e.g. "SomeClass@a1b2c3"), not a deep dump. | ||
| 189 | + Multi-process apps may attach to the wrong sub-process — see this | ||
| 190 | + module's docstring. | ||
| 191 | + | ||
| 192 | + Args: | ||
| 193 | + package: Target app package name, e.g. 'com.android.settings'. | ||
| 194 | + timeout: Max seconds to wait for script output (default 10). | ||
| 195 | + """ | ||
| 196 | + result = _frida_call(package, _INTENT_SCRIPT, timeout) | ||
| 197 | + if "error" in result: | ||
| 198 | + return json.dumps(result, ensure_ascii=False) | ||
| 199 | + output = result.get("output") or [] | ||
| 200 | + activities = output[0].get("activities", []) if output else [] | ||
| 201 | + if not activities: | ||
| 202 | + result["note"] = _EMPTY_RESULT_NOTE | ||
| 203 | + return json.dumps(result, ensure_ascii=False) | ||
| 204 | + | ||
| 205 | + @mcp.tool() | ||
| 206 | + def dump_live_bitmaps(package: str, max_export: int = 3, timeout: int = 15) -> str: | ||
| 207 | + """ | ||
| 208 | + Enumerate live (non-recycled) Bitmap objects in a running app's heap | ||
| 209 | + via Frida, and export the largest few as base64-encoded PNGs. | ||
| 210 | + | ||
| 211 | + Injects a fixed, generic script (no per-app reverse engineering) that | ||
| 212 | + walks the target process's heap for every live android.graphics.Bitmap, | ||
| 213 | + collects width/height/config/allocationByteCount for all of them, then | ||
| 214 | + PNG-compresses and base64-encodes only the `max_export` largest ones | ||
| 215 | + (compression happens inside the target app's own process — keep this | ||
| 216 | + small; it is slow and memory-heavy for large bitmaps). | ||
| 217 | + | ||
| 218 | + Requires root + frida-server on device, and frida_bridge.py running | ||
| 219 | + on the Mac (start_frida_bridge.sh) reachable via adb reverse | ||
| 220 | + tcp:8767 tcp:8767. | ||
| 221 | + | ||
| 222 | + Note: this only captures whatever happens to be allocated in memory | ||
| 223 | + at the moment of the call — not a guarantee of "the photo you | ||
| 224 | + wanted," just a snapshot. Multi-process apps may attach to the wrong | ||
| 225 | + sub-process — see this module's docstring. | ||
| 226 | + | ||
| 227 | + Args: | ||
| 228 | + package: Target app package name, e.g. 'com.android.settings'. | ||
| 229 | + max_export: How many of the largest live bitmaps to export as PNG | ||
| 230 | + (default 3, keep small — see note above). | ||
| 231 | + timeout: Max seconds to wait for script output (default 15). | ||
| 232 | + """ | ||
| 233 | + script = _BITMAP_SCRIPT.replace("__MAX_EXPORT__", str(int(max_export))) | ||
| 234 | + result = _frida_call(package, script, timeout) | ||
| 235 | + if "error" in result: | ||
| 236 | + return json.dumps(result, ensure_ascii=False) | ||
| 237 | + output = result.get("output") or [] | ||
| 238 | + total = output[0].get("totalLiveBitmaps", 0) if output else 0 | ||
| 239 | + if not total: | ||
| 240 | + result["note"] = _EMPTY_RESULT_NOTE | ||
| 241 | + return json.dumps(result, ensure_ascii=False) |
phoneuse/start_frida_bridge.sh
0 → 100755
| 1 | +#!/bin/bash | ||
| 2 | +# Start frida_bridge.py on Mac: tunnels device->Mac for android_mcp.py's | ||
| 3 | +# frida_call / frida_inspect_tools, then runs the bridge in this venv | ||
| 4 | +# (pinned frida==17.9.10 to match the on-device frida-server version). | ||
| 5 | +set -euo pipefail | ||
| 6 | +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" | ||
| 7 | +ADB="${SCRIPT_DIR}/../platform-tools/adb" | ||
| 8 | + | ||
| 9 | +"$ADB" reverse tcp:8767 tcp:8767 | ||
| 10 | +exec "${SCRIPT_DIR}/.venv/bin/python3" "${SCRIPT_DIR}/frida_bridge.py" |