Commit 263efd3399f89c5f7641a33e5d3779b022ad7497

Authored by yibo
1 parent 0f730253

feat(lifelog/photolog): 全相册自动索引 — 分层管线 + 搜图 + VL 暂停/停止

把整个手机相册自动跑成可搜索内容(v0.8)。本地 VL ~30-40s/张,
全量是按天计的长任务 → 分层:快的全量先铺,慢的 VL 排队慢慢补。

- GalleryScanner:MediaStore 游标扫全量,content:// uri 原地登记不拷文件,
  mediaId 去重增量;photos 加 source/mediaId/scanState/captionEmbedding(Room v14)
- Tier-0 PhotoFaceWorker:扫描 + ML Kit 人脸 + 512d emb + GPS,秒级全量铺,
  battery-not-low;Tier-1 PhotoVlWorker:VL 场景描述 + caption 语义向量,
  优先级队列(有人脸>近期),充电时跑,尊重暂停标志
- 搜图:关键词(caption+人名 LIKE) + 语义(caption 向量余弦,EmbedClient 打
  llama-swap embed 模型;未配置优雅降级)
- VL 控制(PhotoVlState):运行中「暂停/停止」、空闲「立即理解/继续」切换,
  正在处理的格子转圈,格子角标 待索引/待理解/已描述
- 修 ImageUtil.decodeUri:bounds 预解码 decodeStream 在 inJustDecodeBounds
  下按设计返回 null,原 `?:` 误判致每张提前 return(人脸全跑空)

迁移 13→14 纯增量:老的已处理照片直接标 scanState=2 不重跑。

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Showing 19 changed files with 921 additions and 99 deletions
lifelog-recorder/HANDOFF.md
@@ -70,10 +70,26 @@ app/src/main/kotlin/com/lemon/lifelog/ @@ -70,10 +70,26 @@ app/src/main/kotlin/com/lemon/lifelog/
70 │ ├── HermesClient.kt — OpenAI 兼容(纪要 worker 用,指本地 llama-swap;readTimeout 600s) 70 │ ├── HermesClient.kt — OpenAI 兼容(纪要 worker 用,指本地 llama-swap;readTimeout 600s)
71 │ ├── ModelDownloader.kt — 首次启动下模型(paraformer 232 MB + speaker 28 MB) 71 │ ├── ModelDownloader.kt — 首次启动下模型(paraformer 232 MB + speaker 28 MB)
72 │ └── XfyunIatClient.kt — 讯飞大模型 WS 客户端,仅 fallback / 云复核用 72 │ └── XfyunIatClient.kt — 讯飞大模型 WS 客户端,仅 fallback / 云复核用
  73 +├── PhotologScreen.kt — 「人物」+「相册」一级 tab(通讯录/档案/照片网格/全相册索引+搜索+VL 控制)
  74 +├── PhotologViewModel.kt — photolog 总线(导入/标注/合并/绑声纹/AI 印象/搜图/VL 控制)
  75 +├── PersonProfileScreen.kt — 联系人档案(照片/录音/微信聊天/朋友圈/AI 印象,tab 式 + 记录搜索)
  76 +├── vision/ — ★ photolog 视觉层(人脸 + 全相册自动索引,见 §四下半)
  77 +│ ├── GalleryScanner.kt — 全相册 MediaStore 游标扫描,content:// uri 原地登记(不拷文件),mediaId 去重增量
  78 +│ ├── PhotoFaceWorker.kt — Tier-0:扫描 + ML Kit 人脸 + 512d emb + GPS(快,battery-not-low)
  79 +│ ├── PhotoVlWorker.kt — Tier-1:Qwen-VL 场景描述 + caption 语义向量(慢,充电时;尊重暂停标志)
  80 +│ ├── PhotoVlState.kt — ★ VL 运行态 singleton(running/currentPhotoId/paused 持久化),UI 控制按钮 + 单张转圈
  81 +│ ├── PhotoScheduler.kt — 两条周期 worker(face 15min / vl 15min+charging)+ runNow/runVlNow/cancelVl
  82 +│ ├── PhotoProcessor.kt — faceStage(Tier-0)/ captionStage(Tier-1),按来源解码
  83 +│ ├── FaceEngine.kt — ML Kit 检测+5点对齐 → TFLite MobileFaceNet 512d(非 onnx,避与 sherpa 冲突)
  84 +│ ├── FaceMatcher.kt — 人脸归属(人物锚点+已标注脸 gallery,余弦 0.45)+ backfill
  85 +│ ├── VisionClient.kt — llama-swap "vl" 模型出场景描述(不报人名)
  86 +│ ├── EmbedClient.kt — llama-swap "embed" 模型出 caption 文本向量(语义搜;未配置则优雅降级)
  87 +│ ├── ImageUtil.kt — EXIF/解码/缩放/embedding 打包;decode 按 source 走 file 或 content uri
  88 +│ └── WeChatEnrollWorker.kt — 微信头像 → 人脸锚点入库
73 └── data/ 89 └── data/
74 - ├── Entities.kt — Segment / Transcript / Summary / SpeakerProfile / TranscriptFts / ChatConversation / ChatMessage  
75 - ├── Daos.kt — observeTimeline + searchTimeline(FTS4) + Speaker/Summary/ChatDao + 范围查询  
76 - ├── AppDatabase.kt — Room v3,addMigrations(MIGRATION_2_3),★ 已删 fallbackToDestructiveMigration(改 schema 不写迁移会崩,绝不静默清库) 90 + ├── Entities.kt — Segment / Transcript / Summary / SpeakerProfile / TranscriptFts / Chat* / Person / Photo / Face / Moment / WechatMessage
  91 + ├── Daos.kt — observeTimeline + searchTimeline(FTS4) + Photo/Face/Person/... DAO;★ photos 分层队列(pendingFace/pendingCaption) + observeProgress + searchByKeyword + captionVecs
  92 + ├── AppDatabase.kt — Room **v14**,addMigrations(... MIGRATION_13_14),★ 已删 fallbackToDestructiveMigration(改 schema 不写迁移会崩,绝不静默清库)
77 └── Repository.kt 93 └── Repository.kt
78 ``` 94 ```
79 95
@@ -188,6 +204,43 @@ VAD 参数(在 `RecorderService.runPipeline` 里调): @@ -188,6 +204,43 @@ VAD 参数(在 `RecorderService.runPipeline` 里调):
188 204
189 --- 205 ---
190 206
  207 +### photolog 全相册自动索引(v0.8,2026-06-09)
  208 +
  209 +把手机**整个相册**自动跑成可搜索内容。核心矛盾:本地 VL(Qwen2.5-VL-3B)真机 **~30-40s/张**,几千张全量 VL 是按天计的长任务 → **分层(tiered)**:快的全量先铺,慢的 VL 排队慢慢补。
  210 +
  211 +```
  212 +GalleryScanner.scan() ── MediaStore 游标遍历全部图片
  213 + └─ 只存 content:// uri(不拷文件!全相册拷一份会撑爆私有目录)+ mediaId 去重 + DATE_TAKEN
  214 + → photos 行 (source='gallery', scanState=0)
  215 + │
  216 + ┌──────────────────────────┴───────────────────────────┐
  217 +Tier-0 PhotoFaceWorker (15min, battery-not-low) Tier-1 PhotoVlWorker (15min, +charging)
  218 + 每轮先 scan() 增量登记,再处理 scanState=0: 处理 scanState=1,优先级队列(有人脸>近期):
  219 + ├─ ImageUtil.decode(ctx,photo) ← 按 source 解码 ├─ VisionClient.caption(jpeg 448px) → 场景描述
  220 + ├─ FaceEngine 人脸+512d emb + FaceMatcher 认人 ├─ EmbedClient.embed(caption) → 语义向量(可选)
  221 + ├─ ImageUtil.readGps (ACCESS_MEDIA_LOCATION) └─ markCaptionDone → scanState=2 + processedAt
  222 + └─ markFaceDone → scanState=1 失败 markCaptionSkipped(=2,caption 空) 不卡队列
  223 + 秒级,几小时铺满全库 → 立刻可按人/时间搜 每张 7min 预算 + retry,避开 WM 10min 上限
  224 +```
  225 +
  226 +**scanState 状态机**:`0 待索引 → 1 人脸/时间已铺 → 2 已出场景描述`。迁移 13→14 把老的已处理照片(`processedAt` 非空)直接标 2 不重跑;imported 源(手动导入,拷进私有目录)走同一管线,只是 `source='imported'` 且 `decode` 走 file path。
  227 +
  228 +**搜索**:① 关键词 `searchByKeyword` = caption + 人名 LIKE 子串(中文子串比 FTS4 unicode61 靠谱,跟 transcript 的 searchTextLike 一致);② 语义 `searchSemantic` = embed query → 全库 `captionVecs()` 余弦 top-K(Kotlin 算,floor 0.20)。语义要 llama-swap 配 `embed` 模型(`--embedding` 的中文 GGUF,如 bge-small-zh),没配 `EmbedClient` 抛 IOException,UI 提示「未配置 embed 端点」,不影响关键词搜。
  229 +
  230 +**VL 控制(PhotoVlState)**:worker 推 `running`/`currentPhotoId`,相册页据此:
  231 +- 运行中 → 顶部「⏸ 暂停」「⏹ 停止」+ 正在 VL 的那格半透明遮罩转圈;
  232 +- 空闲 → 「⚡ 立即理解」(暂停且未跑完 → 「▶ 继续理解」);
  233 +- **暂停** = `paused` 持久化到 prefs,worker 每张前 `isPaused()` 检查 break,**连充电时的周期 worker 也尊重**,需「继续」才恢复;**停止** = cancelVl 取消手动这轮 + 清 paused(之后充电仍自动补完)。
  234 +- 格子角标:`👤N` 人脸数 / `📝` 已出描述(可搜) / `待索引·待理解·无描述` 未处理态。
  235 +
  236 +> ⚠️ 关键坑(已修):`ImageUtil.decodeUri` 里 bounds 预解码用 `inJustDecodeBounds=true`,`BitmapFactory.decodeStream` **按设计返回 null**(只填 bounds)——不能拿它的返回值判失败,否则**每张都提前 return null**,人脸全跑空。`?:` 要判 `openInputStream` 的 null。
  237 +
  238 +> ⚠️ WM worker 互斥:periodic + 手动 + retry 可能并发,多 VL worker 打同一 llama-server 会 502 → `PhotoVlWorker`/`PhotoFaceWorker` 各有 `AtomicBoolean isRunning` 守。`PhotoVlState.running` 只由拿到锁那个置位。
  239 +
  240 +> 全相册扫描没有 sqlite3 在设备上:dev 改 photos 表用 `adb exec-out run-as com.lemon.lifelog cat databases/lifelog.db{,-wal,-shm}` 拉到 Mac,`sqlite3 'PRAGMA wal_checkpoint(TRUNCATE);'` 合并 WAL 后改,再 `run-as sh -c 'cat > databases/lifelog.db'` 推回 + 删 wal/shm。
  241 +
  242 +---
  243 +
191 ## §五 §六 待办(按优先级) 244 ## §五 §六 待办(按优先级)
192 245
193 ✅ **已完成**: 246 ✅ **已完成**:
@@ -196,7 +249,8 @@ VAD 参数(在 `RecorderService.runPipeline` 里调): @@ -196,7 +249,8 @@ VAD 参数(在 `RecorderService.runPipeline` 里调):
196 - v0.4 (2026-06):段播放 (MediaPlayer Opus) / bottom sheet 操作菜单 / enroll 后自动回填老段 249 - v0.4 (2026-06):段播放 (MediaPlayer Opus) / bottom sheet 操作菜单 / enroll 后自动回填老段
197 - v0.5 (2026-06):hermes api_server 端点 + SummaryWorker + 纪要 UI section 250 - v0.5 (2026-06):hermes api_server 端点 + SummaryWorker + 纪要 UI section
198 - v0.6 (2026-06-04):**声纹在线聚类**(SpeakerClusterer,自动「说话人N」+ reclusterAll)/ **5-tab UI**(录音/对话/时间轴/声纹/纪要,时间轴全 CRUD + 批量删导出 + 按说话人/时间/未匹配筛选 + 排序)/ **真暂停**(停采集留模型)/ **会话级纪要**(按时间间隔切会话、可调阈值、要点带 [HH:mm:ss] 点击跳回时间轴、折叠头显示说话人名单)/ **App 内对话**(本地 2B/4B + Hermes 多端点流式)/ **本地 LLM 栈**(llama.cpp HEAD + llama-swap 8080,纪要+对话全本地,见 §六)/ **Room v3 迁移**(去 destructive 改 fail-loud) 251 - v0.6 (2026-06-04):**声纹在线聚类**(SpeakerClusterer,自动「说话人N」+ reclusterAll)/ **5-tab UI**(录音/对话/时间轴/声纹/纪要,时间轴全 CRUD + 批量删导出 + 按说话人/时间/未匹配筛选 + 排序)/ **真暂停**(停采集留模型)/ **会话级纪要**(按时间间隔切会话、可调阈值、要点带 [HH:mm:ss] 点击跳回时间轴、折叠头显示说话人名单)/ **App 内对话**(本地 2B/4B + Hermes 多端点流式)/ **本地 LLM 栈**(llama.cpp HEAD + llama-swap 8080,纪要+对话全本地,见 §六)/ **Room v3 迁移**(去 destructive 改 fail-loud)
199 -- v0.7 (2026-06-07):**block-based diarization-first 录音管线**(攒块→整块 pyannote 分轮→逐 turn 转写+嵌入+认人,turn=时间轴一行;去实时字幕;BlockBuffer/BlockProcessor;崩溃 listPending 续跑;commit 07f3d22)/ **声纹纠偏 L1+L2**(L1=reclusterAll & assign 冻结命名 anchor 不漂移止串人;L2=重命名/合并时 rebuildAnchor 用该人全部段均值重建多 anchor)/ 阈值默认 0.45、设置页 0.35-0.65 可调 252 +- v0.7 (2026-06-07):**block-based diarization-first 录音管线**(攒块→整块 pyannote 分轮→逐 turn 转写+嵌入+认人,turn=时间轴一行;去实时字幕;BlockBuffer/BlockProcessor;崩溃 listPending 续跑;commit 07f3d22)/ **声纹纠偏 L1+L2**(L1=reclusterAll & assign 冻结命名 anchor 不漂移止串人;L2=重命名/合并时 rebuildAnchor 用该人全部段均值重建多 anchor)/ 阈值默认 0.45、设置页 0.35-0.65 可调 / **photolog 人脉 CRM 层**(人物档案 + 相册人脸聚类 + 微信通讯录/朋友圈/AI 印象)
  253 +- v0.8 (2026-06-09):**全相册自动索引 + 分层管线**(GalleryScanner 扫 MediaStore 全量、content uri 原地登记不拷文件;Tier-0 人脸/时间秒级全量铺、Tier-1 VL 充电时排队补;Room v14;详见 §四 photolog 段)/ **搜图**(关键词 caption+人名 LIKE / 语义 caption 向量余弦,embed 走 llama-swap)/ **VL 控制**(PhotoVlState:暂停/继续/停止切换 + 单张转圈 + 格子状态角标)
200 254
201 剩下: 255 剩下:
202 256
@@ -315,6 +369,9 @@ openrouter/nous metadata 超时),对开机服务可接受。 @@ -315,6 +369,9 @@ openrouter/nous metadata 超时),对开机服务可接受。
315 | hf-mirror.com 偶尔 truncate | HF mirror 限速 / 中断 | `ModelDownloader` 已带断点续传 | 369 | hf-mirror.com 偶尔 truncate | HF mirror 限速 / 中断 | `ModelDownloader` 已带断点续传 |
316 | `am start-foreground-service` 返回 "Requires permission not exported" | 服务 `android:exported="false"`,adb shell 没权 | 用 `su -c "am ..."` 提权 | 370 | `am start-foreground-service` 返回 "Requires permission not exported" | 服务 `android:exported="false"`,adb shell 没权 | 用 `su -c "am ..."` 提权 |
317 | `adb shell 'su 10288 -c "..."` 网络全断 | Magisk su 没传 supplementary groups | 加 `-G 3003 -G 9997 -G 20<n> -G 50<n>` | 371 | `adb shell 'su 10288 -c "..."` 网络全断 | Magisk su 没传 supplementary groups | 加 `-G 3003 -G 9997 -G 20<n> -G 50<n>` |
  372 +| 全相册照片人脸全 0、`decode failed (face)` 刷屏 | `decodeUri` bounds 预解码处 `decodeStream(inJustDecodeBounds=true)` **按设计返回 null**,被 `?:` 当失败提前 return | `?:` 改判 `openInputStream` 的 null,不是 `decodeStream` 返回值(已修,见 `ImageUtil.decodeUri`) |
  373 +| 语义搜 `embed 404: no router for requested model` | llama-swap 没配 `embed` 模型 | config.yaml 加一个 `embed` model(`llama-server --embedding` + 中文 GGUF);不配则只用关键词搜 |
  374 +| VL 进度不动 / 点「立即理解」像没反应 | 单张 VL ~30-40s(含 llama-swap 热加载 vl 模型),且要 battery-not-low | 正常现象;看「场景描述 N/总」进度条;想自动补完插充电(周期 worker 要 charging) |
318 375
319 --- 376 ---
320 377
lifelog-recorder/app/src/main/AndroidManifest.xml
@@ -12,6 +12,12 @@ @@ -12,6 +12,12 @@
12 <uses-permission android:name="android.permission.INTERNET" /> 12 <uses-permission android:name="android.permission.INTERNET" />
13 <uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" /> 13 <uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
14 <uses-permission android:name="android.permission.READ_CONTACTS" /> 14 <uses-permission android:name="android.permission.READ_CONTACTS" />
  15 + <!-- 全相册自动索引:读图片(13+ 用 READ_MEDIA_IMAGES,12 及以下用 READ_EXTERNAL_STORAGE) -->
  16 + <uses-permission android:name="android.permission.READ_MEDIA_IMAGES" />
  17 + <uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE"
  18 + android:maxSdkVersion="32" />
  19 + <!-- 读照片 EXIF GPS(脱敏原图需要) -->
  20 + <uses-permission android:name="android.permission.ACCESS_MEDIA_LOCATION" />
15 21
16 <application 22 <application
17 android:name=".LifelogApp" 23 android:name=".LifelogApp"
lifelog-recorder/app/src/main/kotlin/com/lemon/lifelog/LifelogApp.kt
@@ -23,6 +23,7 @@ class LifelogApp : Application() { @@ -23,6 +23,7 @@ class LifelogApp : Application() {
23 SummaryScheduler.schedulePeriodic(this) 23 SummaryScheduler.schedulePeriodic(this)
24 // 相册:未处理照片的周期后台处理(人脸+VL),空闲时跑,不阻塞导入。 24 // 相册:未处理照片的周期后台处理(人脸+VL),空闲时跑,不阻塞导入。
25 com.lemon.lifelog.vision.PhotoScheduler.schedulePeriodic(this) 25 com.lemon.lifelog.vision.PhotoScheduler.schedulePeriodic(this)
  26 + com.lemon.lifelog.vision.PhotoVlState.init(this) // 载入持久化的「暂停」标志
26 // 存储 TTL:每天清理超期录音 .opus(文字永留);默认关闭,设置页选保留天数才生效。 27 // 存储 TTL:每天清理超期录音 .opus(文字永留);默认关闭,设置页选保留天数才生效。
27 com.lemon.lifelog.storage.StorageScheduler.schedulePeriodic(this) 28 com.lemon.lifelog.storage.StorageScheduler.schedulePeriodic(this)
28 } 29 }
lifelog-recorder/app/src/main/kotlin/com/lemon/lifelog/PersonProfileScreen.kt
@@ -148,7 +148,7 @@ internal fun PersonProfileScreen( @@ -148,7 +148,7 @@ internal fun PersonProfileScreen(
148 ) { 148 ) {
149 gridItems(fPhotos, key = { it.id }) { ph -> 149 gridItems(fPhotos, key = { it.id }) { ph ->
150 AsyncImage( 150 AsyncImage(
151 - model = File(ph.path), contentDescription = ph.caption, contentScale = ContentScale.Crop, 151 + model = photoModel(ph.source, ph.path), contentDescription = ph.caption, contentScale = ContentScale.Crop,
152 modifier = Modifier.aspectRatio(1f).clip(RoundedCornerShape(6.dp)).clickable { onOpenPhoto(ph.id) }, 152 modifier = Modifier.aspectRatio(1f).clip(RoundedCornerShape(6.dp)).clickable { onOpenPhoto(ph.id) },
153 ) 153 )
154 } 154 }
lifelog-recorder/app/src/main/kotlin/com/lemon/lifelog/PhotologScreen.kt
@@ -2,6 +2,8 @@ package com.lemon.lifelog @@ -2,6 +2,8 @@ package com.lemon.lifelog
2 2
3 import android.Manifest 3 import android.Manifest
4 import android.content.pm.PackageManager 4 import android.content.pm.PackageManager
  5 +import android.net.Uri
  6 +import android.os.Build
5 import androidx.activity.compose.rememberLauncherForActivityResult 7 import androidx.activity.compose.rememberLauncherForActivityResult
6 import androidx.activity.result.PickVisualMediaRequest 8 import androidx.activity.result.PickVisualMediaRequest
7 import androidx.activity.result.contract.ActivityResultContracts 9 import androidx.activity.result.contract.ActivityResultContracts
@@ -22,6 +24,7 @@ import androidx.compose.foundation.layout.fillMaxWidth @@ -22,6 +24,7 @@ import androidx.compose.foundation.layout.fillMaxWidth
22 import androidx.compose.foundation.layout.height 24 import androidx.compose.foundation.layout.height
23 import androidx.compose.foundation.layout.padding 25 import androidx.compose.foundation.layout.padding
24 import androidx.compose.foundation.layout.size 26 import androidx.compose.foundation.layout.size
  27 +import androidx.compose.foundation.layout.width
25 import androidx.compose.foundation.shape.CircleShape 28 import androidx.compose.foundation.shape.CircleShape
26 import androidx.compose.ui.draw.clip 29 import androidx.compose.ui.draw.clip
27 import androidx.compose.ui.window.Dialog 30 import androidx.compose.ui.window.Dialog
@@ -41,6 +44,7 @@ import androidx.compose.material3.CircularProgressIndicator @@ -41,6 +44,7 @@ import androidx.compose.material3.CircularProgressIndicator
41 import androidx.compose.material3.ExperimentalMaterial3Api 44 import androidx.compose.material3.ExperimentalMaterial3Api
42 import androidx.compose.material3.FilterChip 45 import androidx.compose.material3.FilterChip
43 import androidx.compose.material3.HorizontalDivider 46 import androidx.compose.material3.HorizontalDivider
  47 +import androidx.compose.material3.LinearProgressIndicator
44 import androidx.compose.material3.MaterialTheme 48 import androidx.compose.material3.MaterialTheme
45 import androidx.compose.material3.ModalBottomSheet 49 import androidx.compose.material3.ModalBottomSheet
46 import androidx.compose.material3.OutlinedButton 50 import androidx.compose.material3.OutlinedButton
@@ -70,9 +74,12 @@ import androidx.lifecycle.viewmodel.compose.viewModel @@ -70,9 +74,12 @@ import androidx.lifecycle.viewmodel.compose.viewModel
70 import coil.compose.AsyncImage 74 import coil.compose.AsyncImage
71 import com.lemon.lifelog.data.Face 75 import com.lemon.lifelog.data.Face
72 import com.lemon.lifelog.data.Person 76 import com.lemon.lifelog.data.Person
  77 +import com.lemon.lifelog.data.PhotoProgress
73 import com.lemon.lifelog.data.PhotoRow 78 import com.lemon.lifelog.data.PhotoRow
74 import com.lemon.lifelog.data.Repository 79 import com.lemon.lifelog.data.Repository
  80 +import com.lemon.lifelog.vision.PhotoVlState
75 import kotlinx.coroutines.Dispatchers 81 import kotlinx.coroutines.Dispatchers
  82 +import kotlinx.coroutines.delay
76 import kotlinx.coroutines.flow.flowOf 83 import kotlinx.coroutines.flow.flowOf
77 import kotlinx.coroutines.withContext 84 import kotlinx.coroutines.withContext
78 import java.io.File 85 import java.io.File
@@ -122,38 +129,121 @@ internal fun ContactsTab(repo: Repository) { @@ -122,38 +129,121 @@ internal fun ContactsTab(repo: Repository) {
122 } 129 }
123 } 130 }
124 131
125 -/** 「相册」一级 tab:照片网格(导入 + 详情 + 标注)。 */ 132 +/** PhotoRow → Coil model:gallery 源是 content uri,imported 源是私有文件。 */
  133 +internal fun photoModel(source: String, path: String): Any =
  134 + if (source == "gallery") Uri.parse(path) else File(path)
  135 +
  136 +/** 「相册」一级 tab:全相册自动索引 + 进度 + 搜索 + 网格(详情 + 标注)。 */
126 @OptIn(ExperimentalMaterial3Api::class) 137 @OptIn(ExperimentalMaterial3Api::class)
127 @Composable 138 @Composable
128 internal fun AlbumTab(repo: Repository) { 139 internal fun AlbumTab(repo: Repository) {
129 val vm: PhotologViewModel = viewModel() 140 val vm: PhotologViewModel = viewModel()
  141 + val ctx = LocalContext.current
130 var detailPhotoId by remember { mutableStateOf<Long?>(null) } 142 var detailPhotoId by remember { mutableStateOf<Long?>(null) }
131 val persons by vm.repo.persons.observeAll().collectAsState(initial = emptyList()) 143 val persons by vm.repo.persons.observeAll().collectAsState(initial = emptyList())
132 - val pendingCount by vm.repo.photos.observeUnprocessedCount().collectAsState(initial = 0) 144 + val progress by vm.repo.photos.observeProgress().collectAsState(initial = PhotoProgress(0, 0, 0))
133 val photos by remember { vm.repo.photos.observeGrid(1000) }.collectAsState(initial = emptyList()) 145 val photos by remember { vm.repo.photos.observeGrid(1000) }.collectAsState(initial = emptyList())
  146 + // VL 运行态:控制按钮切换 + 单张处理指示。
  147 + val vlRunning by PhotoVlState.running.collectAsState()
  148 + val vlPaused by PhotoVlState.paused.collectAsState()
  149 + val vlCurrentId by PhotoVlState.currentPhotoId.collectAsState()
  150 +
  151 + // 读相册权限(13+ READ_MEDIA_IMAGES;12 及以下 READ_EXTERNAL_STORAGE)。授予即扫全相册。
  152 + val mediaPerm = if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.TIRAMISU)
  153 + Manifest.permission.READ_MEDIA_IMAGES else Manifest.permission.READ_EXTERNAL_STORAGE
  154 + var granted by remember {
  155 + mutableStateOf(ContextCompat.checkSelfPermission(ctx, mediaPerm) == PackageManager.PERMISSION_GRANTED)
  156 + }
  157 + val permLauncher = rememberLauncherForActivityResult(
  158 + ActivityResultContracts.RequestMultiplePermissions(),
  159 + ) { res -> granted = res[mediaPerm] == true }
  160 + LaunchedEffect(granted) { if (granted) vm.scanGallery() }
  161 + // 读图片 + 读 EXIF GPS(ACCESS_MEDIA_LOCATION 是独立的危险权限,要一起请求)。
  162 + val requestPerms = remember(mediaPerm) {
  163 + arrayOf(mediaPerm, Manifest.permission.ACCESS_MEDIA_LOCATION)
  164 + }
  165 +
  166 + // 搜索:results=null 显示全部网格;非 null 显示搜索结果。
  167 + var query by remember { mutableStateOf("") }
  168 + var results by remember { mutableStateOf<List<PhotoRow>?>(null) }
  169 + var searchMsg by remember { mutableStateOf<String?>(null) }
  170 + // 关键词即时搜(去抖 300ms)。语义搜走显式按钮。
  171 + LaunchedEffect(query) {
  172 + if (query.isBlank()) { results = null; searchMsg = null; return@LaunchedEffect }
  173 + delay(300)
  174 + vm.searchKeyword(query) { results = it; searchMsg = "关键词命中 ${it.size}" }
  175 + }
134 176
135 val picker = rememberLauncherForActivityResult( 177 val picker = rememberLauncherForActivityResult(
136 ActivityResultContracts.PickMultipleVisualMedia(30), 178 ActivityResultContracts.PickMultipleVisualMedia(30),
137 ) { uris -> vm.importUris(uris) } 179 ) { uris -> vm.importUris(uris) }
138 180
  181 + val shown = results ?: photos
  182 +
139 Column(modifier = Modifier.fillMaxSize()) { 183 Column(modifier = Modifier.fillMaxSize()) {
  184 + // 权限 + 进度。
  185 + if (!granted) {
  186 + Row(
  187 + modifier = Modifier.fillMaxWidth().padding(horizontal = 12.dp, vertical = 6.dp),
  188 + verticalAlignment = Alignment.CenterVertically, horizontalArrangement = Arrangement.spacedBy(8.dp),
  189 + ) {
  190 + Button(onClick = { permLauncher.launch(requestPerms) }) { Text("开启全相册索引") }
  191 + Text("授予读相册权限后自动扫描全部照片", style = MaterialTheme.typography.labelSmall)
  192 + }
  193 + } else {
  194 + ProgressHeader(
  195 + p = progress, running = vlRunning, paused = vlPaused,
  196 + onRun = {
  197 + vm.scanGallery(); vm.runVlNow()
  198 + android.widget.Toast.makeText(
  199 + ctx, "已开始场景理解(每张约 30–40 秒,看「场景描述」进度)", android.widget.Toast.LENGTH_LONG,
  200 + ).show()
  201 + },
  202 + onPause = { vm.pauseVl() },
  203 + onStop = { vm.stopVl() },
  204 + )
  205 + }
  206 + // 搜索框 + 语义按钮。
140 Row( 207 Row(
141 - modifier = Modifier.fillMaxWidth().padding(horizontal = 12.dp, vertical = 6.dp),  
142 - verticalAlignment = Alignment.CenterVertically,  
143 - horizontalArrangement = Arrangement.spacedBy(8.dp), 208 + modifier = Modifier.fillMaxWidth().padding(horizontal = 12.dp, vertical = 4.dp),
  209 + verticalAlignment = Alignment.CenterVertically, horizontalArrangement = Arrangement.spacedBy(8.dp),
144 ) { 210 ) {
145 - Button(onClick = { 211 + OutlinedTextField(
  212 + value = query, onValueChange = { query = it },
  213 + placeholder = { Text("搜照片:场景 / 人名 / 物体…") }, singleLine = true,
  214 + modifier = Modifier.weight(1f),
  215 + )
  216 + Button(
  217 + enabled = query.isNotBlank(),
  218 + onClick = {
  219 + searchMsg = "语义搜索中…"
  220 + vm.searchSemantic(
  221 + query,
  222 + onResult = { results = it; searchMsg = "语义命中 ${it.size}" },
  223 + onError = { searchMsg = "⚠ $it" },
  224 + )
  225 + },
  226 + ) { Text("🧠语义") }
  227 + }
  228 + Row(
  229 + modifier = Modifier.fillMaxWidth().padding(horizontal = 12.dp).padding(bottom = 4.dp),
  230 + verticalAlignment = Alignment.CenterVertically, horizontalArrangement = Arrangement.spacedBy(8.dp),
  231 + ) {
  232 + OutlinedButton(onClick = {
146 picker.launch(PickVisualMediaRequest(ActivityResultContracts.PickVisualMedia.ImageOnly)) 233 picker.launch(PickVisualMediaRequest(ActivityResultContracts.PickVisualMedia.ImageOnly))
147 - }) { Text("+ 导入照片") }  
148 - if (pendingCount > 0) {  
149 - CircularProgressIndicator(modifier = Modifier.height(18.dp), strokeWidth = 2.dp)  
150 - Text("待处理 $pendingCount(后台识别中)", style = MaterialTheme.typography.labelSmall)  
151 - } 234 + }) { Text("+ 手动导入") }
  235 + searchMsg?.let { Text(it, style = MaterialTheme.typography.labelSmall) }
152 } 236 }
153 - if (photos.isEmpty()) { 237 + HorizontalDivider()
  238 + if (shown.isEmpty()) {
154 Box(Modifier.fillMaxSize().padding(24.dp), contentAlignment = Alignment.Center) { 239 Box(Modifier.fillMaxSize().padding(24.dp), contentAlignment = Alignment.Center) {
155 Text( 240 Text(
156 - if (pendingCount > 0) "已导入 $pendingCount 张,后台识别中…稍后刷新" else "还没有照片。点「+ 导入照片」加入相册。\n导入瞬时完成,人脸+场景识别后台定时处理。", 241 + when {
  242 + results != null -> "没搜到。换个词,或等场景描述(VL)跑得更多再搜。"
  243 + !granted -> "点上方「开启全相册索引」,自动把手机相册所有照片纳入。"
  244 + progress.total > 0 -> "正在索引 ${progress.total} 张…稍后刷新。"
  245 + else -> "还没有照片。开启全相册索引,或「+ 手动导入」。"
  246 + },
157 style = MaterialTheme.typography.bodyMedium, 247 style = MaterialTheme.typography.bodyMedium,
158 ) 248 )
159 } 249 }
@@ -164,7 +254,9 @@ internal fun AlbumTab(repo: Repository) { @@ -164,7 +254,9 @@ internal fun AlbumTab(repo: Repository) {
164 verticalArrangement = Arrangement.spacedBy(4.dp), 254 verticalArrangement = Arrangement.spacedBy(4.dp),
165 horizontalArrangement = Arrangement.spacedBy(4.dp), 255 horizontalArrangement = Arrangement.spacedBy(4.dp),
166 ) { 256 ) {
167 - gridItems(photos, key = { it.id }) { p -> PhotoCell(p) { detailPhotoId = p.id } } 257 + gridItems(shown, key = { it.id }) { p ->
  258 + PhotoCell(p, processing = p.id == vlCurrentId) { detailPhotoId = p.id }
  259 + }
168 } 260 }
169 } 261 }
170 } 262 }
@@ -173,15 +265,118 @@ internal fun AlbumTab(repo: Repository) { @@ -173,15 +265,118 @@ internal fun AlbumTab(repo: Repository) {
173 } 265 }
174 } 266 }
175 267
  268 +/**
  269 + * 分层进度:人脸(Tier-0)+ 场景描述(Tier-1 VL)两条进度条 + VL 控制。
  270 + * 运行中显示「暂停 / 停止」,空闲显示「立即理解」(暂停且未跑完时显示「继续」)。
  271 + */
  272 +@Composable
  273 +private fun ProgressHeader(
  274 + p: PhotoProgress,
  275 + running: Boolean,
  276 + paused: Boolean,
  277 + onRun: () -> Unit,
  278 + onPause: () -> Unit,
  279 + onStop: () -> Unit,
  280 +) {
  281 + if (p.total == 0) {
  282 + Row(
  283 + modifier = Modifier.fillMaxWidth().padding(horizontal = 12.dp, vertical = 6.dp),
  284 + verticalAlignment = Alignment.CenterVertically, horizontalArrangement = Arrangement.spacedBy(8.dp),
  285 + ) {
  286 + CircularProgressIndicator(modifier = Modifier.height(16.dp), strokeWidth = 2.dp)
  287 + Text("等待扫描相册…", style = MaterialTheme.typography.labelSmall)
  288 + }
  289 + return
  290 + }
  291 + Column(modifier = Modifier.fillMaxWidth().padding(horizontal = 12.dp, vertical = 6.dp)) {
  292 + Row(verticalAlignment = Alignment.CenterVertically, horizontalArrangement = Arrangement.spacedBy(8.dp)) {
  293 + Text("人脸/时间 ${p.faceDone}/${p.total}", style = MaterialTheme.typography.labelSmall, modifier = Modifier.width(120.dp))
  294 + LinearProgressIndicator(
  295 + progress = { if (p.total > 0) p.faceDone.toFloat() / p.total else 0f },
  296 + modifier = Modifier.weight(1f),
  297 + )
  298 + }
  299 + Spacer(Modifier.height(4.dp))
  300 + Row(verticalAlignment = Alignment.CenterVertically, horizontalArrangement = Arrangement.spacedBy(8.dp)) {
  301 + Text("场景描述 ${p.captionDone}/${p.total}", style = MaterialTheme.typography.labelSmall, modifier = Modifier.width(120.dp))
  302 + LinearProgressIndicator(
  303 + progress = { if (p.total > 0) p.captionDone.toFloat() / p.total else 0f },
  304 + modifier = Modifier.weight(1f),
  305 + )
  306 + }
  307 + Spacer(Modifier.height(6.dp))
  308 + if (running) {
  309 + Row(verticalAlignment = Alignment.CenterVertically, horizontalArrangement = Arrangement.spacedBy(8.dp)) {
  310 + CircularProgressIndicator(modifier = Modifier.size(16.dp), strokeWidth = 2.dp)
  311 + Button(onClick = onPause) { Text("⏸ 暂停") }
  312 + OutlinedButton(onClick = onStop) { Text("⏹ 停止") }
  313 + Text("理解中 ${p.captionDone}/${p.total}", style = MaterialTheme.typography.labelSmall)
  314 + }
  315 + } else if (p.captionDone < p.total) {
  316 + Row(verticalAlignment = Alignment.CenterVertically, horizontalArrangement = Arrangement.spacedBy(8.dp)) {
  317 + Button(onClick = onRun) { Text(if (paused) "▶ 继续理解" else "⚡ 立即理解") }
  318 + if (paused) OutlinedButton(onClick = onStop) { Text("⏹ 停止") }
  319 + Text(if (paused) "已暂停" else "默认充电时跑", style = MaterialTheme.typography.labelSmall)
  320 + }
  321 + } else {
  322 + Text("✓ 全部已理解", style = MaterialTheme.typography.labelSmall)
  323 + }
  324 + }
  325 +}
  326 +
  327 +/** 角标:按 scanState/处理态显示「转圈 / 待理解 / 待索引」。 */
176 @Composable 328 @Composable
177 -private fun PhotoCell(p: PhotoRow, onClick: () -> Unit) { 329 +private fun PhotoCell(p: PhotoRow, processing: Boolean, onClick: () -> Unit) {
178 Box(modifier = Modifier.aspectRatio(1f).clickable(onClick = onClick)) { 330 Box(modifier = Modifier.aspectRatio(1f).clickable(onClick = onClick)) {
179 AsyncImage( 331 AsyncImage(
180 - model = File(p.path), 332 + model = photoModel(p.source, p.path),
181 contentDescription = null, 333 contentDescription = null,
182 contentScale = ContentScale.Crop, 334 contentScale = ContentScale.Crop,
183 modifier = Modifier.fillMaxSize(), 335 modifier = Modifier.fillMaxSize(),
184 ) 336 )
  337 + // 正在 VL 这张:半透明遮罩 + 转圈。
  338 + if (processing) {
  339 + Box(
  340 + modifier = Modifier.fillMaxSize().background(MaterialTheme.colorScheme.scrim.copy(alpha = 0.45f)),
  341 + contentAlignment = Alignment.Center,
  342 + ) {
  343 + CircularProgressIndicator(modifier = Modifier.size(22.dp), strokeWidth = 2.dp, color = MaterialTheme.colorScheme.onPrimary)
  344 + }
  345 + }
  346 + // 左下角状态角标:0 待索引 / 1 待理解 / 2(无 caption) 无描述;已理解(有 caption)不标。
  347 + val statusLabel = when {
  348 + processing -> null
  349 + p.scanState == 0 -> "待索引"
  350 + p.scanState == 1 -> "待理解"
  351 + p.scanState >= 2 && p.caption.isNullOrBlank() -> "无描述"
  352 + else -> null
  353 + }
  354 + statusLabel?.let { lbl ->
  355 + Surface(
  356 + color = MaterialTheme.colorScheme.surface.copy(alpha = 0.75f),
  357 + shape = RoundedCornerShape(6.dp),
  358 + modifier = Modifier.align(Alignment.BottomStart).padding(4.dp),
  359 + ) {
  360 + Text(
  361 + lbl, style = MaterialTheme.typography.labelSmall,
  362 + color = MaterialTheme.colorScheme.onSurface,
  363 + modifier = Modifier.padding(horizontal = 5.dp, vertical = 1.dp),
  364 + )
  365 + }
  366 + }
  367 + // 已出场景描述:左下角一个绿点提示「可搜」。
  368 + if (!processing && p.scanState >= 2 && !p.caption.isNullOrBlank()) {
  369 + Surface(
  370 + color = MaterialTheme.colorScheme.primary.copy(alpha = 0.85f),
  371 + shape = RoundedCornerShape(8.dp),
  372 + modifier = Modifier.align(Alignment.BottomStart).padding(4.dp),
  373 + ) {
  374 + Text(
  375 + "📝", style = MaterialTheme.typography.labelSmall,
  376 + modifier = Modifier.padding(horizontal = 4.dp, vertical = 1.dp),
  377 + )
  378 + }
  379 + }
185 if (p.faceCount > 0) { 380 if (p.faceCount > 0) {
186 Surface( 381 Surface(
187 color = MaterialTheme.colorScheme.primary.copy(alpha = 0.8f), 382 color = MaterialTheme.colorScheme.primary.copy(alpha = 0.8f),
@@ -214,7 +409,7 @@ private fun PhotoDetailSheet(vm: PhotologViewModel, photoId: Long, persons: List @@ -214,7 +409,7 @@ private fun PhotoDetailSheet(vm: PhotologViewModel, photoId: Long, persons: List
214 Column(modifier = Modifier.fillMaxWidth().padding(16.dp)) { 409 Column(modifier = Modifier.fillMaxWidth().padding(16.dp)) {
215 photo?.let { p -> 410 photo?.let { p ->
216 AsyncImage( 411 AsyncImage(
217 - model = File(p.path), 412 + model = photoModel(p.source, p.path),
218 contentDescription = null, 413 contentDescription = null,
219 contentScale = ContentScale.Fit, 414 contentScale = ContentScale.Fit,
220 modifier = Modifier.fillMaxWidth().height(280.dp), 415 modifier = Modifier.fillMaxWidth().height(280.dp),
lifelog-recorder/app/src/main/kotlin/com/lemon/lifelog/PhotologViewModel.kt
@@ -7,6 +7,7 @@ import androidx.lifecycle.AndroidViewModel @@ -7,6 +7,7 @@ import androidx.lifecycle.AndroidViewModel
7 import androidx.lifecycle.viewModelScope 7 import androidx.lifecycle.viewModelScope
8 import com.lemon.lifelog.data.Person 8 import com.lemon.lifelog.data.Person
9 import com.lemon.lifelog.data.Photo 9 import com.lemon.lifelog.data.Photo
  10 +import com.lemon.lifelog.data.PhotoRow
10 import com.lemon.lifelog.data.Repository 11 import com.lemon.lifelog.data.Repository
11 import com.lemon.lifelog.data.SpeakerProfile 12 import com.lemon.lifelog.data.SpeakerProfile
12 import androidx.work.ExistingWorkPolicy 13 import androidx.work.ExistingWorkPolicy
@@ -21,9 +22,12 @@ import com.lemon.lifelog.chat.ChatClient @@ -21,9 +22,12 @@ import com.lemon.lifelog.chat.ChatClient
21 import com.lemon.lifelog.chat.ChatConfig 22 import com.lemon.lifelog.chat.ChatConfig
22 import com.lemon.lifelog.service.RecorderService 23 import com.lemon.lifelog.service.RecorderService
23 import com.lemon.lifelog.service.RecorderState 24 import com.lemon.lifelog.service.RecorderState
  25 +import com.lemon.lifelog.vision.EmbedClient
  26 +import com.lemon.lifelog.vision.FaceEngine
24 import com.lemon.lifelog.vision.FaceMatcher 27 import com.lemon.lifelog.vision.FaceMatcher
25 import com.lemon.lifelog.vision.ImageUtil 28 import com.lemon.lifelog.vision.ImageUtil
26 import com.lemon.lifelog.vision.PhotoScheduler 29 import com.lemon.lifelog.vision.PhotoScheduler
  30 +import com.lemon.lifelog.vision.PhotoVlState
27 import com.lemon.lifelog.vision.WeChatEnrollWorker 31 import com.lemon.lifelog.vision.WeChatEnrollWorker
28 import kotlinx.coroutines.Dispatchers 32 import kotlinx.coroutines.Dispatchers
29 import kotlinx.coroutines.Job 33 import kotlinx.coroutines.Job
@@ -69,6 +73,66 @@ class PhotologViewModel(app: Application) : AndroidViewModel(app) { @@ -69,6 +73,66 @@ class PhotologViewModel(app: Application) : AndroidViewModel(app) {
69 } 73 }
70 } 74 }
71 75
  76 + /** 进相册页 / 拿到读相册权限后催一次全相册扫描 + Tier-0(幂等,已在跑则忽略)。 */
  77 + fun scanGallery() = PhotoScheduler.scanNow(getApplication())
  78 +
  79 + /** 「立即理解」/「继续」:清暂停标志 + 催一次 VL(绕开充电约束)。 */
  80 + fun runVlNow() {
  81 + PhotoVlState.setPaused(getApplication(), false)
  82 + PhotoScheduler.runVlNow(getApplication())
  83 + }
  84 +
  85 + /** 暂停:worker 处理完当前张就停,标志持久化(周期/充电触发也不跑),可「继续」。 */
  86 + fun pauseVl() = PhotoVlState.setPaused(getApplication(), true)
  87 +
  88 + /** 停止:取消手动 VL + 清运行态;不留暂停标志(之后充电时周期照常推进)。 */
  89 + fun stopVl() {
  90 + val app = getApplication<Application>()
  91 + PhotoScheduler.cancelVl(app)
  92 + PhotoVlState.setPaused(app, false)
  93 + PhotoVlState.running.value = false
  94 + PhotoVlState.currentPhotoId.value = null
  95 + }
  96 +
  97 + /** 关键词搜图(caption + 人名 LIKE 子串),即时。 */
  98 + fun searchKeyword(kw: String, onResult: (List<PhotoRow>) -> Unit) {
  99 + viewModelScope.launch {
  100 + val r = withContext(Dispatchers.IO) { repo.photos.searchByKeyword(kw.trim()) }
  101 + onResult(r)
  102 + }
  103 + }
  104 +
  105 + /**
  106 + * 语义搜图:query → 本地 embed 端点出向量 → 跟所有 caption 向量算余弦 → top-K。
  107 + * embed 端点没配/挂了会走 onError(UI 提示),不影响关键词搜。
  108 + */
  109 + fun searchSemantic(query: String, onResult: (List<PhotoRow>) -> Unit, onError: (String) -> Unit) {
  110 + viewModelScope.launch {
  111 + val app = getApplication<Application>()
  112 + try {
  113 + val rows = withContext(Dispatchers.IO) {
  114 + val embed = EmbedClient(ChatConfig.localBase(app), ChatConfig.modelEmbed(app))
  115 + val q = embed.embed(query.trim())
  116 + val vecs = repo.photos.captionVecs()
  117 + if (vecs.isEmpty()) return@withContext emptyList<PhotoRow>()
  118 + val scored = vecs.asSequence()
  119 + .map { it.id to FaceEngine.cosine(q, ImageUtil.unpackEmbedding(it.captionEmbedding)) }
  120 + .filter { it.second >= SEMANTIC_FLOOR }
  121 + .sortedByDescending { it.second }
  122 + .take(60)
  123 + .map { it.first }
  124 + .toList()
  125 + if (scored.isEmpty()) return@withContext emptyList<PhotoRow>()
  126 + val byId = repo.photos.rowsByIds(scored).associateBy { it.id }
  127 + scored.mapNotNull { byId[it] } // 保持相似度降序
  128 + }
  129 + onResult(rows)
  130 + } catch (t: Throwable) {
  131 + onError(t.message ?: "语义搜索失败(检查设置里的 embed 端点)")
  132 + }
  133 + }
  134 + }
  135 +
72 fun createPersonAndTag(faceId: Long, name: String) { 136 fun createPersonAndTag(faceId: Long, name: String) {
73 viewModelScope.launch { 137 viewModelScope.launch {
74 // 手动建的人 = 我关心的 → 直接 managed。 138 // 手动建的人 = 我关心的 → 直接 managed。
@@ -371,6 +435,8 @@ class PhotologViewModel(app: Application) : AndroidViewModel(app) { @@ -371,6 +435,8 @@ class PhotologViewModel(app: Application) : AndroidViewModel(app) {
371 } 435 }
372 } 436 }
373 437
  438 +private const val SEMANTIC_FLOOR = 0.20f // 语义余弦下限,低于此基本不相关;主要靠 top-K 排序
  439 +
374 /** 手机系统通讯录里的一条(内存态,未入库)。phoneDisplay 留原格式给 UI,phone 是纯数字去重键。 */ 440 /** 手机系统通讯录里的一条(内存态,未入库)。phoneDisplay 留原格式给 UI,phone 是纯数字去重键。 */
375 data class PhoneContact( 441 data class PhoneContact(
376 val id: Long, 442 val id: Long,
lifelog-recorder/app/src/main/kotlin/com/lemon/lifelog/SettingsScreen.kt
@@ -76,6 +76,7 @@ fun SettingsScreen(activity: Context) { @@ -76,6 +76,7 @@ fun SettingsScreen(activity: Context) {
76 var localBase by remember { mutableStateOf(ChatConfig.localBase(activity)) } 76 var localBase by remember { mutableStateOf(ChatConfig.localBase(activity)) }
77 var m2b by remember { mutableStateOf(ChatConfig.model2b(activity)) } 77 var m2b by remember { mutableStateOf(ChatConfig.model2b(activity)) }
78 var m4b by remember { mutableStateOf(ChatConfig.model4b(activity)) } 78 var m4b by remember { mutableStateOf(ChatConfig.model4b(activity)) }
  79 + var mEmbed by remember { mutableStateOf(ChatConfig.modelEmbed(activity)) }
79 var endpointMsg by remember { mutableStateOf("") } 80 var endpointMsg by remember { mutableStateOf("") }
80 81
81 var batteryExempt by remember { mutableStateOf(isBatteryExempt(activity)) } 82 var batteryExempt by remember { mutableStateOf(isBatteryExempt(activity)) }
@@ -133,9 +134,12 @@ fun SettingsScreen(activity: Context) { @@ -133,9 +134,12 @@ fun SettingsScreen(activity: Context) {
133 OutlinedTextField(value = m4b, onValueChange = { m4b = it }, 134 OutlinedTextField(value = m4b, onValueChange = { m4b = it },
134 label = { Text("4B model 名") }, singleLine = true, modifier = Modifier.weight(1f)) 135 label = { Text("4B model 名") }, singleLine = true, modifier = Modifier.weight(1f))
135 } 136 }
  137 + OutlinedTextField(value = mEmbed, onValueChange = { mEmbed = it },
  138 + label = { Text("embed model 名(语义搜图,llama-swap 里配 --embedding)") },
  139 + singleLine = true, modifier = Modifier.fillMaxWidth())
136 Row(verticalAlignment = Alignment.CenterVertically, horizontalArrangement = Arrangement.spacedBy(8.dp)) { 140 Row(verticalAlignment = Alignment.CenterVertically, horizontalArrangement = Arrangement.spacedBy(8.dp)) {
137 Button(onClick = { 141 Button(onClick = {
138 - ChatConfig.save(activity, localBase, m2b, m4b) 142 + ChatConfig.save(activity, localBase, m2b, m4b, mEmbed)
139 endpointMsg = "已保存" 143 endpointMsg = "已保存"
140 }) { Text("保存端点") } 144 }) { Text("保存端点") }
141 if (endpointMsg.isNotEmpty()) Text(endpointMsg, style = MaterialTheme.typography.labelSmall) 145 if (endpointMsg.isNotEmpty()) Text(endpointMsg, style = MaterialTheme.typography.labelSmall)
lifelog-recorder/app/src/main/kotlin/com/lemon/lifelog/chat/ChatConfig.kt
@@ -26,10 +26,13 @@ object ChatConfig { @@ -26,10 +26,13 @@ object ChatConfig {
26 private const val K_LOCAL_BASE = "local_base" 26 private const val K_LOCAL_BASE = "local_base"
27 private const val K_MODEL_2B = "model_2b" 27 private const val K_MODEL_2B = "model_2b"
28 private const val K_MODEL_4B = "model_4b" 28 private const val K_MODEL_4B = "model_4b"
  29 + private const val K_MODEL_EMBED = "model_embed"
29 30
30 const val DEFAULT_LOCAL_BASE = "http://127.0.0.1:8080" 31 const val DEFAULT_LOCAL_BASE = "http://127.0.0.1:8080"
31 const val DEFAULT_MODEL_2B = "2b" 32 const val DEFAULT_MODEL_2B = "2b"
32 const val DEFAULT_MODEL_4B = "4b" 33 const val DEFAULT_MODEL_4B = "4b"
  34 + /** 语义搜索的文本 embedding model 名(llama-swap 里配 --embedding 的 GGUF,如 bge-small-zh)。 */
  35 + const val DEFAULT_MODEL_EMBED = "embed"
33 36
34 const val ID_2B = "local-2b" 37 const val ID_2B = "local-2b"
35 const val ID_4B = "local-4b" 38 const val ID_4B = "local-4b"
@@ -46,11 +49,15 @@ object ChatConfig { @@ -46,11 +49,15 @@ object ChatConfig {
46 fun model4b(ctx: Context): String = 49 fun model4b(ctx: Context): String =
47 prefs(ctx).getString(K_MODEL_4B, DEFAULT_MODEL_4B)?.ifBlank { DEFAULT_MODEL_4B } ?: DEFAULT_MODEL_4B 50 prefs(ctx).getString(K_MODEL_4B, DEFAULT_MODEL_4B)?.ifBlank { DEFAULT_MODEL_4B } ?: DEFAULT_MODEL_4B
48 51
49 - fun save(ctx: Context, localBase: String, model2b: String, model4b: String) { 52 + fun modelEmbed(ctx: Context): String =
  53 + prefs(ctx).getString(K_MODEL_EMBED, DEFAULT_MODEL_EMBED)?.ifBlank { DEFAULT_MODEL_EMBED } ?: DEFAULT_MODEL_EMBED
  54 +
  55 + fun save(ctx: Context, localBase: String, model2b: String, model4b: String, modelEmbed: String = DEFAULT_MODEL_EMBED) {
50 prefs(ctx).edit() 56 prefs(ctx).edit()
51 .putString(K_LOCAL_BASE, localBase.trim()) 57 .putString(K_LOCAL_BASE, localBase.trim())
52 .putString(K_MODEL_2B, model2b.trim()) 58 .putString(K_MODEL_2B, model2b.trim())
53 .putString(K_MODEL_4B, model4b.trim()) 59 .putString(K_MODEL_4B, model4b.trim())
  60 + .putString(K_MODEL_EMBED, modelEmbed.trim())
54 .apply() 61 .apply()
55 } 62 }
56 63
lifelog-recorder/app/src/main/kotlin/com/lemon/lifelog/data/AppDatabase.kt
@@ -22,7 +22,7 @@ import androidx.sqlite.db.SupportSQLiteDatabase @@ -22,7 +22,7 @@ import androidx.sqlite.db.SupportSQLiteDatabase
22 WechatMessage::class, 22 WechatMessage::class,
23 Moment::class, 23 Moment::class,
24 ], 24 ],
25 - version = 13, 25 + version = 14,
26 exportSchema = false, 26 exportSchema = false,
27 ) 27 )
28 abstract class AppDatabase : RoomDatabase() { 28 abstract class AppDatabase : RoomDatabase() {
@@ -132,6 +132,24 @@ abstract class AppDatabase : RoomDatabase() { @@ -132,6 +132,24 @@ abstract class AppDatabase : RoomDatabase() {
132 } 132 }
133 } 133 }
134 134
  135 + /**
  136 + * v0.8 全相册自动索引:photos 加来源/MediaStore 去重键/分层进度/caption 语义向量。
  137 + * 老的「导入照片」(source 默认 imported);已处理过的(processedAt 非空)直接标 scanState=2,不重跑。
  138 + */
  139 + private val MIGRATION_13_14 = object : Migration(13, 14) {
  140 + override fun migrate(db: SupportSQLiteDatabase) {
  141 + db.execSQL("ALTER TABLE `photos` ADD COLUMN `source` TEXT NOT NULL DEFAULT 'imported'")
  142 + db.execSQL("ALTER TABLE `photos` ADD COLUMN `mediaId` INTEGER")
  143 + db.execSQL("ALTER TABLE `photos` ADD COLUMN `scanState` INTEGER NOT NULL DEFAULT 0")
  144 + db.execSQL("ALTER TABLE `photos` ADD COLUMN `captionEmbedding` BLOB")
  145 + // 已处理过的老照片不重跑(人脸+VL 都做完了)。
  146 + db.execSQL("UPDATE `photos` SET `scanState` = 2 WHERE `processedAt` IS NOT NULL")
  147 + // 加速 Tier-0/Tier-1 取队列 + MediaStore 去重。
  148 + db.execSQL("CREATE INDEX IF NOT EXISTS `index_photos_scanState` ON `photos` (`scanState`)")
  149 + db.execSQL("CREATE INDEX IF NOT EXISTS `index_photos_mediaId` ON `photos` (`mediaId`)")
  150 + }
  151 + }
  152 +
135 fun get(ctx: Context): AppDatabase = 153 fun get(ctx: Context): AppDatabase =
136 instance ?: synchronized(this) { 154 instance ?: synchronized(this) {
137 instance ?: Room.databaseBuilder( 155 instance ?: Room.databaseBuilder(
@@ -139,7 +157,7 @@ abstract class AppDatabase : RoomDatabase() { @@ -139,7 +157,7 @@ abstract class AppDatabase : RoomDatabase() {
139 AppDatabase::class.java, 157 AppDatabase::class.java,
140 "lifelog.db", 158 "lifelog.db",
141 ) 159 )
142 - .addMigrations(MIGRATION_2_3, MIGRATION_3_4, MIGRATION_4_5, MIGRATION_5_6, MIGRATION_6_7, MIGRATION_7_9, MIGRATION_9_10, MIGRATION_10_11, MIGRATION_11_12, MIGRATION_12_13) 160 + .addMigrations(MIGRATION_2_3, MIGRATION_3_4, MIGRATION_4_5, MIGRATION_5_6, MIGRATION_6_7, MIGRATION_7_9, MIGRATION_9_10, MIGRATION_10_11, MIGRATION_11_12, MIGRATION_12_13, MIGRATION_13_14)
143 // 不再 fallbackToDestructiveMigration:以后改 schema 必须写 Migration, 161 // 不再 fallbackToDestructiveMigration:以后改 schema 必须写 Migration,
144 // 否则启动直接抛 IllegalStateException(fail-loud),绝不静默清掉真实录音。 162 // 否则启动直接抛 IllegalStateException(fail-loud),绝不静默清掉真实录音。
145 .build() 163 .build()
lifelog-recorder/app/src/main/kotlin/com/lemon/lifelog/data/Daos.kt
@@ -287,13 +287,32 @@ interface SummaryDao { @@ -287,13 +287,32 @@ interface SummaryDao {
287 data class PhotoRow( 287 data class PhotoRow(
288 val id: Long, 288 val id: Long,
289 val path: String, 289 val path: String,
  290 + /** 'imported' = File(path);'gallery' = Uri.parse(path)。UI 据此选解码方式。 */
  291 + val source: String,
290 val takenTs: Long, 292 val takenTs: Long,
291 val caption: String?, 293 val caption: String?,
292 val faceCount: Int, 294 val faceCount: Int,
  295 + /** 0=待索引 1=人脸已铺待理解 2=已出场景描述。格子角标据此显示状态。 */
  296 + val scanState: Int,
293 val lat: Double?, 297 val lat: Double?,
294 val lon: Double?, 298 val lon: Double?,
295 ) 299 )
296 300
  301 +/** 语义搜索用的轻投影:只取 id + caption 向量,cosine 在 Kotlin 里算(Room 算不了)。 */
  302 +data class PhotoVec(
  303 + val id: Long,
  304 + val captionEmbedding: ByteArray,
  305 +)
  306 +
  307 +/** 相册分层进度(进度条用)。 */
  308 +data class PhotoProgress(
  309 + val total: Int,
  310 + /** Tier-0 已铺(人脸+时间,scanState≥1) */
  311 + val faceDone: Int,
  312 + /** Tier-1 已出场景描述(scanState≥2) */
  313 + val captionDone: Int,
  314 +)
  315 +
297 @Dao 316 @Dao
298 interface PersonDao { 317 interface PersonDao {
299 @Insert 318 @Insert
@@ -400,33 +419,75 @@ interface PhotoDao { @@ -400,33 +419,75 @@ interface PhotoDao {
400 @Query("SELECT * FROM photos WHERE path = :path LIMIT 1") 419 @Query("SELECT * FROM photos WHERE path = :path LIMIT 1")
401 suspend fun getByPath(path: String): Photo? 420 suspend fun getByPath(path: String): Photo?
402 421
403 - /** 还没跑过人脸+VL 的照片(处理 worker 取这个)。 */  
404 - @Query("SELECT * FROM photos WHERE processedAt IS NULL ORDER BY takenTs DESC")  
405 - suspend fun unprocessed(): List<Photo> 422 + // ----- 全相册扫描:MediaStore 去重 + 增量 -----
  423 +
  424 + /** 已登记的所有 MediaStore _ID(gallery 源去重)。 */
  425 + @Query("SELECT mediaId FROM photos WHERE mediaId IS NOT NULL")
  426 + suspend fun knownMediaIds(): List<Long>
  427 +
  428 + /** 已登记的最大 MediaStore _ID(增量扫描可只看更大的,但默认仍全量去重)。 */
  429 + @Query("SELECT MAX(mediaId) FROM photos WHERE source = 'gallery'")
  430 + suspend fun maxMediaId(): Long?
  431 +
  432 + // ----- 分层管线队列 -----
  433 +
  434 + /** Tier-0 队列:还没铺人脸(scanState=0),近期优先。 */
  435 + @Query("SELECT * FROM photos WHERE scanState = 0 ORDER BY takenTs DESC LIMIT :limit")
  436 + suspend fun pendingFace(limit: Int = 5000): List<Photo>
  437 +
  438 + /**
  439 + * Tier-1 VL 队列:人脸已铺待出场景描述(scanState=1)。
  440 + * 优先级:有人脸的优先、再近期优先(合照/有人的更值得先理解)。
  441 + */
  442 + @Query(
  443 + "SELECT * FROM photos WHERE scanState = 1 ORDER BY (faceCount > 0) DESC, takenTs DESC LIMIT :limit",
  444 + )
  445 + suspend fun pendingCaption(limit: Int = 5000): List<Photo>
  446 +
  447 + // ----- 分层进度 -----
  448 +
  449 + @Query(
  450 + """
  451 + SELECT COUNT(*) AS total,
  452 + COALESCE(SUM(CASE WHEN scanState >= 1 THEN 1 ELSE 0 END), 0) AS faceDone,
  453 + COALESCE(SUM(CASE WHEN scanState >= 2 THEN 1 ELSE 0 END), 0) AS captionDone
  454 + FROM photos
  455 + """,
  456 + )
  457 + fun observeProgress(): Flow<PhotoProgress>
  458 +
  459 + // ----- 分层写回 -----
406 460
407 - /** 未处理照片数(UI 显示"待处理 N")。 */  
408 - @Query("SELECT COUNT(*) FROM photos WHERE processedAt IS NULL")  
409 - fun observeUnprocessedCount(): Flow<Int> 461 + /** Tier-0 完成:写人脸数 + GPS,进度推到 1。 */
  462 + @Query("UPDATE photos SET faceCount = :n, lat = :lat, lon = :lon, scanState = 1 WHERE id = :id")
  463 + suspend fun markFaceDone(id: Long, n: Int, lat: Double?, lon: Double?)
410 464
411 - @Query("UPDATE photos SET caption = :caption, captionModel = :model WHERE id = :id")  
412 - suspend fun updateCaption(id: Long, caption: String, model: String) 465 + /** Tier-1 完成:写 caption + 语义向量,进度推到 2 + processedAt。 */
  466 + @Query(
  467 + """
  468 + UPDATE photos SET caption = :caption, captionModel = :model,
  469 + captionEmbedding = :emb, scanState = 2, processedAt = :ts WHERE id = :id
  470 + """,
  471 + )
  472 + suspend fun markCaptionDone(id: Long, caption: String?, model: String?, emb: ByteArray?, ts: Long)
413 473
414 - @Query("UPDATE photos SET faceCount = :n, processedAt = :ts WHERE id = :id")  
415 - suspend fun markProcessed(id: Long, n: Int, ts: Long) 474 + /** Tier-1 失败也别卡队列:标 scanState=2(caption 留空),下轮不再取。 */
  475 + @Query("UPDATE photos SET scanState = 2, processedAt = :ts WHERE id = :id")
  476 + suspend fun markCaptionSkipped(id: Long, ts: Long)
416 477
417 @Query("UPDATE photos SET lat = :lat, lon = :lon WHERE id = :id") 478 @Query("UPDATE photos SET lat = :lat, lon = :lon WHERE id = :id")
418 suspend fun updateGeo(id: Long, lat: Double?, lon: Double?) 479 suspend fun updateGeo(id: Long, lat: Double?, lon: Double?)
419 480
420 @Query( 481 @Query(
421 - "SELECT id, path, takenTs, caption, faceCount, lat, lon FROM photos ORDER BY takenTs DESC LIMIT :limit", 482 + "SELECT id, path, source, takenTs, caption, faceCount, scanState, lat, lon FROM photos ORDER BY takenTs DESC LIMIT :limit",
422 ) 483 )
423 fun observeGrid(limit: Int = 500): Flow<List<PhotoRow>> 484 fun observeGrid(limit: Int = 500): Flow<List<PhotoRow>>
424 485
425 /** 某个人出现的所有照片(faces.personId join)。 */ 486 /** 某个人出现的所有照片(faces.personId join)。 */
426 @Query( 487 @Query(
427 """ 488 """
428 - SELECT DISTINCT p.id AS id, p.path AS path, p.takenTs AS takenTs,  
429 - p.caption AS caption, p.faceCount AS faceCount, p.lat AS lat, p.lon AS lon 489 + SELECT DISTINCT p.id AS id, p.path AS path, p.source AS source, p.takenTs AS takenTs,
  490 + p.caption AS caption, p.faceCount AS faceCount, p.scanState AS scanState, p.lat AS lat, p.lon AS lon
430 FROM photos p INNER JOIN faces f ON f.photoId = p.id 491 FROM photos p INNER JOIN faces f ON f.photoId = p.id
431 WHERE f.personId = :personId 492 WHERE f.personId = :personId
432 ORDER BY p.takenTs DESC 493 ORDER BY p.takenTs DESC
@@ -434,6 +495,35 @@ interface PhotoDao { @@ -434,6 +495,35 @@ interface PhotoDao {
434 ) 495 )
435 fun observePhotosOfPerson(personId: Long): Flow<List<PhotoRow>> 496 fun observePhotosOfPerson(personId: Long): Flow<List<PhotoRow>>
436 497
  498 + // ----- 搜索(P3 关键词 LIKE:caption + 人名;中文子串比 FTS4 unicode61 更靠谱) -----
  499 +
  500 + @Query(
  501 + """
  502 + SELECT DISTINCT p.id AS id, p.path AS path, p.source AS source, p.takenTs AS takenTs,
  503 + p.caption AS caption, p.faceCount AS faceCount, p.scanState AS scanState, p.lat AS lat, p.lon AS lon
  504 + FROM photos p
  505 + LEFT JOIN faces f ON f.photoId = p.id
  506 + LEFT JOIN persons pe ON pe.id = f.personId
  507 + WHERE p.caption LIKE '%' || :kw || '%' OR pe.name LIKE '%' || :kw || '%'
  508 + ORDER BY p.takenTs DESC
  509 + LIMIT :limit
  510 + """,
  511 + )
  512 + suspend fun searchByKeyword(kw: String, limit: Int = 500): List<PhotoRow>
  513 +
  514 + // ----- 搜索(P4 语义向量:取全部 caption 向量,cosine 在 Kotlin 里算) -----
  515 +
  516 + @Query("SELECT id, captionEmbedding FROM photos WHERE captionEmbedding IS NOT NULL")
  517 + suspend fun captionVecs(): List<PhotoVec>
  518 +
  519 + @Query(
  520 + """
  521 + SELECT id, path, source, takenTs, caption, faceCount, scanState, lat, lon
  522 + FROM photos WHERE id IN (:ids)
  523 + """,
  524 + )
  525 + suspend fun rowsByIds(ids: List<Long>): List<PhotoRow>
  526 +
437 @Query("DELETE FROM photos WHERE id = :id") 527 @Query("DELETE FROM photos WHERE id = :id")
438 suspend fun delete(id: Long) 528 suspend fun delete(id: Long)
439 } 529 }
lifelog-recorder/app/src/main/kotlin/com/lemon/lifelog/data/Entities.kt
@@ -2,6 +2,7 @@ package com.lemon.lifelog.data @@ -2,6 +2,7 @@ package com.lemon.lifelog.data
2 2
3 import androidx.room.Entity 3 import androidx.room.Entity
4 import androidx.room.Fts4 4 import androidx.room.Fts4
  5 +import androidx.room.Index
5 import androidx.room.PrimaryKey 6 import androidx.room.PrimaryKey
6 7
7 @Entity(tableName = "segments") 8 @Entity(tableName = "segments")
@@ -121,11 +122,19 @@ data class Person( @@ -121,11 +122,19 @@ data class Person(
121 ) 122 )
122 123
123 /** 一张照片 + Qwen-VL 场景描述 + EXIF 时间/GPS。 */ 124 /** 一张照片 + Qwen-VL 场景描述 + EXIF 时间/GPS。 */
124 -@Entity(tableName = "photos") 125 +@Entity(
  126 + tableName = "photos",
  127 + indices = [Index("scanState"), Index("mediaId")],
  128 +)
125 data class Photo( 129 data class Photo(
126 @PrimaryKey(autoGenerate = true) val id: Long = 0, 130 @PrimaryKey(autoGenerate = true) val id: Long = 0,
  131 + /** imported 源 = app 私有目录文件绝对路径;gallery 源 = MediaStore content:// uri 字符串(原地读,不拷贝)。 */
127 val path: String, 132 val path: String,
128 - /** 拍摄时间(EXIF DateTimeOriginal,回退文件时间) */ 133 + /** 来源:'imported'(手动选进来,拷进私有目录)| 'gallery'(全相册自动扫描,原地引用)。 */
  134 + val source: String = "imported",
  135 + /** MediaStore _ID(gallery 源用于去重 + 增量扫描);imported 源为 null。 */
  136 + val mediaId: Long? = null,
  137 + /** 拍摄时间(EXIF DateTimeOriginal / MediaStore DATE_TAKEN,回退文件时间) */
129 val takenTs: Long, 138 val takenTs: Long,
130 val lat: Double? = null, 139 val lat: Double? = null,
131 val lon: Double? = null, 140 val lon: Double? = null,
@@ -133,7 +142,14 @@ data class Photo( @@ -133,7 +142,14 @@ data class Photo(
133 val caption: String? = null, 142 val caption: String? = null,
134 val captionModel: String? = null, 143 val captionModel: String? = null,
135 val faceCount: Int = 0, 144 val faceCount: Int = 0,
136 - /** 人脸+VL 处理完成时间,null = 还没处理 */ 145 + /**
  146 + * 分层管线进度:0 = 已登记未处理;1 = Tier-0(人脸 + EXIF)已铺;2 = Tier-1(VL 场景描述)已出。
  147 + * 全相册几千张,Tier-0 秒级全量铺开先可按人/时间搜,Tier-1 VL 慢、按优先级队列慢慢补。
  148 + */
  149 + val scanState: Int = 0,
  150 + /** caption 的本地文本 embedding(LE float bytes),用于语义搜索;null = 还没算/无 caption。 */
  151 + val captionEmbedding: ByteArray? = null,
  152 + /** Tier-1 VL 处理完成时间,null = 还没出 caption。 */
137 val processedAt: Long? = null, 153 val processedAt: Long? = null,
138 ) 154 )
139 155
lifelog-recorder/app/src/main/kotlin/com/lemon/lifelog/vision/EmbedClient.kt 0 → 100644
  1 +package com.lemon.lifelog.vision
  2 +
  3 +import okhttp3.MediaType.Companion.toMediaType
  4 +import okhttp3.OkHttpClient
  5 +import okhttp3.Request
  6 +import okhttp3.RequestBody.Companion.toRequestBody
  7 +import org.json.JSONArray
  8 +import org.json.JSONObject
  9 +import java.io.IOException
  10 +import java.util.concurrent.TimeUnit
  11 +import kotlin.math.sqrt
  12 +
  13 +/**
  14 + * 文本 embedding(P4 语义搜索):打本地 llama-swap 的 OpenAI 兼容 /v1/embeddings。
  15 + *
  16 + * 部署:llama-swap config.yaml 加一个 model(默认名 "embed"),用 llama-server --embedding 起一个
  17 + * 中文 embedding GGUF(如 bge-small-zh / gte-small-zh)。没配这个 model 时调用会抛 IOException,
  18 + * 语义搜索优雅降级(UI 提示「未配置 embed 端点」),不影响关键词搜。
  19 + *
  20 + * 返回向量统一 L2 归一化 → 余弦相似度 = 点积(与人脸/声纹 embedding 同约定)。
  21 + */
  22 +class EmbedClient(
  23 + private val baseUrl: String,
  24 + private val model: String = "embed",
  25 +) {
  26 + companion object {
  27 + private val JSON = "application/json; charset=utf-8".toMediaType()
  28 + }
  29 +
  30 + private val http = OkHttpClient.Builder()
  31 + .connectTimeout(10, TimeUnit.SECONDS)
  32 + .readTimeout(120, TimeUnit.SECONDS) // 含 llama-swap 切模型
  33 + .build()
  34 +
  35 + @Throws(IOException::class)
  36 + fun embed(text: String): FloatArray {
  37 + val payload = JSONObject().apply {
  38 + put("model", model)
  39 + put("input", text)
  40 + }
  41 + val req = Request.Builder()
  42 + .url("$baseUrl/v1/embeddings")
  43 + .header("Content-Type", "application/json")
  44 + .post(payload.toString().toRequestBody(JSON))
  45 + .build()
  46 + http.newCall(req).execute().use { resp ->
  47 + val body = resp.body?.string().orEmpty()
  48 + if (!resp.isSuccessful) throw IOException("embed ${resp.code}: ${body.take(200)}")
  49 + val data = JSONObject(body).optJSONArray("data") ?: throw IOException("no data")
  50 + if (data.length() == 0) throw IOException("empty data")
  51 + val arr: JSONArray = data.getJSONObject(0).optJSONArray("embedding")
  52 + ?: throw IOException("no embedding")
  53 + val e = FloatArray(arr.length()) { arr.getDouble(it).toFloat() }
  54 + return l2norm(e)
  55 + }
  56 + }
  57 +
  58 + private fun l2norm(e: FloatArray): FloatArray {
  59 + var n = 0f
  60 + for (v in e) n += v * v
  61 + n = sqrt(n)
  62 + if (n > 0f) for (i in e.indices) e[i] /= n
  63 + return e
  64 + }
  65 +}
lifelog-recorder/app/src/main/kotlin/com/lemon/lifelog/vision/GalleryScanner.kt 0 → 100644
  1 +package com.lemon.lifelog.vision
  2 +
  3 +import android.content.ContentUris
  4 +import android.content.Context
  5 +import android.provider.MediaStore
  6 +import android.util.Log
  7 +import com.lemon.lifelog.data.Photo
  8 +import com.lemon.lifelog.data.Repository
  9 +
  10 +/**
  11 + * 全相册扫描:游标遍历 MediaStore 所有图片,把还没登记的注册成 photos 行(scanState=0)。
  12 + *
  13 + * 关键取舍:**不拷贝文件**——gallery 源只存 content:// uri,原地引用(全相册几千上万张,
  14 + * 拷一份会撑爆私有目录)。拍摄时间直接取 MediaStore DATE_TAKEN(秒级全量铺好时间轴),
  15 + * GPS 留到 Tier-0 解码时按需补(要 ACCESS_MEDIA_LOCATION)。
  16 + *
  17 + * 去重按 MediaStore _ID(mediaId):已登记的跳过,所以可反复跑、增量补新图。
  18 + */
  19 +object GalleryScanner {
  20 + private const val TAG = "GalleryScanner"
  21 +
  22 + /** 扫一遍相册,登记新图片。返回本轮新增数。 */
  23 + suspend fun scan(ctx: Context, repo: Repository): Int {
  24 + val known = repo.photos.knownMediaIds().toHashSet()
  25 + val collection = MediaStore.Images.Media.getContentUri(MediaStore.VOLUME_EXTERNAL)
  26 + val proj = arrayOf(
  27 + MediaStore.Images.Media._ID,
  28 + MediaStore.Images.Media.DATE_TAKEN, // ms(可能为 0/null)
  29 + MediaStore.Images.Media.DATE_ADDED, // 秒
  30 + )
  31 + var added = 0
  32 + try {
  33 + ctx.contentResolver.query(
  34 + collection, proj, null, null,
  35 + "${MediaStore.Images.Media._ID} DESC",
  36 + )?.use { c ->
  37 + val idIdx = c.getColumnIndexOrThrow(MediaStore.Images.Media._ID)
  38 + val takenIdx = c.getColumnIndexOrThrow(MediaStore.Images.Media.DATE_TAKEN)
  39 + val addedIdx = c.getColumnIndexOrThrow(MediaStore.Images.Media.DATE_ADDED)
  40 + while (c.moveToNext()) {
  41 + val id = c.getLong(idIdx)
  42 + if (!known.add(id)) continue // 已登记
  43 + val takenMs = if (!c.isNull(takenIdx)) c.getLong(takenIdx) else 0L
  44 + val ts = if (takenMs > 0) takenMs else c.getLong(addedIdx) * 1000L
  45 + val uri = ContentUris.withAppendedId(collection, id)
  46 + repo.photos.insert(
  47 + Photo(
  48 + path = uri.toString(),
  49 + source = "gallery",
  50 + mediaId = id,
  51 + takenTs = ts,
  52 + scanState = 0,
  53 + ),
  54 + )
  55 + added++
  56 + }
  57 + }
  58 + } catch (t: Throwable) {
  59 + Log.w(TAG, "scan failed: ${t.message}")
  60 + }
  61 + if (added > 0) Log.i(TAG, "registered $added new gallery photos")
  62 + return added
  63 + }
  64 +}
lifelog-recorder/app/src/main/kotlin/com/lemon/lifelog/vision/ImageUtil.kt
1 package com.lemon.lifelog.vision 1 package com.lemon.lifelog.vision
2 2
  3 +import android.content.Context
3 import android.graphics.Bitmap 4 import android.graphics.Bitmap
4 import android.graphics.BitmapFactory 5 import android.graphics.BitmapFactory
5 import android.graphics.Matrix 6 import android.graphics.Matrix
  7 +import android.net.Uri
  8 +import android.provider.MediaStore
6 import androidx.exifinterface.media.ExifInterface 9 import androidx.exifinterface.media.ExifInterface
  10 +import com.lemon.lifelog.data.Photo
7 import java.io.ByteArrayOutputStream 11 import java.io.ByteArrayOutputStream
8 import java.io.File 12 import java.io.File
  13 +import java.io.InputStream
9 import java.nio.ByteBuffer 14 import java.nio.ByteBuffer
10 import java.nio.ByteOrder 15 import java.nio.ByteOrder
11 import java.text.SimpleDateFormat 16 import java.text.SimpleDateFormat
12 import java.util.Locale 17 import java.util.Locale
13 import kotlin.math.min 18 import kotlin.math.min
14 19
15 -/** 照片 EXIF + 解码 + 缩放工具。 */ 20 +/** 照片 EXIF + 解码 + 缩放工具。imported 源读私有文件,gallery 源原地读 MediaStore content uri。 */
16 object ImageUtil { 21 object ImageUtil {
17 data class Meta(val takenTs: Long, val lat: Double?, val lon: Double?) 22 data class Meta(val takenTs: Long, val lat: Double?, val lon: Double?)
18 23
@@ -31,6 +36,67 @@ object ImageUtil { @@ -31,6 +36,67 @@ object ImageUtil {
31 } 36 }
32 } 37 }
33 38
  39 + /** 按来源解码:gallery 源走 content uri(原地),imported 源走文件路径。 */
  40 + fun decode(ctx: Context, photo: Photo, reqMax: Int = 2048): Bitmap? =
  41 + if (photo.source == "gallery") decodeUri(ctx, Uri.parse(photo.path), reqMax)
  42 + else decodeOriented(photo.path, reqMax)
  43 +
  44 + /**
  45 + * 读 GPS(Tier-0 用)。gallery 源需 ACCESS_MEDIA_LOCATION + setRequireOriginal 拿未脱敏 EXIF;
  46 + * 没权限/失败就返回 (null,null),不阻断流程。takenTs 在扫描时已从 MediaStore 取,这里只补经纬度。
  47 + */
  48 + fun readGps(ctx: Context, photo: Photo): Pair<Double?, Double?> = try {
  49 + val stream: InputStream? = if (photo.source == "gallery") {
  50 + val orig = runCatching { MediaStore.setRequireOriginal(Uri.parse(photo.path)) }
  51 + .getOrDefault(Uri.parse(photo.path))
  52 + ctx.contentResolver.openInputStream(orig)
  53 + } else {
  54 + File(photo.path).inputStream()
  55 + }
  56 + stream?.use {
  57 + val ll = ExifInterface(it).latLong
  58 + if (ll != null) ll[0] to ll[1] else null to null
  59 + } ?: (null to null)
  60 + } catch (t: Throwable) {
  61 + null to null
  62 + }
  63 +
  64 + /** 从 content uri 解码 + 按 EXIF 方向摆正(gallery 源)。 */
  65 + private fun decodeUri(ctx: Context, uri: Uri, reqMax: Int): Bitmap? {
  66 + val cr = ctx.contentResolver
  67 + val bounds = BitmapFactory.Options().apply { inJustDecodeBounds = true }
  68 + // 注意:inJustDecodeBounds 模式下 decodeStream 本就返回 null(只填 bounds),
  69 + // 不能拿它的返回值判失败——要判的是 openInputStream。
  70 + (cr.openInputStream(uri) ?: return null).use { BitmapFactory.decodeStream(it, null, bounds) }
  71 + var sample = 1
  72 + val longest = maxOf(bounds.outWidth, bounds.outHeight)
  73 + while (longest / sample > reqMax) sample *= 2
  74 + val decoded = cr.openInputStream(uri)?.use {
  75 + BitmapFactory.decodeStream(it, null, BitmapFactory.Options().apply { inSampleSize = sample })
  76 + } ?: return null
  77 + val orientation = try {
  78 + cr.openInputStream(uri)?.use {
  79 + ExifInterface(it).getAttributeInt(ExifInterface.TAG_ORIENTATION, ExifInterface.ORIENTATION_NORMAL)
  80 + } ?: ExifInterface.ORIENTATION_NORMAL
  81 + } catch (t: Throwable) {
  82 + ExifInterface.ORIENTATION_NORMAL
  83 + }
  84 + return applyOrientation(decoded, orientation)
  85 + }
  86 +
  87 + private fun applyOrientation(decoded: Bitmap, orientation: Int): Bitmap {
  88 + val m = Matrix()
  89 + when (orientation) {
  90 + ExifInterface.ORIENTATION_ROTATE_90 -> m.postRotate(90f)
  91 + ExifInterface.ORIENTATION_ROTATE_180 -> m.postRotate(180f)
  92 + ExifInterface.ORIENTATION_ROTATE_270 -> m.postRotate(270f)
  93 + else -> return decoded
  94 + }
  95 + val rotated = Bitmap.createBitmap(decoded, 0, 0, decoded.width, decoded.height, m, true)
  96 + if (rotated != decoded) decoded.recycle()
  97 + return rotated
  98 + }
  99 +
34 /** 解码并按 EXIF 方向摆正。可选下采样 reqMax(最长边)以省内存。 */ 100 /** 解码并按 EXIF 方向摆正。可选下采样 reqMax(最长边)以省内存。 */
35 fun decodeOriented(path: String, reqMax: Int = 2048): Bitmap? { 101 fun decodeOriented(path: String, reqMax: Int = 2048): Bitmap? {
36 val opts = BitmapFactory.Options() 102 val opts = BitmapFactory.Options()
lifelog-recorder/app/src/main/kotlin/com/lemon/lifelog/vision/PhotoWorker.kt renamed to lifelog-recorder/app/src/main/kotlin/com/lemon/lifelog/vision/PhotoFaceWorker.kt
@@ -4,44 +4,48 @@ import android.content.Context @@ -4,44 +4,48 @@ import android.content.Context
4 import android.util.Log 4 import android.util.Log
5 import androidx.work.CoroutineWorker 5 import androidx.work.CoroutineWorker
6 import androidx.work.WorkerParameters 6 import androidx.work.WorkerParameters
7 -import com.lemon.lifelog.chat.ChatConfig  
8 import com.lemon.lifelog.data.Repository 7 import com.lemon.lifelog.data.Repository
9 import java.util.concurrent.atomic.AtomicBoolean 8 import java.util.concurrent.atomic.AtomicBoolean
10 9
11 /** 10 /**
12 - * 后台批处理未处理照片(processedAt=null):人脸 + Qwen-VL 场景描述。  
13 - * 导入只登记照片行,处理交给这里——闲时/定时跑,不让用户在导入时干等。  
14 - * 单轮有时间预算,超了下轮(retry/periodic)继续,避开 WorkManager 执行上限。 11 + * Tier-0 worker:① 扫全相册登记新图 ② 给未铺人脸的照片(scanState=0)跑 ML Kit 人脸 + GPS。
  12 + *
  13 + * 快(<1s/张),不依赖 llama-server,battery-not-low 即可跑——目标是几小时内把整个相册
  14 + * 的人脸/时间线铺满,先能「按人/按时间」搜。VL 场景描述是另一条慢队列(PhotoVlWorker)。
  15 + *
  16 + * 单轮 7min 预算 + retry,避开 WorkManager ~10min 执行上限。
15 */ 17 */
16 -class PhotoWorker(ctx: Context, params: WorkerParameters) : CoroutineWorker(ctx, params) { 18 +class PhotoFaceWorker(ctx: Context, params: WorkerParameters) : CoroutineWorker(ctx, params) {
17 19
18 override suspend fun doWork(): Result { 20 override suspend fun doWork(): Result {
19 if (!isRunning.compareAndSet(false, true)) return Result.success() 21 if (!isRunning.compareAndSet(false, true)) return Result.success()
20 try { 22 try {
21 val repo = Repository(applicationContext) 23 val repo = Repository(applicationContext)
22 - val pending = repo.photos.unprocessed() 24 + // 先增量登记全相册新图(需要已授予读相册权限;没权限就只处理已登记的)。
  25 + runCatching { GalleryScanner.scan(applicationContext, repo) }
  26 +
  27 + val pending = repo.photos.pendingFace()
23 if (pending.isEmpty()) return Result.success() 28 if (pending.isEmpty()) return Result.success()
24 29
25 val engine = FaceEngine.load(applicationContext) 30 val engine = FaceEngine.load(applicationContext)
26 - val vision = VisionClient(ChatConfig.localBase(applicationContext))  
27 - val proc = PhotoProcessor(repo, engine, vision) 31 + val proc = PhotoProcessor(applicationContext, repo)
28 val deadline = System.currentTimeMillis() + RUN_BUDGET_MS 32 val deadline = System.currentTimeMillis() + RUN_BUDGET_MS
29 var done = 0 33 var done = 0
30 try { 34 try {
31 for (p in pending) { 35 for (p in pending) {
32 if (System.currentTimeMillis() > deadline) break 36 if (System.currentTimeMillis() > deadline) break
33 try { 37 try {
34 - proc.process(p) 38 + proc.faceStage(p, engine)
35 } catch (t: Throwable) { 39 } catch (t: Throwable) {
36 - Log.w(TAG, "process failed ${p.path}: ${t.message}") 40 + Log.w(TAG, "faceStage failed ${p.path}: ${t.message}")
  41 + repo.photos.markFaceDone(p.id, 0, null, null) // 别卡队列
37 } 42 }
38 done++ 43 done++
39 } 44 }
40 } finally { 45 } finally {
41 engine.close() 46 engine.close()
42 } 47 }
43 - Log.i(TAG, "processed $done / ${pending.size} pending")  
44 - // 还有剩的就 retry 继续(带退避),否则结束等下个周期。 48 + Log.i(TAG, "tier0 face: $done / ${pending.size}")
45 return if (done < pending.size) Result.retry() else Result.success() 49 return if (done < pending.size) Result.retry() else Result.success()
46 } catch (t: Throwable) { 50 } catch (t: Throwable) {
47 Log.e(TAG, "worker failed", t) 51 Log.e(TAG, "worker failed", t)
@@ -52,10 +56,9 @@ class PhotoWorker(ctx: Context, params: WorkerParameters) : CoroutineWorker(ctx, @@ -52,10 +56,9 @@ class PhotoWorker(ctx: Context, params: WorkerParameters) : CoroutineWorker(ctx,
52 } 56 }
53 57
54 companion object { 58 companion object {
55 - const val TAG = "PhotoWorker"  
56 - /** 单轮预算 7min(VL ~45s/张,约 9 张/轮),避开 WorkManager ~10min 上限。 */ 59 + const val TAG = "PhotoFaceWorker"
  60 + /** 单轮预算 7min,避开 WorkManager ~10min 上限。 */
57 const val RUN_BUDGET_MS = 7 * 60 * 1000L 61 const val RUN_BUDGET_MS = 7 * 60 * 1000L
58 - /** 防并发:periodic + one-shot + retry 可能同时触发,多 worker 打同一 llama-server 会 502。 */  
59 private val isRunning = AtomicBoolean(false) 62 private val isRunning = AtomicBoolean(false)
60 } 63 }
61 } 64 }
lifelog-recorder/app/src/main/kotlin/com/lemon/lifelog/vision/PhotoProcessor.kt
1 package com.lemon.lifelog.vision 1 package com.lemon.lifelog.vision
2 2
  3 +import android.content.Context
3 import android.util.Log 4 import android.util.Log
4 import com.lemon.lifelog.data.Face 5 import com.lemon.lifelog.data.Face
5 import com.lemon.lifelog.data.Photo 6 import com.lemon.lifelog.data.Photo
6 import com.lemon.lifelog.data.Repository 7 import com.lemon.lifelog.data.Repository
7 import kotlinx.coroutines.Dispatchers 8 import kotlinx.coroutines.Dispatchers
8 import kotlinx.coroutines.withContext 9 import kotlinx.coroutines.withContext
9 -import java.io.File  
10 10
11 /** 11 /**
12 - * 单张照片处理:人脸(FaceEngine) + 场景描述(VisionClient) → 更新已登记的 photos 行 + 写 faces。  
13 - * 导入时只登记 photos 行(processedAt=null);这里由后台 PhotoWorker 闲时调用,不阻塞 UI。 12 + * 单张照片的分层处理:
  13 + * - Tier-0 [faceStage]:人脸检测 + 512d embedding + GPS。秒级,全相册先铺满(按人/时间可搜)。
  14 + * - Tier-1 [captionStage]:Qwen-VL 场景描述 + caption 语义向量。慢,按优先级队列充电时跑。
  15 + *
  16 + * 两段都从来源(imported 私有文件 / gallery content uri)解码,互不依赖,可在不同时机/约束下跑。
14 */ 17 */
15 class PhotoProcessor( 18 class PhotoProcessor(
  19 + private val ctx: Context,
16 private val repo: Repository, 20 private val repo: Repository,
17 - private val face: FaceEngine,  
18 - private val vision: VisionClient,  
19 ) { 21 ) {
20 - /** 处理一张已登记但未处理的照片(processedAt=null)。 */  
21 - suspend fun process(photo: Photo) { 22 + /** Tier-0:人脸 + GPS → faces 行 + 自动认人,scanState 推到 1。 */
  23 + suspend fun faceStage(photo: Photo, face: FaceEngine) {
22 withContext(Dispatchers.IO) { 24 withContext(Dispatchers.IO) {
23 - val bmp = ImageUtil.decodeOriented(photo.path) ?: run {  
24 - Log.w(TAG, "decode failed: ${photo.path}")  
25 - repo.photos.markProcessed(photo.id, 0, System.currentTimeMillis()) 25 + val bmp = ImageUtil.decode(ctx, photo) ?: run {
  26 + Log.w(TAG, "decode failed (face): ${photo.path}")
  27 + repo.photos.markFaceDone(photo.id, 0, null, null)
26 return@withContext 28 return@withContext
27 } 29 }
28 -  
29 val faces = try { 30 val faces = try {
30 face.analyze(bmp) 31 face.analyze(bmp)
31 } catch (t: Throwable) { 32 } catch (t: Throwable) {
32 Log.w(TAG, "face analyze failed: ${t.message}"); emptyList() 33 Log.w(TAG, "face analyze failed: ${t.message}"); emptyList()
33 } 34 }
34 - val caption = try {  
35 - vision.caption(ImageUtil.toJpeg(bmp))  
36 - } catch (t: Throwable) {  
37 - Log.w(TAG, "caption failed: ${t.message}"); null  
38 - }  
39 bmp.recycle() 35 bmp.recycle()
40 36
41 - if (caption != null) repo.photos.updateCaption(photo.id, caption, "vl") 37 + val (lat, lon) = ImageUtil.readGps(ctx, photo)
42 38
43 // 匹配 gallery = 微信头像锚点 + 已标注脸样本。 39 // 匹配 gallery = 微信头像锚点 + 已标注脸样本。
44 val gallery = FaceMatcher.gallery(repo) 40 val gallery = FaceMatcher.gallery(repo)
@@ -55,8 +51,38 @@ class PhotoProcessor( @@ -55,8 +51,38 @@ class PhotoProcessor(
55 ), 51 ),
56 ) 52 )
57 } 53 }
58 - repo.photos.markProcessed(photo.id, faces.size, System.currentTimeMillis())  
59 - Log.i(TAG, "✓ ${File(photo.path).name}: ${faces.size} 脸, caption=${caption?.take(30)}") 54 + repo.photos.markFaceDone(photo.id, faces.size, lat, lon)
  55 + }
  56 + }
  57 +
  58 + /** Tier-1:VL 场景描述 + caption 语义向量 → caption/captionEmbedding,scanState 推到 2。 */
  59 + suspend fun captionStage(photo: Photo, vision: VisionClient, embed: EmbedClient?) {
  60 + withContext(Dispatchers.IO) {
  61 + val bmp = ImageUtil.decode(ctx, photo) ?: run {
  62 + Log.w(TAG, "decode failed (caption): ${photo.path}")
  63 + repo.photos.markCaptionSkipped(photo.id, System.currentTimeMillis())
  64 + return@withContext
  65 + }
  66 + val caption = try {
  67 + vision.caption(ImageUtil.toJpeg(bmp, maxDim = VL_MAX_DIM)).takeIf { it.isNotBlank() }
  68 + } catch (t: Throwable) {
  69 + Log.w(TAG, "caption failed: ${t.message}"); null
  70 + }
  71 + bmp.recycle()
  72 +
  73 + // caption 出来了就顺手算语义向量(embed 端点没配/失败就只存文本,不卡进度)。
  74 + val emb = if (caption != null && embed != null) {
  75 + try {
  76 + ImageUtil.packEmbedding(embed.embed(caption))
  77 + } catch (t: Throwable) {
  78 + Log.w(TAG, "embed failed: ${t.message}"); null
  79 + }
  80 + } else {
  81 + null
  82 + }
  83 +
  84 + repo.photos.markCaptionDone(photo.id, caption, caption?.let { "vl" }, emb, System.currentTimeMillis())
  85 + Log.i(TAG, "✓ photo#${photo.id}: caption=${caption?.take(30)} emb=${emb != null}")
60 } 86 }
61 } 87 }
62 88
@@ -64,5 +90,7 @@ class PhotoProcessor( @@ -64,5 +90,7 @@ class PhotoProcessor(
64 private const val TAG = "PhotoProcessor" 90 private const val TAG = "PhotoProcessor"
65 /** 同人余弦阈值(实测同人 0.96 / 异人 0.13,取 0.45 稳)。 */ 91 /** 同人余弦阈值(实测同人 0.96 / 异人 0.13,取 0.45 稳)。 */
66 const val THRESH = 0.45f 92 const val THRESH = 0.45f
  93 + /** 喂 VL 的最长边:448 比 512 再省些 vision token,粗粒度场景描述精度几乎不掉。 */
  94 + const val VL_MAX_DIM = 448
67 } 95 }
68 } 96 }
lifelog-recorder/app/src/main/kotlin/com/lemon/lifelog/vision/PhotoScheduler.kt
@@ -12,40 +12,67 @@ import androidx.work.WorkManager @@ -12,40 +12,67 @@ import androidx.work.WorkManager
12 import java.util.concurrent.TimeUnit 12 import java.util.concurrent.TimeUnit
13 13
14 /** 14 /**
15 - * 相册照片处理调度(与纪要 SummaryScheduler 同构):  
16 - * - schedulePeriodic:每 15min(WorkManager 硬下限)扫一遍未处理照片,闲时跑。  
17 - * - runNow:导入后触发一次,让新照片尽快在后台开始处理(不阻塞导入 UI)。 15 + * 相册分层处理调度:
  16 + * - Tier-0(人脸+扫描,PhotoFaceWorker):每 15min,只要不低电就跑——快,尽快铺满全相册人脸/时间线。
  17 + * - Tier-1(VL 场景描述,PhotoVlWorker):每 15min,**充电 + 不低电**才跑——慢、耗电,长期后台补。
  18 + * - runNow / scanNow:导入或进相册页时手动催一次(不阻塞 UI)。
18 */ 19 */
19 object PhotoScheduler { 20 object PhotoScheduler {
20 - private const val PERIODIC_TAG = "lifelog-photo-periodic"  
21 - private const val ONE_SHOT_TAG = "lifelog-photo-once" 21 + private const val FACE_PERIODIC = "lifelog-photo-face-periodic"
  22 + private const val VL_PERIODIC = "lifelog-photo-vl-periodic"
  23 + private const val FACE_ONCE = "lifelog-photo-face-once"
  24 + private const val VL_ONCE = "lifelog-photo-vl-once"
22 25
23 fun schedulePeriodic(ctx: Context) { 26 fun schedulePeriodic(ctx: Context) {
24 - val constraints = Constraints.Builder()  
25 - .setRequiredNetworkType(NetworkType.NOT_REQUIRED) // VL 在本机  
26 - .setRequiresBatteryNotLow(true) 27 + val wm = WorkManager.getInstance(ctx)
  28 +
  29 + val faceReq = PeriodicWorkRequestBuilder<PhotoFaceWorker>(15, TimeUnit.MINUTES)
  30 + .setConstraints(
  31 + Constraints.Builder()
  32 + .setRequiredNetworkType(NetworkType.NOT_REQUIRED)
  33 + .setRequiresBatteryNotLow(true)
  34 + .build(),
  35 + )
  36 + .setBackoffCriteria(BackoffPolicy.LINEAR, 1, TimeUnit.MINUTES)
27 .build() 37 .build()
28 - val request = PeriodicWorkRequestBuilder<PhotoWorker>(15, TimeUnit.MINUTES)  
29 - .setConstraints(constraints) 38 + wm.enqueueUniquePeriodicWork(FACE_PERIODIC, ExistingPeriodicWorkPolicy.UPDATE, faceReq)
  39 +
  40 + val vlReq = PeriodicWorkRequestBuilder<PhotoVlWorker>(15, TimeUnit.MINUTES)
  41 + .setConstraints(
  42 + Constraints.Builder()
  43 + .setRequiredNetworkType(NetworkType.NOT_REQUIRED)
  44 + .setRequiresBatteryNotLow(true)
  45 + .setRequiresCharging(true) // 全量 VL 很烧电,只在充电时推进
  46 + .build(),
  47 + )
30 .setBackoffCriteria(BackoffPolicy.LINEAR, 1, TimeUnit.MINUTES) 48 .setBackoffCriteria(BackoffPolicy.LINEAR, 1, TimeUnit.MINUTES)
31 .build() 49 .build()
32 - WorkManager.getInstance(ctx).enqueueUniquePeriodicWork(  
33 - PERIODIC_TAG, ExistingPeriodicWorkPolicy.UPDATE, request,  
34 - ) 50 + wm.enqueueUniquePeriodicWork(VL_PERIODIC, ExistingPeriodicWorkPolicy.UPDATE, vlReq)
35 } 51 }
36 52
37 - /** 导入后调一次,让后台尽快开工。已在跑则忽略(KEEP)。 */ 53 + /** 导入后催一次:先铺人脸(不要求充电),VL 留给周期/充电时。 */
38 fun runNow(ctx: Context) { 54 fun runNow(ctx: Context) {
39 - val constraints = Constraints.Builder()  
40 - .setRequiresBatteryNotLow(true) 55 + val req = OneTimeWorkRequestBuilder<PhotoFaceWorker>()
  56 + .setConstraints(Constraints.Builder().setRequiresBatteryNotLow(true).build())
  57 + .setBackoffCriteria(BackoffPolicy.LINEAR, 1, TimeUnit.MINUTES)
41 .build() 58 .build()
42 - val request = OneTimeWorkRequestBuilder<PhotoWorker>()  
43 - .setConstraints(constraints)  
44 - .addTag(ONE_SHOT_TAG) 59 + WorkManager.getInstance(ctx).enqueueUniqueWork(FACE_ONCE, ExistingWorkPolicy.KEEP, req)
  60 + }
  61 +
  62 + /** 进相册页催一次全相册扫描 + Tier-0(同 runNow,仅语义更明确)。 */
  63 + fun scanNow(ctx: Context) = runNow(ctx)
  64 +
  65 + /** 手动催一次 VL(设置/相册页「立即理解」按钮用,绕开充电约束)。 */
  66 + fun runVlNow(ctx: Context) {
  67 + val req = OneTimeWorkRequestBuilder<PhotoVlWorker>()
  68 + .setConstraints(Constraints.Builder().setRequiresBatteryNotLow(true).build())
45 .setBackoffCriteria(BackoffPolicy.LINEAR, 1, TimeUnit.MINUTES) 69 .setBackoffCriteria(BackoffPolicy.LINEAR, 1, TimeUnit.MINUTES)
46 .build() 70 .build()
47 - WorkManager.getInstance(ctx).enqueueUniqueWork(  
48 - ONE_SHOT_TAG, ExistingWorkPolicy.KEEP, request,  
49 - ) 71 + WorkManager.getInstance(ctx).enqueueUniqueWork(VL_ONCE, ExistingWorkPolicy.KEEP, req)
  72 + }
  73 +
  74 + /** 停止 VL:取消手动的一次性 VL(周期由 paused 标志另行 gate)。 */
  75 + fun cancelVl(ctx: Context) {
  76 + WorkManager.getInstance(ctx).cancelUniqueWork(VL_ONCE)
50 } 77 }
51 } 78 }
lifelog-recorder/app/src/main/kotlin/com/lemon/lifelog/vision/PhotoVlState.kt 0 → 100644
  1 +package com.lemon.lifelog.vision
  2 +
  3 +import android.content.Context
  4 +import kotlinx.coroutines.flow.MutableStateFlow
  5 +import kotlinx.coroutines.flow.StateFlow
  6 +
  7 +/**
  8 + * Tier-1 VL 处理的运行态(给相册 UI:进度/暂停/停止 + 单张处理指示)。
  9 + * 跟 RecorderState 同形——纯进程内 StateFlow,worker 推、UI 收。paused 持久化到 prefs,
  10 + * 这样周期/充电时的自动 worker 也能尊重「暂停」。
  11 + */
  12 +object PhotoVlState {
  13 + /** VL worker 是否正在循环处理(拿到锁那个)。 */
  14 + val running = MutableStateFlow(false)
  15 +
  16 + /** 当前正在出场景描述的照片 id(null = 没有)。UI 在对应格上转圈。 */
  17 + val currentPhotoId = MutableStateFlow<Long?>(null)
  18 +
  19 + private val _paused = MutableStateFlow(false)
  20 +
  21 + /** 用户「暂停」标志(持久化)。worker 每张前检查;为 true 时不取新照片。 */
  22 + val paused: StateFlow<Boolean> = _paused
  23 +
  24 + private const val PREFS = "photo_vl"
  25 + private const val KEY_PAUSED = "paused"
  26 +
  27 + /** App 启动时把持久化的暂停标志载入 StateFlow(在 LifelogApp.onCreate 调)。 */
  28 + fun init(ctx: Context) {
  29 + _paused.value = ctx.getSharedPreferences(PREFS, Context.MODE_PRIVATE).getBoolean(KEY_PAUSED, false)
  30 + }
  31 +
  32 + fun setPaused(ctx: Context, v: Boolean) {
  33 + ctx.getSharedPreferences(PREFS, Context.MODE_PRIVATE).edit().putBoolean(KEY_PAUSED, v).apply()
  34 + _paused.value = v
  35 + }
  36 +
  37 + /** worker 读这个(持久化值为准,跨进程/重启可靠)。 */
  38 + fun isPaused(ctx: Context): Boolean =
  39 + ctx.getSharedPreferences(PREFS, Context.MODE_PRIVATE).getBoolean(KEY_PAUSED, false)
  40 +}
lifelog-recorder/app/src/main/kotlin/com/lemon/lifelog/vision/PhotoVlWorker.kt 0 → 100644
  1 +package com.lemon.lifelog.vision
  2 +
  3 +import android.content.Context
  4 +import android.util.Log
  5 +import androidx.work.CoroutineWorker
  6 +import androidx.work.WorkerParameters
  7 +import com.lemon.lifelog.chat.ChatConfig
  8 +import com.lemon.lifelog.data.Repository
  9 +import java.util.concurrent.atomic.AtomicBoolean
  10 +
  11 +/**
  12 + * Tier-1 worker:给已铺人脸的照片(scanState=1)按优先级跑 Qwen-VL 场景描述 + caption 语义向量。
  13 + *
  14 + * 慢(~10-40s/张,打本地 llama-server),所以:充电 + 不低电才跑、优先级队列(有人脸>近期)、
  15 + * 单轮 7min 预算 + retry。全相册全量 VL 是个长期后台任务,进度在相册页可见。
  16 + *
  17 + * 并发互斥:periodic + 手动 + retry 可能并发,多 worker 打同一 llama-server 会 502。
  18 + */
  19 +class PhotoVlWorker(ctx: Context, params: WorkerParameters) : CoroutineWorker(ctx, params) {
  20 +
  21 + override suspend fun doWork(): Result {
  22 + // 用户暂停时彻底不跑(含周期/充电触发)。
  23 + if (PhotoVlState.isPaused(applicationContext)) return Result.success()
  24 + if (!isRunning.compareAndSet(false, true)) return Result.success()
  25 + PhotoVlState.running.value = true
  26 + try {
  27 + val repo = Repository(applicationContext)
  28 + val pending = repo.photos.pendingCaption()
  29 + if (pending.isEmpty()) return Result.success()
  30 +
  31 + val base = ChatConfig.localBase(applicationContext)
  32 + val vision = VisionClient(base)
  33 + val embed = EmbedClient(base, ChatConfig.modelEmbed(applicationContext))
  34 + val proc = PhotoProcessor(applicationContext, repo)
  35 + val deadline = System.currentTimeMillis() + RUN_BUDGET_MS
  36 + var done = 0
  37 + for (p in pending) {
  38 + if (System.currentTimeMillis() > deadline) break
  39 + if (PhotoVlState.isPaused(applicationContext)) break // 暂停:当前张不再开新的
  40 + PhotoVlState.currentPhotoId.value = p.id
  41 + try {
  42 + proc.captionStage(p, vision, embed)
  43 + } catch (t: Throwable) {
  44 + Log.w(TAG, "captionStage failed ${p.path}: ${t.message}")
  45 + repo.photos.markCaptionSkipped(p.id, System.currentTimeMillis()) // 别卡队列
  46 + }
  47 + done++
  48 + }
  49 + Log.i(TAG, "tier1 vl: $done / ${pending.size}")
  50 + // 暂停时别 retry(否则一直重排);正常没跑完才 retry。
  51 + val paused = PhotoVlState.isPaused(applicationContext)
  52 + return if (!paused && done < pending.size) Result.retry() else Result.success()
  53 + } catch (t: Throwable) {
  54 + Log.e(TAG, "worker failed", t)
  55 + return Result.retry()
  56 + } finally {
  57 + PhotoVlState.currentPhotoId.value = null
  58 + PhotoVlState.running.value = false
  59 + isRunning.set(false)
  60 + }
  61 + }
  62 +
  63 + companion object {
  64 + const val TAG = "PhotoVlWorker"
  65 + /** 单轮预算 7min(VL ~10-40s/张),避开 WorkManager ~10min 上限。 */
  66 + const val RUN_BUDGET_MS = 7 * 60 * 1000L
  67 + private val isRunning = AtomicBoolean(false)
  68 + }
  69 +}