Commit ab6176a45f9976455983f2f82d758881a2fb71ed
1 parent
e8e21050
Pin Stage2 default duration to 10s and add default dance-video prompt template
Duration: fixed 10s (within H3's 4-15s hard constraint), frames=240 @ 24fps. Prompt: follows the internal H3 prompt-writing methodology already sitting in the repo (docs/h3skill.md, from a parallel session) — the "all-purpose reference mode" six-section structure. Character appearance details are auto-captioned from the Stage1 composed image (new composed_images. appearanceDescriptor field); dance motion phrasing is admin-authored once per reference video (new dance_refs.motionDescriptor field), since the format requires concrete descriptors and forbids "as shown in the picture." Flagged as needing a pass through h3skill.md's own 22-item mechanical acceptance checklist before production use, not claimed as fully compliant. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SdtXTsDRz5QWMjvt1HBXLz
Showing
2 changed files
with
111 additions
and
4 deletions
docs/todo/dressup-dance-room-mvcp.md
| @@ -82,6 +82,7 @@ | @@ -82,6 +82,7 @@ | ||
| 82 | | `videoUrl` | string | | | 82 | | `videoUrl` | string | | |
| 83 | | `styleTags` | array | | | 83 | | `styleTags` | array | | |
| 84 | | `durationSec` | number | | | 84 | | `durationSec` | number | | |
| 85 | +| `motionDescriptor` | string | **管ç†å‘˜æ‰‹å†™çš„动作æè¿°**(如"拉ä¸é£Žæ ¼æ‘†èƒ¯+举臂旋转"),拼进 Stage2 prompt 模æ¿çš„ `detailed_description`(已确认,需求文档 4.2/6.3 èŠ‚ï¼‰ï¼Œä¸Šä¼ æ—¶å¿…å¡« | | ||
| 85 | | `copyrightSource` | enum | `test_placeholder` \| `self_made` \| `licensed`(第一期全部 `test_placeholder`,6.3/8 节) | | 86 | | `copyrightSource` | enum | `test_placeholder` \| `self_made` \| `licensed`(第一期全部 `test_placeholder`,6.3/8 节) | |
| 86 | | `isActive` | boolean | | | 87 | | `isActive` | boolean | | |
| 87 | | `createdAt` | number (ts) | | | 88 | | `createdAt` | number (ts) | | |
| @@ -95,6 +96,7 @@ | @@ -95,6 +96,7 @@ | ||
| 95 | | `garmentId` | string | 外键 | | 96 | | `garmentId` | string | 外键 | |
| 96 | | `imageUrl` | string | åˆæˆç»“æžœ | | 97 | | `imageUrl` | string | åˆæˆç»“æžœ | |
| 97 | | `model` | string | 实际调用的 fal.ai æ¨¡åž‹æ ‡è¯†ï¼ˆ4.1/8 节,待选型) | | 98 | | `model` | string | 实际调用的 fal.ai æ¨¡åž‹æ ‡è¯†ï¼ˆ4.1/8 节,待选型) | |
| 99 | +| `appearanceDescriptor` | string | **Stage1 完æˆåŽè‡ªåŠ¨è°ƒè§†è§‰æ¨¡åž‹æè¿°è¿™å¼ æ¢è£…图**ï¼Œç”Ÿæˆ 2-3 æ¡å…·ä½“特å¾ï¼ˆå‘色/å‘åž‹ã€æœè£…颜色款å¼ç‰ï¼‰ï¼Œæ‹¼è¿› Stage2 prompt 模æ¿çš„ `subject_definitions`/`detailed_description`(已确认,需求文档 4.2 节)。生æˆå¤±è´¥ä¸é˜»å¡žâ€”—留空则 Stage2 ç›´æŽ¥æ‹’ç»æäº¤å¹¶é‡è¯•æ‰“æ ‡ï¼Œè€Œä¸æ˜¯æ‹¿"如图所示"è¿™ç§ä¸åˆè§„æè¿°ç¡¬å‡‘(H3 规范明确ä¸å…许) | | ||
| 98 | | `status` | enum | `generating` \| `done` \| `failed` | | 100 | | `status` | enum | `generating` \| `done` \| `failed` | |
| 99 | | `createdAt` | number (ts) | | | 101 | | `createdAt` | number (ts) | | |
| 100 | 102 | ||
| @@ -157,6 +159,78 @@ erDiagram | @@ -157,6 +159,78 @@ erDiagram | ||
| 157 | garments ||--o{ moderation_log : "å®¡æ ¸è®°å½•" | 159 | garments ||--o{ moderation_log : "å®¡æ ¸è®°å½•" |
| 158 | ``` | 160 | ``` |
| 159 | 161 | ||
| 162 | +### M.11 Stage2 默认 Prompt 模æ¿ï¼ˆå·²ç¡®è®¤ï¼Œéœ€æ±‚文档 4.2 节) | ||
| 163 | + | ||
| 164 | +便®ä»“库里的 `docs/h3skill.md`(内部 H3 æç¤ºè¯å†™ä½œæ–¹æ³•论)"全能å‚考 | ||
| 165 | +模å¼å…段å¼"结构写æˆã€‚**`{{...}}` 是å ä½ç¬¦**,由 M.6/M.5 çš„ | ||
| 166 | +`appearanceDescriptor`/`motionDescriptor` å—æ®µåœ¨æ‹¼è£…时填入: | ||
| 167 | + | ||
| 168 | +- `<Picture 1>` = Stage1 æ¢è£…åˆæˆå›¾ï¼ˆ`composed_images.imageUrl`) | ||
| 169 | +- `<Video 1>` = å‚考跳舞视频(`dance_refs.videoUrl`),åªä½œåŠ¨ä½œ/è¿é•œ/ | ||
| 170 | + 节å¥å‚è€ƒï¼Œä¸æ˜¯è§†è§‰èº«ä»½æ¥æº | ||
| 171 | +- `<Audio 1>` = å¤ç”¨ `<Video 1>` çš„é…ä¹è½¨ï¼ˆH3 规范里å‚考视频ä¸å› 为 | ||
| 172 | + å«å£°å°±è‡ªåŠ¨äº§ç”Ÿ `<Audio N>`ï¼Œéœ€è¦æ˜¾å¼å®šä¹‰ï¼Œè§ `docs/h3skill.md` H15) | ||
| 173 | + | ||
| 174 | +``` | ||
| 175 | +subject_definitions: | ||
| 176 | +<Subject 1> is the young woman whose identity is defined by <Picture 1>. | ||
| 177 | +<Picture 1> is the character reference frame: preserve her | ||
| 178 | +{{appearanceDescriptor}} exactly as shown, changing only her pose and | ||
| 179 | +motion to match the choreography described below. | ||
| 180 | +<Video 1> is the reference choreography clip that defines the camera | ||
| 181 | +rhythm and dance structure the target video should follow; it is not a | ||
| 182 | +visual identity source. | ||
| 183 | + | ||
| 184 | +summary: | ||
| 185 | +[reference generation] The target video shows <Subject 1>, wearing the | ||
| 186 | +outfit from <Picture 1>, performing {{motionDescriptor}} following the | ||
| 187 | +pacing and camera movement established in <Video 1>. | ||
| 188 | + | ||
| 189 | +retention_analysis: | ||
| 190 | +<Subject 1> (appears in [Shot 1]): fully_preserved - her face, hairstyle, | ||
| 191 | +and outfit from <Picture 1> are retained throughout. | ||
| 192 | +<Video 1>: weak_reference - only the choreography rhythm and camera | ||
| 193 | +movement pattern are referenced, no visual content is retained. | ||
| 194 | +<Audio 1>: fully_copy - the background music track from <Video 1> is | ||
| 195 | +reused verbatim. | ||
| 196 | + | ||
| 197 | +detailed_description: | ||
| 198 | +A vibrant, mobile-first vertical dance video with warm, glossy studio | ||
| 199 | +lighting. [Shot 1] <Subject 1>, established from <Picture 1> with | ||
| 200 | +{{appearanceDescriptor}}, stands centered in a medium-full shot against a | ||
| 201 | +softly lit backdrop. She performs {{motionDescriptor}}, her weight | ||
| 202 | +shifting smoothly on the beat, arms tracing clean arcs in rhythm with the | ||
| 203 | +music. The camera holds mostly static with a slow, subtle Push In across | ||
| 204 | +the shot, keeping her fully in frame throughout. Her expression is | ||
| 205 | +confident and playful, eyes occasionally meeting the camera. Fabric of | ||
| 206 | +her outfit moves naturally with each step and turn. The routine follows | ||
| 207 | +the pacing and structure established in <Video 1> — building energy | ||
| 208 | +through the middle of the clip before settling. The ending pose returns | ||
| 209 | +close to the opening stance so the clip can loop cleanly for repeated | ||
| 210 | +live playback. | ||
| 211 | + | ||
| 212 | +overall_soundscape: | ||
| 213 | +Soft ambient room tone under the music, with faint fabric rustle as | ||
| 214 | +<Subject 1> moves and rhythmic footfall on the studio floor. | ||
| 215 | + | ||
| 216 | +non_diegetic_music: | ||
| 217 | +An upbeat track carries the routine; its tempo, instrumentation, and | ||
| 218 | +dynamics are reused from <Audio 1>, driving the pacing of her movement | ||
| 219 | +throughout. | ||
| 220 | +``` | ||
| 221 | + | ||
| 222 | +**填入示例**(仅示æ„,éžçœŸå®žç´ æï¼‰ï¼š`{{appearanceDescriptor}}` = | ||
| 223 | +"shoulder-length wavy black hair, a fitted red qipao with gold trim, and | ||
| 224 | +a slender, graceful build"ï¼›`{{motionDescriptor}}` = "a lively | ||
| 225 | +Latin-inspired routine with hip sways and raised-arm turns"。 | ||
| 226 | + | ||
| 227 | +**已知未过机械验收的地方**(对应需求文档 8 节第 9 æ¡ï¼Œæ£å¼æŽ¥å…¥å‰éœ€è¦ | ||
| 228 | +按 `docs/h3skill.md` §1.7 çš„ 22 项清å•逿¡æ ¸å®žï¼Œè¿™é‡Œä¸å‡è£…å·²ç»è¿‡å®¡ï¼‰ï¼š | ||
| 229 | +`detailed_description` çš„è‹±æ–‡è¯æ•°éœ€è¦å¡åœ¨ 350–500 è¯åŒºé—´ï¼ˆH6ï¼‰â€”â€”ä¸Šé¢ | ||
| 230 | +这版是手写的示例长度,没有åšé€è¯è®¡æ•°ï¼›`retention_analysis` 用的四个 | ||
| 231 | +å–值(`fully_preserved`/`weak_reference`/`fully_copy`)都在 H9 白åå• | ||
| 232 | +å†…ï¼Œä½†æ•´ä½“å…æ®µçš„äº”å¤„æ ‡ç¾åŒä¹‰ï¼ˆH14)ç‰ç»†èŠ‚å»ºè®®æ£å¼æŽ¥å…¥å‰å†æ ¸ä¸€é。 | ||
| 233 | + | ||
| 160 | --- | 234 | --- |
| 161 | 235 | ||
| 162 | ## V — View:å‰åŽç«¯ç•Œé¢è®¾è®¡ | 236 | ## V — View:å‰åŽç«¯ç•Œé¢è®¾è®¡ |
| @@ -245,7 +319,8 @@ POST /api/admin/credit/adjust # 人工调整积分(客æœåœºæ™¯ï | @@ -245,7 +319,8 @@ POST /api/admin/credit/adjust # 人工调整积分(客æœåœºæ™¯ï | ||
| 245 | 319 | ||
| 246 | å†…éƒ¨ï¼ˆç”Ÿæˆæµæ°´çº¿ç¼–排,ä¸å¯¹å‰ç«¯æš´éœ²ï¼‰ï¼š | 320 | å†…éƒ¨ï¼ˆç”Ÿæˆæµæ°´çº¿ç¼–排,ä¸å¯¹å‰ç«¯æš´éœ²ï¼‰ï¼š |
| 247 | generateComposedImage(characterId, garmentId) # Stage1,查缓å˜(M.6唯一索引)→未命ä¸è°ƒfal.ai | 321 | generateComposedImage(characterId, garmentId) # Stage1,查缓å˜(M.6唯一索引)→未命ä¸è°ƒfal.ai |
| 248 | -generateDanceVideo(composedImageId, refDanceVideoId, backend) # Stage2,backend按4.3节路由 | 322 | +captionComposedImage(composedImageId) # Stage1完æˆåŽè‡ªåŠ¨æ‰“æ ‡ï¼Œå†™ appearanceDescriptor(M.11) |
| 323 | +generateDanceVideo(composedImageId, refDanceVideoId, backend) # Stage2,按 M.11 æ¨¡æ¿æ‹¼ prompt,backend按4.3节路由 | ||
| 249 | ``` | 324 | ``` |
| 250 | 325 | ||
| 251 | --- | 326 | --- |
| @@ -304,6 +379,9 @@ sequenceDiagram | @@ -304,6 +379,9 @@ sequenceDiagram | ||
| 304 | else æœªå‘½ä¸ | 379 | else æœªå‘½ä¸ |
| 305 | S->>Fal: Stage1 æ¢è£…åˆæˆ | 380 | S->>Fal: Stage1 æ¢è£…åˆæˆ |
| 306 | Fal-->>S: composedImageUrl | 381 | Fal-->>S: composedImageUrl |
| 382 | + S->>Fal: captionComposedImage è‡ªåŠ¨æ‰“æ ‡ï¼ˆM.11ï¼Œç”Ÿæˆ appearanceDescriptor) | ||
| 383 | + Fal-->>S: 2-3 æ¡å¤–è§‚ç‰¹å¾æ–‡å— | ||
| 384 | + S->>S: 按 M.11 æ¨¡æ¿æ‹¼è£… Stage2 prompt | ||
| 307 | S->>H3: Stage2(冷å¯åŠ¨ç®—åŠ›ï¼Œ4.3节) | 385 | S->>H3: Stage2(冷å¯åŠ¨ç®—åŠ›ï¼Œ4.3节) |
| 308 | H3-->>S: è§†é¢‘å®Œæˆ | 386 | H3-->>S: è§†é¢‘å®Œæˆ |
| 309 | S->>S: 写入 generated_videos + composed_images ç¼“å˜ | 387 | S->>S: 写入 generated_videos + composed_images ç¼“å˜ |
docs/todo/dressup-dance-room.md
| @@ -88,12 +88,32 @@ | @@ -88,12 +88,32 @@ | ||
| 88 | 后确定),复用 `server.js` 已有的 `/fal/*` 反代通道——H3 Console 本身 | 88 | 后确定),复用 `server.js` 已有的 `/fal/*` 反代通道——H3 Console 本身 |
| 89 | 不产出图片,Stage1 无论如何都得走别的模型。 | 89 | 不产出图片,Stage1 无论如何都得走别的模型。 |
| 90 | 90 | ||
| 91 | -### 4.2 Stage 2:换装参考图 + 参考跳舞视频 → 生成跳舞视频 | 91 | +### 4.2 Stage 2:换装参考图 + 参考跳舞视频 → 生成跳舞视频(已确认:默认时长 10 秒) |
| 92 | 92 | ||
| 93 | 换装图(`reference_image_urls`)+ 参考跳舞视频(`reference_video_urls`, | 93 | 换装图(`reference_image_urls`)+ 参考跳舞视频(`reference_video_urls`, |
| 94 | 提供舞蹈动作/运镜/配乐节奏,系统随机/轮换挑选,观众不选风格)→ 生成 | 94 | 提供舞蹈动作/运镜/配乐节奏,系统随机/轮换挑选,观众不选风格)→ 生成 |
| 95 | -10 秒左右的跳舞视频片段。产物字段是 v1 `library.json` clip 的超集,加 | ||
| 96 | -`characterId`/`garmentId`/`composedImageId`/`refDanceVideoId` 溯源字段。 | 95 | +**固定 10 秒**的跳舞视频片段(H3 输出时长硬约束是 4–15 秒,10 秒是我们 |
| 96 | +在这个区间内选定的固定默认值,不随请求变化;对应 `frames = 240`,按 | ||
| 97 | +`H3-CONSOLE.md` 的 frames/24fps=秒 换算公式)。产物字段是 v1 | ||
| 98 | +`library.json` clip 的超集,加 `characterId`/`garmentId`/ | ||
| 99 | +`composedImageId`/`refDanceVideoId` 溯源字段。 | ||
| 100 | + | ||
| 101 | +**Stage2 的 prompt 遵循内部 H3 提示词写作方法论**(仓库里的 | ||
| 102 | +`docs/h3skill.md`——这是另一并行工作产出的官方 H3 提示词写作规范, | ||
| 103 | +内容独立于本项目其他文档但直接适用,复用而不是另起一套),走它定义的 | ||
| 104 | +"全能参考模式六段式"结构(`subject_definitions` → `summary` → | ||
| 105 | +`retention_analysis` → `detailed_description` → `overall_soundscape` → | ||
| 106 | +`non_diegetic_music`)。具体默认模板见 | ||
| 107 | +`docs/todo/dressup-dance-room-mvcp.md` 新增章节,两处需要自动/人工填空 | ||
| 108 | +才能拼出合规 prompt(该规范要求人物特征必须具体,不能写"如图所示"): | ||
| 109 | + | ||
| 110 | +- **人物外观描述**(发色/发型/服装颜色款式等 2-3 条具体特征):Stage1 | ||
| 111 | + 换装图生成完成后,**自动调一次视觉模型描述该图**,生成这几条特征 | ||
| 112 | + 文字,随 `composed_images` 记录一起落库,不用每次生成视频都重新 | ||
| 113 | + 描述(已确认)。 | ||
| 114 | +- **舞蹈动作描述**("拉丁风格摆胯+举臂旋转"这类具体动作文字):参考 | ||
| 115 | + 跳舞视频总量有限(后台预置的一个小集合),**由管理员在上传参考视频 | ||
| 116 | + 时手写一段短描述**,一次性录入、长期复用(已确认,见 6.3 节)。 | ||
| 97 | 117 | ||
| 98 | ### 4.3 双算力策略(已确认) | 118 | ### 4.3 双算力策略(已确认) |
| 99 | 119 | ||
| @@ -201,6 +221,9 @@ Stage1(换装图合成)在两条路径下大概率都是 fal.ai(H3 Console | @@ -201,6 +221,9 @@ Stage1(换装图合成)在两条路径下大概率都是 fal.ai(H3 Console | ||
| 201 | ### 6.3 参考跳舞视频/音乐管理 | 221 | ### 6.3 参考跳舞视频/音乐管理 |
| 202 | 222 | ||
| 203 | - CRUD:上传参考跳舞视频(含配乐),命名、风格标签、时长、上下架开关。 | 223 | - CRUD:上传参考跳舞视频(含配乐),命名、风格标签、时长、上下架开关。 |
| 224 | +- **动作描述字段(已确认,4.2 节)**:上传时管理员手写一段短描述具体 | ||
| 225 | + 动作(如"拉丁风格摆胯+举臂旋转"),供 Stage2 prompt 模板拼装用, | ||
| 226 | + 一次录入、后续所有用到这条参考视频的生成都复用,不必每次重写。 | ||
| 204 | - 版权来源字段:当前**先用测试素材占位**(正式版权素材待确认,见第 8 | 227 | - 版权来源字段:当前**先用测试素材占位**(正式版权素材待确认,见第 8 |
| 205 | 节),后台需要能记录/后续批量替换。 | 228 | 节),后台需要能记录/后续批量替换。 |
| 206 | 229 | ||
| @@ -288,6 +311,12 @@ Stage1(换装图合成)在两条路径下大概率都是 fal.ai(H3 Console | @@ -288,6 +311,12 @@ Stage1(换装图合成)在两条路径下大概率都是 fal.ai(H3 Console | ||
| 288 | 7. **"我的作品"页面的第三方实名认证需求**:第一期只做"我已满18岁" | 311 | 7. **"我的作品"页面的第三方实名认证需求**:第一期只做"我已满18岁" |
| 289 | 勾选式确认(5.1 节),是否需要更严格的第三方实名认证,看实际运营 | 312 | 勾选式确认(5.1 节),是否需要更严格的第三方实名认证,看实际运营 |
| 290 | 风险再评估,不预设一定要做。 | 313 | 风险再评估,不预设一定要做。 |
| 314 | +8. **人物外观自动打标的视觉模型选型**:Stage1 完成后自动描述换装图 | ||
| 315 | + (4.2 节),具体调哪个视觉模型/接口待测试几个候选。 | ||
| 316 | +9. **Stage2 默认 prompt 模板的机械验收**:`docs/h3skill.md` 附带一套 | ||
| 317 | + 22 项输出前机械验收清单(含 `detailed_description` 350–500 英文词 | ||
| 318 | + 区间等硬约束),MVCP 文档里给的模板是按该规范手写的示例,正式接入 | ||
| 319 | + 前建议照那份清单逐条过一遍再定稿。 | ||
| 291 | 320 | ||
| 292 | ## 9. 分期建议(不是承诺的排期,是依赖顺序参考) | 321 | ## 9. 分期建议(不是承诺的排期,是依赖顺序参考) |
| 293 | 322 |