This commit is contained in:
Codex
2026-11-13 11:46:25 +08:00
parent f5223075c5
commit 702671e61b
103 changed files with 7080 additions and 201 deletions
+40 -1
View File
@@ -13,6 +13,29 @@
- 固定数据目录、一键启动、升级备份与真人实拍、仙侠风格预设。 - 固定数据目录、一键启动、升级备份与真人实拍、仙侠风格预设。
- 跨集结尾上下文、分镜起止状态、前镜头尾帧接续与持久化依赖队列。 - 跨集结尾上下文、分镜起止状态、前镜头尾帧接续与持久化依赖队列。
- 中英文视频提示词同步保存,MiniMax H3 提交前使用英文制作描述,保留中文对白与素材引用。 - 中英文视频提示词同步保存,MiniMax H3 提交前使用英文制作描述,保留中文对白与素材引用。
- 场景固定设定与场景参考图自动随每个分镜请求提交,覆盖单次、批量、换卡重做和高清制作。
- LTX-2.3 / Wan2.2 先用全部绑定的人物、场景、道具图合成完整分镜首帧,再将单张首帧交给视频模型;超过 6 个素材分轮融合,不静默丢图。
- 豆包英文补译关闭深度思考,拆分镜保存时最多同时补译 3 条,全部通过对白与素材标记校验后才保存。
## 场景一致性
视频制作的「场景固定设定」位于镜头衔接上方。同一场景 ID 的分镜(包括跨集复用)共用空间、固定陈设、时间和光源;没有单独固定设定时直接采用已有场景描述。编辑保存用于后续生成,不改写原分镜或旧视频。灯光开关、移动道具等剧情变化应记录在镜头起止状态中,换机位保持物理空间关系而非屏幕左右不变。
后端从绑定场景读取环境,单独加入每个视频/分镜图片请求,避免只写人物表演而遗漏背景。普通参考模式补带场景图并保留现有图片编号;尾帧接续和 VDN 高清沿用视频参考帧,同时保留环境文字。MiniMax H3 的环境描述翻译为英文,台词保持原文。缺失/失效的场景、超出模型参考图数量会在换卡取消旧任务之前报错;任务记录保存当时场景设定快照。
场景图应与文字设定一致。文字与参考图可减少漂移,但不保证生成画面逐像素一致;连续动作可使用已有「尾帧接续」。角色三视图、道具白底图不加入场景约束。没有场景图时页面明确提示仅有文字约束。
回归验证:`integration-work/test_scene_continuity.py` 使用独立 SQLite 和本机模拟模型服务,覆盖实际请求体、参考编号、换卡、跨集、转场、尾帧、高清、图片与数据持久化;设置 `SCENE_UI_TEST=1` 可同时验证页面保存和刷新。需先构建桌面后端及前端,并具备现有 Electron 测试运行环境。
## LTX / Wan 分镜首帧
视频制作选择 LTX-2.3、LTX-2.3-Text-Preserve 或 Wan2.2-I2V 后,点击生成会先使用顶部选定的 Qwen Image 2.1 图片配置制作完整画面。绑定的场景、人物、道具全部参与:6 个以内一次提交;更多素材以「上一轮合成图 + 最多 5 个新增素材」继续融合,保留每轮结果与素材清单。缺少参考图片会明确报错,需先补图;不再只选第一张场景图。
图片显卡跟随顶部图片配置,视频显卡跟随当前镜头设置。首帧准备完成后才提交视频;开启尾帧接续时,前镜头末帧也加入融合。右侧「分镜首帧」显示最终画面、素材清单和图片模型;输入未变时重试视频复用首帧,也可点击「重做首帧并生成视频」。修改分镜或绑定素材后重新合成,生成中修改会阻止旧首帧提交视频。
准备首帧期间可取消父视频任务,同时取消其当前图片任务;重启继续跟踪已提交图片,不重复生成。首帧失败不会提交视频。中间图与最终图进入任务历史,不覆盖人物原图、场景原图或旧视频。全参考图流程减少身份与环境漂移,生成结果仍需检查;跨多个融合轮次可能出现误差。
回归脚本:`integration-work/test_composed_first_frame.py`,设置 `COMPOSE_UI_TEST=1` 同时检查真实界面。使用隔离数据库和本机模拟服务,覆盖 15 张素材、GPU 路由、尾帧衔接、失败、取消、重启与避免重复提交,不消耗真实模型算力。
## 英文视频提示词 ## 英文视频提示词
@@ -26,7 +49,7 @@ MiniMax-H3 系列(含加速版、VDN 高清)提交前自动补齐或更新
- **独立镜头 / 转场**:不依赖前段,适合换场景、时间跳跃或开场。 - **独立镜头 / 转场**:不依赖前段,适合换场景、时间跳跃或开场。
- **状态承接**:将前镜头结束状态与本镜头起止状态加入生成约束;允许换角度,仍使用本镜头的资产参考图。 - **状态承接**:将前镜头结束状态与本镜头起止状态加入生成约束;允许换角度,仍使用本镜头的资产参考图。
- **尾帧接续**:同场景连续动作。等待前段完成,从所选视频结束前 0.05–3 秒提取画面作为单张首帧,再向选定 GPU 提交。支持当前本地 MiniMax-H3、MiniMax-H3-Hybrid、Wan2.2-I2V;不与多张参考图或参考音色混用。 - **尾帧接续**:同场景连续动作。等待前段完成,从所选视频结束前 0.05–3 秒提取画面,再向选定 GPU 提交。当前本地 MiniMax-H3、MiniMax-H3-Hybrid 直接使用单张尾帧;LTX-2.3 和 Wan2.2-I2V 将尾帧与全部绑定素材融合后使用新首帧。不与参考音色混用。
来源默认为按集号、镜号排序的前一镜,也可选择本项目更早的镜头。不能引用未来、自身、其他项目或已删除的镜头。第一镜、转场需用独立模式。拆分镜工具会读取上一集结尾剧本及最后三个分镜;起止状态是文字设定,不代表系统已经识别或审核了成片画面。 来源默认为按集号、镜号排序的前一镜,也可选择本项目更早的镜头。不能引用未来、自身、其他项目或已删除的镜头。第一镜、转场需用独立模式。拆分镜工具会读取上一集结尾剧本及最后三个分镜;起止状态是文字设定,不代表系统已经识别或审核了成片画面。
@@ -36,12 +59,28 @@ MiniMax-H3 系列(含加速版、VDN 高清)提交前自动补齐或更新
## 视频进度与预计耗时 ## 视频进度与预计耗时
分镜拆分按每批最多 4 个段落校验中文、英文、时间轴和素材绑定,全部通过后一次性替换旧分镜。取消、超时、重启或校验失败均保留旧分镜;拆分期间修改剧本或原分镜,会阻止旧结果覆盖新编辑。
没有上传音色不影响拆分:生成草稿中误加的角色音色引用会改回普通人物引用;真实音色和对白原文保留。素材引用中的多余类别前缀会统一为实际资产名称。对白提到“血红文字”等内容不再误判为画面画字;实际要求画字仍会退回修正。连续三轮没有保存进展会停止并显示具体错误,诊断信息保存在本地任务状态中。
拆分任务按原文场次逐场运行,为每段原文分配来源编号,当前场次覆盖完成后才进入下一场。每场核对来源编号、按原文顺序保留的对白、最少镜头数和时长;仅出现结尾台词不能判定整集完成。预计镜头数量是节奏参考,不是固定上限;不能为了达到数量添加剧情。
长台词按原文切成小段,并给出约 4.5 汉字/秒的发声时长参考。校验具体对白时间段,不允许用几秒承载几十字台词;小幅时间分配错误会同步修正中英文时间轴,超过单镜头 15 秒容量时必须继续细分。全部场次通过校验才替换原分镜。来源编号和对白校验不等于人工审片,表演、调度及动作覆盖仍需检查。旧版本过度压缩的结果会提示重新细分,不自动改写已有镜头或视频。
视频预览下方、任务列表和高清制作页会显示排队、模型加载、采样、解码与编码阶段,以及已提交时长。采样步数来自对应 ComfyUI 任务的实时消息;积累足够的步速记录后显示当前采样阶段的剩余时间区间,后续解码、编码和保存另计。 视频预览下方、任务列表和高清制作页会显示排队、模型加载、采样、解码与编码阶段,以及已提交时长。采样步数来自对应 ComfyUI 任务的实时消息;积累足够的步速记录后显示当前采样阶段的剩余时间区间,后续解码、编码和保存另计。
整段视频耗时只参考同显卡、同工作流及规格的成功记录,不包含排队与下载;样本不足时不报估算。采样完成不代表成片完成,连接中断或数据过期会隐藏旧估算。进度读取不会取消、重新提交或改换显卡。 整段视频耗时只参考同显卡、同工作流及规格的成功记录,不包含排队与下载;样本不足时不报估算。采样完成不代表成片完成,连接中断或数据过期会隐藏旧估算。进度读取不会取消、重新提交或改换显卡。
服务器的进度扩展源文件为 `integration-work/comfy_progress.py`,由 `install_comfy_progress.py` 备份后嵌入现有适配器;无需重启 ComfyUI。图片、视频及私有配置仍不进入 Git。 服务器的进度扩展源文件为 `integration-work/comfy_progress.py`,由 `install_comfy_progress.py` 备份后嵌入现有适配器;无需重启 ComfyUI。图片、视频及私有配置仍不进入 Git。
## 下载后自动清理 ComfyUI 视频
本地视频任务(包括高清版)成功下载后,先完整解码校验、写入磁盘并保存到项目,才发送带文件长度和 SHA-256 的清理确认。服务器再次核对实际交付的视频内容,从该任务的历史记录中定位唯一输出,仅删除对应视频文件和该条 ComfyUI 历史记录。火宝中的本地成片、海报、历史版本、人物和场景参考图保持可用。
清理失败单独重试,不把已完成的视频改成失败;重启后继续确认,任务列表显示等待清理或已清理。未收到本地下载确认、正在运行、被其他任务引用、文件校验不符或路径不明确的任务不删除;既有已完成任务不批量追溯清理。
服务端运行 `comfy_video_cleanup.py` 用户服务,只消费两个私有桥接器保存的确认记录,没有新增网络监听,不重启 ComfyUI。先记录精确文件标识再删除,异常退出后可继续完成同一条记录。`test_video_cleanup_delivery.py`、`test_video_cleanup_api.py` 和 Linux 隔离测试 `test_comfy_video_cleanup.py` 分别验证本地下载、确认接口及删除范围。
## 取消任务 ## 取消任务
打开右上角「任务」,在对应图片或视频记录上点击「取消任务」。只停止这一条任务,不影响其他镜头、其他显卡任务或已有素材。等待前镜头的任务也可以直接取消。 打开右上角「任务」,在对应图片或视频记录上点击「取消任务」。只停止这一条任务,不影响其他镜头、其他显卡任务或已有素材。等待前镜头的任务也可以直接取消。
+9 -1
View File
@@ -19,6 +19,7 @@ import { loadAgentPromptFile, loadBasePromptFile } from './prompts.js'
import { buildLanguageDirective } from './language.js' import { buildLanguageDirective } from './language.js'
import { getContentLanguageFromRC } from './context.js' import { getContentLanguageFromRC } from './context.js'
import { getVoiceReferenceInstructions } from './voice-reference-instructions.js' import { getVoiceReferenceInstructions } from './voice-reference-instructions.js'
import { sceneContinuityInstructions } from '../services/scene-continuity.js'
// Default prompts (used when workspace/prompts/<type>.md 文件缺失时兜底) // Default prompts (used when workspace/prompts/<type>.md 文件缺失时兜底)
export const DEFAULT_PROMPTS: Record<string, { name: string; instructions: string }> = { export const DEFAULT_PROMPTS: Record<string, { name: string; instructions: string }> = {
@@ -372,7 +373,14 @@ function buildInstructions(type: string) {
const bilingualInstructions = ['storyboard_breaker', 'prompt_generator'].includes(type) ? ` const bilingualInstructions = ['storyboard_breaker', 'prompt_generator'].includes(type) ? `
双语视频提示词(优先于旧的单字段保存规则):每个分镜同步产出 video_prompt(中文供编辑)和 video_prompt_en(完整英文供视频模型)。英文版逐项忠实翻译画面、人物动作、运镜、光线、声音与否定约束,时间段、顺序和时长与中文完全一致;不得增减剧情。所有引号内对白、旁白逐字保留中文原文,不翻译、不音译;对白原文前紧接 says: 或 Dialogue:(不要写 says in Mandarin Chinese:,以兼容语音解析),表演说明写在该标记之前。@角色名/@场景名/@道具名/#音色名原样保留,不改名,不加新对白或字幕。英文仅用于制作描述,中文台词仍使用中文发声。不要把说明朗读为台词。 双语视频提示词(优先于旧的单字段保存规则):每个分镜同步产出 video_prompt(中文供编辑)和 video_prompt_en(完整英文供视频模型)。英文版逐项忠实翻译画面、人物动作、运镜、光线、声音与否定约束,时间段、顺序和时长与中文完全一致;不得增减剧情。所有引号内对白、旁白逐字保留中文原文,不翻译、不音译;对白原文前紧接 says: 或 Dialogue:(不要写 says in Mandarin Chinese:,以兼容语音解析),表演说明写在该标记之前。@角色名/@场景名/@道具名/#音色名原样保留,不改名,不加新对白或字幕。英文仅用于制作描述,中文台词仍使用中文发声。不要把说明朗读为台词。
save_storyboards 每条同时提供两种提示词。单独生成/更新视频提示词时 update_storyboard 只传 storyboard_id、video_prompt、video_prompt_en 三个字段,其他字段不动。` : '' save_storyboards 每条同时提供两种提示词。单独生成/更新视频提示词时 update_storyboard 只传 storyboard_id、video_prompt、video_prompt_en 三个字段,其他字段不动。` : ''
return [baseInstructions, skillInstructions, voiceInstructions, languageDirective, continuityInstructions, bilingualInstructions] const storyboardJobInstructions = type === 'storyboard_breaker' && requestContext?.get('storyboardJobId' as never) ? `
本次使用带进度的整集拆分任务,以下保存规则覆盖旧提示词中的批量保存说明:每批最多 4 个分镜,顺序调用 save_storyboards,禁止并行保存。只有第一批带 replace_existing=true,后续省略。所有批次先暂存校验,全部完成后系统统一替换原分镜;不要调用 update_storyboard 修改原分镜。读取上下文后直接生成第一批,不要输出规划文字;继续到剧本最后一句,不要把“已暂存”误当成整集已完成。
原文覆盖与合理时长同时满足才完成:系统按场次单独调用模型,每轮只拆 readStoryboardContext 返回的 script/source_units,禁止跳到下一场或整集结尾。每个分镜填写 source_ids。必须覆盖当前场次全部原文段落、全部对白,达到 current_section.min_seconds 和 min_shots;达到结尾台词本身不代表完成。时长不足应细分原有动作和对白,不能新增剧情或续写结尾。
对白按约 4.5 汉字/秒安排到具体时间段,镜头 8–15 秒。系统已将长对白切成保留原文的小段,source_units 的 speech_seconds 是这小段至少需要的发声秒数;一个镜头优先只放一小段对白,不要把多段长台词重新合并。给对白时间段至少留出 speech_seconds,剩余时间安排动作。不能用 3–4 秒承载几十字长台词,不能只延长总 duration 而不调整对白所在时间段。一个时间段可以长于 3 秒;保证时间轴连续即可。原文必须逐字保留,不能增删台词。返回 section_complete 后立即结束当前轮,下一场由系统安排,不得继续调用工具。
对白或来源未通过时先查看 coverage/current_section_coverage。repair_shot_numbers 列出的已有镜头必须原位修改并重存,不能把同一句话反复追加为新镜头。对白用“@角色名 + 表演动作 + 说:「原文」”的明确格式;怒吼、惨叫等表演不应改变对白原文。上一批没有减少缺失内容时立即修正,不要靠增加镜头数量假装完成。
素材标记直接使用 @ 加素材列表中的完整 name/location,不额外加“场景”“角色”等类别前缀。英文中所有引号内原文(含对白、拟声、称谓)与素材标记完整保留;不得漏译画面说明。` : ''
return [baseInstructions, skillInstructions, voiceInstructions, languageDirective, continuityInstructions, bilingualInstructions, storyboardJobInstructions,
['storyboard_breaker', 'prompt_generator'].includes(type) ? sceneContinuityInstructions : '']
.filter(Boolean) .filter(Boolean)
.join('\n\n') .join('\n\n')
} }
@@ -12,9 +12,12 @@ import { logTaskProgress, logTaskSuccess } from '../../utils/task-logger.js'
import { getDramaId, getEpisodeId } from '../context.js' import { getDramaId, getEpisodeId } from '../context.js'
import { previousEpisodeContext } from '../../services/continuity.js' import { previousEpisodeContext } from '../../services/continuity.js'
import { resolveVoiceMentions } from '../../services/character-voice-references.js' import { resolveVoiceMentions } from '../../services/character-voice-references.js'
import { normalizeDraftVoices, normalizeDraftAssets } from '../../utils/storyboard-generated-prompt.js'
import { sectionBatchProblems, sectionCoverage, fitDraftDialogueTiming } from '../../utils/storyboard-coverage.js'
import { isScriptStale, targetEpisodeSeconds } from '../../utils/script-source.js' import { isScriptStale, targetEpisodeSeconds } from '../../utils/script-source.js'
import { storyboardSegmentProblems } from '../../utils/storyboard-checks.js' import { storyboardSegmentProblems } from '../../utils/storyboard-checks.js'
import { bilingualFields, englishPromptStatus } from '../../services/english-video-prompt.js' import { bilingualFields, bilingualBatchFields, englishPromptStatus, promptAssetNames } from '../../services/english-video-prompt.js'
import { assertStoryboardJob, assertStoryboardSourceUnchanged, getStoryboardJob, rejectStoryboardBatch, stageStoryboardBatch, updateStoryboardJob } from '../../services/storyboard-job-state.js'
async function validateVideoPromptVoices(prompt: string | undefined, dramaId: number, characterIds: number[] = []) { async function validateVideoPromptVoices(prompt: string | undefined, dramaId: number, characterIds: number[] = []) {
if (!prompt?.includes('#')) return if (!prompt?.includes('#')) return
@@ -73,7 +76,7 @@ async function getEpisodePropIds(episodeId: number) {
return new Set(links.map(link => link.propId)) return new Set(links.map(link => link.propId))
} }
async function validateStoryboardBindings(episodeId: number, dramaId: number, sceneId: number | null | undefined, characterIds: number[] | undefined, propIds?: number[] | undefined) { async function validateStoryboardBindings(episodeId: number, dramaId: number, sceneId: number | null | undefined, characterIds: number[] | undefined, propIds?: number[] | undefined, dryRun = false) {
const episodeSceneIds = await getEpisodeSceneIds(episodeId) const episodeSceneIds = await getEpisodeSceneIds(episodeId)
const episodeCharacterIds = await getEpisodeCharacterIds(episodeId) const episodeCharacterIds = await getEpisodeCharacterIds(episodeId)
const episodePropIds = await getEpisodePropIds(episodeId) const episodePropIds = await getEpisodePropIds(episodeId)
@@ -84,7 +87,7 @@ async function validateStoryboardBindings(episodeId: number, dramaId: number, sc
if (!scene || scene.dramaId !== dramaId || scene.deletedAt) { if (!scene || scene.dramaId !== dramaId || scene.deletedAt) {
throw new Error(`scene_id ${sceneId} 不属于当前项目`) throw new Error(`scene_id ${sceneId} 不属于当前项目`)
} }
await db.insert(schema.episodeScenes).values({ episodeId, sceneId, createdAt: now() }) if (!dryRun) await db.insert(schema.episodeScenes).values({ episodeId, sceneId, createdAt: now() })
} }
const uniqueCharacterIds = [...new Set((characterIds || []).filter(Boolean))] const uniqueCharacterIds = [...new Set((characterIds || []).filter(Boolean))]
@@ -94,7 +97,7 @@ async function validateStoryboardBindings(episodeId: number, dramaId: number, sc
if (!character || character.dramaId !== dramaId || character.deletedAt) { if (!character || character.dramaId !== dramaId || character.deletedAt) {
throw new Error(`character_id ${characterId} 不属于当前项目`) throw new Error(`character_id ${characterId} 不属于当前项目`)
} }
await db.insert(schema.episodeCharacters).values({ episodeId, characterId, createdAt: now() }) if (!dryRun) await db.insert(schema.episodeCharacters).values({ episodeId, characterId, createdAt: now() })
} }
const uniquePropIds = [...new Set((propIds || []).filter(Boolean))] const uniquePropIds = [...new Set((propIds || []).filter(Boolean))]
@@ -104,11 +107,12 @@ async function validateStoryboardBindings(episodeId: number, dramaId: number, sc
if (!prop || prop.dramaId !== dramaId || prop.deletedAt) { if (!prop || prop.dramaId !== dramaId || prop.deletedAt) {
throw new Error(`prop_id ${propId} 不属于当前项目`) throw new Error(`prop_id ${propId} 不属于当前项目`)
} }
await db.insert(schema.episodeProps).values({ episodeId, propId, createdAt: now() }) if (!dryRun) await db.insert(schema.episodeProps).values({ episodeId, propId, createdAt: now() })
} }
} }
type ToolContext = ToolExecutionContext | undefined type ToolContext = ToolExecutionContext | undefined
const storyboardJobId = (context: ToolContext) => context?.requestContext?.get('storyboardJobId' as never) as string|undefined
function requireIds(context: ToolContext): { episodeId: number; dramaId: number } | { error: string } { function requireIds(context: ToolContext): { episodeId: number; dramaId: number } | { error: string } {
const episodeId = getEpisodeId(context?.requestContext) const episodeId = getEpisodeId(context?.requestContext)
@@ -125,6 +129,8 @@ const readStoryboardContext = createTool({
const ids = requireIds(context) const ids = requireIds(context)
if ('error' in ids) return ids if ('error' in ids) return ids
const { episodeId, dramaId } = ids const { episodeId, dramaId } = ids
const jobId=storyboardJobId(context)
if (jobId) assertStoryboardJob(episodeId,jobId)
const [ep] = await db.select().from(schema.episodes) const [ep] = await db.select().from(schema.episodes)
.where(eq(schema.episodes.id, episodeId)) .where(eq(schema.episodes.id, episodeId))
if (!ep) return { error: 'Episode not found' } if (!ep) return { error: 'Episode not found' }
@@ -134,6 +140,8 @@ const readStoryboardContext = createTool({
} }
const script = ep.scriptContent || ep.content const script = ep.scriptContent || ep.content
if (!script) return { error: 'Episode has no script' } if (!script) return { error: 'Episode has no script' }
const splitJob=jobId?assertStoryboardJob(episodeId,jobId):null
const sectionIndex=splitJob?.sectionIndex||0,section=splitJob?.sections?.[sectionIndex]
const charLinks = await db.select().from(schema.episodeCharacters) const charLinks = await db.select().from(schema.episodeCharacters)
.where(eq(schema.episodeCharacters.episodeId, episodeId)) .where(eq(schema.episodeCharacters.episodeId, episodeId))
@@ -153,7 +161,7 @@ const readStoryboardContext = createTool({
const prps = await db.select().from(schema.props) const prps = await db.select().from(schema.props)
.where(eq(schema.props.dramaId, dramaId)) .where(eq(schema.props.dramaId, dramaId))
const existingStoryboards = await db.select().from(schema.storyboards) const existingStoryboards = await db.select().from(schema.storyboards)
.where(eq(schema.storyboards.episodeId, episodeId)) .where(and(eq(schema.storyboards.episodeId, episodeId), isNull(schema.storyboards.deletedAt)))
const characters = chars const characters = chars
.filter(c => !c.deletedAt) .filter(c => !c.deletedAt)
@@ -180,6 +188,7 @@ const readStoryboardContext = createTool({
time: s.time, time: s.time,
prompt: s.prompt || '', prompt: s.prompt || '',
lighting: s.lighting || '', lighting: s.lighting || '',
environment_lock: s.continuityPrompt || s.prompt || '',
image_url: s.imageUrl || '', image_url: s.imageUrl || '',
storyboard_count: s.storyboardCount || 0, storyboard_count: s.storyboardCount || 0,
})) }))
@@ -230,16 +239,24 @@ const readStoryboardContext = createTool({
episode_number: ep.episodeNumber, episode_number: ep.episodeNumber,
description: ep.description || '', description: ep.description || '',
}, },
script, script:section?.script||script,
source_units:section?.units,
current_section:section?{id:section.id,index:sectionIndex+1,total:splitJob?.sectionCount,min_seconds:section.minSeconds,min_shots:section.minShots,target_seconds:section.targetSeconds}:undefined,
next_shot_number:splitJob?(splitJob.drafts.at(-1)?.shot_number||0)+1:1,
current_section_drafts:splitJob?.drafts.filter(s=>s.source_section===sectionIndex),
current_section_coverage:section&&splitJob?sectionCoverage(section,splitJob.drafts.filter(s=>s.source_section===sectionIndex)):undefined,
previous_section_end:splitJob?.drafts.filter(s=>s.source_section<sectionIndex).slice(-2).map(s=>({description:s.description,continuity_end:s.continuity_end})),
completion_rule: '只拆当前场次,覆盖 source_units 的全部段落与对白;每个分镜填写 source_ids。达到本场 min_seconds/min_shots 且对白全部出现才完成,禁止跳场或续写剧情。',
// 总量锚定的确定性数值(剧本去空白字数 ÷ 500 字/分钟),save_storyboards 会按它回报进度 // 总量锚定的确定性数值(剧本去空白字数 ÷ 500 字/分钟),save_storyboards 会按它回报进度
target_total_seconds: targetEpisodeSeconds(script), target_total_seconds: splitJob?.targetSeconds||targetEpisodeSeconds(script),
target_segment_count: Math.max(1, Math.round(targetEpisodeSeconds(script) / 12)), target_segment_count: splitJob?.target||Math.max(1, Math.round(targetEpisodeSeconds(script) / 12)),
characters, characters,
scenes, scenes,
props, props,
previous_episode: await previousEpisodeContext(episodeId), previous_episode: await previousEpisodeContext(episodeId),
existing_storyboards: existingStoryboardPayload, existing_storyboards: jobId ? [] : existingStoryboardPayload,
} }
if (jobId) updateStoryboardJob(episodeId,jobId,{phase:`正在拆分第 ${sectionIndex+1}/${splitJob?.sectionCount||1} 个场次(每批最多 4 个)`})
logTaskSuccess('StoryboardTool', 'read-context', { logTaskSuccess('StoryboardTool', 'read-context', {
episodeId, episodeId,
dramaId, dramaId,
@@ -255,6 +272,7 @@ const readStoryboardContext = createTool({
const storyboardFields = z.object({ const storyboardFields = z.object({
shot_number: z.number(), shot_number: z.number(),
source_ids:z.array(z.string()).optional().describe('Exact source_units IDs covered by this shot in the current section. A long dialogue may span several shots with the same source ID. Do not claim IDs from another section.'),
continuity_mode: z.enum(['none', 'context', 'first_frame']).optional().describe('none: independent/new scene; context: inherit state across a cut; first_frame: same-scene continuous action using previous video tail; automatic predecessor is previous chronological shot'), continuity_mode: z.enum(['none', 'context', 'first_frame']).optional().describe('none: independent/new scene; context: inherit state across a cut; first_frame: same-scene continuous action using previous video tail; automatic predecessor is previous chronological shot'),
continuity_start: z.string().max(1200).optional().describe('Opening state: positions, facing, costumes, injuries, held objects/hands, lighting, ongoing action; unknowns explicitly marked'), continuity_start: z.string().max(1200).optional().describe('Opening state: positions, facing, costumes, injuries, held objects/hands, lighting, ongoing action; unknowns explicitly marked'),
continuity_end: z.string().max(1200).optional().describe('Planned ending state for the next shot; not an observation of generated video'), continuity_end: z.string().max(1200).optional().describe('Planned ending state for the next shot; not an observation of generated video'),
@@ -280,7 +298,7 @@ const storyboardFields = z.object({
const saveStoryboards = createTool({ const saveStoryboards = createTool({
id: 'save_storyboards', id: 'save_storyboards',
description: 'Save storyboards for this episode. Call in batches of at most 8 storyboards: the first batch must set replace_existing: true (clears all old storyboards for the episode, then writes), every following batch omits replace_existing (appends). Rows are upserted by shot_number, so overlapping batches and retries never create duplicates.', description: 'Save storyboards for this episode. Call in batches of at most 4 storyboards: the first batch sets replace_existing: true, every following batch omits replace_existing and appends sequential shot_numbers. During a tracked job, batches are staged and only replace existing storyboards when the entire screenplay is complete. Validation errors must be corrected on the current batch before proceeding.',
inputSchema: z.object({ inputSchema: z.object({
replace_existing: z.boolean().optional(), replace_existing: z.boolean().optional(),
storyboards: z.array(storyboardFields), storyboards: z.array(storyboardFields),
@@ -289,6 +307,34 @@ const saveStoryboards = createTool({
const ids = requireIds(context) const ids = requireIds(context)
if ('error' in ids) return ids if ('error' in ids) return ids
const { episodeId, dramaId } = ids const { episodeId, dramaId } = ids
const jobId=storyboardJobId(context)
const committing=context?.requestContext?.get('storyboardCommit' as never)===true
if (jobId) {
const job=assertStoryboardJob(episodeId,jobId)
const requestedSection=context?.requestContext?.get('storyboardSectionIndex' as never)
if(!committing && requestedSection!==undefined && requestedSection!==job.sectionIndex) return {section_complete:true,message:'本轮场次已完成,请立即结束,由系统启动下一场。'}
if(!committing && job.sectionIndex===job.sectionCount) return {script_complete:true,message:'所有原文场次已完成,不能追加后续剧情。'}
storyboards=[...storyboards].sort((a,b)=>a.shot_number-b.shot_number)
if (!committing && storyboards.length>4) {
const error='每批最多保存 4 个分镜,请拆小批次以减少等待,保留全部剧情。'
rejectStoryboardBatch(episodeId,jobId,error);return {error}
}
updateStoryboardJob(episodeId,jobId,{phase:committing?'正在保存完整分镜':'正在校验中文分镜和英文提示词'})
}
try {
if(jobId&&!committing) {
storyboards=storyboards.map(fitDraftDialogueTiming)
const job=assertStoryboardJob(episodeId,jobId),section=job.sections?.[job.sectionIndex||0]
if(!section)throw new Error('当前场次已完成,请结束本轮')
const problems=sectionBatchProblems(section,storyboards)
const previous=job.drafts.filter(s=>s.source_section<(job.sectionIndex||0)).at(-1)?.shot_number||0
if(storyboards.some(s=>s.shot_number<=previous))problems.push('不能覆盖已完成场次的分镜,请从当前场次继续编号')
const numbers=[...new Set([...job.drafts.map(s=>s.shot_number),...storyboards.map(s=>s.shot_number)])].sort((a,b)=>a-b)
const gap=numbers.findIndex((n,i)=>n!==i+1)
if(gap>=0)problems.push(`分镜编号缺少 ${gap+1}。上批未保存的分镜不能跳过,请从 ${gap+1} 开始重新提交,source_ids 仍使用当前场次原文编号。`)
if(new Set(storyboards.map(s=>s.shot_number)).size!==storyboards.length)problems.push('本批 shot_number 不能重复')
if(problems.length)throw new Error(problems.join('\n'))
}
const ts = now() const ts = now()
logTaskProgress('StoryboardTool', 'save-begin', { logTaskProgress('StoryboardTool', 'save-begin', {
episodeId, episodeId,
@@ -297,7 +343,15 @@ const saveStoryboards = createTool({
count: storyboards.length, count: storyboards.length,
shotNumbers: storyboards.map(sb => sb.shot_number).join(','), shotNumbers: storyboards.map(sb => sb.shot_number).join(','),
}) })
// Check all voice references before replace_existing can remove the previous storyboards. // Missing voice uploads must not prevent writing new storyboard drafts.
// Normalize model-invented #markers on both languages, never actual video submissions.
const voiceCharacters = (await db.select().from(schema.characters).where(eq(schema.characters.dramaId, dramaId))).filter(c=>!c.deletedAt)
const assetNames=await promptAssetNames(dramaId)
storyboards = storyboards.map(sb=>({...sb,
video_prompt: normalizeDraftAssets(normalizeDraftVoices(sb.video_prompt, voiceCharacters, sb.character_ids || []),assetNames),
video_prompt_en: normalizeDraftAssets(normalizeDraftVoices(sb.video_prompt_en, voiceCharacters, sb.character_ids || []),assetNames),
}))
// Check all remaining voice references before any old storyboard is touched.
for (const sb of storyboards) { for (const sb of storyboards) {
await validateVideoPromptVoices(sb.video_prompt, dramaId, sb.character_ids) await validateVideoPromptVoices(sb.video_prompt, dramaId, sb.character_ids)
} }
@@ -308,73 +362,62 @@ const saveStoryboards = createTool({
storyboardSegmentProblems(sb, dramaCharacters).map(p => `段落 ${sb.shot_number}:${p}`)) storyboardSegmentProblems(sb, dramaCharacters).map(p => `段落 ${sb.shot_number}:${p}`))
if (segmentProblems.length) { if (segmentProblems.length) {
logTaskProgress('StoryboardTool', 'save-rejected', { episodeId, problems: segmentProblems.length }) logTaskProgress('StoryboardTool', 'save-rejected', { episodeId, problems: segmentProblems.length })
if (jobId) rejectStoryboardBatch(episodeId,jobId,segmentProblems.join('\n'))
return { error: `本批未保存,请逐条修正后用同样的 shot_number 重新调用 save_storyboards:\n${segmentProblems.join('\n')}` } return { error: `本批未保存,请逐条修正后用同样的 shot_number 重新调用 save_storyboards:\n${segmentProblems.join('\n')}` }
} }
const english = new Map<number, Awaited<ReturnType<typeof bilingualFields>>>() for (const sb of storyboards) await validateStoryboardBindings(episodeId,dramaId,sb.scene_id,sb.character_ids,sb.prop_ids,true)
for (const sb of storyboards) { const english = await bilingualBatchFields(storyboards, dramaId, {
if (sb.video_prompt?.trim()) english.set(sb.shot_number, await bilingualFields(sb.video_prompt, sb.video_prompt_en, dramaId, configId: context?.requestContext?.get('textConfigId' as never) as number | undefined,
{ configId: context?.requestContext?.get('textConfigId' as never) as number | undefined })) model: context?.requestContext?.get('modelOverride' as never) as string | undefined,
})
if (jobId) {
const job=assertStoryboardJob(episodeId,jobId)
if (!committing) return stageStoryboardBatch(episodeId,jobId,storyboards.map(sb=>({...sb,video_prompt_en:english.get(sb.shot_number)?.videoPromptEn||sb.video_prompt_en})),replace_existing===true)
if(job.sections?.some((section,index)=>!sectionCoverage(section,job.drafts.filter(s=>s.source_section===index)).ready))throw new Error('仍有场次原文、对白或时长未完整覆盖,未替换原分镜')
assertStoryboardSourceUnchanged(job)
} }
if (replace_existing === true) { // Validate the entire batch before touching existing rows; the final swap is atomic.
const existingStoryboardRows = await db.select().from(schema.storyboards) db.transaction(tx => {
.where(eq(schema.storyboards.episodeId, episodeId)) if (jobId) assertStoryboardSourceUnchanged(assertStoryboardJob(episodeId,jobId))
for (const storyboardId of existingStoryboardRows.map(sb => sb.id)) { if (replace_existing === true) {
await db.delete(schema.storyboardCharacters) const previous=tx.select().from(schema.storyboards).where(and(eq(schema.storyboards.episodeId,episodeId),isNull(schema.storyboards.deletedAt))).all()
.where(eq(schema.storyboardCharacters.storyboardId, storyboardId)) for (const row of previous) {
await db.delete(schema.storyboardProps) tx.delete(schema.storyboardCharacters).where(eq(schema.storyboardCharacters.storyboardId,row.id)).run()
.where(eq(schema.storyboardProps.storyboardId, storyboardId)) tx.delete(schema.storyboardProps).where(eq(schema.storyboardProps.storyboardId,row.id)).run()
}
tx.delete(schema.storyboards).where(and(eq(schema.storyboards.episodeId,episodeId),isNull(schema.storyboards.deletedAt))).run()
} }
await db.delete(schema.storyboards).where(eq(schema.storyboards.episodeId, episodeId)) const rows=tx.select().from(schema.storyboards).where(eq(schema.storyboards.episodeId,episodeId)).all()
} const shotToId=new Map(rows.filter(s=>!s.deletedAt).map(s=>[s.storyboardNumber,s.id]))
for (const sb of storyboards) {
// shot_number → id 索引(含本次调用内新增的行),保证按分镜号幂等 upsert if (sb.scene_id && !tx.select().from(schema.episodeScenes).where(and(eq(schema.episodeScenes.episodeId,episodeId),eq(schema.episodeScenes.sceneId,sb.scene_id))).get())
const existingRows = await db.select().from(schema.storyboards) tx.insert(schema.episodeScenes).values({episodeId,sceneId:sb.scene_id,createdAt:ts}).run()
.where(eq(schema.storyboards.episodeId, episodeId)) for (const characterId of new Set(sb.character_ids||[])) {
const shotToId = new Map<number, number>( if (!tx.select().from(schema.episodeCharacters).where(and(eq(schema.episodeCharacters.episodeId,episodeId),eq(schema.episodeCharacters.characterId,characterId))).get())
existingRows.filter(sb => !sb.deletedAt).map(sb => [sb.storyboardNumber, sb.id]), tx.insert(schema.episodeCharacters).values({episodeId,characterId,createdAt:ts}).run()
) }
for (const propId of new Set(sb.prop_ids||[])) {
for (const sb of storyboards) { if (!tx.select().from(schema.episodeProps).where(and(eq(schema.episodeProps.episodeId,episodeId),eq(schema.episodeProps.propId,propId))).get())
await validateStoryboardBindings(episodeId, dramaId, sb.scene_id, sb.character_ids, sb.prop_ids) tx.insert(schema.episodeProps).values({episodeId,propId,createdAt:ts}).run()
const existingId = shotToId.get(sb.shot_number) }
if (existingId !== undefined) { const fields={title:sb.title,shotType:sb.shot_type,angle:sb.angle,movement:sb.movement,
await db.update(schema.storyboards).set({ location:sb.location,time:sb.time,description:sb.description,result:sb.result,
title: sb.title, shotType: sb.shot_type, atmosphere:sb.atmosphere,imagePrompt:sb.image_prompt,videoPrompt:sb.video_prompt,
angle: sb.angle, movement: sb.movement, bgmPrompt:sb.bgm_prompt,...english.get(sb.shot_number),soundEffect:sb.sound_effect,
location: sb.location, time: sb.time, continuityMode:sb.continuity_mode,continuityStart:sb.continuity_start,continuityEnd:sb.continuity_end,
description: sb.description, result: sb.result, sceneId:sb.scene_id,duration:sb.duration||10,updatedAt:ts}
atmosphere: sb.atmosphere, imagePrompt: sb.image_prompt, let storyboardId=shotToId.get(sb.shot_number)
videoPrompt: sb.video_prompt, bgmPrompt: sb.bgm_prompt, if (storyboardId!==undefined) tx.update(schema.storyboards).set(fields).where(eq(schema.storyboards.id,storyboardId)).run()
...english.get(sb.shot_number), else {
soundEffect: sb.sound_effect, storyboardId=getInsertId(tx.insert(schema.storyboards).values({...fields,episodeId,storyboardNumber:sb.shot_number,createdAt:ts}).run())
continuityMode: sb.continuity_mode, continuityStart: sb.continuity_start, continuityEnd: sb.continuity_end, shotToId.set(sb.shot_number,storyboardId)
sceneId: sb.scene_id, duration: sb.duration || 10, }
updatedAt: ts, tx.delete(schema.storyboardCharacters).where(eq(schema.storyboardCharacters.storyboardId,storyboardId)).run()
}).where(eq(schema.storyboards.id, existingId)) tx.delete(schema.storyboardProps).where(eq(schema.storyboardProps.storyboardId,storyboardId)).run()
await syncStoryboardCharacters(existingId, sb.character_ids || []) for (const characterId of new Set(sb.character_ids||[])) tx.insert(schema.storyboardCharacters).values({storyboardId,characterId}).run()
await syncStoryboardProps(existingId, sb.prop_ids || []) for (const propId of new Set(sb.prop_ids||[])) tx.insert(schema.storyboardProps).values({storyboardId,propId}).run()
} else {
const res = await db.insert(schema.storyboards).values({
episodeId,
storyboardNumber: sb.shot_number,
title: sb.title, shotType: sb.shot_type,
angle: sb.angle, movement: sb.movement,
location: sb.location, time: sb.time,
description: sb.description, result: sb.result,
atmosphere: sb.atmosphere, imagePrompt: sb.image_prompt,
videoPrompt: sb.video_prompt, bgmPrompt: sb.bgm_prompt,
...english.get(sb.shot_number),
soundEffect: sb.sound_effect,
continuityMode: sb.continuity_mode, continuityStart: sb.continuity_start, continuityEnd: sb.continuity_end,
sceneId: sb.scene_id, duration: sb.duration || 10,
createdAt: ts, updatedAt: ts,
})
const newId = getInsertId(res)
shotToId.set(sb.shot_number, newId)
await syncStoryboardCharacters(newId, sb.character_ids || [])
await syncStoryboardProps(newId, sb.prop_ids || [])
} }
} })
// 整集时长 = 当前全部存活分镜时长之和(分批保存时不能再按单批累加) // 整集时长 = 当前全部存活分镜时长之和(分批保存时不能再按单批累加)
const allRows = await db.select().from(schema.storyboards) const allRows = await db.select().from(schema.storyboards)
@@ -399,6 +442,12 @@ const saveStoryboards = createTool({
? `已保存 ${totalDuration} 秒,目标约 ${target} 秒(剧本字数 ÷ 500 字/分钟)。剧本还没拆完,继续按顺序保存后续段落,直到剧本最后一句。` ? `已保存 ${totalDuration} 秒,目标约 ${target} 秒(剧本字数 ÷ 500 字/分钟)。剧本还没拆完,继续按顺序保存后续段落,直到剧本最后一句。`
: `已保存 ${totalDuration} 秒,目标约 ${target} 秒。` : `已保存 ${totalDuration} 秒,目标约 ${target} 秒。`
return { message: `Saved ${storyboards.length} storyboards. ${progress}`, count: storyboards.length, total_duration: totalDuration, target_total_seconds: target } return { message: `Saved ${storyboards.length} storyboards. ${progress}`, count: storyboards.length, total_duration: totalDuration, target_total_seconds: target }
} catch(error) {
if (!jobId || committing) throw error
const message=error instanceof Error?error.message:'分镜保存失败'
if (jobId && !committing && getStoryboardJob(episodeId)?.status==='running') rejectStoryboardBatch(episodeId,jobId,message)
return {error:message}
}
}, },
}) })
@@ -430,6 +479,11 @@ const updateStoryboard = createTool({
const ids = requireIds(context) const ids = requireIds(context)
if ('error' in ids) return ids if ('error' in ids) return ids
const { episodeId, dramaId } = ids const { episodeId, dramaId } = ids
const jobId = storyboardJobId(context)
if (jobId) {
assertStoryboardJob(episodeId, jobId)
return { error: '整集拆分期间只允许通过 save_storyboards 暂存新分镜,不能修改原分镜。' }
}
const [storyboard] = await db.select().from(schema.storyboards).where(eq(schema.storyboards.id, storyboard_id)) const [storyboard] = await db.select().from(schema.storyboards).where(eq(schema.storyboards.id, storyboard_id))
if (!storyboard || storyboard.episodeId !== episodeId || storyboard.deletedAt) return { error: `Storyboard ${storyboard_id} not found in this episode` } if (!storyboard || storyboard.episodeId !== episodeId || storyboard.deletedAt) return { error: `Storyboard ${storyboard_id} not found in this episode` }
@@ -487,7 +541,8 @@ const updateStoryboard = createTool({
if ('duration' in fields) updates.duration = fields.duration if ('duration' in fields) updates.duration = fields.duration
if ('video_prompt' in fields || 'video_prompt_en' in fields) Object.assign(updates, if ('video_prompt' in fields || 'video_prompt_en' in fields) Object.assign(updates,
await bilingualFields(fields.video_prompt ?? storyboard.videoPrompt ?? '', fields.video_prompt_en, dramaId, await bilingualFields(fields.video_prompt ?? storyboard.videoPrompt ?? '', fields.video_prompt_en, dramaId,
{ configId: context?.requestContext?.get('textConfigId' as never) as number | undefined })) { configId: context?.requestContext?.get('textConfigId' as never) as number | undefined,
model: context?.requestContext?.get('modelOverride' as never) as string | undefined }))
await db.update(schema.storyboards).set(updates).where(eq(schema.storyboards.id, storyboard_id)) await db.update(schema.storyboards).set(updates).where(eq(schema.storyboards.id, storyboard_id))
if ('character_ids' in fields) await syncStoryboardCharacters(storyboard_id, fields.character_ids || []) if ('character_ids' in fields) await syncStoryboardCharacters(storyboard_id, fields.character_ids || [])
if ('prop_ids' in fields) await syncStoryboardProps(storyboard_id, fields.prop_ids || []) if ('prop_ids' in fields) await syncStoryboardProps(storyboard_id, fields.prop_ids || [])
@@ -6,7 +6,7 @@ const instructions: Record<string, string> = {
- read_storyboard_context 的角色提供 has_voice 与 voice_reference;已有分镜提供 available_voice_references。只有 has_voice 为 true 的角色才有可用音色。 - read_storyboard_context 的角色提供 has_voice 与 voice_reference;已有分镜提供 available_voice_references。只有 has_voice 为 true 的角色才有可用音色。
- 为某个分镜生成或重写 video_prompt 时,只使用该分镜 available_voice_references 中的音色;首次拆分时,只使用本段 character_ids 对应角色的 voice_reference。 - 为某个分镜生成或重写 video_prompt 时,只使用该分镜 available_voice_references 中的音色;首次拆分时,只使用本段 character_ids 对应角色的 voice_reference。
- 先从 description 的台词确定说话角色。画面动作使用 @角色名;有台词且有可用音色时,在对应对白的说话人位置直接使用 #角色名,紧接语气、发声动作与台词。例如:0-3秒:@林小雨抬头,#林小雨轻声说:「早上好。」 - 先从 description 的台词确定说话角色。画面动作使用 @角色名;有台词且有可用音色时,在对应对白的说话人位置直接使用 #角色名,紧接语气、发声动作与台词。例如:0-3秒:@林小雨抬头,#林小雨轻声说:「早上好。」
- 音色引用必须与对白写在同一个时间段内,不新增独立的「音色:」信息头。每次对白都明确标出自己的 #说话人;多人对话逐句对应,不能把所有音色集中列在开头。 - 有可用音色时,音色引用与对白写在同一个时间段内,不新增独立的「音色:」信息头;没有可用音色时直接写 @角色名说:「原文台词」,无需上传音色也能拆分镜。多人对话逐句对应。
- 直接复制 voice_reference 的完整值(如 #林小雨),禁止改名、缩写、翻译或追加“音色”后缀。@角色名引用形象,#角色名引用音色,两者分别放在画面动作与发声位置。 - 直接复制 voice_reference 的完整值(如 #林小雨),禁止改名、缩写、翻译或追加“音色”后缀。@角色名引用形象,#角色名引用音色,两者分别放在画面动作与发声位置。
- 没有上传音色的角色、未绑定角色、已移除音色和不说话的角色,不添加 # 引用。旁白只有明确对应某个已绑定、已有音色的角色时才可引用,不借用其他角色的音色。 - 没有上传音色的角色、未绑定角色、已移除音色和不说话的角色,不添加 # 引用。旁白只有明确对应某个已绑定、已有音色的角色时才可引用,不借用其他角色的音色。
- 音频仅用于参考音色,台词内容与说话时机遵循 description;不要为了使用音色新增台词、人物或镜头。 - 音频仅用于参考音色,台词内容与说话时机遵循 description;不要为了使用音色新增台词、人物或镜头。
+1
View File
@@ -99,6 +99,7 @@ export const scenes = sqliteTable('scenes', {
time: text('time').notNull(), time: text('time').notNull(),
prompt: text('prompt').notNull(), prompt: text('prompt').notNull(),
lighting: text('lighting'), lighting: text('lighting'),
continuityPrompt: text('continuity_prompt'),
finalPrompt: text('final_prompt'), finalPrompt: text('final_prompt'),
storyboardCount: integer('storyboard_count').default(1), storyboardCount: integer('storyboard_count').default(1),
imageUrl: text('image_url'), imageUrl: text('image_url'),
@@ -403,6 +403,7 @@ export function initSqliteSchema(sqlite: Database.Database) {
// Reference inputs are independent of generated outputs; preserve all existing rows. // Reference inputs are independent of generated outputs; preserve all existing rows.
const sceneColumns = new Set((sqlite.prepare('PRAGMA table_info(scenes)').all() as { name: string }[]).map(c => c.name)) const sceneColumns = new Set((sqlite.prepare('PRAGMA table_info(scenes)').all() as { name: string }[]).map(c => c.name))
if (!sceneColumns.has('reference_images')) sqlite.exec('ALTER TABLE scenes ADD COLUMN reference_images TEXT') if (!sceneColumns.has('reference_images')) sqlite.exec('ALTER TABLE scenes ADD COLUMN reference_images TEXT')
if (!sceneColumns.has('continuity_prompt')) sqlite.exec('ALTER TABLE scenes ADD COLUMN continuity_prompt TEXT')
const upgradeSeed = sqlite.prepare(UPGRADE_SQL) const upgradeSeed = sqlite.prepare(UPGRADE_SQL)
const removeSeed = sqlite.prepare(REMOVE_SQL) const removeSeed = sqlite.prepare(REMOVE_SQL)
for (const s of stylePresetSeeds) { for (const s of stylePresetSeeds) {
+2
View File
@@ -26,6 +26,7 @@ import storage from './routes/storage.js'
import serverUpdate from './routes/serverUpdate.js' import serverUpdate from './routes/serverUpdate.js'
import { requestLogger, errorHandler } from './middleware/logger.js' import { requestLogger, errorHandler } from './middleware/logger.js'
import { resumeInterruptedTasks } from './services/generation.js' import { resumeInterruptedTasks } from './services/generation.js'
import { resumeVideoCleanups } from './services/video-cleanup.js'
import { DATA_ROOT } from './utils/paths.js' import { DATA_ROOT } from './utils/paths.js'
const __dirname = path.dirname(fileURLToPath(import.meta.url)) const __dirname = path.dirname(fileURLToPath(import.meta.url))
@@ -89,5 +90,6 @@ const port = Number(process.env.PORT || 5679)
console.log(`🚀 Huobao Drama TS server on http://localhost:${port}`) console.log(`🚀 Huobao Drama TS server on http://localhost:${port}`)
await resumeInterruptedTasks() await resumeInterruptedTasks()
resumeVideoCleanups()
serve({ fetch: app.fetch, port }) serve({ fetch: app.fetch, port })
+14
View File
@@ -7,9 +7,23 @@ import { buildAgentRequestContext } from '../agents/context.js'
import { mastra } from '../mastra/index.js' import { mastra } from '../mastra/index.js'
import { success, badRequest } from '../utils/response.js' import { success, badRequest } from '../utils/response.js'
import { logTaskError, logTaskPayload, logTaskProgress, logTaskStart, logTaskSuccess } from '../utils/task-logger.js' import { logTaskError, logTaskPayload, logTaskProgress, logTaskStart, logTaskSuccess } from '../utils/task-logger.js'
import { startStoryboardJob, cancelStoryboardJob } from '../services/storyboard-jobs.js'
import { getStoryboardJob, publicStoryboardJob } from '../services/storyboard-job-state.js'
const app = new Hono() const app = new Hono()
app.post('/storyboard_breaker/jobs',async c=>{
const body=await c.req.json()
if (!Number.isInteger(body.episode_id)||!Number.isInteger(body.drama_id)||typeof body.message!=='string'||!body.message.trim()) return badRequest(c,'需要有效的项目、集数和拆分要求')
if(body.resume_from!==undefined&&(typeof body.resume_from!=='string'||!body.resume_from))return badRequest(c,'需要有效的暂存任务编号')
try { return success(c,startStoryboardJob(body)) } catch(error:any) { return badRequest(c,error.message) }
})
app.get('/storyboard_breaker/jobs/:episodeId',c=>success(c,publicStoryboardJob(getStoryboardJob(Number(c.req.param('episodeId'))))))
app.post('/storyboard_breaker/jobs/:episodeId/cancel',async c=>{
const body=await c.req.json()
try { return success(c,cancelStoryboardJob(Number(c.req.param('episodeId')),body.id)) } catch(error:any) { return badRequest(c,error.message) }
})
// Mastra v1.17 的 ToolCallChunk / ToolResultChunk 结构: // Mastra v1.17 的 ToolCallChunk / ToolResultChunk 结构:
// { type: 'tool-call', payload: { toolCallId, toolName, args } } // { type: 'tool-call', payload: { toolCallId, toolName, args } }
// { type: 'tool-result', payload: { toolCallId, toolName, result, isError } } // { type: 'tool-result', payload: { toolCallId, toolName, result, isError } }
+6 -3
View File
@@ -179,7 +179,7 @@ app.get('/:id/video-prompts-status', async (c) => {
app.get('/:episode_id/storyboards', async (c) => { app.get('/:episode_id/storyboards', async (c) => {
const episodeId = Number(c.req.param('episode_id')) const episodeId = Number(c.req.param('episode_id'))
const rows = await db.select().from(schema.storyboards) const rows = await db.select().from(schema.storyboards)
.where(eq(schema.storyboards.episodeId, episodeId)) .where(and(eq(schema.storyboards.episodeId, episodeId), isNull(schema.storyboards.deletedAt)))
.orderBy(schema.storyboards.storyboardNumber) .orderBy(schema.storyboards.storyboardNumber)
const links = await db.select().from(schema.storyboardCharacters) const links = await db.select().from(schema.storyboardCharacters)
@@ -262,7 +262,10 @@ app.get('/:id/generation-tasks', async (c) => {
.slice(0, 20) .slice(0, 20)
return success(c, { return success(c, {
tasks: toSnakeCaseArray(tasks.map(task => ({ ...task, routing: taskRouting(task), canCancel: canCancelGenerationTask(task) }))), tasks: toSnakeCaseArray(tasks.map(task => ({ ...task,
storyboardNumber: sbs.find(sb => sb.id === task.storyboardId)?.storyboardNumber,
storyboardDeleted: !!sbs.find(sb => sb.id === task.storyboardId)?.deletedAt,
routing: taskRouting(task), canCancel: canCancelGenerationTask(task) }))),
merges: toSnakeCaseArray(merges), merges: toSnakeCaseArray(merges),
}) })
}) })
@@ -274,7 +277,7 @@ app.get('/:id/pipeline-status', async (c) => {
const chars = await db.select().from(schema.characters).where(eq(schema.characters.dramaId, ep.dramaId)) const chars = await db.select().from(schema.characters).where(eq(schema.characters.dramaId, ep.dramaId))
const scenes = await db.select().from(schema.scenes).where(eq(schema.scenes.dramaId, ep.dramaId)) const scenes = await db.select().from(schema.scenes).where(eq(schema.scenes.dramaId, ep.dramaId))
const sbs = await db.select().from(schema.storyboards).where(eq(schema.storyboards.episodeId, episodeId)) const sbs = await db.select().from(schema.storyboards).where(and(eq(schema.storyboards.episodeId, episodeId), isNull(schema.storyboards.deletedAt)))
const merges = await db.select().from(schema.videoMerges).where(eq(schema.videoMerges.episodeId, episodeId)) const merges = await db.select().from(schema.videoMerges).where(eq(schema.videoMerges.episodeId, episodeId))
const sbsWithImage = sbs.filter(s => s.composedImage) const sbsWithImage = sbs.filter(s => s.composedImage)
+3 -3
View File
@@ -1,5 +1,5 @@
import { Hono } from 'hono' import { Hono } from 'hono'
import { and, asc, eq } from 'drizzle-orm' import { and, asc, eq, isNull } from 'drizzle-orm'
import fs from 'node:fs' import fs from 'node:fs'
import path from 'node:path' import path from 'node:path'
import { randomUUID } from 'node:crypto' import { randomUUID } from 'node:crypto'
@@ -31,7 +31,7 @@ async function versions(id: number) {
} }
async function context(id: number) { async function context(id: number) {
const [sb] = await db.select().from(schema.storyboards).where(eq(schema.storyboards.id, id)) const [sb] = await db.select().from(schema.storyboards).where(eq(schema.storyboards.id, id))
if (!sb) throw new Error('分镜不存在') if (!sb || sb.deletedAt) throw new Error('分镜不存在或已删除')
const tasks = await versions(id) const tasks = await versions(id)
const original = tasks.map(t => params(t.params).originalVideoUrl).find(Boolean) || sb.videoUrl const original = tasks.map(t => params(t.params).originalVideoUrl).find(Boolean) || sb.videoUrl
return { sb, tasks, original } return { sb, tasks, original }
@@ -40,7 +40,7 @@ async function context(id: number) {
app.get('/episodes/:id', async c => { app.get('/episodes/:id', async c => {
const id = Number(c.req.param('id')) const id = Number(c.req.param('id'))
if (!Number.isSafeInteger(id) || id <= 0) return badRequest(c, '集 ID 无效') if (!Number.isSafeInteger(id) || id <= 0) return badRequest(c, '集 ID 无效')
const boards = await db.select().from(schema.storyboards).where(eq(schema.storyboards.episodeId, id)) const boards = await db.select().from(schema.storyboards).where(and(eq(schema.storyboards.episodeId, id), isNull(schema.storyboards.deletedAt)))
const configs = (await db.select().from(schema.aiServiceConfigs)).filter(row => row.isActive && row.serviceType === 'video' && row.provider === 'localvideo' && row.model?.includes(MODEL)).sort((a, b) => (b.priority || 0) - (a.priority || 0)) const configs = (await db.select().from(schema.aiServiceConfigs)).filter(row => row.isActive && row.serviceType === 'video' && row.provider === 'localvideo' && row.model?.includes(MODEL)).sort((a, b) => (b.priority || 0) - (a.priority || 0))
return success(c, { return success(c, {
configured: configs.length > 0, configured: configs.length > 0,
@@ -17,6 +17,7 @@ app.post('/', async (c) => {
const body = await c.req.json() const body = await c.req.json()
if (!body.drama_id) return badRequest(c, 'drama_id 必填') if (!body.drama_id) return badRequest(c, 'drama_id 必填')
if (!body.location?.trim()) return badRequest(c, '地点必填') if (!body.location?.trim()) return badRequest(c, '地点必填')
if (body.continuity_prompt !== undefined && (typeof body.continuity_prompt !== 'string' || body.continuity_prompt.length > 4000)) return badRequest(c, '场景固定设定须为 4000 字以内的文本')
const ts = now() const ts = now()
const res = await db.insert(schema.scenes).values({ const res = await db.insert(schema.scenes).values({
dramaId: body.drama_id, dramaId: body.drama_id,
@@ -25,6 +26,7 @@ app.post('/', async (c) => {
time: body.time || '', time: body.time || '',
prompt: body.prompt || body.description || body.location, prompt: body.prompt || body.description || body.location,
lighting: body.lighting || '', lighting: body.lighting || '',
continuityPrompt: body.continuity_prompt?.trim() || null,
createdAt: ts, createdAt: ts,
updatedAt: ts, updatedAt: ts,
}) })
@@ -51,6 +53,10 @@ app.put('/:id', async (c) => {
if (body.prompt !== undefined) updates.prompt = body.prompt if (body.prompt !== undefined) updates.prompt = body.prompt
if (body.description !== undefined) updates.prompt = body.description if (body.description !== undefined) updates.prompt = body.description
if (body.lighting !== undefined) updates.lighting = body.lighting if (body.lighting !== undefined) updates.lighting = body.lighting
if (body.continuity_prompt !== undefined) {
if (typeof body.continuity_prompt !== 'string' || body.continuity_prompt.length > 4000) return badRequest(c, '场景固定设定须为 4000 字以内的文本')
updates.continuityPrompt = body.continuity_prompt.trim() || null
}
// 用户上传场景图:直接写入图片地址与本地路径 // 用户上传场景图:直接写入图片地址与本地路径
const uploadedImage = body.image_url ?? body.imageUrl const uploadedImage = body.image_url ?? body.imageUrl
if (uploadedImage !== undefined) { if (uploadedImage !== undefined) {
@@ -1,5 +1,5 @@
import { Hono } from 'hono' import { Hono } from 'hono'
import { eq } from 'drizzle-orm' import { and, desc, eq, isNull } from 'drizzle-orm'
import { db, getInsertId, schema } from '../db/index.js' import { db, getInsertId, schema } from '../db/index.js'
import { success, created, now, badRequest } from '../utils/response.js' import { success, created, now, badRequest } from '../utils/response.js'
import { toSnakeCase } from '../utils/transform.js' import { toSnakeCase } from '../utils/transform.js'
@@ -7,8 +7,26 @@ import { logTaskPayload, logTaskStart, logTaskSuccess } from '../utils/task-logg
import { continuityDetails, continuityUpdate, extractContinuityFrame, resolveContinuity, sourceReadiness } from '../services/continuity.js' import { continuityDetails, continuityUpdate, extractContinuityFrame, resolveContinuity, sourceReadiness } from '../services/continuity.js'
import { ensureStoryboardEnglish, englishSourceHash, promptAssetNames, validateEnglishPrompt, withEnglishStatus } from '../services/english-video-prompt.js' import { ensureStoryboardEnglish, englishSourceHash, promptAssetNames, validateEnglishPrompt, withEnglishStatus } from '../services/english-video-prompt.js'
import { resolveSceneLock } from '../services/scene-continuity.js'
import { deleteStoryboards } from '../services/storyboard-delete.js'
const app = new Hono() const app = new Hono()
app.post('/batch-delete', async c => {
try {
const body = await c.req.json()
return success(c, deleteStoryboards(body.episode_id, body.storyboard_ids))
} catch (error: any) { return badRequest(c, error.message) }
})
app.get('/:id/environment', async c => {
try {
const id = Number(c.req.param('id'))
const current = await resolveSceneLock(id)
const latest = await db.select().from(schema.sysTask).where(and(eq(schema.sysTask.storyboardId, id), eq(schema.sysTask.type, 'video'))).orderBy(desc(schema.sysTask.id)).get()
let used: any = null
try { used = JSON.parse(latest?.params || '{}').sceneLock || null } catch { /* legacy task */ }
return success(c, { current, used, taskId: latest?.id, stale: !!latest && used?.hash !== current.hash })
} catch (error: any) { return badRequest(c, error.message) }
})
app.post('/:id/english-prompt', async c => { app.post('/:id/english-prompt', async c => {
try { try {
const body = await c.req.json().catch(() => ({})) const body = await c.req.json().catch(() => ({}))
@@ -151,7 +169,7 @@ app.put('/:id', async (c) => {
const id = Number(c.req.param('id')) const id = Number(c.req.param('id'))
const body = await c.req.json() const body = await c.req.json()
const [storyboard] = await db.select().from(schema.storyboards).where(eq(schema.storyboards.id, id)) const [storyboard] = await db.select().from(schema.storyboards).where(eq(schema.storyboards.id, id))
if (!storyboard) return badRequest(c, '镜头不存在') if (!storyboard || storyboard.deletedAt) return badRequest(c, '镜头不存在或已删除')
logTaskStart('StoryboardAPI', 'update', { logTaskStart('StoryboardAPI', 'update', {
storyboardId: id, storyboardId: id,
episodeId: storyboard.episodeId, episodeId: storyboard.episodeId,
@@ -190,7 +208,7 @@ app.put('/:id', async (c) => {
'prop_ids' in body ? body.prop_ids : await getStoryboardPropIds(id), 'prop_ids' in body ? body.prop_ids : await getStoryboardPropIds(id),
) )
await db.update(schema.storyboards).set(updates).where(eq(schema.storyboards.id, id)) await db.update(schema.storyboards).set(updates).where(and(eq(schema.storyboards.id, id), isNull(schema.storyboards.deletedAt)))
if ('character_ids' in body) await syncStoryboardCharacters(id, body.character_ids || []) if ('character_ids' in body) await syncStoryboardCharacters(id, body.character_ids || [])
if ('prop_ids' in body) await syncStoryboardProps(id, body.prop_ids || []) if ('prop_ids' in body) await syncStoryboardProps(id, body.prop_ids || [])
logTaskSuccess('StoryboardAPI', 'update', { logTaskSuccess('StoryboardAPI', 'update', {
@@ -205,12 +223,13 @@ app.put('/:id', async (c) => {
// DELETE /storyboards/:id // DELETE /storyboards/:id
app.delete('/:id', async (c) => { app.delete('/:id', async (c) => {
const id = Number(c.req.param('id')) const id = Number(c.req.param('id'))
logTaskStart('StoryboardAPI', 'delete', { storyboardId: id }) try {
await db.delete(schema.storyboardCharacters).where(eq(schema.storyboardCharacters.storyboardId, id)) const shot = db.select().from(schema.storyboards).where(eq(schema.storyboards.id, id)).get()
await db.delete(schema.storyboardProps).where(eq(schema.storyboardProps.storyboardId, id)) if (!shot) return badRequest(c, '分镜不存在')
await db.delete(schema.storyboards).where(eq(schema.storyboards.id, id)) const result = deleteStoryboards(shot.episodeId, [id])
logTaskSuccess('StoryboardAPI', 'delete', { storyboardId: id }) logTaskSuccess('StoryboardAPI', 'delete', { storyboardId: id })
return success(c) return success(c, result)
} catch (error: any) { return badRequest(c, error.message) }
}) })
export default app export default app
+8 -1
View File
@@ -6,6 +6,7 @@ import { generateImage, generateVideo, recoverGenerationTask, cancelWaitingVideo
import { safeTaskError } from '../services/video-download.js' import { safeTaskError } from '../services/video-download.js'
import { taskRouting } from '../services/worker-routing.js' import { taskRouting } from '../services/worker-routing.js'
import { getActiveConfig, getConfigById } from '../services/ai.js' import { getActiveConfig, getConfigById } from '../services/ai.js'
import { needsComposedFirstFrame } from '../services/first-frame-composition.js'
import { getDramaStylePrompt } from '../services/style-preset.js' import { getDramaStylePrompt } from '../services/style-preset.js'
import { resolveStoryboardVoiceReferences } from '../services/character-voice-references.js' import { resolveStoryboardVoiceReferences } from '../services/character-voice-references.js'
import { logTaskError, logTaskPayload, logTaskStart, logTaskSuccess } from '../utils/task-logger.js' import { logTaskError, logTaskPayload, logTaskStart, logTaskSuccess } from '../utils/task-logger.js'
@@ -124,7 +125,8 @@ function validateVideoRequest(body: any, provider?: string): string | null {
const total = imgs + vids + auds + Number(first) + Number(last) + Number(file) + Number(link) const total = imgs + vids + auds + Number(first) + Number(last) + Number(file) + Number(link)
if (total > 20) return 'Wan 3.0 input.media 最多 20 项' if (total > 20) return 'Wan 3.0 input.media 最多 20 项'
} else { } else {
if (imgs > 9 || vids > 3 || auds > 3) return '参考素材超限:图片≤9、视频≤3、音频≤3' const composing = body.storyboard_id && needsComposedFirstFrame(body.model) && !first && !body.image_url
if ((!composing && imgs > 9) || vids > 3 || auds > 3) return '参考素材超限:图片≤9、视频≤3、音频≤3'
if (auds > 0 && imgs + vids === 0) return '参考音频需要至少 1 个参考图片或视频' if (auds > 0 && imgs + vids === 0) return '参考音频需要至少 1 个参考图片或视频'
} }
@@ -163,6 +165,7 @@ app.post('/', async (c) => {
let storyboardDramaId: number | undefined let storyboardDramaId: number | undefined
if (body.storyboard_id) { if (body.storyboard_id) {
const [sb] = await db.select().from(schema.storyboards).where(eq(schema.storyboards.id, Number(body.storyboard_id))) const [sb] = await db.select().from(schema.storyboards).where(eq(schema.storyboards.id, Number(body.storyboard_id)))
if (!sb || sb.deletedAt) return badRequest(c, '分镜不存在或已删除')
if (sb) { if (sb) {
const [ep] = await db.select().from(schema.episodes).where(eq(schema.episodes.id, sb.episodeId)) const [ep] = await db.select().from(schema.episodes).where(eq(schema.episodes.id, sb.episodeId))
const locked = type === 'image' ? ep?.imageConfigId : ep?.videoConfigId const locked = type === 'image' ? ep?.imageConfigId : ep?.videoConfigId
@@ -176,6 +179,7 @@ app.post('/', async (c) => {
const effectiveConfig = configId const effectiveConfig = configId
? (await getConfigById(configId)) ?? await getActiveConfig('video') ? (await getConfigById(configId)) ?? await getActiveConfig('video')
: await getActiveConfig('video') : await getActiveConfig('video')
videoBody.model ||= effectiveConfig?.model
const validationError = validateVideoRequest(videoBody, effectiveConfig?.provider) const validationError = validateVideoRequest(videoBody, effectiveConfig?.provider)
if (validationError) return badRequest(c, validationError) if (validationError) return badRequest(c, validationError)
const voices = await resolveStoryboardVoiceReferences(String(videoBody.prompt || ''), Number(body.storyboard_id) || undefined, videoBody.reference_audio_urls) const voices = await resolveStoryboardVoiceReferences(String(videoBody.prompt || ''), Number(body.storyboard_id) || undefined, videoBody.reference_audio_urls)
@@ -217,6 +221,9 @@ app.post('/', async (c) => {
configId, configId,
}) })
: await generateVideo({ : await generateVideo({
frameImageConfigId: body.frame_image_config_id,
frameImageModel: body.frame_image_model,
regenerateFirstFrame: body.regenerate_first_frame === true,
gpu: body.gpu, gpu: body.gpu,
replaceTaskId: body.replace_task_id, replaceTaskId: body.replace_task_id,
forceNew: body.force_new === true, forceNew: body.force_new === true,
@@ -12,6 +12,14 @@ export class LocalVideoAdapter extends MiniMaxVideoAdapter {
if (record.referenceFileUrl || record.referenceLinkUrl) throw new Error('本地视频暂不支持 file / link 参考素材') if (record.referenceFileUrl || record.referenceLinkUrl) throw new Error('本地视频暂不支持 file / link 参考素材')
const request = super.buildGenerateRequest(config, { ...record, model: 'MiniMax-H3' }) const request = super.buildGenerateRequest(config, { ...record, model: 'MiniMax-H3' })
const body = request.body as Record<string, unknown> const body = request.body as Record<string, unknown>
const content = body.content as Array<{ type: string; role?: string }>
const images = content.filter(item => item.type === 'image_url')
const singleImage = ['LTX-2.3', 'LTX-2.3-Text-Preserve', 'Wan2.2-I2V'].includes(model)
const limit = singleImage ? 1 : 9
if (images.length > limit) throw new Error(`${model} 最多支持 ${limit} 张参考图,当前 ${images.length} 张;请选择单张参考图或改用 MiniMax-H3`)
if (['Wan2.2-I2V', 'MiniMax-H3-R2V-4Step', 'MiniMax-H3-VDN-H3'].includes(model) && !images.length) throw new Error(`${model} 需要至少 1 张参考图`)
if (content.some(item => !['text', 'image_url'].includes(item.type) || item.role === 'last_frame')) throw new Error('本地视频工作流暂不支持参考视频、音色或尾帧,请移除后提交')
if (images.some(item => item.role === 'first_frame') && images.length !== 1) throw new Error('首帧模式只能提交 1 张图片,不能混用多张参考图')
const resolution = String(record.resolution || '768P').toUpperCase() const resolution = String(record.resolution || '768P').toUpperCase()
if (!['480P', '720P', '768P', '1080P', '2K', '4K'].includes(resolution)) throw new Error('不支持的本地视频分辨率') if (!['480P', '720P', '768P', '1080P', '2K', '4K'].includes(resolution)) throw new Error('不支持的本地视频分辨率')
if (resolution === '4K' && model !== 'MiniMax-H3-VDN-H3') throw new Error('4K 需要 VDN-H3 高清制作模型') if (resolution === '4K' && model !== 'MiniMax-H3-VDN-H3') throw new Error('4K 需要 VDN-H3 高清制作模型')
@@ -84,8 +84,8 @@ export function validateContinuityModel(plan: ContinuityPlan, config: AIConfig,
let local = false let local = false
try { const u = new URL(config.baseUrl); local = u.hostname === '192.168.110.111' && ['18792', '18794'].includes(u.port) } catch {} try { const u = new URL(config.baseUrl); local = u.hostname === '192.168.110.111' && ['18792', '18794'].includes(u.port) } catch {}
// These local workflows implement genuine image-to-video; VDN/R2V-only workflows do not. // These local workflows implement genuine image-to-video; VDN/R2V-only workflows do not.
if (!local || !['MiniMax-H3', 'MiniMax-H3-Hybrid', 'Wan2.2-I2V'].includes(model || config.model || '')) { if (!local || !['MiniMax-H3', 'MiniMax-H3-Hybrid', 'Wan2.2-I2V', 'LTX-2.3', 'LTX-2.3-Text-Preserve'].includes(model || config.model || '')) {
throw new Error('尾帧接续请选择本地 MiniMax-H3、MiniMax-H3-Hybrid 或 Wan2.2-I2V;当前模型可使用状态承接') throw new Error('尾帧接续请选择本地 MiniMax-H3、MiniMax-H3-Hybrid、LTX-2.3 或 Wan2.2-I2V;当前模型可使用状态承接')
} }
} }
@@ -4,6 +4,7 @@ import { and, eq } from 'drizzle-orm'
import { db, schema } from '../db/index.js' import { db, schema } from '../db/index.js'
import { getConfigById, getTextConfig, getTextProviderBaseUrl } from './ai.js' import { getConfigById, getTextConfig, getTextProviderBaseUrl } from './ai.js'
import { now } from '../utils/response.js' import { now } from '../utils/response.js'
import { isSpeechPrefix } from '../utils/spoken-literals.js'
type Shot = typeof schema.storyboards.$inferSelect type Shot = typeof schema.storyboards.$inferSelect
type TextOptions = { configId?: number; model?: string } type TextOptions = { configId?: number; model?: string }
@@ -49,7 +50,7 @@ export function validateEnglishPrompt(source: string, candidate: string, names:
// Broadcasts/off-screen speech also need an explicit cue understood by H3's // Broadcasts/off-screen speech also need an explicit cue understood by H3's
// audio parser. Add only a label to speech already present in the source. // audio parser. Add only a label to speech already present in the source.
for (const match of source.matchAll(/([^。\n!?]{0,90})[::]\s*(「[^」]*」|“[^”]*”|"[^"\n]*")/g)) { for (const match of source.matchAll(/([^。\n!?]{0,90})[::]\s*(「[^」]*」|“[^”]*”|"[^"\n]*")/g)) {
if (!/(说|旁白|台词|广播|响起|喊|问|答|低语|低声|画外|念|嘀咕|声音|语音|系统音|提示音|播报|开口|心声|says|dialogue|voice|narrat)/i.test(match[1])) continue if (!isSpeechPrefix(match[1]+':')) continue
const literal = match[2], before = candidate const literal = match[2], before = candidate
candidate = candidate.replaceAll(literal, (value, offset) => /(?:says|Dialogue):\s*$/i.test(before.slice(0,offset)) ? value : 'Dialogue: ' + value) candidate = candidate.replaceAll(literal, (value, offset) => /(?:says|Dialogue):\s*$/i.test(before.slice(0,offset)) ? value : 'Dialogue: ' + value)
} }
@@ -74,6 +75,9 @@ export async function translateVisualPrompt(source: string, names: string[] = []
} else { } else {
url = `${base}/chat/completions`; headers = { Authorization: `Bearer ${config.apiKey}`, 'Content-Type': 'application/json' } url = `${base}/chat/completions`; headers = { Authorization: `Bearer ${config.apiKey}`, 'Content-Type': 'application/json' }
body = { model: opts.model || config.model, temperature: 0.1, max_tokens: 5000, messages: [{ role: 'system', content: instruction }, { role: 'user', content: protectedPrompt.text }] } body = { model: opts.model || config.model, temperature: 0.1, max_tokens: 5000, messages: [{ role: 'system', content: instruction }, { role: 'user', content: protectedPrompt.text }] }
// This direct fetch bypasses the Agent's thinking-off wrapper. Doubao
// otherwise defaults to deep reasoning for each individual translation.
if (config.provider === 'volcengine' && /^doubao-seed-/.test(opts.model || config.model)) body.thinking = { type: 'disabled' }
} }
const response = await fetch(url, { method: 'POST', headers, body: JSON.stringify(body), signal: AbortSignal.timeout(90_000) }) const response = await fetch(url, { method: 'POST', headers, body: JSON.stringify(body), signal: AbortSignal.timeout(90_000) })
if (!response.ok) throw new Error(`英文提示词生成失败(文本服务 HTTP ${response.status}),未提交视频,请稍后重试`) if (!response.ok) throw new Error(`英文提示词生成失败(文本服务 HTTP ${response.status}),未提交视频,请稍后重试`)
@@ -103,6 +107,24 @@ export async function bilingualFields(source: string, candidate?: string, dramaI
return { videoPromptEn: text, videoPromptEnSourceHash: englishSourceHash(source) } return { videoPromptEn: text, videoPromptEnSourceHash: englishSourceHash(source) }
} }
/** Validate supplied EN immediately; at most three independent fallback translations
* run at a time. All finish before the caller replaces or writes any storyboards. */
export async function bilingualBatchFields(items: { shot_number: number; video_prompt?: string; video_prompt_en?: string }[], dramaId: number, opts: TextOptions = {}) {
const results: Array<[number, Awaited<ReturnType<typeof bilingualFields>>]> = []
let next = 0, failure: unknown
const worker = async () => {
while (failure === undefined && next < items.length) {
const index = next++, item = items[index]
if (!item.video_prompt?.trim()) continue
try { results[index] = [item.shot_number, await bilingualFields(item.video_prompt, item.video_prompt_en, dramaId, opts)] }
catch (error) { failure = error }
}
}
await Promise.all(Array.from({ length: Math.min(3, items.length) }, worker))
if (failure !== undefined) throw failure
return new Map(results.filter(Boolean))
}
const englishJobs = new Map<number, Promise<Shot>>() const englishJobs = new Map<number, Promise<Shot>>()
export async function ensureStoryboardEnglish(id: number, force = false, opts: TextOptions = {}): Promise<Shot> { export async function ensureStoryboardEnglish(id: number, force = false, opts: TextOptions = {}): Promise<Shot> {
if (englishJobs.has(id)) return englishJobs.get(id)! if (englishJobs.has(id)) return englishJobs.get(id)!
@@ -130,11 +152,12 @@ export async function englishSubmissionPrompt(input: string, storyboardId?: numb
const ep = await db.select().from(schema.episodes).where(eq(schema.episodes.id,shot.episodeId)).get() const ep = await db.select().from(schema.episodes).where(eq(schema.episodes.id,shot.episodeId)).get()
dramaId = ep?.dramaId dramaId = ep?.dramaId
const original = shot.videoPrompt?.trim() const original = shot.videoPrompt?.trim()
const normalized = input.replace(/@图片\d+/g,'@') const names = await promptAssetNames(dramaId)
// Non-greedy index matching preserves numbered place names such as 404宿舍.
const normalized = names.length ? input.replace(new RegExp(`@图片\\d+?(?=(?:${names.map(escape).join('|')}))`, 'g'), '@') : input
if (original && normalized.includes(original)) { if (original && normalized.includes(original)) {
const english = await ensureStoryboardEnglish(shot.id) const english = await ensureStoryboardEnglish(shot.id)
prompt = normalized.replace(original,english.videoPromptEn!) prompt = normalized.replace(original,english.videoPromptEn!)
const names = await promptAssetNames(dramaId)
for (const name of names) { for (const name of names) {
const match = input.match(new RegExp('@图片(\\d+)' + escape(name))) const match = input.match(new RegExp('@图片(\\d+)' + escape(name)))
if (match) prompt = prompt.replace(new RegExp('@' + escape(name), 'g'), `@图片${match[1]}${name}`) if (match) prompt = prompt.replace(new RegExp('@' + escape(name), 'g'), `@图片${match[1]}${name}`)
@@ -5,7 +5,7 @@ import fs from 'fs'
import path from 'path' import path from 'path'
import { v4 as uuid } from 'uuid' import { v4 as uuid } from 'uuid'
import { db, getInsertId, schema } from '../db/index.js' import { db, getInsertId, schema } from '../db/index.js'
import { eq } from 'drizzle-orm' import { and, eq, isNull } from 'drizzle-orm'
import { now } from '../utils/response.js' import { now } from '../utils/response.js'
import { logTaskError, logTaskStart, logTaskSuccess } from '../utils/task-logger.js' import { logTaskError, logTaskStart, logTaskSuccess } from '../utils/task-logger.js'
import { extractVideoPoster } from '../utils/video-poster.js' import { extractVideoPoster } from '../utils/video-poster.js'
@@ -25,7 +25,7 @@ function toAbsPath(relativePath: string): string {
*/ */
export async function mergeEpisodeVideos(episodeId: number, dramaId: number, storyboardIds?: number[]): Promise<number> { export async function mergeEpisodeVideos(episodeId: number, dramaId: number, storyboardIds?: number[]): Promise<number> {
let storyboards = await db.select().from(schema.storyboards) let storyboards = await db.select().from(schema.storyboards)
.where(eq(schema.storyboards.episodeId, episodeId)) .where(and(eq(schema.storyboards.episodeId, episodeId), isNull(schema.storyboards.deletedAt)))
.orderBy(schema.storyboards.storyboardNumber) .orderBy(schema.storyboards.storyboardNumber)
if (storyboardIds?.length) { if (storyboardIds?.length) {
@@ -0,0 +1,100 @@
/** Durable all-reference preparation for single-image video workflows. */
import { createHash } from 'node:crypto'
import { eq } from 'drizzle-orm'
import { db, schema } from '../db/index.js'
import { getActiveConfig, getConfigById } from './ai.js'
import { resolveSceneLock, type SceneLock } from './scene-continuity.js'
import type { AIConfig } from './adapters/types.js'
export const needsComposedFirstFrame = (model?: string | null) => ['LTX-2.3', 'LTX-2.3-Text-Preserve', 'Wan2.2-I2V'].includes(model || '')
export interface FrameAsset { kind: 'scene' | 'character' | 'prop'; id: number; name: string; url: string; description: string }
export interface FrameComposition {
version: 1; sourceHash: string; assets: FrameAsset[]; scene: SceneLock
shot: { description: string; videoPrompt: string; start: string; end: string }
sourcePrompt: string; size: string
imageConfigId: number; imageProvider: string; imageBaseUrl: string; imageModel: string; imageGpu: string
rounds: number; round: number; childTaskIds: number[]; images: string[]
frameUrl?: string; reusedFromTaskId?: number; continuityFrameUrl?: string
}
async function shotInputs(id: number) {
const shot = await db.select().from(schema.storyboards).where(eq(schema.storyboards.id, id)).get()
if (!shot || shot.deletedAt) throw new Error('首帧合成:分镜不存在或已删除')
const episode = await db.select().from(schema.episodes).where(eq(schema.episodes.id, shot.episodeId)).get()
if (!episode || episode.deletedAt) throw new Error('首帧合成:当前集不存在')
const scene = await resolveSceneLock(id)
const assets: FrameAsset[] = [{ kind: 'scene', id: scene.sceneId, name: scene.location, url: scene.imageUrl, description: scene.environment }]
const characters = await db.select().from(schema.storyboardCharacters).where(eq(schema.storyboardCharacters.storyboardId, id))
for (const link of characters.sort((a,b) => a.characterId-b.characterId)) {
const item = await db.select().from(schema.characters).where(eq(schema.characters.id, link.characterId)).get()
if (!item || item.deletedAt || item.dramaId !== episode.dramaId) throw new Error('首帧合成:绑定角色已失效,请重新绑定')
assets.push({ kind:'character', id:item.id, name:item.name, url:item.imageUrl || item.localPath || '', description:[item.appearance,item.description].filter(Boolean).join(';') })
}
const props = await db.select().from(schema.storyboardProps).where(eq(schema.storyboardProps.storyboardId, id))
for (const link of props.sort((a,b) => a.propId-b.propId)) {
const item = await db.select().from(schema.props).where(eq(schema.props.id, link.propId)).get()
if (!item || item.deletedAt || item.dramaId !== episode.dramaId) throw new Error('首帧合成:绑定道具已失效,请重新绑定')
assets.push({ kind:'prop', id:item.id, name:item.name, url:item.imageUrl || item.localPath || '', description:item.description || '' })
}
const missing = assets.filter(a => !a.url)
if (missing.length) throw new Error(`首帧合成:${missing.map(a=>a.name).join('、')}尚无参考图,请先生成或上传;未忽略任何绑定素材`)
return { episode, scene, assets, shot:{ description:shot.description || '', videoPrompt:shot.videoPrompt || '', start:shot.continuityStart || '', end:shot.continuityEnd || '' } }
}
function inputHash(value: Pick<FrameComposition,'assets'|'scene'|'shot'>) {
return createHash('sha256').update(JSON.stringify({ assets:value.assets, scene:value.scene.hash, shot:value.shot })).digest('hex')
}
export async function assertFrameInputsCurrent(id: number, plan: FrameComposition) {
if (inputHash(await shotInputs(id)) !== plan.sourceHash) throw new Error('首帧合成:分镜、绑定素材或场景设定已变化,请重新生成;旧首帧保留,未提交视频')
}
export function frameImageSize(ratio = '16:9') {
const sizes: Record<string,string> = { '16:9':'1280x720', '9:16':'720x1280', '1:1':'1024x1024', '4:3':'1152x864', '3:4':'864x1152', '21:9':'1344x576' }
if (!sizes[ratio]) throw new Error('首帧合成:不支持的画面比例')
return sizes[ratio]
}
/** First request carries six originals. Later requests carry the prior full scene + five new originals. */
export function frameBatches<T>(assets: T[], continuation = false): T[][] {
const batches: T[][] = []
for (let i=0; i<assets.length;) { const count = batches.length || continuation ? 5 : 6; batches.push(assets.slice(i,i+count)); i+=count }
return batches
}
export async function prepareFrameComposition(id: number, sourcePrompt: string, ratio?: string, configId?: number, model?: string) {
const inputs = await shotInputs(id)
const selectedId = configId ?? inputs.episode.imageConfigId ?? undefined
const config = selectedId ? await getConfigById(selectedId) : await getActiveConfig('image')
const row = config?.id ? await db.select().from(schema.aiServiceConfigs).where(eq(schema.aiServiceConfigs.id,config.id)).get() : null
if (!config || !row || row.serviceType !== 'image') throw new Error('首帧合成:请先选择已启用的 Qwen Image 2.1 图片配置')
const chosen = model || config.model
if (config.provider !== 'openai' || chosen !== 'qwen_image_2_1' || !JSON.parse(row.model || '[]').includes(chosen)) throw new Error('首帧合成:当前流程需要支持多参考图的 Qwen Image 2.1,请在顶部图片模型中选择')
const plan: FrameComposition = { version:1, sourceHash:inputHash(inputs), assets:inputs.assets, scene:inputs.scene, shot:inputs.shot,
sourcePrompt, size:frameImageSize(ratio), imageConfigId:config.id!, imageProvider:config.provider, imageBaseUrl:config.baseUrl,
imageModel:chosen, imageGpu:config.gpu || 'auto', rounds:frameBatches(inputs.assets).length, round:0, childTaskIds:[], images:[] }
return { plan, config:{ ...config, model:chosen } }
}
export async function frameImageConfig(plan: FrameComposition): Promise<AIConfig> {
const config = await getConfigById(plan.imageConfigId)
if (!config || config.baseUrl !== plan.imageBaseUrl || config.provider !== plan.imageProvider) throw new Error('首帧合成:原图片服务配置已停用或变更,请重新选择后生成')
return { ...config, model:plan.imageModel, gpu:plan.imageGpu as AIConfig['gpu'] }
}
export function withoutImageIndexes(text: string, assets: FrameAsset[]) {
const escape = (s:string) => s.replace(/[.*+?^${}()|[\]\\]/g,'\\$&')
const names = [...new Set(assets.map(a=>a.name))].sort((a,b)=>b.length-a.length)
return text.replace(new RegExp(`@图片\\d+?(?=(?:${names.map(escape).join('|')}))`,'g'),'@')
}
export function compositionRound(plan: FrameComposition, round: number) {
const batches = frameBatches(plan.assets, !!plan.continuityFrameUrl)
const batch = batches[round]
if (!batch) throw new Error('首帧合成:轮次记录无效')
const previous = round ? plan.images[round-1] : plan.continuityFrameUrl
const references = [...(previous ? [previous] : []), ...batch.map(a=>a.url)]
const list = batch.map((asset,index) => `图片${index+1+(previous?1:0)}:${asset.kind==='scene'?'场景环境':asset.kind==='character'?'人物身份与服装':'道具外形'}「${asset.name}」;${asset.description}`).join('\n')
const initialShot = plan.shot.description.split(/【镜头\s*2】/)[0]
const prompt = `生成一张完整的电影分镜首帧,只有一个连续画面,禁止拼图、分屏、三视图、人物设定板、文字标签和字幕。\n`+
`全部绑定素材:${plan.assets.map(a=>a.name).join('、')}。严格区分人物身份、服装、道具与场景;人物参考图中的白底或背景不替换场景。参考图只定义素材,不强行让后续才登场的人物提前进入首帧。\n`+
(previous ? `图片1是${round?'上轮融合的完整首帧':'前镜头的最后一帧'}:保留已有的人脸、服装、场景布局、道具位置和机位,只按本轮新参考图补充或校正相应素材,不能删除或替换其他已确定元素。\n` : '')+
`本轮图像顺序与用途:\n${list}\n`+
`固定场景:${plan.scene.location};${plan.scene.environment}\n时间:${plan.scene.time};光源:${plan.scene.lighting}\n`+
`首个镜头描述:${initialShot}\n开始状态:${plan.shot.start || '采用第一个镜头/第一个时间段的起始状态'}\n`+
`下面是整段视频的风格和动作上下文,仅用于确定首帧。只画动作开始前或开始瞬间,不把多个时间段同时画进一张图,不提前画结束状态,不把对白画成文字:\n${withoutImageIndexes(plan.sourcePrompt,plan.assets)}`
if (prompt.length > 32000) throw new Error('首帧合成:提示词过长,请精简分镜描述后重试')
return { references, prompt, assets:batch, rounds:batches.length }
}
+222 -15
View File
@@ -7,6 +7,7 @@ import { and, desc, eq, gt, ne, or, isNull } from 'drizzle-orm'
import fs from 'node:fs' import fs from 'node:fs'
import { getAbsolutePath } from '../utils/storage.js' import { getAbsolutePath } from '../utils/storage.js'
import { downloadVideo, safeTaskError } from './video-download.js' import { downloadVideo, safeTaskError } from './video-download.js'
import { supportsVideoCleanup, videoOutputIndex, videoReceipt, verifyRetainedVideo, pendingVideoCleanup, cleanupSavedVideo } from './video-cleanup.js'
import { prepareReferenceAudio, resolveReferenceAudioUrls } from './reference-audio.js' import { prepareReferenceAudio, resolveReferenceAudioUrls } from './reference-audio.js'
import { getActiveConfig, getConfigById } from './ai.js' import { getActiveConfig, getConfigById } from './ai.js'
import { parseWorkerRouting, type WorkerRouting } from './worker-routing.js' import { parseWorkerRouting, type WorkerRouting } from './worker-routing.js'
@@ -22,6 +23,8 @@ import { logTaskError, logTaskPayload, logTaskProgress, logTaskStart, logTaskSuc
import { continuityPrompt, extractContinuityFrame, resolveContinuity, sourceReadiness, validateContinuityModel, type ContinuityPlan } from './continuity.js' import { continuityPrompt, extractContinuityFrame, resolveContinuity, sourceReadiness, validateContinuityModel, type ContinuityPlan } from './continuity.js'
import { englishSubmissionPrompt, usesEnglishVideoPrompt } from './english-video-prompt.js' import { englishSubmissionPrompt, usesEnglishVideoPrompt } from './english-video-prompt.js'
import { resolveSceneLock, assertSceneLockCurrent, addSceneReference, sceneLockPrompt, stripSceneLock } from './scene-continuity.js'
import { needsComposedFirstFrame, prepareFrameComposition, assertFrameInputsCurrent, frameImageConfig, frameBatches, compositionRound, withoutImageIndexes, type FrameComposition } from './first-frame-composition.js'
type TaskType = 'image' | 'video' type TaskType = 'image' | 'video'
@@ -51,6 +54,9 @@ interface GenerateImageParams {
} }
interface GenerateVideoParams { interface GenerateVideoParams {
frameImageConfigId?: number
frameImageModel?: string
regenerateFirstFrame?: boolean
gpu?: AIConfig['gpu'] gpu?: AIConfig['gpu']
replaceTaskId?: number replaceTaskId?: number
variant?: 'hd' variant?: 'hd'
@@ -86,6 +92,13 @@ export async function generateImage(params: GenerateImageParams): Promise<number
: await getActiveConfig('image') : await getActiveConfig('image')
if (!config) throw new Error('未配置图片模型,请先到「设置」页添加并启用 AI 服务') if (!config) throw new Error('未配置图片模型,请先到「设置」页添加并启用 AI 服务')
// Only storyboard images use a physical set; character/prop sheets keep their own backgrounds.
const sceneLock = params.storyboardId ? await resolveSceneLock(params.storyboardId) : null
if (sceneLock) {
const references = addSceneReference(sceneLock, params.referenceImages)
params = { ...params, prompt: await sceneLockPrompt(params.prompt, sceneLock, { referenceIndex: references.index }), referenceImages: references.urls }
}
const id = await createTask('image', config, { const id = await createTask('image', config, {
storyboardId: params.storyboardId, storyboardId: params.storyboardId,
dramaId: params.dramaId, dramaId: params.dramaId,
@@ -98,6 +111,7 @@ export async function generateImage(params: GenerateImageParams): Promise<number
size: params.size || '1920x1080', size: params.size || '1920x1080',
frameType: params.frameType, frameType: params.frameType,
referenceImages: params.referenceImages, referenceImages: params.referenceImages,
sceneLock,
}) })
logTaskStart('ImageTask', 'enqueue', { logTaskStart('ImageTask', 'enqueue', {
@@ -136,13 +150,51 @@ export async function generateVideo(params: GenerateVideoParams): Promise<number
params.referenceAudioUrls = await prepareReferenceAudio(params.referenceAudioUrls, config.provider) params.referenceAudioUrls = await prepareReferenceAudio(params.referenceAudioUrls, config.provider)
const continuity = storyboardId && params.variant !== 'hd' ? (await resolveContinuity(storyboardId)).plan : null const continuity = storyboardId && params.variant !== 'hd' ? (await resolveContinuity(storyboardId)).plan : null
const sceneLock = storyboardId ? await resolveSceneLock(storyboardId) : null
params = { ...params, prompt: stripSceneLock(params.prompt) }
let composition: FrameComposition | undefined
if (storyboardId && params.variant !== 'hd' && needsComposedFirstFrame(params.model || config.model) && !params.firstFrameUrl && !params.imageUrl) {
if (params.referenceVideoUrls?.length || params.referenceAudioUrls?.length || params.lastFrameUrl || params.referenceFileUrl || params.referenceLinkUrl) throw new Error('首帧合成:LTX/Wan2.2 当前流程只支持绑定图片,不支持参考视频、音色或尾帧')
const prepared = await prepareFrameComposition(storyboardId, params.prompt, params.aspectRatio, params.frameImageConfigId, params.frameImageModel)
composition = prepared.plan
// Resolve every bound image before admission/cancelling an old job. No truncation.
for (let i=0; i<composition.assets.length; i+=6) await normalizeReferenceImages(composition.assets.slice(i,i+6).map(a=>a.url))
const firstBatch = compositionRound(composition,0)
getImageAdapter(prepared.config.provider).buildGenerateRequest(prepared.config,{id:0,model:composition.imageModel,prompt:firstBatch.prompt,size:composition.size,
referenceImages:JSON.stringify(await normalizeReferenceImages(firstBatch.references,true))})
if (continuity?.mode !== 'first_frame' && !params.regenerateFirstFrame) {
const previous = await db.select().from(schema.sysTask).where(and(eq(schema.sysTask.type,'video'),eq(schema.sysTask.storyboardId,storyboardId))).orderBy(desc(schema.sysTask.id))
for (const task of previous) {
const old = parseTaskParams(task.params).firstFrameComposition as FrameComposition | undefined
if (old?.frameUrl && old.sourceHash===composition.sourceHash && old.sourcePrompt===composition.sourcePrompt && old.size===composition.size
&& old.imageConfigId===composition.imageConfigId && old.imageModel===composition.imageModel && old.imageGpu===composition.imageGpu
&& old.imageProvider===composition.imageProvider && old.imageBaseUrl===composition.imageBaseUrl
&& fs.existsSync(getAbsolutePath(old.frameUrl))) {
composition = { ...composition, frameUrl:old.frameUrl, images:old.images, childTaskIds:old.childTaskIds, round:old.rounds, rounds:old.rounds, reusedFromTaskId:task.id }; break
}
}
}
params.prompt = withoutImageIndexes(params.prompt,composition.assets)
params.referenceImageUrls = []
}
// Frame workflows already carry the environment in their single visual input.
const sceneFrame = !!composition || continuity?.mode === 'first_frame' || params.variant === 'hd' || !!(params.firstFrameUrl || params.imageUrl)
let sceneReferenceIndex: number | null = null
if (sceneLock && !sceneFrame && !params.referenceFileUrl && !params.referenceLinkUrl) {
const limit = /^(?:LTX-2\.3|Wan2\.2-I2V)/.test(params.model || config.model || '') ? 1 : config.provider === 'aliyun' ? 10 : 9
const references = addSceneReference(sceneLock, params.referenceImageUrls, limit)
params.referenceImageUrls = references.urls
sceneReferenceIndex = references.index
}
if (continuity) validateContinuityModel(continuity, config, params.model) if (continuity) validateContinuityModel(continuity, config, params.model)
if (continuity?.mode === 'first_frame' && params.referenceAudioUrls?.length) throw new Error('当前尾帧接续工作流不支持音色参考,请改为状态承接') if (continuity?.mode === 'first_frame' && params.referenceAudioUrls?.length) throw new Error('当前尾帧接续工作流不支持音色参考,请改为状态承接')
const previewParams = { ...params, prompt: continuity ? continuityPrompt(params.prompt, continuity) : params.prompt } const previewParams = { ...params, prompt: continuity ? continuityPrompt(params.prompt, continuity) : params.prompt }
const english = usesEnglishVideoPrompt(params.model || config.model) const english = usesEnglishVideoPrompt(params.model || config.model)
// Translate and validate BEFORE cancelling an old job or admitting a new one. // Translate and validate BEFORE cancelling an old job or admitting a new one.
if (english) previewParams.prompt = await englishSubmissionPrompt(previewParams.prompt, storyboardId, params.dramaId) if (english) previewParams.prompt = await englishSubmissionPrompt(previewParams.prompt, storyboardId, params.dramaId)
if (continuity?.mode === 'first_frame') { // Append after translation so the deterministic layout rules cannot disappear in an LLM rewrite.
if (sceneLock) previewParams.prompt = await sceneLockPrompt(previewParams.prompt, sceneLock, { english, referenceIndex: sceneReferenceIndex, frame: sceneFrame })
if (continuity?.mode === 'first_frame' || composition) {
// A validation-only placeholder; never submitted as a generation input. // A validation-only placeholder; never submitted as a generation input.
previewParams.firstFrameUrl = 'data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAwMCAO+aN1kAAAAASUVORK5CYII=' previewParams.firstFrameUrl = 'data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAwMCAO+aN1kAAAAASUVORK5CYII='
previewParams.imageUrl = undefined; previewParams.lastFrameUrl = undefined previewParams.imageUrl = undefined; previewParams.lastFrameUrl = undefined
@@ -169,6 +221,7 @@ export async function generateVideo(params: GenerateVideoParams): Promise<number
if (!response.ok) throw new Error(`新显卡未就绪,原任务保持运行:${safeTaskError(await response.text())}`) if (!response.ok) throw new Error(`新显卡未就绪,原任务保持运行:${safeTaskError(await response.text())}`)
} }
} }
if (sceneLock && storyboardId) await assertSceneLockCurrent(storyboardId, sceneLock)
if (storyboardId) { if (storyboardId) {
const tasks = await db.select().from(schema.sysTask).where(and(eq(schema.sysTask.type, 'video'), eq(schema.sysTask.storyboardId, storyboardId))) const tasks = await db.select().from(schema.sysTask).where(and(eq(schema.sysTask.type, 'video'), eq(schema.sysTask.storyboardId, storyboardId)))
.orderBy(desc(schema.sysTask.id)) .orderBy(desc(schema.sysTask.id))
@@ -182,14 +235,16 @@ export async function generateVideo(params: GenerateVideoParams): Promise<number
const id = await createTask('video', config, { const id = await createTask('video', config, {
storyboardId: params.storyboardId, storyboardId: params.storyboardId,
dramaId: params.dramaId, dramaId: params.dramaId,
prompt: english ? previewParams.prompt : params.prompt, prompt: previewParams.prompt,
model: params.model || config.model, model: params.model || config.model,
}, { }, {
continuity, continuity,
sceneLock: sceneLock ? { ...sceneLock, referenceIndex: sceneReferenceIndex, referenceMode: sceneFrame ? 'frame' : sceneReferenceIndex ? 'scene_image' : 'text_only' } : null,
promptLanguage: english ? 'en' : 'original', promptLanguage: english ? 'en' : 'original',
promptIncludesContinuity: english, promptIncludesContinuity: true,
sourcePrompt: english ? params.prompt : undefined, sourcePrompt: params.prompt,
referenceMode: params.referenceMode || 'reference', referenceMode: params.referenceMode || 'reference',
firstFrameComposition: composition,
variant: params.variant, variant: params.variant,
originalVideoUrl: params.originalVideoUrl, originalVideoUrl: params.originalVideoUrl,
replacesTaskId: params.replaceTaskId, replacesTaskId: params.replaceTaskId,
@@ -246,6 +301,9 @@ async function cancelVideoForReplacement(record: SysTaskRecord) {
export function canCancelGenerationTask(record: SysTaskRecord): boolean { export function canCancelGenerationTask(record: SysTaskRecord): boolean {
if (record.status !== 'processing' || record.resultUrl || ['downloading', 'saving', 'transferring'].includes(record.stage || '')) return false if (record.status !== 'processing' || record.resultUrl || ['downloading', 'saving', 'transferring'].includes(record.stage || '')) return false
if (!record.taskId && record.type === 'video' && parseTaskParams(record.params).firstFrameComposition
&& ['preparing_first_frame','first_frame_ready','waiting_predecessor','blocked_predecessor','cancelling'].includes(record.stage || '')) return true
if (!record.taskId && record.type === 'image' && record.stage === 'prepared' && parseTaskParams(record.params).compositionParentTaskId) return true
if (record.type === 'video' && !record.taskId && ['waiting_predecessor', 'blocked_predecessor'].includes(record.stage || '')) return true if (record.type === 'video' && !record.taskId && ['waiting_predecessor', 'blocked_predecessor'].includes(record.stage || '')) return true
return !!record.taskId && record.stage !== 'submitting' return !!record.taskId && record.stage !== 'submitting'
&& (record.type === 'video' ? ['minimax', 'localvideo'].includes(record.provider || '') : record.type === 'image' && record.provider === 'openai') && (record.type === 'video' ? ['minimax', 'localvideo'].includes(record.provider || '') : record.type === 'image' && record.provider === 'openai')
@@ -266,6 +324,12 @@ export async function cancelGenerationTask(id: number, reason: 'user' | 'replace
if (record.status !== 'processing') throw new Error('任务已结束,请刷新列表') if (record.status !== 'processing') throw new Error('任务已结束,请刷新列表')
if (cancellingTasks.has(id)) return { outcome: 'pending' } if (cancellingTasks.has(id)) return { outcome: 'pending' }
if (!canCancelGenerationTask(record)) throw new Error('当前任务尚未返回编号、正在保存或不支持取消,请稍后刷新') if (!canCancelGenerationTask(record)) throw new Error('当前任务尚未返回编号、正在保存或不支持取消,请稍后刷新')
if (!record.taskId && parseTaskParams(record.params).firstFrameComposition) {
await db.update(schema.sysTask).set({ stage:'cancelling', params:JSON.stringify({...parseTaskParams(record.params),cancellation:{requestedAt:now(),reason}}),updatedAt:now() })
.where(and(eq(schema.sysTask.id,id),eq(schema.sysTask.status,'processing')))
return { outcome:await settleCompositionCancellation(id) }
}
if (!record.taskId && record.type === 'image' && record.stage === 'prepared') { await markTaskCancelled(id,reason); return {outcome:'cancelled'} }
if (!record.taskId) { await cancelWaitingVideo(id); return { outcome: 'cancelled' } } if (!record.taskId) { await cancelWaitingVideo(id); return { outcome: 'cancelled' } }
cancellingTasks.add(id) cancellingTasks.add(id)
try { try {
@@ -327,17 +391,23 @@ async function createTask(
params: Record<string, unknown>, params: Record<string, unknown>,
): Promise<number> { ): Promise<number> {
const ts = now() const ts = now()
const res = await db.insert(schema.sysTask).values({ const res = db.transaction(tx => {
if (fields.storyboardId) {
const shot = tx.select().from(schema.storyboards).where(eq(schema.storyboards.id, fields.storyboardId)).get()
if (!shot || shot.deletedAt) throw new Error('分镜不存在或已删除,未提交生成任务')
}
return tx.insert(schema.sysTask).values({
type, type,
...fields, ...fields,
provider: config.provider, provider: config.provider,
configId: config.id, configId: config.id,
configBaseUrl: config.baseUrl, configBaseUrl: config.baseUrl,
stage: (params.continuity as ContinuityPlan | undefined)?.mode === 'first_frame' ? 'waiting_predecessor' : 'submitting', stage: (params.continuity as ContinuityPlan | undefined)?.mode === 'first_frame' ? 'waiting_predecessor' : params.firstFrameComposition ? 'preparing_first_frame' : 'submitting',
params: JSON.stringify({ ...params, gpu: config.gpu || 'auto' }), params: JSON.stringify({ ...params, gpu: config.gpu || 'auto' }),
status: 'processing', status: 'processing',
createdAt: ts, createdAt: ts,
updatedAt: ts, updatedAt: ts,
}).run()
}) })
const id = getInsertId(res) const id = getInsertId(res)
@@ -354,6 +424,97 @@ function parseTaskParams(raw: string | null | undefined): Record<string, any> {
} }
} }
async function settleCompositionCancellation(id: number): Promise<'cancelled'|'pending'> {
if (cancellingTasks.has(id)) return 'pending'
cancellingTasks.add(id)
try {
const parent = await db.select().from(schema.sysTask).where(eq(schema.sysTask.id,id)).get()
if (!parent || parent.status !== 'processing') return 'cancelled'
const params = parseTaskParams(parent.params), plan = params.firstFrameComposition as FrameComposition
const childId = plan.childTaskIds.at(-1)
const child = childId ? await db.select().from(schema.sysTask).where(eq(schema.sysTask.id,childId)).get() : null
if (child?.status === 'processing') {
if (!canCancelGenerationTask(child)) return 'pending'
const result = await cancelGenerationTask(child.id)
if (result.outcome === 'pending') return 'pending'
}
await markTaskCancelled(id,params.cancellation?.reason)
return 'cancelled'
} finally { cancellingTasks.delete(id) }
}
async function prepareComposedFirstFrame(id: number): Promise<boolean> {
while (true) {
const parent = await db.select().from(schema.sysTask).where(eq(schema.sysTask.id,id)).get()
if (!parent || parent.status !== 'processing') return false
const params = parseTaskParams(parent.params), plan = params.firstFrameComposition as FrameComposition | undefined
if (!plan || !parent.storyboardId) return true
if (params.cancellation) {
if (await settleCompositionCancellation(id) === 'cancelled') return false
await new Promise(r=>setTimeout(r,1000)); continue
}
if (plan.frameUrl) {
await assertFrameInputsCurrent(parent.storyboardId,plan)
if (!fs.existsSync(getAbsolutePath(plan.frameUrl))) throw new Error('首帧合成:已保存的首帧文件不存在,请重做首帧')
params.firstFrameUrl=plan.frameUrl; params.imageUrl=undefined; params.lastFrameUrl=undefined
params.referenceImageUrls=[]; params.referenceVideoUrls=[]; params.referenceAudioUrls=[]
params.referenceMode='first_frame'
const claimed = db.transaction(tx=>{
const current = tx.select().from(schema.sysTask).where(eq(schema.sysTask.id,id)).get()
if (!current || current.status!=='processing' || parseTaskParams(current.params).cancellation) return false
tx.update(schema.sysTask).set({params:JSON.stringify(params),stage:'first_frame_ready',updatedAt:now()}).where(eq(schema.sysTask.id,id)).run()
tx.update(schema.storyboards).set({firstFrameImage:plan.frameUrl,updatedAt:now()}).where(eq(schema.storyboards.id,parent.storyboardId!)).run()
return true
})
return claimed
}
const round = plan.round
const childId = plan.childTaskIds[round]
if (childId) {
const child = await db.select().from(schema.sysTask).where(eq(schema.sysTask.id,childId)).get()
if (!child) throw new Error('首帧合成:关联图片任务已删除,未提交视频,请重做首帧')
if (child.status==='failed') {
if (child.errorCode==='user_cancelled') { await markTaskCancelled(id); return false }
throw new Error(`首帧合成:第 ${round+1}/${plan.rounds} 轮失败,未提交视频。${safeTaskError(child.errorMsg || '请检查图片任务后重试')}`)
}
if (child.status==='completed') {
if (!child.localPath) throw new Error('首帧合成:图片尚未保存到本地,未提交视频')
await assertFrameInputsCurrent(parent.storyboardId,plan)
plan.images[round]=child.localPath; plan.round=round+1
if (plan.round>=plan.rounds) plan.frameUrl=child.localPath
// Preserve a cancellation that arrived while checking image/scene state.
db.transaction(tx=>{
const current = tx.select().from(schema.sysTask).where(eq(schema.sysTask.id,id)).get()
if (current?.status==='processing' && !parseTaskParams(current.params).cancellation) {
tx.update(schema.sysTask).set({params:JSON.stringify(params),updatedAt:now()}).where(eq(schema.sysTask.id,id)).run()
}
})
continue
}
await new Promise(r=>setTimeout(r,1000)); continue
}
await assertFrameInputsCurrent(parent.storyboardId,plan)
const config = await frameImageConfig(plan)
const batch = compositionRound(plan,round)
const prepared = await normalizeReferenceImages(batch.references,true)
getImageAdapter(config.provider).buildGenerateRequest(config,{id:0,model:plan.imageModel,prompt:batch.prompt,size:plan.size,referenceImages:JSON.stringify(prepared)})
const child = db.transaction(tx=>{
const current = tx.select().from(schema.sysTask).where(eq(schema.sysTask.id,id)).get()
if (!current || current.status!=='processing' || parseTaskParams(current.params).cancellation) return 0
const inserted=tx.insert(schema.sysTask).values({type:'image',storyboardId:parent.storyboardId,dramaId:parent.dramaId,
provider:config.provider,configId:config.id,configBaseUrl:config.baseUrl,model:plan.imageModel,prompt:batch.prompt,
stage:'prepared',status:'processing',createdAt:now(),updatedAt:now(),params:JSON.stringify({size:plan.size,
referenceImages:batch.references,frameType:'video_first_frame_stage',compositionParentTaskId:id,
compositionRound:round+1,compositionRounds:batch.rounds,referenceAssets:batch.assets,gpu:plan.imageGpu})}).run()
const imageId=getInsertId(inserted)
plan.childTaskIds[round]=imageId; plan.rounds=batch.rounds
tx.update(schema.sysTask).set({params:JSON.stringify(params),stage:'preparing_first_frame',updatedAt:now()}).where(eq(schema.sysTask.id,id)).run()
return imageId
})
if (child) runBackground(child,()=>processTask(child,config))
}
}
async function prepareContinuityTask(record: SysTaskRecord): Promise<boolean> { async function prepareContinuityTask(record: SysTaskRecord): Promise<boolean> {
const params = parseTaskParams(record.params) const params = parseTaskParams(record.params)
const plan = params.continuity as ContinuityPlan | undefined const plan = params.continuity as ContinuityPlan | undefined
@@ -361,8 +522,9 @@ async function prepareContinuityTask(record: SysTaskRecord): Promise<boolean> {
while (true) { while (true) {
const current = await db.select().from(schema.sysTask).where(eq(schema.sysTask.id, record.id)).get() const current = await db.select().from(schema.sysTask).where(eq(schema.sysTask.id, record.id)).get()
if (!current || current.status !== 'processing') return false if (!current || current.status !== 'processing') return false
if (parseTaskParams(current.params).cancellation) return false
const fresh = (await resolveContinuity(record.storyboardId)).plan const fresh = (await resolveContinuity(record.storyboardId)).plan
if (JSON.stringify(fresh) !== JSON.stringify(plan)) throw new Error('镜头衔接设置或前镜头结束状态已变化,请重新提交此镜头') if (JSON.stringify(fresh) !== JSON.stringify(Object.fromEntries(Object.keys(fresh).map(key=>[key,plan[key as keyof ContinuityPlan]])))) throw new Error('镜头衔接设置或前镜头结束状态已变化,请重新提交此镜头')
const source = plan.sourceId ? await sourceReadiness(plan.sourceId) : null const source = plan.sourceId ? await sourceReadiness(plan.sourceId) : null
if (plan.mode === 'first_frame' && source?.state !== 'ready') { if (plan.mode === 'first_frame' && source?.state !== 'ready') {
await db.update(schema.sysTask).set({ stage: source?.state || 'waiting_predecessor', updatedAt: now() }) await db.update(schema.sysTask).set({ stage: source?.state || 'waiting_predecessor', updatedAt: now() })
@@ -375,6 +537,10 @@ async function prepareContinuityTask(record: SysTaskRecord): Promise<boolean> {
const after = await sourceReadiness(plan.sourceId!) const after = await sourceReadiness(plan.sourceId!)
if (after.state !== 'ready' || after.video !== source.video || after.taskId !== source.taskId) continue if (after.state !== 'ready' || after.video !== source.video || after.taskId !== source.taskId) continue
params.firstFrameUrl = frameUrl; params.imageUrl = undefined; params.lastFrameUrl = undefined params.firstFrameUrl = frameUrl; params.imageUrl = undefined; params.lastFrameUrl = undefined
if (params.firstFrameComposition) {
params.firstFrameComposition.continuityFrameUrl = frameUrl
params.firstFrameComposition.rounds = frameBatches(params.firstFrameComposition.assets,true).length
}
params.referenceImageUrls = []; params.referenceVideoUrls = []; params.referenceAudioUrls = [] params.referenceImageUrls = []; params.referenceVideoUrls = []; params.referenceAudioUrls = []
params.referenceFileUrl = undefined; params.referenceLinkUrl = undefined params.referenceFileUrl = undefined; params.referenceLinkUrl = undefined
params.referenceMode = 'first_frame' params.referenceMode = 'first_frame'
@@ -383,9 +549,13 @@ async function prepareContinuityTask(record: SysTaskRecord): Promise<boolean> {
params.continuity = { ...plan, sourceVideo: source?.video || '', sourceTaskId: source?.taskId ?? null } params.continuity = { ...plan, sourceVideo: source?.video || '', sourceTaskId: source?.taskId ?? null }
} }
const prompt = params.promptIncludesContinuity ? record.prompt || '' : continuityPrompt(record.prompt || '', plan) const prompt = params.promptIncludesContinuity ? record.prompt || '' : continuityPrompt(record.prompt || '', plan)
const claimed = await db.update(schema.sysTask).set({ params: JSON.stringify(params), prompt, stage: 'submitting', updatedAt: now() }) return db.transaction(tx=>{
.where(and(eq(schema.sysTask.id, record.id), eq(schema.sysTask.status, 'processing'), isNull(schema.sysTask.taskId))).returning({ id: schema.sysTask.id }) const latest=tx.select().from(schema.sysTask).where(eq(schema.sysTask.id,record.id)).get()
return claimed.length > 0 if (!latest || latest.status!=='processing' || latest.taskId || parseTaskParams(latest.params).cancellation) return false
tx.update(schema.sysTask).set({params:JSON.stringify(params),prompt,stage:params.firstFrameComposition?'preparing_first_frame':'submitting',updatedAt:now()})
.where(eq(schema.sysTask.id,record.id)).run()
return true
})
} }
} }
@@ -393,7 +563,14 @@ async function processTask(id: number, config: AIConfig) {
try { try {
let [record] = await db.select().from(schema.sysTask).where(eq(schema.sysTask.id, id)) let [record] = await db.select().from(schema.sysTask).where(eq(schema.sysTask.id, id))
if (!record || record.status !== 'processing') return if (!record || record.status !== 'processing') return
if (record.type === 'video' && !await prepareContinuityTask(record)) return if (record.type === 'video') {
if (parseTaskParams(record.params).cancellation && parseTaskParams(record.params).firstFrameComposition) {
while (await settleCompositionCancellation(id)==='pending') await new Promise(r=>setTimeout(r,1000))
return
}
if (!await prepareContinuityTask(record)) return
if (!await prepareComposedFirstFrame(id)) return
}
;[record] = await db.select().from(schema.sysTask).where(eq(schema.sysTask.id, id)) ;[record] = await db.select().from(schema.sysTask).where(eq(schema.sysTask.id, id))
const type = record.type as TaskType const type = record.type as TaskType
const label = taskLabel(type) const label = taskLabel(type)
@@ -410,7 +587,7 @@ async function processTask(id: number, config: AIConfig) {
if (type === 'image') { if (type === 'image') {
const adapter = getImageAdapter(config.provider) const adapter = getImageAdapter(config.provider)
const resolvedReferenceImages = await normalizeReferenceImages(params.referenceImages) const resolvedReferenceImages = await normalizeReferenceImages(params.referenceImages,!!params.compositionParentTaskId)
;({ url, method, headers, body } = adapter.buildGenerateRequest(config, { ;({ url, method, headers, body } = adapter.buildGenerateRequest(config, {
id: record.id, id: record.id,
model: record.model, model: record.model,
@@ -456,12 +633,18 @@ async function processTask(id: number, config: AIConfig) {
const beforeSend = await db.select().from(schema.sysTask).where(eq(schema.sysTask.id, id)).get() const beforeSend = await db.select().from(schema.sysTask).where(eq(schema.sysTask.id, id)).get()
if (!beforeSend || beforeSend.status !== 'processing') return if (!beforeSend || beforeSend.status !== 'processing') return
if (parseTaskParams(beforeSend.params).cancellation) return
if (params.continuity?.mode === 'first_frame') { if (params.continuity?.mode === 'first_frame') {
const source = await sourceReadiness(params.continuity.sourceId) const source = await sourceReadiness(params.continuity.sourceId)
if (source.state !== 'ready' || source.video !== params.continuity.sourceVideo || source.taskId !== params.continuity.sourceTaskId) { if (source.state !== 'ready' || source.video !== params.continuity.sourceVideo || source.taskId !== params.continuity.sourceTaskId) {
throw new Error('前镜头版本已变化,请重新提交此镜头;尚未发送到显卡') throw new Error('前镜头版本已变化,请重新提交此镜头;尚未发送到显卡')
} }
} }
if (type === 'video' && params.firstFrameComposition && record.storyboardId) await assertFrameInputsCurrent(record.storyboardId,params.firstFrameComposition)
// Only this state may have sent an unacknowledged request; never auto-resubmit it on restart.
const claimed = await db.update(schema.sysTask).set({ stage:'submitting', updatedAt:now() })
.where(and(eq(schema.sysTask.id,id),eq(schema.sysTask.status,'processing'),ne(schema.sysTask.stage,'cancelling'))).returning({id:schema.sysTask.id})
if (!claimed.length) return
const resp = await fetch(url, { const resp = await fetch(url, {
method, method,
headers, headers,
@@ -712,6 +895,9 @@ async function handleImageCompleteBase64(record: SysTaskRecord, base64Data: stri
// 图片完成后回写业务表:分镜(按 frameType)、角色、场景、道具 // 图片完成后回写业务表:分镜(按 frameType)、角色、场景、道具
async function writeBackImageAssets(record: SysTaskRecord, localPath: string) { async function writeBackImageAssets(record: SysTaskRecord, localPath: string) {
const params = parseTaskParams(record.params) const params = parseTaskParams(record.params)
// Intermediate frames belong to this video attempt. Only the finished parent
// may publish its final frame; never overwrite character/scene source artwork.
if (params.compositionParentTaskId) return
// A late result is kept in task history, never substituted for a newer attempt. // A late result is kept in task history, never substituted for a newer attempt.
const newer = await db.select().from(schema.sysTask).where(and(eq(schema.sysTask.type, 'image'), gt(schema.sysTask.id, record.id), or( const newer = await db.select().from(schema.sysTask).where(and(eq(schema.sysTask.type, 'image'), gt(schema.sysTask.id, record.id), or(
eq(schema.sysTask.characterId, record.characterId || -1), eq(schema.sysTask.sceneId, record.sceneId || -1), eq(schema.sysTask.characterId, record.characterId || -1), eq(schema.sysTask.sceneId, record.sceneId || -1),
@@ -747,6 +933,13 @@ async function handleVideoComplete(record: SysTaskRecord, videoUrl: string, dura
try { try {
const localPath = record.localPath && fs.existsSync(getAbsolutePath(record.localPath)) const localPath = record.localPath && fs.existsSync(getAbsolutePath(record.localPath))
? record.localPath : await downloadVideo(videoUrl, record.id) ? record.localPath : await downloadVideo(videoUrl, record.id)
const autoCleanup=supportsVideoCleanup(current)
// Keep the original downloaded-byte receipt even if native HD padding is cropped below.
const originalReceipt=autoCleanup ? (record.localPath && params.downloadReceipt ? params.downloadReceipt : await videoReceipt(localPath)) : null
if (originalReceipt) {
params.downloadReceipt=originalReceipt
await db.update(schema.sysTask).set({params:JSON.stringify(params)}).where(eq(schema.sysTask.id,record.id))
}
if (params.variant === 'hd') { if (params.variant === 'hd') {
const readDimensions = () => new Promise<{ width?: number; height?: number }>((resolve, reject) => { const readDimensions = () => new Promise<{ width?: number; height?: number }>((resolve, reject) => {
ffmpeg.ffprobe(getAbsolutePath(localPath), (error, meta) => { ffmpeg.ffprobe(getAbsolutePath(localPath), (error, meta) => {
@@ -775,6 +968,10 @@ async function handleVideoComplete(record: SysTaskRecord, videoUrl: string, dura
} }
await db.update(schema.sysTask).set({ localPath, stage: 'saving', updatedAt: now() }).where(eq(schema.sysTask.id, record.id)) await db.update(schema.sysTask).set({ localPath, stage: 'saving', updatedAt: now() }).where(eq(schema.sysTask.id, record.id))
await extractVideoPoster(localPath) await extractVideoPoster(localPath)
if (autoCleanup && originalReceipt) {
const retained=await verifyRetainedVideo(localPath)
params.remoteCleanup=pendingVideoCleanup(originalReceipt as {bytes:number;sha256:string},retained,videoOutputIndex(videoUrl,current.configBaseUrl!,current.taskId!))
}
// 完成状态与业务回写原子提交;恢复旧任务不能覆盖用户后来生成的视频。 // 完成状态与业务回写原子提交;恢复旧任务不能覆盖用户后来生成的视频。
db.transaction(tx => { db.transaction(tx => {
const newer = record.storyboardId ? tx.select({ id: schema.sysTask.id }).from(schema.sysTask) const newer = record.storyboardId ? tx.select({ id: schema.sysTask.id }).from(schema.sysTask)
@@ -782,9 +979,11 @@ async function handleVideoComplete(record: SysTaskRecord, videoUrl: string, dura
if (record.storyboardId && !newer && parseTaskParams(record.params).variant !== 'hd') tx.update(schema.storyboards) if (record.storyboardId && !newer && parseTaskParams(record.params).variant !== 'hd') tx.update(schema.storyboards)
.set({ videoUrl: localPath, duration: duration || undefined, updatedAt: now() }) .set({ videoUrl: localPath, duration: duration || undefined, updatedAt: now() })
.where(eq(schema.storyboards.id, record.storyboardId)).run() .where(eq(schema.storyboards.id, record.storyboardId)).run()
tx.update(schema.sysTask).set({ localPath, status: 'completed', stage: 'completed', errorCode: null, tx.update(schema.sysTask).set({ localPath, params:JSON.stringify(params), status: 'completed', stage: 'completed', errorCode: null,
errorMsg: null, completedAt: now(), updatedAt: now() }).where(eq(schema.sysTask.id, record.id)).run() errorMsg: null, completedAt: now(), updatedAt: now() }).where(eq(schema.sysTask.id, record.id)).run()
}) })
// Cleanup has its own retry state; it must never invalidate a saved video.
if (params.remoteCleanup) void cleanupSavedVideo(record.id).catch(()=>{})
} catch (error) { } catch (error) {
await failTask(record.id, safeTaskError(error), 'delivery_failed') await failTask(record.id, safeTaskError(error), 'delivery_failed')
} }
@@ -861,6 +1060,14 @@ export async function resumeInterruptedTasks() {
const tasks = candidates.filter(task => task.status === 'processing' || (!task.errorCode && /^Timeout: (polling attempts exhausted|Polling exceeded 10 minutes)/.test(task.errorMsg || '') const tasks = candidates.filter(task => task.status === 'processing' || (!task.errorCode && /^Timeout: (polling attempts exhausted|Polling exceeded 10 minutes)/.test(task.errorMsg || '')
&& isLocalQueuedTask({ provider: task.provider || '', baseUrl: task.configBaseUrl || '' }, task.taskId))) && isLocalQueuedTask({ provider: task.provider || '', baseUrl: task.configBaseUrl || '' }, task.taskId)))
for (const task of tasks) { for (const task of tasks) {
const params = parseTaskParams(task.params)
if (task.status === 'processing' && !task.taskId && (
task.type === 'video' && params.firstFrameComposition && ['preparing_first_frame','first_frame_ready','cancelling','waiting_predecessor','blocked_predecessor'].includes(task.stage || '')
|| task.type === 'image' && params.compositionParentTaskId && task.stage === 'prepared')) {
try { const config = await recoveryConfig(task); runBackground(task.id,()=>processTask(task.id,{...config,gpu:params.gpu || config.gpu})) }
catch (error) { await failTask(task.id,safeTaskError(error),'recovery_failed') }
continue
}
if (task.type === 'video' && !task.taskId && task.status === 'processing' && ['waiting_predecessor', 'blocked_predecessor'].includes(task.stage || '') && parseTaskParams(task.params).continuity?.mode === 'first_frame') { if (task.type === 'video' && !task.taskId && task.status === 'processing' && ['waiting_predecessor', 'blocked_predecessor'].includes(task.stage || '') && parseTaskParams(task.params).continuity?.mode === 'first_frame') {
try { try {
const config = await recoveryConfig(task) const config = await recoveryConfig(task)
@@ -879,7 +1086,7 @@ export async function resumeInterruptedTasks() {
// ─── 参考素材归一化 ─────────────────────────────────────────────── // ─── 参考素材归一化 ───────────────────────────────────────────────
async function normalizeReferenceImages(refs: string[] | null | undefined): Promise<string[]> { async function normalizeReferenceImages(refs: string[] | null | undefined, preserveOrder = false): Promise<string[]> {
if (!Array.isArray(refs) || !refs.length) return [] if (!Array.isArray(refs) || !refs.length) return []
const deduped = Array.from( const deduped = Array.from(
@@ -890,7 +1097,7 @@ async function normalizeReferenceImages(refs: string[] | null | undefined): Prom
), ),
) )
const normalized = await Promise.all(deduped.map(async (value) => { const normalized = await Promise.all((preserveOrder ? refs : deduped).map(async (value) => {
if (value.startsWith('data:image/')) return value if (value.startsWith('data:image/')) return value
if (value.startsWith('static/') || value.startsWith('/static/')) { if (value.startsWith('static/') || value.startsWith('/static/')) {
const localPath = value.startsWith('/static/') ? value.slice(1) : value const localPath = value.startsWith('/static/') ? value.slice(1) : value
@@ -0,0 +1,93 @@
/** Shared environment constraints, independent of character identity and shot-to-shot motion. */
import { createHash } from 'node:crypto'
import { eq } from 'drizzle-orm'
import { db, schema } from '../db/index.js'
import { translateVisualPrompt } from './english-video-prompt.js'
export interface SceneLock {
version: 1
sceneId: number
location: string
time: string
environment: string
lighting: string
imageUrl: string
hash: string
source: 'fixed' | 'scene'
}
export function sceneLockFromScene(scene: typeof schema.scenes.$inferSelect): SceneLock {
// finalPrompt is deliberately excluded: empty-set image prompts prohibit people
// and prescribe a fixed camera, which would contradict a performance shot.
const fields = { version: 1 as const, sceneId: scene.id, location: scene.location.trim(),
time: scene.time?.trim() || '', environment: (scene.continuityPrompt || scene.prompt || '').trim(),
lighting: scene.lighting?.trim() || '', imageUrl: (scene.imageUrl || scene.localPath || '').trim(),
source: scene.continuityPrompt?.trim() ? 'fixed' as const : 'scene' as const }
return { ...fields, hash: createHash('sha256').update(JSON.stringify(fields)).digest('hex') }
}
export async function resolveSceneLock(storyboardId: number): Promise<SceneLock> {
const shot = await db.select().from(schema.storyboards).where(eq(schema.storyboards.id, storyboardId)).get()
if (!shot || shot.deletedAt) throw new Error('分镜不存在')
if (!shot.sceneId) throw new Error('请先为此分镜绑定场景,避免模型随机重画背景')
const episode = await db.select().from(schema.episodes).where(eq(schema.episodes.id, shot.episodeId)).get()
const scene = await db.select().from(schema.scenes).where(eq(schema.scenes.id, shot.sceneId)).get()
if (!episode || episode.deletedAt || !scene || scene.deletedAt || scene.dramaId !== episode.dramaId) {
throw new Error('分镜绑定的场景已失效,请重新绑定当前项目的场景')
}
const lock = sceneLockFromScene(scene)
if (!lock.environment || lock.environment === lock.location && !lock.lighting && !lock.imageUrl) {
throw new Error('场景只有名称,请先补充空间、布景与灯光描述,或上传场景参考图')
}
return lock
}
export async function assertSceneLockCurrent(id: number, lock: SceneLock) {
if ((await resolveSceneLock(id)).hash !== lock.hash) throw new Error('准备期间场景设定或绑定已变化,请重新提交;未替换原任务')
}
const referenceKey = (url: string) => url.replace(/^https?:\/\/(?:localhost|127\.0\.0\.1)(?::\d+)?(?=\/static\/)/i, '').replace(/^\/(?=static\/)/, '')
export function addSceneReference(lock: SceneLock, references: string[] = [], limit = 9) {
// Append only. Reordering/deduplicating the existing list corrupts @图片N mappings.
const urls = [...references]
if (urls.length > limit) throw new Error(`所选模型最多支持 ${limit} 张参考图,当前 ${urls.length} 张;请减少参考图或选择多图模型`)
let index = lock.imageUrl ? urls.findIndex(url => referenceKey(url) === referenceKey(lock.imageUrl)) : -1
if (lock.imageUrl && index < 0) {
if (urls.length >= limit) throw new Error(`场景参考图尚未包含,所选模型最多支持 ${limit} 张参考图;请减少其他参考图或使用包含场景的首帧`)
index = urls.length
urls.push(lock.imageUrl)
}
return { urls, index: index < 0 ? null : index + 1 }
}
const translations = new Map<string, Promise<string>>()
async function englishEnvironment(lock: SceneLock) {
const source = [`Set: @${lock.location}`, `Time: ${lock.time || 'as established in the scene reference'}`,
`Environment: ${lock.environment}`, `Lighting: ${lock.lighting || 'preserve the established scene lighting'}`].join('\n')
if (!translations.has(lock.hash)) {
const job = translateVisualPrompt(source, [lock.location])
translations.set(lock.hash, job)
void job.catch(() => { if (translations.get(lock.hash) === job) translations.delete(lock.hash) })
if (translations.size > 100) translations.delete(translations.keys().next().value!)
}
return translations.get(lock.hash)!
}
export const stripSceneLock = (prompt: string) => prompt.replace(/\s*\[HB_SCENE_LOCK_V1\][\s\S]*?\[\/HB_SCENE_LOCK_V1\]/g, '').trim()
export async function sceneLockPrompt(prompt: string, lock: SceneLock, options: { english?: boolean; referenceIndex?: number | null; frame?: boolean } = {}) {
const environment = options.english ? await englishEnvironment(lock)
: [`场景:@${lock.location}`, `时间:${lock.time}`, `固定环境:${lock.environment}`, `光源与色调:${lock.lighting}`].filter(line => !line.endsWith(':')).join('\n')
const reference = options.frame
? 'Use the supplied first frame / original-video frame as the visual environment anchor; continue its physical set.'
: options.referenceIndex
? `Reference image ${options.referenceIndex} defines the ENVIRONMENT, not character identity. Preserve its architecture, materials, furniture and relative object positions. Character reference backgrounds must NOT replace this set.`
: 'No scene image is supplied. Use the written environment consistently; do not invent a different room.'
return `${stripSceneLock(prompt)}\n\n[HB_SCENE_LOCK_V1]\nEnvironment continuity — visual directions only, never dialogue, narration or on-screen text.\n${environment}\n${reference}\nApply this environment to EVERY shot and time segment, including close-ups and reverse angles. Keep the same room geometry, doors, windows, wall materials, fixed furnishings, key prop positions and light sources. A tighter crop or changed viewpoint does not change the physical set; do not force off-screen objects into frame. Use this bound scene rather than the previous scene when cutting to a new location. Explicit scripted actions may move props or change lights; preserve the resulting state, do not reset it. Keep the scripted characters and actions; an empty scene reference does not mean removing the actors.\n[/HB_SCENE_LOCK_V1]`
}
export const sceneContinuityInstructions = `
场景一致性:read_storyboard_context 中 scenes 的 environment_lock 是本场景共用的固定设定(来源是 continuity_prompt,未设置时取 prompt);lighting 固定光源与色调。每个分镜必须绑定正确 scene_id;没有对应场景时说明需要先补充场景,不要拿人物参考背景或上一场景凑数。
生成 description、video_prompt 和 video_prompt_en 时,每条分镜都写入当前场景的空间结构、门窗/墙面、关键陈设相对位置、时间与光源等已知信息,不能仅在标题/信息头写场景名,也不能只写人物表演。每个子镜头/时间段写出该景别可见的背景锚点;特写只描述合理可见部分,保持物理位置而非所有视角的屏幕左右相同。不得臆造未提供的房间布局、移动固定家具或用氛围词替换已有光源。角色三视图/LoRA 只用于人物身份,场景图用于环境。
同一 scene_id(包括跨集复用)共用固定设定;不同 scene_id 表示新场景,不把旧房间复制过去。道具移动、灯光开关和破损等剧情变化记入 continuity_start/continuity_end 并承接,不随切镜复位。不要将空场景生图的“无人/固定机位”规则带入有人物表演的视频。已有明确地点冲突应报告修正绑定,不能擅自更改剧情。
画面英文版包含同样的环境约束,中文对白逐字保持。环境约束不作为旁白、对白或字幕。`
@@ -0,0 +1,35 @@
import { and, eq, inArray, isNull } from 'drizzle-orm'
import { db, schema } from '../db/index.js'
import { now } from '../utils/response.js'
import { getStoryboardJob } from './storyboard-job-state.js'
/** Remove shots from production, keeping task history, bindings and media. */
export function deleteStoryboards(episodeId: number, input: unknown) {
if (!Number.isSafeInteger(episodeId) || episodeId <= 0 || !Array.isArray(input) || !input.length || input.length > 500 ||
input.some(id => !Number.isSafeInteger(id) || id <= 0)) throw new Error('请选择当前集要删除的分镜')
const ids = [...new Set(input as number[])]
if (getStoryboardJob(episodeId)?.status === 'running') throw new Error('正在拆分分镜,请先取消拆分或等待完成后再删除')
return db.transaction(tx => {
const rows = tx.select().from(schema.storyboards).where(inArray(schema.storyboards.id, ids)).all()
if (rows.length !== ids.length || rows.some(row => row.episodeId !== episodeId)) throw new Error('所选分镜不存在或不属于当前集,未删除任何分镜')
const active = rows.filter(row => !row.deletedAt)
const running = tx.select().from(schema.sysTask).where(inArray(schema.sysTask.status, ['processing', 'pending', 'queued', 'running'])).all()
for (const task of running) {
let sourceId: number | undefined
try { sourceId = JSON.parse(task.params || '{}').continuity?.sourceId } catch { /* legacy task */ }
if ((task.storyboardId && ids.includes(task.storyboardId)) || (sourceId && ids.includes(sourceId))) {
throw new Error(`任务 #${task.id} 正在生成或依赖所选分镜,请先取消该任务或等待完成;未删除任何分镜`)
}
}
const ts = now()
if (active.length) {
tx.update(schema.storyboards).set({ deletedAt: ts, updatedAt: ts })
.where(inArray(schema.storyboards.id, active.map(row => row.id))).run()
const remaining = tx.select().from(schema.storyboards)
.where(and(eq(schema.storyboards.episodeId, episodeId), isNull(schema.storyboards.deletedAt))).all()
tx.update(schema.episodes).set({ duration: Math.ceil(remaining.reduce((sum, row) => sum + (row.duration || 0), 0) / 60), updatedAt: ts })
.where(eq(schema.episodes.id, episodeId)).run()
}
return { deleted_ids: active.map(row => row.id), count: active.length }
})
}
@@ -0,0 +1,133 @@
import { createHash, randomUUID } from 'node:crypto'
import { eq } from 'drizzle-orm'
import { db, schema } from '../db/index.js'
import { buildStoryboardPlan, sectionCoverage, coverageAdvanced, resumableDraftPrefix, type StoryboardSection } from '../utils/storyboard-coverage.js'
export interface StoryboardJob {
id: string; episodeId: number; dramaId: number
status: 'running'|'done'|'error'|'cancelled'; phase: string
startedAt: string; finishedAt?: string; error?: string
completed: number; target: number; targetSeconds: number; seconds: number
steps: number; rejections: number; lastValidation?: string
model?: string; responseModel?: string; configId?: number
stepTrace?: Array<{step:number;tools:string[];finish:string;error?:string;completed:number}>
endingCue?:string; narrativeEndShot?:number
sections?:StoryboardSection[]; sectionIndex?:number; sectionCount?:number
resumedFrom?:string
drafts: any[]; sourceHash: string; rowsHash: string
}
const jobs = new Map<number, StoryboardJob>()
const controllers = new Map<string, AbortController>()
const key = (id: number) => `storyboard_job:${id}`
const hash = (v: unknown) => createHash('sha256').update(JSON.stringify(v)).digest('hex')
export function storyboardSource(episodeId: number) {
const ep = db.select().from(schema.episodes).where(eq(schema.episodes.id,episodeId)).get()
const rows = db.select().from(schema.storyboards).where(eq(schema.storyboards.episodeId,episodeId)).all()
const ids = new Set(rows.map(r=>r.id))
const characters = db.select().from(schema.storyboardCharacters).all().filter(r=>ids.has(r.storyboardId))
const props = db.select().from(schema.storyboardProps).all().filter(r=>ids.has(r.storyboardId))
return { ep, sourceHash: hash([ep?.content,ep?.scriptContent,ep?.scriptSourceHash]), rowsHash: hash([rows,characters,props]) }
}
export function persistStoryboardJob(job: StoryboardJob) {
db.insert(schema.appSettings).values({key:key(job.episodeId),value:JSON.stringify(job),updatedAt:new Date().toISOString()})
.onConflictDoUpdate({target:schema.appSettings.key,set:{value:JSON.stringify(job),updatedAt:new Date().toISOString()}}).run()
jobs.set(job.episodeId,job)
}
export function getStoryboardJob(episodeId: number): StoryboardJob|null {
let job=jobs.get(episodeId)
if (!job) {
const row=db.select().from(schema.appSettings).where(eq(schema.appSettings.key,key(episodeId))).get()
if (!row) return null
try { job=JSON.parse(row.value) } catch { return null }
if (!job) return null
if (job.status==='running') {
job.status='error';job.phase='拆分已中断';job.error='程序重启中断了拆分,原分镜保留,请重新拆分。';job.finishedAt=new Date().toISOString()
persistStoryboardJob(job)
} else jobs.set(episodeId,job)
}
return job
}
export function publicStoryboardJob(job: StoryboardJob|null) {
if (!job) return null
const {drafts,sourceHash,rowsHash,rejections,stepTrace,endingCue,sections,...view}=job
if(job.status==='done'&&!sections&&job.targetSeconds&&job.seconds<job.targetSeconds*.8) return {...view,phase:'旧分镜需要重新细分',qualityWarning:`此前只生成 ${job.completed} 个分镜、${job.seconds} 秒,低于预计 ${job.targetSeconds} 秒。旧结果未校验逐场覆盖和对白时长,请按新版重新拆分。`}
if((job.status==='error'||job.status==='cancelled')&&sections?.length) {
const source=storyboardSource(job.episodeId)
const canResume=source.sourceHash===sourceHash&&source.rowsHash===rowsHash
const prefix=canResume?resumableDraftPrefix(sections,drafts,job.sectionIndex||0):null
return {...view,canResume,resumeFromSection:prefix?prefix.sectionIndex+1:undefined,reusableShots:prefix?.drafts.length||0}
}
return view
}
export function createStoryboardJob(episodeId: number, dramaId: number, resumeFrom?:string) {
const previous=getStoryboardJob(episodeId)
if (previous?.status==='running') return {job:previous,created:false,controller:controllers.get(previous.id)!}
const source=storyboardSource(episodeId)
if (!source.ep || source.ep.dramaId!==dramaId) throw new Error('当前集不存在或不属于该项目')
if(resumeFrom) {
if(!previous || previous.id!==resumeFrom || !['error','cancelled'].includes(previous.status) || !previous.sections?.length)throw new Error('暂存任务已变更,请刷新后重试')
assertStoryboardSourceUnchanged(previous)
}
const job:StoryboardJob={id:randomUUID(),episodeId,dramaId,status:'running',phase:'正在读取剧本和素材',startedAt:new Date().toISOString(),completed:0,target:0,targetSeconds:0,seconds:0,steps:0,rejections:0,drafts:[],sourceHash:source.sourceHash,rowsHash:source.rowsHash}
job.sections=buildStoryboardPlan(source.ep.scriptContent||source.ep.content||'')
job.sectionIndex=0;job.sectionCount=job.sections.length
if(resumeFrom&&previous) {
const prefix=resumableDraftPrefix(job.sections,previous.drafts,previous.sectionIndex||0)
job.drafts=structuredClone(prefix.drafts);job.sectionIndex=prefix.sectionIndex
job.completed=job.drafts.length;job.seconds=job.drafts.reduce((sum,s)=>sum+(s.duration||0),0)
job.resumedFrom=previous.id;job.phase=`已保留 ${job.completed} 个分镜,从第 ${job.sectionIndex+1} 场继续拆分`
}
job.targetSeconds=job.sections.reduce((sum,s)=>sum+s.targetSeconds,0);job.target=Math.round(job.targetSeconds/12)
if(previous) db.insert(schema.appSettings).values({key:`storyboard_job_archive:${previous.id}`,value:JSON.stringify(previous),updatedAt:new Date().toISOString()}).onConflictDoNothing().run()
const controller=new AbortController();controllers.set(job.id,controller);persistStoryboardJob(job)
return {job,created:true,controller}
}
export function assertStoryboardJob(episodeId: number, id: string) {
const job=jobs.get(episodeId)
if (!job || job.id!==id || job.status!=='running' || controllers.get(id)?.signal.aborted) throw new Error('本次拆分已结束或取消,迟到的结果不会覆盖分镜')
return job
}
export function updateStoryboardJob(episodeId: number, id: string, changes: Partial<StoryboardJob>) {
const job=assertStoryboardJob(episodeId,id);Object.assign(job,changes);persistStoryboardJob(job);return job
}
export function finishStoryboardJob(job: StoryboardJob, status: StoryboardJob['status'], message?: string) {
if (job.status!=='running') return
job.status=status;job.finishedAt=new Date().toISOString();job.phase=status==='done'?'分镜拆分完成':status==='cancelled'?'已取消拆分':'拆分未完成'
if (message) job.error=message.slice(0,800)
if (status!=='done') controllers.get(job.id)?.abort(new Error(message||'已取消拆分'))
persistStoryboardJob(job);controllers.delete(job.id)
}
export function rejectStoryboardBatch(episodeId:number,id:string,message:string) {
const job=assertStoryboardJob(episodeId,id)
job.rejections++;job.lastValidation=message.slice(0,800);job.phase='正在修正分镜校验问题';persistStoryboardJob(job)
if (job.rejections>=3) { finishStoryboardJob(job,'error',`同一批分镜连续三次校验未通过:${message}。原分镜已保留。`);throw new Error(job.error) }
}
export function stageStoryboardBatch(episodeId:number,id:string,items:any[],replace:boolean) {
const job=assertStoryboardJob(episodeId,id)
// A model may repeat replace_existing at the start of each scene. Tracked
// calls always upsert private drafts; replacement happens only at commit.
const byNumber=new Map(job.drafts.map(s=>[s.shot_number,s]))
for (const item of items) byNumber.set(item.shot_number,{...item,source_section:job.sectionIndex||0})
const candidate=[...byNumber.values()].sort((a,b)=>a.shot_number-b.shot_number)
const index=job.sectionIndex||0,section=job.sections?.[index]
const current=job.drafts.filter(s=>s.source_section===index)
const coverage=section?sectionCoverage(section,candidate.filter(s=>s.source_section===index)):null
const previousCoverage=section?sectionCoverage(section,current):null
if(coverage&&previousCoverage&&!coverageAdvanced(previousCoverage,coverage)) {
const missingIds=new Set(previousCoverage.missingDialogue.map(p=>p.id))
const repair=current.filter(s=>s.source_ids?.some((id:string)=>missingIds.has(id))).map(s=>s.shot_number)
throw new Error(`本批未减少缺失原文或对白,已拦截重复分镜。请修改已有镜头 ${repair.join('、')||current.map(s=>s.shot_number).join('、')},不要追加重复内容;使用明确的“角色说:「原文」”格式。`)
}
job.drafts=candidate;job.completed=job.drafts.length;job.seconds=job.drafts.reduce((sum,s)=>sum+(s.duration||10),0)
if(coverage?.ready) job.sectionIndex=index+1
if(job.sections?.length && job.sectionIndex===job.sections.length) job.narrativeEndShot=job.drafts.at(-1)?.shot_number
job.rejections=0;delete job.lastValidation;job.phase=`已暂存 ${job.completed} 个分镜 · 已完成 ${job.sectionIndex||0}/${job.sectionCount||1} 个场次`
persistStoryboardJob(job)
const missingIds=new Set(coverage?.missingDialogue.map(p=>p.id)||[])
const repair=job.drafts.filter(s=>s.source_section===index&&s.source_ids?.some((id:string)=>missingIds.has(id))).map(s=>s.shot_number)
return {count:items.length,total_duration:job.seconds,target_total_seconds:job.targetSeconds,section_complete:!!coverage?.ready,script_complete:!!job.narrativeEndShot,coverage,repair_shot_numbers:repair,message:`已暂存 ${job.completed} 个双语分镜,共 ${job.seconds} 秒。${coverage?.ready?'当前场次已完整覆盖,立即结束本轮,下一场由系统继续安排。':'当前场次尚未完成:缺失对白请修改 repair_shot_numbers 对应已有镜头,原位重存,不要追加重复剧情;缺失来源或时长不足才继续细分原有动作/对白。'}不要跳到后面的场次或结尾,不要压缩对白、删词或加速念完。后续每批最多 4 条并省略 replace_existing。`}
}
export function assertStoryboardSourceUnchanged(job: StoryboardJob) {
const current=storyboardSource(job.episodeId)
if (current.sourceHash!==job.sourceHash || current.rowsHash!==job.rowsHash) throw new Error('拆分期间剧本或分镜已修改,本次结果未覆盖新内容,请重新拆分。')
}
@@ -0,0 +1,91 @@
import { mastra } from '../mastra/index.js'
import { buildAgentRequestContext } from '../agents/context.js'
import { storyboardTools } from '../agents/tools/storyboard-tools.js'
import { isScriptStale } from '../utils/script-source.js'
import { sectionCoverage } from '../utils/storyboard-coverage.js'
import { assertStoryboardJob, createStoryboardJob, finishStoryboardJob, getStoryboardJob, persistStoryboardJob, publicStoryboardJob, storyboardSource } from './storyboard-job-state.js'
export function startStoryboardJob(body: {episode_id:number;drama_id:number;message:string;model?:string;config_id?:number;resume_from?:string}) {
const source=storyboardSource(body.episode_id)
if (!source.ep || source.ep.dramaId!==body.drama_id) throw new Error('当前集不存在或不属于该项目')
if (isScriptStale(source.ep)) throw new Error('格式化剧本已过期,请先重新 AI 改写或跳过改写。')
if (!(source.ep.scriptContent||source.ep.content)?.trim()) throw new Error('请先保存剧本内容')
const {job,created,controller}=createStoryboardJob(body.episode_id,body.drama_id,body.resume_from)
if (!created) return publicStoryboardJob(job)
job.model=body.model;job.configId=body.config_id;persistStoryboardJob(job)
const requestContext=buildAgentRequestContext({episodeId:job.episodeId,dramaId:job.dramaId,modelOverride:body.model,textConfigId:body.config_id})
requestContext.set('storyboardJobId' as never,job.id as never)
const instruction=body.message+`\n系统已按原文划分场次。本轮只拆 readStoryboardContext 返回的当前场次,不能自行跳场。每次 save_storyboards 最多 4 个分镜;每个分镜填写 source_ids,逐项覆盖 source_units。长对白按 4.5 汉字/秒安排时间,8–15 秒装不下就拆成多个镜头;一句可以跨镜头,原文不得删减。只有全任务第一批使用 replace_existing=true,后续按 next_shot_number 追加;如校验失败,修正当前批次,不要跳过。收到 section_complete 后结束本轮,下一场由系统调用。`
const timeout=Number(process.env.HUOBAO_STORYBOARD_TIMEOUT_MS||Math.max(900000,Math.min(3600000,(job.sectionCount||1)*180000)))
const timer=setTimeout(()=>finishStoryboardJob(job,'error',`拆分超过 ${Math.round(timeout/60000)} 分钟仍未完成,已停止等待,原分镜保留。请重试或切换文本模型。`),timeout)
timer.unref()
void (async()=>{
let previousDrafts='',noProgress=0
const cancelled=new Promise<never>((_,reject)=>{
controller.signal.addEventListener('abort',()=>reject(controller.signal.reason||new Error('已停止拆分')),{once:true})
})
try {
for(let sectionIndex=job.sectionIndex||0;sectionIndex<(job.sectionCount||0);sectionIndex++) {
noProgress=0
requestContext.set('storyboardSectionIndex' as never,sectionIndex as never)
for(let attempt=0;attempt<2 && job.sectionIndex===sectionIndex;attempt++) {
assertStoryboardJob(job.episodeId,job.id)
const generate=mastra.getAgent('storyboard_breaker').generate([{role:'user',content:instruction}],{
requestContext,abortSignal:controller.signal,toolCallConcurrency:1,
activeTools:['readStoryboardContext','saveStoryboards'],
// Mastra's maxSteps overrides stopWhen; keep both limits in one condition.
stopWhen:({steps}:{steps:unknown[]})=>(job.sectionIndex||0)>sectionIndex||steps.length>=12,
modelSettings:{maxRetries:0},
onStepFinish:(step:any)=>{
if (job.status!=='running') return
job.steps++
const reason=step?.finishReason
const results=(step?.toolResults||[]).map((r:any)=>r.payload||r)
const names=(step?.toolCalls||[]).map((r:any)=>(r.payload||r).toolName).filter(Boolean)
const errors=results.map((r:any)=>{
let value=r.result
if (typeof value==='string') { try { value=JSON.parse(value) } catch {} }
return value?.error || (r.isError ? (typeof value==='string'?value:value?.message||'分镜工具执行失败') : '')
}).filter(Boolean).map(String)
if (errors.length) { job.lastValidation=errors.join('\n').slice(0,800);job.phase='正在修正分镜保存问题' }
if (reason==='length') job.lastValidation='本批输出达到模型长度上限,请减少本批分镜数量。'
if (step.response?.modelId) job.responseModel=step.response.modelId
job.stepTrace=[...(job.stepTrace||[]),{step:job.steps,tools:names,finish:reason||'',error:errors.join('\n').slice(0,800)||undefined,completed:job.completed}].slice(-12)
const nextDrafts=JSON.stringify(job.drafts)
if (nextDrafts!==previousDrafts) noProgress=0
else if (job.steps>1 && !names.includes('readStoryboardContext')) noProgress++
previousDrafts=nextDrafts
persistStoryboardJob(job)
if (noProgress>=3) finishStoryboardJob(job,'error',`连续三轮未能保存新的分镜,已停止重复请求。${job.lastValidation||'模型没有调用保存工具,请重试或切换文本模型。'} 原分镜保留。`)
},
})
// Reject promptly even if a provider ignores AbortSignal. Tools recheck the
// job identity after awaited translation and before any commit.
await Promise.race([generate,cancelled])
}
if(job.sectionIndex===sectionIndex) {
const coverage=sectionCoverage(job.sections![sectionIndex],job.drafts.filter(s=>s.source_section===sectionIndex))
throw new Error(`第 ${sectionIndex+1} 场尚未完成:缺少 ${coverage.missing.length} 段原文、${coverage.missingDialogue.length} 处对白。本场已暂存 ${coverage.shots} 个镜头、${coverage.seconds} 秒。暂存结果已保留,可从未完成场次继续。`)
}
}
assertStoryboardJob(job.episodeId,job.id)
if (!job.drafts.length) throw new Error(job.lastValidation||'模型结束了回复,但没有生成可保存的分镜,请重试。')
if (job.drafts.some((s,i)=>s.shot_number!==i+1)) throw new Error('分镜编号不连续,未覆盖原分镜,请重新拆分。')
if(job.sectionIndex!==job.sectionCount) throw new Error('还有未拆分的场次,不能标记整集完成')
if (job.targetSeconds && job.seconds<job.targetSeconds*.8) throw new Error(`仅完成 ${job.completed} 个分镜、${job.seconds} 秒,低于本集约 ${job.targetSeconds} 秒的完整拆分要求,未替换原分镜。`)
requestContext.set('storyboardCommit' as never,true as never)
const result=await (storyboardTools.saveStoryboards.execute as any)({replace_existing:true,storyboards:job.drafts},{requestContext})
if (result?.error) throw new Error(result.error)
finishStoryboardJob(job,'done')
} catch(error:any) {
finishStoryboardJob(job,'error',(error?.message||'分镜拆分失败')+' 原分镜保留。')
} finally { clearTimeout(timer) }
})()
return publicStoryboardJob(job)
}
export function cancelStoryboardJob(episodeId:number,id:string) {
const job=getStoryboardJob(episodeId)
if (!job || job.id!==id) throw new Error('拆分任务已变更,请刷新')
finishStoryboardJob(job,'cancelled','已取消拆分,原分镜保留。')
return publicStoryboardJob(job)
}
@@ -0,0 +1,111 @@
import fs from 'node:fs'
import path from 'node:path'
import { createHash } from 'node:crypto'
import { and, eq } from 'drizzle-orm'
import { db, schema } from '../db/index.js'
import { getConfigById } from './ai.js'
import { STORAGE_ROOT } from '../utils/paths.js'
import { ffmpeg, checkFfmpegSuite } from '../utils/ffmpeg.js'
import { safeTaskError } from './video-download.js'
export interface VideoReceipt { bytes: number; sha256: string }
interface CleanupState {
version: 1; status: 'pending'|'cleaned'; receipt: VideoReceipt; localReceipt: VideoReceipt
index: number; attempts: number; nextAttemptAt?: string; lastError?: string; completedAt?: string
}
const paramsOf = (raw?: string|null): Record<string,any> => { try { return JSON.parse(raw || '{}') } catch { return {} } }
/** Only our authenticated local bridges implement a download acknowledgement. */
export function supportsVideoCleanup(record: {provider?:string|null; configBaseUrl?:string|null; taskId?:string|null}) {
if (!['minimax','localvideo'].includes(record.provider || '') || !record.taskId) return false
try {
const url=new URL(record.configBaseUrl || '')
return url.protocol==='http:' && url.hostname==='192.168.110.111' && ['18792','18794'].includes(url.port) && !url.username && !url.password
} catch { return false }
}
export function videoOutputIndex(url: string, base: string, taskId: string) {
const parsed=new URL(url), origin=new URL(base)
const parts=parsed.pathname.split('/').filter(Boolean)
if (parsed.origin!==origin.origin || parts[0]!=='media' || decodeURIComponent(parts[1] || '')!==taskId || !/^\d+$/.test(parts[2] || '')) throw new Error('视频来源与原任务不匹配,未清理服务器文件')
return Number(parts[2])
}
async function localVideoFile(relative:string) {
if (!/^static\/videos\/[^/\\]+\.(mp4|webm)$/i.test(relative)) throw new Error('本地视频路径无效,未清理服务器文件')
const root=await fs.promises.realpath(STORAGE_ROOT)
const file=await fs.promises.realpath(path.join(root,relative.slice(7)))
if (!file.startsWith(root+path.sep)) throw new Error('本地视频超出素材目录,未清理服务器文件')
return file
}
export async function videoReceipt(relative:string):Promise<VideoReceipt> {
const file=await localVideoFile(relative), digest=createHash('sha256')
let bytes=0
for await (const chunk of fs.createReadStream(file)) { digest.update(chunk); bytes+=chunk.length }
if (!bytes) throw new Error('本地视频为空,未清理服务器文件')
return {bytes,sha256:digest.digest('hex')}
}
/** Decode every video/audio packet, then flush and fingerprint the retained file. */
export async function verifyRetainedVideo(relative:string):Promise<VideoReceipt> {
const file=await localVideoFile(relative), suite=await checkFfmpegSuite()
if (!suite.ffmpeg || !suite.ffprobe) throw new Error('视频校验工具不可用,服务器原文件保持不变')
await new Promise<void>((resolve,reject)=>ffmpeg.ffprobe(file,(error,metadata)=>{
const video=metadata?.streams?.find(s=>s.codec_type==='video')
if (error || !video?.width || !video.height || !(Number(metadata.format.duration)>0)) reject(new Error('视频结构或时长无效,服务器原文件保持不变'))
else resolve()
}))
await new Promise<void>((resolve,reject)=>{
const command=ffmpeg(file).inputOptions(['-v error','-xerror']).outputOptions(['-map 0:v:0','-map 0:a?']).format('null').output('-')
const timer=setTimeout(()=>{command.kill('SIGKILL');reject(new Error('视频完整性校验超时,服务器原文件保持不变'))},180_000)
command.on('end',()=>{clearTimeout(timer);resolve()}).on('error',()=>{clearTimeout(timer);reject(new Error('视频解码校验失败,服务器原文件保持不变'))}).run()
})
const handle=await fs.promises.open(file,'r+')
try { await handle.sync() } finally { await handle.close() }
return videoReceipt(relative)
}
export function pendingVideoCleanup(receipt:VideoReceipt,localReceipt:VideoReceipt,index:number):CleanupState {
return {version:1,status:'pending',receipt,localReceipt,index,attempts:0}
}
const active=new Set<number>()
export async function cleanupSavedVideo(id:number) {
if (active.has(id)) return
active.add(id)
try {
const record=await db.select().from(schema.sysTask).where(eq(schema.sysTask.id,id)).get()
const params=paramsOf(record?.params), state=params.remoteCleanup as CleanupState|undefined
if (!record || record.type!=='video' || record.status!=='completed' || !record.localPath || !state || state.status==='cleaned' || !supportsVideoCleanup(record)) return
if (state.nextAttemptAt && Date.parse(state.nextAttemptAt)>Date.now()) return
try {
const receipt=await videoReceipt(record.localPath)
if (receipt.sha256!==state.localReceipt.sha256 || receipt.bytes!==state.localReceipt.bytes) throw new Error('本地视频已改变,暂停发送新的清理确认')
const config=record.configId ? await getConfigById(record.configId) : null
if (!config || config.baseUrl!==record.configBaseUrl || config.provider!==record.provider) throw new Error('原服务配置不可用,等待恢复后清理')
const response=await fetch(`${config.baseUrl.replace(/\/$/,'')}/v2/video_generation/${encodeURIComponent(record.taskId!)}/cleanup`,{
method:'POST',headers:{Authorization:`Bearer ${config.apiKey}`,'Content-Type':'application/json'},
body:JSON.stringify({version:1,index:state.index,...state.receipt}),signal:AbortSignal.timeout(20_000)})
if (!response.ok) throw new Error(`服务器清理确认暂不可用(HTTP ${response.status})`)
const result=await response.json() as any
if (result.status==='cleaned' && result.sha256===state.receipt.sha256 && result.bytes===state.receipt.bytes && result.index===state.index) {
state.status='cleaned';state.completedAt=new Date().toISOString();delete state.lastError;delete state.nextAttemptAt
} else if (result.status==='pending') {
state.nextAttemptAt=new Date(Date.now()+60_000).toISOString()
if (result.retrying) state.lastError='服务器清理暂未成功,将自动重试'
else delete state.lastError
}
else throw new Error('服务器尚未确认清理,保留本地视频并等待重试')
} catch(error) { state.lastError=safeTaskError(error);state.nextAttemptAt=new Date(Date.now()+Math.min(15,2**Math.min(state.attempts,4))*60_000).toISOString() }
state.attempts++
db.transaction(tx=>{
const latest=tx.select().from(schema.sysTask).where(eq(schema.sysTask.id,id)).get()
if (latest?.status==='completed') tx.update(schema.sysTask).set({params:JSON.stringify({...paramsOf(latest.params),remoteCleanup:state})}).where(eq(schema.sysTask.id,id)).run()
})
} finally { active.delete(id) }
}
let timer:ReturnType<typeof setInterval>|undefined
export function resumeVideoCleanups() {
const scan=async()=>{
const records=await db.select().from(schema.sysTask).where(and(eq(schema.sysTask.type,'video'),eq(schema.sysTask.status,'completed')))
// Only acknowledged downloads participate. Never sweep old completed tasks.
for (const record of records) if (paramsOf(record.params).remoteCleanup?.status==='pending') await cleanupSavedVideo(record.id)
}
void scan().catch(()=>{})
if (!timer) { timer=setInterval(()=>{void scan().catch(()=>{})},60_000);timer.unref() }
}
@@ -52,6 +52,8 @@ export async function downloadVideo(url: string, taskId: number, options: {
callback(null, chunk) callback(null, chunk)
} }) } })
await pipeline(Readable.fromWeb(resp.body as any), validator, fs.createWriteStream(partial), { signal }) await pipeline(Readable.fromWeb(resp.body as any), validator, fs.createWriteStream(partial), { signal })
const length=Number(resp.headers.get('content-length'))
if (length>0 && !resp.headers.get('content-encoding') && bytes!==length) throw new VideoDownloadError('download_incomplete','Video download is incomplete',true)
const mp4 = head.length >= 12 && head.toString('ascii', 4, 8) === 'ftyp' const mp4 = head.length >= 12 && head.toString('ascii', 4, 8) === 'ftyp'
const webm = head.length >= 4 && head.readUInt32BE(0) === 0x1a45dfa3 const webm = head.length >= 4 && head.readUInt32BE(0) === 0x1a45dfa3
if (!mp4 && !webm) throw new VideoDownloadError('download_format', 'Response is not an MP4 or WebM video') if (!mp4 && !webm) throw new VideoDownloadError('download_format', 'Response is not an MP4 or WebM video')
@@ -0,0 +1,36 @@
/** One speech classifier for coverage, duration and screen-text validation. */
export function isSpeechPrefix(prefix:string) {
const clause=prefix.slice(-160).split(/[。!?\n]/).at(-1)||''
if (!/[::]\s*$/.test(clause)) return false
// A named object displaying a quotation is not someone speaking it.
if (/(?:写着|写有|印着|刻着|字幕|文字|标语|门牌|屏幕|显示|字样)[^,,;;::]{0,35}[::]\s*$/.test(clause)) return false
return /(?:说|念|喊|吼|叫|问|答|低语|耳语|嘀咕|嘟囔|喃喃|咆哮|呵斥|哭诉|哀嚎|惊呼|叹道|旁白|台词|播报|广播|画外|声音|嗓音|says?|shouts?|screams?|whispers?|dialogue|narrat\w*)[^::「」“”"]{0,100}[::]\s*$/i.test(clause)
|| /^\s*[@#][\p{L}\p{N}·]+(?:\s*[((][^))]*[))])?\s*[::]\s*$/u.test(clause)
}
/** Outermost quotes only. Nested words inside a spoken line count once. */
export function quotedLiterals(text:string):Array<{text:string;start:number;end:number}> {
const pairs:Record<string,string>={'「':'」','『':'』','“':'”','"':'"'}
const result:Array<{text:string;start:number;end:number}>=[]
for(let i=0;i<text.length;i++) {
const end=pairs[text[i]];if(!end)continue
const start=i,stack=[end]
for(i++;i<text.length;i++) {
if(text[i]===stack.at(-1)) {stack.pop();if(!stack.length)break}
else if(pairs[text[i]])stack.push(pairs[text[i]])
}
if(!stack.length)result.push({text:text.slice(start+1,i),start,end:i+1})
}
return result
}
export function spokenLiterals(text:string) {
return quotedLiterals(text).filter(q=>isSpeechPrefix(text.slice(0,q.start))).map(q=>q.text)
}
export function withoutSpokenLiterals(text:string) {
for(const q of quotedLiterals(text).reverse()) {
if(isSpeechPrefix(text.slice(0,q.start)))text=text.slice(0,q.start)+text.slice(q.end)
}
return text
}
@@ -1,3 +1,5 @@
import { withoutSpokenLiterals } from './storyboard-generated-prompt.js'
/** /**
* 分镜段落保存前的确定性检查(save_storyboards 调用)。 * 分镜段落保存前的确定性检查(save_storyboards 调用)。
* 只查能机械判定的三类问题,查出来就把问题清单回给 Agent 让它改了重存: * 只查能机械判定的三类问题,查出来就把问题清单回给 Agent 让它改了重存:
@@ -54,7 +56,7 @@ export function storyboardSegmentProblems(sb: SegmentLike, characters: Array<{ i
} }
for (const [field, text] of [['video_prompt', prompt], ['image_prompt', sb.image_prompt || '']] as const) { for (const [field, text] of [['video_prompt', prompt], ['image_prompt', sb.image_prompt || '']] as const) {
const cleaned = text.replace(NEGATED, '') const cleaned = withoutSpokenLiterals(text).replace(NEGATED, '')
const hit = cleaned.match(SCREEN_TEXT) const hit = cleaned.match(SCREEN_TEXT)
if (hit) problems.push(`${field} 要求画面里出现文字(「${hit[0]}」)——规则字、系统提示、字幕一律后期叠加,画面只写人物反应,并注明画面中不出现任何文字`) if (hit) problems.push(`${field} 要求画面里出现文字(「${hit[0]}」)——规则字、系统提示、字幕一律后期叠加,画面只写人物反应,并注明画面中不出现任何文字`)
} }
@@ -0,0 +1,173 @@
import { targetEpisodeSeconds } from './script-source.js'
import { normalizeNarrative } from './storyboard-script-ending.js'
import { spokenLiterals } from './spoken-literals.js'
export { spokenLiterals, quotedLiterals } from './spoken-literals.js'
export interface SourceUnit { id:string; text:string; dialogue?:string; speech_seconds?:number }
export interface StoryboardSection { id:string; title:string; script:string; units:SourceUnit[]; targetSeconds:number; minSeconds:number; minShots:number }
type CoverageShot={shot_number:number;duration?:number;video_prompt?:string;description?:string;source_ids?:string[];source_section?:number}
/** A deterministic source inventory. Scene headers are boundaries, never hints
* the model may skip. Unformatted scripts use bounded paragraph groups. */
export function buildStoryboardPlan(script:string):StoryboardSection[] {
const blocks:Array<{title:string;lines:string[]}>=[]
let current={title:'正文',lines:[] as string[]}
for(const line of script.replace(/\r\n/g,'\n').split('\n')) {
if(/^\s*#{1,3}\s*(?:S\d+\b|场(?:次|景)?\s*[一二三四五六七八九十\d]+)/i.test(line)) {
if(current.lines.some(s=>s.trim())) blocks.push(current)
current={title:line.trim(),lines:[]}
} else current.lines.push(line)
}
if(current.lines.some(s=>s.trim())) blocks.push(current)
const bounded:Array<{title:string;text:string}>=[]
for(const block of blocks) {
const paragraphs=block.lines.join('\n').split(/\n\s*\n/).map(s=>s.trim()).filter(Boolean)
let part:string[]=[],size=0
for(const paragraph of paragraphs) {
if(size>900) { bounded.push({title:block.title,text:part.join('\n\n')});part=[];size=0 }
part.push(paragraph);size+=paragraph.length
}
if(part.length) bounded.push({title:block.title,text:part.join('\n\n')})
}
return bounded.map((block,index)=>{
const id=`S${String(index+1).padStart(2,'0')}`
const units=block.text.split(/\n+/).map(s=>s.trim()).filter(Boolean).flatMap<SourceUnit>((text,i)=>{
const spoken=text.match(/^[\p{L}\p{N}· ]{1,30}[::]\s*(?:[((][^))]*[))]\s*)?([^\n]+)$/u)
const unitId=`${id}P${String(i+1).padStart(2,'0')}`
if(!spoken)return [{id:unitId,text}]
const parts=splitSpeech(spoken[1]),prefix=text.slice(0,text.length-spoken[1].length)
return parts.map((part,p)=>({id:parts.length===1?unitId:`${unitId}.${p+1}`,text:prefix+part,dialogue:part,speech_seconds:Math.ceil(speechSeconds(part))}))
})
const dialogueSeconds=units.reduce((sum,u)=>sum+(u.dialogue?speechSeconds(u.dialogue):0),0)
const targetSeconds=Math.max(targetEpisodeSeconds(block.text),Math.ceil(dialogueSeconds+units.filter(u=>!u.dialogue).length))
const minSeconds=Math.max(Math.ceil(targetSeconds*.8),Math.ceil(dialogueSeconds))
return {id,title:block.title,script:block.title+'\n\n'+units.map(u=>u.text).join('\n\n'),units,targetSeconds,minSeconds,minShots:Math.max(1,Math.ceil(minSeconds/12))}
})
}
// Chinese: 4.5 characters/sec; other languages: 2.5 words/sec. This is a
// configurable writing pace estimate, not a speech synthesizer measurement.
export function speechSeconds(text:string) {
const chinese=(text.match(/\p{Script=Han}/gu)||[]).length
const other=(text.replace(/\p{Script=Han}/gu,' ').match(/[\p{L}\p{N}]+/gu)||[]).length
return chinese/4.5+other/2.5
}
/** Split long original speech without rewriting a character. Prefer a nearby
* clause boundary; the concatenated parts equal the exact original string. */
export function splitSpeech(text:string):string[] {
const parts:string[]=[]
let remaining=text
while(speechSeconds(remaining)>10) {
let end=0,boundary=0
for(const char of remaining) {
if(speechSeconds(remaining.slice(0,end+char.length))>10)break
end+=char.length
if(/[,,。.!!??;;\s]/.test(char))boundary=end
}
const cut=boundary>=end/2?boundary:Math.max(1,end)
parts.push(remaining.slice(0,cut));remaining=remaining.slice(cut)
}
if(remaining)parts.push(remaining)
return parts
}
const timeMarkers=(prompt:string)=>[...prompt.matchAll(/^[ \t]*(\d+(?:\.\d+)?)\s*[-–—~~至到]\s*(\d+(?:\.\d+)?)\s*(?:秒|s(?![a-z])|sec)\s*[::]/gim)]
/** Correct small timing errors in generated drafts, in both languages together.
* Never remove speech/actions, change order, or exceed the shot length limit. */
export function fitDraftDialogueTiming<T extends CoverageShot & {video_prompt_en?:string}>(shot:T):T {
const prompt=shot.video_prompt||'',markers=timeMarkers(prompt)
if(!markers.length)return shot
let originalCursor=0
for(const m of markers) { if(Number(m[1])!==originalCursor||Number(m[2])<=originalCursor)return shot;originalCursor=Number(m[2]) }
if(originalCursor!==(shot.duration||0))return shot
const minimum=markers.map((m,i)=>{
const part=prompt.slice(m.index!+m[0].length,markers[i+1]?.index??prompt.length)
return Math.max(1,Math.ceil(spokenLiterals(part).reduce((sum,s)=>sum+speechSeconds(s),0)))
})
const needed=minimum.reduce((a,b)=>a+b,0)
if(needed>15||minimum.every((s,i)=>s<=Number(markers[i][2])-Number(markers[i][1])))return shot
const duration=Math.max(8,shot.duration||0,needed)
if(duration>15)return shot
const lengths=[...minimum];let remaining=duration-needed
for(let i=0;i<lengths.length&&remaining>0;i++) {
const extra=Math.min(remaining,Math.max(0,Number(markers[i][2])-Number(markers[i][1])-lengths[i]));lengths[i]+=extra;remaining-=extra
}
lengths[lengths.length-1]+=remaining
const replace=(text:string,english:boolean)=>{
const found=timeMarkers(text)
if(found.length!==markers.length||found.some((m,i)=>Number(m[1])!==Number(markers[i][1])||Number(m[2])!==Number(markers[i][2])))return text
let cursor=0,output='',end=0
found.forEach((m,i)=>{output+=text.slice(end,m.index);const next=cursor+lengths[i];output+=`${cursor}-${next}${english?'s:':'秒:'}`;end=m.index!+m[0].length;cursor=next})
return output+text.slice(end)
}
return {...shot,duration,video_prompt:replace(prompt,false),video_prompt_en:shot.video_prompt_en?replace(shot.video_prompt_en,true):shot.video_prompt_en}
}
export function dialogueTimingProblems(shot:CoverageShot):string[] {
const prompt=shot.video_prompt||''
const markers=timeMarkers(prompt)
const problems:string[]=[]
if(!markers.length) return ['缺少从 0 秒开始的完整时间轴']
for(let i=0;i<markers.length;i++) {
const marker=markers[i],part=prompt.slice(marker.index!+marker[0].length,markers[i+1]?.index??prompt.length)
const speech=spokenLiterals(part).reduce((sum,s)=>sum+speechSeconds(s),0)
const available=Number(marker[2])-Number(marker[1])
if(speech>available+.25) problems.push(`${marker[1]}-${marker[2]} 秒的对白至少需要 ${Math.ceil(speech)} 秒,目前只有 ${available} 秒;拆开长对白,分配更长时间段或更多分镜,不要删词或加速念完`)
}
const duration=shot.duration||0
if(duration<8||duration>15) problems.push(`段落时长 ${duration} 秒不在 8–15 秒范围内,请细分镜头`)
return problems
}
export function sectionCoverage(section:StoryboardSection,shots:CoverageShot[]) {
const ids=new Set(shots.flatMap(s=>s.source_ids||[]))
const missing=section.units.filter(u=>!ids.has(u.id)).map(u=>u.id)
const speech=normalizeNarrative(shots.flatMap(s=>spokenLiterals(s.video_prompt||'')).join(''))
let speechCursor=0
const missingDialogue=section.units.filter(u=>u.dialogue).flatMap(u=>{
// Allow one original line to span multiple shots, while retaining every
// phrase. Punctuation, quoting and breathing marks may differ.
const phrases=u.dialogue!.split(/[,,。.!!??;;::\n]/).map(normalizeNarrative).filter(Boolean)
return phrases.filter(p=>{
const found=speech.indexOf(p,speechCursor)
if(found<0)return true
speechCursor=found+p.length;return false
}).map(p=>({id:u.id,text:p}))
})
const seconds=shots.reduce((sum,s)=>sum+(s.duration||0),0)
const ready=!missing.length&&!missingDialogue.length&&seconds>=section.minSeconds&&shots.length>=section.minShots
return {ready,missing,missingDialogue,seconds,minSeconds:section.minSeconds,shots:shots.length,minShots:section.minShots}
}
/** More duplicate shots are not progress when the same words remain missing. */
export function coverageAdvanced(before:ReturnType<typeof sectionCoverage>,after:ReturnType<typeof sectionCoverage>) {
return after.ready || after.missing.length<before.missing.length
|| after.missingDialogue.length<before.missingDialogue.length
|| Math.min(after.seconds,after.minSeconds)>Math.min(before.seconds,before.minSeconds)
|| Math.min(after.shots,after.minShots)>Math.min(before.shots,before.minShots)
}
/** Reuse only whole, validated scenes before the failed scene. Incomplete
* retries remain archived; never replay their duplicate shots into the result. */
export function resumableDraftPrefix(sections:StoryboardSection[],drafts:any[],completedSections:number) {
const kept:any[]=[];let sectionIndex=0
for(let index=0;index<Math.min(completedSections,sections.length);index++) {
const shots=drafts.filter(s=>s.source_section===index)
if(!sectionCoverage(sections[index],shots).ready || sectionBatchProblems(sections[index],shots).length
|| shots.some((s,i)=>s.shot_number!==kept.length+i+1))break
kept.push(...shots);sectionIndex++
}
return {drafts:kept,sectionIndex}
}
export function sectionBatchProblems(section:StoryboardSection,items:CoverageShot[]) {
const allowed=new Set(section.units.map(u=>u.id))
return items.flatMap(s=>{
const issues=dialogueTimingProblems(s)
if(!s.source_ids?.length) issues.push('必须填写 source_ids,指明本镜头对应当前场次的哪些原文段落')
for(const id of s.source_ids||[]) if(!allowed.has(id))issues.push(`source_ids ${id} 不属于当前场次 ${section.id},不能跳场或提前写结尾`)
return issues.map(p=>`段落 ${s.shot_number}:${p}`)
})
}
@@ -0,0 +1,28 @@
const QUOTED = /「[^」]*」|『[^』]*』|“[^”]*”|"[^"\n]*"/g
export { withoutSpokenLiterals } from './spoken-literals.js'
const escape = (s: string) => s.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')
/** Normalize invented voice references only in AI drafts, preserving quotes,
* real uploaded voices and unknown/unbound speakers for explicit validation. */
export function normalizeDraftVoices(prompt: string | undefined, characters: Array<{id:number;name:string;voiceAudioUrl:string|null}>, boundIds: number[]) {
if (!prompt?.includes('#')) return prompt
const quoted: string[] = []
let text = prompt.replace(QUOTED, value => `\u0000${quoted.push(value)-1}\u0000`)
const names = characters.map(c => c.name).filter(Boolean).sort((a,b) => b.length-a.length)
if (names.length) text = text.replace(new RegExp(`#(${names.map(escape).join('|')})(?:音色)?`, 'g'), (marker, name) => {
const character = characters.find(c => c.name === name)
return character && boundIds.includes(character.id) && !character.voiceAudioUrl?.trim() ? `@${name}` : marker
})
return text.replace(/\u0000(\d+)\u0000/g, (_, i) => quoted[Number(i)])
}
/** Models sometimes invent category prefixes (@场景房间 / @角色小明).
* Canonicalize only exact existing asset names, outside literal speech. */
export function normalizeDraftAssets(prompt: string | undefined, names: string[]) {
if (!prompt?.includes('@') || !names.length) return prompt
const quoted:string[]=[]
const text=prompt.replace(QUOTED,value=>`\u0000${quoted.push(value)-1}\u0000`)
const alternatives=[...names].sort((a,b)=>b.length-a.length).map(escape).join('|')
return text.replace(new RegExp(`@(?:场景|角色|人物|道具)(${alternatives})`,'g'),'@$1')
.replace(/\u0000(\d+)\u0000/g,(_,i)=>quoted[Number(i)])
}
@@ -0,0 +1,17 @@
export const normalizeNarrative = (value:string) => value.replace(/[^\p{L}\p{N}]/gu,'')
/** Only use a distinctive, unique terminal dialogue as a mechanical stop cue.
* Short utterances and free-form action endings remain model-directed. */
export function scriptEndingCue(script:string) {
const line=script.trim().split('\n').map(s=>s.trim()).filter(Boolean).at(-1)||''
const match=line.match(/^[\p{L}\p{N}· ]{1,30}[::]\s*(?:[((][^))]*[))]\s*)?(.+)$/u)
if (!match) return ''
const cue=normalizeNarrative(match[1])
const normalized=normalizeNarrative(script)
return cue.length>=16 && normalized.split(cue).length===2 ? cue : ''
}
export function endingShotIndex(items:Array<{description?:string;video_prompt?:string}>,cue?:string) {
if(!cue) return -1
return items.findIndex(s=>[s.description,s.video_prompt].some(p=>p&&normalizeNarrative(p).includes(cue)))
}
@@ -0,0 +1,23 @@
import { test } from 'node:test'
import assert from 'node:assert/strict'
import { LocalVideoAdapter } from '../src/services/adapters/local-video.js'
import type { AIConfig, VideoGenerationRecord } from '../src/services/adapters/types.js'
const adapter = new LocalVideoAdapter()
const config = { provider:'localvideo', baseUrl:'http://fixture.invalid', apiKey:'fixture', model:'LTX-2.3', gpu:'gpu3' } as AIConfig
const base = { id:1, prompt:'A girl in a fixed dormitory.', duration:9, resolution:'720p', model:'LTX-2.3' } as VideoGenerationRecord
test('LTX rejects the screenshot failure before a provider job is admitted', () => {
assert.throws(() => adapter.buildGenerateRequest(config, {...base, referenceImageUrls:JSON.stringify(['scene','girl','prop'])}), /LTX-2.3 最多支持 1 张参考图/)
const request = adapter.buildGenerateRequest(config, {...base, referenceImageUrls:JSON.stringify(['scene'])})
const body = request.body as any
assert.equal(body.model,'LTX-2.3')
assert.equal(body.gpu,'gpu3')
assert.equal(body.content.filter((x:any)=>x.type==='image_url').length,1)
})
test('local model-specific references do not regress H3 or image-required models', () => {
assert.doesNotThrow(() => adapter.buildGenerateRequest(config, {...base, model:'MiniMax-H3', referenceImageUrls:JSON.stringify(['scene','girl','prop'])}))
assert.doesNotThrow(() => adapter.buildGenerateRequest(config, base))
for (const model of ['Wan2.2-I2V','MiniMax-H3-R2V-4Step','MiniMax-H3-VDN-H3']) {
assert.throws(() => adapter.buildGenerateRequest(config, {...base,model}), /需要至少 1 张参考图/)
}
assert.throws(() => adapter.buildGenerateRequest(config, {...base,referenceAudioUrls:'["voice"]'}), /暂不支持参考视频、音色或尾帧/)
})
@@ -0,0 +1,72 @@
import test from 'node:test'
import assert from 'node:assert/strict'
import {buildStoryboardPlan,dialogueTimingProblems,sectionBatchProblems,sectionCoverage,spokenLiterals,splitSpeech,speechSeconds,fitDraftDialogueTiming} from '../src/utils/storyboard-coverage.js'
const dialogue='请先检查每一扇窗户是否关好,再把桌上的图纸按照编号摆放整齐,确认材料数量以后我们才能开始修理屋顶。'
test('long dialogue cannot fit in a four-second timeline slot even with a long total duration',()=>{
assert.ok(dialogueTimingProblems({shot_number:1,duration:15,video_prompt:`0-4秒:老师说:「${dialogue}」\n4-15秒:大家检查工具。`}).some(s=>s.includes('对白至少需要')))
assert.deepEqual(dialogueTimingProblems({shot_number:1,duration:15,video_prompt:`0-14秒:老师说:「${dialogue}」\n14-15秒:大家点头。`}),[])
})
test('nested quotes counted once and offscreen speech counted as dialogue',()=>{
assert.deepEqual(spokenLiterals('老师(画外):「请回答『准备好了』,然后开门。」'),['请回答『准备好了』,然后开门。'])
assert.deepEqual(spokenLiterals('特写:门牌写着「图书室」。'),[])
})
test('plan retains each scene and paragraph in order',()=>{
const plan=buildStoryboardPlan(`## S01 | 工作室 | 白天\n\n甲检查窗户。\n\n甲:(认真)${dialogue}\n\n## S02 | 屋顶 | 白天\n\n乙走上屋顶。\n\n乙:工具已经准备好了,我们现在可以开始修理了。`)
assert.equal(plan.length,2);assert.ok(plan[0].units.length>=3)
assert.equal(plan[0].units.map(u=>u.dialogue||'').join(''),dialogue)
assert.ok(plan[0].minSeconds>=Math.ceil(dialogue.replace(/[^\p{Script=Han}]/gu,'').length/4.5))
const skipped=sectionBatchProblems(plan[0],[{shot_number:1,duration:12,video_prompt:'0-12秒:修理屋顶。',source_ids:['S02P01']}])
assert.ok(skipped.some(s=>s.includes('不能跳场')))
})
test('ending sentence does not bypass missing source, speech or duration',()=>{
const [section]=buildStoryboardPlan(`## S01 | 工作室 | 白天\n\n甲检查窗户。\n\n甲:${dialogue}\n\n乙:工具已经准备好了,我们现在可以开始修理了。`)
const rushed=[{shot_number:1,duration:12,source_ids:section.units.map(u=>u.id),video_prompt:'0-12秒:乙说:「工具已经准备好了,我们现在可以开始修理了。」'}]
const coverage=sectionCoverage(section,rushed)
assert.equal(coverage.ready,false);assert.ok(coverage.missingDialogue.length)
})
test('dialogue can span shots while all source units and minimum duration remain required',()=>{
const [section]=buildStoryboardPlan(`## S01 | 工作室 | 白天\n\n甲:${dialogue}`)
const [a,b]=dialogue.split('确认')
const shots=[{shot_number:1,duration:12,source_ids:section.units.map(u=>u.id),video_prompt:`0-12秒:甲说:「${a}」`},{shot_number:2,duration:12,source_ids:section.units.map(u=>u.id),video_prompt:`0-12秒:甲说:「确认${b}」`}]
assert.equal(sectionCoverage(section,shots).ready,true)
})
test('adjacent lines without blank separators still inventory dialogue',()=>{
const [section]=buildStoryboardPlan('大家来到工作室。\n老师:'+dialogue)
assert.ok(section.units.length>=3);assert.equal(section.units.map(u=>u.dialogue||'').join(''),dialogue)
assert.equal(sectionCoverage(section,[{shot_number:1,duration:15,source_ids:section.units.map(u=>u.id),video_prompt:'0-15秒:大家检查工具。'}]).ready,false)
})
test('long speech chunks preserve every character and fit their declared speaking budget',()=>{
for(const text of [dialogue.repeat(3),'Please check every window and put the drawings in order before we begin repairing the roof. '.repeat(5),'没有标点的长台词'.repeat(20)]) {
const chunks=splitSpeech(text)
assert.ok(chunks.length>1);assert.equal(chunks.join(''),text)
assert.ok(chunks.every(c=>speechSeconds(c)<=10))
}
})
test('retiming preserves all speech and actions with identical Chinese and English time ranges',()=>{
const cn=`0-2秒:老师打开窗户。\n2-6秒:老师说:「${dialogue}」\n6-9秒:大家点头。`
const en=`0-2s: The teacher opens a window.\n2-6s: The teacher says: 「${dialogue}」\n6-9s: Everyone nods.`
const fixed=fitDraftDialogueTiming({shot_number:1,duration:9,video_prompt:cn,video_prompt_en:en})
assert.deepEqual(dialogueTimingProblems(fixed),[])
assert.ok(fixed.duration<=15&&fixed.duration>9)
const strip=(s:string)=>s.replace(/^\d+-\d+(?:秒:|s:)/gm,'')
assert.equal(strip(fixed.video_prompt),strip(cn));assert.equal(strip(fixed.video_prompt_en),strip(en))
const ranges=(s:string)=>s.match(/^\d+-\d+/gm)
assert.deepEqual(ranges(fixed.video_prompt),ranges(fixed.video_prompt_en))
})
test('retiming never shortens speech to squeeze an oversized line into one shot',()=>{
const shot={shot_number:1,duration:15,video_prompt:`0-15秒:老师说:「${dialogue.repeat(3)}」`}
assert.deepEqual(fitDraftDialogueTiming(shot),shot)
assert.ok(dialogueTimingProblems(shot).length)
})
test('coverage requires repeated speech in original order',()=>{
const [section]=buildStoryboardPlan('甲:准备好了。\n乙:检查窗户。\n甲:准备好了。')
const shot={shot_number:1,duration:15,source_ids:section.units.map(u=>u.id),video_prompt:'0-15秒:甲说:「准备好了。」乙说:「检查窗户。」'}
assert.equal(sectionCoverage(section,[shot]).ready,false)
shot.video_prompt+='甲说:「准备好了。」'
assert.equal(sectionCoverage(section,[shot]).ready,true)
})
@@ -0,0 +1,33 @@
import test from 'node:test'
import assert from 'node:assert/strict'
import { normalizeDraftVoices, normalizeDraftAssets } from '../src/utils/storyboard-generated-prompt.js'
import { storyboardSegmentProblems } from '../src/utils/storyboard-checks.js'
const characters=[{id:1,name:'乔雨灵',voiceAudioUrl:null},{id:2,name:'高天明',voiceAudioUrl:'voice.wav'},
{id:3,name:'林',voiceAudioUrl:null},{id:4,name:'林婉',voiceAudioUrl:'voice2.wav'}]
test('AI drafts do not require voices for known bound characters without uploaded audio',()=>{
assert.equal(normalizeDraftVoices('0-3秒:#乔雨灵音色说:「别过来!」',characters,[1]),'0-3秒:@乔雨灵说:「别过来!」')
})
test('keep real voices, unbound names, and dialogue literals intact',()=>{
const text='0-3秒:#高天明说:「#乔雨灵,别过来!」 #陌生人 #乔雨灵'
assert.equal(normalizeDraftVoices(text,characters,[2]),text)
assert.equal(normalizeDraftVoices('#林婉音色 #林',characters,[3,4]),'#林婉音色 @林')
})
test('spoken mention of blood-red text does not request drawing text',()=>{
const p='0-9秒:@高天明低声念道:「血红文字写着,所有人都得死。」画面中不出现任何文字。'
assert.deepEqual(storyboardSegmentProblems({shot_number:1,duration:9,video_prompt:p,character_ids:[2]},characters),[])
})
test('screen text instructions are still rejected, including quoted instructions',()=>{
for(const p of ['0-9秒:墙面显示血红文字。','0-9秒:屏幕显示:「血红文字」。']) {
assert.ok(storyboardSegmentProblems({shot_number:1,duration:9,video_prompt:p},[]).some(p=>p.includes('画面里出现文字')))
}
})
test('timeline and character binding checks still apply',()=>{
const problems=storyboardSegmentProblems({shot_number:1,duration:12,video_prompt:'0-9秒:@高天明说:「血红文字。」'},characters)
assert.ok(problems.some(p=>p.includes('时间轴')))
assert.ok(problems.some(p=>p.includes('没有绑定')))
})
test('normalize category-prefixed asset references without changing dialogue',()=>{
assert.equal(normalizeDraftAssets('@场景民国八角戏楼·候场台,@角色乔雨灵说:「@场景民国八角戏楼·候场台」',['民国八角戏楼·候场台','乔雨灵']),
'@民国八角戏楼·候场台,@乔雨灵说:「@场景民国八角戏楼·候场台」')
})
@@ -0,0 +1,13 @@
import test from 'node:test'
import assert from 'node:assert/strict'
import {scriptEndingCue,endingShotIndex} from '../src/utils/storyboard-script-ending.js'
test('identify unique long terminal speech and ignore punctuation differences',()=>{
const cue=scriptEndingCue('前情。\n高天明:(低声)乔雨灵,信我一次!按剧本喊出来!剩下的交给老子!')
assert.ok(cue)
assert.equal(endingShotIndex([{description:'前半段'},{video_prompt:'0-9秒:@高天明说:「乔雨灵,信我一次,按剧本喊出来!剩下的交给老子!」'},{description:'不应生成的新剧情'}],cue),1)
})
test('do not prematurely stop on repeated or short dialogue or an action ending',()=>{
assert.equal(scriptEndingCue('高天明:好!'),'')
assert.equal(scriptEndingCue('他离开房间,关上门。'),'')
assert.equal(scriptEndingCue('甲:这句对白前面已经出现了很多次一定不能作为终点。\n乙:这句对白前面已经出现了很多次一定不能作为终点。'),'')
})
@@ -0,0 +1,27 @@
<template>
<section v-if="plan" class="composed-frame" data-testid="composed-first-frame">
<div class="frame-heading"><strong>分镜首帧</strong><span>{{ plan.reusedFromTaskId ? '复用已验证素材清单的首帧' : `${plan.assets?.length || 0} 个绑定素材 · ${plan.rounds} 轮融合` }}</span></div>
<p>图片模型:{{ plan.imageModel }} · {{ plan.imageGpu === 'auto' ? '自动分配显卡' : plan.imageGpu?.toUpperCase() }}</p>
<p>{{ plan.assets?.map(a => a.name).join('、') }}</p>
<button v-if="plan.frameUrl || plan.images?.length" type="button" class="frame-image" @click="$emit('preview', plan.frameUrl || plan.images.at(-1))">
<img :src="src(plan.frameUrl || plan.images.at(-1))" alt="本次视频任务的分镜首帧" />
<span>{{ plan.frameUrl ? '已合成的分镜首帧 · 点击查看' : `第 ${plan.images.length} 轮中间图 · 尚未提交视频` }}</span>
</button>
<p v-else>正在准备完整画面。全部首帧步骤成功后,才会提交视频。</p>
<small>这是最近一次视频任务的素材快照。修改人物、场景或分镜后,下次生成会重新准备首帧。</small>
</section>
</template>
<script setup lang="ts">
const props = defineProps<{ task?: Record<string,any> | null }>()
defineEmits(['preview'])
const plan = computed(() => { try { return JSON.parse(props.task?.params || '{}').firstFrameComposition || null } catch { return null } })
const src = (value:string) => value?.startsWith('static/') ? '/' + value : value
</script>
<style scoped>
.composed-frame { margin:12px 16px; padding:12px; border:1px solid var(--border); border-radius:12px; background:var(--bg-1); font-size:12px; line-height:1.6; }
.frame-heading { display:flex; justify-content:space-between; gap:8px; flex-wrap:wrap; }
.frame-heading span, small { color:var(--text-3); } p { margin:6px 0; overflow-wrap:anywhere; }
.frame-image { display:block; width:100%; padding:0; border:0; background:transparent; cursor:pointer; text-align:center; color:inherit; }
.frame-image img { display:block; width:100%; max-height:230px; object-fit:contain; border-radius:8px; background:#111; }
.frame-image span { display:block; padding:5px; } small { display:block; margin-top:6px; }
</style>
@@ -0,0 +1,60 @@
<template>
<section class="scene-lock-panel" aria-label="场景固定设定">
<div class="heading"><strong>场景固定设定</strong><span>{{ scene?.location || '未绑定场景' }}</span></div>
<p v-if="!scene" class="warning">请在参考素材中绑定场景后再生成。人物参考图不能代替场景背景。</p>
<template v-else>
<p>同一场景的所有分镜共用,生成时自动加入每条请求;换角度、特写也保留同一布景。</p>
<label>空间与固定陈设
<textarea v-model="draft" :disabled="saving" rows="4" maxlength="4000" placeholder="房间结构、门窗和墙面材质、家具与关键物件的相对位置。以场景为坐标描述,不固定所有机位的画面左右。" />
</label>
<p>时间:{{ scene.time || '未填写' }} · 光源:{{ scene.lighting || '未填写,请在场景资产中补充' }}</p>
<p v-if="scene.image_url || scene.imageUrl">已绑定场景图:普通参考模式自动携带;尾帧接续或高清重制使用对应视频帧。</p>
<p v-else class="warning">尚无场景参考图,目前仅用文字约束。上传或生成固定场景图可进一步减少背景漂移。</p>
<p v-if="info?.stale" class="warning">已有任务未使用当前设定。保存后用于下一次生成,已有视频保留。</p>
<p v-else-if="info?.used">最近任务已使用本场景设定{{ info.used.referenceMode === 'scene_image' ? '及场景参考图' : info.used.referenceMode === 'frame' ? '及视频参考帧' : '(文字约束)' }}。</p>
<p class="note">人物动作、机位与对白由分镜决定。剧情中移动道具或开关灯的变化,请写在镜头起止状态里。</p>
<div class="actions"><span>{{ dirty ? '有未保存的场景设定' : '已保存 · 同场景共用' }}</span><button class="btn btn-sm" :disabled="saving || !dirty" @click="save">{{ saving ? '保存中…' : '保存场景设定' }}</button></div>
</template>
<p v-if="error" class="warning" role="alert">{{ error }}</p>
</section>
</template>
<script setup lang="ts">
import { api, sceneAPI } from '~/composables/useApi'
const props = defineProps<{ shot: any; scene?: any }>()
const emit = defineEmits<{ saved: []; dirty: [value: boolean] }>()
const draft = ref(''), saved = ref(''), saving = ref(false), error = ref(''), info = ref<any>(null)
const dirty = computed(() => draft.value !== saved.value)
watch(dirty, value => emit('dirty', value))
async function load() {
const id = props.shot.id
try { const result = await api.get(`/storyboards/${id}/environment`); if (id === props.shot.id) info.value = result }
catch (e: any) { if (id === props.shot.id) error.value = e.message }
}
watch(() => [props.shot.id, props.scene?.id, props.scene?.continuity_prompt ?? props.scene?.continuityPrompt, props.scene?.prompt, props.scene?.lighting, props.scene?.image_url ?? props.scene?.imageUrl], () => {
if (dirty.value || saving.value) return
draft.value = props.scene?.continuity_prompt || props.scene?.continuityPrompt || props.scene?.prompt || ''
saved.value = draft.value; info.value = null; error.value = ''; void load()
}, { immediate: true })
async function save() {
if (!props.scene) return
const value = draft.value, id = props.scene.id
saving.value = true; error.value = ''
try {
await sceneAPI.update(id, { continuity_prompt: value })
saved.value = value; await load(); emit('saved')
} catch (e: any) { error.value = e.message }
finally { saving.value = false }
}
onBeforeUnmount(() => emit('dirty', false))
</script>
<style scoped>
.scene-lock-panel { display:flex; flex-direction:column; gap:10px; border:1px solid var(--border,#ddd); border-radius:14px; padding:16px; margin:12px 0; background:var(--bg-1,#fff); }
.heading,.actions { display:flex; align-items:center; justify-content:space-between; gap:8px; }
.heading span,.actions span,.note { color:var(--text-3,#777); font-size:12px; }
p { margin:0; font-size:12px; line-height:1.6; }
label { display:flex; flex-direction:column; gap:6px; font-size:12px; font-weight:600; }
textarea { width:100%; box-sizing:border-box; resize:vertical; border:1px solid var(--border,#ddd); border-radius:8px; background:var(--bg-0,#fff); color:var(--text-0,#222); padding:9px; font:inherit; font-weight:400; line-height:1.55; }
.warning { color:#a1490a; background:#fff3e8; padding:9px; border-radius:8px; }
</style>
@@ -12,6 +12,8 @@
<p v-if="fresh && progress?.historyMaxSeconds">同显卡、同规格通常耗时约 {{ timeRange(progress.historyMinSeconds, progress.historyMaxSeconds) }}<small>参考 {{ progress.historySamples }} 次成功执行,不含排队与下载;属于估算。</small></p> <p v-if="fresh && progress?.historyMaxSeconds">同显卡、同规格通常耗时约 {{ timeRange(progress.historyMinSeconds, progress.historyMaxSeconds) }}<small>参考 {{ progress.historySamples }} 次成功执行,不含排队与下载;属于估算。</small></p>
<p v-else-if="!waiting && !finishing">整段视频预计耗时:暂无足够的同规格记录。</p> <p v-else-if="!waiting && !finishing">整段视频预计耗时:暂无足够的同规格记录。</p>
<p v-if="finishing">视频生成已结束,正在回传并保存到本地。</p> <p v-if="finishing">视频生成已结束,正在回传并保存到本地。</p>
<p v-else-if="task?.stage === 'preparing_first_frame'">正在融合全部绑定素材生成完整首帧,完成后自动提交视频。</p>
<p v-else-if="task?.stage === 'first_frame_ready'">首帧已保存,正在准备视频请求。</p>
<p v-else-if="task?.stage === 'cancelling'">已请求停止此任务,正在等待显卡确认。</p> <p v-else-if="task?.stage === 'cancelling'">已请求停止此任务,正在等待显卡确认。</p>
<p v-else-if="!fresh && !waiting" class="progress-note">{{ progress ? '进度更新暂时延迟,仍在跟踪原任务。' : '正在读取显卡进度…' }}</p> <p v-else-if="!fresh && !waiting" class="progress-note">{{ progress ? '进度更新暂时延迟,仍在跟踪原任务。' : '正在读取显卡进度…' }}</p>
</section> </section>
@@ -30,10 +32,10 @@ const progress = computed(() => {
const finishing = computed(() => ['transferring', 'downloading', 'saving'].includes(props.task?.stage)) const finishing = computed(() => ['transferring', 'downloading', 'saving'].includes(props.task?.stage))
const fresh = computed(() => !finishing.value && props.task?.stage !== 'cancelling' && progress.value && clock.value - progress.value.observedAt < 90000) const fresh = computed(() => !finishing.value && props.task?.stage !== 'cancelling' && progress.value && clock.value - progress.value.observedAt < 90000)
const samplingFresh = computed(() => fresh.value && clock.value - progress.value.observedAt < 30000) const samplingFresh = computed(() => fresh.value && clock.value - progress.value.observedAt < 30000)
const waiting = computed(() => ['waiting_predecessor', 'blocked_predecessor', 'cancelling'].includes(props.task?.stage)) const waiting = computed(() => ['waiting_predecessor', 'blocked_predecessor', 'cancelling', 'preparing_first_frame', 'first_frame_ready'].includes(props.task?.stage))
const phases: Record<string, string> = { queued:'等待显卡队列', running:'显卡正在处理', sampling:'视频采样中', decoding:'视频 / 音频解码中', const phases: Record<string, string> = { queued:'等待显卡队列', running:'显卡正在处理', sampling:'视频采样中', decoding:'视频 / 音频解码中',
encoding:'编码与保存中', loading:'模型 / 素材加载中', conditioning:'参考素材与提示词处理', postprocessing:'声音 / 字幕后处理', encoding:'编码与保存中', loading:'模型 / 素材加载中', conditioning:'参考素材与提示词处理', postprocessing:'声音 / 字幕后处理',
waiting_predecessor:'等待前镜头完成', blocked_predecessor:'等待处理前镜头', cancelling:'正在确认取消', transferring:'视频回传中', downloading:'视频下载中', saving:'保存视频中', submitting:'提交任务中' } preparing_first_frame:'第 1 步 · 生成完整分镜首帧', first_frame_ready:'第 2 步 · 提交视频', waiting_predecessor:'等待前镜头完成', blocked_predecessor:'等待处理前镜头', cancelling:'正在确认取消', transferring:'视频回传中', downloading:'视频下载中', saving:'保存视频中', submitting:'提交任务中' }
const phaseLabel = computed(() => { const phaseLabel = computed(() => {
const stage = props.task?.stage const stage = props.task?.stage
return phases[waiting.value || ['transferring','downloading','saving','submitting'].includes(stage) ? stage : fresh.value ? progress.value.phase : stage] || '生成任务处理中' return phases[waiting.value || ['transferring','downloading','saving','submitting'].includes(stage) ? stage : fresh.value ? progress.value.phase : stage] || '生成任务处理中'
@@ -59,6 +59,7 @@ export const episodeAPI = {
} }
export const storyboardAPI = { export const storyboardAPI = {
deleteBatch: (episodeId: number, ids: number[]) => api.post('/storyboards/batch-delete', { episode_id: episodeId, storyboard_ids: ids }),
englishPrompt: (id: number, data?: any) => api.post(`/storyboards/${id}/english-prompt`, data || {}), englishPrompt: (id: number, data?: any) => api.post(`/storyboards/${id}/english-prompt`, data || {}),
create: (data: any) => api.post('/storyboards', data), create: (data: any) => api.post('/storyboards', data),
update: (id: number, data: any) => api.put(`/storyboards/${id}`, data), update: (id: number, data: any) => api.put(`/storyboards/${id}`, data),
@@ -0,0 +1,55 @@
import { computed, onBeforeUnmount, ref, watch, type Ref } from 'vue'
import { toast } from 'vue-sonner'
export function useStoryboardJob(episodeId:Ref<number>,onDone:()=>Promise<void>) {
const job=ref<any>(null),starting=ref(false),connectionError=ref(''),clock=ref(Date.now())
let timer:ReturnType<typeof setTimeout>|undefined,disposed=false,epoch=0
const running=computed(()=>starting.value||job.value?.status==='running')
const elapsed=computed(()=>{
if (!job.value?.startedAt) return ''
const seconds=Math.max(0,Math.round(((job.value.finishedAt?Date.parse(job.value.finishedAt):clock.value)-Date.parse(job.value.startedAt))/1000))
return `${Math.floor(seconds/60)} 分 ${seconds%60} 秒`
})
async function request(path:string,body?:any) {
const response=await fetch('/api/v1/agent/storyboard_breaker/jobs'+path,{method:body?'POST':'GET',headers:{'Content-Type':'application/json'},body:body?JSON.stringify(body):undefined,signal:AbortSignal.timeout(12000)})
const json=await response.json()
if (!response.ok||(json.code&&json.code>=400)) throw new Error(json.message||'无法连接分镜任务')
return Object.prototype.hasOwnProperty.call(json,'data')?json.data:json
}
function schedule() {
if (timer) clearTimeout(timer)
if (!disposed&&job.value?.status==='running') timer=setTimeout(()=>void load(),2000)
}
async function load() {
if (!episodeId.value||disposed) return
const id=episodeId.value,token=epoch
try {
const state=await request('/'+id)
if (disposed||token!==epoch||id!==episodeId.value) return
const finished=job.value?.status==='running'&&state?.id===job.value?.id&&state?.status!=='running'
job.value=state;clock.value=Date.now();connectionError.value=''
if (finished) {
if (state.status==='done') { toast.success(`分镜拆分完成:${state.completed} 个`);await onDone() }
else if (state.status==='error') toast.error(state.error||'分镜拆分未完成')
}
} catch { if (token===epoch) connectionError.value='暂时无法获取进度,正在重连;请勿重复提交。' }
finally { schedule() }
}
async function start(message:string,dramaId:number,model?:string,configId?:number,resumeFrom?:string) {
if (running.value) return
starting.value=true
try {
job.value=await request('',{message,drama_id:dramaId,episode_id:episodeId.value,model,config_id:configId,resume_from:resumeFrom||undefined})
clock.value=Date.now();connectionError.value='';schedule()
} catch(error:any) { toast.error(error.message||'拆分启动失败');await load() }
finally { starting.value=false }
}
async function cancel() {
if (!job.value||job.value.status!=='running') return
try { job.value=await request('/'+episodeId.value+'/cancel',{id:job.value.id});connectionError.value='';if(timer)clearTimeout(timer) }
catch(error:any) { toast.error(error.message||'取消失败,请重试') }
}
watch(episodeId,()=>{epoch++;if(timer)clearTimeout(timer);job.value=null;connectionError.value='';void load()},{immediate:true})
onBeforeUnmount(()=>{disposed=true;epoch++;if(timer)clearTimeout(timer)})
return {job,running,elapsed,connectionError,start,cancel}
}
@@ -9,6 +9,7 @@
*/ */
import { toast } from 'vue-sonner' import { toast } from 'vue-sonner'
import { i18n } from '~/composables/i18n' import { i18n } from '~/composables/i18n'
import { videoReferenceError } from '~/utils/video-reference-error'
const t = (key: string, params?: Record<string, unknown>) => i18n.global.t(key, params) const t = (key: string, params?: Record<string, unknown>) => i18n.global.t(key, params)
@@ -40,6 +41,9 @@ function messageOf(err: unknown): string {
export function mapError(err: unknown, opts?: { fallback?: string }): string { export function mapError(err: unknown, opts?: { fallback?: string }): string {
const msg = messageOf(err).trim() const msg = messageOf(err).trim()
if (!msg) return opts?.fallback ? t(opts.fallback) : t('errors.unknown') if (!msg) return opts?.fallback ? t(opts.fallback) : t('errors.unknown')
const referenceError = videoReferenceError(msg)
if (referenceError) return referenceError
if (msg.startsWith('首帧合成:') && !/[{}<>]|https?:\/\//i.test(msg)) return msg.slice(0,220)
// Application-authored continuity errors contain actionable model/shot guidance. // Application-authored continuity errors contain actionable model/shot guidance.
if (/^(尾帧接续请选择|当前尾帧接续工作流|镜头衔接设置或前镜头|前镜头版本已变化|衔接来源必须|尾帧接续仅用于|没有可承接的前镜头)/.test(msg) && msg.length < 180 && !/https?:\/\//i.test(msg)) return msg if (/^(尾帧接续请选择|当前尾帧接续工作流|镜头衔接设置或前镜头|前镜头版本已变化|衔接来源必须|尾帧接续仅用于|没有可承接的前镜头)/.test(msg) && msg.length < 180 && !/https?:\/\//i.test(msg)) return msg
@@ -0,0 +1,8 @@
// Known model limits can be explained without displaying arbitrary upstream JSON.
export function videoReferenceError(message) {
const upstream = message.match(/\b(LTX-2\.3(?:-Text-Preserve)?|Wan2\.2-I2V) requires (?:0-1|1-1|1) images\b/)
if (upstream) return `${upstream[1]} 最多支持 1 张参考图;请按当前模型重新提交,或选择支持多图的 MiniMax-H3`
if (/^(?:(?:LTX-2\.3(?:-Text-Preserve)?|Wan2\.2-I2V|MiniMax-H3(?:-Hybrid|-R2V-4Step|-VDN-H3)?) (?:最多支持|需要至少)|所选模型最多支持|本地视频工作流暂不支持|首帧模式只能提交)/.test(message)
&& message.length < 180 && !/[{}<>]|https?:\/\//i.test(message)) return message
return ''
}
@@ -0,0 +1,34 @@
// Resolve the effective per-shot model, never the episode default by accident.
export function referenceImageLimit(model = '', provider = '') {
if (['LTX-2.3', 'LTX-2.3-Text-Preserve', 'Wan2.2-I2V'].includes(model)) return 1
return provider === 'aliyun' || model.startsWith('wan3.0-video') ? 10 : 9
}
export function usesComposedFirstFrame(model = '') {
return ['LTX-2.3', 'LTX-2.3-Text-Preserve', 'Wan2.2-I2V'].includes(model)
}
export function selectVideoReferences(assets, limit) {
const seen = new Set()
const selected = []
for (const asset of assets) {
if (!asset.url || seen.has(asset.url) || selected.length >= limit) continue
seen.add(asset.url)
selected.push(asset)
}
return selected
}
export function resolveReferenceMentions(prompt, assets, boundAssets = assets) {
const names = [...new Set(boundAssets.map(x => x.name).filter(Boolean))].sort((a,b) => b.length - a.length)
const escape = s => s.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')
// Match the known name before stripping a stale index: 404宿舍 starts with digits.
return prompt.replace(/@([^\s@#]+)/g, (match, raw) => {
for (const name of names) {
const numbered = raw.match(new RegExp(`^图片\\d+?${escape(name)}`))
if (!raw.startsWith(name) && !numbered) continue
const index = assets.findIndex(x => x.name === name)
return `@${index >= 0 ? `图片${index + 1}` : ''}${name}${raw.slice(numbered ? numbered[0].length : name.length)}`
}
return match
})
}
@@ -492,8 +492,8 @@
<span class="tag mono">{{ t('episode.sb.segmentStat', { n: sbs.length, dur: totalDuration }) }}</span> <span class="tag mono">{{ t('episode.sb.segmentStat', { n: sbs.length, dur: totalDuration }) }}</span>
<span class="tag mono" :title="t('episode.vid.aspectRatio')">{{ dramaAspectRatio }}</span> <span class="tag mono" :title="t('episode.vid.aspectRatio')">{{ dramaAspectRatio }}</span>
<div class="ml-auto flex gap-1"> <div class="ml-auto flex gap-1">
<button class="btn btn-sm" :disabled="rn" @click="doBreakdown"> <button class="btn btn-sm" :disabled="rn || breakdownRunning" @click="doBreakdown">
<Loader2 v-if="rt === 'storyboard_breaker'" :size="11" class="animate-spin" /> <Loader2 v-if="breakdownRunning" :size="11" class="animate-spin" />
<svg v-else width="11" height="11" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round"><polygon points="13 2 3 14 12 14 11 22 21 10 12 10 13 2"/></svg> <svg v-else width="11" height="11" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round"><polygon points="13 2 3 14 12 14 11 22 21 10 12 10 13 2"/></svg>
{{ sbs.length ? t('episode.sb.rebreak') : t('episode.sb.startBreak') }} {{ sbs.length ? t('episode.sb.rebreak') : t('episode.sb.startBreak') }}
</button> </button>
@@ -516,15 +516,28 @@
</button> </button>
</div> </div>
</div> </div>
<div v-if="breakdownJob" class="storyboard-job-status" data-testid="storyboard-job-status" role="status">
<div class="flex items-center gap-2"><strong>{{ breakdownJob.phase }}</strong><span class="dim">{{ breakdownElapsed }}</span><button v-if="breakdownRunning" class="btn btn-sm ml-auto" @click="cancelBreakdown" data-testid="cancel-storyboard-job">取消拆分</button></div>
<p>{{ breakdownJob.status === 'done' ? '已保存' : '已暂存' }} {{ breakdownJob.completed || 0 }} 个分镜<span v-if="breakdownJob.target">,预计约 {{ breakdownJob.target }} 个</span> · {{ breakdownJob.seconds || 0 }} 秒<span v-if="breakdownJob.sectionCount"> · 已完成 {{ breakdownJob.sectionIndex || 0 }}/{{ breakdownJob.sectionCount }} 场</span></p>
<p v-if="breakdownConnectionError" class="storyboard-job-error">{{ breakdownConnectionError }}</p>
<p v-else-if="breakdownJob.error" class="storyboard-job-error">{{ breakdownJob.error }}</p>
<p v-else-if="breakdownJob.qualityWarning" class="storyboard-job-error">{{ breakdownJob.qualityWarning }}</p>
<p v-else-if="breakdownJob.lastValidation" class="storyboard-job-error">{{ breakdownJob.lastValidation }}</p>
<p v-else-if="breakdownRunning" class="dim">每批最多 4 个分镜,中英文一起校验;全部完成后统一保存。可刷新页面继续查看进度。</p>
<div v-if="breakdownJob.canResume && !breakdownRunning" class="flex items-center gap-2 flex-wrap mt-2">
<button class="btn btn-primary btn-sm" data-testid="resume-storyboard-job" @click="doBreakdown(breakdownJob.id)">继续未完成拆分</button>
<span class="dim">复用 {{ breakdownJob.reusableShots || 0 }} 个已完成场次的分镜,从第 {{ breakdownJob.resumeFromSection }} 场重试。草稿会在整集完成后进入列表。</span>
</div>
</div>
<div v-if="!sbs.length" class="step-empty video-task-empty-state"> <div v-if="!sbs.length" class="step-empty video-task-empty-state">
<div class="empty-visual"> <div class="empty-visual">
<svg width="32" height="32" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.3" stroke-linecap="round"><rect x="2" y="2" width="20" height="20" rx="2.5"/><line x1="7" y1="8" x2="7" y2="16"/><line x1="10" y1="8" x2="10" y2="16"/><line x1="13" y1="8" x2="13" y2="16"/></svg> <svg width="32" height="32" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.3" stroke-linecap="round"><rect x="2" y="2" width="20" height="20" rx="2.5"/><line x1="7" y1="8" x2="7" y2="16"/><line x1="10" y1="8" x2="10" y2="16"/><line x1="13" y1="8" x2="13" y2="16"/></svg>
</div> </div>
<div class="empty-title">{{ t('episode.sb.emptyTitle') }}</div> <div class="empty-title">{{ breakdownRunning ? '正在拆分分镜' : t('episode.sb.emptyTitle') }}</div>
<div class="empty-desc">{{ t('episode.sb.emptyDesc') }}</div> <div class="empty-desc">{{ t('episode.sb.emptyDesc') }}</div>
<div class="locked-config-banner">{{ t('episode.vid.lockedModel') }}{{ effectiveVideoModelLabel }}</div> <div class="locked-config-banner">{{ t('episode.vid.lockedModel') }}{{ effectiveVideoModelLabel }}</div>
<button class="btn btn-primary" :disabled="rn" @click="doBreakdown"> <button class="btn btn-primary" :disabled="rn || breakdownRunning" @click="doBreakdown">
<Loader2 v-if="rt === 'storyboard_breaker'" :size="13" class="animate-spin" /> <Loader2 v-if="breakdownRunning" :size="13" class="animate-spin" />
<svg v-else width="13" height="13" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.2" stroke-linecap="round"><polygon points="13 2 3 14 12 14 11 22 21 10 12 10 13 2"/></svg> <svg v-else width="13" height="13" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.2" stroke-linecap="round"><polygon points="13 2 3 14 12 14 11 22 21 10 12 10 13 2"/></svg>
{{ t('episode.sb.startBreak') }} {{ t('episode.sb.startBreak') }}
</button> </button>
@@ -546,17 +559,19 @@
<button class="shot-quick-btn" @click="toggleSelectAllVideos">{{ t('episode.sb.selectAll') }}</button> <button class="shot-quick-btn" @click="toggleSelectAllVideos">{{ t('episode.sb.selectAll') }}</button>
<button class="shot-quick-btn" @click="selectMissingVideos">{{ t('episode.vid.selectMissing') }}</button> <button class="shot-quick-btn" @click="selectMissingVideos">{{ t('episode.vid.selectMissing') }}</button>
<button class="shot-quick-btn" @click="selectedVideoSbIds = []">{{ t('episode.sb.clear') }}</button> <button class="shot-quick-btn" @click="selectedVideoSbIds = []">{{ t('episode.sb.clear') }}</button>
<button class="shot-quick-btn storyboard-delete-action" data-testid="delete-selected-storyboards" :disabled="!selectedVideoSbIds.length || storyboardDelete.loading || breakdownRunning" @click="askDeleteStoryboards(selectedVideoSbIds)"><Trash2 :size="12" />删除所选({{ selectedVideoSbIds.length }})</button>
</div> </div>
<div class="video-task-table"> <div class="video-task-table">
<div <div
v-for="task in videoTaskRows" v-for="task in videoTaskRows"
:key="task.id" :key="task.id"
:data-storyboard-id="task.id"
:class="['video-task-row', 'is-' + videoTaskState(task.storyboard), { active: !videoSelectMode && selectedSb?.id === task.storyboard.id, 'is-selected': videoSelectMode && isVideoSbSelected(task.id) }]" :class="['video-task-row', 'is-' + videoTaskState(task.storyboard), { active: !videoSelectMode && selectedSb?.id === task.storyboard.id, 'is-selected': videoSelectMode && isVideoSbSelected(task.id) }]"
role="button" role="button"
tabindex="0" tabindex="0"
@click="onVideoTaskRowClick(task.storyboard)" @click="onVideoTaskRowClick(task.storyboard)"
@keydown.enter.prevent="onVideoTaskRowClick(task.storyboard)" @keydown.enter.self.prevent="onVideoTaskRowClick(task.storyboard)"
@keydown.space.prevent="onVideoTaskRowClick(task.storyboard)" @keydown.space.self.prevent="onVideoTaskRowClick(task.storyboard)"
> >
<div class="video-task-preview"> <div class="video-task-preview">
<span <span
@@ -578,7 +593,7 @@
<div v-else class="video-task-empty"> <div v-else class="video-task-empty">
<svg width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.3" stroke-linecap="round"><polygon points="23 7 16 12 23 17 23 7"/><rect x="1" y="5" width="15" height="14" rx="2" ry="2"/></svg> <svg width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.3" stroke-linecap="round"><polygon points="23 7 16 12 23 17 23 7"/><rect x="1" y="5" width="15" height="14" rx="2" ry="2"/></svg>
</div> </div>
<span class="video-task-index">#{{ String(task.index + 1).padStart(2, '0') }}</span> <span class="video-task-index">#{{ String(shotNumber(task.storyboard)).padStart(2, '0') }}</span>
</div> </div>
<div class="video-task-main"> <div class="video-task-main">
<div class="video-task-line"> <div class="video-task-line">
@@ -605,6 +620,7 @@
<div v-if="videoModerationHint(task.error)" class="video-task-error-hint">{{ videoModerationHint(task.error) }}</div> <div v-if="videoModerationHint(task.error)" class="video-task-error-hint">{{ videoModerationHint(task.error) }}</div>
</div> </div>
</div> </div>
<div class="video-task-actions">
<button <button
class="btn btn-icon btn-sm video-task-action" class="btn btn-icon btn-sm video-task-action"
:title="videoTaskActionLabel(task.storyboard)" :title="videoTaskActionLabel(task.storyboard)"
@@ -613,6 +629,8 @@
> >
<svg width="12" height="12" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round"><polygon points="23 7 16 12 23 17 23 7"/><rect x="1" y="5" width="15" height="14" rx="2" ry="2"/></svg> <svg width="12" height="12" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round"><polygon points="23 7 16 12 23 17 23 7"/><rect x="1" y="5" width="15" height="14" rx="2" ry="2"/></svg>
</button> </button>
<button type="button" class="btn btn-icon btn-sm storyboard-delete-action" :aria-label="`删除分镜 ${shotNumber(task.storyboard)}`" title="删除分镜" :disabled="storyboardDelete.loading || breakdownRunning" @click.stop="askDeleteStoryboards([task.id])"><Trash2 :size="13" /></button>
</div>
</div> </div>
</div> </div>
</section> </section>
@@ -674,6 +692,7 @@
</section> </section>
</div> </div>
<SceneContinuity v-if="selectedSb" :key="`scene-${selectedSb.id}-${selectedSb.scene_id || selectedSb.sceneId || 0}`" :shot="selectedSb" :scene="getStoryboardScene(selectedSb)" @saved="refresh" @dirty="sceneLockDirty = $event" />
<ShotContinuity v-if="selectedSb" :key="selectedSb.id" :shot="selectedSb" :task="latestVideoTask(selectedSb.id)" @saved="refresh" @dirty="continuityDirty = $event" /> <ShotContinuity v-if="selectedSb" :key="selectedSb.id" :shot="selectedSb" :task="latestVideoTask(selectedSb.id)" @saved="refresh" @dirty="continuityDirty = $event" />
<section class="video-inspector-section"> <section class="video-inspector-section">
<div class="video-inspector-prompt-head"> <div class="video-inspector-prompt-head">
@@ -769,6 +788,7 @@
</aside> </aside>
<TaskProgress :task="latestVideoTask(selectedSb?.id)" /> <TaskProgress :task="latestVideoTask(selectedSb?.id)" />
<ComposedFirstFrame :task="latestVideoTask(selectedSb?.id)" @preview="url => openImageViewer(assetImageSrc({ imageUrl:url }), '分镜首帧')" />
<div v-if="sbVideoHistory.length" class="video-player-history"> <div v-if="sbVideoHistory.length" class="video-player-history">
<div class="video-player-history-head"> <div class="video-player-history-head">
<span>{{ t('episode.vid.history') }}</span> <span>{{ t('episode.vid.history') }}</span>
@@ -819,6 +839,9 @@
<div class="video-inspector-settings"> <div class="video-inspector-settings">
<p class="video-param-hint" style="margin-bottom:8px">仅设置当前镜头,其他镜头和批量默认不变。</p> <p class="video-param-hint" style="margin-bottom:8px">仅设置当前镜头,其他镜头和批量默认不变。</p>
<ModelSelect v-model="shotVideoModel" label="当前镜头模型" :options="videoModelOptions" :show-config="true" default-label="跟随顶部视频默认" /> <ModelSelect v-model="shotVideoModel" label="当前镜头模型" :options="videoModelOptions" :show-config="true" default-label="跟随顶部视频默认" />
<p v-if="shotReferenceHint" class="video-param-hint" style="margin:8px 0" data-testid="shot-reference-limit">{{ shotReferenceHint }}</p>
<button v-if="selectedShotComposesFrame && taskParams(latestVideoTask(selectedSb.id)).firstFrameComposition?.frameUrl" class="btn btn-sm" type="button"
:disabled="videoSubmittingIds.includes(selectedSb.id) || isPendingVideo(selectedSb.id)" @click="genVid(selectedSb, { forceNew:true, regenerateFirstFrame:true })">重做首帧并生成视频</button>
<p class="video-param-hint" style="margin:8px 0" data-testid="shot-actual-routing">{{ videoTaskRoutingLabel(latestVideoTask(selectedSb.id)) }}</p> <p class="video-param-hint" style="margin:8px 0" data-testid="shot-actual-routing">{{ videoTaskRoutingLabel(latestVideoTask(selectedSb.id)) }}</p>
<p v-if="/^MiniMax-H3/i.test(bareModelName(modelForShot(selectedSb.id)) || '')" class="video-param-hint">生成时使用英文画面描述,中文对白保持原文。</p> <p v-if="/^MiniMax-H3/i.test(bareModelName(modelForShot(selectedSb.id)) || '')" class="video-param-hint">生成时使用英文画面描述,中文对白保持原文。</p>
<label v-if="supportsVideoGpu" class="video-param-row" style="margin-bottom:10px"> <label v-if="supportsVideoGpu" class="video-param-row" style="margin-bottom:10px">
@@ -1104,6 +1127,10 @@
<span>#{{ row.id }}</span> <span>#{{ row.id }}</span>
</div> </div>
<TaskProgress v-if="row.kind === 'video'" :task="row.sourceTask" /> <TaskProgress v-if="row.kind === 'video'" :task="row.sourceTask" />
<p v-if="row.kind === 'video' && taskParams(row.sourceTask).remoteCleanup" class="video-param-hint" data-testid="video-remote-cleanup">
{{ taskParams(row.sourceTask).remoteCleanup.status === 'cleaned' ? '视频已保存到本地 · ComfyUI 对应任务记录和视频源文件已清理' : '视频已保存到本地 · 正在确认清理 ComfyUI 源文件' }}
<span v-if="taskParams(row.sourceTask).remoteCleanup.lastError">(暂未清理,将自动重试)</span>
</p>
<div v-if="row.canCancel" class="task-cancel-actions"> <div v-if="row.canCancel" class="task-cancel-actions">
<button class="btn btn-sm" :disabled="cancellingTaskIds.includes(row.id)" @click.stop="cancelTask(row)"> <button class="btn btn-sm" :disabled="cancellingTaskIds.includes(row.id)" @click.stop="cancelTask(row)">
{{ cancellingTaskIds.includes(row.id) ? '正在确认取消…' : row.stage === 'cancelling' ? '重试取消任务' : '取消任务' }} {{ cancellingTaskIds.includes(row.id) ? '正在确认取消…' : row.stage === 'cancelling' ? '重试取消任务' : '取消任务' }}
@@ -1474,6 +1501,15 @@
</div> </div>
</div> </div>
<ConfirmDialog
:open="storyboardDelete.open"
:title="`删除 ${storyboardDelete.ids.length} 个分镜?`"
:message="storyboardDelete.message"
:loading="storyboardDelete.loading"
confirm-text="确认删除"
@confirm="confirmDeleteStoryboards"
@cancel="!storyboardDelete.loading && (storyboardDelete.open = false)"
/>
<ConfirmDialog <ConfirmDialog
:open="assetDelete.open" :open="assetDelete.open"
:title="t('episode.delete.title', { type: assetDeleteTypeLabel })" :title="t('episode.delete.title', { type: assetDeleteTypeLabel })"
@@ -1492,9 +1528,10 @@ import { toast } from 'vue-sonner'
import { useI18n } from 'vue-i18n' import { useI18n } from 'vue-i18n'
import { import {
Users, FileText, FolderKanban, Clapperboard, Download, Loader2, Users, FileText, FolderKanban, Clapperboard, Download, Loader2,
Plus, X, ListTodo, CircleHelp, Plus, X, ListTodo, CircleHelp, Trash2,
} from 'lucide-vue-next' } from 'lucide-vue-next'
import { api, dramaAPI, episodeAPI, storyboardAPI, characterAPI, sceneAPI, propAPI, taskAPI, mergeAPI, aiConfigAPI, uploadAPI } from '~/composables/useApi' import { api, dramaAPI, episodeAPI, storyboardAPI, characterAPI, sceneAPI, propAPI, taskAPI, mergeAPI, aiConfigAPI, uploadAPI } from '~/composables/useApi'
import { referenceImageLimit, selectVideoReferences, resolveReferenceMentions, usesComposedFirstFrame } from '~/utils/video-references'
import { startTour, autoTour } from '~/composables/useTour' import { startTour, autoTour } from '~/composables/useTour'
import { useAgent } from '~/composables/useAgent' import { useAgent } from '~/composables/useAgent'
import { resolveTextModelSelection } from '~/utils/text-model-selection' import { resolveTextModelSelection } from '~/utils/text-model-selection'
@@ -1509,6 +1546,7 @@ const route = useRoute()
const dramaId = Number(route.params.id) const dramaId = Number(route.params.id)
const episodeNumber = Number(route.params.episodeNumber) const episodeNumber = Number(route.params.episodeNumber)
const continuityDirty = ref(false) const continuityDirty = ref(false)
const sceneLockDirty = ref(false)
const drama = ref(null), episode = ref(null), chars = ref([]), scenes = ref([]), propItems = ref([]), sbs = ref([]), mergeData = ref(null) const drama = ref(null), episode = ref(null), chars = ref([]), scenes = ref([]), propItems = ref([]), sbs = ref([]), mergeData = ref(null)
// 工作台面板位置记忆(按剧集隔离):仅页面刷新(reload)时恢复到上次所在步骤; // 工作台面板位置记忆(按剧集隔离):仅页面刷新(reload)时恢复到上次所在步骤;
@@ -1530,6 +1568,7 @@ const localRaw = ref(''), localScript = ref('')
const rawContent = computed(() => episode.value?.content || '') const rawContent = computed(() => episode.value?.content || '')
const scriptContent = computed(() => episode.value?.script_content || episode.value?.scriptContent || '') const scriptContent = computed(() => episode.value?.script_content || episode.value?.scriptContent || '')
const epId = computed(() => episode.value?.id || 0) const epId = computed(() => episode.value?.id || 0)
const {job:breakdownJob,running:breakdownRunning,elapsed:breakdownElapsed,connectionError:breakdownConnectionError,start:startBreakdown,cancel:cancelBreakdown}=useStoryboardJob(epId,onBreakdownDone)
const rawLen = computed(() => localRaw.value.replace(/\s/g, '').length || 0) const rawLen = computed(() => localRaw.value.replace(/\s/g, '').length || 0)
const scriptLen = computed(() => localScript.value.replace(/\s/g, '').length || 0) const scriptLen = computed(() => localScript.value.replace(/\s/g, '').length || 0)
@@ -1678,8 +1717,7 @@ const imageViewer = ref({ open: false, src: '', title: '' })
const activeMerge = ref(null) // 成片大预览弹窗中正在播放的拼接记录 const activeMerge = ref(null) // 成片大预览弹窗中正在播放的拼接记录
const previewShot = ref(null) // 导出页镜头素材预览弹窗中正在播放的分镜 const previewShot = ref(null) // 导出页镜头素材预览弹窗中正在播放的分镜
function shotNumberOf(sb) { function shotNumberOf(sb) {
const i = sbs.value.findIndex(s => s.id === sb?.id) return sb ? shotNumber(sb) : 0
return i >= 0 ? i + 1 : 0
} }
// 导出步骤完成 = 用户手动标记(episodes.status = 'completed'),不再按最新拼接记录推算 // 导出步骤完成 = 用户手动标记(episodes.status = 'completed'),不再按最新拼接记录推算
const exportDone = computed(() => episode.value?.status === 'completed') const exportDone = computed(() => episode.value?.status === 'completed')
@@ -2119,7 +2157,7 @@ const allVideoTaskRows = computed(() => sbs.value.map((sb, index) => {
id: sb.id, id: sb.id,
index, index,
storyboard: sb, storyboard: sb,
title: sb.description || t('episode.vid.shotN', { n: String(index + 1).padStart(2, '0') }), title: sb.description || t('episode.vid.shotN', { n: String(shotNumber(sb)).padStart(2, '0') }),
meta: sceneName, meta: sceneName,
duration: Number.isFinite(duration) ? duration : 5, duration: Number.isFinite(duration) ? duration : 5,
referenceCount, referenceCount,
@@ -2143,6 +2181,36 @@ function toggleVideoFilter(state) {
const videoSelectMode = ref(false) const videoSelectMode = ref(false)
const selectedVideoSbIds = ref([]) const selectedVideoSbIds = ref([])
const batchVideoConfirm = ref({ open: false, targets: [] }) const batchVideoConfirm = ref({ open: false, targets: [] })
const storyboardDelete = ref({ open: false, ids: [], message: '', loading: false })
function shotNumber(sb) { return sb?.storyboard_number ?? sb?.storyboardNumber ?? sbs.value.indexOf(sb) + 1 }
function askDeleteStoryboards(ids) {
if (storyboardDelete.value.loading || breakdownRunning.value) return
if (continuityDirty.value || sceneLockDirty.value) { toast.error('请先保存当前镜头的衔接和场景设定'); return }
const selected = sbs.value.filter(sb => ids.includes(sb.id))
if (!selected.length) return
const numbers = selected.map(sb => `#${String(shotNumber(sb)).padStart(2, '0')}`)
storyboardDelete.value = { open: true, ids: selected.map(sb => sb.id), loading: false,
message: `将删除分镜 ${numbers.slice(0, 12).join('、')}${numbers.length > 12 ? ` 等 ${numbers.length} 个` : ''},从制作和拼接列表移除。已生成视频、任务历史、人物和场景素材保留。其他镜头若引用它们衔接,需要重新选择来源;正在生成的分镜请先取消任务。` }
}
async function confirmDeleteStoryboards() {
if (!storyboardDelete.value.open || storyboardDelete.value.loading) return
const ids = [...storyboardDelete.value.ids], previousIndex = sbs.value.findIndex(sb => sb.id === selectedSb.value?.id)
storyboardDelete.value.loading = true
try {
await Promise.all(ids.map(id => shotSaves.get(id)))
const result = await storyboardAPI.deleteBatch(epId.value, ids)
const removed = new Set(ids)
sbs.value = sbs.value.filter(sb => !removed.has(sb.id))
selectedVideoSbIds.value = selectedVideoSbIds.value.filter(id => !removed.has(id))
exportSelectedIds.value = exportSelectedIds.value.filter(id => !removed.has(id))
batchVideoConfirm.value = { open: false, targets: [] }
if (removed.has(selectedSb.value?.id)) selectedSb.value = sbs.value[Math.max(0, Math.min(previousIndex, sbs.value.length - 1))] || null
storyboardDelete.value.open = false
toast.success(`已删除 ${result.count} 个分镜,视频和素材已保留`)
await refresh()
} catch (error) { toast.error(error?.message || '删除失败,请重试') }
finally { storyboardDelete.value.loading = false }
}
function isVideoSbSelected(id) { return selectedVideoSbIds.value.includes(id) } function isVideoSbSelected(id) { return selectedVideoSbIds.value.includes(id) }
function toggleVideoSbSelect(id) { function toggleVideoSbSelect(id) {
@@ -2155,7 +2223,7 @@ function toggleVideoSelectMode() {
if (!videoSelectMode.value) selectedVideoSbIds.value = [] if (!videoSelectMode.value) selectedVideoSbIds.value = []
} }
function onVideoTaskRowClick(sb) { function onVideoTaskRowClick(sb) {
if (continuityDirty.value && selectedSb.value?.id !== sb.id) { toast.error('请先保存当前镜头的衔接设置'); return } if ((continuityDirty.value || sceneLockDirty.value) && selectedSb.value?.id !== sb.id) { toast.error('请先保存当前镜头的衔接和场景设定'); return }
if (videoSelectMode.value) toggleVideoSbSelect(sb.id) if (videoSelectMode.value) toggleVideoSbSelect(sb.id)
else selectedSb.value = sb else selectedSb.value = sb
} }
@@ -2291,6 +2359,7 @@ function configComfyLabel(config) {
const configGpuLabel = computed(() => configGpu(selectedShotVideoConfig.value) === 'auto' ? '自动分配' : configGpu(selectedShotVideoConfig.value).toUpperCase()) const configGpuLabel = computed(() => configGpu(selectedShotVideoConfig.value) === 'auto' ? '自动分配' : configGpu(selectedShotVideoConfig.value).toUpperCase())
function videoTaskRoutingLabel(task, compact = false) { function videoTaskRoutingLabel(task, compact = false) {
if (!task) return '尚未提交生成任务' if (!task) return '尚未提交生成任务'
if (taskParams(task).firstFrameComposition && !task.task_id && task.status==='processing') return '正在准备首帧,视频尚未提交显卡'
const route = task.routing || taskParams(task).routing const route = task.routing || taskParams(task).routing
const status = ({ processing: '当前任务', completed: '已完成', failed: '上次任务' })[task.status] || '任务' const status = ({ processing: '当前任务', completed: '已完成', failed: '上次任务' })[task.status] || '任务'
const worker = route?.gpu ? `${route.gpu.toUpperCase()} · ${route.legacy ? '旧节点' : `ComfyUI ${route.port}`}` : '等待节点确认' const worker = route?.gpu ? `${route.gpu.toUpperCase()} · ${route.legacy ? '旧节点' : `ComfyUI ${route.port}`}` : '等待节点确认'
@@ -2301,15 +2370,10 @@ function canReplaceVideo(sb) {
const task = latestVideoTask(sb?.id) const task = latestVideoTask(sb?.id)
if (task?.stage === 'cancelling') return false if (task?.stage === 'cancelling') return false
if (task?.status === 'processing' && ['waiting_predecessor', 'blocked_predecessor'].includes(task.stage)) return true if (task?.status === 'processing' && ['waiting_predecessor', 'blocked_predecessor'].includes(task.stage)) return true
if (task?.status === 'processing' && taskParams(task).firstFrameComposition && ['preparing_first_frame','first_frame_ready'].includes(task.stage)) return true
return task?.status === 'processing' && !!task.task_id && ['minimax', 'localvideo'].includes(task.provider) return task?.status === 'processing' && !!task.task_id && ['minimax', 'localvideo'].includes(task.provider)
&& !task.result_url && !['submitting', 'transferring', 'downloading', 'saving'].includes(task.stage) && !task.result_url && !['submitting', 'transferring', 'downloading', 'saving'].includes(task.stage)
} }
const isWan3Video = computed(() => selectedVideoConfig.value?.provider === 'aliyun'
|| bareModelName(videoModel.value).startsWith('wan3.0-video'))
// 参考图上限(Wan 3.0 官方 10 张,其他模型 9 张),绑定素材收集与 @名字 映射统一读取
const refImageLimit = computed(() => isWan3Video.value ? 10 : 9)
// 本次生成的生效配置(模型/分辨率/时长),用于右侧小结与批量确认弹窗 // 本次生成的生效配置(模型/分辨率/时长),用于右侧小结与批量确认弹窗
const effectiveVideoModelLabel = computed(() => { const effectiveVideoModelLabel = computed(() => {
const explicit = bareModelName(videoModel.value) const explicit = bareModelName(videoModel.value)
@@ -2331,7 +2395,7 @@ function openBatchVideoConfirm(pool) {
batchVideoConfirm.value = { open: true, targets } batchVideoConfirm.value = { open: true, targets }
} }
function batchVideos() { function batchVideos() {
if (continuityDirty.value) { toast.error('请先保存当前镜头的衔接设置'); return } if (continuityDirty.value || sceneLockDirty.value) { toast.error('请先保存当前镜头的衔接和场景设定'); return }
// 选择模式且有勾选 → 仅所选(允许重出已完成镜头);否则全部未完成(待生成+失败) // 选择模式且有勾选 → 仅所选(允许重出已完成镜头);否则全部未完成(待生成+失败)
const useSelection = videoSelectMode.value && selectedVideoSbIds.value.length const useSelection = videoSelectMode.value && selectedVideoSbIds.value.length
const pool = useSelection const pool = useSelection
@@ -2451,7 +2515,9 @@ async function cancelTask(row) {
function genTaskTargetLabel(task) { function genTaskTargetLabel(task) {
if (task.storyboard_id) { if (task.storyboard_id) {
const sb = sbs.value.find(x => x.id === task.storyboard_id) const sb = sbs.value.find(x => x.id === task.storyboard_id)
return t('episode.tasks.sbN', { n: sb?.storyboard_number ?? sb?.storyboardNumber ?? task.storyboard_id }) const label = t('episode.tasks.sbN', { n: sb?.storyboard_number ?? sb?.storyboardNumber ?? task.storyboard_number ?? task.storyboard_id }) + (task.storyboard_deleted ? '(已删除分镜)' : '')
const params = taskParams(task)
return params.compositionParentTaskId ? `${label} · 首帧 ${params.compositionRound}/${params.compositionRounds}` : label
} }
if (task.character_id) { if (task.character_id) {
const c = chars.value.find(x => x.id === task.character_id) const c = chars.value.find(x => x.id === task.character_id)
@@ -2515,6 +2581,9 @@ function taskSubmittedInEnglish(task) {
} }
function trackedStageLabel(stage) { function trackedStageLabel(stage) {
if (stage === 'preparing_first_frame') return '生成分镜首帧'
if (stage === 'first_frame_ready') return '首帧就绪'
if (stage === 'prepared') return '准备提交首帧'
if (stage === 'cancelling') return '正在确认取消' if (stage === 'cancelling') return '正在确认取消'
if (stage === 'waiting_predecessor') return '等待前镜头' if (stage === 'waiting_predecessor') return '等待前镜头'
if (stage === 'blocked_predecessor') return '前镜头待处理' if (stage === 'blocked_predecessor') return '前镜头待处理'
@@ -2551,9 +2620,10 @@ function genTaskDuration(row) {
// 后端任务在刷新/重启后仍可能运行,工作台持续同步,不依赖抽屉是否打开。 // 后端任务在刷新/重启后仍可能运行,工作台持续同步,不依赖抽屉是否打开。
let refreshingTasks = false let refreshingTasks = false
watch(genTaskActiveCount, active => { const taskPollingNeeded = computed(() => genTaskActiveCount.value > 0 || genTasks.value.some(task => task.status === 'completed' && taskParams(task).remoteCleanup?.status === 'pending'))
watch(taskPollingNeeded, needed => {
stopGenTasksPolling() stopGenTasksPolling()
if (active > 0) genTasksTimer = setInterval(async () => { if (needed) genTasksTimer = setInterval(async () => {
if (refreshingTasks) return if (refreshingTasks) return
refreshingTasks = true refreshingTasks = true
try { await refresh() } finally { refreshingTasks = false } try { await refresh() } finally { refreshingTasks = false }
@@ -2750,8 +2820,7 @@ const currentSubStageLabel = computed(() => currentStageLabel.value)
const totalDuration = computed(() => sbs.value.reduce((s, sb) => s + (sb.duration || 10), 0)) const totalDuration = computed(() => sbs.value.reduce((s, sb) => s + (sb.duration || 10), 0))
const selectedSb = ref(null) const selectedSb = ref(null)
const selectedVideoTaskNumber = computed(() => { const selectedVideoTaskNumber = computed(() => {
const index = videoTaskRows.value.findIndex(task => String(task.id) === String(selectedSb.value?.id)) return selectedSb.value ? shotNumber(selectedSb.value) : 0
return index >= 0 ? index + 1 : 0
}) })
const shotSaves = new Map() const shotSaves = new Map()
@@ -3019,7 +3088,7 @@ function pollVideoPromptBatch(attempts = 240) {
} }
setTimeout(() => tick(attempts), 2500) setTimeout(() => tick(attempts), 2500)
} }
function doBreakdown() { function doBreakdown(resumeFrom = '') {
const charList = chars.value.length const charList = chars.value.length
? chars.value.map(c => `${c.name}(ID:${c.id})`).join('、') ? chars.value.map(c => `${c.name}(ID:${c.id})`).join('、')
: '(当前集还没有角色)' : '(当前集还没有角色)'
@@ -3029,7 +3098,7 @@ function doBreakdown() {
const propList = propItems.value.length const propList = propItems.value.length
? propItems.value.map(p => `${p.name}(ID:${p.id})`).join('、') ? propItems.value.map(p => `${p.name}(ID:${p.id})`).join('、')
: '(当前集还没有道具)' : '(当前集还没有道具)'
runAgent('storyboard_breaker', `请基于当前集剧本拆分分镜,并为每个分镜段落同时生成 video_prompt(中文视频提示词)和 video_prompt_en(英文制作提示词,中文对白及 @/# 素材引用原样保留)。 startBreakdown(`请基于当前集剧本拆分分镜,并为每个分镜段落同时生成 video_prompt(中文视频提示词)和 video_prompt_en(英文制作提示词,中文对白及 @/# 素材引用原样保留)。
本次视频模型:${effectiveVideoModelLabel.value},请按该模型的特性与时长限制生成 video_prompt。 本次视频模型:${effectiveVideoModelLabel.value},请按该模型的特性与时长限制生成 video_prompt。
当前集已有角色:${charList} 当前集已有角色:${charList}
@@ -3040,7 +3109,7 @@ function doBreakdown() {
- 每个镜头必须根据剧本内容,从上述当前集已有角色中选出出场的角色绑定 character_ids(ID 必须来自上述列表;有角色出场就必须绑定,不要遗漏) - 每个镜头必须根据剧本内容,从上述当前集已有角色中选出出场的角色绑定 character_ids(ID 必须来自上述列表;有角色出场就必须绑定,不要遗漏)
- 每个镜头尽量匹配上述已有场景填写 scene_id(ID 必须来自上述列表),不要凭空创造新场景 - 每个镜头尽量匹配上述已有场景填写 scene_id(ID 必须来自上述列表),不要凭空创造新场景
- 每个镜头出现关键道具(被使用、交接、特写或在画面中明显可见)时,从上述当前集已有道具中绑定 prop_ids(ID 必须来自上述列表);没有道具出现可传空数组 - 每个镜头出现关键道具(被使用、交接、特写或在画面中明显可见)时,从上述当前集已有道具中绑定 prop_ids(ID 必须来自上述列表);没有道具出现可传空数组
- 只有纯环境空镜头才可以不绑定角色`, dramaId, epId.value, onBreakdownDone, chatModelOverride(), chatConfigId()) - 只有纯环境空镜头才可以不绑定角色`, dramaId, chatModelOverride(), chatConfigId(), typeof resumeFrom === 'string' ? resumeFrom : undefined)
} }
/** 拆分完成后刷新并自动补齐缺失的视频提示词(兜住 Agent 漏写/截断) */ /** 拆分完成后刷新并自动补齐缺失的视频提示词(兜住 Agent 漏写/截断) */
@@ -3053,7 +3122,7 @@ async function onBreakdownDone() {
// 按需为单个分镜生成视频提示词:由 prompt_generator 读取分镜字段生成并保存到 video_prompt // 按需为单个分镜生成视频提示词:由 prompt_generator 读取分镜字段生成并保存到 video_prompt
async function genVideoPrompt(sb) { async function genVideoPrompt(sb) {
if (!sb || videoPromptGeneratingIds.value.includes(sb.id)) return if (!sb || videoPromptGeneratingIds.value.includes(sb.id)) return
const idx = sbs.value.indexOf(sb) + 1 const idx = shotNumber(sb)
const cfg = selectedVideoConfig.value const cfg = selectedVideoConfig.value
const label = cfg ? `${cfg.name} (${cfg.provider})` : '默认' const label = cfg ? `${cfg.name} (${cfg.provider})` : '默认'
const charNames = getStoryboardCharacters(sb).map(c => c.name).join('、') || '无' const charNames = getStoryboardCharacters(sb).map(c => c.name).join('、') || '无'
@@ -3291,21 +3360,24 @@ function formatHistoryTime(iso) {
return `${p(d.getMonth() + 1)}-${p(d.getDate())} ${p(d.getHours())}:${p(d.getMinutes())}` return `${p(d.getMonth() + 1)}-${p(d.getDate())} ${p(d.getHours())}:${p(d.getMinutes())}`
} }
function getShotReferenceImages(sb) { function getShotBoundImages(sb) {
const refs = []
const pushRef = (value) => {
if (!value || refs.includes(value) || refs.length >= refImageLimit.value) return
refs.push(value)
}
const scene = getStoryboardScene(sb) const scene = getStoryboardScene(sb)
pushRef(scene?.image_url || scene?.imageUrl) return [
for (const char of getStoryboardCharacters(sb)) { ...(scene ? [{ name: scene.location, url: scene.image_url || scene.imageUrl }] : []),
pushRef(char?.image_url || char?.imageUrl) ...getStoryboardCharacters(sb).map(c => ({ name: c.name, url: c.image_url || c.imageUrl })),
} ...getStoryboardProps(sb).map(p => ({ name: p.name, url: p.image_url || p.imageUrl })),
for (const prop of getStoryboardProps(sb)) { ]
pushRef(prop?.image_url || prop?.imageUrl) }
}
return refs function getShotReferenceAssets(sb, modelKey = modelForShot(sb.id)) {
const config = configForModel(modelKey)
const model = bareModelName(modelKey) || configModels(config)[0] || ''
if (usesComposedFirstFrame(model)) return getShotBoundImages(sb).filter(a=>a.url)
return selectVideoReferences(getShotBoundImages(sb), referenceImageLimit(model, config?.provider))
}
function getShotReferenceImages(sb, modelKey = modelForShot(sb.id)) {
return getShotReferenceAssets(sb, modelKey).map(a => a.url)
} }
// 右侧参考素材面板:本集全部可绑定素材(场景单选、角色/道具多选),bound 标记是否已绑定 // 右侧参考素材面板:本集全部可绑定素材(场景单选、角色/道具多选),bound 标记是否已绑定
@@ -3364,8 +3436,20 @@ const refBindableAssets = computed(() => {
return sb ? shotBindableAssets(sb) : [] return sb ? shotBindableAssets(sb) : []
}) })
// 右栏「绑定参考图」:当前分镜已绑定素材(生成时作为参考图提交),按分组顺序平铺展示 // 保留全部绑定;提交时按当前模型选择图片,并明确显示单图限制。
const boundRefAssets = computed(() => refBindableAssets.value.filter(a => a.bound)) const boundRefAssets = computed(() => refBindableAssets.value.filter(a => a.bound))
const selectedShotComposesFrame = computed(() => !!selectedSb.value && usesComposedFirstFrame(bareModelName(modelForShot(selectedSb.value.id)) || configModels(selectedShotVideoConfig.value)[0] || ''))
const shotReferenceHint = computed(() => {
const sb = selectedSb.value
if (!sb) return ''
const key = modelForShot(sb.id)
const config = configForModel(key)
const model = bareModelName(key) || configModels(config)[0] || ''
if (!usesComposedFirstFrame(model)) return ''
const count = boundRefAssets.value.length
const missing = boundRefAssets.value.filter(a=>!a.ready).length
return `先用顶部选择的 Qwen 图片模型融合全部 ${count} 个绑定素材,生成完整分镜首帧,再交给 ${model} 生成视频。${count>6 ? '超过 6 张将分轮融合,保留每轮图片,不丢弃素材。' : ''}${missing ? `还有 ${missing} 个素材没有图片,请先生成或上传。` : ''}${(sb.continuity_mode || sb.continuityMode)==='first_frame' ? '尾帧接续会将前镜头末帧一同用于首帧融合。' : ''}`
})
// 参考面板分组顺序(kind code 驱动,label 渲染时求值) // 参考面板分组顺序(kind code 驱动,label 渲染时求值)
const REF_KINDS = computed(() => ([ const REF_KINDS = computed(() => ([
@@ -3376,6 +3460,7 @@ const REF_KINDS = computed(() => ([
// 右侧面板切换绑定:场景单选(切换/解绑),角色/道具多选(kind code 判断,不依赖显示文案) // 右侧面板切换绑定:场景单选(切换/解绑),角色/道具多选(kind code 判断,不依赖显示文案)
function toggleShotBind(sb, asset) { function toggleShotBind(sb, asset) {
if (sceneLockDirty.value) { toast.error('请先保存场景固定设定'); return }
if (asset.kind === 'scene') { if (asset.kind === 'scene') {
const current = sb?.scene_id || sb?.sceneId const current = sb?.scene_id || sb?.sceneId
updateField(sb, 'scene_id', current === asset.id ? null : asset.id) updateField(sb, 'scene_id', current === asset.id ? null : asset.id)
@@ -3427,42 +3512,11 @@ const mentionOptions = computed(() => {
] ]
}) })
// 按参考图顺序(场景图在前、角色图居中、道具图在后)为 @名字 建立索引映射,供视频提示词引用替换 // 编号和实际提交图片共用同一份清单,换模型后不留下错误的图片编号。
function getShotReferenceIndexMap(sb) { function resolveVideoPromptRefs(sb, modelKey = modelForShot(sb.id)) {
const ordered = []
const seen = new Set()
const push = (name, url) => {
if (!url || seen.has(url) || ordered.length >= refImageLimit.value) return
seen.add(url)
ordered.push({ name, imageUrl: url })
}
const scene = getStoryboardScene(sb)
push(scene?.location || '', scene?.image_url || scene?.imageUrl)
for (const char of getStoryboardCharacters(sb)) {
push(char.name || '', char?.image_url || char?.imageUrl)
}
for (const prop of getStoryboardProps(sb)) {
push(prop.name || '', prop?.image_url || prop?.imageUrl)
}
const nameToIndex = {}
ordered.forEach((a, i) => { if (a.name && !(a.name in nameToIndex)) nameToIndex[a.name] = i + 1 })
return nameToIndex
}
// 将视频提示词里的 @名字 替换为 @图片N名字(N 为参考图序号,1 起),生成时使用
function resolveVideoPromptRefs(sb) {
const prompt = sb.video_prompt || sb.videoPrompt || '' const prompt = sb.video_prompt || sb.videoPrompt || ''
const map = getShotReferenceIndexMap(sb) if (usesComposedFirstFrame(bareModelName(modelKey) || configModels(configForModel(modelKey))[0] || '')) return prompt
const names = Object.keys(map).sort((a, b) => b.length - a.length) return resolveReferenceMentions(prompt, getShotReferenceAssets(sb, modelKey), getShotBoundImages(sb))
if (!names.length) return prompt
return prompt.replace(/@([^\s@#]+)/g, (m, raw) => {
for (const name of names) {
if (raw.startsWith(name)) {
return `@图片${map[name]}${name}${raw.slice(name.length)}`
}
}
return m
})
} }
// 分镜时长(视频生成参数区直接编辑并保存到分镜): // 分镜时长(视频生成参数区直接编辑并保存到分镜):
@@ -3519,21 +3573,24 @@ function uploadAssetImage(kind, id) {
async function genVid(sb, opts = {}) { async function genVid(sb, opts = {}) {
try { await waitShotSave(sb.id) } catch { toast.error('分镜尚未保存成功,请保存后再生成视频'); return } try { await waitShotSave(sb.id) } catch { toast.error('分镜尚未保存成功,请保存后再生成视频'); return }
if (continuityDirty.value) { toast.error('请先保存当前镜头的衔接设置'); return } if (continuityDirty.value || sceneLockDirty.value) { toast.error('请先保存当前镜头的衔接和场景设定'); return }
if (videoSubmittingIds.value.includes(sb.id)) return if (videoSubmittingIds.value.includes(sb.id)) return
if (!opts.forceNew && canRecoverVideo(sb.id)) return recoverVideo(sb) if (!opts.forceNew && canRecoverVideo(sb.id)) return recoverVideo(sb)
const latest = latestVideoTask(sb.id) const latest = latestVideoTask(sb.id)
if (latest?.status === 'processing' && !opts.replace) return if (latest?.status === 'processing' && !opts.replace) return
const forceNew = opts.forceNew || latest?.status === 'failed' const forceNew = opts.forceNew || latest?.status === 'failed'
if (!opts.replace && latest?.status === 'failed' && !window.confirm(t('episode.recovery.confirmNew'))) return if (!opts.replace && latest?.status === 'failed' && !window.confirm(t('episode.recovery.confirmNew'))) return
const referenceImages = getShotReferenceImages(sb)
const modelKey = opts.batch ? videoModel.value : modelForShot(sb.id) const modelKey = opts.batch ? videoModel.value : modelForShot(sb.id)
const referenceImages = getShotReferenceImages(sb, modelKey)
const chosenConfig = configForModel(modelKey) const chosenConfig = configForModel(modelKey)
// 参考素材完全来自分镜绑定的角色/场景/道具图片 // 参考素材完全来自分镜绑定的角色/场景/道具图片
const params = { const params = {
frame_image_config_id: ownerConfigId(imageModelOptions.value,imageModel.value),
frame_image_model: bareModelName(imageModel.value) || undefined,
regenerate_first_frame: opts.regenerateFirstFrame === true,
storyboard_id: sb.id, storyboard_id: sb.id,
drama_id: dramaId, drama_id: dramaId,
prompt: resolveVideoPromptRefs(sb), prompt: resolveVideoPromptRefs(sb, modelKey),
duration: Number(sb.duration || 10), duration: Number(sb.duration || 10),
aspect_ratio: dramaAspectRatio.value, aspect_ratio: dramaAspectRatio.value,
generate_audio: true, generate_audio: true,
@@ -3662,6 +3719,9 @@ onMounted(() => setTimeout(() => autoTour('episode', EPISODE_TOUR, t), 900))
</script> </script>
<style scoped> <style scoped>
.storyboard-job-status { margin: 0 16px 12px; padding: 14px 18px; border: 1px solid var(--border-color); border-radius: 14px; background: var(--bg-2); flex-shrink: 0; font-size: 13px; }
.storyboard-job-status p { margin: 7px 0 0; white-space: pre-wrap; }
.storyboard-job-error { color: #b45309; }
.task-cancel-actions { display:flex; align-items:center; gap:10px; margin:8px 0; font-size:12px; color:var(--text-3); } .task-cancel-actions { display:flex; align-items:center; gap:10px; margin:8px 0; font-size:12px; color:var(--text-3); }
.task-cancelled-note { margin-top:6px; color:var(--text-3); font-size:13px; } .task-cancelled-note { margin-top:6px; color:var(--text-3); font-size:13px; }
.video-task-status.is-cancelled, .video-task-metric.is-cancelled { color:var(--text-3); background:var(--bg-2); border-color:var(--border-color); } .video-task-status.is-cancelled, .video-task-metric.is-cancelled { color:var(--text-3); background:var(--bg-2); border-color:var(--border-color); }
@@ -5260,6 +5320,10 @@ button.video-task-metric.on { box-shadow: 0 0 0 2px var(--accent); }
transition: background 0.16s var(--ease-out), border-color 0.16s var(--ease-out); transition: background 0.16s var(--ease-out), border-color 0.16s var(--ease-out);
cursor: pointer; cursor: pointer;
} }
.video-task-actions { display: flex; flex-direction: column; gap: 5px; align-items: center; }
.storyboard-delete-action { color: var(--action-danger); }
.storyboard-delete-action:hover { background: var(--action-danger-bg); color: var(--action-danger); }
.storyboard-delete-action:disabled { opacity: .45; cursor: not-allowed; }
.video-task-row:first-child { .video-task-row:first-child {
border-top: 0; border-top: 0;
} }
@@ -0,0 +1,36 @@
import { test } from 'node:test'
import assert from 'node:assert/strict'
import { referenceImageLimit, selectVideoReferences, resolveReferenceMentions, usesComposedFirstFrame } from '../app/utils/video-references.js'
import { videoReferenceError } from '../app/utils/video-reference-error.js'
const bound = [
{ name: '404宿舍', url: '/scene.png' },
{ name: '乔雨灵', url: '/girl.png' },
{ name: '乔雨', url: '/other.png' },
{ name: '无图道具', url: '' },
{ name: '重复图片', url: '/scene.png' },
]
test('native single-image limits and reference numbering stay separate from frame composition', () => {
const single = selectVideoReferences(bound, referenceImageLimit('LTX-2.3', 'localvideo'))
assert.deepEqual(single, [bound[0]])
const full = selectVideoReferences(bound, referenceImageLimit('MiniMax-H3', 'minimax'))
assert.deepEqual(full, bound.slice(0, 3))
const prompt = '@图片2乔雨灵 在 @图片1404宿舍 看向 @图片3乔雨。'
assert.equal(resolveReferenceMentions(prompt, single, bound), '@乔雨灵 在 @图片1404宿舍 看向 @乔雨。')
assert.equal(resolveReferenceMentions(prompt, full), prompt)
assert.equal(bound.length, 5)
})
test('single-image models and Wan 3 retain their actual limits', () => {
for (const model of ['LTX-2.3','LTX-2.3-Text-Preserve','Wan2.2-I2V']) assert.equal(usesComposedFirstFrame(model), true)
for (const model of ['MiniMax-H3','MiniMax-H3-VDN-H3','wan3.0-video']) assert.equal(usesComposedFirstFrame(model), false)
assert.equal(referenceImageLimit('LTX-2.3-Text-Preserve', 'localvideo'), 1)
assert.equal(referenceImageLimit('Wan2.2-I2V', 'localvideo'), 1)
assert.equal(referenceImageLimit('wan3.0-video', 'aliyun'), 10)
assert.equal(referenceImageLimit('MiniMax-H3-VDN-H3', 'localvideo'), 9)
})
test('historic task errors explain the limit without leaking raw provider details', () => {
const raw = 'API error 400: {"error":{"message":"LTX-2.3 requires 0-1 images","type":"bridge_error","code":400}}'
assert.match(videoReferenceError(raw), /LTX-2.3 最多支持 1 张参考图/)
assert.equal(videoReferenceError('API error 401: {"secret":"test"}'), '')
assert.equal(videoReferenceError('首帧模式只能提交 1 张图片,不能混用多张参考图'), '首帧模式只能提交 1 张图片,不能混用多张参考图')
})
@@ -0,0 +1,24 @@
"""Read-only verification of isolated real-model storyboard results."""
from pathlib import Path
import hashlib,json,sqlite3,sys
ROOT=Path(__file__).resolve().parent.parent
reports=[]
for argument in sys.argv[1:]:
folder=Path(argument).resolve()
assert folder.is_relative_to(ROOT/'integration-work')
with sqlite3.connect((folder/'huobao.sqlite3').as_uri()+'?mode=ro',uri=True) as db:
db.row_factory=sqlite3.Row
assert db.execute('pragma integrity_check').fetchone()[0]=='ok'
job=json.loads(db.execute("select value from app_settings where key='storyboard_job:8'").fetchone()[0])
assert job['status']=='done',job['status']
shots=[dict(r) for r in db.execute('select * from storyboards where episode_id=8 and deleted_at is null order by storyboard_number')]
assert len(shots)==job['completed'] and shots[-1]['storyboard_number']==job['narrativeEndShot']
assert [s['storyboard_number'] for s in shots]==list(range(1,len(shots)+1))
for shot in shots:
assert shot['video_prompt_en'] and shot['video_prompt_en_source_hash']==hashlib.sha256(('english-v1-speech-cues\n'+shot['video_prompt'].strip()).encode()).hexdigest()
assert db.execute('select count(*) from sys_task').fetchone()[0]==0,'No media jobs should be submitted'
from datetime import datetime
elapsed=(datetime.fromisoformat(job['finishedAt'])-datetime.fromisoformat(job['startedAt'])).total_seconds()
reports.append({'fixture':str(folder),'status':'done','shots':len(shots),'video_seconds':job['seconds'],'elapsed_seconds':round(elapsed,1),'bilingual_hashes_valid':True,'stopped_at_script_ending':True,'media_jobs_started':0})
(ROOT/'integration-work/storyboard-real-model-audit.json').write_text(json.dumps(reports,indent=2),'utf-8')
print(json.dumps(reports))
@@ -0,0 +1,602 @@
"""Private adapters for Huobao: Dify, Qwen Image, local H3, and Grok CLI."""
from __future__ import annotations
import asyncio
import base64
from contextlib import closing
from email import policy
from email.parser import BytesParser
import hashlib
import hmac
import json
import math
import os
from pathlib import Path
import re
import secrets
import sqlite3
import struct
import tempfile
import time
from urllib.parse import quote
import httpx
from fastapi import Depends, FastAPI, Header, HTTPException, Request
from fastapi.responses import JSONResponse, StreamingResponse
def chat_query(body):
messages = body.get("messages")
if not isinstance(messages, list) or not messages:
raise HTTPException(400, "messages must be a non-empty array")
if body.get("tool_choice") == "required" or isinstance(body.get("tool_choice"), dict):
raise HTTPException(400, "This chatflow does not support required tool calls; use DeepSeek")
parts = []
for item in messages:
content = item.get("content", "")
if isinstance(content, list):
if any(p.get("type") != "text" for p in content):
raise HTTPException(400, "This adapter accepts text messages only")
content = "\n".join(p.get("text", "") for p in content)
if content:
parts.append(f"[{item.get('role', 'user')}]\n{content}")
fmt = body.get("response_format") or {}
if fmt.get("type") == "json_schema":
parts.append("Return only JSON matching this schema:\n" + json.dumps(fmt.get("json_schema", {}).get("schema", {}), ensure_ascii=False))
elif fmt.get("type") == "json_object":
parts.append("Return only a valid JSON object, without Markdown fences.")
if not parts:
raise HTTPException(400, "Empty messages")
return "\n\n".join(parts)
def image_dimensions(data_url):
try:
data = base64.b64decode(data_url.split(",", 1)[1], validate=True)
if data.startswith(b"\x89PNG\r\n\x1a\n"):
return struct.unpack(">II", data[16:24])
if data.startswith((b"GIF87a", b"GIF89a")):
return struct.unpack("<HH", data[6:10])
if data[:2] == b"\xff\xd8":
offset = 2
while offset + 4 < len(data):
if data[offset] != 255:
offset += 1
continue
while data[offset] == 255:
offset += 1
marker = data[offset]
offset += 1
if marker in {0xD8, 0xD9} or 0xD0 <= marker <= 0xD7:
continue
size = int.from_bytes(data[offset:offset + 2], "big")
if marker in {0xC0, 0xC1, 0xC2, 0xC3, 0xC5, 0xC6, 0xC7, 0xC9, 0xCA, 0xCB, 0xCD, 0xCE, 0xCF}:
height, width = struct.unpack(">HH", data[offset + 3:offset + 7])
return width, height
if size < 2:
break
offset += size
if data[:4] == b"RIFF" and data[8:12] == b"WEBP":
if data[12:16] == b"VP8X":
return 1 + int.from_bytes(data[24:27], "little"), 1 + int.from_bytes(data[27:30], "little")
if data[12:16] == b"VP8L" and data[20] == 0x2F:
bits = int.from_bytes(data[21:25], "little")
return (bits & 0x3FFF) + 1, ((bits >> 14) & 0x3FFF) + 1
if data[12:16] == b"VP8 " and data[23:26] == b"\x9d\x01\x2a":
width, height = struct.unpack("<HH", data[26:30])
return width & 0x3FFF, height & 0x3FFF
except (ValueError, IndexError, struct.error):
pass
raise HTTPException(400, "Unable to read first-frame dimensions; use PNG, JPEG or WebP")
VIDEO_MODELS = {
"MiniMax-H3-VDN-H3": {"reference": "minimax_h3_vdn_hd", "min_images": 1, "max_images": 9},
"MiniMax-H3": {"text": "minimax_h3_t2v_official", "first": "minimax_h3_i2v", "reference": "minimax_h3_r2v", "min_images": 0, "max_images": 9},
"MiniMax-H3-Hybrid": {"text": "minimax_h3_hybrid_v0331_t2v", "first": "minimax_h3_hybrid_v0331_i2v", "reference": "minimax_h3_hybrid_v0331_i2v", "min_images": 0, "max_images": 9},
"MiniMax-H3-R2V-4Step": {"reference": "minimax_h3_r2v_4step", "min_images": 1, "max_images": 9},
"LTX-2.3": {"text": "ltx23_t2v", "first": "ltx23_t2v", "reference": "ltx23_t2v", "min_images": 0, "max_images": 1},
"LTX-2.3-Text-Preserve": {"text": "ltx23_t2v_text_preserve", "first": "ltx23_t2v_text_preserve", "reference": "ltx23_t2v_text_preserve", "min_images": 0, "max_images": 1},
"Wan2.2-I2V": {"first": "wan22_simple", "reference": "wan22_simple", "min_images": 1, "max_images": 1},
}
def video_workflows(spec):
return {spec[mode] for mode in ("text", "first", "reference") if mode in spec}
def gpu_routing(body):
gpu = body.get("gpu", "auto")
if gpu not in ("auto", "gpu1", "gpu2", "gpu3"):
raise HTTPException(400, "gpu must be auto, gpu1, gpu2 or gpu3")
if gpu == "auto":
return {"provider_id": "default", "huobao_panels": True}
return {"provider_id": {"gpu1": "gpu1_8189", "gpu2": "default", "gpu3": "gpu3"}[gpu], "strict_provider": True, "huobao_panels": True}
def worker_routing(provider_id):
worker = {"default": ("gpu2", 8188), "gpu1_8189": ("gpu1", 8189), "gpu3": ("gpu3", 8191), "gpu1": ("gpu1", None)}.get(provider_id)
if not worker:
return None
gpu, port = worker
return {"gpu": gpu, "port": port, "providerId": provider_id, "legacy": provider_id == "gpu1"}
def h3_request(body):
"""Translate the local video protocol. 'default' means upstream auto-balance."""
model = body.get("model", "MiniMax-H3")
spec = VIDEO_MODELS.get(model)
if spec is None:
raise HTTPException(400, "Unknown local video model; query /v1/models")
content = body.get("content") or []
if not isinstance(content, list):
raise HTTPException(400, "content must be an array")
prompt = "\n".join(str(c.get("text", "")) for c in content if c.get("type") == "text").strip()
if not prompt or len(prompt) > 7000:
raise HTTPException(400, "A prompt of 1-7000 characters is required")
images, roles = [], []
for item in content:
kind = item.get("type")
if kind == "text":
continue
if kind != "image_url" or item.get("role") == "last_frame":
raise HTTPException(400, "Local video supports first-frame/reference images; last frames, reference video and voice samples are unsupported")
raw = item.get("image_url") or {}
url = raw.get("url", "") if isinstance(raw, dict) else raw
if not isinstance(url, str) or not url.startswith("data:image/"):
raise HTTPException(400, "Local video requires inline image data; enable Huobao's local-image conversion")
images.append(url)
roles.append(item.get("role", "reference_image"))
if not spec["min_images"] <= len(images) <= spec["max_images"]:
raise HTTPException(400, f"{model} requires {spec['min_images']}-{spec['max_images']} images")
if "first_frame" in roles and len(images) != 1:
raise HTTPException(400, "Use either one fixed first frame or reference images; these modes cannot be combined")
mode = "text" if not images else "first" if roles == ["first_frame"] else "reference"
if mode not in spec:
raise HTTPException(400, f"{model} does not support {mode} mode; use reference images")
try:
duration = float(body.get("duration", 5))
except (ValueError, TypeError):
raise HTTPException(400, "Invalid duration")
if not math.isfinite(duration) or not 4 <= duration <= 15:
raise HTTPException(400, "Duration must be 4-15 seconds")
ratio = body.get("ratio", "16:9")
if ratio not in {"16:9", "9:16", "1:1", "4:3", "3:4", "21:9"}:
raise HTTPException(400, "Unsupported aspect ratio")
resolution = str(body.get("resolution", "768P")).upper()
resolutions = {"480P": 480, "720P": 720, "1080P": 1080, "768P": 768, "2K": 1152}
if model == "MiniMax-H3-VDN-H3":
if ratio not in {"16:9", "9:16"}:
raise HTTPException(400, "VDN HD supports 16:9 or 9:16")
resolutions = {"2K": 1440, "4K": 2160}
if resolution not in resolutions:
raise HTTPException(400, "Unsupported video resolution")
short = resolutions[resolution]
a, b = map(int, ratio.split(":"))
if roles == ["first_frame"] and "ratio" not in body:
a, b = image_dimensions(images[0])
if a <= 0 or b <= 0:
raise HTTPException(400, "Invalid first-frame dimensions")
width = round(short * max(1, a / b) / 64) * 64
height = round(short * max(1, b / a) / 64) * 64
if model == "MiniMax-H3-VDN-H3":
width, height = (short * 16 // 9, short) if a > b else (short, short * 16 // 9)
workflow = spec[mode]
request = {**gpu_routing(body), "workflow_key": workflow, "prompt": prompt,
"input_images": images, "duration": duration, "width": width, "height": height,
"metadata": {"source": "huobao-drama", "model": model}}
if body.get("seed") is not None:
if not isinstance(body["seed"], int) or not 0 <= body["seed"] < 2**63:
raise HTTPException(400, "Invalid seed")
request["seed"] = body["seed"]
return request
def qwen_image_request(body, images=None):
if body.get("model") != "qwen_image_2_1":
raise HTTPException(400, "Use model qwen_image_2_1")
prompt = body.get("prompt")
if not isinstance(prompt, str) or not prompt.strip() or len(prompt) > 32000:
raise HTTPException(400, "A prompt of 1-32000 characters is required")
if str(body.get("n", 1)) != "1":
raise HTTPException(400, "This image adapter supports n=1 per task")
if body.get("response_format", "url") != "url":
raise HTTPException(400, "Asynchronous image tasks require response_format=url")
size = str(body.get("size", "1024x1024")).lower()
if size == "auto":
size = "1024x1024"
match = re.fullmatch(r"(\d{3,4})x(\d{3,4})", size)
if not match or not all(256 <= int(d) <= 4096 for d in match.groups()):
raise HTTPException(400, "size must be WIDTHxHEIGHT with dimensions 256-4096")
width, height = map(int, match.groups())
refs = images or []
if len(refs) > 6:
raise HTTPException(400, "Qwen Image 2.1 supports at most 6 reference images")
return {**gpu_routing(body), "workflow_key": "qwen_image_2_1", "prompt": prompt.strip(),
"width": width, "height": height, "batch_size": 1, "input_images": refs,
"metadata": {"source": "huobao-drama", "output_type": "image"}}
async def parse_image_edit(request):
content_type = request.headers.get("content-type", "")
if not content_type.lower().startswith("multipart/form-data;"):
raise HTTPException(415, "Image edits require multipart/form-data")
raw = bytearray()
async for chunk in request.stream():
raw.extend(chunk)
if len(raw) > 64 * 1024 * 1024:
raise HTTPException(413, "Reference uploads exceed 64 MiB")
message = BytesParser(policy=policy.default).parsebytes(
("Content-Type: " + content_type + "\r\nMIME-Version: 1.0\r\n\r\n").encode() + raw)
if not message.is_multipart() or message.defects:
raise HTTPException(400, "Invalid multipart image request")
fields, images = {}, []
for part in message.iter_parts():
name = part.get_param("name", header="content-disposition")
data = part.get_payload(decode=True)
if not isinstance(data, bytes):
raise HTTPException(400, "Invalid multipart field")
if name in {"image", "image[]"}:
mime = part.get_content_type()
if mime not in {"image/png", "image/jpeg", "image/webp"} or not data or len(data) > 10 * 1024 * 1024:
raise HTTPException(400, "Each reference must be PNG, JPEG or WebP, up to 10 MiB")
url = "data:" + mime + ";base64," + base64.b64encode(data).decode()
width, height = image_dimensions(url)
if not (0 < width <= 16384 and 0 < height <= 16384):
raise HTTPException(400, "Invalid reference image dimensions")
images.append(url)
if len(images) > 6:
raise HTTPException(400, "Qwen Image 2.1 supports at most 6 reference images")
elif name in {"model", "prompt", "size", "n", "response_format", "gpu"}:
if name in fields or len(data) > 128000:
raise HTTPException(400, "Duplicate or oversized form field")
try:
fields[name] = data.decode("utf-8")
except UnicodeDecodeError:
raise HTTPException(400, "Form text must be UTF-8")
else:
raise HTTPException(400, "Unsupported image edit field: " + str(name)[:60])
if not images:
raise HTTPException(400, "At least one reference image is required")
return qwen_image_request(fields, images)
def create_app(config):
app = FastAPI(title="Huobao private model bridge", docs_url=None, redoc_url=None)
state_dir = Path(config.get("state_dir", "./state"))
state_dir.mkdir(parents=True, exist_ok=True)
db_path = state_dir / "jobs.sqlite3"
with closing(sqlite3.connect(db_path)) as db, db:
db.execute("CREATE TABLE IF NOT EXISTS jobs (id TEXT PRIMARY KEY, duration REAL NOT NULL, created REAL NOT NULL)")
db.execute("CREATE TABLE IF NOT EXISTS image_jobs (id TEXT PRIMARY KEY, created REAL NOT NULL)")
def auth(authorization: str | None = Header(default=None)):
expected = config.get("token", "")
if not expected or not secrets.compare_digest(authorization or "", "Bearer " + expected):
raise HTTPException(401, "Invalid bridge token")
@app.exception_handler(HTTPException)
async def http_error(request, exc):
return JSONResponse({"error": {"message": str(exc.detail), "type": "bridge_error", "code": exc.status_code}}, status_code=exc.status_code)
@app.exception_handler(httpx.HTTPError)
async def upstream_error(request, exc):
return JSONResponse({"error": {"message": "Upstream connection failed: " + type(exc).__name__, "type": "upstream_error"}}, status_code=502)
def upstream_headers():
return {"Authorization": "Bearer " + config["h3"]["token"]}
async def upstream(method, path, body=None):
if not config.get("h3"):
raise HTTPException(404, "H3 is not configured on this bridge")
async with httpx.AsyncClient(timeout=httpx.Timeout(180, connect=10), trust_env=False) as client:
r = await client.request(method, config["h3"]["url"] + path, headers=upstream_headers(), json=body)
if r.is_error:
try:
detail = r.json().get("detail", "Model service request failed")
except ValueError:
detail = "Model service request failed"
raise HTTPException(r.status_code, str(detail)[:600])
return r.json()
@app.get("/health")
async def health():
return {"status": "ok", "service": "huobao-model-bridge"}
@app.get("/v1/models", dependencies=[Depends(auth)])
async def models():
names = list(config.get("dify", {}))
if config.get("h3"):
# Verify the real service, instead of making a local catalogue look healthy.
registry = await upstream("GET", "/v1/models")
available = {m["id"] for m in registry.get("data", [])}
names.extend(name for name, spec in VIDEO_MODELS.items() if video_workflows(spec) <= available)
if any(m.get("id") == "qwen_image_2_1" and m.get("model_type") == "image" for m in registry.get("data", [])):
names.append("qwen_image_2_1")
if config.get("grok"):
names.append("grok-local")
return {"object": "list", "data": [{"id": n, "object": "model", "owned_by": "private"} for n in names]}
async def dify_events(model, query):
upstream_config = config["dify"][model]
payload = {"inputs": {}, "query": query, "response_mode": "streaming",
"user": "huobao-" + secrets.token_hex(8), "auto_generate_name": False}
finished, usage = False, {}
async with httpx.AsyncClient(timeout=httpx.Timeout(240, connect=10), trust_env=False) as client:
async with client.stream("POST", upstream_config["url"] + "/chat-messages",
headers={"Authorization": "Bearer " + upstream_config["key"]}, json=payload) as response:
if response.is_error:
await response.aread()
try:
message = response.json().get("message", "Dify request failed")
except ValueError:
message = "Dify request failed"
raise HTTPException(response.status_code, str(message)[:400])
async for line in response.aiter_lines():
if not line.startswith("data:"):
continue
event = json.loads(line[5:].strip())
kind = event.get("event")
if kind in {"message", "agent_message"}:
yield "text", event.get("answer", "")
elif kind == "error":
raise HTTPException(502, str(event.get("message", "Dify generation failed"))[:400])
elif kind == "message_end":
finished = True
usage = event.get("metadata", {}).get("usage", {})
elif kind == "workflow_finished":
data = event.get("data") or {}
if data.get("status") not in {None, "succeeded"}:
raise HTTPException(502, str(data.get("error") or "Dify workflow failed")[:400])
if not finished:
raise HTTPException(502, "Dify stream ended before message_end")
yield "usage", usage
async def grok_events(query):
item = config["grok"]
args = [item["exe"], "--cwd", item["cwd"], "--output-format", "json", "--verbatim",
"--no-subagents", "--no-memory", "--disable-web-search", "--max-turns", "1",
"--tools", "", "--deny", "MCPTool", "--permission-mode", "dontAsk",
"--system-prompt-override", "Answer the supplied writing request using text only. Do not use tools."]
options = {"creationflags": 0x08000000} if os.name == "nt" else {}
# A file avoids the Windows command-line length limit for full scripts.
with tempfile.TemporaryDirectory(prefix="huobao-grok-") as scratch:
prompt_file = Path(scratch) / "prompt.txt"
prompt_file.write_text(query, encoding="utf-8")
args += ["--prompt-file", str(prompt_file)]
process = await asyncio.create_subprocess_exec(*args, stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.PIPE, **options)
try:
out, err = await asyncio.wait_for(process.communicate(), timeout=240)
except (asyncio.TimeoutError, asyncio.CancelledError):
process.kill()
await process.wait()
raise
try:
data = json.loads(out.decode("utf-8"))
except (ValueError, UnicodeDecodeError):
raise HTTPException(502, "Grok CLI did not return JSON; run Grok login on this computer")
if process.returncode or not data.get("text"):
raise HTTPException(502, "Grok CLI failed; check local login and available quota")
yield "text", data["text"]
yield "usage", data.get("usage", {})
@app.post("/v1/chat/completions", dependencies=[Depends(auth)])
async def chat(body: dict):
model = body.get("model", "")
if model not in config.get("dify", {}) and not (model == "grok-local" and config.get("grok")):
raise HTTPException(404, "Unknown text model")
query = chat_query(body)
events = grok_events(query) if model == "grok-local" else dify_events(model, query)
cid, created = "chatcmpl-" + secrets.token_hex(12), int(time.time())
def chunk(delta, finish=None):
return {"id": cid, "object": "chat.completion.chunk", "created": created, "model": model,
"choices": [{"index": 0, "delta": delta, "finish_reason": finish}]}
if body.get("stream"):
async def stream():
def encode(value):
return "data: " + json.dumps(value, ensure_ascii=False) + "\n\n"
yield encode(chunk({"role": "assistant", "content": ""}))
try:
async for kind, value in events:
if kind == "text":
yield encode(chunk({"content": value}))
yield encode(chunk({}, "stop"))
except (HTTPException, httpx.HTTPError) as error:
message = str(error.detail) if isinstance(error, HTTPException) else "Upstream connection failed"
yield encode({"error": {"message": message, "type": "upstream_error"}})
yield "data: [DONE]\n\n"
return StreamingResponse(stream(), media_type="text/event-stream", headers={"Cache-Control": "no-cache"})
text, usage = "", {}
async for kind, value in events:
if kind == "text":
text += value
else:
usage = value
return {"id": cid, "object": "chat.completion", "created": created, "model": model,
"choices": [{"index": 0, "message": {"role": "assistant", "content": text}, "finish_reason": "stop"}],
"usage": {"prompt_tokens": usage.get("prompt_tokens", usage.get("input_tokens", 0)),
"completion_tokens": usage.get("completion_tokens", usage.get("output_tokens", 0)),
"total_tokens": usage.get("total_tokens", 0)}}
async def require_qwen_image():
registry = await upstream("GET", "/v1/models")
if not any(m.get("id") == "qwen_image_2_1" and m.get("model_type") == "image" for m in registry.get("data", [])):
raise HTTPException(503, "Qwen Image 2.1 workflow is unavailable")
@app.get("/v1/images/pool", dependencies=[Depends(auth)])
async def image_pool():
await require_qwen_image()
health, catalogue = await asyncio.gather(upstream("GET", "/health"), upstream("GET", "/v1/providers"))
return {"model": "qwen_image_2_1", "auto_balance": health.get("autoBalance") is True,
"routing": "shortest-queue-with-reservations" if health.get("autoBalance") is True else "single-provider",
"configured_workers": [{"id": p["id"], "name": p.get("name", p["id"])} for p in catalogue.get("providers", [])]}
async def submit_image(payload):
await require_qwen_image()
result = await upstream("POST", "/v1/jobs", payload)
jid = result.get("id")
if not isinstance(jid, str) or not jid:
raise HTTPException(502, "Image service did not return a task ID")
with closing(sqlite3.connect(db_path)) as db, db:
db.execute("INSERT INTO image_jobs VALUES(?,?)", (jid, time.time()))
return {"task_id": jid, "status": "processing", "provider_id": result.get("providerId")}
@app.post("/v1/images/validate", dependencies=[Depends(auth)])
async def validate_image(body: dict):
payload = qwen_image_request(body)
await require_qwen_image()
return {"valid": True, "workflow": payload["workflow_key"], "requested_size": [payload["width"], payload["height"]], "max_reference_images": 6,
"pool": await image_pool()}
@app.post("/v1/images/generations", dependencies=[Depends(auth)])
async def generate_image(body: dict):
return await submit_image(qwen_image_request(body))
@app.post("/v1/images/edits", dependencies=[Depends(auth)])
async def edit_image(request: Request):
return await submit_image(await parse_image_edit(request))
@app.post("/v1/images/task/{jid}/cancel", dependencies=[Depends(auth)])
async def cancel_image(jid: str):
with closing(sqlite3.connect(db_path)) as db:
found = db.execute("SELECT 1 FROM image_jobs WHERE id=?", (jid,)).fetchone()
if not found:
raise HTTPException(404, "Unknown image task")
result = await upstream("POST", "/v1/jobs/" + quote(jid, safe="") + "/cancel", {})
return {"task": {"id": jid, "status": result.get("status", "running"),
"cancelRequested": result.get("cancelRequested", False)}}
@app.get("/v1/images/task/{jid}", dependencies=[Depends(auth)])
async def poll_image(jid: str):
with closing(sqlite3.connect(db_path)) as db:
found = db.execute("SELECT 1 FROM image_jobs WHERE id=?", (jid,)).fetchone()
if not found:
raise HTTPException(404, "Unknown image task")
result = await upstream("GET", "/v1/huobao/jobs/" + quote(jid, safe=""))
state = result.get("status", "running")
if state in {"failed", "cancelled"}:
return {"task_id": jid, "status": "failed", "cancelled": state == "cancelled", "error": {"message": str(result.get("error") or "Image generation failed")[:600]}}
if state != "succeeded":
return {"task_id": jid, "status": "processing", "provider_id": result.get("providerId"), "worker_state": result.get("worker_state"), "queue_position": result.get("queue_position")}
output = next((o for o in result.get("outputs", []) if o.get("kind") == "image"), None)
if output is None:
return {"task_id": jid, "status": "failed", "error": {"message": "Task completed without an image output"}}
index, expiry = int(output["index"]), int(time.time()) + 86400
extension = {"image/png": "png", "image/jpeg": "jpg", "image/webp": "webp"}.get(output.get("mimeType"), "png")
# Huobao determines the saved extension from the URL, not Content-Type.
url = f"{config['public_url']}/media/{quote(jid, safe='')}/{index}/image.{extension}?expires={expiry}&sig={signature(jid, index, expiry)}"
return {"task_id": jid, "status": "completed", "image_url": url, "data": [{"url": url}], "provider_id": result.get("providerId")}
@app.post("/v2/video_generation/validate", dependencies=[Depends(auth)])
async def validate_video(body: dict):
request = h3_request(body)
registry = await upstream("GET", "/v1/models")
if request["workflow_key"] not in {m["id"] for m in registry.get("data", [])}:
raise HTTPException(503, "Selected video workflow is unavailable")
plan = await upstream("POST", "/v1/audio-plan", request)
route = await upstream("POST", "/v1/huobao/route", request)
return {"valid": plan.get("status", "ready") == "ready", "workflow": request["workflow_key"],
"width": request["width"], "height": request["height"], "duration": request["duration"], "audio_plan": plan,
"routing": worker_routing(route.get("providerId"))}
@app.post("/v2/video_generation", dependencies=[Depends(auth)])
async def video(body: dict):
# Huobao probes video providers with an empty body. Check the real
# upstream without submitting a GPU task, then report readiness.
if not body:
registry = await upstream("GET", "/v1/models")
available = {m["id"] for m in registry.get("data", [])}
if not {"minimax_h3_t2v_official", "minimax_h3_i2v", "minimax_h3_r2v"} <= available:
raise HTTPException(503, "Required H3 workflows are unavailable")
return {"status": "ready", "probe": True, "model": "MiniMax-H3"}
request = h3_request(body)
registry = await upstream("GET", "/v1/models")
if request["workflow_key"] not in {m["id"] for m in registry.get("data", [])}:
raise HTTPException(503, "Selected video workflow is unavailable")
result = await upstream("POST", "/v1/jobs", request)
jid = result.get("id")
if not isinstance(jid, str) or not jid:
raise HTTPException(502, "Video service did not return a job ID")
with closing(sqlite3.connect(db_path)) as db, db:
db.execute("INSERT INTO jobs VALUES(?,?,?)", (jid, request["duration"], time.time()))
return {"task_id": jid, "status": "submitted", "provider_id": result.get("providerId"), "routing": worker_routing(result.get("providerId"))}
def get_job(jid):
with closing(sqlite3.connect(db_path)) as db:
row = db.execute("SELECT duration FROM jobs WHERE id=?", (jid,)).fetchone()
if not row:
raise HTTPException(404, "Unknown bridge job")
return row[0]
def signature(jid, index, expiry):
data = f"{jid}:{index}:{expiry}".encode()
return hmac.new(config["token"].encode(), data, hashlib.sha256).hexdigest()
@app.post("/v2/video_generation/{jid}/cancel", dependencies=[Depends(auth)])
async def cancel_video(jid: str):
# Only a video submitted through this authenticated bridge may be cancelled.
get_job(jid)
result = await upstream("POST", "/v1/jobs/" + quote(jid, safe="") + "/cancel", {})
return {"task": {"id": jid, "status": result.get("status", "running"),
"cancelRequested": result.get("cancelRequested", False)}}
@app.get("/v2/query/video_generation/{jid}", dependencies=[Depends(auth)])
async def poll(jid: str):
duration = get_job(jid)
result = await upstream("GET", "/v1/huobao/jobs/" + quote(jid, safe=""))
state = result.get("status", "running")
task = {"id": jid, "status": state, "duration": duration, "provider_id": result.get("providerId"), "routing": worker_routing(result.get("providerId")), "worker_state": result.get("worker_state"), "queue_position": result.get("queue_position")}
task["progress"] = result.get("progress")
if state == "succeeded":
output = next((o for o in result.get("outputs", []) if o.get("kind") == "video"), None)
if not output:
task.update(status="failed", error="Video task completed without a video output")
else:
index, expiry = int(output["index"]), int(time.time()) + 86400
url = f"{config['public_url']}/media/{quote(jid, safe='')}/{index}/video.mp4?expires={expiry}&sig={signature(jid, index, expiry)}"
task["content"] = {"url": url}
elif state in {"failed", "cancelled"}:
task["error"] = str(result.get("error") or "Video task failed")[:600]
return {"task": task}
@app.get("/media/{jid}/{index}")
@app.get("/media/{jid}/{index}/{filename}")
async def media(jid: str, index: int, expires: int, sig: str, filename: str | None = None):
if filename is not None and not re.fullmatch(r"image\.(png|jpg|webp)|video\.mp4", filename):
raise HTTPException(404, "Unknown media filename")
with closing(sqlite3.connect(db_path)) as db:
found = db.execute("SELECT 1 FROM jobs WHERE id=? UNION ALL SELECT 1 FROM image_jobs WHERE id=?", (jid, jid)).fetchone()
if not found:
raise HTTPException(404, "Unknown bridge task")
if expires < time.time() or not hmac.compare_digest(sig, signature(jid, index, expires)):
raise HTTPException(403, "Invalid or expired media link; poll the task again")
client = httpx.AsyncClient(timeout=httpx.Timeout(180, connect=10), trust_env=False)
url = config["h3"]["url"] + f"/v1/jobs/{quote(jid, safe='')}/outputs/{index}"
try:
response = await client.send(client.build_request("GET", url, headers=upstream_headers()), stream=True)
if response.is_error:
await response.aclose()
await client.aclose()
raise HTTPException(response.status_code, "Model output is not available")
except BaseException:
await client.aclose()
raise
async def download():
try:
async for part in response.aiter_bytes():
yield part
finally:
await response.aclose()
await client.aclose()
return StreamingResponse(download(), media_type=response.headers.get("content-type", "video/mp4"))
from video_cleanup_api import install_video_cleanup
install_video_cleanup(app, auth, db_path)
return app
def create_from_file():
path = Path(os.environ.get("HUOBAO_BRIDGE_CONFIG", "/bridge/config.json"))
return create_app(json.loads(path.read_text(encoding="utf-8")))
@@ -0,0 +1,47 @@
const {chromium}=require('playwright-core');
const assert=require('node:assert/strict');
const path=require('node:path');
(async()=>{
const browser=await chromium.launch({channel:'chrome',headless:true});
const page=await browser.newPage({viewport:{width:1920,height:1200}});
const errors=[],submissions=[];
try {
page.on('pageerror',e=>errors.push(e.message));page.on('dialog',d=>d.accept());
await page.addInitScript(()=>localStorage.setItem('huobao:tours',JSON.stringify(['episode'])));
await page.route('**/api/v1/tasks',async route=>{
if(route.request().method()!=='POST')return route.continue();
submissions.push(route.request().postDataJSON());
await route.fulfill({status:400,contentType:'application/json',body:JSON.stringify({message:'测试拦截:未提交模型'})});
});
await page.goto('http://127.0.0.1:18970/drama/1/episode/1',{waitUntil:'networkidle'});
const close=page.locator('.driver-popover-close-btn');if(await close.isVisible())await close.click();
await page.getByText('视频制作',{exact:true}).first().click();
await page.locator('.video-task-row').filter({has:page.getByText('#23',{exact:true})}).click();
const model=page.locator('.video-inspector-settings .model-select').filter({hasText:'当前镜头模型'});
await model.getByRole('button').click();
await page.locator('.model-select-option-name').filter({hasText:/LTX-2\.3$/}).click();
const panel=page.getByTestId('composed-first-frame');await panel.waitFor();
assert.match(await panel.innerText(),/15 个绑定素材 · 3 轮融合/);
assert.match(await panel.innerText(),/qwen_image_2_1 · GPU1/);
assert.match(await panel.innerText(),/404宿舍.*Girl.*Lamp/s);
assert.equal(await panel.locator('img').evaluate(img=>img.complete && img.naturalWidth>0),true);
const hint=page.getByTestId('shot-reference-limit');assert.match(await hint.innerText(),/全部 15 个绑定素材/);
assert.match(await hint.innerText(),/超过 6 张/);
await panel.scrollIntoViewIfNeeded();await page.screenshot({path:path.join(process.env.COMPOSE_FIXTURE,'composed-frame-ui.png')});
await page.getByRole('button',{name:'重做首帧并生成视频',exact:true}).click();
await page.waitForFunction(()=>!document.querySelector('.video-inspector-action').disabled);
assert.equal(submissions.length,1);assert.equal(submissions[0].regenerate_first_frame,true);
assert.equal(submissions[0].reference_image_urls.length,15);
// Select an untouched shot and prove Wan's actual outgoing request includes all 4 bindings.
await page.locator('.video-task-row').filter({has:page.getByText('#19',{exact:true})}).click();
await model.getByRole('button').click();await page.locator('.model-select-option-name').filter({hasText:/Wan2\.2-I2V$/}).click();
await page.locator('.video-inspector-action').click();
await page.waitForFunction(()=>!document.querySelector('.video-inspector-action').disabled);
assert.equal(submissions.length,2);assert.equal(submissions[1].model,'Wan2.2-I2V');
assert.equal(submissions[1].reference_image_urls.length,4);assert.equal(submissions[1].regenerate_first_frame,false);
assert.match(submissions[1].prompt,/@404宿舍/);assert.doesNotMatch(submissions[1].prompt,/@图片/);
assert.deepEqual(errors,[]);
console.log('PASS UI: full bindings for LTX/Wan, frame preview and provenance, explicit remake, no real model requests');
}catch(e){await page.screenshot({path:path.join(process.env.COMPOSE_FIXTURE,'ui-failure.png')});throw e}
finally{await browser.close()}
})().catch(e=>{console.error(e.stack);process.exit(1)});
@@ -0,0 +1,39 @@
const { chromium } = require('playwright-core');
const assert = require('node:assert/strict');
const path = require('node:path');
(async () => {
const browser = await chromium.launch({channel:'chrome', headless:true});
try {
const page = await browser.newPage({viewport:{width:1600,height:1120}});
const errors=[];
page.on('pageerror',e=>errors.push(e.message));
await page.addInitScript(()=>localStorage.setItem('huobao:tours',JSON.stringify(['episode'])));
await page.route('**/api/v1/tasks',route=>route.request().method()==='POST'?route.abort():route.continue());
await page.goto('http://127.0.0.1:18950/drama/1/episode/1',{waitUntil:'networkidle'});
const close=page.locator('.driver-popover-close-btn');
if(await close.isVisible()) await close.click();
await page.getByText('视频制作',{exact:true}).first().click();
const form=page.locator('.scene-lock-panel');
try { await form.waitFor({timeout:8000}); }
catch(e) { await page.screenshot({path:path.join(process.env.SCENE_FIXTURE,'ui-failure.png')}); console.log('UI:',(await page.locator('body').innerText()).slice(0,2400),errors); throw e; }
assert.match(await form.innerText(),/Dormitory/);
assert.match(await form.innerText(),/所有分镜共用/);
await form.getByLabel('空间与固定陈设').fill('Blue plaster walls; east window; oak desk below the window.');
assert.match(await form.innerText(),/未保存/);
await form.getByRole('button',{name:'保存场景设定',exact:true}).click();
await page.waitForFunction(()=>document.querySelector('.scene-lock-panel .actions')?.textContent.includes('已保存'));
const value=await page.evaluate(async()=> (await (await fetch('/api/v1/storyboards/1/environment')).json()).data);
assert.equal(value.current.environment,'Blue plaster walls; east window; oak desk below the window.');
assert.equal(value.stale,true);
await page.reload({waitUntil:'networkidle'});
await page.getByText('视频制作',{exact:true}).first().click();
await form.waitFor();
assert.equal(await form.getByLabel('空间与固定陈设').inputValue(),'Blue plaster walls; east window; oak desk below the window.');
await form.scrollIntoViewIfNeeded();
await page.screenshot({path:path.join(process.env.SCENE_FIXTURE,'scene-lock-ui.png')});
await page.setViewportSize({width:1280,height:850});
assert.equal(await form.evaluate(e=>e.scrollWidth<=e.clientWidth+2),true);
assert.deepEqual(errors,[]);
console.log('PASS: scene panel, shared setting save, stale version warning, reload persistence, narrow layout; no real generation');
} finally {await browser.close();}
})().catch(e=>{console.error(e.stack);process.exit(1)});
@@ -0,0 +1,22 @@
const {chromium}=require('playwright-core');const assert=require('node:assert/strict');const path=require('node:path');
(async()=>{
const browser=await chromium.launch({channel:'chrome',headless:true});
try {
const page=await browser.newPage({viewport:{width:1600,height:1100}}),errors=[];page.on('pageerror',e=>errors.push(e.message));
await page.goto('http://127.0.0.1:18986/drama/1/episode/1',{waitUntil:'networkidle'});
await page.getByText('视频制作',{exact:true}).first().click();
const row=id=>page.locator(`[data-storyboard-id="${id}"]`),dialog=page.getByRole('alertdialog');
await page.getByRole('button',{name:'删除分镜 5',exact:true}).click();await dialog.waitFor();
assert.match(await dialog.innerText(),/#05/);await dialog.getByRole('button',{name:'取消',exact:true}).click();assert.equal(await row(5).count(),1);
await page.getByRole('button',{name:'删除分镜 2',exact:true}).click();await dialog.getByRole('button',{name:'确认删除',exact:true}).click();
await page.getByText(/任务 #2 正在生成/).waitFor();assert.equal(await row(2).count(),1);await dialog.getByRole('button',{name:'取消',exact:true}).click();
await row(5).click();await page.getByRole('button',{name:'删除分镜 5',exact:true}).click();await dialog.getByRole('button',{name:'确认删除',exact:true}).click();await row(5).waitFor({state:'detached'});
assert.match(await row(6).innerText(),/#06/);
await page.getByRole('button',{name:'选择',exact:true}).click();await row(6).click();await row(7).click();
await page.getByTestId('delete-selected-storyboards').click();assert.match(await dialog.innerText(),/#06、#07/);
await dialog.getByRole('button',{name:'确认删除',exact:true}).click();await row(6).waitFor({state:'detached'});await row(7).waitFor({state:'detached'});
await page.reload({waitUntil:'networkidle'});assert.equal(await row(5).count(),0);assert.equal(await row(6).count(),0);assert.equal(await row(7).count(),0);assert.equal(await row(2).count(),1);
await page.screenshot({path:path.join(process.env.STORYBOARD_FIXTURE,'storyboard-delete-ui.png')});assert.deepEqual(errors,[]);
console.log('PASS UI: single delete confirmation, cancel, active rejection, batch, stable numbers, reload, no page errors');
} finally {await browser.close()}
})().catch(e=>{console.error(e.stack);process.exit(1)});
@@ -0,0 +1,17 @@
const {chromium}=require('playwright-core');const assert=require('node:assert/strict');const path=require('node:path');
(async()=>{
const browser=await chromium.launch({channel:'chrome',headless:true});
try {
const page=await browser.newPage({viewport:{width:1600,height:1100}}),errors=[];page.on('pageerror',e=>errors.push(e.message));
await page.goto('http://127.0.0.1:18984/drama/1/episode/1',{waitUntil:'networkidle'});
const close=page.locator('.driver-popover-close-btn');if(await close.isVisible())await close.click();
await page.getByText('视频制作',{exact:true}).first().click();
const panel=page.getByTestId('storyboard-job-status');await panel.waitFor();
assert.match(await panel.innerText(),/已校验 4 个分镜/);await page.getByTestId('cancel-storyboard-job').waitFor();
await page.reload({waitUntil:'networkidle'});if(await close.isVisible())await close.click();
if(!await panel.isVisible())await page.getByText('视频制作',{exact:true}).first().click();
assert.match(await panel.innerText(),/已校验 4 个分镜/);assert.deepEqual(errors,[]);
await page.screenshot({path:path.join(process.env.STORYBOARD_FIXTURE,'storyboard-progress.png')});
console.log('PASS UI: progress visible, cancel available, reload retains active job, no page errors');
} finally {await browser.close()}
})().catch(e=>{console.error(e.stack);process.exit(1)});
@@ -0,0 +1,25 @@
const {chromium}=require('playwright-core');
const assert=require('node:assert/strict');
const path=require('node:path');
(async()=>{
const browser=await chromium.launch({channel:'chrome',headless:true});
try{
const page=await browser.newPage({viewport:{width:1600,height:1100}}),errors=[];
let polls=0;
page.on('pageerror',e=>errors.push(e.message));
page.on('request',r=>{if(r.url().includes('/episodes/1/generation-tasks'))polls++});
await page.addInitScript(()=>localStorage.setItem('huobao:tours',JSON.stringify(['episode'])));
await page.goto('http://127.0.0.1:18980/drama/1/episode/1',{waitUntil:'networkidle'});
const close=page.locator('.driver-popover-close-btn');if(await close.isVisible())await close.click();
await page.getByText('任务',{exact:true}).first().click();
const states=page.getByTestId('video-remote-cleanup');await states.first().waitFor();
const text=(await states.allTextContents()).join('\n');
assert.match(text,/任务记录和视频源文件已清理/);assert.match(text,/暂未清理,将自动重试/);
const before=polls,deadline=Date.now()+10000;
while(polls<=before && Date.now()<deadline)await new Promise(r=>setTimeout(r,100));
assert.ok(polls>before,'Completed tasks with pending cleanup keep polling');
assert.deepEqual(errors,[]);
await page.screenshot({path:path.join(process.env.CLEANUP_FIXTURE,'cleanup-status-ui.png')});
console.log('PASS UI: completed video cleanup status, independent retry note, polling continues without running video jobs');
}finally{await browser.close()}
})().catch(e=>{console.error(e.stack);process.exit(1)});
@@ -0,0 +1,48 @@
const { chromium } = require('playwright-core');
const assert = require('node:assert/strict');
const path = require('node:path');
(async () => {
const browser = await chromium.launch({channel:'chrome', headless:true});
try {
const page = await browser.newPage({viewport:{width:1600,height:1120}});
const errors=[], submissions=[];
page.on('pageerror',e=>errors.push(e.message));
page.on('dialog',d=>d.accept());
await page.addInitScript(()=>localStorage.setItem('huobao:tours',JSON.stringify(['episode'])));
await page.route('**/api/v1/tasks',async route=>{
if(route.request().method()!=='POST') return route.continue();
submissions.push(route.request().postDataJSON());
// Inspect actual browser payload without starting a generation even in this test fixture.
await route.fulfill({status:400,contentType:'application/json',body:JSON.stringify({message:'测试拦截:未提交模型'})});
});
await page.goto('http://127.0.0.1:18950/drama/1/episode/1',{waitUntil:'networkidle'});
const close=page.locator('.driver-popover-close-btn');
if(await close.isVisible()) await close.click();
await page.getByText('视频制作',{exact:true}).first().click();
await page.locator('.video-task-row').filter({hasText:'Reference fixture'}).click();
const model=page.locator('.video-inspector-settings .model-select').filter({hasText:'当前镜头模型'});
await model.getByRole('button').click();
await page.locator('.model-select-option-name').filter({hasText:/^LTX-2\.3 ·/}).click();
const hint=page.getByTestId('shot-reference-limit');
await hint.waitFor();
assert.match(await hint.innerText(),/全部 3 个绑定素材/);
await page.locator('.video-inspector-action').click();
await page.waitForFunction(()=>!document.querySelector('.video-inspector-action').disabled);
assert.equal(submissions.length,1);
assert.equal(submissions[0].model,'LTX-2.3');
assert.deepEqual(submissions[0].reference_image_urls,['static/images/scene.png','static/images/girl.png','static/images/other.png']);
assert.equal(submissions[0].prompt,'0-4s: @Dormitory, @Girl looks at @Other.');
await hint.scrollIntoViewIfNeeded();
await page.screenshot({path:path.join(process.env.SCENE_FIXTURE,'ltx-reference-ui.png')});
// Per-shot switch back restores all bound images; the episode default stays H3 throughout.
await model.getByRole('button').click();
await page.locator('.model-select-option-name').filter({hasText:/^MiniMax-H3 ·/}).click();
await page.locator('.video-inspector-action').click();
await page.waitForFunction(()=>!document.querySelector('.video-inspector-action').disabled);
assert.equal(submissions.length,2);
assert.deepEqual(submissions[1].reference_image_urls,['static/images/scene.png','static/images/girl.png','static/images/other.png']);
assert.equal(submissions[1].prompt,'0-4s: @图片1Dormitory, @图片2Girl looks at @图片3Other.');
assert.deepEqual(errors,[]);
console.log('PASS: LTX sends all 3 bindings for first-frame composition, H3 retains direct refs; no model jobs');
} finally {await browser.close();}
})().catch(e=>{console.error(e.stack);process.exit(1)});
+600
View File
@@ -0,0 +1,600 @@
"""Private adapters for Huobao: Dify, Qwen Image, local H3, and Grok CLI."""
from __future__ import annotations
import asyncio
import base64
from contextlib import closing
from email import policy
from email.parser import BytesParser
import hashlib
import hmac
import json
import math
import os
from pathlib import Path
import re
import secrets
import sqlite3
import struct
import tempfile
import time
from urllib.parse import quote
import httpx
from fastapi import Depends, FastAPI, Header, HTTPException, Request
from fastapi.responses import JSONResponse, StreamingResponse
def chat_query(body):
messages = body.get("messages")
if not isinstance(messages, list) or not messages:
raise HTTPException(400, "messages must be a non-empty array")
if body.get("tool_choice") == "required" or isinstance(body.get("tool_choice"), dict):
raise HTTPException(400, "This chatflow does not support required tool calls; use DeepSeek")
parts = []
for item in messages:
content = item.get("content", "")
if isinstance(content, list):
if any(p.get("type") != "text" for p in content):
raise HTTPException(400, "This adapter accepts text messages only")
content = "\n".join(p.get("text", "") for p in content)
if content:
parts.append(f"[{item.get('role', 'user')}]\n{content}")
fmt = body.get("response_format") or {}
if fmt.get("type") == "json_schema":
parts.append("Return only JSON matching this schema:\n" + json.dumps(fmt.get("json_schema", {}).get("schema", {}), ensure_ascii=False))
elif fmt.get("type") == "json_object":
parts.append("Return only a valid JSON object, without Markdown fences.")
if not parts:
raise HTTPException(400, "Empty messages")
return "\n\n".join(parts)
def image_dimensions(data_url):
try:
data = base64.b64decode(data_url.split(",", 1)[1], validate=True)
if data.startswith(b"\x89PNG\r\n\x1a\n"):
return struct.unpack(">II", data[16:24])
if data.startswith((b"GIF87a", b"GIF89a")):
return struct.unpack("<HH", data[6:10])
if data[:2] == b"\xff\xd8":
offset = 2
while offset + 4 < len(data):
if data[offset] != 255:
offset += 1
continue
while data[offset] == 255:
offset += 1
marker = data[offset]
offset += 1
if marker in {0xD8, 0xD9} or 0xD0 <= marker <= 0xD7:
continue
size = int.from_bytes(data[offset:offset + 2], "big")
if marker in {0xC0, 0xC1, 0xC2, 0xC3, 0xC5, 0xC6, 0xC7, 0xC9, 0xCA, 0xCB, 0xCD, 0xCE, 0xCF}:
height, width = struct.unpack(">HH", data[offset + 3:offset + 7])
return width, height
if size < 2:
break
offset += size
if data[:4] == b"RIFF" and data[8:12] == b"WEBP":
if data[12:16] == b"VP8X":
return 1 + int.from_bytes(data[24:27], "little"), 1 + int.from_bytes(data[27:30], "little")
if data[12:16] == b"VP8L" and data[20] == 0x2F:
bits = int.from_bytes(data[21:25], "little")
return (bits & 0x3FFF) + 1, ((bits >> 14) & 0x3FFF) + 1
if data[12:16] == b"VP8 " and data[23:26] == b"\x9d\x01\x2a":
width, height = struct.unpack("<HH", data[26:30])
return width & 0x3FFF, height & 0x3FFF
except (ValueError, IndexError, struct.error):
pass
raise HTTPException(400, "Unable to read first-frame dimensions; use PNG, JPEG or WebP")
VIDEO_MODELS = {
"MiniMax-H3-VDN-H3": {"reference": "minimax_h3_vdn_hd", "min_images": 1, "max_images": 9},
"MiniMax-H3": {"text": "minimax_h3_t2v_official", "first": "minimax_h3_i2v", "reference": "minimax_h3_r2v", "min_images": 0, "max_images": 9},
"MiniMax-H3-Hybrid": {"text": "minimax_h3_hybrid_v0331_t2v", "first": "minimax_h3_hybrid_v0331_i2v", "reference": "minimax_h3_hybrid_v0331_i2v", "min_images": 0, "max_images": 9},
"MiniMax-H3-R2V-4Step": {"reference": "minimax_h3_r2v_4step", "min_images": 1, "max_images": 9},
"LTX-2.3": {"text": "ltx23_t2v", "first": "ltx23_t2v", "reference": "ltx23_t2v", "min_images": 0, "max_images": 1},
"LTX-2.3-Text-Preserve": {"text": "ltx23_t2v_text_preserve", "first": "ltx23_t2v_text_preserve", "reference": "ltx23_t2v_text_preserve", "min_images": 0, "max_images": 1},
"Wan2.2-I2V": {"first": "wan22_simple", "reference": "wan22_simple", "min_images": 1, "max_images": 1},
}
def video_workflows(spec):
return {spec[mode] for mode in ("text", "first", "reference") if mode in spec}
def gpu_routing(body):
gpu = body.get("gpu", "auto")
if gpu not in ("auto", "gpu1", "gpu2", "gpu3"):
raise HTTPException(400, "gpu must be auto, gpu1, gpu2 or gpu3")
if gpu == "auto":
return {"provider_id": "default", "huobao_panels": True}
return {"provider_id": {"gpu1": "gpu1_8189", "gpu2": "default", "gpu3": "gpu3"}[gpu], "strict_provider": True, "huobao_panels": True}
def worker_routing(provider_id):
worker = {"default": ("gpu2", 8188), "gpu1_8189": ("gpu1", 8189), "gpu3": ("gpu3", 8191), "gpu1": ("gpu1", None)}.get(provider_id)
if not worker:
return None
gpu, port = worker
return {"gpu": gpu, "port": port, "providerId": provider_id, "legacy": provider_id == "gpu1"}
def h3_request(body):
"""Translate the local video protocol. 'default' means upstream auto-balance."""
model = body.get("model", "MiniMax-H3")
spec = VIDEO_MODELS.get(model)
if spec is None:
raise HTTPException(400, "Unknown local video model; query /v1/models")
content = body.get("content") or []
if not isinstance(content, list):
raise HTTPException(400, "content must be an array")
prompt = "\n".join(str(c.get("text", "")) for c in content if c.get("type") == "text").strip()
if not prompt or len(prompt) > 7000:
raise HTTPException(400, "A prompt of 1-7000 characters is required")
images, roles = [], []
for item in content:
kind = item.get("type")
if kind == "text":
continue
if kind != "image_url" or item.get("role") == "last_frame":
raise HTTPException(400, "Local video supports first-frame/reference images; last frames, reference video and voice samples are unsupported")
raw = item.get("image_url") or {}
url = raw.get("url", "") if isinstance(raw, dict) else raw
if not isinstance(url, str) or not url.startswith("data:image/"):
raise HTTPException(400, "Local video requires inline image data; enable Huobao's local-image conversion")
images.append(url)
roles.append(item.get("role", "reference_image"))
if not spec["min_images"] <= len(images) <= spec["max_images"]:
raise HTTPException(400, f"{model} requires {spec['min_images']}-{spec['max_images']} images")
if "first_frame" in roles and len(images) != 1:
raise HTTPException(400, "Use either one fixed first frame or reference images; these modes cannot be combined")
mode = "text" if not images else "first" if roles == ["first_frame"] else "reference"
if mode not in spec:
raise HTTPException(400, f"{model} does not support {mode} mode; use reference images")
try:
duration = float(body.get("duration", 5))
except (ValueError, TypeError):
raise HTTPException(400, "Invalid duration")
if not math.isfinite(duration) or not 4 <= duration <= 15:
raise HTTPException(400, "Duration must be 4-15 seconds")
ratio = body.get("ratio", "16:9")
if ratio not in {"16:9", "9:16", "1:1", "4:3", "3:4", "21:9"}:
raise HTTPException(400, "Unsupported aspect ratio")
resolution = str(body.get("resolution", "768P")).upper()
resolutions = {"480P": 480, "720P": 720, "1080P": 1080, "768P": 768, "2K": 1152}
if model == "MiniMax-H3-VDN-H3":
if ratio not in {"16:9", "9:16"}:
raise HTTPException(400, "VDN HD supports 16:9 or 9:16")
resolutions = {"2K": 1440, "4K": 2160}
if resolution not in resolutions:
raise HTTPException(400, "Unsupported video resolution")
short = resolutions[resolution]
a, b = map(int, ratio.split(":"))
if roles == ["first_frame"] and "ratio" not in body:
a, b = image_dimensions(images[0])
if a <= 0 or b <= 0:
raise HTTPException(400, "Invalid first-frame dimensions")
width = round(short * max(1, a / b) / 64) * 64
height = round(short * max(1, b / a) / 64) * 64
if model == "MiniMax-H3-VDN-H3":
width, height = (short * 16 // 9, short) if a > b else (short, short * 16 // 9)
workflow = spec[mode]
request = {**gpu_routing(body), "workflow_key": workflow, "prompt": prompt,
"input_images": images, "duration": duration, "width": width, "height": height,
"metadata": {"source": "huobao-drama", "model": model}}
if body.get("seed") is not None:
if not isinstance(body["seed"], int) or not 0 <= body["seed"] < 2**63:
raise HTTPException(400, "Invalid seed")
request["seed"] = body["seed"]
return request
def qwen_image_request(body, images=None):
if body.get("model") != "qwen_image_2_1":
raise HTTPException(400, "Use model qwen_image_2_1")
prompt = body.get("prompt")
if not isinstance(prompt, str) or not prompt.strip() or len(prompt) > 32000:
raise HTTPException(400, "A prompt of 1-32000 characters is required")
if str(body.get("n", 1)) != "1":
raise HTTPException(400, "This image adapter supports n=1 per task")
if body.get("response_format", "url") != "url":
raise HTTPException(400, "Asynchronous image tasks require response_format=url")
size = str(body.get("size", "1024x1024")).lower()
if size == "auto":
size = "1024x1024"
match = re.fullmatch(r"(\d{3,4})x(\d{3,4})", size)
if not match or not all(256 <= int(d) <= 4096 for d in match.groups()):
raise HTTPException(400, "size must be WIDTHxHEIGHT with dimensions 256-4096")
width, height = map(int, match.groups())
refs = images or []
if len(refs) > 6:
raise HTTPException(400, "Qwen Image 2.1 supports at most 6 reference images")
return {**gpu_routing(body), "workflow_key": "qwen_image_2_1", "prompt": prompt.strip(),
"width": width, "height": height, "batch_size": 1, "input_images": refs,
"metadata": {"source": "huobao-drama", "output_type": "image"}}
async def parse_image_edit(request):
content_type = request.headers.get("content-type", "")
if not content_type.lower().startswith("multipart/form-data;"):
raise HTTPException(415, "Image edits require multipart/form-data")
raw = bytearray()
async for chunk in request.stream():
raw.extend(chunk)
if len(raw) > 64 * 1024 * 1024:
raise HTTPException(413, "Reference uploads exceed 64 MiB")
message = BytesParser(policy=policy.default).parsebytes(
("Content-Type: " + content_type + "\r\nMIME-Version: 1.0\r\n\r\n").encode() + raw)
if not message.is_multipart() or message.defects:
raise HTTPException(400, "Invalid multipart image request")
fields, images = {}, []
for part in message.iter_parts():
name = part.get_param("name", header="content-disposition")
data = part.get_payload(decode=True)
if not isinstance(data, bytes):
raise HTTPException(400, "Invalid multipart field")
if name in {"image", "image[]"}:
mime = part.get_content_type()
if mime not in {"image/png", "image/jpeg", "image/webp"} or not data or len(data) > 10 * 1024 * 1024:
raise HTTPException(400, "Each reference must be PNG, JPEG or WebP, up to 10 MiB")
url = "data:" + mime + ";base64," + base64.b64encode(data).decode()
width, height = image_dimensions(url)
if not (0 < width <= 16384 and 0 < height <= 16384):
raise HTTPException(400, "Invalid reference image dimensions")
images.append(url)
if len(images) > 6:
raise HTTPException(400, "Qwen Image 2.1 supports at most 6 reference images")
elif name in {"model", "prompt", "size", "n", "response_format", "gpu"}:
if name in fields or len(data) > 128000:
raise HTTPException(400, "Duplicate or oversized form field")
try:
fields[name] = data.decode("utf-8")
except UnicodeDecodeError:
raise HTTPException(400, "Form text must be UTF-8")
else:
raise HTTPException(400, "Unsupported image edit field: " + str(name)[:60])
if not images:
raise HTTPException(400, "At least one reference image is required")
return qwen_image_request(fields, images)
def create_app(config):
app = FastAPI(title="Huobao private model bridge", docs_url=None, redoc_url=None)
state_dir = Path(config.get("state_dir", "./state"))
state_dir.mkdir(parents=True, exist_ok=True)
db_path = state_dir / "jobs.sqlite3"
with closing(sqlite3.connect(db_path)) as db, db:
db.execute("CREATE TABLE IF NOT EXISTS jobs (id TEXT PRIMARY KEY, duration REAL NOT NULL, created REAL NOT NULL)")
db.execute("CREATE TABLE IF NOT EXISTS image_jobs (id TEXT PRIMARY KEY, created REAL NOT NULL)")
def auth(authorization: str | None = Header(default=None)):
expected = config.get("token", "")
if not expected or not secrets.compare_digest(authorization or "", "Bearer " + expected):
raise HTTPException(401, "Invalid bridge token")
@app.exception_handler(HTTPException)
async def http_error(request, exc):
return JSONResponse({"error": {"message": str(exc.detail), "type": "bridge_error", "code": exc.status_code}}, status_code=exc.status_code)
@app.exception_handler(httpx.HTTPError)
async def upstream_error(request, exc):
return JSONResponse({"error": {"message": "Upstream connection failed: " + type(exc).__name__, "type": "upstream_error"}}, status_code=502)
def upstream_headers():
return {"Authorization": "Bearer " + config["h3"]["token"]}
async def upstream(method, path, body=None):
if not config.get("h3"):
raise HTTPException(404, "H3 is not configured on this bridge")
async with httpx.AsyncClient(timeout=httpx.Timeout(180, connect=10), trust_env=False) as client:
r = await client.request(method, config["h3"]["url"] + path, headers=upstream_headers(), json=body)
if r.is_error:
try:
detail = r.json().get("detail", "Model service request failed")
except ValueError:
detail = "Model service request failed"
raise HTTPException(r.status_code, str(detail)[:600])
return r.json()
@app.get("/health")
async def health():
return {"status": "ok", "service": "huobao-model-bridge"}
@app.get("/v1/models", dependencies=[Depends(auth)])
async def models():
names = list(config.get("dify", {}))
if config.get("h3"):
# Verify the real service, instead of making a local catalogue look healthy.
registry = await upstream("GET", "/v1/models")
available = {m["id"] for m in registry.get("data", [])}
names.extend(name for name, spec in VIDEO_MODELS.items() if video_workflows(spec) <= available)
if any(m.get("id") == "qwen_image_2_1" and m.get("model_type") == "image" for m in registry.get("data", [])):
names.append("qwen_image_2_1")
if config.get("grok"):
names.append("grok-local")
return {"object": "list", "data": [{"id": n, "object": "model", "owned_by": "private"} for n in names]}
async def dify_events(model, query):
upstream_config = config["dify"][model]
payload = {"inputs": {}, "query": query, "response_mode": "streaming",
"user": "huobao-" + secrets.token_hex(8), "auto_generate_name": False}
finished, usage = False, {}
async with httpx.AsyncClient(timeout=httpx.Timeout(240, connect=10), trust_env=False) as client:
async with client.stream("POST", upstream_config["url"] + "/chat-messages",
headers={"Authorization": "Bearer " + upstream_config["key"]}, json=payload) as response:
if response.is_error:
await response.aread()
try:
message = response.json().get("message", "Dify request failed")
except ValueError:
message = "Dify request failed"
raise HTTPException(response.status_code, str(message)[:400])
async for line in response.aiter_lines():
if not line.startswith("data:"):
continue
event = json.loads(line[5:].strip())
kind = event.get("event")
if kind in {"message", "agent_message"}:
yield "text", event.get("answer", "")
elif kind == "error":
raise HTTPException(502, str(event.get("message", "Dify generation failed"))[:400])
elif kind == "message_end":
finished = True
usage = event.get("metadata", {}).get("usage", {})
elif kind == "workflow_finished":
data = event.get("data") or {}
if data.get("status") not in {None, "succeeded"}:
raise HTTPException(502, str(data.get("error") or "Dify workflow failed")[:400])
if not finished:
raise HTTPException(502, "Dify stream ended before message_end")
yield "usage", usage
async def grok_events(query):
item = config["grok"]
args = [item["exe"], "--cwd", item["cwd"], "--output-format", "json", "--verbatim",
"--no-subagents", "--no-memory", "--disable-web-search", "--max-turns", "1",
"--tools", "", "--deny", "MCPTool", "--permission-mode", "dontAsk",
"--system-prompt-override", "Answer the supplied writing request using text only. Do not use tools."]
options = {"creationflags": 0x08000000} if os.name == "nt" else {}
# A file avoids the Windows command-line length limit for full scripts.
with tempfile.TemporaryDirectory(prefix="huobao-grok-") as scratch:
prompt_file = Path(scratch) / "prompt.txt"
prompt_file.write_text(query, encoding="utf-8")
args += ["--prompt-file", str(prompt_file)]
process = await asyncio.create_subprocess_exec(*args, stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.PIPE, **options)
try:
out, err = await asyncio.wait_for(process.communicate(), timeout=240)
except (asyncio.TimeoutError, asyncio.CancelledError):
process.kill()
await process.wait()
raise
try:
data = json.loads(out.decode("utf-8"))
except (ValueError, UnicodeDecodeError):
raise HTTPException(502, "Grok CLI did not return JSON; run Grok login on this computer")
if process.returncode or not data.get("text"):
raise HTTPException(502, "Grok CLI failed; check local login and available quota")
yield "text", data["text"]
yield "usage", data.get("usage", {})
@app.post("/v1/chat/completions", dependencies=[Depends(auth)])
async def chat(body: dict):
model = body.get("model", "")
if model not in config.get("dify", {}) and not (model == "grok-local" and config.get("grok")):
raise HTTPException(404, "Unknown text model")
query = chat_query(body)
events = grok_events(query) if model == "grok-local" else dify_events(model, query)
cid, created = "chatcmpl-" + secrets.token_hex(12), int(time.time())
def chunk(delta, finish=None):
return {"id": cid, "object": "chat.completion.chunk", "created": created, "model": model,
"choices": [{"index": 0, "delta": delta, "finish_reason": finish}]}
if body.get("stream"):
async def stream():
def encode(value):
return "data: " + json.dumps(value, ensure_ascii=False) + "\n\n"
yield encode(chunk({"role": "assistant", "content": ""}))
try:
async for kind, value in events:
if kind == "text":
yield encode(chunk({"content": value}))
yield encode(chunk({}, "stop"))
except (HTTPException, httpx.HTTPError) as error:
message = str(error.detail) if isinstance(error, HTTPException) else "Upstream connection failed"
yield encode({"error": {"message": message, "type": "upstream_error"}})
yield "data: [DONE]\n\n"
return StreamingResponse(stream(), media_type="text/event-stream", headers={"Cache-Control": "no-cache"})
text, usage = "", {}
async for kind, value in events:
if kind == "text":
text += value
else:
usage = value
return {"id": cid, "object": "chat.completion", "created": created, "model": model,
"choices": [{"index": 0, "message": {"role": "assistant", "content": text}, "finish_reason": "stop"}],
"usage": {"prompt_tokens": usage.get("prompt_tokens", usage.get("input_tokens", 0)),
"completion_tokens": usage.get("completion_tokens", usage.get("output_tokens", 0)),
"total_tokens": usage.get("total_tokens", 0)}}
async def require_qwen_image():
registry = await upstream("GET", "/v1/models")
if not any(m.get("id") == "qwen_image_2_1" and m.get("model_type") == "image" for m in registry.get("data", [])):
raise HTTPException(503, "Qwen Image 2.1 workflow is unavailable")
@app.get("/v1/images/pool", dependencies=[Depends(auth)])
async def image_pool():
await require_qwen_image()
health, catalogue = await asyncio.gather(upstream("GET", "/health"), upstream("GET", "/v1/providers"))
return {"model": "qwen_image_2_1", "auto_balance": health.get("autoBalance") is True,
"routing": "shortest-queue-with-reservations" if health.get("autoBalance") is True else "single-provider",
"configured_workers": [{"id": p["id"], "name": p.get("name", p["id"])} for p in catalogue.get("providers", [])]}
async def submit_image(payload):
await require_qwen_image()
result = await upstream("POST", "/v1/jobs", payload)
jid = result.get("id")
if not isinstance(jid, str) or not jid:
raise HTTPException(502, "Image service did not return a task ID")
with closing(sqlite3.connect(db_path)) as db, db:
db.execute("INSERT INTO image_jobs VALUES(?,?)", (jid, time.time()))
return {"task_id": jid, "status": "processing", "provider_id": result.get("providerId")}
@app.post("/v1/images/validate", dependencies=[Depends(auth)])
async def validate_image(body: dict):
payload = qwen_image_request(body)
await require_qwen_image()
return {"valid": True, "workflow": payload["workflow_key"], "requested_size": [payload["width"], payload["height"]], "max_reference_images": 6,
"pool": await image_pool()}
@app.post("/v1/images/generations", dependencies=[Depends(auth)])
async def generate_image(body: dict):
return await submit_image(qwen_image_request(body))
@app.post("/v1/images/edits", dependencies=[Depends(auth)])
async def edit_image(request: Request):
return await submit_image(await parse_image_edit(request))
@app.post("/v1/images/task/{jid}/cancel", dependencies=[Depends(auth)])
async def cancel_image(jid: str):
with closing(sqlite3.connect(db_path)) as db:
found = db.execute("SELECT 1 FROM image_jobs WHERE id=?", (jid,)).fetchone()
if not found:
raise HTTPException(404, "Unknown image task")
result = await upstream("POST", "/v1/jobs/" + quote(jid, safe="") + "/cancel", {})
return {"task": {"id": jid, "status": result.get("status", "running"),
"cancelRequested": result.get("cancelRequested", False)}}
@app.get("/v1/images/task/{jid}", dependencies=[Depends(auth)])
async def poll_image(jid: str):
with closing(sqlite3.connect(db_path)) as db:
found = db.execute("SELECT 1 FROM image_jobs WHERE id=?", (jid,)).fetchone()
if not found:
raise HTTPException(404, "Unknown image task")
result = await upstream("GET", "/v1/huobao/jobs/" + quote(jid, safe=""))
state = result.get("status", "running")
if state in {"failed", "cancelled"}:
return {"task_id": jid, "status": "failed", "cancelled": state == "cancelled", "error": {"message": str(result.get("error") or "Image generation failed")[:600]}}
if state != "succeeded":
return {"task_id": jid, "status": "processing", "provider_id": result.get("providerId"), "worker_state": result.get("worker_state"), "queue_position": result.get("queue_position")}
output = next((o for o in result.get("outputs", []) if o.get("kind") == "image"), None)
if output is None:
return {"task_id": jid, "status": "failed", "error": {"message": "Task completed without an image output"}}
index, expiry = int(output["index"]), int(time.time()) + 86400
extension = {"image/png": "png", "image/jpeg": "jpg", "image/webp": "webp"}.get(output.get("mimeType"), "png")
# Huobao determines the saved extension from the URL, not Content-Type.
url = f"{config['public_url']}/media/{quote(jid, safe='')}/{index}/image.{extension}?expires={expiry}&sig={signature(jid, index, expiry)}"
return {"task_id": jid, "status": "completed", "image_url": url, "data": [{"url": url}], "provider_id": result.get("providerId")}
@app.post("/v2/video_generation/validate", dependencies=[Depends(auth)])
async def validate_video(body: dict):
request = h3_request(body)
registry = await upstream("GET", "/v1/models")
if request["workflow_key"] not in {m["id"] for m in registry.get("data", [])}:
raise HTTPException(503, "Selected video workflow is unavailable")
plan = await upstream("POST", "/v1/audio-plan", request)
route = await upstream("POST", "/v1/huobao/route", request)
return {"valid": plan.get("status", "ready") == "ready", "workflow": request["workflow_key"],
"width": request["width"], "height": request["height"], "duration": request["duration"], "audio_plan": plan,
"routing": worker_routing(route.get("providerId"))}
@app.post("/v2/video_generation", dependencies=[Depends(auth)])
async def video(body: dict):
# Huobao probes video providers with an empty body. Check the real
# upstream without submitting a GPU task, then report readiness.
if not body:
registry = await upstream("GET", "/v1/models")
available = {m["id"] for m in registry.get("data", [])}
if not {"minimax_h3_t2v_official", "minimax_h3_i2v", "minimax_h3_r2v"} <= available:
raise HTTPException(503, "Required H3 workflows are unavailable")
return {"status": "ready", "probe": True, "model": "MiniMax-H3"}
request = h3_request(body)
registry = await upstream("GET", "/v1/models")
if request["workflow_key"] not in {m["id"] for m in registry.get("data", [])}:
raise HTTPException(503, "Selected video workflow is unavailable")
result = await upstream("POST", "/v1/jobs", request)
jid = result.get("id")
if not isinstance(jid, str) or not jid:
raise HTTPException(502, "Video service did not return a job ID")
with closing(sqlite3.connect(db_path)) as db, db:
db.execute("INSERT INTO jobs VALUES(?,?,?)", (jid, request["duration"], time.time()))
return {"task_id": jid, "status": "submitted", "provider_id": result.get("providerId"), "routing": worker_routing(result.get("providerId"))}
def get_job(jid):
with closing(sqlite3.connect(db_path)) as db:
row = db.execute("SELECT duration FROM jobs WHERE id=?", (jid,)).fetchone()
if not row:
raise HTTPException(404, "Unknown bridge job")
return row[0]
def signature(jid, index, expiry):
data = f"{jid}:{index}:{expiry}".encode()
return hmac.new(config["token"].encode(), data, hashlib.sha256).hexdigest()
@app.post("/v2/video_generation/{jid}/cancel", dependencies=[Depends(auth)])
async def cancel_video(jid: str):
# Only a video submitted through this authenticated bridge may be cancelled.
get_job(jid)
result = await upstream("POST", "/v1/jobs/" + quote(jid, safe="") + "/cancel", {})
return {"task": {"id": jid, "status": result.get("status", "running"),
"cancelRequested": result.get("cancelRequested", False)}}
@app.get("/v2/query/video_generation/{jid}", dependencies=[Depends(auth)])
async def poll(jid: str):
duration = get_job(jid)
result = await upstream("GET", "/v1/huobao/jobs/" + quote(jid, safe=""))
state = result.get("status", "running")
task = {"id": jid, "status": state, "duration": duration, "provider_id": result.get("providerId"), "routing": worker_routing(result.get("providerId")), "worker_state": result.get("worker_state"), "queue_position": result.get("queue_position")}
task["progress"] = result.get("progress")
if state == "succeeded":
output = next((o for o in result.get("outputs", []) if o.get("kind") == "video"), None)
if not output:
task.update(status="failed", error="Video task completed without a video output")
else:
index, expiry = int(output["index"]), int(time.time()) + 86400
url = f"{config['public_url']}/media/{quote(jid, safe='')}/{index}/video.mp4?expires={expiry}&sig={signature(jid, index, expiry)}"
task["content"] = {"url": url}
elif state in {"failed", "cancelled"}:
task["error"] = str(result.get("error") or "Video task failed")[:600]
return {"task": task}
@app.get("/media/{jid}/{index}")
@app.get("/media/{jid}/{index}/{filename}")
async def media(jid: str, index: int, expires: int, sig: str, filename: str | None = None):
if filename is not None and not re.fullmatch(r"image\.(png|jpg|webp)|video\.mp4", filename):
raise HTTPException(404, "Unknown media filename")
with closing(sqlite3.connect(db_path)) as db:
found = db.execute("SELECT 1 FROM jobs WHERE id=? UNION ALL SELECT 1 FROM image_jobs WHERE id=?", (jid, jid)).fetchone()
if not found:
raise HTTPException(404, "Unknown bridge task")
if expires < time.time() or not hmac.compare_digest(sig, signature(jid, index, expires)):
raise HTTPException(403, "Invalid or expired media link; poll the task again")
client = httpx.AsyncClient(timeout=httpx.Timeout(180, connect=10), trust_env=False)
url = config["h3"]["url"] + f"/v1/jobs/{quote(jid, safe='')}/outputs/{index}"
try:
response = await client.send(client.build_request("GET", url, headers=upstream_headers()), stream=True)
if response.is_error:
await response.aclose()
await client.aclose()
raise HTTPException(response.status_code, "Model output is not available")
except BaseException:
await client.aclose()
raise
async def download():
try:
async for part in response.aiter_bytes():
yield part
finally:
await response.aclose()
await client.aclose()
return StreamingResponse(download(), media_type=response.headers.get("content-type", "video/mp4"))
return app
def create_from_file():
path = Path(os.environ.get("HUOBAO_BRIDGE_CONFIG", "/bridge/config.json"))
return create_app(json.loads(path.read_text(encoding="utf-8")))
@@ -0,0 +1,829 @@
from __future__ import annotations
import base64
import asyncio
import ipaddress
import json
import mimetypes
import os
import secrets
import socket
from pathlib import Path
from job_cancellation import dispatched as cancellation_dispatched, record as record_cancellation
from dataclasses import dataclass, replace
from typing import Any
from urllib.parse import quote, urlencode, urlsplit
import httpx
from fastapi import Depends, FastAPI, Header, HTTPException, Response
from fastapi.responses import StreamingResponse
from pydantic import BaseModel, Field
from workflow import MAX_VIDEO_DURATION_SECONDS, compile_workflow, load_registry, public_model, public_workflow, validate_h3_references
from h3_audio_intent import plan_h3_audio
from h3_audio_validation import AudioOutputValidator, attach_audio_policy
from h3_visual_guard import VISUAL_REVISION, validate_visual_dimensions
from h3_audio_conditioning import AudioConditioningError, prepare_audio_conditioning
from h3_audio_output_guard import AudioQualityError
@dataclass(frozen=True)
class Provider:
id: str
name: str
base_url: str
api_key: str = ""
control_token_file: str = ""
class JobRequest(BaseModel):
provider_id: str = "default"
strict_provider: bool = False
huobao_panels: bool = False
workflow_key: str
prompt: str
negative_prompt: str = ""
input_images: list[str] = Field(default_factory=list, max_length=9)
width: int | None = Field(default=None, ge=64, le=8192)
height: int | None = Field(default=None, ge=64, le=8192)
duration: float | None = Field(default=None, gt=0, le=MAX_VIDEO_DURATION_SECONDS)
generate_audio: bool | None = None
seed: int | None = Field(default=None, ge=0)
batch_size: int = Field(default=1, ge=1, le=4)
denoise: float | None = Field(default=None, ge=0, le=1)
metadata: dict[str, Any] = Field(default_factory=dict)
app = FastAPI(title="Canvas ComfyUI Workflow Adapter", version="1.0.0")
audio_validator = AudioOutputValidator()
provider_selection_lock = asyncio.Lock()
provider_reservations: dict[str, int] = {}
def auto_balance_enabled() -> bool:
return os.getenv("COMFY_AUTO_BALANCE", "").strip().lower() in {"1", "true", "yes", "on"}
def providers() -> dict[str, Provider]:
raw = os.getenv("COMFY_PROVIDERS_JSON", "").strip()
rows: list[dict[str, Any]] = []
if raw:
parsed = json.loads(raw)
rows = parsed if isinstance(parsed, list) else []
elif os.getenv("COMFY_URL", "").strip():
rows = [{"id": "default", "name": "ComfyUI", "base_url": os.getenv("COMFY_URL"), "api_key": os.getenv("COMFY_API_KEY", "")}]
extra_path = Path('/app/huobao-extra-providers.json')
if extra_path.exists():
extra = json.loads(extra_path.read_text())
if not isinstance(extra, list) or any(not isinstance(row, dict) or row.get('id') not in ('gpu1', 'gpu1_8189') for row in extra):
raise ValueError('Invalid Huobao extra provider configuration')
existing_ids = {row.get('id') for row in rows}
rows.extend(row for row in extra if row['id'] not in existing_ids)
result: dict[str, Provider] = {}
for row in rows:
provider = Provider(id=str(row.get("id") or "").strip(), name=str(row.get("name") or row.get("id") or "ComfyUI"), base_url=str(row.get("base_url") or "").strip().rstrip("/"), api_key=str(row.get("api_key") or ""), control_token_file=str(row.get("control_token_file") or ""))
if provider.id and provider.base_url.startswith(("http://", "https://")):
result[provider.id] = provider
return result
def require_token(authorization: str | None = Header(default=None)) -> None:
expected = os.getenv("COMFY_ADAPTER_TOKEN", "").strip()
if expected and not secrets.compare_digest(authorization or "", f"Bearer {expected}"):
raise HTTPException(status_code=401, detail="invalid adapter token")
def provider_headers(provider: Provider) -> dict[str, str]:
headers = {"Authorization": f"Bearer {provider.api_key}"} if provider.api_key else {}
if provider.control_token_file:
path = Path(provider.control_token_file)
try:
# Operator-mounted token only; it is never accepted from JobRequest
# or exposed by the public provider catalogue.
if path.parent != Path('/run/h3-workers') or path.name != provider.id + '.token':
raise ValueError('invalid worker control token path')
token = path.read_text().strip()
if not 16 <= len(token) <= 512 or any(ord(x) < 33 or ord(x) > 126 for x in token):
raise ValueError('invalid worker control token')
headers['X-H3-Canary-Token'] = token
except (OSError, ValueError):
raise HTTPException(status_code=503, detail="生成节点控制配置未就绪,任务未提交")
return headers
async def provider_queue_depth(provider: Provider) -> int | None:
try:
if provider.id == 'gpu1_8189':
# Same physical card: finish old accepted jobs without moving or cancelling them.
legacy = providers().get('gpu1')
if legacy:
async with httpx.AsyncClient(timeout=5) as client:
old = await client.get(f'{legacy.base_url}/queue', headers=provider_headers(legacy))
old.raise_for_status()
queue = old.json()
if queue.get('queue_running') or queue.get('queue_pending'):
return None
stats = await client.get(f'{legacy.base_url}/system_stats', headers=provider_headers(legacy))
stats.raise_for_status()
devices = stats.json().get('devices')
if not isinstance(devices, list) or not devices or any(
not isinstance(d, dict) or type(d.get('torch_vram_total')) not in (int, float)
or not 0 <= d['torch_vram_total'] < float('inf') for d in devices
):
return None
# An idle cudaMallocAsync worker retains ~2.2 GiB of persistent tensors.
# Allow at most 4 GiB within the new worker's existing 8 GiB reserve;
# active/queued legacy jobs still block above, and large caches drain below.
if sum(d['torch_vram_total'] for d in devices) > 4 * 1024**3:
# Release only idle model caches; queue contents and results stay intact.
released = await client.post(f'{legacy.base_url}/free', json={'unload_models': True, 'free_memory': True}, headers=provider_headers(legacy))
released.raise_for_status()
return None
async with httpx.AsyncClient(timeout=5) as client:
response = await client.get(f"{provider.base_url}/queue", headers=provider_headers(provider))
response.raise_for_status()
payload = response.json()
except (httpx.HTTPError, ValueError, TypeError):
return None
return len(payload.get("queue_running") or []) + len(payload.get("queue_pending") or [])
def admission_ids(configured: dict[str, Provider]) -> set[str]:
path = os.getenv("COMFY_ADMISSION_FILE", "").strip()
if not path:
return set(configured)
try:
raw = Path(path).read_bytes()
if len(raw) > 4096:
raise ValueError("oversized admission configuration")
value = json.loads(raw)
if not isinstance(value, dict):
raise ValueError("invalid admission configuration")
allowed = value.get("admit")
if type(value.get("schema")) is not int or value.get("schema") != 1 or not isinstance(allowed, list) or any(not isinstance(x, str) for x in allowed):
raise ValueError("invalid admission configuration")
if len(set(allowed)) != len(allowed) or not set(allowed) <= set(configured):
raise ValueError("unknown or duplicate admission provider")
return set(allowed)
except (OSError, ValueError, TypeError):
# Drain changes affect only new submissions; job lookup always retains
# both stable provider IDs and their existing history/output routes.
raise HTTPException(status_code=503, detail="生成服务正在切换,任务尚未提交,请稍后重试")
async def select_provider(requested_provider_id: str, strict_provider: bool = False, huobao_panels: bool = False) -> Provider | None:
configured = providers()
admitted = admission_ids(configured)
if huobao_panels:
ports = {'default': 8188, 'gpu1_8189': 8189, 'gpu3': 8191}
configured = {key: replace(value, base_url=f'http://192.168.110.111:{ports[key]}') for key, value in configured.items() if key in ports}
else:
# 2026-11-10 用户要求:GPU3(8191)只给火宝用,其他应用的新任务不分到 GPU3、指定 GPU3 直接拒绝
if requested_provider_id == 'gpu3':
raise HTTPException(status_code=403, detail="GPU3 仅供火宝使用,任务未提交;请选择自动分配(GPU2)")
configured = {key: value for key, value in configured.items() if key not in ('gpu1_8189', 'gpu3')}
requested = configured.get(requested_provider_id)
if requested is None:
return None
if strict_provider:
if requested_provider_id not in admitted:
raise HTTPException(status_code=503, detail="指定显卡正在维护或排空,任务未提交;请稍后重试或选择自动分配")
if await provider_queue_depth(requested) is None:
raise HTTPException(status_code=503, detail=("GPU1 / 8189 暂不可用:旧 GPU1 任务或显存尚未释放,未取消原任务,也未提交新任务" if requested_provider_id == "gpu1_8189" else "指定显卡不可用或缺少所需工作流,任务未提交"))
return requested
if requested is None or requested_provider_id != "default" or not auto_balance_enabled() or len(configured) < 2:
if requested_provider_id not in admitted:
raise HTTPException(status_code=503, detail="该生成节点正在排空,现有任务继续执行,新任务请稍后提交")
return requested
candidates = [provider for provider in configured.values() if provider.id in admitted]
if not candidates:
raise HTTPException(status_code=503, detail="生成节点正在排空,现有任务和结果保持不变")
depths = await asyncio.gather(*(provider_queue_depth(provider) for provider in candidates))
available = [
(depth + provider_reservations.get(provider.id, 0), 0 if provider.id == requested_provider_id else 1, provider)
for provider, depth in zip(candidates, depths)
if depth is not None
]
if not available:
if os.getenv("COMFY_ADMISSION_FILE", "").strip():
raise HTTPException(status_code=503, detail="生成节点尚未就绪,任务未提交,请稍后重试")
return requested
return min(available, key=lambda item: (item[0], item[1]))[2]
async def reserve_provider(requested_provider_id: str, strict_provider: bool = False, huobao_panels: bool = False) -> Provider | None:
async with provider_selection_lock:
provider = await select_provider(requested_provider_id, strict_provider, huobao_panels)
if provider is not None:
provider_reservations[provider.id] = provider_reservations.get(provider.id, 0) + 1
return provider
async def release_provider(provider: Provider) -> None:
async with provider_selection_lock:
remaining = provider_reservations.get(provider.id, 0) - 1
if remaining > 0:
provider_reservations[provider.id] = remaining
else:
provider_reservations.pop(provider.id, None)
def input_host_allowlist() -> set[str]:
return {value.strip().lower().rstrip(".") for value in os.getenv("COMFY_INPUT_HOST_ALLOWLIST", "").split(",") if value.strip()}
async def validate_input_url(value: str) -> None:
parsed = urlsplit(value)
host = (parsed.hostname or "").lower().rstrip(".")
if parsed.scheme not in {"http", "https"} or not host or parsed.username or parsed.password:
raise HTTPException(status_code=400, detail="reference image URL is invalid")
if host in input_host_allowlist():
return
try:
infos = await asyncio.to_thread(socket.getaddrinfo, host, parsed.port or (443 if parsed.scheme == "https" else 80), type=socket.SOCK_STREAM)
except OSError as exc:
raise HTTPException(status_code=400, detail="reference image host cannot be resolved") from exc
if not infos:
raise HTTPException(status_code=400, detail="reference image host cannot be resolved")
for info in infos:
address = ipaddress.ip_address(info[4][0])
if not address.is_global:
raise HTTPException(status_code=400, detail="reference image URL cannot target a private address")
async def download_input_image(client: httpx.AsyncClient, value: str) -> tuple[bytes, str]:
await validate_input_url(value)
chunks: list[bytes] = []
total = 0
async with client.stream("GET", value, follow_redirects=False) as response:
response.raise_for_status()
if 300 <= response.status_code < 400:
raise HTTPException(status_code=400, detail="reference image redirects are not accepted")
mime_type = response.headers.get("content-type", "image/png").split(";", 1)[0]
if not mime_type.startswith("image/"):
raise HTTPException(status_code=400, detail="reference image URL did not return an image")
async for chunk in response.aiter_bytes():
total += len(chunk)
if total > 30 * 1024 * 1024:
raise HTTPException(status_code=413, detail="reference image exceeds 30 MB")
chunks.append(chunk)
return b"".join(chunks), mime_type
def encode_job(provider_id: str, prompt_id: str) -> str:
return base64.urlsafe_b64encode(f"{provider_id}:{prompt_id}".encode()).decode().rstrip("=")
def decode_job(job_id: str) -> tuple[Provider, str]:
if job_id.startswith(("h3a1.", "h3v2.")):
job_id = job_id.split(".", 1)[1]
try:
raw = base64.urlsafe_b64decode(job_id + "=" * (-len(job_id) % 4)).decode()
provider_id, prompt_id = raw.split(":", 1)
provider = providers()[provider_id]
except Exception as exc:
raise HTTPException(status_code=404, detail="job not found") from exc
return provider, prompt_id
def output_validation_mode() -> str:
mode = os.getenv("H3_OUTPUT_VALIDATION_MODE", "guarded").strip()
if mode not in {"guarded", "prompt-only"}:
raise HTTPException(status_code=503, detail="视频处理模式配置无效,任务未提交")
return mode
@app.get("/health")
async def health() -> dict[str, Any]:
return {
"status": "ok",
"providers": len(providers()),
"workflows": len(load_registry()),
"autoBalance": auto_balance_enabled(),
"outputValidationMode": output_validation_mode(),
}
@app.get("/v1/providers", dependencies=[Depends(require_token)])
async def list_providers() -> dict[str, Any]:
return {"providers": [{"id": item.id, "name": item.name, "baseUrl": item.base_url} for item in providers().values()]}
@app.get("/v1/workflows", dependencies=[Depends(require_token)])
async def list_workflows() -> dict[str, Any]:
return {"workflows": [public_workflow(item) for item in load_registry().values()]}
@app.get("/v1/models", dependencies=[Depends(require_token)])
async def list_models() -> dict[str, Any]:
return {"object": "list", "data": [public_model(item) for item in load_registry().values()]}
async def upload_image(client: httpx.AsyncClient, provider: Provider, value: str, index: int, client_id: str) -> str:
value = value.strip()
if value.startswith("comfy://"):
return value.removeprefix("comfy://")
if value.startswith("data:"):
header, encoded = value.split(",", 1)
mime_type = header.split(";", 1)[0].removeprefix("data:") or "image/png"
if not mime_type.startswith("image/"):
raise HTTPException(status_code=400, detail="reference data URL must contain an image")
try:
data = base64.b64decode(encoded, validate=True)
except (ValueError, base64.binascii.Error) as exc:
raise HTTPException(status_code=400, detail="reference image data URL is invalid") from exc
elif value.startswith(("http://", "https://")):
data, mime_type = await download_input_image(client, value)
else:
raise HTTPException(status_code=400, detail="input_images must contain http(s), data, or comfy URLs")
if len(data) > 30 * 1024 * 1024:
raise HTTPException(status_code=413, detail="reference image exceeds 30 MB")
extension = mimetypes.guess_extension(mime_type) or ".png"
filename = f"{client_id}-reference-{index}{extension}"
response = await client.post(f"{provider.base_url}/upload/image", files={"image": (filename, data, mime_type)}, data={"overwrite": "true"}, headers=provider_headers(provider))
response.raise_for_status()
payload = response.json()
return str(payload.get("name") or filename)
def request_audio_plan(spec, request: JobRequest) -> dict:
if not (spec.raw.get("speech_prompt_policy") == "minimax_h3" and spec.raw.get("native_audio_control")):
return {"supported": False}
if request.generate_audio is False:
return {"supported": True, "status": "ready", "normalized_prompt": request.prompt,
"dialogue_base_prompt": request.prompt, "dialogue": [], "sound_effects": [],
"music": [], "audio_mode": "silent", "silent": True, "speech_requested": False,
"expected_transcript": "", "required_seconds": 0, "max_duration_seconds": 15, "issues": []}
plan = plan_h3_audio(request.prompt, float(request.duration or 5))
if output_validation_mode() == "prompt-only" and not plan["silent"] and plan["audio_mode"] == "silent":
plan["audio_mode"] = "effects"
return {"supported": True, **plan}
@app.post("/v1/audio-plan", dependencies=[Depends(require_token)])
async def preview_audio_plan(request: JobRequest) -> dict:
spec = load_registry().get(request.workflow_key)
if spec is None:
raise HTTPException(status_code=400, detail="workflow is not registered")
return request_audio_plan(spec, request)
@app.post("/v1/jobs", dependencies=[Depends(require_token)])
async def create_job(request: JobRequest) -> dict[str, Any]:
registry = load_registry()
spec = registry.get(request.workflow_key)
if spec is None:
raise HTTPException(status_code=400, detail="workflow is not registered")
try:
validate_h3_references(spec, len(request.input_images))
except ValueError as error:
raise HTTPException(status_code=400, detail=str(error)) from error
audio_plan = request_audio_plan(spec, request)
guarded_output = bool(audio_plan.get("supported")) and output_validation_mode() == "guarded"
if audio_plan.get("supported"):
if audio_plan.get("status") != "ready":
raise HTTPException(status_code=400, detail="H3_AUDIO_CONFIRMATION_REQUIRED: 请先确认台词及视频时长,再提交视频生成")
if not guarded_output:
request = request.model_copy(update={"prompt": audio_plan["normalized_prompt"]})
try:
if guarded_output:
validate_visual_dimensions(request.width, request.height)
await audio_validator.ready(visual_required=True, audio_required=audio_plan.get("audio_mode") in {"dialogue", "effects"})
if audio_plan.get("audio_mode") in {"dialogue", "effects"}:
prompt = await prepare_audio_conditioning(audio_plan, request.prompt, float(request.duration or 5), len(request.input_images))
request = request.model_copy(update={"prompt": prompt})
except AudioQualityError as error:
raise HTTPException(status_code=503 if error.code.endswith("_UNAVAILABLE") else 400, detail=str(error)) from error
except AudioConditioningError as error:
raise HTTPException(status_code=503 if error.code == "H3_AUDIO_NORMALIZER_UNAVAILABLE" else 400, detail=str(error)) from error
provider = await reserve_provider(request.provider_id, request.strict_provider, request.huobao_panels)
if provider is None:
raise HTTPException(status_code=400, detail="provider is not configured")
try:
client_id = ("canvas-h3v2-" if guarded_output else "canvas-") + secrets.token_hex(8)
async with httpx.AsyncClient(timeout=60) as client:
uploaded = [await upload_image(client, provider, value, index, client_id) for index, value in enumerate(request.input_images, start=1)]
try:
workflow = compile_workflow(spec, request.model_dump(), uploaded, client_id, preserve_requested_audio=not guarded_output and bool(audio_plan.get("supported")))
if guarded_output:
attach_audio_policy(workflow, audio_plan, float(request.duration or 5), str(spec.raw.get("save_node")), visual_required=True)
except ValueError as exc:
raise HTTPException(status_code=400, detail=str(exc)) from exc
response = await client.post(f"{provider.base_url}/prompt", json={"prompt": workflow, "client_id": client_id}, headers=provider_headers(provider))
if response.status_code == 400 and spec.raw.get("speech_prompt_policy") == "minimax_h3":
try:
rejected = response.json()
except ValueError:
rejected = {}
if "h3_caption_pipeline" in (rejected.get("node_errors") or {}) or "H3CaptionPipeline" in str((rejected.get("error") or {}).get("details") or ""):
raise HTTPException(status_code=503, detail="字幕工作流预检未通过,处理组件、模型或字体尚未就绪;任务未进入 GPU 生成,请联系管理员。")
response.raise_for_status()
payload = response.json()
finally:
await release_provider(provider)
prompt_id = str(payload.get("prompt_id") or "").strip()
if not prompt_id:
raise HTTPException(status_code=502, detail="ComfyUI did not return prompt_id")
job_id = encode_job(provider.id, prompt_id)
persisted = True
if guarded_output:
job_id = "h3v2." + job_id
try:
audio_validator.register_policy(job_id, audio_plan, float(request.duration or 5), visual_required=True)
except AudioQualityError as error:
raise HTTPException(status_code=502, detail=str(error)) from error
except OSError:
# The provider already accepted this exact job. Return its ID rather
# than an ambiguous 500 inviting another GPU submission. Its graph
# carries the same policy; missing evidence still fails closed.
persisted = False
return {"id": job_id, "promptId": prompt_id, "providerId": provider.id, "workflowKey": spec.key, "workflowRevision": spec.revision, "status": "submitted", "policyPersisted": persisted}
def extract_outputs(entry: dict[str, Any], job_id: str) -> list[dict[str, Any]]:
result: list[dict[str, Any]] = []
for node in (entry.get("outputs") or {}).values():
if not isinstance(node, dict):
continue
for field in ("images", "gifs", "audio"):
for item in node.get(field) or []:
if not isinstance(item, dict) or not item.get("filename"):
continue
filename = str(item["filename"])
mime_type = mimetypes.guess_type(filename)[0] or "application/octet-stream"
kind = "video" if mime_type.startswith("video/") or filename.lower().endswith((".gif", ".webp")) and field == "gifs" else "audio" if mime_type.startswith("audio/") else "image"
index = len(result)
result.append({"index": index, "kind": kind, "mimeType": mime_type, "filename": filename, "url": f"/jobs/{quote(job_id, safe='')}/outputs/{index}"})
return result
async def job_state(job_id: str) -> tuple[Provider, str, dict[str, Any], list[dict[str, Any]]]:
provider, prompt_id = decode_job(job_id)
async with httpx.AsyncClient(timeout=30) as client:
response = await client.get(f"{provider.base_url}/history/{quote(prompt_id, safe='')}", headers=provider_headers(provider))
response.raise_for_status()
history = response.json()
entry = history.get(prompt_id) or {}
outputs = extract_outputs(entry, job_id)
return provider, prompt_id, entry, outputs
@app.get("/v1/jobs/{job_id}", dependencies=[Depends(require_token)])
async def get_job(job_id: str) -> dict[str, Any]:
try:
provider, prompt_id, entry, outputs = await job_state(job_id)
except httpx.TimeoutException:
provider, prompt_id = decode_job(job_id)
return {
"id": job_id,
"promptId": prompt_id,
"providerId": provider.id,
"status": "running",
"outputs": [],
"error": "",
"pollDeferred": True,
}
status_payload = entry.get("status") or {}
if status_payload.get("status_str") == "error":
status = "failed"
elif outputs and not (job_id.startswith("h3v2.") and not status_payload.get("completed")):
status = "succeeded"
elif status_payload.get("completed"):
status = "failed"
else:
status = "running"
if job_id.startswith("h3v2.") and status != "succeeded":
outputs = []
if status != "succeeded" and cancellation_dispatched(job_id):
# An acknowledged interrupt is not terminal until this exact prompt
# disappears from the worker queue. Never infer it from HTTP 200 alone.
try:
async with httpx.AsyncClient(timeout=10) as client:
response = await client.get(f"{provider.base_url}/queue", headers=provider_headers(provider))
response.raise_for_status()
queue = response.json()
if not isinstance(queue, dict) or not isinstance(queue.get("queue_running"), list) or not isinstance(queue.get("queue_pending"), list):
raise ValueError("invalid queue response")
remaining = {str(item[1]) for item in queue["queue_running"] + queue["queue_pending"] if isinstance(item, list) and len(item) > 1}
if prompt_id not in remaining:
status, outputs = "cancelled", []
else:
return {"id": job_id, "promptId": prompt_id, "providerId": provider.id, "status": "running", "outputs": [], "error": "", "stage": "正在取消生成", "cancelRequested": True}
except (httpx.HTTPError, ValueError):
return {"id": job_id, "promptId": prompt_id, "providerId": provider.id, "status": "running", "outputs": [], "error": "", "stage": "正在确认取消", "cancelRequested": True}
error = "ComfyUI execution failed" if status == "failed" else ""
if status == "failed":
for event in status_payload.get("messages") or []:
if not isinstance(event, (list, tuple)) or len(event) < 2 or event[0] != "execution_error" or not isinstance(event[1], dict):
continue
# 只映射已识别的内核错误,不向用户返回节点输入、提示词或 traceback。
if "tensor strides exceed int32 range" in str(event[1].get("exception_message") or ""):
error = "当前时长与分辨率组合超出采样内核范围,请缩短时长后重试(最多 15 秒)"
break
if status == "failed":
hints = {
"H3_SUBTITLE_MASK_UNCERTAIN": "检测到疑似字幕,但无法安全区分字幕和产品文字,任务已停止,未返回未清理视频。",
"H3_SUBTITLE_RESIDUAL": "字幕清理后仍检测到残留,任务已停止,避免生成双层字幕。",
"H3_SPEECH_MISMATCH": "实际口播与原台词不一致,未生成误导字幕;请缩短台词或调整后重试。",
"H3_SUBTITLE_ALIGNMENT": "口播时间轴校验未通过,无法准确烧录字幕;请缩短单句台词后重试。",
"H3_SUBTITLE_FONT": "字幕含字体不支持的字符,已停止以避免乱码;请使用常见文字。",
"H3_SUBTITLE_DEPENDENCY": "字幕处理服务依赖未就绪,请联系管理员;原任务未自动重试。",
"H3_SUBTITLE_FRAME_COUNT": "字幕处理前后帧数或尺寸不一致,已停止输出,请联系管理员。",
}
for event in status_payload.get("messages") or []:
if not isinstance(event, (list, tuple)) or len(event) < 2 or event[0] != "execution_error" or not isinstance(event[1], dict):
continue
message = str(event[1].get("exception_message") or "")
for code, hint in hints.items():
if message.startswith(code + ":"):
error = hint
break
# Never expose a partial/raw output from a failed cleanup/quality gate.
outputs = []
if status == "succeeded":
quality = await audio_validator.poll(job_id, entry, provider.base_url, provider_headers(provider))
if quality["status"] in {"pending", "unavailable"}:
return {"id": job_id, "promptId": prompt_id, "providerId": provider.id, "status": "running",
"outputs": [], "error": "", "stage": "正在校验字幕与音轨" if job_id.startswith("h3v2.") or quality.get("visual_required") else "正在校验视频音轨", "pollDeferred": True}
if quality["status"] == "failed":
status, outputs, error = "failed", [], quality["error"]
return {"id": job_id, "promptId": prompt_id, "providerId": provider.id, "status": status, "outputs": outputs, "error": error}
@app.post("/v1/jobs/{job_id}/cancel", dependencies=[Depends(require_token)])
async def cancel_job(job_id: str) -> dict[str, Any]:
provider, prompt_id = decode_job(job_id)
# Write intent before sending; disk failure must not produce an untracked
# cancellation. The worker API atomically compares the running prompt ID.
record_cancellation(job_id, False)
async with httpx.AsyncClient(timeout=30) as client:
response = await client.post(f"{provider.base_url}/api/jobs/{quote(prompt_id, safe='')}/cancel", json={}, headers=provider_headers(provider))
if response.status_code in (404, 405):
raise HTTPException(status_code=409, detail="生成节点未提供按任务取消接口,未中断其他任务")
response.raise_for_status()
acknowledged = response.json().get("cancelled")
if type(acknowledged) is not bool:
raise HTTPException(status_code=502, detail="生成节点取消回执无效,请核对原任务状态")
if acknowledged:
record_cancellation(job_id, True)
return await get_job(job_id)
@app.get("/v1/jobs/{job_id}/outputs/{index}", dependencies=[Depends(require_token)])
async def get_output(job_id: str, index: int) -> Response:
provider, _, entry, outputs = await job_state(job_id)
if (entry.get("status") or {}).get("status_str") == "error":
raise HTTPException(status_code=409, detail="任务未成功,未返回部分输出")
if not outputs or job_id.startswith("h3v2.") and not (entry.get("status") or {}).get("completed"):
raise HTTPException(status_code=409, detail="视频生成尚未完成,未返回部分输出")
quality = await audio_validator.poll(job_id, entry, provider.base_url, provider_headers(provider))
if quality["status"] != "verified":
raise HTTPException(status_code=422 if quality["status"] == "failed" else 409,
detail=quality.get("error", "视频核验尚未完成,请稍后查询原任务"))
if index < 0 or index >= len(outputs):
raise HTTPException(status_code=404, detail="output not found")
wanted = outputs[index]
locator: dict[str, Any] | None = None
for node in (entry.get("outputs") or {}).values():
if not isinstance(node, dict):
continue
for field in ("images", "gifs", "audio"):
for item in node.get(field) or []:
if isinstance(item, dict) and str(item.get("filename")) == wanted["filename"]:
locator = item
break
if locator: break
if locator: break
if locator is None:
raise HTTPException(status_code=404, detail="output locator not found")
if quality.get("visual_revision") == VISUAL_REVISION:
bound_locator = {key: locator.get(key, "output" if key == "type" else "") for key in ("filename", "subfolder", "type")}
try:
media = await audio_validator.verified_download(quality, bound_locator, provider.base_url, provider_headers(provider))
except AudioQualityError as error:
raise HTTPException(status_code=422, detail=str(error)) from error
except (httpx.HTTPError, OSError, asyncio.TimeoutError):
raise HTTPException(status_code=503, detail="H3_VISUAL_VALIDATION_UNAVAILABLE: 已核验视频暂不可读取,请查询原任务") from None
return Response(content=media, media_type=wanted["mimeType"], headers={"Cache-Control": "private, no-store"})
query = urlencode({"filename": locator.get("filename", ""), "subfolder": locator.get("subfolder", ""), "type": locator.get("type", "output")})
client = httpx.AsyncClient(timeout=120)
response = await client.send(client.build_request("GET", f"{provider.base_url}/view?{query}", headers=provider_headers(provider)), stream=True)
if response.status_code >= 400:
await response.aclose(); await client.aclose()
raise HTTPException(status_code=502, detail="failed to read ComfyUI output")
async def stream():
try:
async for chunk in response.aiter_bytes():
yield chunk
finally:
await response.aclose(); await client.aclose()
return StreamingResponse(stream(), media_type=response.headers.get("content-type", wanted["mimeType"]))
@app.post("/v1/huobao/route", dependencies=[Depends(require_token)])
async def huobao_route(request: JobRequest) -> dict[str, Any]:
provider = await select_provider(request.provider_id, request.strict_provider, True)
if provider is None:
raise HTTPException(status_code=400, detail="unknown requested worker")
return {"providerId": provider.id, "baseUrl": provider.base_url, "ready": True}
@app.get("/v1/huobao/jobs/{job_id}", dependencies=[Depends(require_token)])
async def huobao_job_status(job_id: str) -> dict[str, Any]:
# Keep the general API contract unchanged; Huobao needs actual queue state.
result = await get_job(job_id)
if result.get("status") != "running":
return {**result, "worker_state": result.get("status")}
if result.get("pollDeferred"):
return {**result, "worker_state": "checking"}
provider, prompt_id = decode_job(job_id)
async with httpx.AsyncClient(timeout=10) as client:
response = await client.get(f"{provider.base_url}/queue", headers=provider_headers(provider))
response.raise_for_status()
queue = response.json()
running = [row[1] for row in queue.get("queue_running", [])]
pending = [row[1] for row in queue.get("queue_pending", [])]
if prompt_id in running:
return {**result, "worker_state": "running", "progress": await huobao_progress(provider, prompt_id, queue, provider_headers(provider))}
if prompt_id in pending:
return {**result, "worker_state": "queued", "queue_position": pending.index(prompt_id) + 1, "progress": await huobao_progress(provider, prompt_id, queue, provider_headers(provider))}
# A task may finish between the first history read and the queue read.
fresh = await get_job(job_id)
if fresh.get("status") != "running":
return {**fresh, "worker_state": fresh.get("status")}
if fresh.get("pollDeferred"):
return {**fresh, "worker_state": "checking"}
# Do not invent success/failure or submit again when a worker has lost history.
return {**fresh, "worker_state": "missing"}
# BEGIN HUOBAO READ-ONLY PROGRESS
"""Read-only ComfyUI telemetry, embedded in the existing private normalizers.
Only the requesting Huobao job's client connection is observed. No queue mutation
or image previews are used. Sampling ETA is never presented as whole-video ETA.
"""
import asyncio as _hp_asyncio
import json as _hp_json
import time as _hp_time
import math as _hp_math
import statistics as _hp_stats
from urllib.parse import quote as _hp_quote
import httpx as _hp_httpx
import websockets as _hp_ws
_hp_watchers = {}
_hp_histories = {}
def _hp_phase(kind):
kind = str(kind).lower()
if 'sampler' in kind: return 'sampling'
if 'decode' in kind: return 'decoding'
if 'savevideo' in kind or 'combine' in kind or 'createvideo' in kind: return 'encoding'
if 'loader' in kind or kind == 'loadimage': return 'loading'
if 'caption' in kind: return 'postprocessing'
if 'encode' in kind or 'tovideo' in kind: return 'conditioning'
return 'running'
def _hp_signature(graph):
# Keep graph topology, model weights, image count, dimensions, FPS, duration,
# steps and sampler. Exclude content, random seeds and output filenames.
omitted = {'text', 'prompt', 'positive', 'negative', 'seed', 'noise_seed',
'filename_prefix', 'image', 'audio', 'video', 'file', 'path'}
result = {}
for node, value in graph.items():
inputs = {}
for key, item in value.get('inputs', {}).items():
if key in omitted and not isinstance(item, list): continue
if any(word in key.lower() for word in ['prompt', 'text', 'caption']) and not isinstance(item, list): continue
inputs[key] = item
result[node] = {'class_type': value.get('class_type'), 'inputs': inputs}
return _hp_json.dumps(result, sort_keys=True, separators=(',', ':'))
def _hp_history_estimate(graph, history):
signature = _hp_signature(graph)
durations = []
for entry in history.values():
status = entry.get('status', {})
prompt = entry.get('prompt', [])
if status.get('status_str') != 'success' or not status.get('completed') or len(prompt) < 3: continue
previous = prompt[2]
if not isinstance(previous, dict) or _hp_signature(previous) != signature: continue
times = {}; cached = []
for event in status.get('messages', []):
if len(event) < 2 or not isinstance(event[1], dict): continue
times[event[0]] = event[1].get('timestamp')
if event[0] == 'execution_cached': cached = event[1].get('nodes', [])
if any('sampler' in str(previous.get(str(n), {}).get('class_type', '')).lower() for n in cached): continue
start, end = times.get('execution_start'), times.get('execution_success')
if not isinstance(start, (int, float)) or not isinstance(end, (int, float)): continue
duration = (end - start) / 1000
if 2 <= duration <= 86400: durations.append(duration)
result = {'historySamples': len(durations)}
if len(durations) >= 2:
# A range, not a guarantee; model loading and shared GPU contention vary.
result.update(historyMinSeconds=round(min(durations) * .8), historyMaxSeconds=round(max(durations) * 1.2))
return result
def _hp_event(state, message, timestamp):
data = message.get('data', {})
if not isinstance(data, dict): return
event = message.get('type')
if data.get('prompt_id') not in (None, state['prompt']): return
if event == 'executing':
node = data.get('node')
if node is None: return
node = str(node)
if node != state.get('node'):
state.update(node=node, samples=[], value=None, total=None)
state['phase'] = _hp_phase(state['classes'].get(node))
state['updated'] = timestamp
elif event == 'progress' and data.get('prompt_id') == state['prompt']:
value, total = data.get('value'), data.get('max')
if not all(type(x) in (int, float) and _hp_math.isfinite(x) for x in (value, total)): return
if not 0 <= value <= total <= 1000000 or total <= 0: return
node = str(data.get('node', state.get('node', '')))
if node != state.get('node') or total != state.get('total') or value < (state.get('value') or 0):
state.update(node=node, samples=[])
if state.get('value') != value or not state['samples']:
state['samples'] = (state['samples'] + [(timestamp, value)])[-8:]
state.update(value=value, total=total, updated=timestamp, phase=_hp_phase(state['classes'].get(node)))
def _hp_snapshot(state, timestamp):
result = {'phase': state.get('phase', 'running')}
if not state.get('connected') or timestamp - state.get('updated', 0) > 600:
return {'phase': 'running'}
if state.get('value') is None: return result
result.update(step=state['value'], totalSteps=state['total'], stepUpdatedAt=round(state['updated'] * 1000))
samples = state['samples']
if state.get('phase') == 'sampling' and len(samples) >= 3:
seconds = samples[-1][0] - samples[0][0]; steps = samples[-1][1] - samples[0][1]
if seconds >= 5 and steps > 0 and state['value'] < state['total']:
speed = seconds / steps
if timestamp - state['updated'] <= max(30, speed * 2.5):
remaining = max(1, speed * (state['total'] - state['value']) - (timestamp - state['updated']))
result.update(samplingMinSeconds=round(remaining * .8), samplingMaxSeconds=round(remaining * 1.3))
return result
async def _hp_watch(state, base, headers, client_id):
uri = base.replace('https://', 'wss://').replace('http://', 'ws://') + '/ws?clientId=' + _hp_quote(client_id, safe='')
while _hp_time.time() - state['requested'] < 120:
try:
async with _hp_ws.connect(uri, additional_headers=headers, max_size=32 * 1024 * 1024, open_timeout=5) as ws:
state['connected'] = True
# This observer needs text progress only, never preview images.
await ws.send(_hp_json.dumps({'type':'feature_flags','data':{'supports_preview_metadata':True}}))
while _hp_time.time() - state['requested'] < 120:
try: message = await _hp_asyncio.wait_for(ws.recv(), 15)
except _hp_asyncio.TimeoutError: continue
if isinstance(message, str):
try: _hp_event(state, _hp_json.loads(message), _hp_time.time())
except (ValueError, TypeError, AttributeError): pass
except _hp_asyncio.CancelledError: raise
except Exception:
await _hp_asyncio.sleep(5)
finally: state['connected'] = False
async def huobao_progress(provider, prompt_id, queue, headers):
"""Failure of progress reporting must never fail or resubmit a generation."""
timestamp = _hp_time.time()
running = queue.get('queue_running', []); pending = queue.get('queue_pending', [])
row = next((x for x in running + pending if len(x) > 3 and x[1] == prompt_id), None)
if row is None: return None
result = {'observedAt': round(timestamp * 1000), 'phase': 'queued' if row in pending else 'running'}
if row in pending:
result.update(queuePosition=pending.index(row) + 1, queueAhead=len(running) + pending.index(row))
try:
previous = _hp_watchers.get(provider.id)
if previous and (previous['prompt'] != prompt_id and row in running or previous['task'].done()):
previous['task'].cancel(); _hp_watchers.pop(provider.id, None); previous = None
if row in running:
client_id = row[3].get('client_id')
# Huobao's normalizer generates unique canvas-* client IDs. Never
# replace a human's browser websocket or guess another job's state.
if isinstance(client_id, str) and client_id.startswith('canvas-'):
if previous is None:
previous = {'prompt':prompt_id, 'classes':{str(k):v.get('class_type') for k,v in row[2].items()},
'requested':timestamp, 'samples':[], 'connected':False}
previous['task'] = _hp_asyncio.create_task(_hp_watch(previous, provider.base_url, headers, client_id))
_hp_watchers[provider.id] = previous
previous['requested'] = timestamp
if previous['prompt'] == prompt_id: result.update(_hp_snapshot(previous, timestamp))
cached = _hp_histories.get(provider.id)
if cached is None or timestamp - cached[0] > 90:
async with _hp_httpx.AsyncClient(timeout=5) as client:
response = await client.get(provider.base_url + '/history', params={'max_items':80}, headers=headers)
response.raise_for_status(); history = response.json()
_hp_histories[provider.id] = (timestamp, history)
else: history = cached[1]
result.update(_hp_history_estimate(row[2], history))
except Exception:
pass
return result
@@ -0,0 +1,600 @@
"""Private adapters for Huobao: Dify, Qwen Image, local H3, and Grok CLI."""
from __future__ import annotations
import asyncio
import base64
from contextlib import closing
from email import policy
from email.parser import BytesParser
import hashlib
import hmac
import json
import math
import os
from pathlib import Path
import re
import secrets
import sqlite3
import struct
import tempfile
import time
from urllib.parse import quote
import httpx
from fastapi import Depends, FastAPI, Header, HTTPException, Request
from fastapi.responses import JSONResponse, StreamingResponse
def chat_query(body):
messages = body.get("messages")
if not isinstance(messages, list) or not messages:
raise HTTPException(400, "messages must be a non-empty array")
if body.get("tool_choice") == "required" or isinstance(body.get("tool_choice"), dict):
raise HTTPException(400, "This chatflow does not support required tool calls; use DeepSeek")
parts = []
for item in messages:
content = item.get("content", "")
if isinstance(content, list):
if any(p.get("type") != "text" for p in content):
raise HTTPException(400, "This adapter accepts text messages only")
content = "\n".join(p.get("text", "") for p in content)
if content:
parts.append(f"[{item.get('role', 'user')}]\n{content}")
fmt = body.get("response_format") or {}
if fmt.get("type") == "json_schema":
parts.append("Return only JSON matching this schema:\n" + json.dumps(fmt.get("json_schema", {}).get("schema", {}), ensure_ascii=False))
elif fmt.get("type") == "json_object":
parts.append("Return only a valid JSON object, without Markdown fences.")
if not parts:
raise HTTPException(400, "Empty messages")
return "\n\n".join(parts)
def image_dimensions(data_url):
try:
data = base64.b64decode(data_url.split(",", 1)[1], validate=True)
if data.startswith(b"\x89PNG\r\n\x1a\n"):
return struct.unpack(">II", data[16:24])
if data.startswith((b"GIF87a", b"GIF89a")):
return struct.unpack("<HH", data[6:10])
if data[:2] == b"\xff\xd8":
offset = 2
while offset + 4 < len(data):
if data[offset] != 255:
offset += 1
continue
while data[offset] == 255:
offset += 1
marker = data[offset]
offset += 1
if marker in {0xD8, 0xD9} or 0xD0 <= marker <= 0xD7:
continue
size = int.from_bytes(data[offset:offset + 2], "big")
if marker in {0xC0, 0xC1, 0xC2, 0xC3, 0xC5, 0xC6, 0xC7, 0xC9, 0xCA, 0xCB, 0xCD, 0xCE, 0xCF}:
height, width = struct.unpack(">HH", data[offset + 3:offset + 7])
return width, height
if size < 2:
break
offset += size
if data[:4] == b"RIFF" and data[8:12] == b"WEBP":
if data[12:16] == b"VP8X":
return 1 + int.from_bytes(data[24:27], "little"), 1 + int.from_bytes(data[27:30], "little")
if data[12:16] == b"VP8L" and data[20] == 0x2F:
bits = int.from_bytes(data[21:25], "little")
return (bits & 0x3FFF) + 1, ((bits >> 14) & 0x3FFF) + 1
if data[12:16] == b"VP8 " and data[23:26] == b"\x9d\x01\x2a":
width, height = struct.unpack("<HH", data[26:30])
return width & 0x3FFF, height & 0x3FFF
except (ValueError, IndexError, struct.error):
pass
raise HTTPException(400, "Unable to read first-frame dimensions; use PNG, JPEG or WebP")
VIDEO_MODELS = {
"MiniMax-H3-VDN-H3": {"reference": "minimax_h3_vdn_hd", "min_images": 1, "max_images": 9},
"MiniMax-H3": {"text": "minimax_h3_t2v_official", "first": "minimax_h3_i2v", "reference": "minimax_h3_r2v", "min_images": 0, "max_images": 9},
"MiniMax-H3-Hybrid": {"text": "minimax_h3_hybrid_v0331_t2v", "first": "minimax_h3_hybrid_v0331_i2v", "reference": "minimax_h3_hybrid_v0331_i2v", "min_images": 0, "max_images": 9},
"MiniMax-H3-R2V-4Step": {"reference": "minimax_h3_r2v_4step", "min_images": 1, "max_images": 9},
"LTX-2.3": {"text": "ltx23_t2v", "first": "ltx23_t2v", "reference": "ltx23_t2v", "min_images": 0, "max_images": 1},
"LTX-2.3-Text-Preserve": {"text": "ltx23_t2v_text_preserve", "first": "ltx23_t2v_text_preserve", "reference": "ltx23_t2v_text_preserve", "min_images": 0, "max_images": 1},
"Wan2.2-I2V": {"first": "wan22_simple", "reference": "wan22_simple", "min_images": 1, "max_images": 1},
}
def video_workflows(spec):
return {spec[mode] for mode in ("text", "first", "reference") if mode in spec}
def gpu_routing(body):
gpu = body.get("gpu", "auto")
if gpu not in ("auto", "gpu1", "gpu2", "gpu3"):
raise HTTPException(400, "gpu must be auto, gpu1, gpu2 or gpu3")
if gpu == "auto":
return {"provider_id": "default", "huobao_panels": True}
return {"provider_id": {"gpu1": "gpu1_8189", "gpu2": "default", "gpu3": "gpu3"}[gpu], "strict_provider": True, "huobao_panels": True}
def worker_routing(provider_id):
worker = {"default": ("gpu2", 8188), "gpu1_8189": ("gpu1", 8189), "gpu3": ("gpu3", 8191), "gpu1": ("gpu1", None)}.get(provider_id)
if not worker:
return None
gpu, port = worker
return {"gpu": gpu, "port": port, "providerId": provider_id, "legacy": provider_id == "gpu1"}
def h3_request(body):
"""Translate the local video protocol. 'default' means upstream auto-balance."""
model = body.get("model", "MiniMax-H3")
spec = VIDEO_MODELS.get(model)
if spec is None:
raise HTTPException(400, "Unknown local video model; query /v1/models")
content = body.get("content") or []
if not isinstance(content, list):
raise HTTPException(400, "content must be an array")
prompt = "\n".join(str(c.get("text", "")) for c in content if c.get("type") == "text").strip()
if not prompt or len(prompt) > 7000:
raise HTTPException(400, "A prompt of 1-7000 characters is required")
images, roles = [], []
for item in content:
kind = item.get("type")
if kind == "text":
continue
if kind != "image_url" or item.get("role") == "last_frame":
raise HTTPException(400, "Local video supports first-frame/reference images; last frames, reference video and voice samples are unsupported")
raw = item.get("image_url") or {}
url = raw.get("url", "") if isinstance(raw, dict) else raw
if not isinstance(url, str) or not url.startswith("data:image/"):
raise HTTPException(400, "Local video requires inline image data; enable Huobao's local-image conversion")
images.append(url)
roles.append(item.get("role", "reference_image"))
if not spec["min_images"] <= len(images) <= spec["max_images"]:
raise HTTPException(400, f"{model} requires {spec['min_images']}-{spec['max_images']} images")
if "first_frame" in roles and len(images) != 1:
raise HTTPException(400, "Use either one fixed first frame or reference images; these modes cannot be combined")
mode = "text" if not images else "first" if roles == ["first_frame"] else "reference"
if mode not in spec:
raise HTTPException(400, f"{model} does not support {mode} mode; use reference images")
try:
duration = float(body.get("duration", 5))
except (ValueError, TypeError):
raise HTTPException(400, "Invalid duration")
if not math.isfinite(duration) or not 4 <= duration <= 15:
raise HTTPException(400, "Duration must be 4-15 seconds")
ratio = body.get("ratio", "16:9")
if ratio not in {"16:9", "9:16", "1:1", "4:3", "3:4", "21:9"}:
raise HTTPException(400, "Unsupported aspect ratio")
resolution = str(body.get("resolution", "768P")).upper()
resolutions = {"480P": 480, "720P": 720, "1080P": 1080, "768P": 768, "2K": 1152}
if model == "MiniMax-H3-VDN-H3":
if ratio not in {"16:9", "9:16"}:
raise HTTPException(400, "VDN HD supports 16:9 or 9:16")
resolutions = {"2K": 1440, "4K": 2160}
if resolution not in resolutions:
raise HTTPException(400, "Unsupported video resolution")
short = resolutions[resolution]
a, b = map(int, ratio.split(":"))
if roles == ["first_frame"] and "ratio" not in body:
a, b = image_dimensions(images[0])
if a <= 0 or b <= 0:
raise HTTPException(400, "Invalid first-frame dimensions")
width = round(short * max(1, a / b) / 64) * 64
height = round(short * max(1, b / a) / 64) * 64
if model == "MiniMax-H3-VDN-H3":
width, height = (short * 16 // 9, short) if a > b else (short, short * 16 // 9)
workflow = spec[mode]
request = {**gpu_routing(body), "workflow_key": workflow, "prompt": prompt,
"input_images": images, "duration": duration, "width": width, "height": height,
"metadata": {"source": "huobao-drama", "model": model}}
if body.get("seed") is not None:
if not isinstance(body["seed"], int) or not 0 <= body["seed"] < 2**63:
raise HTTPException(400, "Invalid seed")
request["seed"] = body["seed"]
return request
def qwen_image_request(body, images=None):
if body.get("model") != "qwen_image_2_1":
raise HTTPException(400, "Use model qwen_image_2_1")
prompt = body.get("prompt")
if not isinstance(prompt, str) or not prompt.strip() or len(prompt) > 32000:
raise HTTPException(400, "A prompt of 1-32000 characters is required")
if str(body.get("n", 1)) != "1":
raise HTTPException(400, "This image adapter supports n=1 per task")
if body.get("response_format", "url") != "url":
raise HTTPException(400, "Asynchronous image tasks require response_format=url")
size = str(body.get("size", "1024x1024")).lower()
if size == "auto":
size = "1024x1024"
match = re.fullmatch(r"(\d{3,4})x(\d{3,4})", size)
if not match or not all(256 <= int(d) <= 4096 for d in match.groups()):
raise HTTPException(400, "size must be WIDTHxHEIGHT with dimensions 256-4096")
width, height = map(int, match.groups())
refs = images or []
if len(refs) > 6:
raise HTTPException(400, "Qwen Image 2.1 supports at most 6 reference images")
return {**gpu_routing(body), "workflow_key": "qwen_image_2_1", "prompt": prompt.strip(),
"width": width, "height": height, "batch_size": 1, "input_images": refs,
"metadata": {"source": "huobao-drama", "output_type": "image"}}
async def parse_image_edit(request):
content_type = request.headers.get("content-type", "")
if not content_type.lower().startswith("multipart/form-data;"):
raise HTTPException(415, "Image edits require multipart/form-data")
raw = bytearray()
async for chunk in request.stream():
raw.extend(chunk)
if len(raw) > 64 * 1024 * 1024:
raise HTTPException(413, "Reference uploads exceed 64 MiB")
message = BytesParser(policy=policy.default).parsebytes(
("Content-Type: " + content_type + "\r\nMIME-Version: 1.0\r\n\r\n").encode() + raw)
if not message.is_multipart() or message.defects:
raise HTTPException(400, "Invalid multipart image request")
fields, images = {}, []
for part in message.iter_parts():
name = part.get_param("name", header="content-disposition")
data = part.get_payload(decode=True)
if not isinstance(data, bytes):
raise HTTPException(400, "Invalid multipart field")
if name in {"image", "image[]"}:
mime = part.get_content_type()
if mime not in {"image/png", "image/jpeg", "image/webp"} or not data or len(data) > 10 * 1024 * 1024:
raise HTTPException(400, "Each reference must be PNG, JPEG or WebP, up to 10 MiB")
url = "data:" + mime + ";base64," + base64.b64encode(data).decode()
width, height = image_dimensions(url)
if not (0 < width <= 16384 and 0 < height <= 16384):
raise HTTPException(400, "Invalid reference image dimensions")
images.append(url)
if len(images) > 6:
raise HTTPException(400, "Qwen Image 2.1 supports at most 6 reference images")
elif name in {"model", "prompt", "size", "n", "response_format", "gpu"}:
if name in fields or len(data) > 128000:
raise HTTPException(400, "Duplicate or oversized form field")
try:
fields[name] = data.decode("utf-8")
except UnicodeDecodeError:
raise HTTPException(400, "Form text must be UTF-8")
else:
raise HTTPException(400, "Unsupported image edit field: " + str(name)[:60])
if not images:
raise HTTPException(400, "At least one reference image is required")
return qwen_image_request(fields, images)
def create_app(config):
app = FastAPI(title="Huobao private model bridge", docs_url=None, redoc_url=None)
state_dir = Path(config.get("state_dir", "./state"))
state_dir.mkdir(parents=True, exist_ok=True)
db_path = state_dir / "jobs.sqlite3"
with closing(sqlite3.connect(db_path)) as db, db:
db.execute("CREATE TABLE IF NOT EXISTS jobs (id TEXT PRIMARY KEY, duration REAL NOT NULL, created REAL NOT NULL)")
db.execute("CREATE TABLE IF NOT EXISTS image_jobs (id TEXT PRIMARY KEY, created REAL NOT NULL)")
def auth(authorization: str | None = Header(default=None)):
expected = config.get("token", "")
if not expected or not secrets.compare_digest(authorization or "", "Bearer " + expected):
raise HTTPException(401, "Invalid bridge token")
@app.exception_handler(HTTPException)
async def http_error(request, exc):
return JSONResponse({"error": {"message": str(exc.detail), "type": "bridge_error", "code": exc.status_code}}, status_code=exc.status_code)
@app.exception_handler(httpx.HTTPError)
async def upstream_error(request, exc):
return JSONResponse({"error": {"message": "Upstream connection failed: " + type(exc).__name__, "type": "upstream_error"}}, status_code=502)
def upstream_headers():
return {"Authorization": "Bearer " + config["h3"]["token"]}
async def upstream(method, path, body=None):
if not config.get("h3"):
raise HTTPException(404, "H3 is not configured on this bridge")
async with httpx.AsyncClient(timeout=httpx.Timeout(180, connect=10), trust_env=False) as client:
r = await client.request(method, config["h3"]["url"] + path, headers=upstream_headers(), json=body)
if r.is_error:
try:
detail = r.json().get("detail", "Model service request failed")
except ValueError:
detail = "Model service request failed"
raise HTTPException(r.status_code, str(detail)[:600])
return r.json()
@app.get("/health")
async def health():
return {"status": "ok", "service": "huobao-model-bridge"}
@app.get("/v1/models", dependencies=[Depends(auth)])
async def models():
names = list(config.get("dify", {}))
if config.get("h3"):
# Verify the real service, instead of making a local catalogue look healthy.
registry = await upstream("GET", "/v1/models")
available = {m["id"] for m in registry.get("data", [])}
names.extend(name for name, spec in VIDEO_MODELS.items() if video_workflows(spec) <= available)
if any(m.get("id") == "qwen_image_2_1" and m.get("model_type") == "image" for m in registry.get("data", [])):
names.append("qwen_image_2_1")
if config.get("grok"):
names.append("grok-local")
return {"object": "list", "data": [{"id": n, "object": "model", "owned_by": "private"} for n in names]}
async def dify_events(model, query):
upstream_config = config["dify"][model]
payload = {"inputs": {}, "query": query, "response_mode": "streaming",
"user": "huobao-" + secrets.token_hex(8), "auto_generate_name": False}
finished, usage = False, {}
async with httpx.AsyncClient(timeout=httpx.Timeout(240, connect=10), trust_env=False) as client:
async with client.stream("POST", upstream_config["url"] + "/chat-messages",
headers={"Authorization": "Bearer " + upstream_config["key"]}, json=payload) as response:
if response.is_error:
await response.aread()
try:
message = response.json().get("message", "Dify request failed")
except ValueError:
message = "Dify request failed"
raise HTTPException(response.status_code, str(message)[:400])
async for line in response.aiter_lines():
if not line.startswith("data:"):
continue
event = json.loads(line[5:].strip())
kind = event.get("event")
if kind in {"message", "agent_message"}:
yield "text", event.get("answer", "")
elif kind == "error":
raise HTTPException(502, str(event.get("message", "Dify generation failed"))[:400])
elif kind == "message_end":
finished = True
usage = event.get("metadata", {}).get("usage", {})
elif kind == "workflow_finished":
data = event.get("data") or {}
if data.get("status") not in {None, "succeeded"}:
raise HTTPException(502, str(data.get("error") or "Dify workflow failed")[:400])
if not finished:
raise HTTPException(502, "Dify stream ended before message_end")
yield "usage", usage
async def grok_events(query):
item = config["grok"]
args = [item["exe"], "--cwd", item["cwd"], "--output-format", "json", "--verbatim",
"--no-subagents", "--no-memory", "--disable-web-search", "--max-turns", "1",
"--tools", "", "--deny", "MCPTool", "--permission-mode", "dontAsk",
"--system-prompt-override", "Answer the supplied writing request using text only. Do not use tools."]
options = {"creationflags": 0x08000000} if os.name == "nt" else {}
# A file avoids the Windows command-line length limit for full scripts.
with tempfile.TemporaryDirectory(prefix="huobao-grok-") as scratch:
prompt_file = Path(scratch) / "prompt.txt"
prompt_file.write_text(query, encoding="utf-8")
args += ["--prompt-file", str(prompt_file)]
process = await asyncio.create_subprocess_exec(*args, stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.PIPE, **options)
try:
out, err = await asyncio.wait_for(process.communicate(), timeout=240)
except (asyncio.TimeoutError, asyncio.CancelledError):
process.kill()
await process.wait()
raise
try:
data = json.loads(out.decode("utf-8"))
except (ValueError, UnicodeDecodeError):
raise HTTPException(502, "Grok CLI did not return JSON; run Grok login on this computer")
if process.returncode or not data.get("text"):
raise HTTPException(502, "Grok CLI failed; check local login and available quota")
yield "text", data["text"]
yield "usage", data.get("usage", {})
@app.post("/v1/chat/completions", dependencies=[Depends(auth)])
async def chat(body: dict):
model = body.get("model", "")
if model not in config.get("dify", {}) and not (model == "grok-local" and config.get("grok")):
raise HTTPException(404, "Unknown text model")
query = chat_query(body)
events = grok_events(query) if model == "grok-local" else dify_events(model, query)
cid, created = "chatcmpl-" + secrets.token_hex(12), int(time.time())
def chunk(delta, finish=None):
return {"id": cid, "object": "chat.completion.chunk", "created": created, "model": model,
"choices": [{"index": 0, "delta": delta, "finish_reason": finish}]}
if body.get("stream"):
async def stream():
def encode(value):
return "data: " + json.dumps(value, ensure_ascii=False) + "\n\n"
yield encode(chunk({"role": "assistant", "content": ""}))
try:
async for kind, value in events:
if kind == "text":
yield encode(chunk({"content": value}))
yield encode(chunk({}, "stop"))
except (HTTPException, httpx.HTTPError) as error:
message = str(error.detail) if isinstance(error, HTTPException) else "Upstream connection failed"
yield encode({"error": {"message": message, "type": "upstream_error"}})
yield "data: [DONE]\n\n"
return StreamingResponse(stream(), media_type="text/event-stream", headers={"Cache-Control": "no-cache"})
text, usage = "", {}
async for kind, value in events:
if kind == "text":
text += value
else:
usage = value
return {"id": cid, "object": "chat.completion", "created": created, "model": model,
"choices": [{"index": 0, "message": {"role": "assistant", "content": text}, "finish_reason": "stop"}],
"usage": {"prompt_tokens": usage.get("prompt_tokens", usage.get("input_tokens", 0)),
"completion_tokens": usage.get("completion_tokens", usage.get("output_tokens", 0)),
"total_tokens": usage.get("total_tokens", 0)}}
async def require_qwen_image():
registry = await upstream("GET", "/v1/models")
if not any(m.get("id") == "qwen_image_2_1" and m.get("model_type") == "image" for m in registry.get("data", [])):
raise HTTPException(503, "Qwen Image 2.1 workflow is unavailable")
@app.get("/v1/images/pool", dependencies=[Depends(auth)])
async def image_pool():
await require_qwen_image()
health, catalogue = await asyncio.gather(upstream("GET", "/health"), upstream("GET", "/v1/providers"))
return {"model": "qwen_image_2_1", "auto_balance": health.get("autoBalance") is True,
"routing": "shortest-queue-with-reservations" if health.get("autoBalance") is True else "single-provider",
"configured_workers": [{"id": p["id"], "name": p.get("name", p["id"])} for p in catalogue.get("providers", [])]}
async def submit_image(payload):
await require_qwen_image()
result = await upstream("POST", "/v1/jobs", payload)
jid = result.get("id")
if not isinstance(jid, str) or not jid:
raise HTTPException(502, "Image service did not return a task ID")
with closing(sqlite3.connect(db_path)) as db, db:
db.execute("INSERT INTO image_jobs VALUES(?,?)", (jid, time.time()))
return {"task_id": jid, "status": "processing", "provider_id": result.get("providerId")}
@app.post("/v1/images/validate", dependencies=[Depends(auth)])
async def validate_image(body: dict):
payload = qwen_image_request(body)
await require_qwen_image()
return {"valid": True, "workflow": payload["workflow_key"], "requested_size": [payload["width"], payload["height"]], "max_reference_images": 6,
"pool": await image_pool()}
@app.post("/v1/images/generations", dependencies=[Depends(auth)])
async def generate_image(body: dict):
return await submit_image(qwen_image_request(body))
@app.post("/v1/images/edits", dependencies=[Depends(auth)])
async def edit_image(request: Request):
return await submit_image(await parse_image_edit(request))
@app.post("/v1/images/task/{jid}/cancel", dependencies=[Depends(auth)])
async def cancel_image(jid: str):
with closing(sqlite3.connect(db_path)) as db:
found = db.execute("SELECT 1 FROM image_jobs WHERE id=?", (jid,)).fetchone()
if not found:
raise HTTPException(404, "Unknown image task")
result = await upstream("POST", "/v1/jobs/" + quote(jid, safe="") + "/cancel", {})
return {"task": {"id": jid, "status": result.get("status", "running"),
"cancelRequested": result.get("cancelRequested", False)}}
@app.get("/v1/images/task/{jid}", dependencies=[Depends(auth)])
async def poll_image(jid: str):
with closing(sqlite3.connect(db_path)) as db:
found = db.execute("SELECT 1 FROM image_jobs WHERE id=?", (jid,)).fetchone()
if not found:
raise HTTPException(404, "Unknown image task")
result = await upstream("GET", "/v1/huobao/jobs/" + quote(jid, safe=""))
state = result.get("status", "running")
if state in {"failed", "cancelled"}:
return {"task_id": jid, "status": "failed", "cancelled": state == "cancelled", "error": {"message": str(result.get("error") or "Image generation failed")[:600]}}
if state != "succeeded":
return {"task_id": jid, "status": "processing", "provider_id": result.get("providerId"), "worker_state": result.get("worker_state"), "queue_position": result.get("queue_position")}
output = next((o for o in result.get("outputs", []) if o.get("kind") == "image"), None)
if output is None:
return {"task_id": jid, "status": "failed", "error": {"message": "Task completed without an image output"}}
index, expiry = int(output["index"]), int(time.time()) + 86400
extension = {"image/png": "png", "image/jpeg": "jpg", "image/webp": "webp"}.get(output.get("mimeType"), "png")
# Huobao determines the saved extension from the URL, not Content-Type.
url = f"{config['public_url']}/media/{quote(jid, safe='')}/{index}/image.{extension}?expires={expiry}&sig={signature(jid, index, expiry)}"
return {"task_id": jid, "status": "completed", "image_url": url, "data": [{"url": url}], "provider_id": result.get("providerId")}
@app.post("/v2/video_generation/validate", dependencies=[Depends(auth)])
async def validate_video(body: dict):
request = h3_request(body)
registry = await upstream("GET", "/v1/models")
if request["workflow_key"] not in {m["id"] for m in registry.get("data", [])}:
raise HTTPException(503, "Selected video workflow is unavailable")
plan = await upstream("POST", "/v1/audio-plan", request)
route = await upstream("POST", "/v1/huobao/route", request)
return {"valid": plan.get("status", "ready") == "ready", "workflow": request["workflow_key"],
"width": request["width"], "height": request["height"], "duration": request["duration"], "audio_plan": plan,
"routing": worker_routing(route.get("providerId"))}
@app.post("/v2/video_generation", dependencies=[Depends(auth)])
async def video(body: dict):
# Huobao probes video providers with an empty body. Check the real
# upstream without submitting a GPU task, then report readiness.
if not body:
registry = await upstream("GET", "/v1/models")
available = {m["id"] for m in registry.get("data", [])}
if not {"minimax_h3_t2v_official", "minimax_h3_i2v", "minimax_h3_r2v"} <= available:
raise HTTPException(503, "Required H3 workflows are unavailable")
return {"status": "ready", "probe": True, "model": "MiniMax-H3"}
request = h3_request(body)
registry = await upstream("GET", "/v1/models")
if request["workflow_key"] not in {m["id"] for m in registry.get("data", [])}:
raise HTTPException(503, "Selected video workflow is unavailable")
result = await upstream("POST", "/v1/jobs", request)
jid = result.get("id")
if not isinstance(jid, str) or not jid:
raise HTTPException(502, "Video service did not return a job ID")
with closing(sqlite3.connect(db_path)) as db, db:
db.execute("INSERT INTO jobs VALUES(?,?,?)", (jid, request["duration"], time.time()))
return {"task_id": jid, "status": "submitted", "provider_id": result.get("providerId"), "routing": worker_routing(result.get("providerId"))}
def get_job(jid):
with closing(sqlite3.connect(db_path)) as db:
row = db.execute("SELECT duration FROM jobs WHERE id=?", (jid,)).fetchone()
if not row:
raise HTTPException(404, "Unknown bridge job")
return row[0]
def signature(jid, index, expiry):
data = f"{jid}:{index}:{expiry}".encode()
return hmac.new(config["token"].encode(), data, hashlib.sha256).hexdigest()
@app.post("/v2/video_generation/{jid}/cancel", dependencies=[Depends(auth)])
async def cancel_video(jid: str):
# Only a video submitted through this authenticated bridge may be cancelled.
get_job(jid)
result = await upstream("POST", "/v1/jobs/" + quote(jid, safe="") + "/cancel", {})
return {"task": {"id": jid, "status": result.get("status", "running"),
"cancelRequested": result.get("cancelRequested", False)}}
@app.get("/v2/query/video_generation/{jid}", dependencies=[Depends(auth)])
async def poll(jid: str):
duration = get_job(jid)
result = await upstream("GET", "/v1/huobao/jobs/" + quote(jid, safe=""))
state = result.get("status", "running")
task = {"id": jid, "status": state, "duration": duration, "provider_id": result.get("providerId"), "routing": worker_routing(result.get("providerId")), "worker_state": result.get("worker_state"), "queue_position": result.get("queue_position")}
task["progress"] = result.get("progress")
if state == "succeeded":
output = next((o for o in result.get("outputs", []) if o.get("kind") == "video"), None)
if not output:
task.update(status="failed", error="Video task completed without a video output")
else:
index, expiry = int(output["index"]), int(time.time()) + 86400
url = f"{config['public_url']}/media/{quote(jid, safe='')}/{index}/video.mp4?expires={expiry}&sig={signature(jid, index, expiry)}"
task["content"] = {"url": url}
elif state in {"failed", "cancelled"}:
task["error"] = str(result.get("error") or "Video task failed")[:600]
return {"task": task}
@app.get("/media/{jid}/{index}")
@app.get("/media/{jid}/{index}/{filename}")
async def media(jid: str, index: int, expires: int, sig: str, filename: str | None = None):
if filename is not None and not re.fullmatch(r"image\.(png|jpg|webp)|video\.mp4", filename):
raise HTTPException(404, "Unknown media filename")
with closing(sqlite3.connect(db_path)) as db:
found = db.execute("SELECT 1 FROM jobs WHERE id=? UNION ALL SELECT 1 FROM image_jobs WHERE id=?", (jid, jid)).fetchone()
if not found:
raise HTTPException(404, "Unknown bridge task")
if expires < time.time() or not hmac.compare_digest(sig, signature(jid, index, expires)):
raise HTTPException(403, "Invalid or expired media link; poll the task again")
client = httpx.AsyncClient(timeout=httpx.Timeout(180, connect=10), trust_env=False)
url = config["h3"]["url"] + f"/v1/jobs/{quote(jid, safe='')}/outputs/{index}"
try:
response = await client.send(client.build_request("GET", url, headers=upstream_headers()), stream=True)
if response.is_error:
await response.aclose()
await client.aclose()
raise HTTPException(response.status_code, "Model output is not available")
except BaseException:
await client.aclose()
raise
async def download():
try:
async for part in response.aiter_bytes():
yield part
finally:
await response.aclose()
await client.aclose()
return StreamingResponse(download(), media_type=response.headers.get("content-type", "video/mp4"))
return app
def create_from_file():
path = Path(os.environ.get("HUOBAO_BRIDGE_CONFIG", "/bridge/config.json"))
return create_app(json.loads(path.read_text(encoding="utf-8")))
+68
View File
@@ -0,0 +1,68 @@
"""Install receipt-driven cleanup; restart only private HTTP bridges, never workers."""
import datetime,hashlib,json,pathlib,shutil,sqlite3,subprocess,time,urllib.request,urllib.error
root=pathlib.Path('/home/ps/huobao-video-cleanup')
expected='646600a3c34354689b8d5aec0927b7481583da5478619e4e172f42583e5f4ffc'
bridges=[{'name':'huobao-model-bridge','folder':'/home/ps/huobao-model-bridge','database':'/home/ps/huobao-model-bridge/state/jobs.sqlite3','normalizer':'canvas-h3-prompt-normalizer-healthy-20261026','port':18792},
{'name':'huobao-vdn-bridge','folder':'/home/ps/huobao-vdn/bridge','database':'/home/ps/huobao-vdn/bridge-state/jobs.sqlite3','normalizer':'huobao-vdn-normalizer','port':18794}]
workers={'default':'comfyui','gpu1_8189':'comfyui-manju','gpu3':'h3-caption-gpu3-canary-8a5f','gpu1':'huobao-comfy-gpu1'}
names=list(workers.values())+[b['normalizer'] for b in bridges]
def identities():
return {i['Name']:{'id':i['Id'],'started':i['State']['StartedAt']} for i in json.loads(subprocess.check_output(['docker','inspect',*names],text=True))}
before=identities()
for filename in ['comfy_video_cleanup.candidate.py','video_cleanup_api.candidate.py','bridge-cleanup.candidate.py']:
compile((root/filename).read_text(),filename,'exec')
for entry in bridges:
path=pathlib.Path(entry['folder'])/'bridge.py'
assert hashlib.sha256(path.read_bytes()).hexdigest()==expected,'Bridge changed since review'
assert not (path.parent/'video_cleanup_api.py').exists(),'Unexpected pre-existing cleanup module'
unit=pathlib.Path.home()/'.config/systemd/user/huobao-video-cleanup.service'
assert not unit.exists(),'Unexpected existing service; review first'
backup=root/('backup-'+datetime.datetime.now().strftime('%Y%m%d-%H%M%S'));backup.mkdir(mode=0o700)
for entry in bridges:
folder=pathlib.Path(entry['folder']);shutil.copy2(folder/'bridge.py',backup/(entry['name']+'.py'))
with sqlite3.connect('file:'+entry['database']+'?mode=ro',uri=True) as source,sqlite3.connect(backup/(entry['name']+'.sqlite3')) as target:source.backup(target)
shutil.copy2(root/'comfy_video_cleanup.candidate.py',root/'comfy_video_cleanup.py')
config={'bridges':[{k:b[k] for k in ['name','database','normalizer']}|{'workers':workers} for b in bridges]}
(root/'config.json').write_text(json.dumps(config,indent=2));(root/'config.json').chmod(0o600)
unit.parent.mkdir(parents=True,exist_ok=True)
unit.write_text('[Unit]\nDescription=Huobao verified video cleanup\n\n[Service]\nType=simple\nWorkingDirectory=/home/ps/huobao-video-cleanup\nExecStart=/usr/bin/python3 /home/ps/huobao-video-cleanup/comfy_video_cleanup.py /home/ps/huobao-video-cleanup/config.json\nRestart=on-failure\nRestartSec=10\nTimeoutStopSec=250\nNoNewPrivileges=true\nUMask=0077\n\n[Install]\nWantedBy=default.target\n')
def request(entry,path,body=None,auth=True):
cfg=json.loads((pathlib.Path(entry['folder'])/'config.json').read_text())
headers={'Content-Type':'application/json'}
if auth:headers['Authorization']='Bearer '+cfg['token']
req=urllib.request.Request(f"http://192.168.110.111:{entry['port']}"+path,data=None if body is None else json.dumps(body).encode(),headers=headers)
try:
with urllib.request.urlopen(req,timeout=15) as response:return response.status
except urllib.error.HTTPError as error:return error.code
changed=[]
try:
for entry in bridges:
folder=pathlib.Path(entry['folder']);shutil.copy2(root/'video_cleanup_api.candidate.py',folder/'video_cleanup_api.py')
shutil.copy2(root/'bridge-cleanup.candidate.py',folder/'bridge.py');changed.append(entry)
subprocess.run(['docker','restart','--timeout','60',entry['name']],check=True,capture_output=True)
for attempt in range(15):
try:
if request(entry,'/v1/models')==200:break
except OSError:pass
time.sleep(1)
else:raise RuntimeError('Bridge health check failed')
receipt={'version':1,'index':0,'sha256':'a'*64,'bytes':1}
endpoint='/v2/video_generation/unknown-cleanup-selftest/cleanup'
assert request(entry,endpoint,receipt)==404
assert request(entry,endpoint,receipt,auth=False)==401
with sqlite3.connect(entry['database']) as db:assert db.execute('select count(*) from video_cleanup_acks').fetchone()[0]==0
print(entry['name'],'receipt endpoint ready',flush=True)
subprocess.run(['systemctl','--user','daemon-reload'],check=True,capture_output=True)
subprocess.run(['systemctl','--user','enable','--now','huobao-video-cleanup.service'],check=True,capture_output=True)
time.sleep(1)
assert subprocess.check_output(['systemctl','--user','is-active','huobao-video-cleanup.service'],text=True).strip()=='active'
assert identities()==before,'Unexpected model/normalizer restart'
report={'installed':True,'backup':str(backup),'service':'huobao-video-cleanup.service','model_services_not_restarted':True,'existing_jobs_not_cancelled':True,'pending_cleanup_requests':0}
(root/'deployment.json').write_text(json.dumps(report,indent=2));print(json.dumps(report))
except Exception:
subprocess.run(['systemctl','--user','disable','--now','huobao-video-cleanup.service'],capture_output=True)
for entry in changed:
folder=pathlib.Path(entry['folder']);shutil.copy2(backup/(entry['name']+'.py'),folder/'bridge.py')
(folder/'video_cleanup_api.py').unlink(missing_ok=True)
subprocess.run(['docker','restart','--timeout','60',entry['name']],capture_output=True)
raise
@@ -0,0 +1,62 @@
"""Reproduce the existing storyboard request in a private DB copy; never change live projects or submit media."""
import contextlib,json,os,pathlib,re,shutil,sqlite3,subprocess,time,urllib.request,urllib.error
ROOT=pathlib.Path(__file__).resolve().parent.parent
OUT=ROOT/'integration-work'/('storyboard-loop-diagnostic-'+time.strftime('%Y%m%d-%H%M%S'));OUT.mkdir()
(ROOT/'integration-work/current-storyboard-diagnostic.txt').write_text(str(OUT),'utf-8')
shutil.copytree(pathlib.Path(os.environ['APPDATA'])/'HuobaoDrama/workspace',OUT/'workspace')
dbfile=OUT/'huobao.sqlite3'
with contextlib.closing(sqlite3.connect((ROOT/'huobao-data/huobao.sqlite3').as_uri()+'?mode=ro',uri=True)) as live,contextlib.closing(sqlite3.connect(dbfile)) as db:live.backup(db)
with sqlite3.connect(dbfile) as db:
db.row_factory=sqlite3.Row
db.execute('delete from sys_task');db.execute('delete from video_merges');db.execute("delete from app_settings where key like 'storyboard_job:%'")
ep=dict(db.execute('select * from episodes where id=8').fetchone())
cfg=dict(db.execute("select * from ai_service_configs where service_type='text' and is_active=1 order by priority desc limit 1").fetchone())
model=json.loads(cfg['model'])[0]
chars=[dict(r) for r in db.execute('select id,name from characters where drama_id=? and deleted_at is null',(ep['drama_id'],))]
scenes=[dict(r) for r in db.execute('select id,location from scenes where drama_id=? and deleted_at is null',(ep['drama_id'],))]
body={'drama_id':ep['drama_id'],'episode_id':8,'config_id':cfg['id'],'model':model,'message':'请基于当前集剧本拆分完整分镜,并为每个分镜段落同时生成 video_prompt(中文视频提示词)和 video_prompt_en(英文制作提示词,中文对白及 @/# 素材引用原样保留)。本次视频模型:MiniMax-H3。当前集已有角色:'+json.dumps(chars,ensure_ascii=False)+'。当前集已有场景:'+json.dumps(scenes,ensure_ascii=False)+'。必须从已有角色和场景中选择正确 ID 绑定,按顺序分批保存全部分镜。'}
source=(ROOT/'huobao-source/desktop/build/backend.mjs').read_text('utf-8').replace('maxSteps: 30,', 'maxSteps: 6,')
stage=ROOT/'integration-work/stage-4.0.11/app'
source=source.replace('const require = __hbCreateRequire(import.meta.url)','const require = __hbCreateRequire('+json.dumps(str(stage/'resources/app.asar/build/backend.mjs'))+')',1)
source=re.sub(r'^import (\w+) from "(better-sqlite3|sharp)";',r'const \1 = require("\2");',source,flags=re.M)
instrument=r'''
const __diagFetch=globalThis.fetch; let __diagN=0;
globalThis.fetch=async(input,init)=>{
const url=new URL(typeof input==='string'?input:input.url||String(input));
if(url.hostname==='127.0.0.1')return __diagFetch(input,init);
const n=++__diagN,t=Date.now();let b={};try{b=JSON.parse(init?.body||'{}')}catch{}
console.log('DIAGNOSTIC '+JSON.stringify({event:'request',n,model:b.model,stream:b.stream,max_tokens:b.max_tokens,thinking:b.thinking,messages:b.messages?.length,input_chars:JSON.stringify(b.messages||[]).length,tools:b.tools?.map(x=>x.function.name),recent_tools:b.messages?.filter(x=>x.role==='tool').slice(-2).map(x=>({id:x.tool_call_id,content:String(x.content).slice(0,1200)})),recent_calls:b.messages?.filter(x=>x.tool_calls).slice(-1).map(x=>x.tool_calls.map(t=>({name:t.function.name,args:t.function.arguments.slice(0,150)})))}));
try{const r=await __diagFetch(input,{...init,signal:AbortSignal.any([...(init?.signal?[init.signal]:[]),AbortSignal.timeout(240000)])});
console.log('DIAGNOSTIC '+JSON.stringify({event:'headers',n,status:r.status,seconds:(Date.now()-t)/1000}));
r.clone().text().then(s=>console.log('DIAGNOSTIC '+JSON.stringify({event:'body',n,chars:s.length,seconds:(Date.now()-t)/1000,finish:s.match(/"finish_reason"\s*:\s*"[^" ]+"/g)?.slice(-2)}))).catch(()=>{});return r;
}catch(e){console.log('DIAGNOSTIC '+JSON.stringify({event:'error',n,name:e.name,seconds:(Date.now()-t)/1000}));throw e}
};
'''
(OUT/'backend.mjs').write_text(instrument+source,'utf-8')
env={**os.environ,'ELECTRON_RUN_AS_NODE':'1','PORT':'18988','HUOBAO_DATA_DIR':str(OUT),'SQLITE_PATH':str(dbfile),'FRONTEND_DIST':str(ROOT/'huobao-source/frontend/.output/public'),'WORKSPACE_PATH':str(OUT/'workspace')}
process=None
try:
with (OUT/'backend.log').open('w',encoding='utf-8') as log:
process=subprocess.Popen([str(stage/'HuobaoDrama.exe'),str(OUT/'backend.mjs')],cwd=OUT,env=env,stdout=log,stderr=log,creationflags=subprocess.CREATE_NO_WINDOW)
for _ in range(100):
if process.poll() is not None:raise RuntimeError('Isolated backend failed to start')
try:
urllib.request.urlopen('http://127.0.0.1:18988/api/v1/health',timeout=1).close();break
except OSError:time.sleep(.2)
start=time.monotonic();print(json.dumps({'started':str(OUT),'episode':8,'liveProjectUnmodified':True}),flush=True)
req=urllib.request.Request('http://127.0.0.1:18988/api/v1/agent/storyboard_breaker/jobs',data=json.dumps(body).encode(),headers={'Content-Type':'application/json'})
try:
with urllib.request.urlopen(req,timeout=600) as response:result=json.load(response)
except urllib.error.HTTPError as e:result=json.load(e)
except Exception as e:result={'error':type(e).__name__}
if result.get('data',{}).get('status')=='running':
while time.monotonic()-start<600:
time.sleep(3)
with urllib.request.urlopen('http://127.0.0.1:18988/api/v1/agent/storyboard_breaker/jobs/8',timeout=10) as response: result=json.load(response)
if result.get('data',{}).get('status')!='running':break
(OUT/'result.json').write_text(json.dumps(result,ensure_ascii=False,indent=2),'utf-8')
with sqlite3.connect(dbfile) as db:count=db.execute('select count(*) from storyboards where episode_id=8').fetchone()[0]
print(json.dumps({'seconds':round(time.monotonic()-start,1),'shots':count,'code':result.get('code'),'error':result.get('message') or result.get('error')}),flush=True)
finally:
if process and process.poll() is None:process.terminate();process.wait(15)
@@ -0,0 +1,55 @@
"""Reproduce the existing storyboard request in a private DB copy; never change live projects or submit media."""
import contextlib,json,os,pathlib,re,sqlite3,subprocess,time,urllib.request,urllib.error
ROOT=pathlib.Path(__file__).resolve().parent.parent
OUT=ROOT/'integration-work'/('storyboard-diagnostic-'+time.strftime('%Y%m%d-%H%M%S'));OUT.mkdir()
(ROOT/'integration-work/current-storyboard-diagnostic.txt').write_text(str(OUT),'utf-8')
dbfile=OUT/'huobao.sqlite3'
with contextlib.closing(sqlite3.connect((ROOT/'huobao-data/huobao.sqlite3').as_uri()+'?mode=ro',uri=True)) as live,contextlib.closing(sqlite3.connect(dbfile)) as db:live.backup(db)
with sqlite3.connect(dbfile) as db:
db.row_factory=sqlite3.Row
db.execute('delete from sys_task');db.execute('delete from video_merges')
ep=dict(db.execute('select * from episodes where id=8').fetchone())
cfg=dict(db.execute("select * from ai_service_configs where provider='volcengine' and service_type='text' and is_active=1 order by id desc limit 1").fetchone())
model=json.loads(cfg['model'])[0]
chars=[dict(r) for r in db.execute('select id,name from characters where drama_id=? and deleted_at is null',(ep['drama_id'],))]
scenes=[dict(r) for r in db.execute('select id,location from scenes where drama_id=? and deleted_at is null',(ep['drama_id'],))]
body={'drama_id':ep['drama_id'],'episode_id':8,'config_id':cfg['id'],'model':model,'message':'请基于当前集剧本拆分完整分镜,并为每个分镜段落同时生成 video_prompt(中文视频提示词)和 video_prompt_en(英文制作提示词,中文对白及 @/# 素材引用原样保留)。本次视频模型:MiniMax-H3。当前集已有角色:'+json.dumps(chars,ensure_ascii=False)+'。当前集已有场景:'+json.dumps(scenes,ensure_ascii=False)+'。必须从已有角色和场景中选择正确 ID 绑定,按顺序分批保存全部分镜。'}
source=(ROOT/'huobao-source/desktop/build/backend.mjs').read_text('utf-8')
stage=ROOT/'integration-work/stage-4.0.11/app'
source=source.replace('const require = __hbCreateRequire(import.meta.url)','const require = __hbCreateRequire('+json.dumps(str(stage/'resources/app.asar/build/backend.mjs'))+')',1)
source=re.sub(r'^import (\w+) from "(better-sqlite3|sharp)";',r'const \1 = require("\2");',source,flags=re.M)
instrument=r'''
const __diagFetch=globalThis.fetch; let __diagN=0;
globalThis.fetch=async(input,init)=>{
const url=new URL(typeof input==='string'?input:input.url||String(input));
if(url.hostname==='127.0.0.1')return __diagFetch(input,init);
const n=++__diagN,t=Date.now();let b={};try{b=JSON.parse(init?.body||'{}')}catch{}
console.log('DIAGNOSTIC '+JSON.stringify({event:'request',n,model:b.model,stream:b.stream,max_tokens:b.max_tokens,thinking:b.thinking,messages:b.messages?.length,input_chars:JSON.stringify(b.messages||[]).length}));
try{const r=await __diagFetch(input,{...init,signal:AbortSignal.any([...(init?.signal?[init.signal]:[]),AbortSignal.timeout(240000)])});
console.log('DIAGNOSTIC '+JSON.stringify({event:'headers',n,status:r.status,seconds:(Date.now()-t)/1000}));
r.clone().text().then(s=>console.log('DIAGNOSTIC '+JSON.stringify({event:'body',n,chars:s.length,seconds:(Date.now()-t)/1000,finish:s.match(/"finish_reason"\s*:\s*"[^" ]+"/g)?.slice(-2)}))).catch(()=>{});return r;
}catch(e){console.log('DIAGNOSTIC '+JSON.stringify({event:'error',n,name:e.name,seconds:(Date.now()-t)/1000}));throw e}
};
'''
(OUT/'backend.mjs').write_text(instrument+source,'utf-8')
env={**os.environ,'ELECTRON_RUN_AS_NODE':'1','PORT':'18982','HUOBAO_DATA_DIR':str(OUT),'SQLITE_PATH':str(dbfile),'FRONTEND_DIST':str(ROOT/'huobao-source/frontend/.output/public'),'WORKSPACE_PATH':str(stage/'resources/workspace-template')}
process=None
try:
with (OUT/'backend.log').open('w',encoding='utf-8') as log:
process=subprocess.Popen([str(stage/'HuobaoDrama.exe'),str(OUT/'backend.mjs')],cwd=OUT,env=env,stdout=log,stderr=log,creationflags=subprocess.CREATE_NO_WINDOW)
for _ in range(100):
if process.poll() is not None:raise RuntimeError('Isolated backend failed to start')
try:
urllib.request.urlopen('http://127.0.0.1:18982/api/v1/health',timeout=1).close();break
except OSError:time.sleep(.2)
start=time.monotonic();print(json.dumps({'started':str(OUT),'episode':8,'liveProjectUnmodified':True}),flush=True)
req=urllib.request.Request('http://127.0.0.1:18982/api/v1/agent/storyboard_breaker/chat',data=json.dumps(body).encode(),headers={'Content-Type':'application/json'})
try:
with urllib.request.urlopen(req,timeout=600) as response:result=json.load(response)
except urllib.error.HTTPError as e:result=json.load(e)
except Exception as e:result={'error':type(e).__name__}
(OUT/'result.json').write_text(json.dumps(result,ensure_ascii=False,indent=2),'utf-8')
with sqlite3.connect(dbfile) as db:count=db.execute('select count(*) from storyboards where episode_id=8').fetchone()[0]
print(json.dumps({'seconds':round(time.monotonic()-start,1),'shots':count,'code':result.get('code'),'error':result.get('message') or result.get('error')}),flush=True)
finally:
if process and process.poll() is None:process.terminate();process.wait(15)
@@ -0,0 +1,62 @@
"""Reproduce the existing storyboard request in a private DB copy; never change live projects or submit media."""
import contextlib,json,os,pathlib,re,shutil,sqlite3,subprocess,time,urllib.request,urllib.error
ROOT=pathlib.Path(__file__).resolve().parent.parent
OUT=ROOT/'integration-work'/('storyboard-loop-diagnostic-'+time.strftime('%Y%m%d-%H%M%S'));OUT.mkdir()
(ROOT/'integration-work/current-storyboard-diagnostic.txt').write_text(str(OUT),'utf-8')
shutil.copytree(pathlib.Path(os.environ['APPDATA'])/'HuobaoDrama/workspace',OUT/'workspace')
dbfile=OUT/'huobao.sqlite3'
with contextlib.closing(sqlite3.connect((ROOT/'huobao-data/huobao.sqlite3').as_uri()+'?mode=ro',uri=True)) as live,contextlib.closing(sqlite3.connect(dbfile)) as db:live.backup(db)
with sqlite3.connect(dbfile) as db:
db.row_factory=sqlite3.Row
db.execute('delete from sys_task');db.execute('delete from video_merges');db.execute("delete from app_settings where key like 'storyboard_job:%'")
ep=dict(db.execute('select * from episodes where id=8').fetchone())
cfg=dict(db.execute("select * from ai_service_configs where provider='volcengine' and service_type='text' and is_active=1 order by id desc limit 1").fetchone())
model=json.loads(cfg['model'])[0]
chars=[dict(r) for r in db.execute('select id,name from characters where drama_id=? and deleted_at is null',(ep['drama_id'],))]
scenes=[dict(r) for r in db.execute('select id,location from scenes where drama_id=? and deleted_at is null',(ep['drama_id'],))]
body={'drama_id':ep['drama_id'],'episode_id':8,'config_id':cfg['id'],'model':model,'message':'请基于当前集剧本拆分完整分镜,并为每个分镜段落同时生成 video_prompt(中文视频提示词)和 video_prompt_en(英文制作提示词,中文对白及 @/# 素材引用原样保留)。本次视频模型:MiniMax-H3。当前集已有角色:'+json.dumps(chars,ensure_ascii=False)+'。当前集已有场景:'+json.dumps(scenes,ensure_ascii=False)+'。必须从已有角色和场景中选择正确 ID 绑定,按顺序分批保存全部分镜。'}
source=(ROOT/'huobao-source/desktop/build/backend.mjs').read_text('utf-8').replace('maxSteps: 30,', 'maxSteps: 6,')
stage=ROOT/'integration-work/stage-4.0.11/app'
source=source.replace('const require = __hbCreateRequire(import.meta.url)','const require = __hbCreateRequire('+json.dumps(str(stage/'resources/app.asar/build/backend.mjs'))+')',1)
source=re.sub(r'^import (\w+) from "(better-sqlite3|sharp)";',r'const \1 = require("\2");',source,flags=re.M)
instrument=r'''
const __diagFetch=globalThis.fetch; let __diagN=0;
globalThis.fetch=async(input,init)=>{
const url=new URL(typeof input==='string'?input:input.url||String(input));
if(url.hostname==='127.0.0.1')return __diagFetch(input,init);
const n=++__diagN,t=Date.now();let b={};try{b=JSON.parse(init?.body||'{}')}catch{}
console.log('DIAGNOSTIC '+JSON.stringify({event:'request',n,model:b.model,stream:b.stream,max_tokens:b.max_tokens,thinking:b.thinking,messages:b.messages?.length,input_chars:JSON.stringify(b.messages||[]).length,tools:b.tools?.map(x=>x.function.name),recent_tools:b.messages?.filter(x=>x.role==='tool').slice(-2).map(x=>({id:x.tool_call_id,content:String(x.content).slice(0,1200)})),recent_calls:b.messages?.filter(x=>x.tool_calls).slice(-1).map(x=>x.tool_calls.map(t=>({name:t.function.name,args:t.function.arguments.slice(0,150)})))}));
try{const r=await __diagFetch(input,{...init,signal:AbortSignal.any([...(init?.signal?[init.signal]:[]),AbortSignal.timeout(240000)])});
console.log('DIAGNOSTIC '+JSON.stringify({event:'headers',n,status:r.status,seconds:(Date.now()-t)/1000}));
r.clone().text().then(s=>console.log('DIAGNOSTIC '+JSON.stringify({event:'body',n,chars:s.length,seconds:(Date.now()-t)/1000,finish:s.match(/"finish_reason"\s*:\s*"[^" ]+"/g)?.slice(-2)}))).catch(()=>{});return r;
}catch(e){console.log('DIAGNOSTIC '+JSON.stringify({event:'error',n,name:e.name,seconds:(Date.now()-t)/1000}));throw e}
};
'''
(OUT/'backend.mjs').write_text(instrument+source,'utf-8')
env={**os.environ,'ELECTRON_RUN_AS_NODE':'1','PORT':'18987','HUOBAO_DATA_DIR':str(OUT),'SQLITE_PATH':str(dbfile),'FRONTEND_DIST':str(ROOT/'huobao-source/frontend/.output/public'),'WORKSPACE_PATH':str(OUT/'workspace')}
process=None
try:
with (OUT/'backend.log').open('w',encoding='utf-8') as log:
process=subprocess.Popen([str(stage/'HuobaoDrama.exe'),str(OUT/'backend.mjs')],cwd=OUT,env=env,stdout=log,stderr=log,creationflags=subprocess.CREATE_NO_WINDOW)
for _ in range(100):
if process.poll() is not None:raise RuntimeError('Isolated backend failed to start')
try:
urllib.request.urlopen('http://127.0.0.1:18987/api/v1/health',timeout=1).close();break
except OSError:time.sleep(.2)
start=time.monotonic();print(json.dumps({'started':str(OUT),'episode':8,'liveProjectUnmodified':True}),flush=True)
req=urllib.request.Request('http://127.0.0.1:18987/api/v1/agent/storyboard_breaker/jobs',data=json.dumps(body).encode(),headers={'Content-Type':'application/json'})
try:
with urllib.request.urlopen(req,timeout=600) as response:result=json.load(response)
except urllib.error.HTTPError as e:result=json.load(e)
except Exception as e:result={'error':type(e).__name__}
if result.get('data',{}).get('status')=='running':
while time.monotonic()-start<600:
time.sleep(3)
with urllib.request.urlopen('http://127.0.0.1:18987/api/v1/agent/storyboard_breaker/jobs/8',timeout=10) as response: result=json.load(response)
if result.get('data',{}).get('status')!='running':break
(OUT/'result.json').write_text(json.dumps(result,ensure_ascii=False,indent=2),'utf-8')
with sqlite3.connect(dbfile) as db:count=db.execute('select count(*) from storyboards where episode_id=8').fetchone()[0]
print(json.dumps({'seconds':round(time.monotonic()-start,1),'shots':count,'code':result.get('code'),'error':result.get('message') or result.get('error')}),flush=True)
finally:
if process and process.poll() is None:process.terminate();process.wait(15)
@@ -0,0 +1,61 @@
"""Reproduce the existing storyboard request in a private DB copy; never change live projects or submit media."""
import contextlib,json,os,pathlib,re,sqlite3,subprocess,time,urllib.request,urllib.error
ROOT=pathlib.Path(__file__).resolve().parent.parent
OUT=ROOT/'integration-work'/('storyboard-loop-diagnostic-'+time.strftime('%Y%m%d-%H%M%S'));OUT.mkdir()
(ROOT/'integration-work/current-storyboard-diagnostic.txt').write_text(str(OUT),'utf-8')
dbfile=OUT/'huobao.sqlite3'
with contextlib.closing(sqlite3.connect((ROOT/'huobao-data/huobao.sqlite3').as_uri()+'?mode=ro',uri=True)) as live,contextlib.closing(sqlite3.connect(dbfile)) as db:live.backup(db)
with sqlite3.connect(dbfile) as db:
db.row_factory=sqlite3.Row
db.execute('delete from sys_task');db.execute('delete from video_merges');db.execute("delete from app_settings where key like 'storyboard_job:%'")
ep=dict(db.execute('select * from episodes where id=8').fetchone())
cfg=dict(db.execute("select * from ai_service_configs where provider='volcengine' and service_type='text' and is_active=1 order by id desc limit 1").fetchone())
model=json.loads(cfg['model'])[0]
chars=[dict(r) for r in db.execute('select id,name from characters where drama_id=? and deleted_at is null',(ep['drama_id'],))]
scenes=[dict(r) for r in db.execute('select id,location from scenes where drama_id=? and deleted_at is null',(ep['drama_id'],))]
body={'drama_id':ep['drama_id'],'episode_id':8,'config_id':cfg['id'],'model':model,'message':'请基于当前集剧本拆分完整分镜,并为每个分镜段落同时生成 video_prompt(中文视频提示词)和 video_prompt_en(英文制作提示词,中文对白及 @/# 素材引用原样保留)。本次视频模型:MiniMax-H3。当前集已有角色:'+json.dumps(chars,ensure_ascii=False)+'。当前集已有场景:'+json.dumps(scenes,ensure_ascii=False)+'。必须从已有角色和场景中选择正确 ID 绑定,按顺序分批保存全部分镜。'}
source=(ROOT/'huobao-source/desktop/build/backend.mjs').read_text('utf-8').replace('maxSteps: 30,', 'maxSteps: 6,')
stage=ROOT/'integration-work/stage-4.0.11/app'
source=source.replace('const require = __hbCreateRequire(import.meta.url)','const require = __hbCreateRequire('+json.dumps(str(stage/'resources/app.asar/build/backend.mjs'))+')',1)
source=re.sub(r'^import (\w+) from "(better-sqlite3|sharp)";',r'const \1 = require("\2");',source,flags=re.M)
instrument=r'''
const __diagFetch=globalThis.fetch; let __diagN=0;
globalThis.fetch=async(input,init)=>{
const url=new URL(typeof input==='string'?input:input.url||String(input));
if(url.hostname==='127.0.0.1')return __diagFetch(input,init);
const n=++__diagN,t=Date.now();let b={};try{b=JSON.parse(init?.body||'{}')}catch{}
console.log('DIAGNOSTIC '+JSON.stringify({event:'request',n,model:b.model,stream:b.stream,max_tokens:b.max_tokens,thinking:b.thinking,messages:b.messages?.length,input_chars:JSON.stringify(b.messages||[]).length,tools:b.tools?.map(x=>x.function.name),recent_tools:b.messages?.filter(x=>x.role==='tool').slice(-2).map(x=>({id:x.tool_call_id,content:String(x.content).slice(0,1200)})),recent_calls:b.messages?.filter(x=>x.tool_calls).slice(-1).map(x=>x.tool_calls.map(t=>({name:t.function.name,args:t.function.arguments.slice(0,150)})))}));
try{const r=await __diagFetch(input,{...init,signal:AbortSignal.any([...(init?.signal?[init.signal]:[]),AbortSignal.timeout(240000)])});
console.log('DIAGNOSTIC '+JSON.stringify({event:'headers',n,status:r.status,seconds:(Date.now()-t)/1000}));
r.clone().text().then(s=>console.log('DIAGNOSTIC '+JSON.stringify({event:'body',n,chars:s.length,seconds:(Date.now()-t)/1000,finish:s.match(/"finish_reason"\s*:\s*"[^" ]+"/g)?.slice(-2)}))).catch(()=>{});return r;
}catch(e){console.log('DIAGNOSTIC '+JSON.stringify({event:'error',n,name:e.name,seconds:(Date.now()-t)/1000}));throw e}
};
'''
(OUT/'backend.mjs').write_text(instrument+source,'utf-8')
env={**os.environ,'ELECTRON_RUN_AS_NODE':'1','PORT':'18982','HUOBAO_DATA_DIR':str(OUT),'SQLITE_PATH':str(dbfile),'FRONTEND_DIST':str(ROOT/'huobao-source/frontend/.output/public'),'WORKSPACE_PATH':str(stage/'resources/workspace-template')}
process=None
try:
with (OUT/'backend.log').open('w',encoding='utf-8') as log:
process=subprocess.Popen([str(stage/'HuobaoDrama.exe'),str(OUT/'backend.mjs')],cwd=OUT,env=env,stdout=log,stderr=log,creationflags=subprocess.CREATE_NO_WINDOW)
for _ in range(100):
if process.poll() is not None:raise RuntimeError('Isolated backend failed to start')
try:
urllib.request.urlopen('http://127.0.0.1:18982/api/v1/health',timeout=1).close();break
except OSError:time.sleep(.2)
start=time.monotonic();print(json.dumps({'started':str(OUT),'episode':8,'liveProjectUnmodified':True}),flush=True)
req=urllib.request.Request('http://127.0.0.1:18982/api/v1/agent/storyboard_breaker/jobs',data=json.dumps(body).encode(),headers={'Content-Type':'application/json'})
try:
with urllib.request.urlopen(req,timeout=600) as response:result=json.load(response)
except urllib.error.HTTPError as e:result=json.load(e)
except Exception as e:result={'error':type(e).__name__}
if result.get('data',{}).get('status')=='running':
while time.monotonic()-start<600:
time.sleep(3)
with urllib.request.urlopen('http://127.0.0.1:18982/api/v1/agent/storyboard_breaker/jobs/8',timeout=10) as response: result=json.load(response)
if result.get('data',{}).get('status')!='running':break
(OUT/'result.json').write_text(json.dumps(result,ensure_ascii=False,indent=2),'utf-8')
with sqlite3.connect(dbfile) as db:count=db.execute('select count(*) from storyboards where episode_id=8').fetchone()[0]
print(json.dumps({'seconds':round(time.monotonic()-start,1),'shots':count,'code':result.get('code'),'error':result.get('message') or result.get('error')}),flush=True)
finally:
if process and process.poll() is None:process.terminate();process.wait(15)
@@ -0,0 +1,57 @@
"""Enqueue only the user-authorized, locally fully decoded, preflighted historical videos.
Missing-history tasks require a unique byte-identical file on the original worker,
no live history, no references from any queued/running/other completed task.
The existing exact-file cleanup executor performs the actual unlink and history removal.
"""
import contextlib,importlib.util,json,pathlib,sqlite3,time,traceback
ROOT=pathlib.Path('/home/ps/huobao-video-cleanup');OUT=ROOT/'historical-20261112'
spec=importlib.util.spec_from_file_location('cleaner',ROOT/'comfy_video_cleanup.py');m=importlib.util.module_from_spec(spec);spec.loader.exec_module(m)
bridges={b['name']:b for b in json.loads((ROOT/'config.json').read_text())['bridges']}
inventory={r['id']:r for r in json.loads((ROOT/'historical-inventory.json').read_text())}
probes=json.loads((OUT/'probe-results.json').read_text())+json.loads((OUT/'orphan-probe-results.json').read_text())
probes={r['task']:r for r in probes if r['verified']}
approved=json.loads((ROOT/'historical-approved.json').read_text())
assert len({r['task'] for r in approved})==len(approved)
assert (OUT/'probe-done.json').exists() and (OUT/'orphan-probe-done.json').exists()
# Consistent receipts/journal backup before adding any historical acknowledgements.
for b in bridges.values():
backup=OUT/(b['name']+'-before-enqueue.sqlite3')
if not backup.exists():
with contextlib.closing(sqlite3.connect('file:'+b['database']+'?mode=ro',uri=True)) as db,contextlib.closing(sqlite3.connect(backup)) as target:db.backup(target)
results=[]
for a in approved:
out={'task':a['task'],'id':a['id'],'enqueued':False}
try:
r=inventory[a['task']];p=probes[a['task']];b=bridges[a['bridge']]
assert r['eligible'] and r['localVerified'] and r['status']=='completed'
assert (a['id'],a['receipt'],a['index'])==(r['task_id'],r['receipt'],r['index'])
assert all(a[k]==p[k] for k in ('id','bridge','manifest','worker','fileReceipt')),'Preflight manifest mismatch'
assert b['workers'][a['manifest']['provider']]==a['worker'],'Worker changed'
with contextlib.closing(sqlite3.connect(b['database'],timeout=15)) as db:
db.row_factory=sqlite3.Row
assert db.execute('select 1 from jobs where id=?',(a['id'],)).fetchone(),'Unknown bridge job'
existing=db.execute('select * from video_cleanup_acks where id=?',(a['id'],)).fetchone()
if existing:
assert (existing['output_index'],existing['sha256'],existing['bytes'])==(a['index'],a['receipt']['sha256'],a['receipt']['bytes']),'Conflicting receipt'
assert existing['manifest'] and json.loads(existing['manifest'])['fileReceipt']==a['fileReceipt'],'Conflicting prepared file'
out.update(enqueued=True,status=existing['status']);results.append(out);continue
if a['method']=='live-history-and-download':
fresh=m.docker_python(b['normalizer'],m.NORMALIZER_CODE,{'mode':'verify','id':a['id'],'index':a['index'],**a['receipt']})
assert fresh==a['manifest'],'History changed after preflight'
else:
assert a['method']=='unique-byte-identical-orphan'
assert {k:a['fileReceipt'][k] for k in ('sha256','bytes')}==a['receipt'],'Source is not identical to retained local video'
guard=m.docker_python(b['normalizer'],m.NORMALIZER_CODE,{'mode':'guard','id':a['id'],'manifest':a['manifest']})
if a['method']=='unique-byte-identical-orphan':assert guard['historyMissing'],'History reappeared'
fresh=m.docker_python(a['worker'],m.WORKER_CODE,{'mode':'inspect','locator':a['manifest']['locator']})
assert fresh==a['fileReceipt'],'Video changed since preflight'
manifest={**a['manifest'],'worker':a['worker'],'fileReceipt':a['fileReceipt'],'historicalEvidence':{'method':a['method'],'localTask':a['task'],'localReceipt':a['receipt'],'fullDecodeVerified':True}}
now=time.time()
with db:db.execute("insert into video_cleanup_acks(id,output_index,sha256,bytes,status,manifest,created,updated) values(?,?,?,?,'prepared',?,?,?)",(a['id'],a['index'],a['receipt']['sha256'],a['receipt']['bytes'],json.dumps(manifest),now,now))
out.update(enqueued=True,status='prepared')
except Exception as e:out['reason']=str(e) if type(e) in (RuntimeError,AssertionError) else type(e).__name__
results.append(out)
(OUT/'enqueue-results.json').write_text(json.dumps(results,indent=2))
print(json.dumps(out),flush=True)
(OUT/'enqueue-done.json').write_text(json.dumps({'total':len(results),'enqueued':sum(r['enqueued'] for r in results)}))
@@ -0,0 +1,60 @@
"""Read-only recovery of orphaned history links using unique byte-identical source videos."""
import base64,collections,hashlib,importlib.util,json,pathlib,sqlite3,subprocess
ROOT=pathlib.Path('/home/ps/huobao-video-cleanup');OUT=ROOT/'historical-20261112'
assert (OUT/'probe-done.json').exists()
spec=importlib.util.spec_from_file_location('cleaner',ROOT/'comfy_video_cleanup.py');m=importlib.util.module_from_spec(spec);spec.loader.exec_module(m)
cfg=json.loads((ROOT/'config.json').read_text());bridges={b['name']:b for b in cfg['bridges']}
inventory={r['id']:r for r in json.loads((ROOT/'historical-inventory.json').read_text())}
probes=json.loads((OUT/'probe-results.json').read_text())
pending=[r for r in probes if not r['verified'] and r.get('diagnostic',{}).get('historyPresent') is False]
SCAN=r'''
import hashlib,json,os,pathlib,stat,sys
q=json.load(sys.stdin);sizes=set(q['sizes']);matches=[];examined=0
root=pathlib.Path('/app/output')
for directory,dirs,files in os.walk(root,followlinks=False):
dirs[:]=[n for n in dirs if not (pathlib.Path(directory)/n).is_symlink()]
for name in files:
p=pathlib.Path(directory)/name
if p.suffix.lower() not in ('.mp4','.webm','.mov','.mkv') or p.is_symlink():continue
s=p.stat()
if not stat.S_ISREG(s.st_mode) or s.st_nlink!=1 or s.st_size not in sizes:continue
h=hashlib.sha256();examined+=1
with p.open('rb') as f:
for b in iter(lambda:f.read(1024*1024),b''):h.update(b)
after=p.stat()
if (s.st_size,s.st_mtime_ns,s.st_ino)!=(after.st_size,after.st_mtime_ns,after.st_ino):continue
if h.hexdigest() in q['hashes']:
sub=p.parent.relative_to(root).as_posix()
matches.append({'locator':{'filename':p.name,'subfolder':'' if sub=='.' else sub,'type':'output'},'sha256':h.hexdigest(),'bytes':s.st_size})
print('HB_CLEANUP_RESULT:'+json.dumps({'ok':True,'result':{'matches':matches,'examined':examined}}))
'''
byworker=collections.defaultdict(list)
for p in pending:
b=bridges[p['bridge']];w=b['workers'][p['diagnostic']['provider']];byworker[w].append(p)
scans={}
for worker,items in byworker.items():
receipts=[inventory[p['task']]['receipt'] for p in items]
scans[worker]=m.docker_python(worker,SCAN,{'sizes':list({r['bytes'] for r in receipts}),'hashes':list({r['sha256'] for r in receipts})})
(OUT/'orphan-source-matches.json').write_text(json.dumps(scans,indent=2))
results=[]
for p in pending:
r=inventory[p['task']];b=bridges[p['bridge']];provider=p['diagnostic']['provider'];worker=b['workers'][provider]
out={'task':p['task'],'id':p['id'],'bridge':p['bridge'],'verified':False,'method':'unique-byte-identical-orphan'}
try:
matches=[x for x in scans[worker]['matches'] if x['sha256']==r['receipt']['sha256'] and x['bytes']==r['receipt']['bytes']]
if len(matches)!=1:raise RuntimeError('No byte-identical source video' if not matches else 'Multiple byte-identical source videos; association ambiguous')
same=[x for x in pending if x['task']!=p['task'] and inventory[x['task']]['receipt']==r['receipt'] and x['id']!=p['id']]
if same:raise RuntimeError('Multiple local tasks share identical content; association ambiguous')
raw=base64.urlsafe_b64decode(p['id']+'='*(-len(p['id'])%4)).decode();decoded,uuid=raw.split(':',1)
if decoded!=provider:raise RuntimeError('Provider identity mismatch')
manifest={'provider':provider,'prompt':uuid,'locator':matches[0]['locator'],'historyHash':hashlib.sha256(b'{}').hexdigest()}
guard=m.docker_python(b['normalizer'],m.NORMALIZER_CODE,{'mode':'guard','id':p['id'],'manifest':manifest})
if not guard['historyMissing']:raise RuntimeError('History reappeared; require normal verification')
file=m.docker_python(worker,m.WORKER_CODE,{'mode':'inspect','locator':manifest['locator']})
if {k:file[k] for k in ('sha256','bytes')}!=r['receipt']:raise RuntimeError('Source changed after scan')
out.update(verified=True,manifest=manifest,worker=worker,fileReceipt=file)
except Exception as e:out['reason']=str(e) if type(e) is RuntimeError else type(e).__name__
results.append(out)
(OUT/'orphan-probe-results.json').write_text(json.dumps(results,indent=2))
print(json.dumps({k:v for k,v in out.items() if k in ('task','verified','reason')}),flush=True)
(OUT/'orphan-probe-done.json').write_text(json.dumps({'total':len(results),'verified':sum(r['verified'] for r in results)}))
@@ -0,0 +1,5 @@
import json,pathlib
out=pathlib.Path('/home/ps/huobao-video-cleanup/historical-20261112');p=out/'orphan-probe-results.json'
r=json.loads(p.read_text()) if p.exists() else []
print(json.dumps({'processed':len(r),'verified':sum(x['verified'] for x in r),'done':(out/'orphan-probe-done.json').exists(),'latest':[{k:v for k,v in x.items() if k in ('task','verified','reason')} for x in r[-6:]]}))
if not r:print((out/'orphan-probe.log').read_text()[-1500:])
@@ -0,0 +1,7 @@
import json,subprocess,pathlib
active=subprocess.check_output(['systemctl','--user','is-active','huobao-video-cleanup.service'],text=True).strip()
assert active=='active'
log=subprocess.check_output(['journalctl','--user','-u','huobao-video-cleanup.service','--since','2 minutes ago','--no-pager','-o','cat'],text=True,stderr=subprocess.DEVNULL)
errors=[line for line in log.splitlines() if 'history-reconcile-retry' in line or 'Traceback' in line]
assert not errors,'Cleanup helper reported an error'
print(json.dumps({'active':True,'historyGuardErrors':0}))
@@ -0,0 +1,52 @@
"""Prevent verified deleted historical tasks from resurfacing via signed restore snapshots."""
import contextlib,hashlib,importlib.util,json,pathlib,shutil,sqlite3,subprocess,time
ROOT=pathlib.Path('/home/ps/huobao-video-cleanup');OUT=ROOT/'historical-20261112'
cfg=json.loads((ROOT/'config.json').read_text());approved=json.loads((ROOT/'historical-approved.json').read_text())
assert (OUT/'enqueue-done.json').exists()
candidate=ROOT/'comfy_video_cleanup.candidate.py';compile(candidate.read_text(),str(candidate),'exec')
assert 'def reconcile_cleaned_history(' in candidate.read_text()
bridges={b['name']:b for b in cfg['bridges']};byworker={}
for a in approved:byworker.setdefault(a['worker'],[]).append(a)
workers=json.loads(subprocess.check_output(['docker','inspect',*byworker],text=True))
added=[]
for b in bridges.values():
backup=OUT/(b['name']+'-before-history-guard.sqlite3')
if not backup.exists():
with contextlib.closing(sqlite3.connect('file:'+b['database']+'?mode=ro',uri=True)) as source,contextlib.closing(sqlite3.connect(backup)) as target:source.backup(target)
for i in workers:
env=dict(x.split('=',1) for x in i['Config'].get('Env',[]) if '=' in x);dest=env.get('H3_HISTORY_SNAPSHOT')
if not dest:continue
mount=next((m for m in i['Mounts'] if m['Destination']==dest),None)
if not mount:continue
snapshot=json.loads(pathlib.Path(mount['Source']).read_text())
for a in byworker[i['Name'].lstrip('/')]:
history=snapshot.get('history',{}).get(a['manifest']['prompt'])
if not history:continue
locations=[]
for node in (history.get('outputs') or {}).values():
for field in ('images','gifs','audio'):
for loc in node.get(field) or []:
if isinstance(loc,dict):locations.append({k:loc.get(k,'output' if k=='type' else '') for k in ('filename','subfolder','type')})
assert a['manifest']['locator'] in locations,'Snapshot video locator does not match verified deleted source'
h=hashlib.sha256(json.dumps(history,sort_keys=True,separators=(',',':'),ensure_ascii=False).encode()).hexdigest()
with contextlib.closing(sqlite3.connect(bridges[a['bridge']]['database'],timeout=15)) as db,db:
db.row_factory=sqlite3.Row;row=db.execute('select status,manifest from video_cleanup_acks where id=?',(a['id'],)).fetchone()
assert row and row['status']=='cleaned','Wait for exact video cleanup'
manifest=json.loads(row['manifest']);assert manifest['fileReceipt']==a['fileReceipt']
manifest['restoredHistoryHashes']=sorted(set(manifest.get('restoredHistoryHashes',[])+[h]))
db.execute('update video_cleanup_acks set manifest=? where id=?',(json.dumps(manifest),a['id']))
added.append(a['task'])
# All approved deletions must finish before restarting only the private cleanup helper.
for a in approved:
with contextlib.closing(sqlite3.connect('file:'+bridges[a['bridge']]['database']+'?mode=ro',uri=True)) as db:
assert db.execute('select status from video_cleanup_acks where id=?',(a['id'],)).fetchone()[0]=='cleaned'
old=OUT/'comfy_video_cleanup-before-history-guard.py'
if not old.exists():shutil.copy2(ROOT/'comfy_video_cleanup.py',old)
shutil.copy2(candidate,ROOT/'comfy_video_cleanup.py')
subprocess.run(['systemctl','--user','restart','huobao-video-cleanup.service'],check=True,capture_output=True)
assert subprocess.check_output(['systemctl','--user','is-active','huobao-video-cleanup.service'],text=True).strip()=='active'
before=json.loads((OUT/'services-before.json').read_text())
after={i['Name']:{'id':i['Id'],'started':i['State']['StartedAt']} for i in json.loads(subprocess.check_output(['docker','inspect',*[x.lstrip('/') for x in before]],text=True))}
assert before==after,'Model service identity changed'
report={'historyGuardInstalled':True,'snapshotTasksProtected':added,'pollSeconds':60,'modelServicesUnchanged':True,'snapshotFilesUnchanged':True,'codeSha256':hashlib.sha256((ROOT/'comfy_video_cleanup.py').read_bytes()).hexdigest()}
(OUT/'history-guard-report.json').write_text(json.dumps(report,indent=2));print(json.dumps(report))
@@ -0,0 +1,6 @@
import pathlib,subprocess,json
root=pathlib.Path('/home/ps/huobao-video-cleanup');out=root/'historical-20261112'
assert not (out/'enqueue.log').exists(),'Enqueue already started; inspect its progress'
with (out/'enqueue.log').open('w') as log:
p=subprocess.Popen(['python3',str(root/'historical_cleanup_enqueue_server.py')],stdout=log,stderr=subprocess.STDOUT,start_new_session=True)
print(json.dumps({'started':p.pid,'scope':'approved historical videos only'}))
@@ -0,0 +1,7 @@
import pathlib,subprocess,json
root=pathlib.Path('/home/ps/huobao-video-cleanup');out=root/'historical-20261112'
assert (out/'probe-done.json').exists()
assert not (out/'orphan-probe.log').exists()
with (out/'orphan-probe.log').open('w') as log:
p=subprocess.Popen(['python3',str(root/'historical_cleanup_hash_probe_server.py')],stdout=log,stderr=subprocess.STDOUT,start_new_session=True)
print(json.dumps({'started':p.pid,'mode':'read-only checksum recovery'}))
@@ -0,0 +1,7 @@
import pathlib,subprocess,json
root=pathlib.Path('/home/ps/huobao-video-cleanup')
out=root/'historical-20261112';out.mkdir(exist_ok=True)
assert not (out/'probe.log').exists(),'Preflight already launched'
with (out/'probe.log').open('w') as log:
p=subprocess.Popen(['python3',str(root/'historical_cleanup_probe_server.py')],stdout=log,stderr=subprocess.STDOUT,start_new_session=True)
print(json.dumps({'started':p.pid,'mode':'read-only'}))
@@ -0,0 +1,5 @@
import pathlib,subprocess,json
root=pathlib.Path('/home/ps/huobao-video-cleanup');out=root/'historical-20261112'
with (out/'verify.log').open('w') as log:
p=subprocess.Popen(['python3',str(root/'historical_cleanup_verify_server.py')],stdout=log,stderr=subprocess.STDOUT,start_new_session=True)
print(json.dumps({'started':p.pid,'mode':'read-only final verification'}))
@@ -0,0 +1,22 @@
"""Freeze explicitly verified historical cleanup receipts after rechecking retained local media."""
import contextlib,hashlib,json,pathlib,sqlite3
ROOT=pathlib.Path(__file__).resolve().parent.parent;OUT=ROOT/'integration-work/historical-video-cleanup-20261112';DATA=ROOT/'huobao-data'
inventory={r['id']:r for r in json.loads((OUT/'inventory.json').read_text())}
probes=json.loads((OUT/'server-probe.json').read_text())
if (OUT/'orphan-probe.json').exists():probes+=json.loads((OUT/'orphan-probe.json').read_text())
verified={p['task']:p for p in probes if p['verified']};approved=[]
with contextlib.closing(sqlite3.connect((DATA/'huobao.sqlite3').as_uri()+'?mode=ro',uri=True)) as db:
db.row_factory=sqlite3.Row
for task,p in verified.items():
r=inventory[task];current=db.execute('select * from sys_task where id=?',(task,)).fetchone()
assert current and all(current[k]==r[k] for k in ('status','local_path','task_id','config_id','config_base_url','provider','model')),'Local task changed'
assert current['status']=='completed' and r['localVerified']
f=(DATA/current['local_path']).resolve(strict=True)
assert f.is_relative_to((DATA/'static/videos').resolve())
with f.open('rb') as stream:h=hashlib.file_digest(stream,'sha256').hexdigest()
assert {'bytes':f.stat().st_size,'sha256':h}==r['receipt'],'Local bytes changed'
cfg=db.execute('select base_url,provider,is_active from ai_service_configs where id=?',(r['config_id'],)).fetchone()
assert cfg and cfg['base_url']==r['config_base_url'] and cfg['provider']==r['provider'] and cfg['is_active'],'Original service unavailable'
approved.append({'task':task,'id':r['task_id'],'bridge':p['bridge'],'index':r['index'],'receipt':r['receipt'],'method':p.get('method','live-history-and-download'),'manifest':p['manifest'],'worker':p['worker'],'fileReceipt':p['fileReceipt']})
(OUT/'approved.json').write_text(json.dumps(approved,indent=2),'utf-8')
print(json.dumps({'approved':len(approved),'tasks':[r['task'] for r in approved],'bytes':sum(r['fileReceipt']['bytes'] for r in approved)}))
@@ -0,0 +1,60 @@
"""Read-only preflight for explicitly requested historical-video cleanup."""
import importlib.util,json,pathlib,sqlite3,subprocess,time,traceback
ROOT=pathlib.Path('/home/ps/huobao-video-cleanup')
OUT=ROOT/'historical-20261112';OUT.mkdir(exist_ok=True)
spec=importlib.util.spec_from_file_location('cleaner',ROOT/'comfy_video_cleanup.py');m=importlib.util.module_from_spec(spec);spec.loader.exec_module(m)
config=json.loads((ROOT/'config.json').read_text())
normal=next(b for b in config['bridges'] if 'vdn' not in b['name'])
vdn=next(b for b in config['bridges'] if 'vdn' in b['name'])
SNAPSHOT=r'''
import os,json,pathlib,hashlib
files={}
for folder in ('/app/input','/app/output'):
for root,dirs,names in os.walk(folder,followlinks=False):
for name in names:
p=pathlib.Path(root)/name
if p.suffix.lower() not in ('.png','.jpg','.jpeg','.webp','.bmp','.gif') or p.is_symlink():continue
s=p.stat();files[str(p)]={'bytes':s.st_size,'mtime':s.st_mtime_ns}
print('HB_CLEANUP_RESULT:'+json.dumps({'ok':True,'result':files}))
'''
DIAG=r'''
import asyncio,sys,json,httpx,os,app
q=json.load(sys.stdin)
async def go():
p,pid,e,outputs=await app.job_state(q['id'])
async with httpx.AsyncClient(timeout=30) as c:
r=await c.get('http://127.0.0.1:8105/v1/jobs/'+q['id'],headers={'Authorization':'Bearer '+os.getenv('COMFY_ADAPTER_TOKEN','')});r.raise_for_status();state=r.json()
return {'provider':p.id,'historyPresent':bool(e),'historyStatus':e.get('status',{}).get('status_str'),'historyCompleted':e.get('status',{}).get('completed'),'outputCount':len(outputs),'adapterStatus':state.get('status')}
try:print('HB_CLEANUP_RESULT:'+json.dumps({'ok':True,'result':asyncio.run(go())}))
except Exception as e:print('HB_CLEANUP_RESULT:'+json.dumps({'ok':False,'error':type(e).__name__}))
'''
try:
if not (OUT/'image-snapshot-before.json').exists():
snapshots={w:m.docker_python(w,SNAPSHOT,{}) for w in sorted(set(normal['workers'].values())|set(vdn['workers'].values()))}
(OUT/'image-snapshot-before.json').write_text(json.dumps(snapshots))
containers=list(snapshots)+[normal['normalizer'],vdn['normalizer']]
inspection=json.loads(subprocess.check_output(['docker','inspect',*containers],text=True))
(OUT/'services-before.json').write_text(json.dumps({i['Name']:{'id':i['Id'],'started':i['State']['StartedAt']} for i in inspection}))
results=[]
for r in json.loads((ROOT/'historical-inventory.json').read_text()):
if not r['eligible']:continue
entry=vdn if r['config_base_url'].endswith(':18794') else normal
out={'task':r['id'],'id':r['task_id'],'bridge':entry['name'],'verified':False}
try:
with sqlite3.connect('file:'+entry['database']+'?mode=ro',uri=True) as db:
if not db.execute('select 1 from jobs where id=?',(r['task_id'],)).fetchone():raise RuntimeError('Unknown bridge video')
manifest=m.docker_python(entry['normalizer'],m.NORMALIZER_CODE,{'mode':'verify','id':r['task_id'],'index':r['index'],**r['receipt']})
m.docker_python(entry['normalizer'],m.NORMALIZER_CODE,{'mode':'guard','id':r['task_id'],'manifest':manifest})
worker=entry['workers'][manifest['provider']]
file=m.docker_python(worker,m.WORKER_CODE,{'mode':'inspect','locator':manifest['locator']})
out.update(verified=True,manifest=manifest,worker=worker,fileReceipt=file)
except Exception as e:
out['reason']=str(e) if type(e) is RuntimeError else type(e).__name__
try:out['diagnostic']=m.docker_python(entry['normalizer'],DIAG,{'id':r['task_id']})
except Exception:pass
results.append(out)
(OUT/'probe-results.json').write_text(json.dumps(results,indent=2))
print(json.dumps({k:v for k,v in out.items() if k in ('task','verified','reason','diagnostic')}),flush=True)
(OUT/'probe-done.json').write_text(json.dumps({'total':len(results),'verified':sum(r['verified'] for r in results)}))
except Exception:
(OUT/'probe-failed.txt').write_text(traceback.format_exc());raise
@@ -0,0 +1,4 @@
import pathlib,json
out=pathlib.Path('/home/ps/huobao-video-cleanup/historical-20261112')
p=out/'probe-results.json';r=json.loads(p.read_text()) if p.exists() else []
print(json.dumps({'processed':len(r),'verified':sum(x['verified'] for x in r),'done':(out/'probe-done.json').exists(),'failed':(out/'probe-failed.txt').exists(),'latest':[{k:v for k,v in x.items() if k in ('task','verified','reason','diagnostic')} for x in r[-5:]]}))
@@ -0,0 +1,12 @@
import pathlib,json,subprocess
root=pathlib.Path('/home/ps/huobao-video-cleanup');out=root/'historical-20261112'
approved=json.loads((root/'historical-approved.json').read_text());target={a['manifest']['prompt'] for a in approved}
names=json.loads((out/'services-before.json').read_text());result=[]
for i in json.loads(subprocess.check_output(['docker','inspect',*[n.lstrip('/') for n in names]],text=True)):
env=dict(x.split('=',1) for x in i['Config'].get('Env',[]) if '=' in x);dest=env.get('H3_HISTORY_SNAPSHOT')
if not dest:continue
mount=next((m for m in i['Mounts'] if m['Destination']==dest),None)
if not mount:continue
snapshot=json.loads(pathlib.Path(mount['Source']).read_text());overlap=sorted(set(snapshot.get('history',{}))&target)
result.append({'worker':i['Name'],'matchingCleanedPrompts':overlap})
print(json.dumps(result))
@@ -0,0 +1,13 @@
import pathlib,sqlite3,json,collections
root=pathlib.Path('/home/ps/huobao-video-cleanup');out=root/'historical-20261112';cfg=json.loads((root/'config.json').read_text())
approved=json.loads((root/'historical-approved.json').read_text());ids={r['id']:r['task'] for r in approved};results=[]
for b in cfg['bridges']:
with sqlite3.connect('file:'+b['database']+'?mode=ro',uri=True) as db:
db.row_factory=sqlite3.Row
for r in db.execute('select id,status,attempts,error,manifest from video_cleanup_acks'):
if r['id'] not in ids:continue
results.append({'task':ids[r['id']],'id':r['id'],'bridge':b['name'],'status':r['status'],'attempts':r['attempts'],'error':r['error']})
(out/'cleanup-status.json').write_text(json.dumps(results,indent=2))
print(json.dumps({'approved':len(approved),'acknowledged':len(results),'counts':dict(collections.Counter(r['status'] for r in results)),'enqueueDone':(out/'enqueue-done.json').exists(),'errors':[r for r in results if r['error']]}))
if (out/'enqueue-results.json').exists():
enqueued=json.loads((out/'enqueue-results.json').read_text());print(json.dumps({'prepared':sum(r['enqueued'] for r in enqueued),'blocked':[r for r in enqueued if not r['enqueued']]}))
@@ -0,0 +1,48 @@
"""Mark confirmed server cleanup on existing completed tasks, preserving all local files/records."""
import contextlib,datetime,hashlib,json,pathlib,sqlite3
ROOT=pathlib.Path(__file__).resolve().parent.parent;DATA=ROOT/'huobao-data';OUT=ROOT/'integration-work/historical-video-cleanup-20261112'
approved=json.loads((OUT/'approved.json').read_text());state={r['task']:r for r in json.loads((OUT/'server-status.json').read_text())}
inventory={r['id']:r for r in json.loads((OUT/'inventory.json').read_text())}
assert len(approved)==61 and all(state.get(a['task'],{}).get('status')=='cleaned' for a in approved),'Server cleanup incomplete'
def digest(file):
with file.open('rb') as f:h=hashlib.file_digest(f,'sha256').hexdigest()
return {'sha256':h,'bytes':file.stat().st_size}
for a in approved:
r=inventory[a['task']];f=(DATA/r['local_path']).resolve(strict=True)
assert f.is_relative_to((DATA/'static/videos').resolve()) and digest(f)==a['receipt'],'Retained video changed'
with contextlib.closing(sqlite3.connect(DATA/'huobao.sqlite3',timeout=15)) as db:
db.row_factory=sqlite3.Row
backup=OUT/'before-local-status-sync.sqlite3'
if not backup.exists():
with contextlib.closing(sqlite3.connect(backup)) as target:db.backup(target)
db.execute('BEGIN IMMEDIATE')
for a in approved:
r=inventory[a['task']];current=db.execute('select * from sys_task where id=?',(a['task'],)).fetchone()
assert current and all(current[k]==r[k] for k in ('status','local_path','task_id','config_id','config_base_url','provider','model'))
params=json.loads(current['params'] or '{}')
params['remoteCleanup']={'version':1,'status':'cleaned','receipt':a['receipt'],'localReceipt':a['receipt'],'index':a['index'],'attempts':0,'completedAt':datetime.datetime.now(datetime.timezone.utc).isoformat()}
params.setdefault('downloadReceipt',a['receipt'])
db.execute('update sys_task set params=? where id=?',(json.dumps(params,ensure_ascii=False,separators=(',',':')),a['task']))
db.commit()
for a in approved:
r=db.execute('select status,local_path,params from sys_task where id=?',(a['task'],)).fetchone()
assert r['status']=='completed' and r['local_path']==inventory[a['task']]['local_path'] and json.loads(r['params'])['remoteCleanup']['status']=='cleaned'
assert db.execute('pragma quick_check').fetchone()[0]=='ok'
before_media=json.loads((OUT/'media-before.json').read_text())
missing=[];changed=[]
for relative,receipt in before_media.items():
p=DATA/relative
if not p.is_file():missing.append(relative)
elif digest(p)!=receipt:changed.append(relative)
assert not missing and not changed,'A pre-existing local asset changed'
# Prove this status sync changed only the two cleanup receipt fields.
with contextlib.closing(sqlite3.connect(backup)) as old:
old.row_factory=sqlite3.Row
for a in approved:
left=dict(old.execute('select * from sys_task where id=?',(a['task'],)).fetchone());right=dict(db.execute('select * from sys_task where id=?',(a['task'],)).fetchone())
lp=json.loads(left.pop('params') or '{}');rp=json.loads(right.pop('params') or '{}')
for k in ('remoteCleanup','downloadReceipt'):lp.pop(k,None);rp.pop(k,None)
assert left==right and lp==rp,'Unexpected modification to local task'
report={'tasksMarkedCleaned':len(approved),'completedTasksRetained':len(approved),'localVideosUnchanged':len(approved),'originalLocalMediaUnchanged':len(before_media),'databaseIntegrity':'ok','onlyCleanupReceiptsChanged':True,'backup':str(OUT/'before.sqlite3')}
(OUT/'final-local-report.json').write_text(json.dumps(report,indent=2),'utf-8')
print(json.dumps(report))
@@ -0,0 +1,59 @@
"""Final read-only verification of exact cleanup scope and preserved reference images/services."""
import importlib.util,json,pathlib,sqlite3,subprocess,time
ROOT=pathlib.Path('/home/ps/huobao-video-cleanup');OUT=ROOT/'historical-20261112'
spec=importlib.util.spec_from_file_location('cleaner',ROOT/'comfy_video_cleanup.py');m=importlib.util.module_from_spec(spec);spec.loader.exec_module(m)
bridges={b['name']:b for b in json.loads((ROOT/'config.json').read_text())['bridges']}
approved=json.loads((ROOT/'historical-approved.json').read_text());results=[]
for a in approved:
b=bridges[a['bridge']]
deadline=time.monotonic()+900
while True:
with sqlite3.connect('file:'+b['database']+'?mode=ro',uri=True) as db:
db.row_factory=sqlite3.Row;r=db.execute('select * from video_cleanup_acks where id=?',(a['id'],)).fetchone()
if r and r['status']=='cleaned':break
if time.monotonic()>deadline:raise RuntimeError('Cleanup still pending for task '+str(a['task']))
time.sleep(5)
manifest=json.loads(r['manifest'])
guard=m.docker_python(b['normalizer'],m.NORMALIZER_CODE,{'mode':'guard','id':a['id'],'manifest':manifest})
assert guard['historyMissing'],'History still present'
try:m.docker_python(a['worker'],m.WORKER_CODE,{'mode':'inspect','locator':manifest['locator']})
except RuntimeError as e:
if str(e)!='FileNotFoundError':raise
else:raise RuntimeError('Source video still present')
results.append({'task':a['task'],'historyAbsent':True,'sourceAbsent':True,'sourceBytes':a['fileReceipt']['bytes'],'method':a['method']})
(OUT/'verification-progress.json').write_text(json.dumps(results,indent=2))
images=json.loads((OUT/'image-snapshot-before.json').read_text());changed=[];counts={}
SNAPSHOT=r'''
import os,json,pathlib
files={}
for folder in ('/app/input','/app/output'):
for root,dirs,names in os.walk(folder,followlinks=False):
for name in names:
p=pathlib.Path(root)/name
if p.suffix.lower() not in ('.png','.jpg','.jpeg','.webp','.bmp','.gif') or p.is_symlink():continue
s=p.stat();files[str(p)]={'bytes':s.st_size,'mtime':s.st_mtime_ns}
print('HB_CLEANUP_RESULT:'+json.dumps({'ok':True,'result':files}))
'''
for worker,before in images.items():
after=m.docker_python(worker,SNAPSHOT,{})
different=[p for p,v in before.items() if after.get(p)!=v]
changed.extend({'worker':worker,'path':p} for p in different);counts[worker]={'preserved':len(before)-len(different),'new':len(set(after)-set(before))}
assert not changed,'Reference image snapshot changed'
identities=json.loads((OUT/'services-before.json').read_text())
inspection=json.loads(subprocess.check_output(['docker','inspect',*[name.lstrip('/') for name in identities]],text=True))
after={i['Name']:{'id':i['Id'],'started':i['State']['StartedAt']} for i in inspection}
assert identities==after,'Model service changed'
# Read only operator-configured restore snapshots: identify possible stale history resurrection.
snapshots=[]
targetprompts={a['manifest']['prompt'] for a in approved}
for i in inspection:
env=dict(x.split('=',1) for x in i['Config'].get('Env',[]) if '=' in x)
destination=env.get('H3_HISTORY_SNAPSHOT')
if not destination:continue
mount=next((v for v in i['Mounts'] if v['Destination']==destination),None)
if not mount:continue
value=json.loads(pathlib.Path(mount['Source']).read_text());overlap=sorted(set(value.get('history',{}))&targetprompts)
snapshots.append({'worker':i['Name'],'matchingCleanedPrompts':overlap})
report={'verified':len(results),'removedBytes':sum(r['sourceBytes'] for r in results),'liveHistoryDeleted':sum(r['method']=='live-history-and-download' for r in results),'alreadyAbsentHistory':sum(r['method']=='unique-byte-identical-orphan' for r in results),'referenceImages':counts,'referenceImagesChanged':changed,'modelServicesUnchanged':True,'restoreSnapshots':snapshots,'tasks':results}
(OUT/'final-server-report.json').write_text(json.dumps(report,indent=2))
print(json.dumps({k:v for k,v in report.items() if k!='tasks'}),flush=True)
@@ -0,0 +1,9 @@
import pathlib,json
out=pathlib.Path('/home/ps/huobao-video-cleanup/historical-20261112')
progress=out/'verification-progress.json';done=out/'final-server-report.json'
r=json.loads(progress.read_text()) if progress.exists() else []
print(json.dumps({'verified':len(r),'done':done.exists()}))
if done.exists():print(json.dumps({k:v for k,v in json.loads(done.read_text()).items() if k!='tasks'}))
else:
log=(out/'verify.log').read_text()
if log:print(log[-1500:])
@@ -0,0 +1,56 @@
"""Read-only media verification and consistent DB backup before explicit historical cleanup."""
import concurrent.futures, contextlib, hashlib, json, os, pathlib, re, sqlite3, subprocess, time
from urllib.parse import urlsplit, unquote
ROOT=pathlib.Path(__file__).resolve().parent.parent
DATA=ROOT/'huobao-data'
OUT=ROOT/'integration-work/historical-video-cleanup-20261112'
OUT.mkdir(exist_ok=True)
FF=ROOT/'HuobaoDrama/resources/bin'
def digest(file):
h=hashlib.sha256()
with file.open('rb') as f:
for b in iter(lambda:f.read(1024*1024),b''):h.update(b)
return {'sha256':h.hexdigest(),'bytes':file.stat().st_size}
def verify(row):
result={k:row[k] for k in ('id','status','local_path','task_id','config_id','config_base_url','provider','model')}
try:
relative=row['local_path']
if not re.fullmatch(r'static/videos/[^/\\]+\.(mp4|webm)',relative,re.I):raise ValueError('Unsupported local path')
file=(DATA/relative).resolve(strict=True)
if not file.is_relative_to((DATA/'static/videos').resolve()):raise ValueError('Local path outside video directory')
before=digest(file)
flags={'creationflags':subprocess.CREATE_NO_WINDOW,'capture_output':True,'timeout':180}
p=subprocess.run([str(FF/'ffprobe.exe'),'-v','error','-show_format','-show_streams','-of','json',str(file)],**flags)
if p.returncode:raise ValueError('ffprobe failed')
info=json.loads(p.stdout);v=next((s for s in info['streams'] if s['codec_type']=='video'),{})
duration=float(info['format'].get('duration',0))
if not v.get('width') or not v.get('height') or duration<=0:raise ValueError('Invalid dimensions/duration')
p=subprocess.run([str(FF/'ffmpeg.exe'),'-nostdin','-v','error','-xerror','-threads','2','-i',str(file),'-map','0:v:0','-map','0:a?','-f','null','-'],**flags)
if p.returncode:raise ValueError('Full decode failed')
with file.open('r+b') as f:os.fsync(f.fileno())
if before!=digest(file):raise ValueError('Local video changed during verification')
result.update(localVerified=True,receipt=before,duration=duration,width=v['width'],height=v['height'])
if row['status']!='completed':raise ValueError('Not completed')
if row['provider'] not in ('minimax','localvideo'):raise ValueError('Unsupported provider')
base=urlsplit(row['config_base_url']);url=urlsplit(row['result_url'] or '')
if base.scheme!='http' or base.hostname!='192.168.110.111' or base.port not in (18792,18794) or base.username or base.password:raise ValueError('Unsupported bridge')
parts=url.path.split('/')
if (url.scheme,url.netloc)!=(base.scheme,base.netloc) or len(parts)<5 or parts[1]!='media' or unquote(parts[2])!=row['task_id'] or not parts[3].isdigit():raise ValueError('Source URL does not match task')
result['index']=int(parts[3]);result['eligible']=True
except Exception as e:result.update(eligible=False,reason=str(e) if isinstance(e,ValueError) else type(e).__name__)
return result
with contextlib.closing(sqlite3.connect((DATA/'huobao.sqlite3').as_uri()+'?mode=ro',uri=True)) as db:
db.row_factory=sqlite3.Row
backup=OUT/'before.sqlite3'
if backup.exists():raise RuntimeError('Refusing to replace initial backup')
with contextlib.closing(sqlite3.connect(backup)) as target:db.backup(target)
rows=[dict(r) for r in db.execute("select * from sys_task where type='video' and local_path is not null and local_path!='' order by id")]
media={p.relative_to(DATA).as_posix():digest(p) for p in (DATA/'static').rglob('*') if p.is_file()}
(OUT/'media-before.json').write_text(json.dumps(media,indent=2),'utf-8')
results=[]
with concurrent.futures.ThreadPoolExecutor(max_workers=2) as pool:
for item in pool.map(verify,rows):
results.append(item)
(OUT/'inventory.json').write_text(json.dumps(results,indent=2),'utf-8')
print(json.dumps({'task':item['id'],'verified':item.get('localVerified',False),'eligible':item['eligible'],'reason':item.get('reason')}),flush=True)
print(json.dumps({'total':len(results),'verified':sum(r.get('localVerified',False) for r in results),'eligible':sum(r['eligible'] for r in results),'mediaSnapshot':len(media)}))
@@ -0,0 +1,18 @@
import json,subprocess,pathlib
names=['canvas-h3-prompt-normalizer-healthy-20261026','huobao-vdn-normalizer','comfyui-manju','comfyui','h3-caption-gpu3-canary-8a5f','huobao-comfy-gpu1']
for name in names:
p=subprocess.run(['docker','inspect',name],capture_output=True,text=True)
if p.returncode:print(json.dumps({'container':name,'exists':False}));continue
item=json.loads(p.stdout)[0]
print(json.dumps({'container':name,'workdir':item['Config']['WorkingDir'],'running':item['State']['Running'],'mounts':[{'source':m['Source'],'destination':m['Destination'],'rw':m['RW']} for m in item['Mounts']]}))
code='''import app,json,asyncio,httpx
async def main():
for p in app.providers().values():
async with httpx.AsyncClient(timeout=8) as c:
r=await c.get(p.base_url+'/queue',headers=app.provider_headers(p));r.raise_for_status();q=r.json()
print(json.dumps({'provider':p.id,'base':p.base_url,'running':len(q.get('queue_running',[])),'pending':len(q.get('queue_pending',[]))}))
asyncio.run(main())'''
p=subprocess.run(['docker','exec','canvas-h3-prompt-normalizer-healthy-20261026','python','-c',code],capture_output=True,text=True)
print(p.stdout)
if p.returncode:print('provider inventory failed; inspect locally on server')
print('services',subprocess.check_output(['systemctl','--user','is-system-running'],text=True).strip() if subprocess.run(['systemctl','--user','is-system-running'],capture_output=True).returncode==0 else 'user manager not running or degraded')
+25
View File
@@ -0,0 +1,25 @@
"""Read-only GPU3 history/log inspection for a previously failed task."""
import json, subprocess, re
job = 'Z3B1MzoxZTVhOTVkNC1hMjk5LTQ5OTAtOGU2Yi02M2M1N2EzMTJkMzM'
code = '''import asyncio,httpx,json,app
async def main():
p,pid=app.decode_job(JOB)
async with httpx.AsyncClient(timeout=15) as c:
q=(await c.get(p.base_url+'/queue',headers=app.provider_headers(p))).json()
h=(await c.get(p.base_url+'/history/'+pid,headers=app.provider_headers(p))).json().get(pid,{})
status=h.get('status',{})
errors=[{k:x[1].get(k) for k in ['node_id','node_type','exception_type','exception_message','traceback'] if k in x[1]} for x in status.get('messages',[]) if x[0] in ['execution_error','execution_interrupted']]
print(json.dumps({'prompt_id':pid,'provider':p.id,'running_ids':[x[1] for x in q.get('queue_running',[])],'pending_ids':[x[1] for x in q.get('queue_pending',[])],'history_present':bool(h),'status':status.get('status_str'),'errors':errors,'output_nodes':list(h.get('outputs',{}))},ensure_ascii=False))
asyncio.run(main())'''.replace('JOB',repr(job))
run=subprocess.run(['docker','exec','canvas-h3-prompt-normalizer-healthy-20261026','python','-c',code],capture_output=True,text=True)
print(run.stdout);print(run.stderr)
name='h3-caption-gpu3-canary-8a5f'
details=json.loads(subprocess.check_output(['docker','inspect',name],text=True))[0]
print(json.dumps({'container':name,'started_at':details['State']['StartedAt'],'running':details['State']['Running'],'oom_killed':details['State']['OOMKilled'],'restart_count':details['RestartCount']}))
logs=subprocess.run(['docker','logs','--since','25m','--tail','1800',name],capture_output=True,text=True)
lines=(logs.stdout+'\n'+logs.stderr).splitlines()
matches=[i for i,line in enumerate(lines) if re.search(r'1e5a95d4|execution failed|exception|error|out of memory|Traceback|prompt executed|H3_HISTORY',line,re.I)]
indices=sorted({j for i in matches for j in range(max(0,i-2),min(len(lines),i+6))})
for i in indices:
line=re.sub(r'(?i)(token|api[_-]?key|authorization)(\s*[=:]\s*)\S+',r'\1\2[redacted]',lines[i])
print(line[:700])
+10
View File
@@ -0,0 +1,10 @@
import json,pathlib,hashlib,subprocess
for folder in ['/home/ps/huobao-model-bridge','/home/ps/huobao-vdn/bridge']:
p=pathlib.Path(folder); cfg=json.loads((p/'config.json').read_text())
print(json.dumps({'folder':folder,'hash':hashlib.sha256((p/'bridge.py').read_bytes()).hexdigest(),'upstream':cfg['h3']['url']}))
for item in json.loads(subprocess.check_output(['docker','inspect','huobao-model-bridge','huobao-vdn-bridge','huobao-vdn-normalizer'],text=True)):
print(json.dumps({'container':item['Name'],'image':item['Config']['Image'],'mounts':[{'source':m['Source'],'destination':m['Destination']} for m in item['Mounts']]}))
p=pathlib.Path('/home/ps/huobao-gpu-selection/task-tracking-current.json')
if p.exists():
directory=pathlib.Path(json.loads(p.read_text())['directory'])
print((directory/'manifest.json').read_text())
+10
View File
@@ -0,0 +1,10 @@
"""Create read-only server verification receipts for already downloaded examples; never enqueue cleanup."""
import hashlib,json,pathlib,sqlite3
root=pathlib.Path(__file__).resolve().parent.parent
with sqlite3.connect((root/'huobao-data/huobao.sqlite3').as_uri()+'?mode=ro',uri=True) as db:
db.row_factory=sqlite3.Row;items=[]
for task in db.execute('select * from sys_task where id in (262,273) order by id'):
file=root/'huobao-data'/task['local_path'];parts=task['result_url'].split('/media/',1)[1].split('/')
items.append({'id':task['task_id'],'index':int(parts[1]),'sha256':hashlib.sha256(file.read_bytes()).hexdigest(),'bytes':file.stat().st_size,'localTask':task['id']})
(root/'integration-work/cleanup-readonly-probes.json').write_text(json.dumps(items),'utf-8')
print('Read-only verification examples:',[i['localTask'] for i in items])
@@ -0,0 +1,7 @@
import pathlib,subprocess,json
root=pathlib.Path('/home/ps/huobao-video-cleanup');root.mkdir(mode=0o700,exist_ok=True)
print('private work directory ready')
print(subprocess.check_output(['loginctl','show-user','ps','-p','Linger'],text=True).strip())
for name in ['comfyui','comfyui-manju','h3-caption-gpu3-canary-8a5f','huobao-comfy-gpu1']:
result=subprocess.run(['docker','exec',name,'python','-c','import sys,os; print(sys.version.split()[0],os.path.isdir("/app/output"))'],capture_output=True,text=True)
print(json.dumps({'worker':name,'python_output_dir':result.stdout.strip(),'ok':result.returncode==0}))
+13
View File
@@ -0,0 +1,13 @@
import importlib.util,json,pathlib
root=pathlib.Path('/home/ps/huobao-video-cleanup')
spec=importlib.util.spec_from_file_location('cleaner',root/'comfy_video_cleanup.candidate.py');module=importlib.util.module_from_spec(spec);spec.loader.exec_module(module)
workers={'default':'comfyui','gpu1_8189':'comfyui-manju','gpu3':'h3-caption-gpu3-canary-8a5f','gpu1':'huobao-comfy-gpu1'}
results=[]
for receipt in json.loads((root/'readonly-probes.json').read_text()):
try:
manifest=module.docker_python('canvas-h3-prompt-normalizer-healthy-20261026',module.NORMALIZER_CODE,{'mode':'verify',**receipt})
module.docker_python('canvas-h3-prompt-normalizer-healthy-20261026',module.NORMALIZER_CODE,{'mode':'guard','id':receipt['id'],'manifest':manifest})
file=module.docker_python(workers[manifest['provider']],module.WORKER_CODE,{'mode':'inspect','locator':manifest['locator']})
results.append({'task':receipt['localTask'],'provider':manifest['provider'],'downloadChecksumVerified':True,'historyControlReady':True,'sourceBytes':file['bytes'],'deleted':False})
except RuntimeError as error:results.append({'task':receipt['localTask'],'deleted':False,'blocked':str(error)})
print(json.dumps(results));(root/'readonly-probe-result.json').write_text(json.dumps(results,indent=2))
@@ -0,0 +1,4 @@
import pathlib,subprocess,sys
root=pathlib.Path('/home/ps/huobao-video-cleanup')
result=subprocess.run(['python3',str(root/'test_comfy_video_cleanup.py'),str(root/'comfy_video_cleanup.candidate.py')],capture_output=True,text=True)
print(result.stdout);print(result.stderr);sys.exit(result.returncode)
@@ -0,0 +1,123 @@
"""Linux isolated fixtures for scoped video deletion and crash-safe receipts."""
import asyncio,contextlib,hashlib,importlib.util,io,json,os,pathlib,sqlite3,sys,tempfile,types
source=pathlib.Path(sys.argv[1])
spec=importlib.util.spec_from_file_location('cleaner',source);cleaner=importlib.util.module_from_spec(spec);spec.loader.exec_module(cleaner)
PID='11111111-1111-4111-8111-111111111111';OTHER='22222222-2222-4222-8222-222222222222'
video=b'fixture video data';digest=hashlib.sha256(video).hexdigest()
entry={'status':{'completed':True,'status_str':'success'},'outputs':{'1':{'gifs':[{'filename':'own.mp4','subfolder':'shots','type':'output'}]}}}
history={PID:entry,OTHER:{'outputs':{'2':{'gifs':[{'filename':'other.mp4','subfolder':'shots','type':'output'}]}}}}
queue={'queue_running':[],'queue_pending':[]};state='succeeded';output_kind='video';calls=[]
def execute(code,request):
stdin,stdout=sys.stdin,sys.stdout;capture=io.StringIO()
try:
sys.stdin=io.StringIO(json.dumps(request));sys.stdout=capture
try:exec(code,{'__name__':'fixture'})
except SystemExit:pass
finally:sys.stdin,sys.stdout=stdin,stdout
values=[line.split(':',1)[1] for line in capture.getvalue().splitlines() if line.startswith('HB_CLEANUP_RESULT:')]
assert values,capture.getvalue()
result=json.loads(values[-1])
if not result['ok']:raise RuntimeError(result.get('error'))
return result['result']
app=types.ModuleType('app')
provider=types.SimpleNamespace(id='default',base_url='http://worker')
app.decode_job=lambda _: (provider,PID)
app.provider_headers=lambda _: {'fixture':'only'}
app.providers=lambda: {'default':provider}
async def job_state(_):return provider,PID,history.get(PID,{}),[{'filename':'own.mp4','kind':output_kind}]
app.job_state=job_state
class Reply:
def __init__(self,value):self.value=value
def json(self):return self.value
def raise_for_status(self):pass
async def aiter_bytes(self):yield video
async def __aenter__(self):return self
async def __aexit__(self,*args):pass
class Client:
def __init__(self,**kwargs):pass
async def __aenter__(self):return self
async def __aexit__(self,*args):pass
async def get(self,url,**kwargs):
if '/v1/jobs/' in url:return Reply({'status':state})
if url.endswith('/queue'):return Reply(queue)
if url.endswith('/history'):return Reply(history)
return Reply({PID:history[PID]} if PID in history else {})
async def post(self,url,**kwargs):
assert url=='http://worker/history';body=kwargs['json'];assert set(body)=={'delete'}
assert body['delete'] in ([],[PID]);calls.append(body)
for key in body['delete']:history.pop(key,None)
return Reply({})
def stream(self,*args,**kwargs):return Reply(None)
httpx=types.ModuleType('httpx');httpx.AsyncClient=Client
sys.modules['app']=app;sys.modules['httpx']=httpx
with tempfile.TemporaryDirectory(prefix='huobao-cleanup-fixture-',dir=source.parent) as directory:
root=pathlib.Path(directory);out=root/'output';(out/'shots').mkdir(parents=True)
# All recursive test teardown remains within the newly created private fixture.
assert root.resolve().parent==source.parent.resolve()
code=cleaner.WORKER_CODE.replace("'/app/output'",repr(str(out)))
locator={'filename':'own.mp4','subfolder':'shots','type':'output'}
def worker(mode,loc=locator,**kw):return execute(code,{'mode':mode,'locator':loc,**kw})
def fail(fn):
try:fn()
except RuntimeError:return
raise AssertionError('Unsafe operation accepted')
own=out/'shots/own.mp4';other=out/'shots/other.mp4';own.write_bytes(video);other.write_bytes(video)
receipt=worker('inspect');assert receipt['sha256']==digest
worker('delete',expected=receipt);assert not own.exists() and other.exists()
assert worker('delete',expected=receipt)['missing']
own.write_bytes(video);receipt=worker('inspect');own.write_bytes(b'replaced')
fail(lambda:worker('delete',expected=receipt));assert own.exists()
for loc in [dict(locator,filename='../own.mp4'),dict(locator,subfolder='../output/shots'),dict(locator,type='input'),dict(locator,filename='image.png'),dict(locator,filename='own\x00.mp4')]:
fail(lambda:worker('delete',loc,expected=receipt))
outside=root/'protected.mp4';outside.write_bytes(video)
(out/'symlink.mp4').symlink_to(outside)
fail(lambda:worker('inspect',dict(locator,subfolder='',filename='symlink.mp4')))
(out/'linked').symlink_to(root,target_is_directory=True)
fail(lambda:worker('inspect',dict(locator,subfolder='linked',filename='protected.mp4')))
os.link(outside,out/'hard.mp4')
fail(lambda:worker('inspect',dict(locator,subfolder='',filename='hard.mp4')))
assert outside.read_bytes()==video
verify={'mode':'verify','id':'fixture','index':0,'sha256':digest,'bytes':len(video)}
manifest=execute(cleaner.NORMALIZER_CODE,verify)
fail(lambda:execute(cleaner.NORMALIZER_CODE,{**verify,'sha256':'a'*64}))
state='running';fail(lambda:execute(cleaner.NORMALIZER_CODE,verify));state='succeeded'
output_kind='image';fail(lambda:execute(cleaner.NORMALIZER_CODE,verify));output_kind='video'
guard={'mode':'guard','id':'fixture','manifest':manifest}
queue['queue_running']=[[1,PID]];fail(lambda:execute(cleaner.NORMALIZER_CODE,guard));queue['queue_running']=[]
queue['queue_pending']=[[1,OTHER,{'video':'shots/own.mp4'}]];fail(lambda:execute(cleaner.NORMALIZER_CODE,guard));queue['queue_pending']=[]
other_entry=history[OTHER];history[OTHER]=entry;fail(lambda:execute(cleaner.NORMALIZER_CODE,guard));history[OTHER]=other_entry
entry['changed']=True;fail(lambda:execute(cleaner.NORMALIZER_CODE,guard));del entry['changed']
execute(cleaner.NORMALIZER_CODE,{**guard,'mode':'remove_history'});assert PID not in history and OTHER in history
history[PID]=entry;own.write_bytes(video)
database=root/'jobs.sqlite3'
with contextlib.closing(sqlite3.connect(database)) as db,db:
db.row_factory=sqlite3.Row
db.execute('create table jobs(id text primary key)');db.execute("insert into jobs values('fixture')")
db.execute('create table video_cleanup_acks(id text primary key,output_index integer,sha256 text,bytes integer,status text,manifest text,attempts integer,next_attempt real,error text,created real,updated real)')
db.execute("insert into video_cleanup_acks values('fixture',0,?,?,'pending',null,0,0,null,0,0)",(digest,len(video)))
fault=[True];verifications=[]
def run(container,script,payload):
if script==cleaner.WORKER_CODE:return execute(code,payload)
if payload['mode']=='verify':verifications.append(True)
if payload['mode']=='remove_history' and fault[0]:fault[0]=False;raise RuntimeError('simulated crash after unlink')
return execute(script,payload)
config={'name':'fixture','normalizer':'fixture','workers':{'default':'worker'}}
row=lambda:db.execute("select * from video_cleanup_acks where id='fixture'").fetchone()
fail(lambda:cleaner.process_ack(db,config,row(),run))
assert row()['status']=='prepared' and not own.exists() and PID in history and other.exists()
cleaner.process_ack(db,config,row(),run)
assert row()['status']=='cleaned' and PID not in history and OTHER in history and len(verifications)==1
assert other.read_bytes()==video and outside.read_bytes()==video
# A restored, exact old snapshot is removed; active or altered records survive.
snapshot_hash=hashlib.sha256(json.dumps(entry,sort_keys=True,separators=(',',':'),ensure_ascii=False).encode()).hexdigest()
reconcile={'providers':{'default':[{'prompt':PID,'hashes':[snapshot_hash]}]}}
history[PID]=entry
queue['queue_running']=[[1,PID]]
assert execute(cleaner.CLEANED_HISTORY_CODE,reconcile)['deferred']==[PID] and PID in history
queue['queue_running']=[];entry['changed']=True
assert execute(cleaner.CLEANED_HISTORY_CODE,reconcile)['deferred']==[PID] and PID in history
del entry['changed']
assert execute(cleaner.CLEANED_HISTORY_CODE,reconcile)['removed']==[PID]
assert PID not in history and OTHER in history and other.read_bytes()==video
assert execute(cleaner.CLEANED_HISTORY_CODE,reconcile)['removed']==[]
print(json.dumps({'passed':True,'checks':['exact video only','idempotent missing file retry','changed inode/hash protection','path traversal denied','symlink/hardlink denied','download checksum proof','completed video only','running/queued/shared file protection','single history delete','crash recovery after unlink','other files and history unchanged']}))
@@ -0,0 +1,204 @@
"""End-to-end isolated image->video lifecycle tests. Never calls live models or the live DB."""
import base64, hashlib, json, os, pathlib, re, sqlite3, struct, subprocess, threading, time, urllib.request, urllib.error, zlib
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from email.parser import BytesParser
from email.policy import default
root=pathlib.Path(__file__).resolve().parent.parent
data=root/'integration-work'/('composed-frame-test-'+time.strftime('%Y%m%d-%H%M%S'));data.mkdir()
stage=root/'integration-work/stage-4.0.11/app';dbfile=data/'huobao.sqlite3'
source=(root/'huobao-source/desktop/build/backend.mjs').read_text('utf-8')
source=source.replace('const require = __hbCreateRequire(import.meta.url)','const require = __hbCreateRequire('+json.dumps(str(stage/'resources/app.asar/build/backend.mjs'))+')',1)
source=re.sub(r'^import (\w+) from "(better-sqlite3|sharp)";',r'const \1 = require("\2");',source,flags=re.M)
source=re.sub(r'(\w+)\.hostname === "192\.168\.110\.111" && \["18792", "18794"\]\.includes\(\1.port\)',r'\1.hostname === "127.0.0.1" && \1.port === "18971"',source)
source=re.sub(r'&& !!taskRouting\(\{ provider: config\d*\.provider, taskId \}\)','&& !!taskId',source)
source=source.replace('intervalMs: 1e4','intervalMs: 100').replace('intervalMs: 5e3','intervalMs: 100')
(data/'backend.mjs').write_text(source,'utf-8')
def png(color):
def chunk(tag,raw):return struct.pack('>I',len(raw))+tag+raw+struct.pack('>I',zlib.crc32(tag+raw)&0xffffffff)
return b'\x89PNG\r\n\x1a\n'+chunk(b'IHDR',struct.pack('>IIBBBBB',32,18,8,2,0,0,0))+chunk(b'IDAT',zlib.compress((b'\0'+bytes(color)*32)*18))+chunk(b'IEND',b'')
events=[];image_jobs={};image_mode='complete';image_submit_release=threading.Event();image_submit_release.set();video_submit_release=threading.Event();video_submit_release.set()
class Mock(BaseHTTPRequestHandler):
def log_message(self,*args):pass
def reply(self,status,body):
try:
raw=json.dumps(body).encode();self.send_response(status);self.send_header('Content-Type','application/json');self.end_headers();self.wfile.write(raw)
except (BrokenPipeError,ConnectionResetError):pass
def do_POST(self):
raw=self.rfile.read(int(self.headers.get('Content-Length','0')))
if 'multipart/form-data' in self.headers.get('Content-Type',''):
message=BytesParser(policy=default).parsebytes(('Content-Type: '+self.headers['Content-Type']+'\r\nMIME-Version: 1.0\r\n\r\n').encode()+raw)
body={'images':[]}
for part in message.iter_parts():
name=part.get_param('name',header='content-disposition')
if name=='image[]':body['images'].append(hashlib.sha256(part.get_payload(decode=True)).hexdigest())
else:body[name]=part.get_payload(decode=True).decode()
else:body=json.loads(raw or b'{}')
events.append((self.path,body))
if self.path.endswith('/images/edits'):
jid='img-'+str(len(image_jobs)+1);image_jobs[jid]={'status':image_mode,'color':(len(image_jobs)*11%200,120,210)}
image_submit_release.wait(25);self.reply(200,{'task_id':jid,'status':'processing'});return
if '/images/task/' in self.path and self.path.endswith('/cancel'):
jid=self.path.split('/')[-2];image_jobs[jid]['status']='cancelled';self.reply(200,{'task':{'id':jid,'status':'cancelled'}});return
if self.path.endswith('/validate'):self.reply(200,{'valid':True});return
if self.path.endswith('/cancel'):self.reply(200,{'task':{'id':self.path.split('/')[-2],'status':'cancelled'}});return
if self.path.endswith('/video_generation'):
video_submit_release.wait(25);self.reply(200,{'task_id':'vid-'+str(len(events))});return
if self.path.endswith('/chat/completions'):
content='An establishing shot. '+' '.join(re.findall(r'__HB_KEEP_\d+__',body['messages'][-1]['content']))
self.reply(200,{'choices':[{'message':{'content':json.dumps({'translation':content})}}]});return
self.reply(400,{'error':'unexpected fixture route'})
def do_GET(self):
if self.path.startswith('/image/'):
raw=png(image_jobs[self.path.split('/')[-1]]['color']);self.send_response(200);self.send_header('Content-Type','image/png');self.end_headers();self.wfile.write(raw);return
if '/images/task/' in self.path:
jid=self.path.split('/')[-1];state=image_jobs[jid]['status']
if state=='complete':self.reply(200,{'status':'completed','image_url':'http://127.0.0.1:18971/image/'+jid})
elif state in ('fail','cancelled'):self.reply(200,{'status':'failed','cancelled':state=='cancelled','error':{'message':'Fixture image failure'}})
else:self.reply(200,{'status':'processing','worker_state':'queued'})
return
self.reply(200,{'task':{'id':self.path.split('/')[-1],'status':'running','worker_state':'running'}})
server=ThreadingHTTPServer(('127.0.0.1',18971),Mock);threading.Thread(target=server.serve_forever,daemon=True).start()
env={**os.environ,'FFMPEG_BIN':str(stage/'resources/bin/ffmpeg.exe'),'FFPROBE_BIN':str(stage/'resources/bin/ffprobe.exe'),'ELECTRON_RUN_AS_NODE':'1','PORT':'18970','HUOBAO_DATA_DIR':str(data),'SQLITE_PATH':str(dbfile),'FRONTEND_DIST':str(root/'huobao-source/frontend/.output/public'),'WORKSPACE_PATH':str(stage/'resources/workspace-template')}
log=(data/'backend.log').open('w',encoding='utf-8');process=None
def api(method,path,body=None,expected=200):
req=urllib.request.Request('http://127.0.0.1:18970/api/v1'+path,method=method,data=None if body is None else json.dumps(body).encode(),headers={'Content-Type':'application/json'})
try:
with urllib.request.urlopen(req,timeout=30) as response:status=response.status;value=json.load(response)
except urllib.error.HTTPError as error:status=error.code;value=json.load(error)
assert status==expected,(path,status,value)
return value.get('data',value)
def sql(query,args=()):
with sqlite3.connect(dbfile) as db:db.row_factory=sqlite3.Row;return [dict(r) for r in db.execute(query,args)]
def task(id):return sql('select * from sys_task where id=?',(id,))[0]
def plan(id):return json.loads(task(id)['params'])['firstFrameComposition']
def wait(fn,seconds=15):
deadline=time.monotonic()+seconds
while time.monotonic()<deadline:
value=fn()
if value:return value
time.sleep(.1)
raise AssertionError('Condition did not become true')
def start():
global process
process=subprocess.Popen([str(stage/'HuobaoDrama.exe'),str(data/'backend.mjs')],cwd=data,env=env,stdout=log,stderr=log,creationflags=subprocess.CREATE_NO_WINDOW)
for _ in range(150):
assert process.poll() is None
try:api('GET','/health');return
except OSError:time.sleep(.1)
raise AssertionError('Backend did not start')
def stop():
global process
process.terminate();process.wait(10);process=None
def send(shot,**more):return api('POST','/tasks',{'type':'video','storyboard_id':shot,'drama_id':1,'config_id':1,'model':'LTX-2.3','frame_image_config_id':2,'frame_image_model':'qwen_image_2_1','prompt':'0-4s: @图片1404宿舍, @图片2Girl looks at @Other.','duration':4,'gpu':'gpu3','force_new':True,**more},201)['id']
def videos():return [b for p,b in events if p.endswith('/video_generation')]
def images():return [b for p,b in events if p.endswith('/images/edits')]
def finished_frame(id):
wait(lambda:task(id)['task_id'] or task(id)['status']=='failed')
assert task(id)['task_id'],task(id)
return plan(id)
try:
start();ts='2026-11-12T00:00:00Z'
sql('insert into dramas(id,title,created_at,updated_at) values(1,?,?,?)',('Composition fixture',ts,ts))
sql('insert into episodes(id,drama_id,episode_number,title,content,script_content,image_config_id,created_at,updated_at) values(1,1,1,?,?,?,?,?,?)',('Fixture','A girl in a dormitory.','A girl in a dormitory.',2,ts,ts))
for id,service,provider,model,gpu in [(1,'video','localvideo','LTX-2.3','gpu3'),(2,'image','openai','qwen_image_2_1','gpu1'),(3,'text','openai','fixture-translator','auto')]:
sql('insert into ai_service_configs(id,service_type,provider,name,base_url,api_key,model,settings,is_active,created_at,updated_at) values(?,?,?,?,?,?,?,?,1,?,?)',(id,service,provider,'Fixture '+service,'http://127.0.0.1:18971','fixture-only',json.dumps([model,'Wan2.2-I2V','MiniMax-H3'] if service=='video' else [model]),json.dumps({'gpu':gpu}),ts,ts))
(data/'static/images').mkdir(parents=True)
for i in range(16):(data/f'static/images/input{i}.png').write_bytes(png((i*13%255,15,70)))
sql("insert into scenes(id,drama_id,location,time,prompt,lighting,image_url,created_at,updated_at) values(1,1,?,'night','Blue walls; east window; oak desk.','Warm east lamp.','static/images/input0.png',?,?)",('404宿舍',ts,ts))
sql('insert into episode_scenes(episode_id,scene_id,created_at) values(1,1,?)',(ts,))
for i in range(1,14):
name='Girl' if i==1 else 'Other' if i==2 else 'Character'+str(i)
sql('insert into characters(id,drama_id,name,image_url,created_at,updated_at) values(?,1,?,?,?,?)',(i,name,f'static/images/input{i}.png',ts,ts))
sql('insert into episode_characters(episode_id,character_id,created_at) values(1,?,?)',(i,ts))
sql("insert into props(id,drama_id,name,image_url,created_at,updated_at) values(1,1,'Lamp','static/images/input14.png',?,?)",(ts,ts))
sql('insert into episode_props(episode_id,prop_id,created_at) values(1,1,?)',(ts,))
for i in range(1,24):
sql("insert into storyboards(id,episode_id,storyboard_number,scene_id,description,video_prompt,duration,created_at,updated_at) values(?,1,?,1,'Girl looks up in 404 dormitory.','0-4s: @404宿舍, @Girl looks at @Other.',4,?,?)",(i,i,ts,ts))
for c in [1,2]:sql('insert into storyboard_characters(storyboard_id,character_id) values(?,?)',(i,c))
sql('insert into storyboard_props(storyboard_id,prop_id) values(?,1)',(i,))
sources={name:sql('select * from '+name+' order by id') for name in ['characters','scenes','props']}
# Real multipart edits receive every bound asset, and videos wait for that output.
image_mode='hold';one=send(1);wait(lambda:len(images())==1)
assert not videos() and len(images()[0]['images'])==4 and images()[0]['gpu']=='gpu1'
assert '404宿舍' in images()[0]['prompt'] and '单个' not in images()[0]['prompt']
image_jobs['img-1']['status']='complete';first=finished_frame(one)
assert first['frameUrl'] and sql('select first_frame_image from storyboards where id=1')[0]['first_frame_image']==first['frameUrl']
v=videos()[-1];assert v['model']=='LTX-2.3' and v['gpu']=='gpu3'
refs=[x for x in v['content'] if x['type']=='image_url'];assert len(refs)==1 and refs[0]['role']=='first_frame'
assert '@图片' not in v['content'][0]['text'] and '@404宿舍' in v['content'][0]['text']
assert base64.b64decode(refs[0]['image_url']['url'].split(',')[1])==(data/first['frameUrl']).read_bytes()
# Same exact input reuses the finished frame across LTX/Wan video retries.
sql("update sys_task set status='failed',error_code='generation_failed' where id=?",(one,));before=len(images())
two=send(1,model='Wan2.2-I2V');reused=finished_frame(two)
assert len(images())==before and reused['frameUrl']==first['frameUrl'] and reused['reusedFromTaskId']==one
# Cancellation stops the child image and never sends its video.
cancelled=send(2);wait(lambda:len(images())==before+1);wait(lambda:task(plan(cancelled)['childTaskIds'][0])['task_id'])
before_v=len(videos());result=api('POST',f'/tasks/{cancelled}/cancel',{})
assert result['outcome']=='cancelled';assert task(cancelled)['error_code']=='user_cancelled';assert len(videos())==before_v
# All 15 original references (> route's old 9-image limit) reach Qwen over 3 rounds.
for c in range(3,14):sql('insert into storyboard_characters(storyboard_id,character_id) values(3,?)',(c,))
image_mode='complete';before=len(images());many=send(3,reference_image_urls=[f'static/images/input{i}.png' for i in range(15)])
multi=finished_frame(many);assert len(multi['assets'])==15 and multi['rounds']==3 and len(multi['images'])==3
assert [len(x['images']) for x in images()[before:]]==[6,6,5]
originals=[]
for child_id in multi['childTaskIds']:originals+=json.loads(task(child_id)['params'])['referenceAssets']
assert originals==multi['assets'] and len(videos()[-1]['content'])==2
# Missing references fail before replacement touches an existing running video.
sql("update characters set image_url=null where id=2");before=len(events)
api('POST','/tasks',{'type':'video','storyboard_id':1,'model':'LTX-2.3','config_id':1,'prompt':'test','replace_task_id':two,'force_new':True},400)
assert len(events)==before and task(two)['status']=='processing'
sql("update characters set image_url='static/images/input2.png' where id=2")
# Image failure cannot emit a video or overwrite source assets.
image_mode='fail';before_v=len(videos());failed=send(4)
wait(lambda:task(failed)['status']=='failed');assert '首帧合成' in task(failed)['error_msg'] and len(videos())==before_v
# Edits during generation keep the old output in history but prohibit stale video submission.
image_mode='hold';stale=send(5);wait(lambda:plan(stale)['childTaskIds']);child=plan(stale)['childTaskIds'][0];wait(lambda:task(child)['task_id'])
sql("update storyboards set description='Changed pose' where id=5");image_jobs[task(child)['task_id']]['status']='complete'
wait(lambda:task(stale)['status']=='failed');assert '已变化' in task(stale)['error_msg'] and len(videos())==before_v
assert task(child)['local_path'] and not sql('select first_frame_image from storyboards where id=5')[0]['first_frame_image']
# Restart follows the SAME image task; no duplicate image or video generation.
restart=send(6);wait(lambda:plan(restart)['childTaskIds']);child=plan(restart)['childTaskIds'][0];wait(lambda:task(child)['task_id'])
jid=task(child)['task_id'];before=len(images());stop();start();assert task(child)['task_id']==jid
image_jobs[jid]['status']='complete';finished_frame(restart);assert len(images())==before
# Cancel while the image POST has not returned its ID. Wait for acknowledgement, then cancel exactly that job.
image_submit_release.clear();image_mode='hold';pending=send(7);wait(lambda:len(images())==before+1)
assert api('POST',f'/tasks/{pending}/cancel',{})['outcome']=='pending'
image_submit_release.set();wait(lambda:task(pending)['error_code']=='user_cancelled')
assert image_jobs['img-'+str(len(image_jobs))]['status']=='cancelled'
# Previous-shot tail frame is an EXTRA reference, never a replacement for bound assets.
(data/'static/videos').mkdir(exist_ok=True)
subprocess.run([str(stage/'resources/bin/ffmpeg.exe'),'-hide_banner','-loglevel','error','-f','lavfi','-i','color=c=blue:s=320x192:d=1','-c:v','libx264',str(data/'static/videos/source.mp4')],check=True,creationflags=subprocess.CREATE_NO_WINDOW)
sql("update storyboards set video_url='static/videos/source.mp4',continuity_end='At the east window' where id=20")
api('PUT','/storyboards/21/continuity',{'continuity_mode':'first_frame','continuity_source_id':20,'continuity_start':'At the east window'})
image_mode='complete';before=len(images());continued=send(21,model='Wan2.2-I2V');continued_plan=finished_frame(continued)
assert continued_plan['continuityFrameUrl'] and len(images()[before]['images'])==5
assert len(continued_plan['assets'])==4 and continued_plan['frameUrl']!=continued_plan['continuityFrameUrl']
assert len([x for x in videos()[-1]['content'] if x['type']=='image_url'])==1
# A crash after a video POST may have reached the provider: never submit a duplicate.
before=len(videos());video_submit_release.clear();unknown=send(22);wait(lambda:len(videos())==before+1)
assert task(unknown)['stage']=='submitting' and not task(unknown)['task_id'];stop();start()
wait(lambda:task(unknown)['error_code']=='submission_unknown');video_submit_release.set();assert len(videos())==before+1
# Explicit regeneration bypasses the valid frame cache, then supports normal cancellation.
image_mode='hold';before=len(images());fresh=send(1,replace_task_id=two,regenerate_first_frame=True)
wait(lambda:len(images())==before+1);assert task(two)['error_code']=='user_cancelled';wait(lambda:task(plan(fresh)['childTaskIds'][0])['task_id'])
api('POST',f'/tasks/{fresh}/cancel',{})
# Existing direct multi-reference H3 flow must not create an image task.
image_mode='complete';before=len(images());h3=send(8,model='MiniMax-H3',reference_image_urls=['static/images/input0.png','static/images/input1.png'])
wait(lambda:task(h3)['task_id']);assert len(images())==before and len([x for x in videos()[-1]['content'] if x['type']=='image_url'])==2
# The default selected video config also accepts every binding when model is omitted.
for c in range(3,14):sql('insert into storyboard_characters(storyboard_id,character_id) values(23,?)',(c,))
default_task=send(23,model=None,reference_image_urls=[f'static/images/input{i}.png' for i in range(15)])
assert len(finished_frame(default_task)['assets'])==15
# No source artwork or their metadata was changed by composition.
assert all(sql('select * from '+name+' order by id')==snapshot for name,snapshot in sources.items())
assert sql('pragma integrity_check')[0]['integrity_check']=='ok'
report={'passed':True,'fixture':str(data),'checks':['all bound references','image before video','one final frame sent','correct image and video GPUs','15 refs over 3 rounds','cache reuse','forced remake','missing-reference before cancellation','image failure isolation','stale-edit protection','restart no duplicate','pending submission cancellation','tail-frame plus all bound images','unknown video acknowledgement no resubmission','default model all refs','H3 unchanged','source artwork unchanged']}
(data/'result.json').write_text(json.dumps(report,indent=2));print(json.dumps(report))
if os.environ.get('COMPOSE_UI_TEST'):
sql("update storyboards set title='Composition fixture shot' where id=23")
sql("update sys_task set status='failed',error_code='generation_failed' where id=?",(default_task,))
subprocess.run(['node',str(root/'integration-work/browser-test/verify_composed_frame_ui.cjs')],env={**os.environ,'COMPOSE_FIXTURE':str(data)},check=True)
finally:
image_submit_release.set();video_submit_release.set()
if process:stop()
server.shutdown();log.close()
+1
View File
@@ -81,6 +81,7 @@ try:
for id,ep,num,scene in [(1,1,1,1),(2,1,2,1),(3,2,1,1),(4,2,2,1),(5,2,3,2),(6,3,1,1),(7,4,1,1),(8,2,4,1),(9,2,5,1)]: for id,ep,num,scene in [(1,1,1,1),(2,1,2,1),(3,2,1,1),(4,2,2,1),(5,2,3,2),(6,3,1,1),(7,4,1,1),(8,2,4,1),(9,2,5,1)]:
db.execute('INSERT INTO storyboards(id,episode_id,storyboard_number,scene_id,description,duration,created_at,updated_at) VALUES(?,?,?,?,?,?,?,?)',(id,ep,num,scene,'动作描述',4,ts,ts)) db.execute('INSERT INTO storyboards(id,episode_id,storyboard_number,scene_id,description,duration,created_at,updated_at) VALUES(?,?,?,?,?,?,?,?)',(id,ep,num,scene,'动作描述',4,ts,ts))
# Dependency validation across episodes, scenes, and projects. # Dependency validation across episodes, scenes, and projects.
for scene_id in [1,2]:sql("INSERT INTO scenes(id,drama_id,location,time,prompt,lighting,created_at,updated_at) VALUES(?,1,'Room','night','Fixed stone walls and north window','Moonlight',?,?)",(scene_id,ts,ts))
assert settings(3,'context')['source']['id']==2 assert settings(3,'context')['source']['id']==2
api('POST','/agent/storyboard_breaker/chat',{'drama_id':1,'episode_id':2,'config_id':2,'message':'先读取分镜上下文,核对上一集结尾,不修改任何内容。'}) api('POST','/agent/storyboard_breaker/chat',{'drama_id':1,'episode_id':2,'config_id':2,'message':'先读取分镜上下文,核对上一集结尾,不修改任何内容。'})
assert len(chat_posts)==2 assert len(chat_posts)==2
@@ -12,11 +12,13 @@ source=re.sub(r'(url\d*)\.hostname === "192\.168\.110\.111" && \["18792", "18794
source=re.sub(r'&& !!taskRouting\(\{ provider: config\d*\.provider, taskId \}\)','&& !!taskId',source) source=re.sub(r'&& !!taskRouting\(\{ provider: config\d*\.provider, taskId \}\)','&& !!taskId',source)
(data/'backend.mjs').write_text(source,'utf-8') (data/'backend.mjs').write_text(source,'utf-8')
events=[];translations=[];mode='ok';entered=threading.Event();release=threading.Event();process=None events=[];translations=[];mode='ok';entered=threading.Event();release=threading.Event();process=None
batch_mode=False;active=0;peak=0;counter_lock=threading.Lock()
class Mock(BaseHTTPRequestHandler): class Mock(BaseHTTPRequestHandler):
def log_message(self,*args):pass def log_message(self,*args):pass
def reply(self,status,body): def reply(self,status,body):
raw=json.dumps(body).encode();self.send_response(status);self.send_header('Content-Type','application/json');self.end_headers();self.wfile.write(raw) raw=json.dumps(body).encode();self.send_response(status);self.send_header('Content-Type','application/json');self.end_headers();self.wfile.write(raw)
def do_POST(self): def do_POST(self):
global active,peak
body=json.loads(self.rfile.read(int(self.headers['Content-Length']))) body=json.loads(self.rfile.read(int(self.headers['Content-Length'])))
if self.path.endswith('/chat/completions'): if self.path.endswith('/chat/completions'):
if body.get('tools'): if body.get('tools'):
@@ -25,9 +27,16 @@ class Mock(BaseHTTPRequestHandler):
name=next(t['function']['name'] for t in body['tools'] if 'save' in t['function']['name'].lower() and 'storyboard' in t['function']['name'].lower()) name=next(t['function']['name'] for t in body['tools'] if 'save' in t['function']['name'].lower() and 'storyboard' in t['function']['name'].lower())
en=shot(1)['video_prompt_en'] en=shot(1)['video_prompt_en']
args={'storyboards':[{'shot_number':4,'description':'【镜头1】暴雨中,乔雨灵说:「别开门!」【镜头2】镜头靠近四楼宿舍。','video_prompt':shot(1)['video_prompt'],'video_prompt_en':en,'duration':9,'character_ids':[1],'scene_id':1,'continuity_mode':'none'}]} args={'storyboards':[{'shot_number':4,'description':'【镜头1】暴雨中,乔雨灵说:「别开门!」【镜头2】镜头靠近四楼宿舍。','video_prompt':shot(1)['video_prompt'],'video_prompt_en':en,'duration':9,'character_ids':[1],'scene_id':1,'continuity_mode':'none'}]}
if batch_mode:
args['storyboards']=[{**args['storyboards'][0],'shot_number':40+i,'video_prompt_en':'invalid English without the literal dialogue'} for i in range(6)]
args['replace_existing']=mode=='invalid'
message={'role':'assistant','content':None,'tool_calls':[{'id':'save_bilingual_fixture','type':'function','function':{'name':name,'arguments':json.dumps(args)}}]};finish='tool_calls' message={'role':'assistant','content':None,'tool_calls':[{'id':'save_bilingual_fixture','type':'function','function':{'name':name,'arguments':json.dumps(args)}}]};finish='tool_calls'
self.reply(200,{'id':'fixture','object':'chat.completion','created':1,'model':'fixture-translator','choices':[{'index':0,'message':message,'finish_reason':finish}],'usage':{'prompt_tokens':1,'completion_tokens':1,'total_tokens':2}});return self.reply(200,{'id':'fixture','object':'chat.completion','created':1,'model':'fixture-translator','choices':[{'index':0,'message':message,'finish_reason':finish}],'usage':{'prompt_tokens':1,'completion_tokens':1,'total_tokens':2}});return
translations.append(body) translations.append(body)
if batch_mode:
with counter_lock:active+=1;peak=max(peak,active)
time.sleep(.2)
with counter_lock:active-=1
if mode=='unavailable':self.reply(503,{'error':'fixture'});return if mode=='unavailable':self.reply(503,{'error':'fixture'});return
if mode=='delay':entered.set();release.wait(6) if mode=='delay':entered.set();release.wait(6)
text='Cinematic rain and camera movement. '+ ' '.join(('Dialogue: '+x if x=='__HB_KEEP_0__' else x) for x in re.findall(r'__HB_KEEP_\d+__',body['messages'][-1]['content'])) text='Cinematic rain and camera movement. '+ ' '.join(('Dialogue: '+x if x=='__HB_KEEP_0__' else x) for x in re.findall(r'__HB_KEEP_\d+__',body['messages'][-1]['content']))
@@ -70,6 +79,8 @@ try:
sql('insert into characters(id,drama_id,name,created_at,updated_at) values(1,1,?,?,?)',('乔雨灵',ts,ts)) sql('insert into characters(id,drama_id,name,created_at,updated_at) values(1,1,?,?,?)',('乔雨灵',ts,ts))
sql("insert into scenes(id,drama_id,location,time,prompt,created_at,updated_at) values(1,1,?,'night','rain',?,?)",('四楼宿舍',ts,ts)) sql("insert into scenes(id,drama_id,location,time,prompt,created_at,updated_at) values(1,1,?,'night','rain',?,?)",('四楼宿舍',ts,ts))
for i in [1,2,3]:sql('insert into storyboards(id,episode_id,storyboard_number,video_prompt,created_at,updated_at) values(?,1,?,?,?,?)',(i,i,cn,ts,ts)) for i in [1,2,3]:sql('insert into storyboards(id,episode_id,storyboard_number,video_prompt,created_at,updated_at) values(?,1,?,?,?,?)',(i,i,cn,ts,ts))
sql('update storyboards set scene_id=1')
sql('insert into episode_scenes(episode_id,scene_id,created_at) values(1,1,?)',(ts,))
row=api('POST','/storyboards/1/english-prompt',{}) row=api('POST','/storyboards/1/english-prompt',{})
assert row['video_prompt_en_status']=='ready' and '「别开门!」' in row['video_prompt_en'] and '@乔雨灵' in row['video_prompt_en'] and '0-3s:' in row['video_prompt_en'] assert row['video_prompt_en_status']=='ready' and '「别开门!」' in row['video_prompt_en'] and '@乔雨灵' in row['video_prompt_en'] and '0-3s:' in row['video_prompt_en']
n=len(translations);api('POST','/storyboards/1/english-prompt',{});assert len(translations)==n n=len(translations);api('POST','/storyboards/1/english-prompt',{});assert len(translations)==n
@@ -130,6 +141,20 @@ try:
candidate="0-3s: @乔雨灵's hand rests on @乔雨灵's shoulder; a system voice announces: 「停了。NG,两次。」" candidate="0-3s: @乔雨灵's hand rests on @乔雨灵's shoulder; a system voice announces: 「停了。NG,两次。」"
api('PUT','/storyboards/31',{'video_prompt_en':candidate,'source_prompt':original}) api('PUT','/storyboards/31',{'video_prompt_en':candidate,'source_prompt':original})
assert "@乔雨灵's" in shot(31)['video_prompt_en'] and 'Dialogue: 「停了。NG,两次。」' in shot(31)['video_prompt_en'] assert "@乔雨灵's" in shot(31)['video_prompt_en'] and 'Dialogue: 「停了。NG,两次。」' in shot(31)['video_prompt_en']
# Direct Doubao translations disable thinking and respect the selected model. Batch fallback is bounded at 3.
sql('insert into ai_service_configs(id,service_type,provider,name,base_url,api_key,model,is_active,created_at,updated_at) values(3,?,?,?,?,?,?,1,?,?)',('text','volcengine','fixture','http://127.0.0.1:18941','test-key',json.dumps(['doubao-seed-fixture-default']),ts,ts))
batch_mode=True;before=len(translations)
api('POST','/agent/storyboard_breaker/chat',{'drama_id':1,'episode_id':1,'message':'Save 6 bilingual shots.','config_id':3,'model':'doubao-seed-fixture-selected'})
saved=sql('select * from storyboards where storyboard_number between 40 and 45 order by storyboard_number')
assert len(saved)==6 and peak==3,(len(saved),peak)
assert len(translations[before:])==6
assert all(b['thinking']=={'type':'disabled'} and b['model']=='doubao-seed-fixture-selected' for b in translations[before:])
assert all('「别开门!」' in x['video_prompt_en'] and x['video_prompt_en_source_hash'] for x in saved)
# A failed concurrent batch must not clear existing shots, including when replace_existing=true.
snapshot=sql('select * from storyboards order by id');mode='invalid'
api('POST','/agent/storyboard_breaker/chat',{'drama_id':1,'episode_id':1,'message':'Attempt replacement.','config_id':3,'model':'doubao-seed-fixture-selected'})
assert sql('select * from storyboards order by id')==snapshot
mode='ok';batch_mode=False
process.terminate();process.wait(10);process=None;start() process.terminate();process.wait(10);process=None;start()
assert all(r['video_prompt_en_status']=='ready' for r in api('GET','/episodes/1/storyboards')) assert all(r['video_prompt_en_status']=='ready' for r in api('GET','/episodes/1/storyboards'))
print(json.dumps({'passed':True,'literal_dialogue_preserved':True,'reference_mapping_preserved':True,'actual_submission_english':True,'continuity_english':True,'failure_before_cancel':True,'stale_write_protection':True,'batch_preserves_originals':True,'restart_persistence':True,'fixture':str(data)})) print(json.dumps({'passed':True,'literal_dialogue_preserved':True,'reference_mapping_preserved':True,'actual_submission_english':True,'continuity_english':True,'failure_before_cancel':True,'stale_write_protection':True,'batch_preserves_originals':True,'restart_persistence':True,'fixture':str(data)}))
+195
View File
@@ -0,0 +1,195 @@
"""Isolated bilingual prompt, stale-write, and actual-provider-body checks. No real models/GPU jobs."""
import base64,concurrent.futures,json,os,pathlib,re,sqlite3,subprocess,threading,time,urllib.request,urllib.error
from http.server import BaseHTTPRequestHandler,ThreadingHTTPServer
root=pathlib.Path(__file__).resolve().parent.parent
data=root/'integration-work'/('scene-lock-test-'+time.strftime('%Y%m%d-%H%M%S'));data.mkdir()
stage=root/'integration-work/stage-4.0.11/app';dbfile=data/'huobao.sqlite3'
source=(root/'huobao-source/desktop/build/backend.mjs').read_text('utf-8')
source=source.replace('const require = __hbCreateRequire(import.meta.url)','const require = __hbCreateRequire('+json.dumps(str(stage/'resources/app.asar/build/backend.mjs'))+')',1)
source=re.sub(r'^import (\w+) from "(better-sqlite3|sharp)";',r'const \1 = require("\2");',source,flags=re.M)
source=source.replace('target.hostname === "192.168.110.111" && ["18792", "18794"].includes(target.port)','target.hostname === "127.0.0.1" && target.port === "18951"')
source=re.sub(r'(url\d*)\.hostname === "192\.168\.110\.111" && \["18792", "18794"\]\.includes\(\1.port\)',r'\1.hostname === "127.0.0.1" && \1.port === "18951"',source)
source=re.sub(r'&& !!taskRouting\(\{ provider: config\d*\.provider, taskId \}\)','&& !!taskId',source)
source=source.replace('u.hostname === \"192.168.110.111\" && [\"18792\", \"18794\"].includes(u.port)', 'u.hostname === \"127.0.0.1\" && u.port === \"18951\"')
(data/'backend.mjs').write_text(source,'utf-8')
events=[];translations=[];mode='ok';entered=threading.Event();release=threading.Event();process=None
class Mock(BaseHTTPRequestHandler):
def log_message(self,*args):pass
def reply(self,status,body):
raw=json.dumps(body).encode();self.send_response(status);self.send_header('Content-Type','application/json');self.end_headers();self.wfile.write(raw)
def do_POST(self):
raw=self.rfile.read(int(self.headers['Content-Length']))
if 'multipart/form-data' in self.headers.get('Content-Type',''):
from email.parser import BytesParser
from email.policy import default
message=BytesParser(policy=default).parsebytes(('Content-Type: '+self.headers['Content-Type']+'\r\nMIME-Version: 1.0\r\n\r\n').encode()+raw)
body={'images':[]}
for part in message.iter_parts():
name=part.get_param('name',header='content-disposition')
if name=='image[]':body['images'].append(len(part.get_payload(decode=True)))
else:body[name]=part.get_payload(decode=True).decode()
else:body=json.loads(raw)
if self.path.endswith('/chat/completions'):
if body.get('tools'):
if any(m.get('role')=='tool' for m in body['messages']):message={'role':'assistant','content':'Bilingual storyboard saved.'};finish='stop'
else:
name=next(t['function']['name'] for t in body['tools'] if 'save' in t['function']['name'].lower() and 'storyboard' in t['function']['name'].lower())
en=shot(1)['video_prompt_en']
args={'storyboards':[{'shot_number':4,'description':'【镜头1】暴雨中,乔雨灵说:「别开门!」【镜头2】镜头靠近四楼宿舍。','video_prompt':shot(1)['video_prompt'],'video_prompt_en':en,'duration':9,'character_ids':[1],'scene_id':1,'continuity_mode':'none'}]}
message={'role':'assistant','content':None,'tool_calls':[{'id':'save_bilingual_fixture','type':'function','function':{'name':name,'arguments':json.dumps(args)}}]};finish='tool_calls'
self.reply(200,{'id':'fixture','object':'chat.completion','created':1,'model':'fixture-translator','choices':[{'index':0,'message':message,'finish_reason':finish}],'usage':{'prompt_tokens':1,'completion_tokens':1,'total_tokens':2}});return
translations.append(body)
if mode=='unavailable':self.reply(503,{'error':'fixture'});return
if mode=='delay':entered.set();release.wait(6)
text='Cinematic rain and camera movement. '+ ' '.join(('Dialogue: '+x if x=='__HB_KEEP_0__' else x) for x in re.findall(r'__HB_KEEP_\d+__',body['messages'][-1]['content']))
if mode=='invalid':text='Wrong translation without protected tokens'
self.reply(200,{'choices':[{'message':{'content':json.dumps({'translation':text})}}]});return
events.append((self.path,body))
if self.path.endswith('/validate'):self.reply(200,{'valid':True})
elif self.path.endswith('/cancel'):self.reply(200,{'task':{'id':self.path.split('/')[-2],'status':'cancelled'}})
else:self.reply(200,{'task_id':'new-'+str(len(events))})
def do_GET(self):self.reply(200,{'task':{'id':self.path.split('/')[-1],'status':'running','worker_state':'running'}})
server=ThreadingHTTPServer(('127.0.0.1',18951),Mock);threading.Thread(target=server.serve_forever,daemon=True).start()
env={**os.environ,'FFMPEG_BIN':str(stage/'resources/bin/ffmpeg.exe'),'FFPROBE_BIN':str(stage/'resources/bin/ffprobe.exe'),'ELECTRON_RUN_AS_NODE':'1','PORT':'18950','HUOBAO_DATA_DIR':str(data),'SQLITE_PATH':str(dbfile),'FRONTEND_DIST':str(root/'huobao-source/frontend/.output/public'),'WORKSPACE_PATH':str(stage/'resources/workspace-template')}
log=(data/'backend.log').open('w',encoding='utf-8')
def api(method,path,body=None,expected=200):
req=urllib.request.Request('http://127.0.0.1:18950/api/v1'+path,method=method,data=None if body is None else json.dumps(body).encode(),headers={'Content-Type':'application/json'})
try:
with urllib.request.urlopen(req,timeout=30) as r:status=r.status;result=json.load(r)
except urllib.error.HTTPError as e:status=e.code;result=json.load(e)
assert status==expected,(path,status,result)
return result.get('data',result)
def sql(query,args=()):
with sqlite3.connect(dbfile) as db:db.row_factory=sqlite3.Row;return [dict(r) for r in db.execute(query,args)]
def shot(i):return sql('select * from storyboards where id=?',(i,))[0]
def start():
global process
process=subprocess.Popen([str(stage/'HuobaoDrama.exe'),str(data/'backend.mjs')],cwd=data,env=env,stdout=log,stderr=log,creationflags=subprocess.CREATE_NO_WINDOW)
for _ in range(150):
assert process.poll() is None
try:api('GET','/health');return
except OSError:time.sleep(.1)
raise AssertionError('Backend failed to start')
cn='出场人物:@乔雨灵;场景:@四楼宿舍。\n0-3秒:暴雨中,@乔雨灵 抬头,说:「别开门!」\n3-9秒:镜头缓慢靠近 @四楼宿舍,无字幕。'
png='data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAwMCAO+aN1kAAAAASUVORK5CYII='
def send(i,**extra):
return api('POST','/tasks',{'type':'video','storyboard_id':i,'config_id':1,'model':'MiniMax-H3','prompt':'0-4s: The girl looks up, frightened, medium close-up.','duration':4,**extra},201)
def sent_for(task):
for _ in range(150):
record=sql('select * from sys_task where id=?',(task['id'],))[0]
if record['task_id']:return record,events[-1][1]
assert record['status']!='failed',record['error_msg']
time.sleep(.1)
raise AssertionError('Provider never received task')
try:
start();ts='2026-11-12T00:00:00Z'
sql('insert into dramas(id,title,created_at,updated_at) values(1,?,?,?)',('Scene continuity fixture',ts,ts))
for i in [1,2]:sql('insert into episodes(id,drama_id,episode_number,title,created_at,updated_at) values(?,1,?,?,?,?)',(i,i,'fixture',ts,ts))
sql("update episodes set content='A girl looks up in the dormitory.', script_content='A girl looks up in the dormitory.'")
for id,service,provider,model in [(1,'video','minimax','MiniMax-H3'),(2,'text','openai','fixture-translator'),(3,'image','openai','qwen_image_2_1'),(4,'video','localvideo','MiniMax-H3-VDN-H3')]:
sql('insert into ai_service_configs(id,service_type,provider,name,base_url,api_key,model,is_active,created_at,updated_at) values(?,?,?,?,?,?,?,1,?,?)',(id,service,provider,'fixture','http://127.0.0.1:18951','test-key',json.dumps([model]),ts,ts))
(data/'static/images').mkdir(parents=True)
(data/'static/images/scene.png').write_bytes(base64.b64decode(png.split(',')[1]))
for i,name,layout,lighting in [(1,'Dormitory','Blue plaster walls; one east window; oak desk below the window.','Warm desk lamp from the east.'),(2,'Hallway','Red brick corridor, steel exit door.','Cool overhead strip lights.')]:
sql('insert into scenes(id,drama_id,location,time,prompt,lighting,image_url,final_prompt,created_at,updated_at) values(?,1,?,?,?,?,?,?,?,?)',(i,name,'night',layout,lighting,'static/images/scene.png','EMPTY SET ONLY. NO PEOPLE. FIXED CAMERA.',ts,ts))
sql('insert into episode_scenes(episode_id,scene_id,created_at) values(1,?,?)',(i,ts))
for i in range(1,14):
sql('insert into storyboards(id,episode_id,storyboard_number,scene_id,description,duration,created_at,updated_at) values(?,?,?,?,?,?,?,?)',(i,2 if i==3 else 1,i,2 if i==4 else 1,'Girl looks up',4,ts,ts))
# Every request includes the set, even if the user supplied only a performance.
first,body=sent_for(send(1,reference_image_urls=[png]))
prompt=body['content'][0]['text']
assert 'Blue plaster walls' in prompt and 'Warm desk lamp' in prompt and 'EVERY shot and time segment' in prompt
assert 'Reference image 2' in prompt and 'NO PEOPLE' not in prompt and len(body['content'])==3
snap=json.loads(first['params'])['sceneLock'];assert snap['referenceIndex']==2 and snap['sceneId']==1
# Existing numbered references are not shifted, and the scene image is not duplicated.
second,body=sent_for(send(2,prompt='0-4s: @图片1Girl looks up in @图片2Dormitory.',reference_image_urls=[png,'/static/images/scene.png']))
assert '@图片1Girl' in body['content'][0]['text'] and '@图片2Dormitory' in body['content'][0]['text'] and len(body['content'])==3
third,body=sent_for(send(3));assert 'Blue plaster walls' in body['content'][0]['text']
fourth,body=sent_for(send(4));assert 'Red brick corridor' in body['content'][0]['text'] and 'Blue plaster walls' not in body['content'][0]['text']
# Same-scene settings can change without modifying old prompts, results or task snapshots.
api('PUT','/scenes/1',{'continuity_prompt':'Blue plaster walls; bed against north wall; east window; oak desk below it.'})
assert api('GET','/storyboards/1/environment')['stale'] is True
assert sql('select prompt from sys_task where id=?',(first['id'],))[0]['prompt']==first['prompt']
# Overflow and missing/invalid scenes fail before cancelling an existing running job.
before=len(events)
api('POST','/tasks',{'type':'video','storyboard_id':1,'config_id':1,'prompt':'test','replace_task_id':first['id'],'reference_image_urls':[png]*9},400)
assert len(events)==before and sql('select status from sys_task where id=?',(first['id'],))[0]['status']=='processing'
sql('update storyboards set scene_id=null where id=5')
api('POST','/tasks',{'type':'video','storyboard_id':5,'prompt':'Only a girl'},400)
sql('update storyboards set scene_id=999 where id=5')
api('POST','/tasks',{'type':'video','storyboard_id':5,'prompt':'Only a girl'},400)
assert len(events)==before
# Replacement preflight and actual submission must contain the identical locked prompt.
replacement,body=sent_for(send(1,replace_task_id=first['id'],reference_image_urls=[png],gpu='gpu3'))
assert [p for p,b in events[before:]]==['/v2/video_generation/validate','/v2/video_generation/'+first['task_id']+'/cancel','/v2/video_generation']
assert body['content'][0]['text']==events[before][1]['content'][0]['text'] and 'bed against north wall' in body['content'][0]['text']
# A scene edit during translation cannot cancel the existing job or send stale constraints.
mode='delay';entered.clear();release.clear();before=len(events)
with concurrent.futures.ThreadPoolExecutor() as pool:
f=pool.submit(api,'POST','/tasks',{'type':'video','storyboard_id':1,'prompt':'女孩惊恐地抬头。','config_id':1,'model':'MiniMax-H3','replace_task_id':replacement['id']},400)
assert entered.wait(5)
api('PUT','/scenes/1',{'continuity_prompt':'Blue walls; bed north; east window; oak desk east; lamp on desk.'})
release.set();f.result()
assert not any(p.endswith('/cancel') for p,b in events[before:])
assert sql('select status from sys_task where id=?',(replacement['id'],))[0]['status']=='processing'
mode='ok'
# Real first-frame extraction still submits exactly one frame and retains written environment.
(data/'static/videos').mkdir(parents=True)
subprocess.run([str(stage/'resources/bin/ffmpeg.exe'),'-hide_banner','-loglevel','error','-f','lavfi','-i','color=c=blue:s=320x180:d=2','-c:v','libx264',str(data/'static/videos/source.mp4')],check=True)
sql("update storyboards set video_url='static/videos/source.mp4' where id=5")
sql("update storyboards set scene_id=1 where id=5")
api('PUT','/storyboards/6/continuity',{'continuity_mode':'first_frame','continuity_source_id':5})
frame,body=sent_for(send(6,reference_image_urls=[png]))
images=[x for x in body['content'] if x['type']=='image_url']
assert len(images)==1 and images[0]['role']=='first_frame' and 'Blue walls' in body['content'][0]['text']
# VDN-HD preserves the original frame and receives exactly one fresh environment block.
sql("update storyboards set video_url='static/videos/source.mp4' where id=7")
hd,body=sent_for(api('POST','/hd/storyboards/7',{'config_id':4,'resolution':'2K'},201))
assert body['model']=='MiniMax-H3-VDN-H3' and len([x for x in body['content'] if x['type']=='image_url'])==1
assert body['content'][0]['text'].count('[HB_SCENE_LOCK_V1]')==1 and 'Blue walls' in body['content'][0]['text']
# Storyboard images carry the environment; character portrait generations stay independent.
picture,body=sent_for(api('POST','/tasks',{'type':'image','storyboard_id':8,'config_id':3,'prompt':'A frightened girl, medium shot','reference_images':[png]},201))
assert 'Blue walls' in body['prompt'] and len(body['images'])==2 and 'Reference image 2' in body['prompt']
portrait,body=sent_for(api('POST','/tasks',{'type':'image','character_id':99,'config_id':3,'prompt':'Character sheet on white background'},201))
assert body['prompt']=='Character sheet on white background'
# Explicit first-frame requests still reject mixed references. Auto composition is tested separately.
before=len(events)
api('POST','/tasks',{'type':'video','storyboard_id':9,'config_id':4,'model':'Wan2.2-I2V','prompt':'test','first_frame_url':'static/images/scene.png','reference_image_urls':[png]},400)
assert len(events)==before
single,body=sent_for(send(9,config_id=4,model='Wan2.2-I2V',first_frame_url='static/images/scene.png'))
assert len([x for x in body['content'] if x['type']=='image_url'])==1 and 'Blue walls' in body['content'][0]['text']
# LTX screenshot regression: even when the scene is already present, reject 3 refs before admission.
before=len(events);count=sql('select count(*) n from sys_task')[0]['n']
failure=api('POST','/tasks',{'type':'video','storyboard_id':12,'config_id':4,'model':'LTX-2.3','prompt':'test','first_frame_url':'static/images/scene.png','reference_image_urls':['static/images/scene.png',png,png]},400)
assert len(events)==before and sql('select count(*) n from sys_task')[0]['n']==count
ltx,body=sent_for(send(12,config_id=4,model='LTX-2.3',gpu='gpu3',first_frame_url='static/images/scene.png'))
assert body['model']=='LTX-2.3' and body['gpu']=='gpu3' and len([x for x in body['content'] if x['type']=='image_url'])==1
assert 'Blue walls' in body['content'][0]['text']
# Translation remains English and the immutable layout guard is not lost in the translator output.
api('PUT','/scenes/1',{'continuity_prompt':'蓝灰色墙面;北侧床铺,东侧窗下书桌。'})
translated,body=sent_for(send(10))
assert '蓝灰色墙面' not in body['content'][0]['text'] and 'EVERY shot and time segment' in body['content'][0]['text']
# A scene without an image still has explicit text constraints and truthful metadata.
api('PUT','/scenes/2',{'image_url':None})
sql('update scenes set image_url=null where id=2');sql('update storyboards set scene_id=2 where id=11')
text_only,body=sent_for(send(11));assert 'No scene image is supplied' in body['content'][0]['text']
assert json.loads(text_only['params'])['sceneLock']['referenceMode']=='text_only'
process.terminate();process.wait(10);process=None;start()
assert sql('select continuity_prompt from scenes where id=1')[0]['continuity_prompt']=='蓝灰色墙面;北侧床铺,东侧窗下书桌。'
assert sql('select prompt from sys_task where id=?',(first['id'],))[0]['prompt']==first['prompt']
assert sql('pragma integrity_check')[0]['integrity_check']=='ok'
print(json.dumps({'passed':True,'fixture':str(data),'checks':['per-request scene injection','scene reference numbering/deduplication','cross-episode shared scene','different scene isolation','before-cancel validation','scene edit race','single-frame continuation','VDN HD','storyboard image reference','portrait isolation','limited-reference model','English visual submission','restart/data preservation']}))
if os.environ.get('SCENE_UI_TEST'):
subprocess.run(['node',str(root/'integration-work/browser-test/verify_scene_lock_ui.cjs')],env={**os.environ,'SCENE_FIXTURE':str(data)},check=True)
if os.environ.get('LTX_UI_TEST'):
sql('update ai_service_configs set model=? where id=4',(json.dumps(['MiniMax-H3-VDN-H3','LTX-2.3']),))
for i,name,file in [(1,'Girl','girl.png'),(2,'Other','other.png')]:
(data/'static/images'/file).write_bytes(base64.b64decode(png.split(',')[1]))
sql('insert into characters(id,drama_id,name,image_url,created_at,updated_at) values(?,1,?,?,?,?)',(i,name,'static/images/'+file,ts,ts))
sql('insert into episode_characters(episode_id,character_id,created_at) values(1,?,?)',(i,ts))
sql('insert into storyboard_characters(storyboard_id,character_id) values(13,?)',(i,))
sql("update storyboards set title='Reference fixture',description='Reference fixture',video_prompt='0-4s: @Dormitory, @Girl looks at @Other.' where id=13")
subprocess.run(['node',str(root/'integration-work/browser-test/verify_video_reference_ui.cjs')],env={**os.environ,'SCENE_FIXTURE':str(data)},check=True)
finally:
if process:process.terminate();process.wait(10)
server.shutdown();log.close()
@@ -0,0 +1,82 @@
"""Exercise deletion against a disposable desktop backend; never touch live data."""
import contextlib, hashlib, json, os, pathlib, re, sqlite3, subprocess, time, urllib.request, urllib.error
ROOT = pathlib.Path(__file__).resolve().parent.parent
OUT = ROOT/'integration-work'/('storyboard-delete-test-'+time.strftime('%Y%m%d-%H%M%S')); OUT.mkdir()
STAGE = ROOT/'integration-work/stage-4.0.11/app'; DB = OUT/'huobao.sqlite3'
source = (ROOT/'huobao-source/desktop/build/backend.mjs').read_text('utf-8')
source = source.replace('const require = __hbCreateRequire(import.meta.url)', 'const require = __hbCreateRequire('+json.dumps(str(STAGE/'resources/app.asar/build/backend.mjs'))+')', 1)
source = re.sub(r'^import (\w+) from "(better-sqlite3|sharp)";', r'const \1 = require("\2");', source, flags=re.M)
(OUT/'backend.mjs').write_text(source, 'utf-8')
env = {**os.environ, 'ELECTRON_RUN_AS_NODE':'1','PORT':'18986','HUOBAO_DATA_DIR':str(OUT),'SQLITE_PATH':str(DB),
'FRONTEND_DIST':str(ROOT/'huobao-source/frontend/.output/public'), 'WORKSPACE_PATH':str(STAGE/'resources/workspace-template')}
def sql(query,args=()):
with contextlib.closing(sqlite3.connect(DB)) as c,c:
c.row_factory=sqlite3.Row
return [dict(r) for r in c.execute(query,args)]
def api(method,path,body=None,code=200):
req=urllib.request.Request('http://127.0.0.1:18986/api/v1'+path,method=method, data=None if body is None else json.dumps(body).encode(),headers={'Content-Type':'application/json'})
try:
with urllib.request.urlopen(req,timeout=10) as response: actual=response.status; value=json.load(response)
except urllib.error.HTTPError as error: actual=error.code; value=json.load(error)
assert actual==code,(method,path,actual,value)
return value.get('data',value)
def remove(ids,code=200): return api('POST','/storyboards/batch-delete',{'episode_id':1,'storyboard_ids':ids},code)
def live():return api('GET','/episodes/1/storyboards')
def row(id):return sql('select * from storyboards where id=?',(id,))[0]
log=(OUT/'backend.log').open('w',encoding='utf-8')
process=subprocess.Popen([str(STAGE/'HuobaoDrama.exe'),str(OUT/'backend.mjs')],cwd=OUT,env=env,stdout=log,stderr=log,creationflags=subprocess.CREATE_NO_WINDOW)
try:
for _ in range(100):
assert process.poll() is None,'Backend exited'
try:api('GET','/health');break
except OSError:time.sleep(.15)
ts='2026-11-13T00:00:00Z'
sql("insert into app_settings(key,value,updated_at) values('tours_seen',?,?) on conflict(key) do update set value=excluded.value",(json.dumps(['episode']),ts))
sql('insert into dramas(id,title,created_at,updated_at) values(1,?,?,?)',('Deletion fixture',ts,ts))
for id in (1,2):sql('insert into episodes(id,drama_id,episode_number,title,content,script_content,created_at,updated_at) values(?,1,?,?,?,?,?,?)',(id,id,'fixture','剧本','剧本',ts,ts))
sql('insert into characters(id,drama_id,name,created_at,updated_at) values(1,1,?,?,?)',('角色',ts,ts))
sql("insert into scenes(id,drama_id,location,time,prompt,created_at,updated_at) values(1,1,'房间','night','room',?,?)",(ts,ts))
sql("insert into props(id,drama_id,name,created_at,updated_at) values(1,1,'道具',?,?)",(ts,ts))
for table,key in [('episode_characters','character_id'),('episode_scenes','scene_id'),('episode_props','prop_id')]:
sql(f'insert into {table}(episode_id,{key},created_at) values(1,1,?)',(ts,))
for id in list(range(1,11))+[20]:
sql('insert into storyboards(id,episode_id,storyboard_number,title,description,duration,scene_id,created_at,updated_at) values(?,?,?,?,?,9,1,?,?)',(id,2 if id==20 else 1,id,'镜头 '+str(id),'分镜画面 '+str(id),ts,ts))
sql('insert into storyboard_characters values(1,1)'); sql('insert into storyboard_props values(1,1)')
media=OUT/'static/videos/preserved.mp4';media.parent.mkdir(parents=True);media.write_bytes(b'fixture media bytes kept intact')
original_hash=hashlib.sha256(media.read_bytes()).hexdigest()
sql("update storyboards set video_url='static/videos/preserved.mp4',status='completed' where id=1")
def task(id,sb,status,params=None):
sql("insert into sys_task(id,type,storyboard_id,drama_id,provider,model,prompt,status,params,local_path,created_at,updated_at) values(?,'video',?,1,'minimax','MiniMax-H3','fixture',?,?,?, ?,?)",(id,sb,status,json.dumps(params or {}),'static/videos/preserved.mp4' if id==1 else None,ts,ts))
task(1,1,'completed');task(2,2,'processing');task(90,3,'processing',{'continuity':{'sourceId':4}})
baseline={id:row(id) for id in (2,3,4,20)}
remove([2,20],400);remove([2,3],400);remove([4],400);remove([],400);remove(['2'],400)
assert all(row(id)==baseline[id] for id in baseline)
sql("update sys_task set status='completed' where id=90")
assert api('DELETE','/storyboards/1')['count']==1
assert row(1)['deleted_at'] and row(1)['video_url']=='static/videos/preserved.mp4'
assert sql('select * from storyboard_characters where storyboard_id=1') and sql('select * from storyboard_props where storyboard_id=1')
assert all(r['id']!=1 for r in live())
assert all(r['id']!=1 for r in api('GET','/hd/episodes/1')['items'])
assert api('GET','/episodes/1/pipeline-status')['steps']['extract_storyboards']['count']==9
merged=api('POST','/merge/episodes/1/merge',{'storyboard_ids':[1]},400)
assert '没有可拼接的视频' in merged['message']
history=api('GET','/episodes/1/generation-tasks')['tasks'];old=next(t for t in history if t['id']==1)
assert old['storyboard_deleted'] and old['storyboard_number']==1 and old['local_path']=='static/videos/preserved.mp4'
api('PUT','/storyboards/1',{'description':'should not save'},400)
api('POST','/tasks',{'type':'image','storyboard_id':1,'prompt':'no model call'},400)
api('POST','/tasks',{'type':'video','storyboard_id':1,'prompt':'no model call'},400)
assert remove([3,4])['count']==2 and remove([3,4])['count']==0
assert row(2)==baseline[2] and row(20)==baseline[20]
ui=subprocess.run(['node',str(ROOT/'integration-work/browser-test/verify_storyboard_delete_ui.cjs')],cwd=ROOT/'integration-work/browser-test',env={**os.environ,'STORYBOARD_FIXTURE':str(OUT)},capture_output=True,text=True,encoding='utf-8',timeout=75)
print(ui.stdout,flush=True);assert ui.returncode==0,ui.stderr
assert row(5)['deleted_at'] and row(6)['deleted_at'] and row(7)['deleted_at'] and not row(2)['deleted_at']
sql("update sys_task set status='cancelled' where id=2")
remove([r['id'] for r in live()]);assert live()==[]
assert api('GET','/episodes/1/pipeline-status')['steps']['extract_storyboards']['count']==0
assert hashlib.sha256(media.read_bytes()).hexdigest()==original_hash
assert len(sql('select * from characters'))==1 and len(sql('select * from scenes'))==1 and len(sql('select * from props'))==1
assert len(sql('select * from sys_task'))==3
report={'passed':True,'checks':['single/batch soft delete','batch all-or-none','active and dependent task protection','cross-episode validation','deleted shot update and generation rejected','pipeline, HD and merge filtering','task history and references retained','UI confirmation/cancel/batch/reload','delete all','media bytes unchanged']}
(OUT/'result.json').write_text(json.dumps(report,indent=2),'utf-8');print(json.dumps(report),flush=True)
finally:
process.terminate();process.wait(10);log.close()
+152
View File
@@ -0,0 +1,152 @@
"""Isolated async storyboard lifecycle, cancellation and atomic replacement regression tests."""
import contextlib,json,os,pathlib,re,sqlite3,subprocess,threading,time,urllib.request,urllib.error
from http.server import BaseHTTPRequestHandler,ThreadingHTTPServer
ROOT=pathlib.Path(__file__).resolve().parent.parent
OUT=ROOT/'integration-work'/('storyboard-jobs-test-'+time.strftime('%Y%m%d-%H%M%S'));OUT.mkdir()
(ROOT/'integration-work/current-storyboard-jobs-test.txt').write_text(str(OUT),'utf-8')
STAGE=ROOT/'integration-work/stage-4.0.11/app';DB=OUT/'huobao.sqlite3'
source=(ROOT/'huobao-source/desktop/build/backend.mjs').read_text('utf-8')
source=source.replace('const require = __hbCreateRequire(import.meta.url)','const require = __hbCreateRequire('+json.dumps(str(STAGE/'resources/app.asar/build/backend.mjs'))+')',1)
source=re.sub(r'^import (\w+) from "(better-sqlite3|sharp)";',r'const \1 = require("\2");',source,flags=re.M)
(OUT/'backend.mjs').write_text(source,'utf-8')
release=threading.Event();held=threading.Event();mode='hold';calls=[];process=None
def shot(i,bad=False):
return {'shot_number':i,'source_ids':['S01P01'],'title':'New '+str(i),'duration':9,'description':'【镜头1】角色站在房间。','character_ids':[1],'scene_id':999 if bad else 1,'video_prompt':'0-9秒:@测试角色 站在 @房间,保持静止。','video_prompt_en':'0-9s: @测试角色 stands in @房间, motionless.'}
class Mock(BaseHTTPRequestHandler):
def log_message(self,*a):pass
def reply(self,value):
raw=json.dumps(value).encode()
try:self.send_response(200);self.send_header('Content-Type','application/json');self.end_headers();self.wfile.write(raw)
except (BrokenPipeError,ConnectionResetError,ConnectionAbortedError):pass
def do_POST(self):
body=json.loads(self.rfile.read(int(self.headers['Content-Length'])));which=body['model'];calls.append(which)
assert body.get('tools'),'Unexpected translation call'
assert any('每批最多 4 个分镜' in str(m.get('content','')) for m in body['messages'] if m['role']=='system'),'Tracked batch limit must be in system instructions'
tools=[t['function']['name'] for t in body['tools']]
read=next(t for t in tools if 'read' in t.lower());save=next(t for t in tools if 'save' in t.lower())
n=sum(m['role']=='tool' for m in body['messages'])
context={}
for m in body['messages']:
if m['role']=='tool':
try:
value=json.loads(m.get('content','{}'))
if 'current_section' in value:context=value
except (ValueError,TypeError):pass
args={};name=read
if which=='empty':message={'role':'assistant','content':'Nothing saved.'};finish='stop'
else:
if n==1:
name=save;args={'replace_existing':True,'storyboards':[shot(i) for i in range(1,5)]}
if which=='reject':args['storyboards'][0]['video_prompt']='0-6秒:@测试角色 站在 @房间。'
elif n==2 and which in ('hold','cancel','restart','changed','timeout','bad'):
held.set();release.wait(90);name=save;args={'storyboards':[shot(5,which=='bad'),shot(6)]}
elif n>=2 and which=='reject':
name=save;args={'replace_existing':True,'storyboards':[shot(1)]};args['storyboards'][0]['video_prompt']='0-6秒:@测试角色 站在 @房间。'
elif n==2:name=save;args={'storyboards':[shot(5),shot(6)]}
if which=='inplace' and n==1:
assert not any('update' in t.lower() or 'workspace' in t.lower() for t in tools),'Tracked jobs expose only context and staging tools'
name='updateStoryboard';args={'storyboard_id':rows()[0]['id'],'title':'must not overwrite'}
if which=='toolerror' and n>=1:
name=save;args={'replace_existing':True,'storyboards':[shot(1,True)]}
if which=='voice' and n==1:
args['storyboards'][0]['video_prompt']='0-9秒:#测试角色音色说:「安静。」'
args['storyboards'][0]['video_prompt_en']='0-9s: #测试角色音色 says: 「安静。」'
if which=='ending' and n==1:
args['storyboards'][1]['description']='角色说:「我们在这扇门后结束本集故事,不要再继续编写新的剧情了!」'
if which=='multisection' and n in (1,2):
base=context['next_shot_number']
numbers=range(base,base+4) if n==1 else range(base+4,base+6)
args={'storyboards':[{**shot(i),'source_ids':[context['current_section']['id']+'P01']} for i in numbers]}
args['replace_existing']=True # repeated per-scene flags must never erase staged drafts
if which=='gap' and n>=1:
name=save;args={'storyboards':[shot(6)]}
if which in ('skip','rushed') and n>=1:
name=save;args={'storyboards':[shot(1)]}
if which=='skip':args['storyboards'][0]['source_ids']=['S02P01']
else:args['storyboards'][0]['video_prompt']='0-4秒:@测试角色说:「'+'先检查窗户是否关好再整理图纸清点工具确认材料数量然后才可以开始修理屋顶。'*3+'」\n4-9秒:大家点头。'
if (n>=3 and which not in ('reject','toolerror','skip','rushed','gap')) or (which=='inplace' and n>=2):message={'role':'assistant','content':'All saved.'};finish='stop'
else:message={'role':'assistant','content':None,'tool_calls':[{'id':f'call_{n}','type':'function','function':{'name':name,'arguments':json.dumps(args)}}]};finish='tool_calls'
self.reply({'id':'fixture','object':'chat.completion','created':1,'model':which,'choices':[{'index':0,'message':message,'finish_reason':finish}],'usage':{'prompt_tokens':1,'completion_tokens':1,'total_tokens':2}})
server=ThreadingHTTPServer(('127.0.0.1',18985),Mock);threading.Thread(target=server.serve_forever,daemon=True).start()
env={**os.environ,'ELECTRON_RUN_AS_NODE':'1','PORT':'18984','HUOBAO_DATA_DIR':str(OUT),'SQLITE_PATH':str(DB),'FRONTEND_DIST':str(ROOT/'huobao-source/frontend/.output/public'),'WORKSPACE_PATH':str(STAGE/'resources/workspace-template'),'HUOBAO_STORYBOARD_TIMEOUT_MS':'90000' if os.getenv('STORYBOARD_UI_TEST')=='1' else '20000'}
def sql(q,args=()):
with contextlib.closing(sqlite3.connect(DB)) as d,d:d.row_factory=sqlite3.Row;return [dict(r) for r in d.execute(q,args)]
def api(method,path,body=None):
req=urllib.request.Request('http://127.0.0.1:18984/api/v1'+path,method=method,data=None if body is None else json.dumps(body).encode(),headers={'Content-Type':'application/json'})
with urllib.request.urlopen(req,timeout=8) as r:v=json.load(r)
return v.get('data',v)
def wait(fn,seconds=12):
end=time.monotonic()+seconds
while time.monotonic()<end:
value=fn()
if value:return value
time.sleep(.1)
raise AssertionError('Timed out')
def state():return api('GET','/agent/storyboard_breaker/jobs/1')
def send(which):return api('POST','/agent/storyboard_breaker/jobs',{'episode_id':1,'drama_id':1,'config_id':1,'model':which,'message':'Create the entire storyboard.'})
def rows():return sql('select * from storyboards where episode_id=1 order by id')
def start():
global process
log=(OUT/'backend.log').open('a',encoding='utf-8')
process=subprocess.Popen([str(STAGE/'HuobaoDrama.exe'),str(OUT/'backend.mjs')],cwd=OUT,env=env,stdout=log,stderr=log,creationflags=subprocess.CREATE_NO_WINDOW)
for _ in range(100):
assert process.poll() is None
try:api('GET','/health');return
except OSError:time.sleep(.15)
raise AssertionError('Backend failed')
def stop():
global process
process.terminate();process.wait(10);process=None
try:
start();ts='2026-11-12T00:00:00Z'
sql("insert into app_settings(key,value,updated_at) values('tours_seen',?,?) on conflict(key) do update set value=excluded.value",(json.dumps(['episode']),ts))
sql('insert into dramas(id,title,created_at,updated_at) values(1,?,?,?)',('fixture',ts,ts))
sql('insert into episodes(id,drama_id,episode_number,title,content,script_content,created_at,updated_at) values(1,1,1,?,?,?,?,?)',('fixture','戏'*450,'戏'*450,ts,ts))
sql("insert into ai_service_configs(id,service_type,provider,name,base_url,api_key,model,is_active,created_at,updated_at) values(1,'text','openai','fixture','http://127.0.0.1:18985','fixture','[\"hold\"]',1,?,?)",(ts,ts))
sql('insert into characters(id,drama_id,name,created_at,updated_at) values(1,1,?,?,?)',('测试角色',ts,ts))
sql("insert into scenes(id,drama_id,location,time,prompt,created_at,updated_at) values(1,1,?,'night','room',?,?)",('房间',ts,ts))
sql('insert into storyboards(id,episode_id,storyboard_number,title,video_url,duration,created_at,updated_at) values(1,1,1,?,?,12,?,?)',('old preserved','static/videos/old.mp4',ts,ts))
original=rows();started=time.monotonic();job=send('hold');assert time.monotonic()-started<2
wait(lambda:state()['completed']==4);assert rows()==original and 'drafts' not in state()
try:
api('DELETE','/storyboards/1');raise AssertionError('Deletion during split must be rejected')
except urllib.error.HTTPError as error:assert error.code==400
assert rows()==original
assert send('hold')['id']==job['id'],'Duplicate start must reuse current job'
if os.getenv('STORYBOARD_UI_TEST')=='1':
ui=subprocess.run(['node',str(ROOT/'integration-work/browser-test/verify_storyboard_jobs_ui.cjs')],cwd=ROOT/'integration-work/browser-test',env={**os.environ,'STORYBOARD_FIXTURE':str(OUT)},capture_output=True,text=True,encoding='utf-8',timeout=40)
print(ui.stdout);assert ui.returncode==0,ui.stderr
release.set();wait(lambda:state()['status']=='done');assert len(rows())==6 and all(r['video_prompt_en'] for r in rows())
baseline=rows()
if os.getenv('STORYBOARD_UI_TEST')=='1':
stop();env['HUOBAO_STORYBOARD_TIMEOUT_MS']='20000';start()
for which in ('cancel','changed','bad','timeout','restart'):
release.clear();held.clear();job=send(which);wait(lambda:state()['completed']==4);assert rows()==baseline
if which=='cancel':
api('POST','/agent/storyboard_breaker/jobs/1/cancel',{'id':job['id']});release.set();time.sleep(.4);assert state()['status']=='cancelled' and rows()==baseline
elif which=='changed':
sql("update storyboards set title='user edit' where id=?",(baseline[0]['id'],));release.set();wait(lambda:state()['status']=='error');assert rows()[0]['title']=='user edit';baseline=rows()
elif which=='bad':release.set();wait(lambda:state()['status']=='error');assert rows()==baseline
elif which=='timeout':wait(lambda:state()['status']=='error',25);release.set();time.sleep(.4);assert rows()==baseline
else:
stop();release.set();start();assert state()['status']=='error' and rows()==baseline
print('PASS',which,flush=True)
for which in ('empty','reject','inplace','toolerror'):
send(which);wait(lambda:state()['status']=='error');assert rows()==baseline;print('PASS',which,flush=True)
if which=='toolerror':assert state()['steps']<=4 and state().get('lastValidation'),state()
send('voice');wait(lambda:state()['status']=='done');assert '#测试角色' not in rows()[0]['video_prompt'];print('PASS invented voice normalization',flush=True)
baseline=rows()
terminal='角色:(低声)我们在这扇门后结束本集故事,不要再继续编写新的剧情了!'
sql('update episodes set content=?,script_content=? where id=1',('戏'*450+'\n'+terminal,'戏'*450+'\n'+terminal))
send('ending');wait(lambda:state()['status']=='error');assert rows()==baseline;print('PASS ending cannot bypass missing source coverage or timing',flush=True)
script='## S01 | Room | Day\n\n'+'甲'*450+'\n\n## S02 | Garden | Day\n\n'+'乙'*450
sql('update episodes set content=?,script_content=? where id=1',(script,script))
for which in ('skip','rushed','gap'):
send(which);wait(lambda:state()['status']=='error');assert rows()==baseline
assert {'skip':'不能跳场','rushed':'对白至少需要','gap':'分镜编号缺少'}[which] in state()['error'];print('PASS',which,flush=True)
send('multisection');wait(lambda:state()['status']=='done');assert len(rows())==12 and state()['sectionIndex']==state()['sectionCount']==2;print('PASS full multi-section completion',flush=True)
print(json.dumps({'passed':True,'checks':['immediate response','draft progress','duplicate admission','atomic bilingual replacement','cancel late-write guard','concurrent edit preservation','invalid binding rollback','deadline','restart recovery','empty result failure','validation retry cap']}),flush=True)
finally:
release.set()
if process:stop()
server.shutdown()
@@ -0,0 +1,34 @@
"""Receipt API tests against temporary SQLite through ASGI, without a network listener."""
import asyncio,importlib.util,pathlib,sqlite3,tempfile
from contextlib import closing
import httpx
from fastapi import FastAPI,Header,HTTPException
source=pathlib.Path(__file__).resolve().parent.parent/'model-bridge/video_cleanup_api.py'
spec=importlib.util.spec_from_file_location('video_cleanup_api',source);module=importlib.util.module_from_spec(spec);spec.loader.exec_module(module)
async def main():
with tempfile.TemporaryDirectory(prefix='huobao-cleanup-api-') as directory:
database=pathlib.Path(directory)/'jobs.sqlite3'
with closing(sqlite3.connect(database)) as db,db:
db.execute('create table jobs(id text primary key)');db.execute("insert into jobs values('known-video')")
app=FastAPI()
async def auth(authorization:str|None=Header(default=None)):
if authorization!='Bearer fixture-token':raise HTTPException(401,'Unauthorized')
module.install_video_cleanup(app,auth,database)
receipt={'version':1,'index':0,'sha256':'a'*64,'bytes':1234}
async with httpx.AsyncClient(transport=httpx.ASGITransport(app=app),base_url='http://fixture') as client:
url='/v2/video_generation/known-video/cleanup'
assert (await client.post(url,json=receipt)).status_code==401
client.headers['Authorization']='Bearer fixture-token'
assert (await client.post('/v2/video_generation/unknown/cleanup',json=receipt)).status_code==404
for patch in [{'filename':'../../anything.mp4'},{'index':True},{'bytes':0},{'sha256':'invalid'},{'version':2}]:
assert (await client.post(url,json={**receipt,**patch})).status_code==400
assert (await client.post(url,json=receipt)).json()['status']=='pending'
assert (await client.post(url,json=receipt)).json()['status']=='pending'
assert (await client.post(url,json={**receipt,'sha256':'b'*64})).status_code==409
with closing(sqlite3.connect(database)) as db,db:
assert db.execute('select count(*) from video_cleanup_acks').fetchone()[0]==1
db.execute("update video_cleanup_acks set status='cleaned' where id='known-video'")
result=(await client.post(url,json=receipt)).json()
assert result['status']=='cleaned' and result['sha256']==receipt['sha256'] and result['bytes']==1234
print('PASS: authenticated known-video-only receipts; invalid paths/types denied; retries are idempotent; conflicting receipts denied')
asyncio.run(main())
@@ -0,0 +1,103 @@
"""Isolated verified-download acknowledgement test. No live ComfyUI traffic."""
import base64,hashlib,json,os,pathlib,re,sqlite3,subprocess,threading,time,urllib.request
from http.server import BaseHTTPRequestHandler,ThreadingHTTPServer
root=pathlib.Path(__file__).resolve().parent.parent
data=root/'integration-work'/('video-cleanup-test-'+time.strftime('%Y%m%d-%H%M%S'));data.mkdir()
stage=root/'integration-work/stage-4.0.11/app';dbfile=data/'huobao.sqlite3'
source=(root/'huobao-source/desktop/build/backend.mjs').read_text('utf-8')
source=source.replace('const require = __hbCreateRequire(import.meta.url)','const require = __hbCreateRequire('+json.dumps(str(stage/'resources/app.asar/build/backend.mjs'))+')',1)
source=re.sub(r'^import (\w+) from "(better-sqlite3|sharp)";',r'const \1 = require("\2");',source,flags=re.M)
source=re.sub(r'(\w+)\.hostname === "192\.168\.110\.111" && \["18792", "18794"\]\.includes\(\1.port\)',r'\1.hostname === "127.0.0.1" && \1.port === "18981"',source)
source=source.replace('intervalMs: 1e4','intervalMs: 100')
(data/'backend.mjs').write_text(source,'utf-8')
subprocess.run([str(stage/'resources/bin/ffmpeg.exe'),'-hide_banner','-loglevel','error','-f','lavfi','-i','color=c=blue:s=320x192:d=1','-c:v','libx264',str(data/'valid.mp4')],check=True,creationflags=subprocess.CREATE_NO_WINDOW)
valid=(data/'valid.mp4').read_bytes();digest=hashlib.sha256(valid).hexdigest()
events=[];job_number=0;mode='cleaned';jobs={}
def sql(query,args=()):
with sqlite3.connect(dbfile) as db:db.row_factory=sqlite3.Row;return [dict(r) for r in db.execute(query,args)]
class Mock(BaseHTTPRequestHandler):
def log_message(self,*args):pass
def reply(self,value,status=200):
self.send_response(status);self.send_header('Content-Type','application/json');self.end_headers();self.wfile.write(json.dumps(value).encode())
def do_POST(self):
global job_number
body=json.loads(self.rfile.read(int(self.headers.get('Content-Length','0'))) or '{}')
if self.path.endswith('/cleanup'):
jid=self.path.split('/')[-2];events.append(('cleanup',jid,body))
record=sql('select * from sys_task where task_id=?',(jid,))[0]
assert record['status']=='completed' and (data/record['local_path']).exists()
assert body=={'version':1,'index':0,'sha256':digest,'bytes':len(valid)}
self.reply({'status':mode,'sha256':digest,'bytes':len(valid),'index':0},503 if mode=='error' else 200);return
job_number+=1;jid='vid-'+str(job_number);jobs[jid]=mode;events.append(('generate',jid))
self.reply({'task_id':jid})
def do_GET(self):
if self.path.startswith('/v2/query/'):
jid=self.path.split('/')[-1]
self.reply({'task':{'id':jid,'status':'succeeded','duration':1,'content':{'url':f'http://127.0.0.1:18981/media/{jid}/0/video.mp4'}}});return
jid=self.path.split('/')[2];events.append(('download',jid))
raw=valid[:32] if jobs[jid]=='invalid' else valid
self.send_response(200);self.send_header('Content-Type','video/mp4');self.send_header('Content-Length',str(len(raw)));self.end_headers();self.wfile.write(raw)
server=ThreadingHTTPServer(('127.0.0.1',18981),Mock);threading.Thread(target=server.serve_forever,daemon=True).start()
env={**os.environ,'ELECTRON_RUN_AS_NODE':'1','PORT':'18980','HUOBAO_DATA_DIR':str(data),'SQLITE_PATH':str(dbfile),'FRONTEND_DIST':str(root/'huobao-source/frontend/.output/public'),'WORKSPACE_PATH':str(stage/'resources/workspace-template'),'FFMPEG_BIN':str(stage/'resources/bin/ffmpeg.exe'),'FFPROBE_BIN':str(stage/'resources/bin/ffprobe.exe')}
process=None;log=(data/'backend.log').open('w',encoding='utf-8')
def api(method,path,body=None):
request=urllib.request.Request('http://127.0.0.1:18980/api/v1'+path,method=method,data=None if body is None else json.dumps(body).encode(),headers={'Content-Type':'application/json'})
with urllib.request.urlopen(request,timeout=15) as response:return json.load(response).get('data',{})
def wait(fn):
deadline=time.monotonic()+20
while time.monotonic()<deadline:
result=fn()
if result:return result
time.sleep(.1)
raise AssertionError('Timed out')
def start():
global process
process=subprocess.Popen([str(stage/'HuobaoDrama.exe'),str(data/'backend.mjs')],cwd=data,env=env,stdout=log,stderr=log,creationflags=subprocess.CREATE_NO_WINDOW)
for _ in range(120):
assert process.poll() is None
try:api('GET','/health');return
except OSError:time.sleep(.1)
raise AssertionError('Failed to start')
def stop():
global process
process.terminate();process.wait(10);process=None
def record(id):return sql('select * from sys_task where id=?',(id,))[0]
def cleanup(id):return json.loads(record(id)['params']).get('remoteCleanup',{})
def send():return api('POST','/tasks',{'type':'video','config_id':1,'model':'MiniMax-H3','prompt':'A blue square.','duration':4})['id']
def due(id):
params=json.loads(record(id)['params']);params['remoteCleanup']['nextAttemptAt']='2000-01-01T00:00:00Z'
sql('update sys_task set params=? where id=?',(json.dumps(params),id))
try:
start();ts='2026-11-12T00:00:00Z'
sql("insert into ai_service_configs(id,service_type,provider,name,base_url,api_key,model,is_active,created_at,updated_at) values(1,'video','minimax','fixture','http://127.0.0.1:18981','fixture-only','[\"MiniMax-H3\"]',1,?,?)",(ts,ts))
first=send();wait(lambda:cleanup(first).get('status')=='cleaned')
assert record(first)['status']=='completed' and cleanup(first)['localReceipt']['sha256']==digest
assert [e[0] for e in events]==['generate','download','cleanup']
# A failed acknowledgement cannot turn a successful download into a failed video.
mode='error';retry=send();wait(lambda:cleanup(retry).get('lastError'))
assert record(retry)['status']=='completed';before=sum(e[0]=='generate' for e in events)
mode='cleaned';due(retry);stop();start();wait(lambda:cleanup(retry).get('status')=='cleaned')
assert sum(e[0]=='generate' for e in events)==before and sum(e[0]=='download' for e in events)==2
# Truncated MP4 headers must never authorize deletion.
mode='invalid';bad=send();wait(lambda:record(bad)['status']=='failed')
assert not cleanup(bad) and not any(e[0]=='cleanup' and e[1]==record(bad)['task_id'] for e in events)
# A modified local file blocks later acknowledgement retries.
mode='error';changed=send();wait(lambda:cleanup(changed).get('lastError'));old_calls=len(events)
(data/record(changed)['local_path']).write_bytes(b'changed')
due(changed);stop();start();wait(lambda:'已改变' in cleanup(changed).get('lastError',''))
assert len(events)==old_calls and record(changed)['status']=='completed'
# Already completed historic tasks without a receipt are never swept.
sql("insert into sys_task(type,provider,config_id,config_base_url,task_id,status,local_path,params,created_at,updated_at) values('video','minimax',1,'http://127.0.0.1:18981','historical','completed',?,'{}',?,?)",(record(first)['local_path'],ts,ts))
stop();start();assert len(events)==old_calls
if os.environ.get('CLEANUP_UI_TEST'):
sql("insert into dramas(id,title,created_at,updated_at) values(1,'Cleanup fixture',?,?)",(ts,ts))
sql("insert into episodes(id,drama_id,episode_number,title,content,script_content,created_at,updated_at) values(1,1,1,'Fixture','Test script','Test script',?,?)",(ts,ts))
for i in [1,2]:sql("insert into storyboards(id,episode_id,storyboard_number,description,duration,created_at,updated_at) values(?,1,?,'Cleanup fixture shot',4,?,?)",(i,i,ts,ts))
sql('update sys_task set storyboard_id=1,drama_id=1 where id=?',(first,))
sql('update sys_task set storyboard_id=2,drama_id=1 where id=?',(changed,))
subprocess.run(['node',str(root/'integration-work/browser-test/verify_video_cleanup_ui.cjs')],env={**os.environ,'CLEANUP_FIXTURE':str(data)},check=True)
report={'passed':True,'fixture':str(data),'checks':['decode and durable local save before ack','checksum receipt','cleanup failure keeps completed video','restart retry no redownload','corrupt video no cleanup','modified local file no cleanup','no historic sweep']}
(data/'result.json').write_text(json.dumps(report,indent=2),'utf-8');print(json.dumps(report))
finally:
if process:stop()
server.shutdown();log.close()
@@ -0,0 +1,17 @@
const fs=require('fs'),vm=require('vm');
const ts=require('../huobao-source/backend/node_modules/typescript');
const source=fs.readFileSync('huobao-source/backend/src/services/english-video-prompt.ts','utf8');
const part=source.slice(source.indexOf('const han ='),source.indexOf('export async function promptAssetNames'))+source.slice(source.indexOf('// Do not normalize'),source.indexOf('export async function translateVisualPrompt'));
const ctx={exports:{},console};vm.runInNewContext(ts.transpileModule(part,{compilerOptions:{module:ts.ModuleKind.CommonJS,target:ts.ScriptTarget.ES2022}}).outputText,ctx);
const utils={exports:{}};vm.runInNewContext(ts.transpileModule(fs.readFileSync('huobao-source/backend/src/utils/storyboard-generated-prompt.ts','utf8'),{compilerOptions:{module:ts.ModuleKind.CommonJS,target:ts.ScriptTarget.ES2022}}).outputText,utils);
for(const folder of process.argv.slice(2)){
const fixture=JSON.parse(fs.readFileSync(folder+'/drafts.json','utf8'));
for(const [i,call] of fixture.calls.entries())for(const shot of call.storyboards||[]){
const norm=p=>utils.exports.normalizeDraftAssets(utils.exports.normalizeDraftVoices(p,fixture.characters,shot.character_ids||[]),fixture.names);
try{ctx.exports.validateEnglishPrompt(norm(shot.video_prompt),norm(shot.video_prompt_en)||'',fixture.names);console.log(JSON.stringify({folder,call:i,shot:shot.shot_number,valid:true}));}
catch(e){
const from=ctx.exports.protectPrompt(norm(shot.video_prompt)||'',fixture.names),to=ctx.exports.protectPrompt(norm(shot.video_prompt_en)||'',fixture.names);
console.log(JSON.stringify({folder,call:i,shot:shot.shot_number,error:e.message,unprotectedHan:(to.text.match(/\p{Script=Han}+/gu)||[]),missing:from.literals.filter(x=>!to.literals.includes(x)),extra:to.literals.filter(x=>!from.literals.includes(x))}));
}
}
}
+26
View File
@@ -0,0 +1,26 @@
import json,pathlib,sqlite3,subprocess
root=pathlib.Path('/home/ps/huobao-video-cleanup')
unit=pathlib.Path.home()/'.config/systemd/user/huobao-video-cleanup.service'
# Avoid a target ordering cycle on the next server boot; no restart is needed.
text=unit.read_text()
if 'After=default.target\n' in text:
unit.write_text(text.replace('After=default.target\n',''))
subprocess.run(['systemctl','--user','daemon-reload'],check=True,capture_output=True)
assert subprocess.check_output(['systemctl','--user','is-active','huobao-video-cleanup.service'],text=True).strip()=='active'
assert subprocess.check_output(['systemctl','--user','is-enabled','huobao-video-cleanup.service'],text=True).strip()=='enabled'
assert subprocess.check_output(['loginctl','show-user','ps','-p','Linger'],text=True).strip()=='Linger=yes'
config=json.loads((root/'config.json').read_text());counts={}
for b in config['bridges']:
with sqlite3.connect('file:'+b['database']+'?mode=ro',uri=True) as db:
counts[b['name']]=dict(db.execute('select status,count(*) from video_cleanup_acks group by status'))
print(json.dumps({'active':True,'enabled':True,'bootPersistence':True,'cleanupCounts':counts}))
code='''import app,asyncio,httpx,json
async def main():
async with httpx.AsyncClient(timeout=10) as client:
for p in app.providers().values():
response=await client.post(p.base_url+'/history',headers=app.provider_headers(p),json={'delete':[]});response.raise_for_status()
print(json.dumps({'provider':p.id,'historyControlReady':True,'deleted':0}))
asyncio.run(main())'''
result=subprocess.run(['docker','exec','canvas-h3-prompt-normalizer-healthy-20261026','python','-c',code],capture_output=True,text=True)
assert result.returncode==0,'Worker control test failed'
print(result.stdout)
@@ -0,0 +1,32 @@
"""Read-only data and installation verification after the first-frame feature upgrade."""
import hashlib,json,pathlib,sqlite3,sys,urllib.request
root=pathlib.Path(__file__).resolve().parent.parent
backup=pathlib.Path((root/'integration-work/current-upgrade-backup.txt').read_text('utf-8').strip())
def connect(path):return sqlite3.connect(path.as_uri()+'?mode=ro',uri=True)
with connect(backup/'huobao-data/huobao.sqlite3') as old,connect(root/'huobao-data/huobao.sqlite3') as live:
tables=[r[0] for r in old.execute("select name from sqlite_master where type='table' and name!='sqlite_sequence' order by name")]
for table in tables:
columns=[r[1] for r in old.execute(f'pragma table_info("{table}")')]
query='select '+','.join('"'+c+'"' for c in columns)+' from "'+table+'" order by rowid'
assert old.execute(query).fetchall()==live.execute(query).fetchall(),table+' changed'
assert live.execute('pragma integrity_check').fetchone()[0]=='ok'
counts={name:live.execute('select count(*) from '+name).fetchone()[0] for name in ['ai_service_configs','dramas','episodes','storyboards','style_presets']}
manifest=json.loads((backup/'manifest.json').read_text('utf-8'))
media=[item for item in manifest['data_files'] if item['path'].startswith('static/')]
for item in media:
assert hashlib.sha256((root/'huobao-data'/item['path']).read_bytes()).hexdigest()==item['sha256'],item['path']
runtime=root/'HuobaoDrama/resources'
build=json.loads((runtime/'huobao-local-build.json').read_text())
assert 'ltx-wan-all-reference-first-frame-composition' in build['custom_features']
assert hashlib.sha256((runtime/'app.asar').read_bytes()).hexdigest()==build['archive_sha256']
front=root/'huobao-source/frontend/.output/public'
for file in front.rglob('*'):
if file.is_file():assert file.read_bytes()==(runtime/'frontend'/file.relative_to(front)).read_bytes(),str(file)
launch=json.loads((root/'huobao-data/last-launch.json').read_text('utf-8'))
with urllib.request.urlopen(launch['api']+'/health',timeout=10) as response:assert response.status==200
report={'passed':True,'backup':str(backup),'database_tables_identical':len(tables),'media_files_unchanged':len(media),'counts':counts,'archive_sha256':build['archive_sha256'],'frontend_identical':True,'health':'ok','real_model_jobs_started':0}
report_name=sys.argv[1] if len(sys.argv)>1 else 'composed-frame-deployed-audit.json'
assert pathlib.Path(report_name).name==report_name and report_name.endswith('.json')
if report_name.startswith('video-cleanup'):assert 'verified-local-video-comfy-source-cleanup' in build['custom_features']
(root/'integration-work'/report_name).write_text(json.dumps(report,indent=2),'utf-8')
print(json.dumps(report))
@@ -0,0 +1,81 @@
"""Reproduce the existing storyboard request in a private DB copy; never change live projects or submit media."""
import contextlib,json,os,pathlib,re,shutil,sqlite3,subprocess,time,urllib.request,urllib.error
ROOT=pathlib.Path(__file__).resolve().parent.parent
OUT=ROOT/'integration-work'/('storyboard-coverage-verification-'+time.strftime('%Y%m%d-%H%M%S')+'-'+os.getenv('STORYBOARD_DIAG_PROVIDER','deepseek'));OUT.mkdir()
(ROOT/'integration-work/current-storyboard-diagnostic.txt').write_text(str(OUT),'utf-8')
shutil.copytree(ROOT/'integration-work/stage-4.0.11/app/resources/workspace-template',OUT/'workspace')
dbfile=OUT/'huobao.sqlite3'
with contextlib.closing(sqlite3.connect((ROOT/'huobao-data/huobao.sqlite3').as_uri()+'?mode=ro',uri=True)) as live,contextlib.closing(sqlite3.connect(dbfile)) as db:live.backup(db)
with sqlite3.connect(dbfile) as db:
db.row_factory=sqlite3.Row
db.execute('delete from sys_task');db.execute('delete from video_merges');db.execute("delete from app_settings where key like 'storyboard_job:%'")
assert dbfile.resolve().is_relative_to(OUT.resolve()) and dbfile.resolve()!= (ROOT/'huobao-data/huobao.sqlite3').resolve()
for (table,) in db.execute("select name from sqlite_master where type='table' and name not in ('sqlite_sequence','ai_service_configs','style_presets')").fetchall():
db.execute('delete from "'+table.replace('"','""')+'"')
ts='2026-11-13T00:00:00Z'
script=(ROOT/'integration-work/storyboard-coverage-fixture.txt').read_text('utf-8')
db.execute('insert into dramas(id,title,created_at,updated_at) values(7,?,?,?)',('Museum repair fixture',ts,ts))
db.execute('insert into episodes(id,drama_id,episode_number,title,content,script_content,created_at,updated_at) values(8,7,1,?,?,?,?,?)',('Fixture',script,script,ts,ts))
for ident,name in [(52,'林师傅'),(53,'小周')]:db.execute('insert into characters(id,drama_id,name,appearance,created_at,updated_at) values(?,7,?,?,?,?)',(ident,name,'成年博物馆工作人员,穿工作服',ts,ts))
for ident,name in [(33,'修复工作室'),(34,'博物馆屋顶'),(35,'验收大厅')]:db.execute('insert into scenes(id,drama_id,location,time,prompt,created_at,updated_at) values(?,7,?,?,?,?,?)',(ident,name,'白天','安静整洁的工作空间,自然光',ts,ts))
ep=dict(db.execute('select * from episodes where id=8').fetchone())
cfg=dict(db.execute("select * from ai_service_configs where service_type='text' and is_active=1 order by priority desc limit 1").fetchone())
if os.getenv('STORYBOARD_DIAG_PROVIDER'):
selector=os.environ['STORYBOARD_DIAG_PROVIDER']
cfg=dict(db.execute("select * from ai_service_configs where service_type='text' and is_active=1 and (provider=? or model like ?) order by priority desc limit 1",(selector,'%'+selector+'%')).fetchone())
model=json.loads(cfg['model'])[0]
chars=[dict(r) for r in db.execute('select id,name from characters where drama_id=? and deleted_at is null',(ep['drama_id'],))]
scenes=[dict(r) for r in db.execute('select id,location from scenes where drama_id=? and deleted_at is null',(ep['drama_id'],))]
body={'drama_id':ep['drama_id'],'episode_id':8,'config_id':cfg['id'],'model':model,'message':'请基于当前集剧本拆分完整分镜,并为每个分镜段落同时生成 video_prompt(中文视频提示词)和 video_prompt_en(英文制作提示词,中文对白及 @/# 素材引用原样保留)。本次视频模型:MiniMax-H3。当前集已有角色:'+json.dumps(chars,ensure_ascii=False)+'。当前集已有场景:'+json.dumps(scenes,ensure_ascii=False)+'。必须从已有角色和场景中选择正确 ID 绑定,按顺序分批保存全部分镜。'}
source=(ROOT/'huobao-source/desktop/build/backend.mjs').read_text('utf-8')
stage=ROOT/'integration-work/stage-4.0.11/app'
source=source.replace('const require = __hbCreateRequire(import.meta.url)','const require = __hbCreateRequire('+json.dumps(str(stage/'resources/app.asar/build/backend.mjs'))+')',1)
source=re.sub(r'^import (\w+) from "(better-sqlite3|sharp)";',r'const \1 = require("\2");',source,flags=re.M)
instrument=r'''
const __diagFetch=globalThis.fetch; let __diagN=0;
globalThis.fetch=async(input,init)=>{
const url=new URL(typeof input==='string'?input:input.url||String(input));
if(url.hostname==='127.0.0.1')return __diagFetch(input,init);
const n=++__diagN,t=Date.now();let b={};try{b=JSON.parse(init?.body||'{}')}catch{}
for (const m of b.messages||[]) if(m.tool_calls) for(const t of m.tool_calls) if(t.function?.name==='saveStoryboards') console.log('DRAFT '+JSON.stringify({n,id:t.id,args:t.function.arguments}));
console.log('DIAGNOSTIC '+JSON.stringify({event:'request',n,model:b.model,stream:b.stream,max_tokens:b.max_tokens,thinking:b.thinking,messages:b.messages?.length,input_chars:JSON.stringify(b.messages||[]).length,tools:b.tools?.map(x=>x.function.name),recent_tools:b.messages?.filter(x=>x.role==='tool').slice(-2).map(x=>({id:x.tool_call_id,content:String(x.content).slice(0,1200)})),recent_calls:b.messages?.filter(x=>x.tool_calls).slice(-1).map(x=>x.tool_calls.map(t=>({name:t.function.name,args:t.function.arguments.slice(0,150)})))}));
try{const r=await __diagFetch(input,{...init,signal:AbortSignal.any([...(init?.signal?[init.signal]:[]),AbortSignal.timeout(240000)])});
console.log('DIAGNOSTIC '+JSON.stringify({event:'headers',n,status:r.status,seconds:(Date.now()-t)/1000}));
r.clone().text().then(s=>console.log('DIAGNOSTIC '+JSON.stringify({event:'body',n,chars:s.length,seconds:(Date.now()-t)/1000,finish:s.match(/"finish_reason"\s*:\s*"[^" ]+"/g)?.slice(-2)}))).catch(()=>{});return r;
}catch(e){console.log('DIAGNOSTIC '+JSON.stringify({event:'error',n,name:e.name,seconds:(Date.now()-t)/1000}));throw e}
};
'''
(OUT/'backend.mjs').write_text(instrument+source,'utf-8')
env={**os.environ,'ELECTRON_RUN_AS_NODE':'1','PORT':os.getenv('STORYBOARD_DIAG_PORT','18988'),'HUOBAO_DATA_DIR':str(OUT),'SQLITE_PATH':str(dbfile),'FRONTEND_DIST':str(ROOT/'huobao-source/frontend/.output/public'),'WORKSPACE_PATH':str(OUT/'workspace')}
process=None
try:
with (OUT/'backend.log').open('w',encoding='utf-8') as log:
process=subprocess.Popen([str(stage/'HuobaoDrama.exe'),str(OUT/'backend.mjs')],cwd=OUT,env=env,stdout=log,stderr=log,creationflags=subprocess.CREATE_NO_WINDOW)
for _ in range(100):
if process.poll() is not None:raise RuntimeError('Isolated backend failed to start')
try:
urllib.request.urlopen('http://127.0.0.1:'+os.getenv('STORYBOARD_DIAG_PORT','18988')+'/api/v1/health',timeout=1).close();break
except OSError:time.sleep(.2)
start=time.monotonic();print(json.dumps({'started':str(OUT),'episode':8,'liveProjectUnmodified':True,'neutralTestScript':True,'model':model}),flush=True)
req=urllib.request.Request('http://127.0.0.1:'+os.getenv('STORYBOARD_DIAG_PORT','18988')+'/api/v1/agent/storyboard_breaker/jobs',data=json.dumps(body).encode(),headers={'Content-Type':'application/json'})
try:
with urllib.request.urlopen(req,timeout=600) as response:result=json.load(response)
except urllib.error.HTTPError as e:result=json.load(e)
except Exception as e:result={'error':type(e).__name__}
if result.get('data',{}).get('status')=='running':
while time.monotonic()-start<960:
time.sleep(3)
with urllib.request.urlopen('http://127.0.0.1:'+os.getenv('STORYBOARD_DIAG_PORT','18988')+'/api/v1/agent/storyboard_breaker/jobs/8',timeout=10) as response: result=json.load(response)
if result.get('data',{}).get('status')!='running':break
(OUT/'result.json').write_text(json.dumps(result,ensure_ascii=False,indent=2),'utf-8')
with sqlite3.connect(dbfile) as db:
db.row_factory=sqlite3.Row
saved=[dict(r) for r in db.execute('select storyboard_number,duration,video_prompt,video_prompt_en from storyboards where episode_id=8 order by storyboard_number')]
count=len(saved)
job=json.loads(db.execute("select value from app_settings where key='storyboard_job:8'").fetchone()[0])
(OUT/'coverage-artifact.json').write_text(json.dumps({'job':job,'saved':saved},ensure_ascii=False,indent=2),'utf-8')
print(json.dumps({'seconds':round(time.monotonic()-start,1),'shots':count,'status':result.get('data',{}).get('status'),'completed':result.get('data',{}).get('completed'),'jobError':result.get('data',{}).get('error'),'code':result.get('code'),'error':result.get('message') or result.get('error')}),flush=True)
assert job['status']=='done' and job['sectionIndex']==job['sectionCount']==3 and count==len(job['drafts']), 'Real model must finish every fixture scene and commit all drafts'
finally:
if process and process.poll() is None:process.terminate();process.wait(15)
@@ -0,0 +1,84 @@
"""Read-only deployment audit; allow active video tasks to advance naturally."""
import hashlib
import json
import pathlib
import sqlite3
import urllib.request
ROOT = pathlib.Path(__file__).resolve().parent.parent
BACKUP = pathlib.Path((ROOT / 'integration-work/current-upgrade-backup.txt').read_text('utf-8').strip())
def connect(path):
connection = sqlite3.connect(path.as_uri() + '?mode=ro', uri=True)
connection.row_factory = sqlite3.Row
return connection
with connect(BACKUP / 'huobao-data/huobao.sqlite3') as before, connect(ROOT / 'huobao-data/huobao.sqlite3') as after:
assert after.execute('pragma integrity_check').fetchone()[0] == 'ok'
prior_job_row=before.execute("select value from app_settings where key='storyboard_job:8'").fetchone()
prior_job=json.loads(prior_job_row[0]) if prior_job_row else None
current_job_row=after.execute("select value from app_settings where key='storyboard_job:8'").fetchone()
assert (current_job_row[0] if current_job_row else None)==(prior_job_row[0] if prior_job_row else None), 'Existing split record changed'
unchanged = []
for table in ('dramas', 'episodes', 'ai_service_configs', 'style_presets',
'episode_characters', 'episode_scenes', 'episode_props',
'storyboard_characters', 'storyboard_props'):
query = f'select * from "{table}" order by rowid'
assert [tuple(r) for r in before.execute(query)] == [tuple(r) for r in after.execute(query)], table
unchanged.append(table)
# Preserve every existing asset and storyboard field. Video workers may
# finish during startup and update only these delivery/status columns.
changed = {}
for table in ('storyboards', 'characters', 'scenes', 'props'):
live = {r['id']: dict(r) for r in after.execute(f'select * from "{table}"')}
delta = []
for old in before.execute(f'select * from "{table}"'):
assert old['id'] in live, (table, old['id'], 'missing')
new = live[old['id']]
fields = [k for k in old.keys() if old[k] != new[k]]
allowed = {'updated_at', 'status', 'video_url', 'image_url', 'local_path'}
assert set(fields) <= allowed, (table, old['id'], fields)
if fields:
delta.append({'id': old['id'], 'fields': fields})
changed[table] = delta
live_tasks = {r['id']: dict(r) for r in after.execute('select * from sys_task')}
active = []
for task in before.execute('select * from sys_task'):
assert task['id'] in live_tasks, ('task missing', task['id'])
if task['status'] in ('processing', 'pending', 'running', 'queued'):
new = live_tasks[task['id']]
assert new['status'] != 'cancelled', ('task unexpectedly cancelled', task['id'])
active.append({'id': task['id'], 'before': task['status'], 'after': new['status']})
counts = {table: after.execute(f'select count(*) from "{table}"').fetchone()[0]
for table in ('dramas', 'episodes', 'storyboards', 'ai_service_configs', 'style_presets')}
manifest = json.loads((BACKUP / 'manifest.json').read_text('utf-8'))
media = [item for item in manifest['data_files'] if item['path'].startswith('static/')]
for item in media:
path = ROOT / 'huobao-data' / item['path']
assert path.is_file() and hashlib.sha256(path.read_bytes()).hexdigest() == item['sha256'], item['path']
runtime = ROOT / 'HuobaoDrama/resources'
build = json.loads((runtime / 'huobao-local-build.json').read_text('utf-8'))
assert 'tracked-storyboard-jobs-atomic-bilingual-save' in build['custom_features']
assert 'storyboard-section-coverage-and-dialogue-timing' in build['custom_features']
assert hashlib.sha256((runtime / 'app.asar').read_bytes()).hexdigest() == build['archive_sha256']
frontend = ROOT / 'huobao-source/frontend/.output/public'
for path in frontend.rglob('*'):
if path.is_file():
assert path.read_bytes() == (runtime / 'frontend' / path.relative_to(frontend)).read_bytes(), str(path)
launch = json.loads((ROOT / 'huobao-data/last-launch.json').read_text('utf-8'))
with urllib.request.urlopen(launch['api'] + '/health', timeout=10) as response:
assert response.status == 200
with urllib.request.urlopen(launch['api'] + '/agent/storyboard_breaker/jobs/8', timeout=10) as response:
status = json.load(response)
assert 'data' in status, 'Split job route missing'
assert (status['data'] or {}).get('id')==(prior_job or {}).get('id'), 'Unexpected new production split job'
if prior_job and prior_job.get('status')=='done' and not prior_job.get('sections') and prior_job.get('seconds',0)<prior_job.get('targetSeconds',0)*.8:
assert status['data'].get('qualityWarning'), 'Old compressed split must show warning'
report = {'passed': True, 'backup': str(BACKUP), 'unchanged_tables': unchanged,
'runtime_field_changes': changed, 'existing_tasks_retained': len(live_tasks),
'active_tasks': active, 'counts': counts, 'media_files_unchanged': len(media),
'archive_sha256': build['archive_sha256'], 'frontend_identical': True,
'health': 'ok', 'new_route': 'ok', 'production_model_jobs_started': 0}
(ROOT / 'integration-work/storyboard-jobs-deployed-audit.json').write_text(json.dumps(report, indent=2), 'utf-8')
print(json.dumps(report))
@@ -0,0 +1,64 @@
"""Reproduce the existing storyboard request in a private DB copy; never change live projects or submit media."""
import contextlib,json,os,pathlib,re,shutil,sqlite3,subprocess,time,urllib.request,urllib.error
ROOT=pathlib.Path(__file__).resolve().parent.parent
OUT=ROOT/'integration-work'/('storyboard-real-verification-'+time.strftime('%Y%m%d-%H%M%S')+'-'+os.getenv('STORYBOARD_DIAG_PROVIDER','deepseek'));OUT.mkdir()
(ROOT/'integration-work/current-storyboard-diagnostic.txt').write_text(str(OUT),'utf-8')
shutil.copytree(pathlib.Path(os.environ['APPDATA'])/'HuobaoDrama/workspace',OUT/'workspace')
dbfile=OUT/'huobao.sqlite3'
with contextlib.closing(sqlite3.connect((ROOT/'huobao-data/huobao.sqlite3').as_uri()+'?mode=ro',uri=True)) as live,contextlib.closing(sqlite3.connect(dbfile)) as db:live.backup(db)
with sqlite3.connect(dbfile) as db:
db.row_factory=sqlite3.Row
db.execute('delete from sys_task');db.execute('delete from video_merges');db.execute("delete from app_settings where key like 'storyboard_job:%'")
ep=dict(db.execute('select * from episodes where id=8').fetchone())
cfg=dict(db.execute("select * from ai_service_configs where service_type='text' and is_active=1 order by priority desc limit 1").fetchone())
if os.getenv('STORYBOARD_DIAG_PROVIDER'):cfg=dict(db.execute("select * from ai_service_configs where service_type='text' and is_active=1 and provider=? limit 1",(os.environ['STORYBOARD_DIAG_PROVIDER'],)).fetchone())
model=json.loads(cfg['model'])[0]
chars=[dict(r) for r in db.execute('select id,name from characters where drama_id=? and deleted_at is null',(ep['drama_id'],))]
scenes=[dict(r) for r in db.execute('select id,location from scenes where drama_id=? and deleted_at is null',(ep['drama_id'],))]
body={'drama_id':ep['drama_id'],'episode_id':8,'config_id':cfg['id'],'model':model,'message':'请基于当前集剧本拆分完整分镜,并为每个分镜段落同时生成 video_prompt(中文视频提示词)和 video_prompt_en(英文制作提示词,中文对白及 @/# 素材引用原样保留)。本次视频模型:MiniMax-H3。当前集已有角色:'+json.dumps(chars,ensure_ascii=False)+'。当前集已有场景:'+json.dumps(scenes,ensure_ascii=False)+'。必须从已有角色和场景中选择正确 ID 绑定,按顺序分批保存全部分镜。'}
source=(ROOT/'huobao-source/desktop/build/backend.mjs').read_text('utf-8')
stage=ROOT/'integration-work/stage-4.0.11/app'
source=source.replace('const require = __hbCreateRequire(import.meta.url)','const require = __hbCreateRequire('+json.dumps(str(stage/'resources/app.asar/build/backend.mjs'))+')',1)
source=re.sub(r'^import (\w+) from "(better-sqlite3|sharp)";',r'const \1 = require("\2");',source,flags=re.M)
instrument=r'''
const __diagFetch=globalThis.fetch; let __diagN=0;
globalThis.fetch=async(input,init)=>{
const url=new URL(typeof input==='string'?input:input.url||String(input));
if(url.hostname==='127.0.0.1')return __diagFetch(input,init);
const n=++__diagN,t=Date.now();let b={};try{b=JSON.parse(init?.body||'{}')}catch{}
for (const m of b.messages||[]) if(m.tool_calls) for(const t of m.tool_calls) if(t.function?.name==='saveStoryboards') console.log('DRAFT '+JSON.stringify({n,id:t.id,args:t.function.arguments}));
console.log('DIAGNOSTIC '+JSON.stringify({event:'request',n,model:b.model,stream:b.stream,max_tokens:b.max_tokens,thinking:b.thinking,messages:b.messages?.length,input_chars:JSON.stringify(b.messages||[]).length,tools:b.tools?.map(x=>x.function.name),recent_tools:b.messages?.filter(x=>x.role==='tool').slice(-2).map(x=>({id:x.tool_call_id,content:String(x.content).slice(0,1200)})),recent_calls:b.messages?.filter(x=>x.tool_calls).slice(-1).map(x=>x.tool_calls.map(t=>({name:t.function.name,args:t.function.arguments.slice(0,150)})))}));
try{const r=await __diagFetch(input,{...init,signal:AbortSignal.any([...(init?.signal?[init.signal]:[]),AbortSignal.timeout(240000)])});
console.log('DIAGNOSTIC '+JSON.stringify({event:'headers',n,status:r.status,seconds:(Date.now()-t)/1000}));
r.clone().text().then(s=>console.log('DIAGNOSTIC '+JSON.stringify({event:'body',n,chars:s.length,seconds:(Date.now()-t)/1000,finish:s.match(/"finish_reason"\s*:\s*"[^" ]+"/g)?.slice(-2)}))).catch(()=>{});return r;
}catch(e){console.log('DIAGNOSTIC '+JSON.stringify({event:'error',n,name:e.name,seconds:(Date.now()-t)/1000}));throw e}
};
'''
(OUT/'backend.mjs').write_text(instrument+source,'utf-8')
env={**os.environ,'ELECTRON_RUN_AS_NODE':'1','PORT':os.getenv('STORYBOARD_DIAG_PORT','18988'),'HUOBAO_DATA_DIR':str(OUT),'SQLITE_PATH':str(dbfile),'FRONTEND_DIST':str(ROOT/'huobao-source/frontend/.output/public'),'WORKSPACE_PATH':str(OUT/'workspace')}
process=None
try:
with (OUT/'backend.log').open('w',encoding='utf-8') as log:
process=subprocess.Popen([str(stage/'HuobaoDrama.exe'),str(OUT/'backend.mjs')],cwd=OUT,env=env,stdout=log,stderr=log,creationflags=subprocess.CREATE_NO_WINDOW)
for _ in range(100):
if process.poll() is not None:raise RuntimeError('Isolated backend failed to start')
try:
urllib.request.urlopen('http://127.0.0.1:'+os.getenv('STORYBOARD_DIAG_PORT','18988')+'/api/v1/health',timeout=1).close();break
except OSError:time.sleep(.2)
start=time.monotonic();print(json.dumps({'started':str(OUT),'episode':8,'liveProjectUnmodified':True,'model':model}),flush=True)
req=urllib.request.Request('http://127.0.0.1:'+os.getenv('STORYBOARD_DIAG_PORT','18988')+'/api/v1/agent/storyboard_breaker/jobs',data=json.dumps(body).encode(),headers={'Content-Type':'application/json'})
try:
with urllib.request.urlopen(req,timeout=600) as response:result=json.load(response)
except urllib.error.HTTPError as e:result=json.load(e)
except Exception as e:result={'error':type(e).__name__}
if result.get('data',{}).get('status')=='running':
while time.monotonic()-start<960:
time.sleep(3)
with urllib.request.urlopen('http://127.0.0.1:'+os.getenv('STORYBOARD_DIAG_PORT','18988')+'/api/v1/agent/storyboard_breaker/jobs/8',timeout=10) as response: result=json.load(response)
if result.get('data',{}).get('status')!='running':break
(OUT/'result.json').write_text(json.dumps(result,ensure_ascii=False,indent=2),'utf-8')
with sqlite3.connect(dbfile) as db:count=db.execute('select count(*) from storyboards where episode_id=8').fetchone()[0]
print(json.dumps({'seconds':round(time.monotonic()-start,1),'shots':count,'status':result.get('data',{}).get('status'),'completed':result.get('data',{}).get('completed'),'jobError':result.get('data',{}).get('error'),'code':result.get('code'),'error':result.get('message') or result.get('error')}),flush=True)
finally:
if process and process.poll() is None:process.terminate();process.wait(15)

Some files were not shown because too many files have changed in this diff Show More