FJJ

模型介绍

本模型依托魔搭社区（ModelScope）AIGC专区模型训练环境与算力完成训练。

• 模型类型：LoRA
• 基础模型：Lightricks/LTX-2.3
• 训练代码：DiffSynth-Studio
• 训练数据量：30
• 总训练步数：5000
• 开源协议：Apache-2.0

推理代码

安装 DiffSynth-Studio：

pip install diffsynth

开始推理：

import torch
from diffsynth.pipelines.ltx2_audio_video import LTX2AudioVideoPipeline, ModelConfig
from diffsynth.utils.data.media_io_ltx2 import write_video_audio_ltx2

vram_config = {
    "offload_dtype": torch.float8_e5m2,
    "offload_device": "cpu",
    "onload_dtype": torch.float8_e5m2,
    "onload_device": "cpu",
    "preparing_dtype": torch.float8_e5m2,
    "preparing_device": "cuda",
    "computation_dtype": torch.bfloat16,
    "computation_device": "cuda",
}
pipe = LTX2AudioVideoPipeline.from_pretrained(
    torch_dtype=torch.bfloat16,
    device="cuda",
    model_configs=[
        ModelConfig(model_id="google/gemma-3-12b-it-qat-q4_0-unquantized", origin_file_pattern="model-*.safetensors", **vram_config),
        ModelConfig(model_id="Lightricks/LTX-2.3", origin_file_pattern="ltx-2.3-22b-dev.safetensors", **vram_config),
        ModelConfig(model_id="Lightricks/LTX-2.3", origin_file_pattern="ltx-2.3-spatial-upscaler-x2-1.0.safetensors", **vram_config),
    ],
    tokenizer_config=ModelConfig(model_id="google/gemma-3-12b-it-qat-q4_0-unquantized"),
    stage2_lora_config=ModelConfig(model_id="Lightricks/LTX-2.3", origin_file_pattern="ltx-2.3-22b-distilled-lora-384.safetensors"),
    vram_limit=torch.cuda.mem_get_info("cuda")[1] / (1024 ** 3) - 0.5,
)
pipe.load_lora(pipe.dit, ModelConfig(model_id="yy3yy3/FJJ", origin_file_pattern="FJJ_c520519-st5000.safetensors"))
prompt = "a cat"
video, audio = pipe(
    prompt=prompt,
    negative_prompt=pipe.default_negative_prompt["LTX-2.3"],
    height=1536, width=1024, num_frames=121,
    tiled=True, use_two_stage_pipeline=True,
)
write_video_audio_ltx2(video, audio, 'video.mp4', fps=24)