🤗 Hugging Face  |  🤖 ModelScope  |  💻 GitHub  |  💬 WeChat (微信)  |  🎮 Discord

⚡ QuantFunc — the fastest diffusion inference engine. Native C++/CUDA, zero Python model dependencies, 2x–11x speedup across RTX 20/30/40/50.

The official ComfyUI-QuantFunc plugin and engine: runtime & offline 4-bit quantization for Z-Image, Qwen-Image, Qwen-Image-Edit and FLUX.2 Klein pipelines.

Powered by the QuantFunc ComfyUI plugin — the fastest diffusion inference engine:

• 🚀 2x–11x speedup over standard BF16/FP16 Python pipelines (pre-exported → even faster loading).
• ⚙️ Native C++/CUDA (libquantfunc.so / quantfunc.dll) with zero Python model dependencies.
• 🧩 Dual engine (SVDQ offline + Lighting runtime 4-bit), zero-cost LoRA stacking, reference-image editing & inpainting.
• 🟢 Full GPU coverage — RTX 20/30/40/50 · A100/H100/H200/B100/B200/GB300 · RTX 6000 Ada / PRO Blackwell (CUDA 12 & 13); native FP4 on Blackwell.

👉 Install the plugin: https://github.com/RealJonathanYip/ComfyUI-QuantFunc

QuantFunc Plugin

The Fastest Diffusion Model Inference Engine

Up to 11x faster · Zero dependencies · LoRA at zero cost · ~130 MB binary

English | 中文

QuantFunc is a high-performance native inference engine for text-to-image and image editing diffusion models. Built entirely in C++/CUDA with zero Python dependency, it delivers 2x–11x speedup over mainstream Python frameworks across NVIDIA RTX 20/30/40/50 series GPUs — no manual tuning required, no environment setup, just download and run.

One binary. One command. The fastest image generation at your fingertips.

1. Why QuantFunc?

1.1 Extreme Speed — Automatically Optimized

QuantFunc doesn't just run models faster — it intelligently adapts to your hardware. The auto-optimization engine profiles your GPU at startup, determines the optimal VRAM strategy, and configures everything transparently.

Key capabilities:

• Full-pipeline kernel fusion — cross-layer optimization, not just individual operators
• Intelligent VRAM management — automatic compression and adaptive offloading
• Four-generation GPU support — Turing (RTX 20), Ampere (RTX 30), Ada Lovelace (RTX 40), Blackwell (RTX 50)
• Multi-GPU ready — out of the box, no configuration needed

1.2 Zero Setup — Download and Run

No Python, no virtual environments, no pip install, no CUDA/PyTorch version matrix. A single self-contained binary — download, point at a model, and generate.

./quantfunc --model-dir /path/to/model --prompt "A cute cat"

• ~130 MB on Linux, ~390 MB on Windows — smaller than a typical torch install
• Pre-built binaries — no compilation required
• C shared library for integration into any language (see Section 4.2)

1.3 Dual Engine Architecture

QuantFunc offers two complementary quantization engines:

1.3.1 Lighting Engine — Maximum Speed

An innovative mixed-precision quantization framework drawing on HQQ, HIGGS, and Rotation quantization. Core features:

• Quantize any FP16/FP32 model on-the-fly — no calibration dataset needed
• Per-layer precision control — fine-tune the speed–quality balance with surgical precision
• 20–30% faster inference than SVDQ
• Export pre-quantized models for instant loading thereafter

Recommended workflow: export once → run at full speed forever.

1.3.2 SVDQ Engine — High Quality, Plug-and-Play

Load pre-quantized weight files directly with deep kernel fusion and algorithmic optimization. Core features:

• Rotation quantization for reduced quantization error
• Runtime LoRA hot-loading for dynamic style switching
• Pre-quantized weights available from ModelScope / HuggingFace

1.3.3 Quick Comparison

# Lighting: build a pre-quantized model from FP16 weights
./quantfunc --model-dir /path/to/base-model \
  --model-backend lighting --precision-config precision.json \
  --export-path /path/to/output --auto-optimize

# SVDQ: download pre-quantized weights and run directly
./quantfunc --model-dir /path/to/base-model \
  --transformer /path/to/svdq-weights.safetensors \
  --prompt "A cute cat" --auto-optimize

1.3.4 Precision Options

Precision configs support ratio shorthand — e.g., {"4": 8, "i8": 2} means 80% 4-bit + 20% INT8, with higher-precision layers automatically placed deeper for quality.

Available precisions: "4" (auto 4-bit), "i4" (INT4), "f4" (FP4, SM120+), "8" (auto 8-bit), "i8" (INT8), "f8" (FP8), "af8wf4" (FP8 act + FP4 wgt), "fp16", "bf16".

See Lighting Inference for full config options.

1.4 Zero-Cost LoRA

1.4.1 Lighting: Build Your All-in-One Model

LoRAs are merged into base weights before quantization and export. The result:

• Zero runtime overhead — speed identical whether 1 or 100 LoRAs are merged
• Zero additional size — no extra bytes on disk
• Stack style, quality, character, and concept LoRAs into one personal AOI model

1.4.2 SVDQ: Smart Runtime LoRA

Runtime LoRA enhanced with Rotation quantization and intelligent rank compression:

LoRA_rank128 + LoRA_rank128 ≈ merged_rank192 (not 256)

Multi-LoRA stacking stays fast and memory-efficient without sacrificing quality.

2. Supported Models

Pipeline type is auto-detected from the model directory.

2.1 Model Download

Pre-quantized models are available on ModelScope and HuggingFace.

2.2 Base Model & Transformer Weight Separation

QuantFunc uses a split design — base model and transformer weights are distributed separately:

• Base model (*-base-model) — text encoder, VAE, vision encoder, tokenizer, etc.
• Transformer weights (*.safetensors) — quantized transformer parameters only

Why? A single base model is compatible with all transformer weight variants in the same series. Download once, switch between different engines and configs freely.

QuantFunc/Qwen-Image-Edit-Series/
├── qwen-image-edit-series-50x-below-base-model/   # Base model (RTX 30/40)
│   ├── model_index.json
│   ├── text_encoder/
│   ├── vae/
│   └── ...
├── qwen-image-edit-series-50x-above-base-model/   # Base model (RTX 50)
│   └── ...
└── transformer/                                    # Transformer weights
    ├── qwen-image-2511-50x-below-lighting-4steps.safetensors
    ├── qwen-image-2511-50x-below-svdq-4steps.safetensors
    └── ...

2.3 GPU Variant Guide

Important: Base model and transformer weights must use the same GPU variant.

2.4 Download & Use Example

# 1. Download models (ModelScope)
pip install modelscope
python -c "from modelscope import snapshot_download; snapshot_download('QuantFunc/Qwen-Image-Edit-Series')"

# 2. SVDQ inference
./quantfunc \
  --model-dir /path/to/qwen-image-edit-series-50x-below-base-model \
  --transformer /path/to/transformer/qwen-image-2511-50x-below-svdq-4steps.safetensors \
  --ref-image input.png --prompt "Change the sky to sunset" \
  --steps 4 --auto-optimize

# 3. Lighting inference — same base model, just swap transformer weights
./quantfunc \
  --model-dir /path/to/qwen-image-edit-series-50x-below-base-model \
  --transformer /path/to/transformer/qwen-image-2511-50x-below-lighting-4steps.safetensors \
  --ref-image input.png --prompt "Change the sky to sunset" \
  --steps 4 --auto-optimize

Tip: Once you have the base model, try any other transformer variant without re-downloading.

3. Quick Start

Download the pre-built binary for your platform and start generating immediately.

3.1 Linux

./quantfunc \
  --model-dir /path/to/model \
  --transformer /path/to/quantized-weights.safetensors \
  --prompt "A serene mountain landscape at sunset" \
  --device 0 --auto-optimize

3.2 Windows

quantfunc.exe --model-dir C:\models\my-model --prompt "A cute cat" --auto-optimize

3.3 Usage Examples

3.3.1 SVDQ Text-to-Image

./quantfunc --model-dir /path/to/model \
  --transformer /path/to/svdq-weights.safetensors \
  --prompt "A serene mountain landscape at sunset" \
  --device 0 --auto-optimize --steps 4

3.3.2 Lighting Text-to-Image (On-the-Fly Quantization)

./quantfunc --model-dir /path/to/model \
  --model-backend lighting \
  --precision-config '{"transformer_blocks.attn": "i4", "transformer_blocks.img_mlp": "i4", "transformer_blocks.img_mod": "i8"}' \
  --lora /path/to/lightning-lora.safetensors \
  --prompt "A cute cat" --steps 4 --auto-optimize \
  --quant-method higgs+hqq --rotation-block-size 256 --fused-mod

3.3.3 Export Lighting Model

./quantfunc --model-dir /path/to/model \
  --model-backend lighting --precision-config precision.json \
  --lora /path/to/lora.safetensors \
  --export-path /path/to/exported \
  --quant-method higgs+hqq --rotation-block-size 256 --fused-mod --auto-optimize

3.3.4 Load Exported Model

./quantfunc --model-dir /path/to/model \
  --transformer /path/to/exported/transformer/model.safetensors \
  --model-backend lighting --prompt "A cute cat" --steps 4 --auto-optimize --fused-mod

3.3.5 Image Editing (QwenImage-Edit)

./quantfunc --model-dir /path/to/edit-model \
  --transformer /path/to/weights.safetensors \
  --ref-image input.png --prompt "Change the sky to sunset" \
  --steps 4 --auto-optimize

4. Integration

4.1 ComfyUI

QuantFunc provides first-class ComfyUI support with full feature parity:

• Native acceleration via libquantfunc.so / quantfunc.dll — zero PyTorch overhead
• Full features — SVDQ, Lighting, LoRA, multi-reference editing, auto-optimize
• Process isolation — dedicated subprocess, keeps ComfyUI stable
• Real-time progress — step-by-step callbacks in the UI
• Multi-GPU — assign different nodes to different GPUs

Coming soon: Official ComfyUI custom node package.

4.2 C API — Integrate with Any Language

Clean C shared library with opaque handles — call QuantFunc from Python, Go, Rust, C#, Java, Node.js, Swift, and more.

#include "quantfunc.h"

quantfunc_pipeline_t pipeline;
quantfunc_create(&init_params, &pipeline);

quantfunc_image_t image;
quantfunc_text_to_image(pipeline, &t2i_params, &image);

quantfunc_image_save(image, "output.png");
quantfunc_destroy(pipeline);

Coming soon: Official server-side SDKs and integration examples.

5. Documentation

6. Features at a Glance

7. System Requirements

7.1 Installation (Linux)

QuantFunc is self-contained — no Python, PyTorch, or OpenCV needed. You only need NVIDIA CUDA runtime and cuDNN.

7.1.1 Install NVIDIA Driver

nvidia-smi   # Requires driver ≥ 560

7.1.2 Install CUDA 13 Runtime

# Add NVIDIA CUDA repository (Ubuntu 22.04 example)
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb
sudo apt update

# Install runtime libraries only
sudo apt install cuda-libraries-13-0

If cuda-libraries-13-0 is unavailable, install individual packages:

sudo apt install libcublas-13-0 libcurand-13-0 libcusolver-13-0 libcusparse-13-0 libnvjitlink-13-0

7.1.3 Install cuDNN 9

sudo apt install libcudnn9-cuda-13

7.1.4 Verify Installation

python3 -c "import ctypes; ctypes.CDLL('libquantfunc.so')"
# No error = ready to go

7.2 Installation (Windows)

Download quantfunc.exe and place it anywhere. Runtime dependencies:

• NVIDIA Driver ≥ 560
• Visual C++ Redistributable 2015-2022 (download)
• cuDNN 9.x (download)

7.3 First Run — JIT Notice

QuantFunc ships with pre-compiled SASS for RTX 30/40/50 (SM86/89/120) — instant startup. Other GPUs need a one-time PTX JIT compilation:

JIT results are cached at ~/.nv/ComputeCache/ and persist across reboots.

8. License

8.1 Engine License

QuantFunc engine (binaries and shared libraries) is proprietary software.

• Free to use during the public beta period, with no restrictions on commercial or personal use
• Redistribution, reverse engineering, and decompilation are prohibited
• Pricing and licensing terms will be announced before the beta period ends
• The author reserves the right to modify these terms at any time

8.2 Model Weights License

Pre-quantized model weights distributed by QuantFunc are derivative works of their respective upstream models. These weights inherit the license of the original model (e.g., Apache 2.0 for Qwen series). Users must independently comply with the original model's license terms. QuantFunc does not grant any rights beyond those provided by the upstream model license.

Stop waiting. Start generating.

Community

Join our community for support, updates, and discussions:

• 🎮 Discord server
• 💬 Scan the QR code below to join our WeChat group: