Skip to content

auto works on SwarmUI #37

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
FurkanGozukara wants to merge 4 commits into
base: main
Choose a base branch
from
Open

auto works on SwarmUI #37

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
82be856
auto works on SwarmUI
FurkanGozukara Jun 25, 2026
406a7a9
if comfyui supports bypass
FurkanGozukara Jul 1, 2026
78a43cc
improvement
FurkanGozukara Jul 11, 2026
7bb27d7
fixed bugs and now supports GGUF of Krea2
FurkanGozukara Jul 11, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
123 changes: 89 additions & 34 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,60 +1,115 @@
# ComfyUI-QuantOps

Extended quantization layouts for ComfyUI, enabling loading and inference with models quantized by [convert_to_quant](https://github.com/silveroxides/convert_to_quant).
Additional quantized tensor layouts and model loaders for ComfyUI. The metadata formats are produced by [convert_to_quant](https://github.com/silveroxides/convert_to_quant).

This is experimental and due to lack of proper support and merging of PR in ComfyUI, do not expect this to work without putting in the effort.
I don't have the time or the energy to keep this up and will close entire project if i keep getting bunch of low effort issues posted expecting me go serve a fix up on a silver platter.
## Automatic routing

QuantOps records ComfyUI's native quantization formats before registering its own layouts. Normal checkpoint and diffusion-model loaders then route each model from its embedded metadata:

### tl;dr Go complain at ComfyOrg. Not here.
1. Models using only ComfyUI-native formats stay on the ComfyUI path.
2. Models containing a QuantOps-only format receive QuantOps custom operations.
3. Unquantized and unknown formats are left to ComfyUI instead of being claimed by QuantOps.

### The following is the last update I make regarding this.
This is automatic in normal ComfyUI workflows and in SwarmUI-generated workflows. Swarm presets and dtype overrides continue through the original loader arguments.

In order to use int8_tensorwise(RTX 30xx-series or newer GPU) you will need the following:
The log reports every decision as `decision=native-bypass`, `decision=quantops`, `decision=no-quantization`, or `decision=core-pass-through`.

- torch 2.10+cu130 or higher
- installed the latest of my custom comfy-kitchen fork wheels with the int8-tensorwise support
- enable the use of triton backend by using --enable-triton-backend launch argument in ComfyUI
## QuantOps-only metadata formats

Step 1: Install Triton
Activate your virtual environment used by ComfyUI and install triton.
For Windows you need to use this but linux can install latest triton as usual.
```
# for torch 2.10 and 2.11
pip install -U "triton-windows<3.7"
# for torch 2.12
pip install -U "triton-windows<3.8"
```
These canonical metadata formats are not native to the current ComfyUI quantization registry and have been verified through the automatic loader path:

Step 3: Install my comfy-kitchen
Download the latest uploaded version matching you python of my pre-compiled .whl file from my [HuggingFace repository](https://huggingface.co/silveroxides/comfy-kitchen-int8-wheels/tree/main) (Latest as of 13 June 2026)
| Metadata format | Storage and scaling | Default execution |
| --- | --- | --- |
| `int8_blockwise` | INT8 weights with a 2D block-scale grid, normally 128 x 128 blocks | PyTorch fallback by default; optional Triton backend in the Quantized loaders |
| `float8_e4m3fn_rowwise` | FP8 E4M3 weights with one scale per output row | QuantOps Triton kernel on compatible CUDA systems; dequantized fallback otherwise |
| `float8_e4m3fn_blockwise` | FP8 E4M3 weights with a 2D block-scale grid, normally 64 x 64 blocks | QuantOps Triton kernel on compatible CUDA systems; dequantized fallback otherwise |

Install it directly pointing to the file path:
```
pip install --no-deps --force-reinstall --no-cache-dir "path/to/comfy-kitchen.whl"
The legacy metadata name `int8` is accepted as a blockwise INT8 compatibility alias when the scale shape identifies a block layout. New files should use `int8_blockwise`.

### Conditional format

`hybrid_mxfp8` describes 32-element MXFP8 blocks with an additional weight scalar. It is usable only when the installed `comfy-kitchen` exports `HybridMXFP8Layout`. The installed `comfy-kitchen` 0.2.17 in the tested environment does not export that class, so this format is not available there.

## Dedicated BNB 4-bit formats

The `BNB4bitUNETLoader` node supports bitsandbytes-compatible `NF4` and `FP4` safetensors without requiring bitsandbytes at runtime. It uses pure PyTorch dequantization and currently targets Flux, Flux 2, Chroma, Chroma Radiance, and Chroma Radiance X0 model structures.

NF4 and FP4 use their dedicated loader. They are not automatic `QUANT_ALGOS` metadata formats.

## Formats already native to ComfyUI

QuantOps does not replace ComfyUI operations for these formats:

- `convrot_w4a4`
- `float8_e4m3fn`
- `float8_e5m2`
- `int8_tensorwise`, including per-channel files whose names may say rowwise
- `mxfp8`
- `nvfp4`

For these models the expected log is `decision=native-bypass` followed by `path=core result=completed`. Embedded metadata is authoritative; filenames are not used to select a quantization layout.

## GGUF integration

GGUF is delegated to an installed provider that registers `UnetLoaderGGUF`, such as [ComfyUI-GGUF](https://github.com/city96/ComfyUI-GGUF). QuantOps does not duplicate or replace its GGUF tensor implementation.

When the provider is present, QuantOps adds automatic image-architecture recognition for:

- `krea2`
- `ideogram`

SwarmUI already selects `UnetLoaderGGUF` for `.gguf` diffusion models. QuantOps extends the provider in memory and logs the detected architecture, GGUF tensor types, state-key count, and acceptance or error result. If no provider is installed, GGUF integration is disabled without affecting other formats.

## Loader coverage

- Stock ComfyUI diffusion-model loaders: automatic native or QuantOps routing.
- Stock ComfyUI checkpoint loaders: automatic routing for the diffusion-model portion.
- `QuantizedModelLoader` and `QuantizedUNETLoader`: explicit format and backend controls.
- `QuantizedCLIPLoader` and `QuantizedDualCLIPLoader`: quantized text-encoder loading.
- `BNB4bitUNETLoader`: dedicated NF4 and FP4 loading.

## Runtime diagnostics

Useful successful log sequences include:

```text
ComfyUI-QuantOps diagnostic: loader=diffusion decision=native-bypass ...
ComfyUI-QuantOps diagnostic: loader=diffusion path=core result=completed

ComfyUI-QuantOps diagnostic: loader=diffusion decision=quantops ...
ComfyUI-QuantOps FP8 blockwise: path=triton-dynamic ...
ComfyUI-QuantOps diagnostic: loader=diffusion path=quantops result=completed

ComfyUI-QuantOps GGUF diagnostic: decision=gguf-provider ...
ComfyUI-QuantOps GGUF diagnostic: result=accepted architecture=krea2 ...
```

Step 4: Install/Update ComfyUI-QuantOps
You just need to ensure it's fully up to date to read the new model formats.
Run these commands:
Repeated kernel-path messages are rate-limited. If an FP8 Triton kernel fails, QuantOps disables that kernel path for the session, emits one concise warning, and uses the dequantized fallback.

## Triton

Use a Triton build compatible with the PyTorch and CUDA versions in the ComfyUI virtual environment. On Windows this normally means `triton-windows`. The ComfyUI launch option below enables its `comfy-kitchen` Triton backend:

```text
--enable-triton-backend
```

QuantOps FP8 kernels are selected when a compatible Triton installation and CUDA device are available. The explicit Quantized loaders expose the blockwise INT8 backend choice.

## Updating

```text
cd custom_nodes/ComfyUI-QuantOps
git pull
```

When launching Comfyui add launch argument:
```
--enable-triton-backend
```

You can get most of the models here: https://huggingface.co/silveroxides
Quantized model releases are available from [silveroxides on Hugging Face](https://huggingface.co/silveroxides).

## License

MIT License

## Acknowledgements

- [lyogavin](https://github.com/lyogavin) for [PR #10864](https://github.com/comfyanonymous/ComfyUI/pull/10864) to ComfyUI.
- [Clybius](https://github.com/Clybius) for inspiring me to take on quantization and his [Learned-Rounding](https://github.com/Clybius/Learned-Rounding) repository.
- [lyogavin](https://github.com/lyogavin) for [ComfyUI PR #10864](https://github.com/comfyanonymous/ComfyUI/pull/10864).
- [Clybius](https://github.com/Clybius) for the [Learned-Rounding](https://github.com/Clybius/Learned-Rounding) project.
125 changes: 79 additions & 46 deletions __init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@

This custom node extends ComfyUI's quantization system with additional layouts:
- INT8 blockwise (with optional Triton acceleration)
- INT8 tensorwise (uses torch._int_mm with dynamic activation quant)
- Row-wise and Block-wise FP8 variants

All layouts are lazy-loaded to avoid import errors when optional dependencies
Expand All @@ -12,6 +11,12 @@

import logging

import torch
from comfy.quant_ops import QUANT_ALGOS, register_layout_class


_NATIVE_COMFY_FORMATS = frozenset(QUANT_ALGOS)

# =============================================================================
# Module-level state for comfy-kitchen backend integration
# =============================================================================
Expand All @@ -33,8 +38,6 @@ def is_ck_triton_available() -> bool:
def _setup_comfy_kitchen_backends():
"""
Configure comfy-kitchen backends for QuantOps.

1. Re-enable triton backend (ComfyUI disables it by default)
"""
global _CK_AVAILABLE, _CK_TRITON_AVAILABLE

Expand All @@ -47,23 +50,23 @@ def _setup_comfy_kitchen_backends():
_CK_TRITON_AVAILABLE = False
return

# Step 1: Re-enable triton backend (ComfyUI disables it)
try:
ck.enable_backend("triton")

backends = ck.list_backends()
triton_info = backends.get("triton", {})

if triton_info.get("available") and not triton_info.get("disabled"):
_CK_TRITON_AVAILABLE = True
logging.info("ComfyUI-QuantOps: Enabled comfy-kitchen triton backend")
logging.info("ComfyUI-QuantOps: comfy-kitchen triton backend available")
elif triton_info.get("available"):
logging.info("ComfyUI-QuantOps: comfy-kitchen triton backend disabled")
_CK_TRITON_AVAILABLE = False
else:
unavail_reason = triton_info.get("unavailable_reason", "unknown")
logging.info(f"ComfyUI-QuantOps: comfy-kitchen triton unavailable: {unavail_reason}")
_CK_TRITON_AVAILABLE = False

except Exception as e:
logging.warning(f"ComfyUI-QuantOps: Failed to enable ck triton backend: {e}")
logging.warning(f"ComfyUI-QuantOps: Failed to inspect ck triton backend: {e}")
_CK_TRITON_AVAILABLE = False


Expand All @@ -75,37 +78,40 @@ def _setup_comfy_kitchen_backends():
def _register_layouts():
"""Register our custom layouts into ComfyUI's layout registry and QUANT_ALGOS dict."""
try:
from comfy.quant_ops import QUANT_ALGOS, register_layout_class
import torch

# Import our layouts (this also registers their operation handlers)
from .quant_layouts.int8_layout import BlockWiseINT8Layout
from .quant_layouts.fp8_variants import RowWiseFP8Layout, BlockWiseFP8Layout

# Register layouts using the new comfy_kitchen API
register_layout_class("BlockWiseINT8Layout", BlockWiseINT8Layout)
register_layout_class("RowWiseFP8Layout", RowWiseFP8Layout)
register_layout_class("BlockWiseFP8Layout", BlockWiseFP8Layout)

# Tensorwise INT8 from comfy_kitchen
try:
from comfy_kitchen.tensor.int8 import TensorWiseINT8Layout
register_layout_class("TensorWiseINT8Layout", TensorWiseINT8Layout)
# Load our patch for per-channel scale support
from .quant_layouts import tensorwise_int8_layout
logging.info("ComfyUI-QuantOps: Registered TensorWiseINT8Layout")
except ImportError:
logging.debug("ComfyUI-QuantOps: TensorWiseINT8Layout not available")
registered = []
if "int8_blockwise" not in QUANT_ALGOS:
from .quant_layouts.int8_layout import BlockWiseINT8Layout

register_layout_class("BlockWiseINT8Layout", BlockWiseINT8Layout)
registered.append("BlockWiseINT8Layout")
if "float8_e4m3fn_rowwise" not in QUANT_ALGOS:
from .quant_layouts.fp8_variants import RowWiseFP8Layout

register_layout_class("RowWiseFP8Layout", RowWiseFP8Layout)
registered.append("RowWiseFP8Layout")
if "float8_e4m3fn_blockwise" not in QUANT_ALGOS:
from .quant_layouts.fp8_variants import BlockWiseFP8Layout

register_layout_class("BlockWiseFP8Layout", BlockWiseFP8Layout)
registered.append("BlockWiseFP8Layout")

# Tensorwise INT8 is native in current ComfyUI. Only register it on
# older installs that do not already expose the format.
if "int8_tensorwise" not in QUANT_ALGOS:
try:
from comfy_kitchen.tensor.int8 import TensorWiseINT8Layout
register_layout_class("TensorWiseINT8Layout", TensorWiseINT8Layout)
QUANT_ALGOS["int8_tensorwise"] = {
"storage_t": torch.int8,
"parameters": {"weight_scale", "input_scale"},
"comfy_tensor_layout": "TensorWiseINT8Layout",
}
registered.append("TensorWiseINT8Layout")
logging.info("ComfyUI-QuantOps: Registered TensorWiseINT8Layout")
except ImportError:
logging.debug("ComfyUI-QuantOps: TensorWiseINT8Layout not available")

# Register QUANT_ALGOS
QUANT_ALGOS.setdefault(
"int8_tensorwise",
{
"storage_t": torch.int8,
"parameters": {"weight_scale", "input_scale"}, # Keep input_scale if checkpoints have it
"comfy_tensor_layout": "TensorWiseINT8Layout", # Must match the class name above
}
)
QUANT_ALGOS.setdefault(
"int8_blockwise",
{
Expand Down Expand Up @@ -137,8 +143,10 @@ def _register_layouts():
# MXFP8 from comfy_kitchen
try:
from comfy_kitchen.tensor import TensorCoreMXFP8Layout
register_layout_class("TensorCoreMXFP8Layout", TensorCoreMXFP8Layout)
logging.info("ComfyUI-QuantOps: Registered TensorCoreMXFP8Layout")
if "mxfp8" not in QUANT_ALGOS:
register_layout_class("TensorCoreMXFP8Layout", TensorCoreMXFP8Layout)
registered.append("TensorCoreMXFP8Layout")
logging.info("ComfyUI-QuantOps: Registered TensorCoreMXFP8Layout")
except ImportError:
logging.debug("ComfyUI-QuantOps: TensorCoreMXFP8Layout not available")

Expand All @@ -153,12 +161,14 @@ def _register_layouts():
)

# Hybrid MXFP8 from comfy_kitchen
try:
from comfy_kitchen.tensor import HybridMXFP8Layout
register_layout_class("HybridMXFP8Layout", HybridMXFP8Layout)
logging.info("ComfyUI-QuantOps: Registered HybridMXFP8Layout")
except ImportError:
logging.debug("ComfyUI-QuantOps: HybridMXFP8Layout not available")
if "hybrid_mxfp8" not in QUANT_ALGOS:
try:
from comfy_kitchen.tensor import HybridMXFP8Layout
register_layout_class("HybridMXFP8Layout", HybridMXFP8Layout)
registered.append("HybridMXFP8Layout")
logging.info("ComfyUI-QuantOps: Registered HybridMXFP8Layout")
except ImportError:
logging.debug("ComfyUI-QuantOps: HybridMXFP8Layout not available")

QUANT_ALGOS.setdefault(
"hybrid_mxfp8",
Expand All @@ -182,7 +192,6 @@ def _register_layouts():
)

# Verify registration
registered = ["BlockWiseINT8Layout", "TensorWiseINT8Layout", "RowWiseFP8Layout", "BlockWiseFP8Layout", "TensorCoreMXFP8Layout"]
logging.info(f"ComfyUI-QuantOps: Registered layouts: {registered}")

except Exception as e:
Expand All @@ -195,9 +204,33 @@ def _register_layouts():

# Setup backends first (enables ck triton, registers our kernels)
_setup_comfy_kitchen_backends()
logging.info(
"ComfyUI-QuantOps diagnostic: native formats before extension: %s",
", ".join(sorted(_NATIVE_COMFY_FORMATS)),
)

# Register layouts
_register_layouts()
logging.info(
"ComfyUI-QuantOps diagnostic: formats added to registry: %s",
", ".join(sorted(set(QUANT_ALGOS) - _NATIVE_COMFY_FORMATS)) or "none",
)

# Patch stock ComfyUI loaders so QuantOps-only metadata works from normal loaders.
try:
from .auto_patch import install_auto_patch

install_auto_patch(_NATIVE_COMFY_FORMATS)
except Exception as e:
logging.warning(f"ComfyUI-QuantOps: failed to install stock-loader auto patch: {e}")

# Extend an installed GGUF provider without replacing its loader implementation.
try:
from .gguf_integration import install_gguf_integration

install_gguf_integration()
except Exception as e:
logging.warning(f"ComfyUI-QuantOps: failed to install GGUF integration: {e}")

# Import nodes for ComfyUI discovery
from .nodes.loader_nodes import NODE_CLASS_MAPPINGS, NODE_DISPLAY_NAME_MAPPINGS
Expand Down
Loading