Skip to content

Fix gptqmodel backend check #2420

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
IlyasMoutawwakil merged 6 commits into from
Apr 15, 2026
Merged

Fix gptqmodel backend check #2420

Changes from 2 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
feac642
fix backend
jiqing-feng Apr 8, 2026
228215b
rm exllama_set_max_input_length
jiqing-feng Apr 8, 2026
e94a212
add backend check
jiqing-feng Apr 10, 2026
2c95366
add comments and format
jiqing-feng Apr 13, 2026
0101682
install pcre and setuptools for gptqmodel
jiqing-feng Apr 15, 2026
33334e0
Apply suggestion from @IlyasMoutawwakil
IlyasMoutawwakil Apr 15, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 2 additions & 3 deletions optimum/gptq/quantizer.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@
from accelerate.hooks import remove_hook_from_module

if is_gptqmodel_available():
from gptqmodel import BACKEND, QuantizeConfig, exllama_set_max_input_length
from gptqmodel import BACKEND, QuantizeConfig
from gptqmodel.quantization import FORMAT, GPTQ, METHOD
from gptqmodel.utils.importer import hf_select_quant_linear_v2
from gptqmodel.utils.model import hf_convert_gptq_v1_to_v2_format, hf_convert_gptq_v2_to_v1_format
Expand Down Expand Up @@ -669,8 +669,7 @@ class StoreAttr(object):
model.quantize_config = StoreAttr()
model.quantize_config.desc_act = self.desc_act
model = gptq_post_init(model, use_act_order=self.desc_act)
if self.desc_act and self.backend == BACKEND.EXLLAMA_V1 and self.max_input_length is not None:
model = exllama_set_max_input_length(model, self.max_input_length)
Copy link
Copy Markdown
Contributor

@Qubitium Qubitium Apr 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jiqing-feng Can you check if anyone stil uses this? exllama_set_max_input_length If not and only v1 specific, this helper method can be removed too.

Copy link
Copy Markdown
Contributor Author

@jiqing-feng jiqing-feng Apr 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked no use of this func, removed. Thanks for the review.


return model

def pack_model(
Expand Down