HGQ QMHA and QLinformer support for oneAPI by laurilaatu · Pull Request #1432 · fastmachinelearning/hls4ml

laurilaatu · 2026-02-09T16:59:16Z

Description

This PR adds the required changes to enable QMHA/QLinformer in oneAPI.
Requires parsing softmax tables from HGQ and support for multidim softmax.

Type of change

New feature (non-breaking change which adds functionality)

Tests

Test Configuration:

Checklist

I have read the guidelines for contributing.
I have commented my code, particularly in hard-to-understand areas.
I have made corresponding changes to the documentation.
My changes generate no new warnings.
I have installed and run pre-commit on the files I edited or added.
I have added tests that prove my fix is effective or that my feature works.

…pi_weights

…api_qmha

jmitrevs · 2026-02-09T17:28:16Z

How are the names of tables associated with the layers? Is it now per-instance?

jmitrevs · 2026-02-11T20:25:08Z

            copyfile(srcpath, dstpath)

-    def __get_table_size(self, model, activation):
+    def __get_table_size(self, model, activation, table_name='table_size'):


table_name and table_size have very different meanings. Maybe table_name_size or something like that would read better?

jmitrevs · 2026-02-11T20:27:25Z

I assume this also needs changes in nnet_activation_stream.h, at least to consistently use the types.

jmitrevs · 2026-02-11T20:28:42Z

Tests should be written (or hgq tests enabled) to show that this works as expected.

jmitrevs · 2026-02-12T17:21:03Z

Scratch that about the Libero backend. Apparently it's just cut and paste code that has not been used or tested. The Libero backend doesn't yet support softmax and other table lookup functions.

…i_qmha

Copilot

Pull request overview

This PR aims to enable HGQ-based QMHA/QLinformer support on the oneAPI backend by (1) parsing/using HGQ softmax LUTs and (2) adding infrastructure for multi-dimensional softmax in oneAPI firmware templates.

Changes:

Add a new insertion point in oneAPI parameters.h and writer logic to include per-layer softmax table headers.
Refactor oneAPI softmax stable implementation to use per-layer exp/invert tables via pointers in CONFIG_T (instead of #includeing .tb files inside the function).
Add a multi-dimensional softmax helper in the oneAPI activation header and update oneAPI template generation to emit per-layer table pointers/types/sizes.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
`hls4ml/writer/oneapi_writer.py`	Generates and includes per-softmax LUT headers in the oneAPI output project.
`hls4ml/templates/oneapi/firmware/parameters.h`	Adds a marker for writer-inserted softmax table includes.
`hls4ml/templates/oneapi/firmware/nnet_utils/nnet_activation.h`	Switches stable softmax to use `CONFIG_T` table pointers; adds multidim softmax helper.
`hls4ml/templates/oneapi/firmware/nnet_utils/nnet_activation_stream.h`	Mirrors stable softmax table-pointer usage for streaming implementation.
`hls4ml/backends/oneapi/passes/core_templates.py`	Updates generated softmax config to include exp/inv table sizes and pointers.
`hls4ml/backends/oneapi/oneapi_backend.py`	Removes the previous oneAPI constraint that blocked multidimensional softmax in `io_parallel`.

Comments suppressed due to low confidence (1)

hls4ml/backends/oneapi/oneapi_backend.py:263

Without the Softmax initializer, multidimensional Softmax with IOType=io_parallel is no longer rejected even though oneAPI codegen does not currently switch to the new softmax_multidim kernel. This can produce incorrect outputs (softmax applied across the flattened tensor). Either add the multidim function-call selection in codegen, or re-introduce the initializer guard until that wiring is in place.

    @layer_optimizer(Activation)
    def init_activation(self, layer):
        if layer.get_attr('activation') == 'tanh':
            layer.set_attr('activation', 'dense_tanh')
        if layer.get_attr('recurrent_activation') == 'tanh':
            layer.set_attr('recurrent_activation', 'dense_tanh')

    @layer_optimizer(Embedding)
    def init_embed(self, layer):
        if layer.attributes['n_in'] is None:
            raise Exception('Input length of Embedding layer must be specified.')

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+                    h_file.write(f'#ifndef {header_name.upper()}_H_\n')
+                    h_file.write(f'#define {header_name.upper()}_H_\n\n')
+
+                    h_file.write(f'static constexpr {table_name}_t {table_name}[{table_size}] = {{')


+                    h_file.write(f'#ifndef {header_name.upper()}_H_\n')
+                    h_file.write(f'#define {header_name.upper()}_H_\n\n')
+
+                    h_file.write(f'static constexpr {table_name}_t {table_name}[{table_size}] = {{')


+                elif '// hls-fpga-machine-learning insert softmax tables' in line:
+                    newline = line
+                    for layer in model.get_layers():
+                        if 'softmax' in layer.name:
+                            newline += f'#include "nnet_utils/activation_tables/{layer.name}_exp_table.h"\n'
+                            newline += f'#include "nnet_utils/activation_tables/{layer.name}_inv_table.h"\n'
+


        for layer in model.get_layers():
-            if layer.name == 'softmax':
-                ac_type = layer.get_input_variable().type
-                if ac_type is not None:
-                    try:
-                        fp_bits = ac_type.precision.integer + ac_type.precision.fractional
-                        fp_integer = ac_type.precision.integer
-                        fp_signed = ac_type.precision.signed
-                    except Exception:
-                        # FixedPrecisionType wasn't correctly stored in layer attributes, use default values
-                        pass
-                    if fp_signed is False:
-                        raise Exception('Softmax types need to be signed')
+            if 'softmax' in layer.name:
+                table_name = layer.name + '_exp_table'


        for layer in model.get_layers():
-            if layer.name == 'softmax':
-                ac_type = layer.get_attr('exp_table_t')
-                if ac_type is not None:
-                    try:
-                        fp_bits = ac_type.precision.integer + ac_type.precision.fractional
-                        fp_integer = ac_type.precision.integer
-                        fp_signed = ac_type.precision.signed
-                    except Exception:
-                        # FixedPrecisionType wasn't correctly stored in layer attributes, use default values
-                        pass
-                    if fp_signed is False:
-                        raise Exception('Softmax types need to be signed')
-
-        sep = ''
-        N = ceil_log2(table_size)
-        for i in range(table_size):
-            f = FixedPointEmulator(fp_bits, fp_integer, signed=fp_signed)
-            b = uint_to_binary(i, N)
-            b.insert(0, 0)
-            f.set_msb_bits(b)
-            real_val = f.inv_float()
-            h_file.write(sep + str(real_val))
-            sep = ', '
+            if 'softmax' in layer.name:
+                table_name = layer.name + '_inv_table'
+                table_size = (


 softmax_config_template = """struct {type}_config{index} : nnet::activ_config {{
    static constexpr unsigned n_in = {n_in};
-    static constexpr unsigned table_size = {table_size};
+    static constexpr unsigned exp_table_size = {exp_table_size};
+    static constexpr unsigned inv_table_size = {inv_table_size};
    static constexpr unsigned io_type = nnet::{iotype};
    static constexpr unsigned reuse_factor = {reuse};
    static constexpr nnet::softmax_implementation implementation = nnet::softmax_implementation::{implementation};
    typedef {exp_table_t.name} exp_table_t;
-    typedef {inv_table_t.name} inv_table_t;
+    typedef {inv_table_t.name} inv_table_t;"""


+                params['exp_table_t'].precision.signed = False
+
+            if 'inp_norm_t' not in params:


    EinsumDense,
    Embedding,
    Layer,
    SimpleRNN,
-    Softmax,
 )


laurilaatu and others added 8 commits January 26, 2026 20:37

weights for dense

3d463b3

hgq2 homogeneous quant fix

d678573

Merge branch 'hgq2_homo_quant' of github.com:calad0i/hls4ml into onea…

77258bc

…pi_weights

Changes required for oneAPI MHA

59bd96f

Original weight implementation

dbb207b

Merge branch 'main' of github.com:fastmachinelearning/hls4ml into one…

0c59255

…api_qmha

Restore oneAPI weight placement

51efff0

pre-commit

6067bea

jmitrevs reviewed Feb 9, 2026

View reviewed changes

Comment thread hls4ml/templates/oneapi/firmware/nnet_utils/nnet_activation.h Outdated

jmitrevs added the please test Trigger testing by creating local PR branch label Feb 9, 2026

Merge branch 'main' into oneapi_qmha

06fda4e

jmitrevs reviewed Feb 11, 2026

View reviewed changes

Comment thread hls4ml/converters/keras_v3/hgq2/multi_head_attention.py

jmitrevs added please test Trigger testing by creating local PR branch and removed please test Trigger testing by creating local PR branch labels Feb 11, 2026

jmitrevs reviewed Feb 11, 2026

View reviewed changes

Comment thread hls4ml/templates/oneapi/firmware/nnet_utils/nnet_activation.h Outdated

jmitrevs reviewed Feb 11, 2026

View reviewed changes

Merge branch 'main' into oneapi_qmha

bf38a6b

jmitrevs added please test Trigger testing by creating local PR branch and removed please test Trigger testing by creating local PR branch labels Feb 13, 2026

laurilaatu and others added 7 commits February 16, 2026 11:03

Merge branch 'main' into oneapi_qmha

e27fd11

Merge branch 'main' into oneapi_qmha

9f4a448

softmax multidim templates

16ca197

Merge branch 'oneapi_qmha' of github.com:laurilaatu/hls4ml into oneap…

564b692

…i_qmha

pre-commit

974e75a

uncomment

060c398

Merge branch 'main' into oneapi_qmha

f78558c

laurilaatu and others added 10 commits February 25, 2026 17:16

int_inp_t to config

772b93a

Merge branch 'oneapi_qmha' of github.com:laurilaatu/hls4ml into oneap…

d2b8921

…i_qmha

Merge branch 'main' into oneapi_qmha

a1ad891

Merge branch 'main' into oneapi_qmha

d65544d

Merge branch 'main' into oneapi_qmha

2d6a5cc

softmax fixed

c3a4584

Merge branch 'main' into oneapi_qmha

9b1cf17

table generation cleanup

31b7ad6

Merge pull request #4 from bugracyln/smax_fix

70b19d1

Merge branch 'main' into oneapi_qmha

29bdbb3

jmitrevs mentioned this pull request Jun 9, 2026

Better handle types and overflows in softmax (especially lagacy and stable) #1476

Draft

8 tasks

jmitrevs reviewed Jun 9, 2026

View reviewed changes

Comment thread hls4ml/backends/oneapi/passes/core_templates.py Outdated

laurilaatu and others added 3 commits June 10, 2026 07:59

Fix formatting of inp_norm_t name string

cab4cbc

pre-commit for core templates

42ece34

pre-commit all

7e2798a

jmitrevs added please test Trigger testing by creating local PR branch and removed please test Trigger testing by creating local PR branch labels Jun 10, 2026

jmitrevs requested a review from Copilot June 10, 2026 19:59

Copilot started reviewing on behalf of jmitrevs June 10, 2026 19:59 View session

Copilot AI reviewed Jun 10, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

HGQ QMHA and QLinformer support for oneAPI #1432

HGQ QMHA and QLinformer support for oneAPI #1432
laurilaatu wants to merge 30 commits into
fastmachinelearning:mainfrom
laurilaatu:oneapi_qmha

laurilaatu commented Feb 9, 2026

Uh oh!

Uh oh!

jmitrevs commented Feb 9, 2026

Uh oh!

Uh oh!

Uh oh!

jmitrevs Feb 11, 2026

Uh oh!

jmitrevs commented Feb 11, 2026

Uh oh!

jmitrevs commented Feb 11, 2026

Uh oh!

jmitrevs commented Feb 12, 2026 •

edited

Loading

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

		params['exp_table_t'].precision.signed = False

		if 'inp_norm_t' not in params:

Uh oh!

Conversation

laurilaatu commented Feb 9, 2026

Description

Type of change

Tests

Checklist

Uh oh!

Uh oh!

jmitrevs commented Feb 9, 2026

Uh oh!

Uh oh!

Uh oh!

jmitrevs Feb 11, 2026

Choose a reason for hiding this comment

Uh oh!

jmitrevs commented Feb 11, 2026

Uh oh!

jmitrevs commented Feb 11, 2026

Uh oh!

jmitrevs commented Feb 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

jmitrevs commented Feb 12, 2026 •

edited

Loading