-
Notifications
You must be signed in to change notification settings - Fork 567
XLS backend #1475
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
vasdommes
wants to merge
118
commits into
fastmachinelearning:main
Choose a base branch
from
vasdommes:xls_backend
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
XLS backend #1475
Changes from 110 commits
Commits
Show all changes
118 commits
Select commit
Hold shift + click to select a range
f88faa0
init: xls backend (not working), implemented xls specific layer infor…
Girjoaba 4496252
test
Girjoaba d8b2415
feat: loading weights and creating infrastructure added to writer
Girjoaba fbe2e82
feat: init writer complete
Girjoaba f9b2863
feat: first end2end working test
Girjoaba 0952f1b
fix: vector input support, change back to current directory
Girjoaba b32405b
refactoring: predict function call
Girjoaba 6d91666
feat: solo relu activation test pass
Girjoaba 86dd94a
debt cleanup: split dslx templates in multiple files
Girjoaba 0b9ad57
refactoring: simplified writer -> attribute factory written as an opt…
Girjoaba a78bd1b
feat: softmax xls implementation of table lookup
Girjoaba b24e581
integrated strategies for the softmax implementation
Girjoaba 039c514
bugfix: softmax latency implementation
Girjoaba dc8f5a9
cleanup: removed junk file
Girjoaba 248c0f0
feat: stable softmax 1 specific precision working
Girjoaba 4ff3f94
cleanup: removed junk
Girjoaba 9a73968
feat: integrated stable softmax with all layers
Girjoaba 5684d26
feat: softmax stable and argmax working any bit precision combination
Girjoaba ff02a02
feat: xls utilization report parsing with vivado
Girjoaba c6a9ccd
wip: cnn
Girjoaba 81af6b6
feat: prepared writer weights for CNNs
Girjoaba 5c42f5c
feat: conv2d_latency is now code generated
Girjoaba 092a809
reverted look_up tables
Girjoaba 90c3231
feat: timing report when building the project
Girjoaba a7fbae3
Fix input type for top function in DSLX
vasdommes 9e7afe0
Add fix_softmax_table_size to XLS optimization passes
vasdommes 912e6cc
Merge remote-tracking branch 'refs/remotes/origin/main' into xls_backend
vasdommes a5cccbe
Fix after merge: remove get_shape() and dim names from XLS.
vasdommes b689c7b
Fix test_softmax.py: replace 'strategy' with 'implementation'
vasdommes 75d5d40
XLS: fix softmax_latency parametric types, refactor func_call generation
vasdommes c1f0bea
XLS: major refactoring, implement rounding and overflow modes.
vasdommes 7c16b59
Support multidimensional input/output.
vasdommes ebbb162
Fix XLSFixedPointType.precision()
vasdommes b820617
XLSBackend.predict(): execute IR code via libxls.so calls instead of …
vasdommes 4afbb9f
Get rid of fxpmath, use xls_types.float_to_significand() instead
vasdommes b2fc8a5
XLS: add SkipSoftmax pass
vasdommes a52e3d5
Fix weights and bias precision for XLS
vasdommes fd9c4ce
Fix XLS conv2d, support strides, padding and data_format.
vasdommes d8ad048
Merge remote-tracking branch 'refs/remotes/origin/main' into xls_backend
vasdommes 52da2ee
Revert unrelated cosmetic changes
vasdommes 127856d
xls_writer: Remove obsolete code for weights
vasdommes 3301bba
XLS: make top-level function accept bits instead of FixedPoint.
vasdommes 92b578c
Append layer name to DSLX module name, e.g.: layer_4_softmax.
vasdommes 8fce6ba
Use pyxls to convert DSLX -> IR and optimize IR.
vasdommes 63918f5
replace open(Path).read() with Path.read_text()
vasdommes 0d8320b
XLS: add activation functions, rewrite lookup table generation.
vasdommes 4624c7f
Fix XLS conv2d for channels_first case
vasdommes ddd187c
XLS: cosmetic changes for dense layer
vasdommes 1646cb8
XLS: add Conv1D layer
vasdommes 16081c5
XLS: build lookup tables for antisymmetric functions (softsign and ta…
vasdommes c63e39a
XLS: fix overflow in fixed_point_util::resize()
vasdommes 4fd978f
remove commented code
vasdommes 2742ece
XLS: Use pyxls API for codegen stage, add config['XLSCodegenFlags'].
vasdommes 7b7c0b0
Remove unused arguments from XLSBackend.create_initial_config(), impl…
vasdommes bc044be
XLS: fix step size underflow in build_tables()
vasdommes f4e82aa
test_softmax: reduce input data range for implementation=latency
vasdommes 987961e
test_activations.py: increase softsign table size for XLS backend to …
vasdommes 9b4dc12
Add xls-python to pyproject.toml dependencies, remove dslx_stdlib_pat…
vasdommes da7e71d
XLS: implement Pooling layers
vasdommes c4098c8
XLS: fix padding in conv1d/conv2d
vasdommes 70c7b7b
test_activations, test_keras_api, test_softmax.py: enable XLS only fo…
vasdommes 7f42695
Add XLS to test_keras_v3_api.py
vasdommes 355e2fb
XLS templates refactoring: extract DataFormat from conv and pooling l…
vasdommes 6e0af55
Delete conda env file added by mistake
vasdommes 0c629ed
XLS: Implement DepthwiseConv1D and DepthwiseConv2D
vasdommes c844823
XLS: fix WARNING: Optimizer "{opt}" is not part of any flow and will …
vasdommes 00cda40
XLS: fix vivado report generation and parsing
vasdommes b64a009
XLS: implement Transpose layer
vasdommes 58730da
XLS: Implement Reshape layer
vasdommes 8e05f9c
Add XLS to test_pytorch_api.py. TODO: many tests still fail.
vasdommes 2ec9ef3
Remove custom XLSBackend.predict(), patch XLSBackend.get_top_function…
vasdommes 17e0a7e
XLS: Allow custom weights and bias precision in dense layer
vasdommes 19f5416
XLS: fix softmax table precision
vasdommes 8551cdf
XLS: support multiple layer inputs and outputs
vasdommes 5351699
XLS: implement Merge layer (Add, Average, Maximum, Minimum, Multiply,…
vasdommes 3fd9243
XLS: implement Dot layer
vasdommes 460039d
XLS: implement Concatenate layer (1d, 2d, 3d)
vasdommes eb41b4b
XLS: fixed thresholded_relu/relu compilation error when output width/…
vasdommes 8f14f5f
XLS: implement BatchNormalization layer.
vasdommes 1af1021
Add more XLS tests
vasdommes 74fa4a3
Disable XLS in test_pytorch_api.py/test_squeeze, add TODO about shape…
vasdommes a1a13cd
XLS: fix binary_tanh and ternary_tanh, allow different threshold prec…
vasdommes 5743912
XLS: convert XnorPrecisionType and ExponentPrecisionType to FixedPrec…
vasdommes d4cf878
XLS: support multiple layer inputs and outputs
vasdommes 0e57ca3
XLS: custom weights and bias name
vasdommes 92de294
XLS: fix XnorPrecisionType handling.
vasdommes 2b9f104
XLS BuildAttr.transform(): use raise ... from to show original except…
vasdommes 22861fc
XLSBackend.predict(): fix output conversion for bit_count > 64
vasdommes 7ead10e
Do not throw ModuleNotFoundError for xls when XLSBackend is not actua…
vasdommes 103bc18
XLSBackend: get rid of os.chdir() calls
vasdommes 91156f6
XLSBackend: save top function to model._xls_top_function, clear on co…
vasdommes 479032e
Merge branch 'refs/heads/main' into xls_backend
vasdommes 137c988
Revert unrelated cosmetic changes
vasdommes 8c6ecf8
Add .idea to .gitignore
vasdommes efe2aca
XLS: fix description in build_prj.tcl
vasdommes 08a32bf
Add XLS to docs
vasdommes 2425a5f
pre-commit
vasdommes 878df24
Update docs/ir/attributes.rst
vasdommes e60a23d
setup.rst: add example pip install hls4ml[xls]
vasdommes 4c41b3a
Add xls to EXTRA_DEPS in pytest/ci-template.yml
vasdommes c94b146
cosmetics: inline variable
vasdommes fbe97e0
XLS docs: add xls_codegen_flags example
vasdommes 12b32a7
pre-commit
vasdommes 8578a62
Reduce XLS dimensions in test_pointwiseconv.py and test_pooling.py to…
vasdommes 6fff642
Fix test_softmax.py: skip argmax implementation for Catapult backend
vasdommes 75a04fb
Reduce XLS dimensions in various tests to make them faster.
vasdommes d340575
XLS: Fix clock_period units for Vivado report (ps -> ns)
vasdommes 102b50f
Add XLS to test_softmax_skipped
vasdommes dad0fa6
XLS report: throw error if .sv file not found
vasdommes 65af749
xls.rst: add blank line before code-block, fix typo
vasdommes 8b31be7
Remove xls_backend.import_xls(), use @requires('xls') instead.
vasdommes 27ad38a
XLS: remove unused make_stamp pass
vasdommes 086d6e8
XLS: cache JIT function instead of Python top function
vasdommes ed74aff
XLSBackend.build(): generate only (System)Verilog code, do not run Vi…
vasdommes 1c3eecb
XLS: assert that XNOR weights are 0 or 1
vasdommes 995edfe
xls_types: remove unused u32() and s32() methods for XLSInteger.
vasdommes 8e28449
XLS: use correct rounding and saturation modes when converting from f…
vasdommes b1aa5d0
Merge branch 'main' into xls_backend
vasdommes File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -7,6 +7,7 @@ sdist/ | |
| *.egg-info/ | ||
| vivado_prj | ||
| .vscode | ||
| .idea | ||
| my-hls-test | ||
| *.tar.gz | ||
| docs/_build | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,49 @@ | ||
| ============ | ||
| XLS | ||
| ============ | ||
|
|
||
| The XLS backend can convert hls4ml models into SystemVerilog via `Google XLS <https://google.github.io/xls/>`_, which can be converted to IP via **Vivado**. | ||
|
|
||
| To enable XLS: | ||
|
|
||
| .. code-block:: bash | ||
|
|
||
| pip install hls4ml[xls] | ||
|
|
||
| hls4ml uses `pyxls <https://calad0i.github.io/pyxls/>`_ package to access XLS API. | ||
| pyxls comes with batteries included, and a separate XLS installation is not required. | ||
|
|
||
| Workflow | ||
| ========================= | ||
|
|
||
| XLS backend performs the following transformations: | ||
|
|
||
| .. code-block:: | ||
|
|
||
| hls4ml representation -> DSLX (<ProjectName>.x) -> XLS IR (<ProjectName>.ir) -> Optimized XLS IR (<ProjectName>.opt.ir) -> SystemVerilog (<ProjectName>.sv) -> IP | ||
|
|
||
| `DSLX <https://google.github.io/xls/dslx_reference/>`_ is a DSL with Rust-like syntax. | ||
| DSLX project generated by hls4ml in ``<OutputDir>/firmware`` contains the main module ``<ProjectName>.x``, layer modules ``layer_<LayerName>.x``, and helper modules in ``ap_types/`` and ``nnet_utils/``. | ||
| You may work with this project either through hls4ml or using your own XLS toolchain. | ||
|
|
||
| hls4ml calls XLS compiler to convert DSLX into `XLS IR <https://google.github.io/xls/ir_overview/>`_ format (``<ProjectName>.ir``) and then runs IR optimization passes (``<ProjectName>.opt.ir``). | ||
|
|
||
| Then, hls4ml uses `XLS Codegen <https://google.github.io/xls/codegen_options/>`_ to generate SystemVerilog (``<ProjectName>.sv``) from IR, and calls **Vivado** for ``<OutputDir>/build_prj.tcl`` to generate IP. | ||
|
|
||
| You can override default codegen options: | ||
| .. code-block:: python | ||
|
|
||
| config = hls4ml.utils.config_from_keras_model(model) | ||
| # This sets hls_model.config['XLSCodegenFlags'] | ||
| hls_model = hls4ml.converters.convert_from_keras_model( | ||
| model, hls_config=config, backend='XLS', | ||
| xls_codegen_flags={'delay_model': 'asap7', 'generator': 'pipeline', 'use_system_verilog': False} | ||
| ) | ||
|
|
||
| I/O Types and Strategy | ||
| ========================= | ||
|
|
||
| Currently, only ``io_parallel`` is supported. ``Strategy`` is ignored. | ||
| All operations are fully unrolled. | ||
|
|
||
| XLS supports only signed ``FixedPoint`` type (similar to ``ap_fixed``). | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -39,6 +39,7 @@ | |
| backend/catapult | ||
| backend/quartus | ||
| backend/sr | ||
| backend/xls | ||
|
|
||
| .. toctree:: | ||
| :hidden: | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Empty file.
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.