abc/abc_new: cache parsed liberty files as .scl to avoid re-parsing across invocations#5813
Conversation
|
I was wondering if SCL could be accepted by Yosys in place of Liberty so that Liberty is parsed only once across both programs, but unfortunately since abc-written SCL skips sequential cells, Yosys couldn't map flip flops with |
|
This is excellent! Thank you for working on this feature.
This is not a current use case of ours so optimizing for it should not be necessary. |
Goubermouche
changed the title
github-merge-queue
Bot
removed this pull request from the merge queue due to failed status checks
| * Return: Path to merged SCL cache file, or empty string if conversion fails | ||
| */ | ||
| #if defined(YOSYS_LINK_ABC) | ||
| inline std::string convert_liberty_files_to_merged_scl(const std::vector<std::string> &, const std::string &, const std::string &) { return ""; } |
There was a problem hiding this comment.
Wait, is it impossible to make this function work with YOSYS_LINK_ABC as well? It's a legitimate option on other platforms, too
There was a problem hiding this comment.
Ah, my thinking was that this is only related to WASI, where this optimization wouldn't really be needed and this would be a quick fix. Will fix using Abc_RealMain. Thanks for the headsup.
| if (last_slash == std::string::npos) { | ||
| first_dir = "."; | ||
| } else { | ||
| first_dir = liberty_files[0].substr(0, last_slash); |
There was a problem hiding this comment.
In most of our flows, the liberty file location and the CWD are non-writeable as you're often inside a container, with some other dir for allocated scratch space. Would it be possible to store this file in the same (or similar) abc tmpdir that's created? This could be done in a follow-up change too to land this one.
https://github.com/YosysHQ/yosys/blob/main/kernel/io.cc#L149
Goubermouche
force-pushed
the
nella/fast-liberty-file
branch
from
2102412 to
d795a4f
Compare
4 participants
When
abcis invoked with-liberty, the liberty file paths are embedded into the generated ABC script asread_libcommands. ABC then re-parses every liberty file from scratch on each invocation. In hierarchical flows whereabcruns once per module, this means the same liberty files are parsed N times for N modules. For large libraries this can drastically slow down the entire flow.This PR aims to fix this issue by adding a cache for the liberty files. The cache is implemented as a merged
sclfile, which is generated on the firstabcinvocation, subsequent runs use the mergedscl. From my research,sclshould preserve everything ABC uses for tech mapping, while stripping the majority of the unneeded data, drastically reducing file size, and being much faster to process.In order to test perf I've ran the following benchmark:
abc -libertyinvoked on a couple simple modules with a varying amount of liberty files, resulting in a ~6x speedup. Module contents are intentionally trivial because the bottleneck being addressed is parse time, not mapping time. See results below:I've also tested an implementation which modified ABC so that it's capable of merging
sclfiles viaread_scl, the idea behind this being that we could cache separate files, not the entire list, which could prove beneficial when a flow invokesabcwith different sets of liberty files, but the speedup went down to about ~4x. I believe that these flows are less common, so I chose to go with the above method.TODO: