backport Python3 support - #25
Open
1715173329 wants to merge 38 commits into
Open
Conversation
Pull this support from these upstream commits: bfbfab0 pylibfdt: Add a means to add and delete notes 9005f41 pylibfdt: Allow delprop() to return errors Signed-off-by: Simon Glass <sjg@chromium.org>
At present we require the caller to manually update the device tree using individual calls to libfdt functions. This is not ideal. It would be better if we could make changes using the Python structure and then call a Sync() function to write them back. Add this feature to the Fdt class. Update binman and the tests to match. Signed-off-by: Simon Glass <sjg@chromium.org>
This should be big endian, since that is what device tree uses. Fix it. Signed-off-by: Simon Glass <sjg@chromium.org>
Add a way to add new nodes and sync them back to the blob. Signed-off-by: Simon Glass <sjg@chromium.org>
Add a few more functions which allow creating and modifying property values. If only we could do this so easily in the real world. Signed-off-by: Simon Glass <sjg@chromium.org>
Support creating an Fdt object without having to write the data to a file first. Signed-off-by: Simon Glass <sjg@chromium.org>
This does not set the correct value at present. Fix it. Signed-off-by: Simon Glass <sjg@chromium.org>
Update all print statements to be functions, as required by Python 3. Signed-off-by: Simon Glass <sjg@chromium.org>
Update all print statements to be functions, as required by Python 3. Signed-off-by: Simon Glass <sjg@chromium.org>
Python 3 requires this, and Python 2 allows it. Convert the code over to ensure compatibility with Python 3. Signed-off-by: Simon Glass <sjg@chromium.org>
Fix an open() statement to use binary mode, so that it works as expected with Python 3. Signed-off-by: Simon Glass <sjg@chromium.org>
The method of multiplying a character by a number works well for creating a repeated string in Python 2. But in Python 3 we need to use bytes() instead, to avoid unicode problems, since 'bytes' is no-longer just an alias of 'str'. Create a function to handle this detail and call it from the relevant places in binman. Signed-off-by: Simon Glass <sjg@chromium.org>
Create helper functions in the tools module to deal with the differences between unicode in Python 2 (where we use the 'unicode' type) and Python 3 (where we use the 'str' type). Signed-off-by: Simon Glass <sjg@chromium.org>
Use this helper function which works on both Python 2 and Python 3. Signed-off-by: Simon Glass <sjg@chromium.org>
This method does not actually use any members of the Prop class. Move it out of the class so that it is easier to add unit tests. Signed-off-by: Simon Glass <sjg@chromium.org>
The difference between the bytes and str types in Python 3 requires a number of minor changes to this function. Update it to handle the input data using the 'bytes' type. Create two useful helper functions which can be used by other modules too. Signed-off-by: Simon Glass <sjg@chromium.org>
In Python 3 bytes and str are separate types. Use bytes to ensure that the code functions correctly with Python 3. Signed-off-by: Simon Glass <sjg@chromium.org>
Update this class to work correctly on Python 3 and to pass its unit tests. Signed-off-by: Simon Glass <sjg@chromium.org>
Update this class to work correctly on Python 3 and to pass its unit tests. The only required change is to deal with a difference in the behaviour of sorting with a None value. Signed-off-by: Simon Glass <sjg@chromium.org>
Since we are now using the bytes type in Python 3, the conversion in fdt32_to_cpu() is not necessary, so drop it. Also use 'int' instead of 'long' to convert the integer value, since 'long' is not present in Python 3. With this, test_fdt passes with both Python 2 and 3: PYTHONPATH=/tmp/b/sandbox_spl/scripts/dtc/pylibfdt python \ ./tools/dtoc/test_fdt -t PYTHONPATH=~/cosarm/dtc/pylibfdt:tools/patman python3 \ ./tools/dtoc/test_fdt -t Signed-off-by: Simon Glass <sjg@chromium.org>
Build this swig module with Python 3. Signed-off-by: Simon Glass <sjg@chromium.org>
Sync up the libfdt Python bindings with upstream, commit: 430419c (tests: fix some python warnings) Signed-off-by: Simon Glass <sjg@chromium.org>
This function should use a void * type, not char *. This causes an error: TypeError: in method 'fdt_property_stub', argument 3 of type 'char const *' Fix it. Signed-off-by: Simon Glass <sjg@chromium.org>
Drop the now-unused Python 2 code to keep code coverage at 100%. Signed-off-by: Simon Glass <sjg@chromium.org>
Update this tool to use Python 3 to meet the 2020 deadline. Unfortunately this introduces a test failure due to a problem in pylibfdt on Python 3. I will investigate. Signed-off-by: Simon Glass <sjg@chromium.org>
If the pylibfdt shared-object file is detected, then Python assumes that the libfdt.py file exists also. Sometimes when an incremental build aborts, the shared-object file is built but the libfdt.py is not. The only way out at this point is to use 'make mkproper', or similar. Fix this by removing the .so file before it is built. This seems to make Python rebuild everything. Signed-off-by: Simon Glass <sjg@chromium.org> Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
At present the build rule for pylibfdt depends on _libfdt.so but modern Python versions add a different suffix to the output file, resulting in something like _libfdt.cpython-38-x86_64-linux-gnu.so The result is that pylibfdt is rebuilt every time. Rename the file the standard name so that the rule works correctly. Also add libfdt.py to the dependencies, so that file is always created if missing. Signed-off-by: Simon Glass <sjg@chromium.org>
Some versions of make complain about using a grouped target without a recipe: .../pylibfdt/Makefile:36: *** grouped targets must provide a recipe. Stop. Fix this by adding a dummy recipe. Signed-off-by: Simon Glass <sjg@chromium.org>
python does not like the u-boot- prefix in the version, drop it.
/usr/lib/python3.10/site-packages/setuptools/dist.py:544: UserWarning:
The version specified ('u-boot-2022.10') is an invalid version, this may
not work as expected with newer versions of setuptools, pip, and PyPI.
Please see PEP 440 for more details.
Signed-off-by: Michal Suchanek <msuchanek@suse.de>
Reviewed-by: Simon Glass <sjg@chromium.org>
The distutils package is deprecated. The upstream libfdt repo uses setuptools for building the pylibfdt module, so bring in that code, suitably modified for U-Boot. Also bring in the README. The modifications include setting the version correctly, making use of the environment variables provided by the Makefile and various tweaks to the directories. Note that the version omits the minus character at the start of EXTRAVERSION, since this creates a warning. The build is really just used within U-Boot itself, so it doesn't matter too much if the version matches upstream, or exactly matches U-Boot. Signed-off-by: Simon Glass <sjg@chromium.org>
Python 3.10 requires defining PY_SSIZE_T_CLEAN. This will be fixed in swig 4.10 but it is not clear when it will be released. There was a warning since python 3.8. Link: swig/swig#2277 Signed-off-by: Michal Suchanek <msuchanek@suse.de> Reviewed-by: Simon Glass <sjg@chromium.org>
Fix the following version normalization warning: " /usr/lib/python3/dist-packages/setuptools/dist.py:530: UserWarning: Normalizing '2023.01' to '2023.1' " Using suggestion from Richard Jones: pypa/setuptools#308 (comment) Signed-off-by: Marek Vasut <marex@denx.de> Reviewed-by: Simon Glass <sjg@chromium.org>
On Arch Linux based systems python setuptools does not contain "setuptools.extern" hence it is failing with the following error-message: " ModuleNotFoundError: No module named 'setuptools.extern' " According to a eschwartz `setuptools.extern` is not a public API and shall not be assumed to be present in the setuptools package. He mentions that the setuptools project anyway wants to drop this. [1] Use the correct solution introduced by python setuptools developers to disable normalization. [2] [1] https://bbs.archlinux.org/viewtopic.php?id=259608 [2] pypa/setuptools#2026 Fixes: 440098c42e73 ("pylibfdt: Fix version normalization warning") Signed-off-by: Philippe Schenker <philippe.schenker@toradex.com> Reviewed-by: Marek Vasut <marex@denx.de> Reviewed-by: Simon Glass <sjg@chromium.org>
In some cases, we might not have the sic portion of setuputils
available. Make our import and use of this be done in try/except blocks
as this is done to suppress a run-time warning that is otherwise
non-fatal.
Reported-by: Pali Rohár <pali@kernel.org>
Fixes: 141659187667 ("pylibfdt: Fix disable version normalization")
Signed-off-by: Tom Rini <trini@konsulko.com>
Once u-boot's build system invokes
python3 scripts/dtc/pylibfdt/setup.py --quiet build_ext --inplace
it may fail with
scripts/dtc/pylibfdt/setup.py:40: SyntaxWarning: invalid escape sequence '\w'
RE_KEY_VALUE = re.compile('(?P<key>\w+) *(?P<plus>[+])?= *(?P<value>.*)$')
depending on the used Python version.
Explicitly mark the regex string as raw string to avoid the warning.
Signed-off-by: Florian Schmaus <flo@geekplace.eu>
Reviewed-by: Heinrich Schuchardt <heinrich.schuchardt@canonical.com>
Swig has changed language specific AppendOutput functions. The helper macro SWIG_AppendOutput remains unchanged. Use that instead of SWIG_Python_AppendOutput, which would require an extra parameter since swig 4.3.0. /home/flk/poky/build-test/tmp/work/qemux86_64-poky-linux/u-boot/2024.10/git/arch/x86/cpu/u-boot-64.lds | scripts/dtc/pylibfdt/libfdt_wrap.c: In function ‘_wrap_fdt_next_node’: | scripts/dtc/pylibfdt/libfdt_wrap.c:5581:17: error: too few arguments to function ‘SWIG_Python_AppendOutput’ | 5581 | resultobj = SWIG_Python_AppendOutput(resultobj, val); | | ^~~~~~~~~~~~~~~~~~~~~~~~ Signed-off-by: Markus Volk <f_l_k@t-online.de> Reported-by: Rudi Heitbaum <rudi@heitbaum.com> Link: dgibson/dtc#154
Backport the binary file handling needed by the mt7621 binman path from
upstream commits 6ed45ba0a831 ("binman: Support updating all device tree
files") and 83d73c2f7c47 ("binman: Support compressed entries").
Keep only the file-I/O changes to avoid pulling in unrelated multi-DTB
and compression support.
Signed-off-by: Tianling Shen <i@cnsztl.eu.org>
Signed-off-by: Tianling Shen <i@cnsztl.eu.org>
Author
|
Kindly ping @DragonBluep |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is not a full Python3 backport but it's good enough for mt7621 build.
The binaries have been verified bit-to-bit compatible after migration.