4368 Commits

Author SHA1 Message Date
Simon Glass
60b8709b98 patman: Allow setting the current directory when sending
Plumb a current-working-directory (cwd) through from send all the way to
the command gitutil libraries. This will allow better testing of this
functionality, since we can use a test directory.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:42 +01:00
Simon Glass
4f7bd6cae8 patman: Add all files to __init__.py
Some files are missing from the __all__ list, so add then. Reformat the
list to use more of the width of each line.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:42 +01:00
Simon Glass
afea95f653 u_boot_pylib: Speed up determining the upstream branch
Use --decorate to quickly detect the upstream branch, since this is much
faster than using 'git name-rev' on every possible commit.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:42 +01:00
Simon Glass
c885917c71 u_boot_pylib: Provide directories to gitutil functions
For testing it is useful to be able to set the current directory used
for git operations, as well as the git-repo directory. Update some of
the functions to support this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:42 +01:00
Simon Glass
0664a956e6 u_boot_pylib: Tidy up quoting of cc and to
The current approach to calling 'git send-email' puts double quotes
around each email address to ensure that it will pass the shell
correctly. This is a bit cumbersome and requires using a shell to sort
it all out.

Drop the quotes and use command.run() instead, to simplify things. This
will also make it possible to (later) set the current directory.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:42 +01:00
Simon Glass
db5d98de55 u_boot_pylib: Support a fatal level in tout
It is convenient to be able to print a message and exit. Add a new
'fatal' level to support this.

Update some assumptions about the level, so that the tools continue to
work as now.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:42 +01:00
Simon Glass
75ae217194 patman: Update Series.ShowActions() to pass alias
Instead of using settings.alias pass this value in. This allows tests to
work without using settings.alias

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:42 +01:00
Simon Glass
abfd67cccf patman: Pass aliases to Series.MakeCcFile()
Rather than accessing settings directly, pass the aliases in, so that
we can do the same from tests. With further work this will allow the
tests to work without using settings.alias

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
e10201aa8c patman: Pass the alias dict into gitutil.email_patches()
Rather than accessing the settings module in this function, require the
alias dict to be passed in.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
7dc55435b2 patman: Pass the alias dict into gitutil.build_email_list()
Rather than accessing the settings module in this function, require the
alias dict to be passed in.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
e70fdbd7e4 patman: Untangle settings from gitutil
The gitutil module is supposed to be independent from patman but one
piece was missed in the series which separated them.

Move the settings setup out of gitutil

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
f9b84f9141 patman: Drop the old status code
Now that we have the new asyncio implementation, drop the unused
ThreadPoolExecutor code.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
605717405f patman: Use a pager when outputting status
The pager makes it easier to review large number of comments. Add it
around the output.

Patman normally sends multiple concurrent requests to the patchwork
server, as this is faster. Provide an option to disable this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
52aef33f95 patman: Provide an option to run in single-threaded mode
Patman normally sends multiple concurrent requests to the patchwork
server, as this is faster. Provide an option to disable this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
45f4f62191 patman: Switch over to asyncio
Now that all the pieces are in place, switch over from using an executor
to using asyncio.

While we are here, import defaultdict to avoid needing to specify its
module.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
39f258494f patman: Use defaultdict directly
Import defaultdict to avoid needing to specify its module in multiple
places. Fix an extra blank line while here.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
0fb0b46200 patman: Add more information to Patch
The cover letter has some information on each patch, so allow this to be
stored in the Patch object.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
7052173fae patman: Show patches in yellow
When comments are shown below patches it can be hard to see the patch
subject. Use yellow instead of blue, since it stands out better.

Pass the colour object into show_responses() rather than creating a new
one, since that is tidier.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
289f4242c5 patman: Simplify test_find_new_responses()
This test uses the find_new_responses() function which combines
accessing patchwork and processing the results.

Since the test is aimed at testing processing, adjust it to call
process_reviews() instead.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
3fd99e2177 patman: Adjust how the fake request() function is provided
Instead of passing the URL and function to each call, put the fake
into the Patchwork object instead.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
3c5f27ce12 patman: Split out the processing of find_new_responses()
This function does patchwork calls and then processes the results. Move
the processing out so we can (later) test it separately.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
d63b075062 patman: Split up check_and_show_status()
This function has three phases:

- collecting things from patchwork
- doing some processing
- showing the results to the user / creating a branch

Refactor into two functions so we can eventually have the patchwork part
fully separated out.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
e2a991398d patman: Rename check_patchwork_status()
This function actually shows the status and does some other things.

Rename it to better reflect its purpose.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
d65490650f patman: Add reading of series and patch status
Expand the patchwork module so that it can match the current
requirements of the 'patman status' command, i.e. reading the state of a
series and the patches associated with it.

Since the format of each patchwork response is a little tricky to
understand, add examples in comments at the top of each function.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
24776ab276 patman: Move Patch and Review to patchwork module
These relate to information obtained from the patchwork server, so move
their definition into the new patchwork module.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
aef44950db patman: Create a module for handling patchwork
At present the patchwork implementation is very simple, just consisting
of a function which calls the REST API.

We want to create a fake patchwork for use in tests. So start a new
module to encapsulate communication with the patchwork server.

Use asyncio since it is easier to handle lots of concurrent requests
from different parts of the code.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
31d458b35d CI: Add a dependency file for patman
Now that patman has an unusual dependency, add a requirements.txt file
and use it in CI

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
770602195c patman: Move code for sending into its own module
The control module includes much of the implementation of patman's
'send' feature. As a first step to separating this out, move the sending
code into its own file.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
eca4653835 patman: Support extra test features
Provide support for the -X flag, which preserves the working directory
used by tests. Also support -N which shows captured output for tests.

Finally, allow selection of a particular test to run.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
6e64ae22c2 patman: Tidy up the start-up code
Much of this was written before the other Python tools and they have
evolved. Make a few updates:

- Rather than calling sys.exit(), return the exit code from the control
  module and use it in __main__
- Set up tout as it is used in some places
- We now have quite a few tests, so update the comment about that

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
f5ec4f05f9 patman: Rename test_basic() in test_checkpatch
This currently has the same name as a test in func_test.py so it isn't
possible to select one or the other with 'patman test test_basic': both
are executed.

Rename this one to avoid confusion.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
5053549532 patman: Correct a stray quote
A quote character was added a few years ago, but was not intended. Drop
it.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
a61635d2af tools: Plumb in capture control
Add control of capturing output into u_boot_pylib and the tools which
use it.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
9dee86cf53 u_boot_pylib: Allow disabling colour output
When running tests there is no situation in which we want ANSI output as
it makes it much harder to see what is going on in logs, tests, etc.

Provide a way to disable this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
55342144af u_boot_pylib: Avoid concurrent execution of only one test
There is no point in spinning up multiple processes if there is only one
test to execute. Add a check for this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
1f18e8798f u_boot_pylib: Add a pager
It is useful to have a pager when outputting a lot of text. Add support
for this in the terminal library, making use of a context manager.

Also add a function to indicate whether the output device is a terminal
or not, while we are here, to avoid duplicating this code.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
f74c6f9a81 u_boot_pylib: Improve ANSI-colour output with backgrounds
The current implementation does not handle background colours very well:

- It outputs an incorrect code in some cases, leading to wrong colours
- Some functions lack a control for the background

Tidy this up so that background colours can be used in more places.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
b697480f19 u_boot_pylib: Allow control of capturing
Tests often capture output so they can check it. This means that if the
test fails it is not easy to see what the output actually was.

Add a -N flag which writes out the output after it has been captured.
This is not a perfect solution but it is simple and seems to work well
in practice.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
84f3deaa99 binman: Correct a typo in the toolpath comment
Fix the 'ist' typo in the comment for TestFunctional

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
ae3695f691 patman: Move capture_sys_output() into terminal and rename
This function is sometimes useful outside tests. Also it can affect how
terminal output is done, e.g. whether ANSI characters should be emitted
or not.

Move it out of the test_util package and into terminal.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
6330f94a35 patman: Move all non-test logic into control module
It is easier for tests if the top-level control logic is all in one
module. Create a new do_patman() function to handle this. Move the
existing code into it.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
92d5d1e1cd binman: x86: Write skip-at-start when end-at-4gb is used
The end-at-4gb property implies a value for skip-at-start so add it into
the output FDT so that U-Boot can read it.

Now that skip-at-start is implemented, we can drop the workarounds used
in the x86 code to obtain the correct image-pos value.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:39 +01:00
Simon Glass
c3d3534806 buildman: Add a way to build a particular target
At present buildman only supports building the default target. Generally
this is what is wanted, but in some cases boards erroneously have a
different target for product extra files.

Add a --target option to help. Also add a comment indicating which
letters are free for new options.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 09:57:06 +01:00
Heinrich Schuchardt
285d265f7d Dockerfile: use lz4 instead of lz4-tools
Since Ubuntu Jammy lz4-tools is only a virtual package which pulls in
lz4 as dependency.

Update documentation too.

Signed-off-by: Heinrich Schuchardt <heinrich.schuchardt@canonical.com>
Reviewed-by: Tom Rini <trini@konsulko.com>
2025-05-09 15:08:09 -06:00
Alice Guo
5f28a6599f tools: imx8image: add i.MX95 support
i.MX95 uses binman to invoke mkimage to create image container. 2 image
containers are needed currently. The first one is composed of
ahab-container.img, LPDDR firmware images, OEI images, System Manager
image and u-boot-spl.bin. The second one is consisted of ARM Trusted
firmware and u-boot.bin.

Because DDR OEI image and LPDDR firmware images have to be packaged
together and named as m33-oei-ddrfw.bin by binman, so imx9_image.sh does
not check if m33-oei-ddrfw.bin exists.

When using "make imx95_19x19_evk_defconfig; make", imx9_image.sh will
delete the line for u-boot.bin in container.cfg. In fact, binman is
always called after the u-boot.bin is built, so imx9_image.sh does not
check if u-boot.bin exists.

Signed-off-by: Alice Guo <alice.guo@nxp.com>
2025-05-03 16:55:32 -03:00
Alice Guo
0318c26c2f binman: add a new entry type for packing DDR PHY firmware images
i.MX95 needs to combine DDR PHY firmware images and their byte counts
together, so add a new entry type nxp-header-ddrfw for this requirement.

Signed-off-by: Alice Guo <alice.guo@nxp.com>
2025-05-03 16:55:32 -03:00
Tom Rini
d75998b476 Docker, CI: Add vexpress_fvp / vexpress_fvp_bloblist support
This adds the vexpress_fvp and vexpress_fvp_bloblist platforms to the
list of platforms we test via emulator in CI. In order to do this we
need to first have our container runtime have TF-A builds for the
vexpress_fvp platform, both with and without transfer list support as
well as installing "telnet" so that we can access console. In the CI
files we check for the existence of /opt/tf-a/${TEST_PY_BD} and if
found, copy bl1.bin and fip.bin to /tmp and set the variables so that we
can later run FVP to run.

Note that we currently disable the hostfs (semihosting) tests as they
trigger a bug in FVP. This has been reported upstream, and can be
enabled when fixed.

Reviewed-by: Harrison Mutai <harrison.mutai@arm.com>
Acked-by: Ilias Apalodimas <ilias.apalodimas@linaro.org>
Signed-off-by: Tom Rini <trini@konsulko.com>
2025-04-29 11:40:40 -06:00
Carlos López
2c529839a9 mkimage: fix option parsing segfault
getopt_long() expects a NULL-terminated list of structures. The current
list in mkimage does not have a zero-filled structure at the end, which
can cause getopt_long() to walk past the end of the array when passing
an unknown option, causing a segmentation fault.

As a reproducer, the following command causes a segmentation fault
(tested in Debian 12):

    mkimage --foobar

Signed-off-by: Carlos López <carlos.lopezr4096@gmail.com>
2025-04-28 13:25:59 -06:00
Tom Rini
efd00b0345 python: Use and refer to the venv module rather than virtualenv
Using some form of sandbox with Python modules is a long standing best
practice with the language. There are a number of ways to have a Python
sandbox be created. At this point in time, it seems the Python community
is moving towards using the "venv" module provided with Python rather
than a separate tool. To match that we make the following changes:

- Refer to a "Python sandbox" rather than virtualenv in comments, etc.
- Install the python3-venv module in our container and not virtualenv.
- In our CI files, invoke "python -m venv" rather than "virtualenv".
- In documentation, tell users to install python3-venv and not
  virtualenv.

Signed-off-by: Tom Rini <trini@konsulko.com>
2025-04-24 15:37:27 -06:00
Xuhui Lin
ec8c6fb0a6 rockchip: mkimage: Add rk3576 support
Add support for rk3576 package header in mkimage tool.

Signed-off-by: Heiko Stuebner <heiko@sntech.de>
Reviewed-by: Jonas Karlman <jonas@kwiboo.se>
Reviewed-by: Kever Yang <kever.yang@rock-chips.com>
2025-04-23 22:12:05 +08:00