494 Commits

Author SHA1 Message Date
Tom Rini
8f16767dcc test/py, buildman: Update filelock package version
The GitHub dependabot tool has reported a "medium" priority bug
CVE-2026-22701, with this package. Update to the patched version.

Reported-by: GitHub dependabot
Signed-off-by: Tom Rini <trini@konsulko.com>
2026-01-19 13:08:19 -06:00
Tom Rini
d24f4ae654 test/py, buildman: Update filelock package version
The GitHub dependabot tool has reported a "medium" priority bug
CVE-2025-68146, with this package. Update to the patched version.

Reported-by: GitHub dependabot
Signed-off-by: Tom Rini <trini@konsulko.com>
2025-12-18 10:28:10 -06:00
Heinrich Schuchardt
ff80e95fed tools: use setuptools 78.1.1
CVE-2025-47273 describes a path traversal vulnerability.

Signed-off-by: Heinrich Schuchardt <heinrich.schuchardt@canonical.com>
Reviewed-by: Tom Rini <trini@konsulko.com>
2025-12-10 09:28:43 -06:00
Adriano Carvalho
1041ce48a9 doc: Capitalize the word Buildman whenever it's used as a proper noun
This consistency reads a bit nicer.

Signed-off-by: Adriano Carvalho <adrianocarvalho.pt@gmail.com>
2025-08-30 20:45:33 +02:00
Adriano Carvalho
0492657c74 doc: Rephrase to be more precise and less confusing (build)
It was "... doing the same build ... will not trigger a rebuild".

Signed-off-by: Adriano Carvalho <adrianocarvalho.pt@gmail.com>
2025-08-30 20:45:33 +02:00
Adriano Carvalho
01bf38a05e doc: Rephrase to read a bit nicer
Reads better.

Signed-off-by: Adriano Carvalho <adrianocarvalho.pt@gmail.com>
2025-08-30 20:45:33 +02:00
Adriano Carvalho
ac0e3d2541 doc: Rephrase to be more clear
It might not be clear what is meant with "to make sure the shell leaves it alone".

Signed-off-by: Adriano Carvalho <adrianocarvalho.pt@gmail.com>
2025-08-30 20:45:33 +02:00
Adriano Carvalho
77c72bcb0d doc: Rephrase in a simpler way
It reads a bit nicer.

Signed-off-by: Adriano Carvalho <adrianocarvalho.pt@gmail.com>
2025-08-30 20:45:33 +02:00
Adriano Carvalho
b14d24af40 doc: Add riscv and unfold the list with the architecture/code name
riscv was missing from the list.
To some, the architecture's name may not be obvious from the code name.

Signed-off-by: Adriano Carvalho <adrianocarvalho.pt@gmail.com>
2025-08-30 20:45:33 +02:00
Adriano Carvalho
76733999b0 doc: Use "supports" instead of "has"
Strictly speaking, "has" doesn't make sense.
"supports" seems like a better word and it probably was what the original author meant.

Signed-off-by: Adriano Carvalho <adrianocarvalho.pt@gmail.com>
2025-08-30 20:45:33 +02:00
Adriano Carvalho
45d32d2911 doc: Quote all long form options using double backticks/grave accents
Otherwise, the two dashes are rendered as just one.

Signed-off-by: Adriano Carvalho <adrianocarvalho.pt@gmail.com>
2025-08-30 20:45:33 +02:00
Adriano Carvalho
6a006c56ee doc: Fix obvious typos and minor improvements
These are fixes to what looks like obvious typos.
Some minor improvments are also included, such as:
- Write "symbolic link" instead of symlink
- Correct capitalization for LLVM (all caps)
- Remove dead link and surrounding sentence

Signed-off-by: Adriano Carvalho <adrianocarvalho.pt@gmail.com>
2025-08-30 20:45:33 +02:00
Tom Rini
9be6c5f967 Merge patch series "kbuild: Update Makefile.extrawarn to 5.1"
This series from Ilias Apalodimas <ilias.apalodimas@linaro.org> largely
finishes the re-sync with the Linux Kernel v5.1 kbuild system.

Link: https://lore.kernel.org/r/20250627185723.342553-1-ilias.apalodimas@linaro.org
2025-07-08 13:10:12 -06:00
Ilias Apalodimas
5f520875bd kbuild: Bump the build system to 5.1
Our last sync with the kernel was 5.1. Even that was a partial one
as some patches from 4.x kernels were already missing making the
transition to a modern kbuild infeasible.

We are so out of sync now, that tracking the patches and backporting
them one by one makes little sense and it's going to take ages.

This is an attempt to sync up Makefile[.lib/.kbuild].
Unfortunately due to sheer amount of patches this is not easy to review,
but that's what we decided during a community call.

One of the biggest changes is get rid of partial linking entirely and
build .a archives isntead of .o.
We diaviate from the kernel on that. Instead of calling a custom script
to create the archive symbol table, we call ar with rcTP (isntead of
rcSTP) since we want a resulting archive that's sauble with the linker.

The only affected platforms are PPC ones. Unfortunately I don't have any
of them around to test, but the objdump of the resulting files --
arch/powerpc/lib/built-in.[oa] looks identical.

Signed-off-by: Ilias Apalodimas <ilias.apalodimas@linaro.org>
2025-07-08 13:10:03 -06:00
Simon Glass
919ba46d72 buildman: Support an in-tree build in the current dir
Allow -w to be used with -i to do a build without a separate output
directory.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-07-01 10:49:32 -06:00
Simon Glass
9002ab0986 buildman: Correct behaviour of --in-tree
This option doesn't work as expected since it sets the cwd to the work
directory, which does not necessarily hold the source code.

It should be left unset, so that the current directory is the source
directory.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-07-01 10:49:32 -06:00
Simon Glass
a61635d2af tools: Plumb in capture control
Add control of capturing output into u_boot_pylib and the tools which
use it.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
ae3695f691 patman: Move capture_sys_output() into terminal and rename
This function is sometimes useful outside tests. Also it can affect how
terminal output is done, e.g. whether ANSI characters should be emitted
or not.

Move it out of the test_util package and into terminal.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
c3d3534806 buildman: Add a way to build a particular target
At present buildman only supports building the default target. Generally
this is what is wanted, but in some cases boards erroneously have a
different target for product extra files.

Add a --target option to help. Also add a comment indicating which
letters are free for new options.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 09:57:06 +01:00
Tom Rini
efd00b0345 python: Use and refer to the venv module rather than virtualenv
Using some form of sandbox with Python modules is a long standing best
practice with the language. There are a number of ways to have a Python
sandbox be created. At this point in time, it seems the Python community
is moving towards using the "venv" module provided with Python rather
than a separate tool. To match that we make the following changes:

- Refer to a "Python sandbox" rather than virtualenv in comments, etc.
- Install the python3-venv module in our container and not virtualenv.
- In our CI files, invoke "python -m venv" rather than "virtualenv".
- In documentation, tell users to install python3-venv and not
  virtualenv.

Signed-off-by: Tom Rini <trini@konsulko.com>
2025-04-24 15:37:27 -06:00
Tom Rini
05a2b69841 buildman: Update to grabbing gcc-14.2.0 toolchains by default
With the switch to using GCC 14.2.0 in commit 001bac5f16 ("Dockerfile:
Update to gcc-14.2.0 and clang-18") in CI, we should make buildman match
this.

Signed-off-by: Tom Rini <trini@konsulko.com>
2025-04-15 11:55:17 -06:00
Heinrich Schuchardt
d8dcfeb778 doc/buildman: typo 'require'
%s/require/required/

Signed-off-by: Heinrich Schuchardt <heinrich.schuchardt@canonical.com>
2025-04-11 13:27:38 +02:00
Simon Glass
3703298e57 u_boot_pylib: Clean up pylint warnings in gitutil.py
This file has about 40 pylint warnings, but no errors.

Quite a few of these warnings have been there for a while, but most are
coming from newer versions of pylint, where people come up with new
warnings.

The f-string warning is the most common one:

   C0209: Formatting a regular string which could be an f-string

That feature was not available when the code was written, but it is
often more convenient than using % with a list of arguments.

This patches reduces the number of warnings in this file, with 7 left
remaining.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-04-01 08:46:41 -06:00
Tom Rini
986ab810fa Merge patch series "tools: Minor clean-ups for the command library"
Simon Glass <sjg@chromium.org> says:

This series adds comments and fixes pylint warnings in the command
library. It also introduces a new, simpler way of running a single
command.

Link: https://lore.kernel.org/r/20250203162704.627469-1-sjg@chromium.org
2025-03-04 13:32:15 -06:00
Simon Glass
3d094ce28a u_boot_pylib: Add a function to run a single command
Add a helper to avoid needing to use a list within a list for this
simple case.

Update existing users of runpipe() to use this where possible.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-03-04 13:31:49 -06:00
Simon Glass
d6900a778a u_boot_pylib: Correct case for test_result
This should be in capitals and defined at the start of the file. Update
it.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-03-04 13:31:49 -06:00
Simon Glass
7520827be3 buildman: Update tests for newer filelock module
Recent versions of this module call time.perf_counter() so add a patch
for this also.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reported-by: Tom Rini <trini@konsulko.com>
2025-02-21 14:11:05 -06:00
Simon Glass
6e628c221e tools: Fix pylint 3.3.4 errors
This newer pylint produces errors about variables possibly being used
before being set. Adjust the code to pass these checks.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reported-by: Tom Rini <trini@konsulko.com>
2025-02-21 08:24:37 -06:00
Simon Glass
5c33fb0288 u_boot_pylib: Move gitutil into the library
Move this file into U-Boot's Python library, so that it is no-longer
part of patman.

This makes a start on:

https://source.denx.de/u-boot/custodians/u-boot-dm/-/issues/35

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-02-17 11:17:55 -06:00
Tom Rini
859621b47f python: Recreate test/py and tools/buildman requirements.txt files
Use the "pipreqs" tool to re-create these files, with a few manual
corrections. We still need to include pytest-xdist which the tool does
not detect. We also for now don't upgrade most of the required tools as
that creates problems with various tests, which should be resolved
independently.

Signed-off-by: Tom Rini <trini@konsulko.com>
2025-02-14 17:11:35 -06:00
Simon Glass
bd7883f346 buildman: Record an error if a toolchain is missing
Buildman has always treated the lack of a toolchain as an infrastructure
problem rather than a build failure.

However the logic for this is not correct, since it does not write a
'done' file in this case.

As a result, one of two things can happen.

1. If a previous build ran in the same (output) directory, the outcome
   of *that* build is recorded as the outcome of this one
2. Otherwise, no outcome is recorded

Obviously this inconsistency is not ideal. While (2) is rare, it can be
very confusing as the build sort-of fails but does not produce any
summary output with 'buildman -s'

Overall it seems better to attribute a toolchain issue to the boards
that it affects. This results in clear failures which can be examined,
no matter what happened in the .bm-work directory previously.

So write a 'done' file for each build when a toolchain is missing.

The end result of this patch is to make missing toolchains much more
obvious. It should be things a bit easier for novice users.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-01-25 16:27:24 -06:00
Simon Glass
dd7138dde7 buildman: Drop unused OUTPUT_FILE constant
This is not actually used but its presence suggests that it is the
filename for the board database. Drop it to avoid confusion.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-01-25 16:27:24 -06:00
Simon Glass
e2ca257287 buildman: Support a tilde to represent the home directory
It is convenient to use ~ to represent the home directory in the
settings file. Add support for this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2024-11-19 10:04:47 -06:00
Simon Glass
eae1eb0e86 buildman: Correct logic for adding a toolchain
If the toolchain is bad, it should not be added. Correct the logic for
this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2024-11-19 10:04:47 -06:00
Simon Glass
a79a9ebee0 buildman: Document the toolchain-prefix section
This is mentioned in examples but should have its own mention in the
'settings' documentation. Add it.

Signed-off-by: Simon Glass <sjg@chromium.org>
2024-11-19 10:04:47 -06:00
Simon Glass
d6ebe17d9d buildman: Correct the indentation in the setting-up section
The example settings file should be indented so that it shows correctly
in the output. Fix it.

Signed-off-by: Simon Glass <sjg@chromium.org>
2024-11-19 10:04:47 -06:00
Simon Glass
6bf74a2e0b buildman: Support #include files in defconfigs
This is used by some boards in U-Boot and is a convenient way to deal
with common settings where using a Kconfig files is not desirable.

Detect #include files and process them as if they were part of the
original file.

Signed-off-by: Simon Glass <sjg@chromium.org>
Fixes: https://source.denx.de/u-boot/custodians/u-boot-dm/-/issues/30
2024-11-19 10:04:47 -06:00
Simon Glass
6d66ded18e buildman: Set up the tout library
Use this to control info and debugging messages. Enable them if -v is
given.

Signed-off-by: Simon Glass <sjg@chromium.org>
2024-11-19 10:04:47 -06:00
Simon Glass
a63fcdb9a4 buildman: Add a lower-level test for KconfigScanner
This code is tested by test_scan_defconfigs() but it is useful to have
some specific tests for the KconfigScanner's operation in U-Boot. Add
a test which checks that the values are obtained correctly.

Signed-off-by: Simon Glass <sjg@chromium.org>
2024-11-19 10:04:47 -06:00
Simon Glass
02eb9f1665 buildman: Correct some warnings about regex strings
With Python 3.12 some warnings have cropped up. Fix them.

Signed-off-by: Simon Glass <sjg@chromium.org>
2024-10-18 14:10:22 -06:00
Simon Glass
ba134c3531 buildman: Allow skipping the dtc build
For most boards, the device-tree compiler is built in-tree, ignoring the
system version. Add a special option to skip this build. This can be
useful when the system dtc is up-to-date, as it speeds up the build.

Signed-off-by: Simon Glass <sjg@chromium.org>
2024-09-26 12:40:30 +02:00
Simon Glass
bf984a6a00 buildman: Retry the build for current source
Buildman retries a failed build when processing a branch, but does not
do this when building current source. It is useful to do this retry in
both cases, so add the logic for it.

Signed-off-by: Simon Glass <sjg@chromium.org>
2024-09-26 12:40:30 +02:00
Jerome Forissier
0c2d7ad3e0 buildman/toolchain.py: do not set CROSS_COMPILE for sandbox
When building for sandbox, self.cross is empty.

In MakeEnvironment(), CROSS_COMPILE is defined to be self.cross (with
or without a full path), optionally prefixed by the toolchain wrapper
defined in ~/.buildman. This is fine when self.cross is not empty, but
it doesn't make sense when it is:
- Either there is no wrapper and we end up with an empty CROSS_COMPILE
which is the same as not defining it (the host compiler will be used),
- Or there is a wrapper and CROSS_COMPILE will contain only the wrapper
which obviously is not a valid compiler, hence an error.

Test case:

 $ sudo apt install ccache
 $ grep -q toolchain-wrapper ~/.buildman || \
     printf "[toolchain-wrapper]\nwrapper = ccache\n" >>~/.buildman
 $ make mrproper
 $ ./tools/buildman/buildman sandbox_noinst
 $ ./tools/buildman/buildman sandbox_noinst
 Building current source for 1 boards (1 thread, 24 jobs per thread)
    sandbox:  +   sandbox_noinst
 +arch/sandbox/lib/reloc_sandbox_efi.c:10:15: error: operator '==' has no left operand
 +   10 | #if HOST_ARCH == HOST_ARCH_X86_64
 +      |               ^~
[...]

The GetEnvArgs function is modified too, since the VAR_CROSS_COMPILE
case has the same issue.

In tools/buildman/test.py, testGetEnvArgs is extended and
testMakeEnvironment is added. They check the 'arm' and 'sandbox'
toolchains, with and without a wrapper.

Signed-off-by: Jerome Forissier <jerome.forissier@linaro.org>
Reviewed-by: Simon Glass <sjg@chromium.org>
2024-09-24 13:41:20 -06:00
Simon Glass
d0f74bd417 buildman: Support building within a Python venv
The Python virtualenv tool sets up a few things in the environment,
putting its path first in the PATH environment variable and setting up
a sys.prefix different from the sys.base_prefix value.

At present buildman puts the toolchain path first in PATH so that it can
be found easily during the build. For sandbox this causes problems since
/usr/bin/gcc (for example) results in '/usr/bin' being prepended to the
PATH variable. As a result, the venv is partially disabled.

The result is that sandbox builds within a venv ignore the venv, e.g.
when looking for packages.

Correct this by detecting the venv and adding the toolchain path after
the venv path.

Signed-off-by: Simon Glass <sjg@chromium.org>
2024-09-06 12:45:54 -06:00
Hiago De Franco
1bf048f21b tools/scripts/Kconfig: Update references from kconfig-language.txt to rst
The Linux kernel documentation has transitioned from using
kconfig-language.txt to kconfig-language.rst. Therefore update all
occurrences of kconfig-language.txt.

Signed-off-by: Hiago De Franco <hiago.franco@toradex.com>
2024-08-27 18:01:32 -06:00
Simon Glass
ab84ffccd6 buildman: Make test_process_limit handle time.monotonic()
Newer versions of filelock use time.monotonic() instead of time.time().
Update the test the handle this.

It would be better if filelock had support for writing unit tests which
use locking.

Signed-off-by: Simon Glass <sjg@chromium.org>
2024-08-26 18:51:48 -06:00
Simon Glass
e8effc3854 buildman: Show board list with -x
When -x is used, buildman does not show the list of boards that will be
built, since there are no terms which cause boards to be added, only
terms which cause them to be removed.

Add a special case to fix this.

Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Tom Rini <trini@konsulko.com>
2024-07-26 08:01:06 -06:00
Tom Rini
e13fcae3fc Revert "buildman: Always use the full path in CROSS_COMPILE"
There are operations in buildman that result in running the cross-tools
(such as performing size checks) and now that we have not modified PATH
to know where our tools are, these operations fail.

This reverts commit 6c0a3cf75f.

Signed-off-by: Tom Rini <trini@konsulko.com>
2024-07-08 09:12:34 -06:00
Simon Glass
6c0a3cf75f buildman: Always use the full path in CROSS_COMPILE
The feature to set the toolchain path does not seem to be needed. It
causes problems with venv (see [1]). Let's remove it.

Add some tests while we are here.

It does not look like any docs changes are needed for this.

[1] https://patchwork.ozlabs.org/project/uboot/patch/20240621131423.2363294-6-sjg@chromium.org/

Signed-off-by: Simon Glass <sjg@chromium.org>
Suggested-by: Tom Rini <trini@konsulko.com>
Reviewed-by: Tom Rini <trini@konsulko.com>
Reviewed-by: Andrejs Cainikovs <andrejs.cainikovs@toradex.com>
2024-07-03 07:36:33 +01:00
Simon Glass
57abd7c549 buildman: Fix a few typos in toolchain code
Fix 'Thie' and capitalise 'unicode'.

Signed-off-by: Simon Glass <sjg@chromium.org>
Suggested-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
2024-07-03 07:36:33 +01:00