53 Commits

Author SHA1 Message Date
Pieter De Gendt
f344ab6b98 cobs: Introduce streaming
This commit does:

- Introduce COBS streaming
- Refactor custom delimiter with XOR'ed encoded data
- Update tests

Signed-off-by: Pieter De Gendt <pieter.degendt@basalte.be>
2026-01-21 17:06:04 +01:00
Jordan Yates
1d92d0b5ff utils: timeutil: option to disable clock skew
Accurately applying clock skew correction in
`timeutil_sync_ref_from_local` and `timeutil_sync_local_from_ref`
requires double-precision floating point operations. If this is the only
usage of double precision logic in the build, but clocks skews are not
used, this can be a significant ROM overhead. Disabling this option can
save ~2.4 kB of ROM.

Signed-off-by: Jordan Yates <jordan@embeint.com>
2025-12-18 09:24:34 +01:00
Benjamin Cabé
bef6cdb127 sys: util: add sys_ prefix to SIGN, gcd, and lcm utils
As per our naming conventions for public symbols, these utility functions
and macros should be prefixed with "sys_".

Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
2025-12-12 04:58:14 -05:00
Fabio Baltieri
4ab97f565e sys: getopt: drop the gcd definition
This is available in util.h since 646f255373 and the existing one
creates a collision and build breakage. Just drop it.

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2025-12-08 22:26:48 -05:00
Alberto Escolar Piedras
72090b73ef lib/utils/getopt: Fix trivial issues detected by SonarCloud
Fix uncontroversional issues detected by SonarCloud, whose fixes are
trivial, and improve code readability.

This commit fixes multiple sonar cloud detected issues like:
"Give names to the parameters of this function prototype."
"Declare the variable "i"/"j" inside the loop"
"Remove this empty statement"

Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
2025-11-19 15:53:28 -05:00
Jukka Rissanen
d164f8b55d lib: posix: Add shim for getopt API
Allow user to use the new sys_getopt API via the old non-Posix
compliant getopt API. This is a temporary solution and the shim
will be deprecated and eventually removed at some point.

Signed-off-by: Jukka Rissanen <jukka.rissanen@nordicsemi.no>
2025-11-18 10:46:29 -05:00
Jukka Rissanen
22f9ef0a33 sys: getopt: Rename getopt to sys_getopt to avoid conflicts with Posix
Summary:

As Zephyr getopt is not really compatible with Posix getopt, rename
Zephyr getopt to sys_getopt.

Background:

Zephyr getopt module was introduced in #31356 to be used by the shell.
Zephyr's getopt is not the standard one. It has multiple APIs which make
it more suited for a system like Zephyr where different components may
want to use it for different purposes. Including APIs to init it,
get and set its internal state etc.
Several Zephyr modules (shell, net, wifi, ztest) use this getopt with
these special APIs. The getopt module is bundled in the POSIX
compatibility API subsystem (CONFIG_POSIX_C_LIB_EXT).

Problem description:

As it is not the standard getopt(), no C library can possibly provide
a Zephyr compatible version (even if such C library were to provide a
standard getopt()). As it is bundled in Zephyr's POSIX API in
CONFIG_POSIX_C_LIB_EXT), multiple components that depend on it are
selecting CONFIG_POSIX_C_LIB_EXT. Zephyr core components should not
depend on the POSIX API in this way.

Changes done in this commit:

Rename the getopt*() APIs to sys_getopt*() and move them into a module
under lib/utils with its own Kconfig option to enable it.
Zephyr's users are changed to use this new component.
The POSIX subsystem can continue providing getopt() by calling the new
sys_getopt() and in that way retain backwards compatibility for external
users.

Signed-off-by: Jukka Rissanen <jukka.rissanen@nordicsemi.no>
2025-11-18 10:46:29 -05:00
BUDKE Gerson Fernando
0ddb8e40f2 lib: json: Fix escape sequence unescaping during decoding
Fix the JSON library to properly unescape sequences during decoding.
Previously, escape sequences like \n, \t, \" etc were validated but
not converted back to their actual character representations during
decoding, causing backslash duplication with each encode/decode cycle.

This fix modifies decode_string_buf() to properly handle escape
sequence unescaping and ensures strings maintain their original
content across multiple encode/decode cycles.

Signed-off-by: BUDKE Gerson Fernando <gerson.budke@leica-geosystems.com>
2025-11-08 10:22:50 +02:00
Fabio Baltieri
700a1a5a28 lib, kernel: use single evaluation min/max/clamp
Replace all in-function instances of MIN/MAX/CLAMP with the single
evaluation version min/max/clamp.

There's probably no race conditions in these files, but the single
evaluation ones save a couple of instructions each so they should save
few code bytes and potentially perform better, so they should be
preferred in general.

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2025-10-24 01:10:40 +03:00
Robin-Charles Guihéneuf
e4bd3cffd4 lib: utils: json: Fix encoded_obj element size
The size of the JSON_TOK_ENCODED_OBJ type was not defined, which
caused errors when generating an array.

Signed-off-by: Robin-Charles Guihéneuf <robin-charles@hotmail.fr>
2025-10-09 22:55:58 +03:00
Chris Friedt
584b73a8d4 timeutil: make timespec_normalize() a regular function
Since the timespec_normalize() function is not branchless, and since it
can also generate a fair bit of code, make the function non-inline and
place it in timeutil.c instead.

This should save some code space on most systems.

Signed-off-by: Chris Friedt <cfriedt@tenstorrent.com>
2025-09-26 09:35:33 -04:00
Fabio Baltieri
fbe0c9feb9 utils: utf8: change utf8_count_chars return to int
Change utf8_count_chars return type to int and drop thesys/types.h, this
way the function does not depend on posix types.

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2025-09-08 11:11:40 -04:00
Fabio Baltieri
8366365c03 util: move utf8 utils to a separate header
This moves the declaration of the utf8 utils defined in lib/utils/utf8.c
in their own header. Main reason to do this is that the current setup
requried adding an include for sys/types.h in util.h, which can result
in a build falure due to a circular header depdenecy when using:

CONFIG_POSIX_API=y
CONFIG_NEWLIB_LIBC=y
_GNU_SOURCE

the loop and error are:

- include/sys/types.h:50: <- this is a NEWLIB one
- include/zephyr/posix/sys/select.h:9:
- include/zephyr/posix/posix_types.h:30:
- include/zephyr/kernel.h:17:
- include/zephyr/kernel_includes.h:25:
- include/zephyr/sys/atomic.h:18:

include/zephyr/sys/util.h:705:1:
error: unknown type name 'ssize_t'

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2025-09-08 11:11:40 -04:00
Krzysztof Chruściński
a6c04cf7d8 lib: utils: bitarray: Use optimized version for 32 bit array
When bitarray is limited to a single 32 bit word then bitmask_find_gap
can be used which is much faster than bit by bit search which is
used in bitarray spanning across multiple 32 bit words. Tests shows
that allocation and freeing is 2-3 times faster.

Signed-off-by: Krzysztof Chruściński <krzysztof.chruscinski@nordicsemi.no>
2025-09-06 10:38:18 +02:00
Krzysztof Chruściński
53c2d58ff8 lib: utils: Add function for finding group of cleared bits in a bit mask
Add function which finds contiguous number of bits which are not set
in the 32 bit mask.

Signed-off-by: Krzysztof Chruściński <krzysztof.chruscinski@nordicsemi.no>
2025-09-06 10:38:18 +02:00
Iliar Rabet
4a4a46c26f util: adds the utf8_count_chars function
The function utf8_count_chars counts the number of UTF-encoded characters

Signed-off-by: Iliar Rabet <iliar.rabet@gmail.com>
2025-09-01 23:27:44 +02:00
Sanjay Vallimanalan
ce4eba52d5 lib: utils: json: add support for mixed array types
Added support for mixed array types in JSON by introducing a new type -
json_mixed_arr_descr. Added APIs for mixed array type parsing and
encoding.

Signed-off-by: Sanjay Vallimanalan <sanjay@linumiz.com>
2025-07-21 21:31:21 -04:00
Christoph Winklhofer
d5354449be json: fix encoding of string null pointer
A segmentation fault occurs when a zero initialized struct with a string
field (JSON_TOK_STRING) is encoded. Encode a string null pointer as an
empty JSON string "".

Signed-off-by: Christoph Winklhofer <cj.winklhofer@gmail.com>
2025-07-11 08:15:36 -10:00
Benjamin Cabé
455280e5aa lib: hex: remove unnecessary defensive programming
The hex2char() calls in bin2hex() can never fail since buf[i] >> 4
and buf[i] & 0xf always produce values in range 0-15.

Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
2025-06-11 10:25:14 -07:00
Alex Fabre
da2ecd3eb8 utils: bitarray: fix comparison of int of different signs
Clang 20.1.0 -Wsign-compare warning on int being compared with size_t
(unsigned int).

Signed-off-by: Alex Fabre <alex.fabre@rtone.fr>
2025-05-13 10:05:52 +02:00
Torsten Rasmussen
c79c4ef9a8 linker: move last section id constant to c-code
Move creation of last section id from ld linker script LONG() usage to
C code with last section attribute.

The use of `LONG()` works correctly with ld but lld emits a warning
because .last_section section is not allocated as there are no matching
input sections and discards the `LONG()` call, meaning the last section
identifier will not be present in the flash.
> ld.lld: warning: ignoring memory region assignment for
>                             non-allocatable section '.last_section'

Placing the last section id in `.last_section` in C code makes lld
allocate the memory for the id and thereby create the output section
with the correct output.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2025-05-08 15:55:40 +02:00
Christoph Winklhofer
a5e295c452 json: improve parsing and serializing of integers
Add support for parsing and serializing of following integer types:
'int8_t', 'uint8_t', 'int16_t', 'uint16_t' and 'uint32_t'.

The generic integer token JSON_TOK_INT and JSON_TOK_UINT, in combination
with the field size (set by JSON_OBJ_DESCR_PRIM) allows to parse different
integer types, for example:

struct foo {
  int64_t i64;
  uint32_t u32;
  int16_t i16;
  uint8_t u8;
};

struct json_obj_descr foo_descr[] = {
  JSON_OBJ_DESCR_PRIM(struct foo, i64, JSON_TOK_INT),
  JSON_OBJ_DESCR_PRIM(struct foo, u32, JSON_TOK_UINT),
  JSON_OBJ_DESCR_PRIM(struct foo, i16, JSON_TOK_INT),
  JSON_OBJ_DESCR_PRIM(struct foo, u8, JSON_TOK_UINT),
};

These tokens also support parsing and serializing enums:

enum unsigned_enum { UA=0, UB=1, UC=2 };
enum signed_enum { SA=-1, SB=0, SC=1 };

struct foo {
  enum unsigned_enum u;
  enum signed_enum s;
};

struct json_obj_descr foo_descr[] = {
  JSON_OBJ_DESCR_PRIM(struct foo, u, JSON_TOK_UINT),
  JSON_OBJ_DESCR_PRIM(struct foo, s, JSON_TOK_INT),
};

Signed-off-by: Christoph Winklhofer <cj.winklhofer@gmail.com>
2025-04-11 06:32:50 +02:00
Christoph Winklhofer
63d33e631c json: support parsing and serializing of strings with char arrays
Support parsing and serializing of struct fields that are defined as a
char array.

Use the token JSON_TOK_STRING_BUF to parse and serialize a string for a
char array, for example:

struct foo {
  const char *str;
  char str_buf[30];
};

struct json_obj_descr foo_descr[] = {
  JSON_OBJ_DESCR_PRIM(struct foo, str, JSON_TOK_STRING),
  JSON_OBJ_DESCR_PRIM(struct foo, str_buf, JSON_TOK_STRING_BUF),
};

The struct 'json_obj_descr' now has an additional union member 'field'
to store the size of the struct field, which is used with the token
'JSON_TOK_STRING_BUF' to determine the element size.

Fixes: #65200
Signed-off-by: Christoph Winklhofer <cj.winklhofer@gmail.com>
2025-04-11 06:32:50 +02:00
Christoph Winklhofer
fc37c02eb1 json: improve parsing and serializing of 'float' and 'double'
Up to now, the handling of type float was offloaded to the users of the
JSON utility, with the token JSON_TOK_FLOAT.

Improve handling of floating point types and support the types 'float'
and 'double' in a built-in way so that they can be directly parsed to
and serialized from variables (of type float or double).

The types are serialized in the shortest representation, either as a
decimal number or in scientific notation:
  * float (with JSON_TOK_FLOAT_FP): encoded with maximal 9 digits
  * double (with JSON_TOK_DOUBLE_FP): encoded with maximal 16 digits
  * NaN, Infinity, -Infinity: encoded and decoded as:
    {"nan_val":NaN,"inf_pos":Infinity,"inf_neg":-Infinity}

Enable the floating point functionality with the Kconfig option:
  JSON_LIBRARY_FP_SUPPORT=y

It requires a libc implementation with support for floating point
functions: strtof(), strtod(), isnan() and isinf().

Fixes: #59412
Signed-off-by: Christoph Winklhofer <cj.winklhofer@gmail.com>
2025-04-09 22:05:14 +02:00
Christoph Winklhofer
c7b7ec2faf json: Support floating point values NaN and Infinity for numbers
Add support to decode the special floating point values NaN, Infinity
and -Infinity. For example:
  {"nan_val":NaN,"inf_pos":Infinity,"inf_neg":-Infinity}

Note that this commit is a preparation for the built-in support of
floating point values and these are only accepted when compiled with
the flag -DCONFIG_JSON_LIBRARY_FP_SUPPORT.

Signed-off-by: Christoph Winklhofer <cj.winklhofer@gmail.com>
2025-04-09 22:05:14 +02:00
Christoph Winklhofer
f07f2da8dc json: support scientific notation for numbers
Add support to decode floating point numbers in scientific notation,
e.g. 3.40282347e+38. Only the lower-case specifier 'e' is allowed.

Signed-off-by: Christoph Winklhofer <cj.winklhofer@gmail.com>
2025-04-09 22:05:14 +02:00
Christoph Winklhofer
061e345972 json: Fix calculation of object size
The calculation of the object size may be incorrect when the size of
a field is smaller than the struct alignment. When such a struct is
used in an array field, the decoded object contains wrong values.

The alignment influences the object size. For example the following
struct has a calculated object size of 8 bytes, however due to
alignment the real size of the struct is 12 bytes:

struct test_bool {
  bool b1; /* offset 0, size 1 */
           /* 3-byte padding */
  int i1;  /* offset 4, size 4 */
  bool b2; /* offset 8, size 1 */
           /* 3-byte padding */
};

This commit changes the object size calculation and computes the size
with the offset and size of the last field in the struct (rounded up
by the struct alignment).

Fixes: #85121
Signed-off-by: Christoph Winklhofer <cj.winklhofer@gmail.com>
2025-04-08 11:44:51 +02:00
Nicolas Pitre
e1eead3925 ring_buffer: shrink size of struct ring_buf
Make struct ring_buf 12 bytes smaller by default. This comes with a 32KB
buffer size limit which covers almost all cases. The previous limit of
2GB can be restored with CONFIG_RING_BUFFER_LARGE=y.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2025-03-11 08:59:05 +01:00
Nicolas Pitre
b97f11a752 ring_buffer: simplify code some more
Some variable renaming made the code clearer, which in turn allowed for
minor optimizations.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2025-03-11 08:59:05 +01:00
Nicolas Pitre
fa73b8b061 ring_buffer: update Kconfig help text
Make it more informative and useful.

Buffer sizes that are a power of 2 didn't provide any advantage since
commit 099850e916 ("ring_buffer: the great simplification").

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2025-03-11 08:59:05 +01:00
Helmut Lord
f31b4a67be lib: utils: add consistent overhead byte stuffing
Implementation uses netbufs.

Signed-off-by: Helmut Lord <kellyhlord@gmail.com>
2025-03-11 05:36:49 +01:00
Pisit Sawangvonganan
e3a4a16594 lib: fix typo in multiple directories
Utilize a code spell-checking tool to scan for and correct spelling errors
in various files within the `lib` directory.

Signed-off-by: Pisit Sawangvonganan <pisit@ndrsolution.com>
2025-01-28 00:06:18 +01:00
Nicolas Pitre
3075a7d906 ring_buffer: factorize almost identical code
Factorize almost identical code. May even improve performance due to
CPU cache locality.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2025-01-25 01:40:03 +01:00
Nicolas Pitre
83279d1524 ring_buffer: optimize the partial buffer loop
It is more efficient to break early when size of claimed area becomes zero.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2025-01-25 01:40:03 +01:00
Nicolas Pitre
67978f8ffc ring_buffer: delete redundant code
No need to clamp requested size as ring_buf_get_claim() does it already.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2025-01-25 01:40:03 +01:00
Mykhailo Lohvynenko
03029e7bd7 json: support parsing and serializing 'uint64_t'
Introduce support for 'uint64_t' type, so that unsigned numbers
can be serialized into JSON payloads.

Signed-off-by: Mykhailo Lohvynenko <Mykhailo_Lohvynenko@epam.com>
2024-10-05 14:04:48 -04:00
Anuj Pathak
836514c9dd sys: timeutil: expose and use macro for base year
timeutil internally uses 1900 as base year for calculation,
that is hidden from caller context. Thus declared a macro in
the public header file, and uses that macro will let caller adjust the
calulations accordingly.

Signed-off-by: Anuj Pathak <anuj@croxel.com>
2024-08-29 11:38:28 +02:00
Marcin Niestroj
878640fa08 json: support parsing and serializing 'int64_t'
Up to now there was only support for parsing/encoding 32-bit integer
numbers, with no support for larger ones.

Introduce support for 'int64_t' type, so that large numbers can be
serialized into JSON payloads.

Signed-off-by: Marcin Niestroj <m.niestroj@emb.dev>
2024-08-13 14:58:06 +02:00
Mykyta Poturai
3afd9429c0 json: Fix alignment calculation for nested objects
Currently, json array macros are passing the outer "container" struct
to the Z_JSON_ELEMENT_DESCR macro, causing the alignment to be calculated
from the outer struct which may be incorrect. Fix this by using the
biggest shift from the nested objects to calculate the alignment as the
struct size is calculated based on the biggest alignment of it's members.

Signed-off-by: Mykyta Poturai <mykyta_poturai@epam.com>
2024-07-09 19:02:25 +02:00
Emil Gydesen
4cf00a6125 zephyr: Add zero-len check for utf8_trunc
The function did not check if the provided string had a zero
length before starting to truncate, which meant that last_byte_p
could possible have pointed to the value before the string.

Signed-off-by: Emil Gydesen <emil.gydesen@nordicsemi.no>
2024-06-27 08:48:31 -04:00
Jordan Yates
07870934e3 everywhere: replace double words
Treewide search and replace on a range of double word combinations:
    * `the the`
    * `to to`
    * `if if`
    * `that that`
    * `on on`
    * `is is`
    * `from from`

Signed-off-by: Jordan Yates <jordan@embeint.com>
2024-06-22 05:40:22 -04:00
frei tycho
f17a67fee4 lib: avoided increments/decrements with side effects
- moved ++/-- before or after the value use

Signed-off-by: frei tycho <tfrei@baumer.com>
2024-06-21 08:48:15 -04:00
Hess Nathan
fe06ffb37f coding guidelines: comply with MISRA Rule 13.4
- avoid to use assignment expression value

Signed-off-by: Hess Nathan <nhess@baumer.com>
2024-06-15 05:17:26 -04:00
TaiJu Wu
a2e97405d0 lib: check null pointer before taking lock for bitarray
In some function in bitarray.c, we should check the pointer
of bitarry before taking bitarray lock.

In this patch, we move the null-check assert before the use of
the struct to provide better validation than a crash.

Signed-off-by: TaiJu Wu <tjwu1217@gmail.com>
2024-06-14 19:18:28 +02:00
Hess Nathan
974bad6242 coding guidelines: comply with MISRA Rule 12.1.
-added parentheses verifying lack of ambiguities

Signed-off-by: Hess Nathan <nhess@baumer.com>
2024-06-13 20:28:23 -04:00
frei tycho
77a3aec34e lib: added missing parenthesis
- added missing parenthesis around macro argument expansion

Signed-off-by: frei tycho <tfrei@baumer.com>
2024-06-13 20:21:08 -04:00
Fin Maaß
01c40dda1f json: add JSON_TOK_ENCODED_OBJ
add support to include a already encoded object
to a object

Signed-off-by: Fin Maaß <f.maass@vogl-electronic.com>
2024-06-12 21:15:27 -04:00
frei tycho
139f16702a lib: utils: avoid Boolean-to-integer type casts
Avoid casting expression to an inappropriate essential type.

Signed-off-by: frei tycho <tfrei@baumer.com>
2024-06-11 20:05:37 +03:00
frei tycho
44782fd8af lib: change controlling expressions in if/while to Boolean
Use `do { ... } while (false)' instead of `do { ... } while (0)'.
Use comparisons with zero instead of implicitly testing integers.
Use comparisons with NULL instead of implicitly testing pointers.
Use comparisons with NUL instead of implicitly testing plain chars.
Use `bool' instead of `int' to represent Boolean values.
Use `while (true)' instead of `while (1)' to express infinite loops.

Signed-off-by: frei tycho <tfrei@baumer.com>
2024-06-11 20:03:34 +03:00
Lucas Romero
3f50197a90 lib: bitarray: add method to find nth bit set in region
This is part one of several changes to add more methods to the bitarray api
so that it can be used for broader usecases, specifically LoRaWAN forward
error correction.

Signed-off-by: Lucas Romero <luqasn@gmail.com>
2024-05-21 16:44:38 -04:00