Skip to content

tools/nxflat: Add an Apache-licensed NXFLAT converter - #20377

Merged
acassis merged 5 commits into
apache:masterfrom
casaroli:nxflat-ldnxflat-apache
Sep 27, 2026
Merged

acassis merged 5 commits into
apache:masterfrom
casaroli:nxflat-ldnxflat-apache

Conversation

@casaroli

@casaroli casaroli commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

Summary

ldnxflat is the last part of the NXFLAT toolchain that NuttX cannot carry, so a board that builds NXFLAT modules has to fetch a GPL tool from buildroot. This adds an Apache-2.0 replacement, and a driver that builds every test module through the toolchain.

mknxflat came in with #19600 and was relicensed with its author's agreement, because Gregory Nutt owned all of it. ldnxflat cannot follow: it descends from elf2flt and obj-res.c through eleven sets of copyright holders reaching back to 1996, several of them companies that no longer exist. It is GPL by that descent rather than by its libbfd dependency, so no set of permissions can be assembled and a rewrite is the only route.

NXFLAT is not an ARM format. binfmt/Kconfig puts no architecture dependency on it and the loader only ever adds a base to a 32-bit word, so an architecture here supplies a table entry, an entry-point convention and a relocation handler, and an object for an unknown machine is refused by name rather than converted wrongly.

Impact

Nothing changes for a configuration that does not set CONFIG_NXFLAT. For one that does, LDNXFLAT names the tool in the tree instead of looking for one on PATH, and mknxflat's output is unchanged: the thunk it generates for each of the eleven test modules is byte identical to the one the merged version generates.

Two defects of the old converter do not survive the rewrite. A GOT entry naming a .bss object now resolves to the object, where the old tool dropped the section's address and pointed it at the start of D-Space — the GOT itself — so a module reaching a static through the GOT read and wrote its own GOT. And the alignment gap before .bss no longer goes missing from h_bssend.

A module can also carry a string, which is a compiler flag rather than the converter, and not a new finding: lm3s6965-ek has had -mno-pic-data-is-text-relative in its own scripts/Make.defs since 2021 (issue #3737) and the CMake build applies it to every PIC configuration. The first commit moves it where it belonged; the other sixteen NXFLAT configurations were without it.

Testing

Nine of the seventeen configurations that set CONFIG_NXFLAT are on the CI blacklist, and they are exactly the ones that build a module — the converter was not there to build them with. eagle100:nxflat and lm3s6965-ek:qemu-nxflat come off it here, and both were built with the toolchain CI uses. The other seven stay off for reasons of their own: the thttpd configurations want CONFIG_BOARDCTL_ROMDISK, which their defconfigs do not set, and some of those boards define their own CPICFLAGS without filtering --fixed-r9, so the compiler refuses r9 as the PIC register.

lm3s6965-ek:qemu-nxflat also runs. Under qemu-system-arm -M lm3s6965evb its ROMFS modules execute to End-of-Test.. Exit-ing, struct reporting pf = 0x13335 — odd, so the Thumb bit survives — and then In dummyfunc() -- PASS. Built with the out-of-tree tool instead, the same image panics in the third test with PC: 00000000 and the remaining five never run: longjmp's jmp_buf is a static, its GOT entry lands on the GOT, and setjmp overwrites it.

tools/nxflat/testsuite.sh builds and converts every module of apps/examples/nxflat/tests and reports the relocations each one carried. It needs no board: a module is built for cortex-m3 against a configured tree's headers.

tools/checkpatch.sh -c -u -m -g passes.

Provenance

The container is defined by include/nxflat.h and by what binfmt/libnxflat does with it, the segment layout by gnu-nxflat-gotoff.ld, and the relocation arithmetic is that of libs/libc/machine/arm/armv7-m/arch_elf.c. Those files and the new one are Apache-2.0.

No part of the out-of-tree tool is used. During development it was run on the same inputs to compare output against, which is how the conventions the container does not state — where the GOT sits, the order of its entries, the order of the relocation records — were matched. Six modules come out byte identical to it, hello++3 converts where it refuses R_ARM_TARGET1, and four differ where it is wrong. Two of its results are deliberately not matched, and the file says which.

apache/nuttx-apps#3803 builds the test modules this exercises. It is not needed for this one to be correct.

@github-actions github-actions Bot added Area: Build system Arch: arm Issues related to ARM (32-bit) architecture Size: XL The size of the change in this PR is very large. Consider breaking down the PR into smaller pieces. labels Sep 27, 2026
@github-actions

github-actions Bot commented Sep 27, 2026 •

Copy link
Copy Markdown

MemBrowse Memory Report

No memory changes detected for:

A module's D-Space is separate from its I-Space, so its read-only data is not
at a fixed offset from its text.  GCC assumes that it is and loads a string
literal PC-relative, which reads I-Space at run time.  A module could
therefore carry no string and reach no static.

lm3s6965-ek has had -mno-pic-data-is-text-relative in its own Make.defs since
2021 (issue apache#3737), and the CMake build gives it to every PIC configuration,
so the flag moves to where it belonged and the board's copy goes.  That copy
also probed for GCC older than 4.9.4, which NuttX no longer supports.  Clang
has no such option, hence the guard.

Assisted-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>
@casaroli
casaroli force-pushed the nxflat-ldnxflat-apache branch 2 times, most recently from 5ca9771 to d93a63d Compare September 27, 2026 09:49
The converter that follows reads the same objects as mknxflat, so the reader
goes into a file of its own before it gains a second user.  nxflat_elf.c
normalises the tables that describe the object -- the headers, the symbols,
the relocation entries -- into the host's order, and leaves the section
contents alone, because those are the target's bytes and the tools write them
out again.  A tool reads a field without knowing whose order it arrived in.

The -d option goes with it.  It chose a dynamic symbol table, which the ld -r
object these tools convert does not have, and which the NXFLAT loader cannot
use anyway: imports reach a module through the array mknxflat generates.

The output is unchanged.  The thunk mknxflat generates for each of the eleven
modules of apps/examples/nxflat/tests is byte identical to the one the
previous version generated.

Assisted-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>
ldnxflat is the last piece of the NXFLAT toolchain that NuttX cannot carry.
It descends from elf2flt through four sets of copyright holders, so it is GPL
by that descent and not merely by its libbfd dependency.  This is a new
implementation, written from include/nxflat.h, from what binfmt/libnxflat does
with the container, and from the ELF specification.  The relocation arithmetic
is that of libs/libc/machine/arm/armv7-m/arch_elf.c, which the ELF loader runs
on the target for the same relocations, and which brings R_ARM_TARGET1 with
it.

NXFLAT is not an ARM format.  Its loader only adds a base to a 32-bit word, so
the segments, the GOT, the relocation records and the header are common to
every architecture.  An architecture supplies a table entry, an entry-point
convention and a relocation handler; an object for a machine with no entry is
refused by name.

The GOT is built here, because ld -r emits none: one entry per symbol that a
GOT-relative reference names, at the start of D-Space, each with a relocation
record of its own.  An entry may hold a function, which is what makes a
function pointer reached through the GOT work.

Two defects of the out-of-tree tool do not survive.  A GOT entry naming a .bss
object lost its section's address and pointed at the start of D-Space, the GOT
itself, so on lm3s6965-ek:qemu-nxflat the longjmp test panics with PC 0 and
the five tests after it never run.  The alignment gap before .bss went missing
from h_bssend as well, leaving D-Space short.

The tool is built and named like the rest of the toolchain.  Makefile.host
builds it, Unix.mk makes a configuration that sets CONFIG_NXFLAT depend on it
beside mknxflat, and LDNXFLAT points at the tool in the tree rather than one
on PATH.

All eight C modules of apps/examples/nxflat/tests convert and run to
completion under qemu-system-arm -M lm3s6965evb.

Assisted-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>
Building an NXFLAT module takes four tools in sequence, and there was no way
to exercise them without a board.  testsuite.sh builds every module of
apps/examples/nxflat/tests for cortex-m3 against a configured tree's headers,
and reports the stage each one stopped at and the relocations it carried.

Assisted-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>
Nine of the seventeen configurations that set CONFIG_NXFLAT are on the CI
blacklist, and they are exactly the ones that build a module: the converter
they need was not in the tree.  ldnxflat is in it now, so eagle100:nxflat and
lm3s6965-ek:qemu-nxflat come off the list.  Both were built with the toolchain
CI uses.

The other seven stay off for reasons of their own.  The thttpd configurations
need CONFIG_BOARDCTL_ROMDISK, which their defconfigs do not set, and the older
boards define their own CPICFLAGS without filtering --fixed-r9 out, so the
compiler refuses r9 as the PIC register.

Assisted-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>
@casaroli
casaroli force-pushed the nxflat-ldnxflat-apache branch from d93a63d to 29e3aec Compare September 27, 2026 09:57
@acassis
acassis marked this pull request as ready for review September 27, 2026 14:25
@acassis
acassis requested a review from yamt as a code owner September 27, 2026 14:25
@acassis
acassis merged commit b3237cf into apache:master Sep 27, 2026
53 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Arch: arm Issues related to ARM (32-bit) architecture Area: Build system Board: arm Size: XL The size of the change in this PR is very large. Consider breaking down the PR into smaller pieces.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants