ebpftracer: report missing BPF tracing program types clearly - #367
ebpftracer: report missing BPF tracing program types clearly#367KR-Ravindra wants to merge 1 commit into
Conversation
On kernels built without CONFIG_BPF_EVENTS (e.g. NVIDIA JetPack 5 / L4T 5.10 with CONFIG_KPROBES=n), BPF_PROG_TYPE_TRACEPOINT and BPF_PROG_TYPE_KPROBE are compiled out and bpf(BPF_PROG_LOAD) fails with a bare EINVAL. The agent then exits with failed to load collection: program sched_process_exit: load program: invalid argument where the program name is whichever program happened to load first, and nothing points at the kernel configuration. Probe TracePoint and Kprobe support with cilium/ebpf/features before loading the collection and fail with an error naming the missing kernel option. Only a conclusive ebpf.ErrNotSupported is reported; any other probe result falls through to the real collection load so existing error paths are unchanged. The probe is behind a package-level function variable so the message mapping is unit-tested without root or a VM. Document the CONFIG_BPF_EVENTS requirement in the README.
0129a76 to
3f9d9d5
Compare
|
Self-review before marking ready. The agent now probes the tracing program types it needs before loading and fails with a message naming the likely kernel option instead of a bare EINVAL; only a conclusive not-supported result short-circuits, so lockdown and permission errors still take the existing path. Unit test pins the exact message; vet, tests and build pass in a golang:1.24 container with libsystemd-dev. |
|
Round 1 self-review. Checked |
Problem
On kernels built without
CONFIG_BPF_EVENTS(for example NVIDIA JetPack 5 / L4T5.10.120-tegra, whereCONFIG_KPROBES=nandCONFIG_UPROBE_EVENTS=n), the agent passes every pre-flight check (tracefs present, kernel version >= 5.1, collection spec loads) and then exits with:Nothing in the message points at the kernel configuration, the program name varies between runs (it is whichever program cilium/ebpf loaded first), and the
*ebpf.VerifierErrorbranch prints nothing because this is not a verifier error. Users end up debugging BTF, kernel version or verifier issues that are not the cause.Root cause
ebpftracer/tracer.go:297-305(ebpf.NewCollectionWithOptions) receives a bareEINVALfrombpf(BPF_PROG_LOAD). WithCONFIG_BPF_EVENTSoff,BPF_PROG_TYPE_TRACEPOINTandBPF_PROG_TYPE_KPROBEare compiled out of the kernel andfind_prog_type()returns-EINVAL. Every program in the agent's collection is atracepoint/,kprobe/oruprobe/program (uprobes areBPF_PROG_TYPE_KPROBE), so nothing can load, but the agent has no check that names this condition.Fix
ebpftracer/tracer.go: before loading the collection, probeebpf.TracePointandebpf.Kprobewithfeatures.HaveProgramTypefrom the already-requiredgithub.com/cilium/ebpfmodule (it maps the probe'sEINVALtoebpf.ErrNotSupported). On a conclusiveErrNotSupportedthe agent now fails with:The error wraps
ebpf.ErrNotSupporteditself rather than the probe error:features.HaveProgramTypereturns an*ebpf.UnsupportedFeatureErrorwhose text isTracePoint not supported (requires >= v4.7), and that kernel-version hint is misleading on a 5.10 kernel that simply has the option disabled. Any other probe result (for exampleEPERMunder kernel lockdown) is ignored so the real collection load surfaces the same error it does today. The probe sits behind a package-level function variable so the mapping can be unit-tested without root or theVM=1harness. Cost on the happy path is two tiny program loads, cached by thefeaturespackage.README.md: one line stating that the kernel must be built withCONFIG_BPF_EVENTS=y.No new dependencies;
go.mod/go.sumare unchanged.How tested
New
ebpftracer/tracer_progtype_test.go(TestCheckProgramTypes) injects a fake probe and checks: supported -> no error; a probe error shaped like the real*ebpf.UnsupportedFeatureError(Kprobe not supported (requires >= v4.1): not supported) -> the returned error wrapsebpf.ErrNotSupportedand its text is exactlykernel does not support BPF Kprobe programs (CONFIG_BPF_EVENTS is not set?): not supported; inconclusive probe error -> no error (falls through to the real load).The exact-string assertion catches the kernel-version tail. With
%wapplied to the probe error instead of the sentinel (the first revision of this PR):With the sentinel wrapped:
The GO job from
.github/workflows/ci.ymlwas run against this revision in agolang:1.24container (linux/amd64,libsystemd-devinstalled):gofmt -l .clean,go vet ./...OK,go test ./...OK for every package with tests (cgroup,common,ebpftracer,ebpftracer/l7,logs,node,proc),go build -mod=readonly .OK.goimports -l ./ebpftraceris clean. The existingVM=1tracer tests were not run here; I have not been able to run the changed binary on the affected Tegra kernel from this environment, so the end-to-end message on real hardware is worth a check by anyone who has one.Links
failed to load collection: program <name>: load program: invalid argument#366invalid argument(different root cause): coroot-node not working with secureboot #205features.HaveProgramType: https://pkg.go.dev/github.com/cilium/ebpf/features#HaveProgramTypeThis change was prepared with an AI agent operated by KR-Ravindra, who reviewed and tested it.