Repository navigation
feat(build): link plugins into the server binary via a generated registry - #9288
Merged
Merged
Conversation
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
license-eye has checked 4817 files.
| Valid | Invalid | Ignored | Fixed |
|---|---|---|---|
| 3728 | 1 | 1088 | 0 |
Click to see the invalid file list
- backend/plugins/all/all.go
Use this command to fix any missing license headers
```bash
docker run -it --rm -v $(pwd):/github/workspace apache/skywalking-eyes header fix
</details>
| @@ -0,0 +1,113 @@ | |||
| //go:build !devlake_external_plugins | |||
Contributor
There was a problem hiding this comment.
Suggested change
| //go:build !devlake_external_plugins | |
| // Licensed to the Apache Software Foundation (ASF) under one | |
| // or more contributor license agreements. See the NOTICE file | |
| // distributed with this work for additional information | |
| // regarding copyright ownership. The ASF licenses this file | |
| // to you under the Apache License, Version 2.0 (the | |
| // "License"); you may not use this file except in compliance | |
| // with the License. You may obtain a copy of the License at | |
| // | |
| // http://www.apache.org/licenses/LICENSE-2.0 | |
| // | |
| // Unless required by applicable law or agreed to in writing, | |
| // software distributed under the License is distributed on an | |
| // "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | |
| // KIND, either express or implied. See the License for the | |
| // specific language governing permissions and limitations | |
| // under the License. | |
| //go:build !devlake_external_plugins |
…stry Plugins were only discoverable at runtime, through the .so files in PLUGIN_DIR. Add a compile-time path so the whole plugin set can be linked into one binary: - core/plugin.MustRegister registers a plugin under the name it reports. - tools/pluginsgen scans plugins/*/impl for the type implementing plugin.PluginMeta and generates plugins/all, which imports each plugin and registers it from an init function. It honours DEVLAKE_PLUGINS so subset builds keep working, and emits an empty package for "none". - server/main.go blank-imports the registry. Building with -tags devlake_external_plugins swaps in an empty package for the shared-object-only build. - runner.LoadGoPlugins treats a missing PLUGIN_DIR as "no shared-object plugins" instead of failing startup, so a single-binary deployment does not need to ship an empty directory. - CI regenerates the registry and fails when the checked-in file drifts. Build scripts and Dockerfiles are deliberately untouched: `make build` still produces the .so files, and they register over the compiled-in ones harmlessly. Repackaging follows separately.
klesh
force-pushed
the
feat/single-binary
branch
from
October 30, 2026 07:40
2997094 to
9217e30
Compare
Reframe the runtime loader around the shipping model: the official binary compiles in every official plugin, and PLUGIN_DIR is where users drop additional or replacement plugins. - A broken or incompatible .so is logged and skipped instead of aborting startup. Sideloaded plugins are third-party code built outside this repo; one bad file must not take the server down. - A shared object reporting a name that is already registered (an official plugin) overrides it, with a warning. That is the existing behaviour and it is what makes patching an official plugin possible, so keep it but make it visible. - Extract open/lookup/assert into openGoPlugin and stop the loader goroutine from writing the walk's `err` variable. - Update the generated registry and the core-only build tag docs to describe sideloading rather than presenting .so as a legacy fallback.
This was referenced Oct 31, 2026
…#9291) `make build` is now `build-server`: the framework, the API and every plugin in the generated registry link into a single bin/lake, so the runtime stage has no plugin directory to copy. - backend/Dockerfile: copy only /app/bin/lake instead of the whole bin/, and create an empty /app/bin/plugins as the sideload mount point. It is created as root, before the chgrp that makes /app group-writable, because the stage otherwise runs as the devlake user. - Drop `COPY --from=build /go/bin /usr/bin`: mockery and swag are build tools and nothing at runtime uses them. - Dockerfile.local: one build step instead of build-plugin plus build-server, and copy only the binary. - build-server.sh: add -trimpath and -s -w. A DEVLAKE_DEBUG build keeps the symbols so delve still works. - Makefile: build, dev and godev no longer build plugins; build-plugin stays for sideload and core-only builds. - env.example: PLUGIN_DIR now documents the sideload directory. GO_PLUGINS keeps working: it reaches the registry generator, so it now selects which plugins are compiled in rather than which .so files are built. plugins-registry clears GOOS/GOARCH: the Dockerfile exports them to cross-compile the server, and `go run` would otherwise build the generator for the target platform and fail with "exec format error" on an amd64 build host.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
First step of #9276. Plugins were only discoverable at runtime, through the
.sofiles inPLUGIN_DIR; this adds the compile-time path so the whole plugin set can be linked into one binary.core/plugin.MustRegisterregisters a plugin under the name it reports, panicking on a nil plugin or an empty name. Overwrite semantics are unchanged, so a plugin that is both compiled in and loaded from a.sokeeps working.tools/pluginsgenscansplugins/*/implfor the type whose method set coversplugin.PluginMeta(Name,Description,RootPkgPath) and generatesplugins/all/all.go, which imports each plugin and registers it frominit. It fails loudly when a package has no such type or has several.plugins/all/external.gois thedevlake_external_pluginsbuild-tag counterpart, for a plugin-free core binary that loads.sofiles instead.server/main.goblank-imports the registry — the one framework→plugin edge the layering guard permits.runner.LoadGoPluginsis hardened for third-party code. A missingPLUGIN_DIRmeans "nothing to sideload" instead of a boot failure; a.sothat fails to open, has noPluginEntryor does not implementPluginMetais logged and skipped rather than aborting startup; and a.sothat reuses a compiled-in plugin's name overrides it with a warning, which is what makes patching an official plugin possible. The per-file open/lookup/assert sequence moved intoopenGoPlugin, which also removes the sharederrthe old goroutine captured.go mod tidycheck.DEVLAKE_PLUGINSis honoured by the generator, sojira,gitlabproduces a two-plugin registry andnoneproduces an empty one — the same valuesscripts/build-plugins.shalready accepts.Does this close any open issues?
Part of #9276. Build scripts and Dockerfiles are deliberately untouched here:
make buildstill produces the.sofiles, and they register over the compiled-in ones harmlessly. Repackaging follows in a separate PR.Screenshots
N/A
Other Information
Verified locally:
A throwaway test asserting
len(plugin.AllPlugins()) == 47passed, including the awkward names (gh-copilot,grafana_irm,issue_trace,gitextractor), and regeneration is deterministic.The shipping model this sets up: the official binary has every official plugin compiled in, and
PLUGIN_DIRremains the supported place to sideload extra or replacement.soplugins. Sideloading has a hard constraint worth flagging — a plugin must be built with the same toolchain, flags and dependency versions as the binary loading it, or Go refuses it withplugin was built with a different version of package .... The follow-up should publish the build recipe (thelake-builderimage already exists) and add an explicitAPIVersion/PluginAPIVersionhandshake so that failure is legible. The loader now skips a broken.soinstead of taking the server down, so a mismatch costs one plugin, not the whole boot.The registry is what makes the single binary possible: with it linked, an unstripped
go build ./serveris 149 MB — the same plugin set as ~3.0 GB of.sofiles. Stripping and the Dockerfile changes are the next PR.