Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/CODEOWNERS
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
* @Workable/systems
138 changes: 74 additions & 64 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -1,19 +1,18 @@
name: CI
name: External Secrets Workable CI

on:
push:
tags:
- workable-*
pull_request:
branches:
- main
pull_request: {}
- workable-*

env:
# Common versions
GOLANGCI_VERSION: 'v2.4.0'
KUBERNETES_VERSION: '1.33.x'

# Sonar
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}

permissions:
contents: read

Expand Down Expand Up @@ -70,23 +69,6 @@ jobs:
skip-pkg-cache: true
skip-build-cache: true

license-check:
permissions:
contents: read # for actions/checkout to fetch code
pull-requests: read # for golangci/golangci-lint-action to fetch pull requests
runs-on: ubuntu-latest
needs: detect-noop
if: needs.detect-noop.outputs.noop != 'true' && github.ref != 'refs/heads/main'

steps:
- uses: step-security/harden-runner@f4a75cfd619ee5ce8d5b864b0d183aff3c69b55a # v2.13.1
with:
egress-policy: audit
- name: Checkout
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Check License Headers
uses: apache/skywalking-eyes/header@61275cc80d0798a405cb070f7d3a8aaf7cf2c2c1 # v0.8.0

check-diff:
runs-on: ubuntu-latest
needs: detect-noop
Expand Down Expand Up @@ -153,48 +135,76 @@ jobs:
run: |
make test

- name: Publish Unit Test Coverage
uses: codecov/codecov-action@5a1091511ad55cbe89839c7260b706298ca349f7 # v5.5.1
publish-artifacts:
needs: [lint, check-diff, unit-tests]
if: ${{ needs.detect-noop.outputs.noop != 'true' && startsWith(github.ref, 'refs/tags/workable-') }}
permissions:
id-token: write
contents: read
runs-on: ubuntu-latest
environment: Workable
steps:
- name: Checkout
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GitHub Action actions/checkout persist Git credentials in workflow - low severity
actions/checkout v2 and above persist the default GITHUB_TOKEN in the repository's local git config when persist-credentials is not set to false, during the workflow run. Subsequent workflow steps or third-party actions can read this token from git configuration, increasing the risk of credential theft or misuse within the pipeline. In order to limit the attack surface when external actions are compromised, ensure persist-credentials is set to false.

Show fix

Remediation: Set persist-credentials: false on actions/checkout steps that do not need to push commits back to the repository. Only keep persist-credentials: true when the workflow explicitly performs authenticated git push operations.

Reply @AikidoSec ignore: [REASON] to ignore this issue.
More info

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GitHub Action actions/checkout persist Git credentials in workflow - low severity
actions/checkout v2 and above persist the default GITHUB_TOKEN in the repository's local git config when persist-credentials is not set to false, during the workflow run. Subsequent workflow steps or third-party actions can read this token from git configuration, increasing the risk of credential theft or misuse within the pipeline. In order to limit the attack surface when external actions are compromised, ensure persist-credentials is set to false.

Show fix

Remediation: Set persist-credentials: false on actions/checkout steps that do not need to push commits back to the repository. Only keep persist-credentials: true when the workflow explicitly performs authenticated git push operations.

Reply @AikidoSec ignore: [REASON] to ignore this issue.
More info

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GitHub Action actions/checkout persist Git credentials in workflow - low severity
actions/checkout v2 and above persist the default GITHUB_TOKEN in the repository's local git config when persist-credentials is not set to false, during the workflow run. Subsequent workflow steps or third-party actions can read this token from git configuration, increasing the risk of credential theft or misuse within the pipeline. In order to limit the attack surface when external actions are compromised, ensure persist-credentials is set to false.

Show fix

Remediation: Set persist-credentials: false on actions/checkout steps that do not need to push commits back to the repository. Only keep persist-credentials: true when the workflow explicitly performs authenticated git push operations.

Reply @AikidoSec ignore: [REASON] to ignore this issue.
More info


- name: Get image tag
id: container-info
run: |
echo "image-tag=${GITHUB_REF#refs/tags/workable-}" >> $GITHUB_OUTPUT

- name: Build image
uses: docker/build-push-action@4f58ea79222b3b9dc2c8bbdd6debcef730109a75 # v6.9.1
with:
context: .
file: Dockerfile.standalone
push: false
tags: Workable/external-secrets:${{ steps.container-info.outputs.image-tag }}
provenance: false

# DISTRIBUTION OF SRE IMAGE
- name: Login to sre registry
uses: docker/login-action@9780b0c442fbb1117ed29e0efdff1e18412f7567 # v3.3.0
with:
registry: us-docker.pkg.dev
username: _json_key
password: ${{ secrets.SRE_GCR_SA }}

- name: Push image to sre registry
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
REGISTRY: us-docker.pkg.dev/sre-artifacts-20e4/gcr.io
run: |
docker tag Workable/external-secrets:${{ steps.container-info.outputs.image-tag }} \
${{ env.REGISTRY }}/external-secrets:${{ steps.container-info.outputs.image-tag }}
docker push ${{ env.REGISTRY }}/external-secrets:${{ steps.container-info.outputs.image-tag }}

# DISTRIBUTION OF STAGING IMAGE
- name: Login to staging registry
uses: docker/login-action@9780b0c442fbb1117ed29e0efdff1e18412f7567 # v3.3.0
with:
flags: unittests
file: ./cover.out
registry: us-docker.pkg.dev
username: _json_key
password: ${{ secrets.STAGING_GCR_SA }}

publish-artifacts:
needs: detect-noop
if: needs.detect-noop.outputs.noop != 'true'
uses: ./.github/workflows/publish.yml
permissions:
contents: read #actions/checkout
packages: write #for publishing artifacts
id-token: write #for keyless sign
strategy:
matrix:
include:
- dockerfile: "Dockerfile"
build-args: "CGO_ENABLED=0"
build-arch: "amd64 arm64 s390x ppc64le"
build-platform: "linux/amd64,linux/arm64,linux/s390x,linux/ppc64le"
tag-suffix: "" # distroless
- dockerfile: "Dockerfile.ubi"
build-args: "CGO_ENABLED=0"
build-arch: "amd64 arm64 ppc64le"
build-platform: "linux/amd64,linux/arm64,linux/ppc64le"
tag-suffix: "-ubi"
- dockerfile: "Dockerfile.ubi"
build-args: "CGO_ENABLED=0 GOEXPERIMENT=boringcrypto"
build-arch: "amd64 ppc64le"
build-platform: "linux/amd64,linux/ppc64le"
tag-suffix: "-ubi-boringssl"
with:
dockerfile: ${{ matrix.dockerfile }}
tag-suffix: ${{ matrix.tag-suffix }}
image-name: ghcr.io/${{ github.repository }}
build-platform: ${{ matrix.build-platform }}
build-args: ${{ matrix.build-args }}
build-arch: ${{ matrix.build-arch }}
ref: ${{ github.ref }}
secrets:
IS_FORK: ${{ secrets.GHCR_USERNAME }} # this is just a secret to verify it is a fork or not, no other utility
- name: Push image to staging registry
env:
REGISTRY: us-docker.pkg.dev/staging-artifacts-786a/gcr.io
run: |
docker tag Workable/external-secrets:${{ steps.container-info.outputs.image-tag }} \
${{ env.REGISTRY }}/external-secrets:${{ steps.container-info.outputs.image-tag }}
docker push ${{ env.REGISTRY }}/external-secrets:${{ steps.container-info.outputs.image-tag }}

# DISTRIBUTION OF PRODUCTION IMAGE
- name: Login to production registry
uses: docker/login-action@9780b0c442fbb1117ed29e0efdff1e18412f7567 # v3.3.0
with:
registry: us-docker.pkg.dev
username: _json_key
password: ${{ secrets.PRODUCTION_GCR_SA }}

- name: Push image to production registry
env:
REGISTRY: us-docker.pkg.dev/production-artifacts-0b0d/gcr.io
run: |
docker tag Workable/external-secrets:${{ steps.container-info.outputs.image-tag }} \
${{ env.REGISTRY }}/external-secrets:${{ steps.container-info.outputs.image-tag }}
docker push ${{ env.REGISTRY }}/external-secrets:${{ steps.container-info.outputs.image-tag }}
9 changes: 2 additions & 7 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -72,13 +72,12 @@ FAIL = (echo ${TIME} ${RED}[FAIL]${CNone} && false)
# ====================================================================================
# Conformance

reviewable: generate docs manifests helm.generate helm.schema.update helm.docs lint license.check helm.test.update test.crds.update tf.fmt ## Ensure a PR is ready for review.
reviewable: generate docs manifests helm.generate helm.schema.update helm.docs lint helm.test.update test.crds.update tf.fmt ## Ensure a PR is ready for review.
@go mod tidy
@cd e2e/ && go mod tidy

check-diff: reviewable ## Ensure branch is clean.
@$(INFO) checking that branch is clean
@test -z "$$(git status --porcelain)" || (echo "$$(git status --porcelain)" && $(FAIL))
@$(OK) branch is clean

update-deps:
Expand All @@ -87,10 +86,6 @@ update-deps:
@go mod tidy
@cd e2e/ && go mod tidy

.PHONY: license.check
license.check:
$(DOCKER) run --rm -u $(shell id -u) -v $(shell pwd):/github/workspace apache/skywalking-eyes:0.6.0 header check

# ====================================================================================
# Golang

Expand Down Expand Up @@ -407,7 +402,7 @@ CTY_VERSION := 1.1.3
.PHONY: envtest
envtest: $(ENVTEST) ## Download envtest-setup locally if necessary.
$(ENVTEST): $(LOCALBIN)
test -s $(LOCALBIN)/setup-envtest || GOBIN=$(LOCALBIN) go install sigs.k8s.io/controller-runtime/tools/setup-envtest@latest
test -s $(LOCALBIN)/setup-envtest || GOBIN=$(LOCALBIN) go install sigs.k8s.io/controller-runtime/tools/setup-envtest@release-0.20

.PHONY: golangci-lint
.PHONY: $(GOLANGCI_LINT)
Expand Down
6 changes: 6 additions & 0 deletions pkg/provider/vault/client_get.go
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,12 @@ func (c *client) GetSecret(ctx context.Context, ref esv1.ExternalSecretDataRemot
}
}

// Replace symlinks
data, err = c.resolveSymlink(ctx, data)
if err != nil {
return nil, err
}

return getSecretValue(data, ref.Property)
}

Expand Down
76 changes: 76 additions & 0 deletions pkg/provider/vault/symlink.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
/*
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package vault

import (
"context"
"regexp"
"strings"
)

const (
// Symlink must start with the prefix vault:// to be valid.
vaultSymlink = `vault://`
// Path can be anything and matches the last # as a separator of key.
vaultSymlinkPath = `(?P<Path>.*)#`
// Key can be any alphanumeric character and stops with the first @.
vaultSymlinkSecret = `(?P<Secret>\w+)`
// Version is optional and will match any number after @.
vaultSymlinkVersion = `(@(?P<Version>\d+)?)?`
vaultSymlinkPattern = vaultSymlink + vaultSymlinkPath + vaultSymlinkSecret + vaultSymlinkVersion
)

// isSymlink tests if secret can be converted to string and if it matches the symlink pattern.
func isSymlink(secret any) bool {
if s, ok := secret.(string); ok {
return strings.HasPrefix(s, vaultSymlink)
}

return false
}

// extractSymlinkParts extract capture group items of regex to a map.
func extractSymlinkParts(secret any) (paramsMap map[string]string) {
r := regexp.MustCompile(vaultSymlinkPattern)
match := r.FindStringSubmatch(secret.(string))
paramsMap = make(map[string]string)

for i, name := range r.SubexpNames() {
if i > 0 && i <= len(match) {
paramsMap[name] = match[i]
}
}

return paramsMap
}

// resolveSymlink test if the data passed has symlinks and resolve them.
func (c *client) resolveSymlink(ctx context.Context, data map[string]any) (map[string]any, error) {
for key, secret := range data {
for isSymlink(secret) {
symlink := extractSymlinkParts(secret)

s, err := c.readSecret(ctx, symlink["Path"], symlink["Version"])
if err != nil {
return nil, err
}

secret = s[symlink["Secret"]]
data[key] = secret
}
}

return data, nil
}
81 changes: 81 additions & 0 deletions pkg/provider/vault/symlink_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
/*
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package vault

import (
"testing"

"github.com/google/go-cmp/cmp"
)

func TestIsSymlink(t *testing.T) {
cases := map[string]struct {
got string
want bool
}{
"ShouldResolveSymlink": {
got: "vault://test",
want: true,
},
"ShouldNotResolveSymlink": {
got: "test",
want: false,
},
}

for name, tc := range cases {
t.Run(name, func(t *testing.T) {
if diff := cmp.Diff(tc.want, isSymlink(tc.got), EquateErrors()); diff != "" {
t.Errorf("\nvault.isSymlink(...): -want error, +got error:\n%s", diff)
}
})
}
}

func TestExtractSymlinkParts(t *testing.T) {
cases := map[string]struct {
pattern string
expectedPath string
expectedSecret string
expectedVersion string
}{
"ShouldExtractPathAndSecret": {
pattern: "vault://test#KEY",
expectedPath: "test",
expectedSecret: "KEY",
expectedVersion: "",
},
"ShouldExtractPathAndSecretAndVersion": {
pattern: "vault://test#KEY@21",
expectedPath: "test",
expectedSecret: "KEY",
expectedVersion: "21",
},
}

for name, tc := range cases {
t.Run(name, func(t *testing.T) {
if diff := cmp.Diff(tc.expectedPath, extractSymlinkParts(tc.pattern)["Path"], EquateErrors()); diff != "" {
t.Errorf("\nvault.extractSymlinkParts(...): -want error, +got error:\n%s", diff)
}
if diff := cmp.Diff(tc.expectedSecret, extractSymlinkParts(tc.pattern)["Secret"], EquateErrors()); diff != "" {
t.Errorf("\nvault.extractSymlinkParts(...): -want error, +got error:\n%s", diff)
}
if diff := cmp.Diff(tc.expectedVersion, extractSymlinkParts(tc.pattern)["Version"], EquateErrors()); diff != "" {
t.Errorf("\nvault.extractSymlinkParts(...): -want error, +got error:\n%s", diff)
}
})
}
}
Loading