-
-
Notifications
You must be signed in to change notification settings - Fork 1
Initial commit of a CLI tool to help with NiFi flow migrations #1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
17 commits
Select commit
Hold shift + click to select a range
f43e904
Initial commit of a CLI tool to help with NiFi flow migrations
lfrancke 487462e
Fix Justfile to properly quote filenames
lfrancke 63a492b
Fix pre-commit warnings and make sure pre-commit shows the same warni…
lfrancke 594c6cb
Add a format-only mode
lfrancke bbb13f5
Update .yamllint.yaml
lfrancke ff9190f
Update .cargo/config.toml
lfrancke a4497a2
Add periods to end of comments
lfrancke 8b172b3
Address review comments
lfrancke 3e214b8
Address review comments
lfrancke 74b18e6
Update Cargo.toml
lfrancke b4d49de
Update Cargo.toml
lfrancke fb06a67
Move migration module to the new standard without mod.rs files
lfrancke 5bb9e7b
Change CLI to take positional arguments for input & output
lfrancke f1e4426
README cleanup
lfrancke 9aa84c8
Update src/cli.rs
lfrancke 56307e7
README improvements
lfrancke 01df15a
Update CLAUDE.md
lfrancke File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| # SPDX-FileCopyrightText: 2025 Stackable GmbH | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| [alias] | ||
| nifi-migrate = ["run", "--bin", "nifi-migrate", "--"] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,66 @@ | ||
| # SPDX-FileCopyrightText: 2025 Stackable GmbH | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
| --- | ||
| name: Build nifi-migrate | ||
|
|
||
| on: | ||
| pull_request: | ||
| paths: | ||
| - ".github/workflows/nifi_migrate_pr.yaml" | ||
| - "rust-toolchain.toml" | ||
| - "src/**.rs" | ||
| - "Cargo.*" | ||
|
|
||
| env: | ||
| RUST_VERSION: 1.87.0 | ||
|
|
||
| jobs: | ||
| # This job is always run to ensure we don't miss any new upstream advisories | ||
| cargo-deny: | ||
| name: Run cargo-deny | ||
| runs-on: ubuntu-latest | ||
| # Prevent sudden announcement of a new advisory from failing CI | ||
| continue-on-error: ${{ matrix.checks == 'advisories' }} | ||
| strategy: | ||
| matrix: | ||
| checks: | ||
| - advisories | ||
| - bans licenses sources | ||
| steps: | ||
| - name: Checkout Repository | ||
| uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 | ||
| with: | ||
| persist-credentials: false | ||
|
|
||
| - name: Run cargo-deny | ||
| uses: EmbarkStudios/cargo-deny-action@f2ba7abc2abebaf185c833c3961145a3c275caad # v2.0.13 | ||
| with: | ||
| command: check ${{ matrix.checks }} | ||
|
|
||
| build: | ||
| name: Build nifi-migrate | ||
| needs: | ||
| - cargo-deny | ||
| strategy: | ||
| fail-fast: false | ||
| matrix: | ||
| targets: | ||
| - { target: aarch64-unknown-linux-gnu, os: ubuntu-24.04-arm } | ||
| - { target: x86_64-unknown-linux-gnu, os: ubuntu-latest } | ||
| - { target: aarch64-apple-darwin, os: macos-latest } | ||
| runs-on: ${{ matrix.targets.os }} | ||
| steps: | ||
| - name: Checkout | ||
| uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 | ||
| with: | ||
| persist-credentials: false | ||
|
|
||
| - uses: dtolnay/rust-toolchain@e97e2d8cc328f1b50210efc529dca0028893a2d9 | ||
| with: | ||
| toolchain: ${{ env.RUST_VERSION }} | ||
| targets: ${{ matrix.targets.target }} | ||
|
|
||
| - name: Build Binary | ||
| env: | ||
| TARGET: ${{ matrix.targets.target }} | ||
| run: cargo build --target "$TARGET" --package nifi-migrate |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,61 @@ | ||
| # SPDX-FileCopyrightText: 2025 Stackable GmbH | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
| --- | ||
| name: Release nifi-migrate | ||
|
|
||
| on: | ||
| push: | ||
| tags: | ||
| - "nifi-migrate-[0-9]+.[0-9]+.[0-9]+**" | ||
|
|
||
| env: | ||
| RUST_VERSION: 1.87.0 | ||
|
|
||
| jobs: | ||
| create-release: | ||
| name: Create Draft Release | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - name: Create Draft Release | ||
| uses: softprops/action-gh-release@72f2c25fcb47643c292f7107632f7a47c1df5cd8 # v2.3.2 | ||
| with: | ||
| draft: true | ||
|
|
||
| build: | ||
| name: Build nifi-migrate | ||
| needs: | ||
| - create-release | ||
| strategy: | ||
| fail-fast: false | ||
| matrix: | ||
| targets: | ||
| - { target: aarch64-unknown-linux-gnu, os: ubuntu-24.04-arm } | ||
| - { target: x86_64-unknown-linux-gnu, os: ubuntu-latest } | ||
| - { target: aarch64-apple-darwin, os: macos-latest } | ||
| runs-on: ${{ matrix.targets.os }} | ||
| steps: | ||
| - name: Checkout | ||
| uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 | ||
| with: | ||
| persist-credentials: false | ||
|
|
||
| - uses: dtolnay/rust-toolchain@e97e2d8cc328f1b50210efc529dca0028893a2d9 | ||
| with: | ||
| toolchain: ${{ env.RUST_VERSION }} | ||
| targets: ${{ matrix.targets.target }} | ||
|
|
||
| - name: Build Binary | ||
| env: | ||
| TARGET: ${{ matrix.targets.target }} | ||
| run: cargo build --target "$TARGET" --release --package nifi-migrate | ||
|
|
||
| - name: Rename Binary | ||
| env: | ||
| TARGET: ${{ matrix.targets.target }} | ||
| run: mv "target/$TARGET/release/nifi-migrate" "nifi-migrate-$TARGET" | ||
|
|
||
| - name: Upload Artifact to Release | ||
| uses: softprops/action-gh-release@72f2c25fcb47643c292f7107632f7a47c1df5cd8 # v2.3.2 | ||
| with: | ||
| draft: false | ||
| files: nifi-migrate-${{ matrix.targets.target }} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| # SPDX-FileCopyrightText: 2025 Stackable GmbH | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
| --- | ||
| name: pre-commit | ||
|
|
||
| on: | ||
| pull_request: | ||
|
|
||
| env: | ||
| RUST_TOOLCHAIN_VERSION: "1.87.0" | ||
|
|
||
| jobs: | ||
| pre-commit: | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 | ||
| with: | ||
| persist-credentials: false | ||
| fetch-depth: 0 | ||
| - uses: stackabletech/actions/run-pre-commit@9a70678a34ec4e4f6927d0f8842b74aa857a9577 # v0.10.2 | ||
| with: | ||
| rust: ${{ env.RUST_TOOLCHAIN_VERSION }} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| # Rust build artifacts | ||
| /target | ||
| Cargo.lock | ||
|
|
||
| # Claude Code local settings | ||
| .claude/ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,27 @@ | ||
| --- | ||
| # SPDX-FileCopyrightText: 2025 Stackable GmbH | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| # All defaults or options can be checked here: | ||
| # https://github.com/DavidAnson/markdownlint/blob/main/schema/.markdownlint.yaml | ||
|
|
||
| # Default state for all rules | ||
| default: true | ||
|
|
||
| # MD013/line-length - Line length | ||
| MD013: | ||
| # Number of characters | ||
| line_length: 9999 | ||
| # Number of characters for headings | ||
| heading_line_length: 9999 | ||
| # Number of characters for code blocks | ||
| code_block_line_length: 9999 | ||
|
|
||
| # MD024/no-duplicate-heading/no-duplicate-header - Multiple headings with the same content | ||
| MD024: | ||
| # Only check sibling headings | ||
| siblings_only: true | ||
|
|
||
| # MD033/no-inline-html Inline HTML | ||
| MD033: | ||
| allowed_elements: [details, summary, img] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,59 @@ | ||
| # SPDX-FileCopyrightText: 2025 Stackable GmbH | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
| --- | ||
| fail_fast: false | ||
| exclude: \.patch$ | ||
|
|
||
| default_language_version: | ||
| node: system | ||
|
|
||
| repos: | ||
| - repo: https://github.com/pre-commit/pre-commit-hooks | ||
| rev: 3e8a8703264a2f4a69428a0aa4dcb512790b2c8c # 6.0.0 | ||
| hooks: | ||
| - id: trailing-whitespace | ||
| - id: end-of-file-fixer | ||
| - id: detect-aws-credentials | ||
| args: ["--allow-missing-credentials"] | ||
| - id: detect-private-key | ||
|
|
||
| - repo: https://github.com/adrienverge/yamllint | ||
| rev: 79a6b2b1392eaf49cdd32ac4f14be1a809bbd8f7 # 1.37.1 | ||
| hooks: | ||
| - id: yamllint | ||
|
|
||
| - repo: https://github.com/igorshubovych/markdownlint-cli | ||
| rev: 192ad822316c3a22fb3d3cc8aa6eafa0b8488360 # 0.45.0 | ||
| hooks: | ||
| - id: markdownlint | ||
|
|
||
| - repo: https://github.com/rhysd/actionlint | ||
| rev: 03d0035246f3e81f36aed592ffb4bebf33a03106 # 1.7.7 | ||
| hooks: | ||
| - id: actionlint | ||
|
|
||
| - repo: local | ||
| hooks: | ||
| - id: cargo-test | ||
| name: cargo-test | ||
| language: system | ||
| entry: cargo test | ||
| stages: [pre-commit, pre-merge-commit] | ||
| pass_filenames: false | ||
| files: \.rs$|Cargo\.(toml|lock) | ||
|
|
||
| - id: cargo-rustfmt | ||
| name: cargo-rustfmt | ||
| language: system | ||
| entry: cargo fmt --all -- --check | ||
| stages: [pre-commit, pre-merge-commit] | ||
| pass_filenames: false | ||
| files: \.rs$ | ||
|
|
||
| - id: cargo-clippy | ||
| name: cargo-clippy | ||
| language: system | ||
| entry: cargo clippy --all-targets -- -D warnings | ||
| stages: [pre-commit, pre-merge-commit] | ||
| pass_filenames: false | ||
| files: \.rs$ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| --- | ||
| # SPDX-FileCopyrightText: 2025 Stackable GmbH | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| extends: default | ||
|
|
||
|
|
||
| rules: | ||
| line-length: disable | ||
| truthy: | ||
| check-keys: false | ||
| comments: | ||
| min-spaces-from-content: 1 # Needed due to https://github.com/adrienverge/yamllint/issues/443 | ||
| indentation: disable | ||
| braces: | ||
| max-spaces-inside: 1 | ||
| max-spaces-inside-empty: 0 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,69 @@ | ||
| <!-- | ||
| SPDX-FileCopyrightText: 2025 Stackable GmbH | ||
| SPDX-License-Identifier: Apache-2.0 | ||
| --> | ||
|
|
||
| # Claude Code Instructions | ||
|
|
||
| Always read the README.md file to understand what this project is about. | ||
|
|
||
| When making changes to this project, always run the following checks in order: | ||
|
|
||
| 1. **Format code**: `cargo fmt` | ||
| 2. **Lint code**: `cargo clippy --all-targets -- -D warnings` | ||
| 3. **Run tests**: `cargo test` | ||
| 4. **Check REUSE compliance**: `reuse lint` | ||
| 5. **Lint GitHub Actions** (after changes to `.github/workflows/*.yaml`): `actionlint` | ||
| 6. **Check dependencies** (after changes to `Cargo.toml` or `Cargo.lock`): `cargo deny check` | ||
|
|
||
| All checks must pass before considering the work complete. | ||
|
|
||
| ## Convenient Commands | ||
|
|
||
| You can run all checks at once using: | ||
|
|
||
| - `just all` - Run all checks individually (fmt, clippy, test, reuse, actionlint, deny) | ||
| - `just pre-commit` - Run pre-commit hooks on all files | ||
|
|
||
| To install pre-commit git hooks: | ||
|
|
||
| - `just pre-commit-install` | ||
|
lfrancke marked this conversation as resolved.
|
||
|
|
||
| ## Project-Specific Notes | ||
|
|
||
| - This project follows FSFE REUSE 3.3 specification | ||
|
lfrancke marked this conversation as resolved.
|
||
| - All source files must have SPDX headers | ||
| - Files covered by `REUSE.toml` don't need individual headers | ||
| - License: Apache-2.0 | ||
| - Copyright holder: Stackable GmbH | ||
| - All full sentences in comments (`//` and `///`) must end with a period | ||
|
|
||
| ## Adding New Migration Rules | ||
|
|
||
| When adding new migration rules, follow these steps in order: | ||
|
|
||
| 1. Create a new file in `src/migration/rules/` (e.g., `my_rule.rs`) | ||
| 2. Implement the `MigrationRule` trait with SPDX headers | ||
|
lfrancke marked this conversation as resolved.
|
||
| - Rules can apply to processors, controller services, or both | ||
| - The trait checks for `type` and `bundle` fields, not component type | ||
| 3. Add the module to `src/migration/rules.rs` and export it | ||
| 4. Register it in `Migrator::default()` in `src/migration.rs` | ||
| - Add to the `rules` vec with appropriate comment (processor/controller service) | ||
| 5. Add comprehensive unit tests in the rule file | ||
| - Test both the rule in isolation and in the full migration flow | ||
| 6. **Update README.md** in the "Supported Migrations" section: | ||
| - Add a new subsection describing the migration | ||
| - Include the old and new type/bundle values | ||
| - Explain why the migration is needed (link to JIRA ticket if available) | ||
| 7. Run all checks listed above (fmt, clippy, test, reuse lint) | ||
|
|
||
| All steps must be completed before considering the migration rule complete. | ||
|
|
||
| ## Project Structure | ||
|
|
||
| - `src/main.rs` - CLI entry point, displays version detection and migration results | ||
| - `src/cli.rs` - Command-line argument parsing with clap | ||
| - `src/migration.rs` - Core migration engine with `Migrator` struct | ||
| - `src/migration/rules.rs` - Module declarations for all migration rules | ||
| - `src/migration/rules/*.rs` - Individual migration rule implementations | ||
| - There might be other files but these are the core ones | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.