diff --git a/.github/instructions/instructions.md b/.github/instructions/instructions.md index 071e813b9..689d368bc 100644 --- a/.github/instructions/instructions.md +++ b/.github/instructions/instructions.md @@ -199,6 +199,9 @@ Pester tests are organized into groups: # Run tests for DSC CLI ./build.ps1 -SkipBuild -Test -ExcludeRustTests -PesterTestGroup dsc +# Run tests for the dsclint CLI +./build.ps1 -SkipBuild -Test -ExcludeRustTests -PesterTestGroup dsclint + # Run tests for adapters ./build.ps1 -SkipBuild -Test -ExcludeRustTests -PesterTestGroup adapters @@ -413,7 +416,7 @@ The build script automatically installs these. If issues persist: | `-Project` | String[] | Build specific projects only | | `-ExcludeRustTests` | Switch | Skip Rust tests (use with -Test) | | `-ExcludePesterTests` | Switch | Skip Pester tests (use with -Test) | -| `-PesterTestGroup` | String[] | Run specific Pester test groups (dsc, adapters, extensions, grammars, resources) | +| `-PesterTestGroup` | String[] | Run specific Pester test groups (dsc, dsclint, adapters, extensions, grammars, resources) | | `-SkipLinkCheck` | Switch | Skip Windows C++ link tools check | | `-UseX64MakeAppx` | Switch | Use x64 version of MakeAppx.exe | | `-UseCFS` | Switch | Use Component Framework Service | diff --git a/.github/workflows/rust.yml b/.github/workflows/rust.yml index 41127ae43..f56709892 100644 --- a/.github/workflows/rust.yml +++ b/.github/workflows/rust.yml @@ -73,7 +73,7 @@ jobs: strategy: fail-fast: false matrix: - group: [dsc, adapters, extensions, resources] + group: [dsc, dsclint, adapters, extensions, resources] runs-on: ubuntu-latest steps: - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 @@ -157,7 +157,7 @@ jobs: strategy: fail-fast: false matrix: - group: [dsc, adapters, extensions, resources] + group: [dsc, dsclint, adapters, extensions, resources] runs-on: macos-latest steps: - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 @@ -258,6 +258,8 @@ jobs: group: dsc/tests/config - name: dsc-other group: dsc/tests/other + - name: dsclint + group: dsclint - name: adapters group: adapters - name: extensions diff --git a/CHANGELOG.md b/CHANGELOG.md index 9ced61d69..8e4c7a48e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -77,6 +77,51 @@ changes since the last release, see the [diff on GitHub][unreleased]. ### Added +- Added the `dsclint` command, a standalone executable that ships with DSC and checks a + configuration document for problems without applying it. The command validates the document + against the configuration document schema and runs a set of rules that report duplicate resource + names, unresolved and circular dependencies, expressions that don't parse or call unknown + functions, undeclared or unused parameters and variables, invalid parameter constraints, + deprecated features, and, with resource discovery, resource types that aren't installed or + literal properties that don't match a resource's schema. You can change the level of a rule for + a run with the `--rule` option or for every run in the new `lint` section of the settings file, + and control which level makes the command fail with `--fail-on`. The `dsc schema` command + exposes the new `lint-result` and `lint-rules` schemas, and the MCP server exposes the linter as + the `lint_dsc_config` tool. Every diagnostic records the line and column of the problem, and + when the output isn't redirected the command renders each diagnostic as a report that shows the + problem in the document text with the rule's identifier and what the rule requires. + +
Related work items + + - Issues: + - [#591][#591] + - [#842][#842] + - PRs: [#1768][#1768] + +
+ +- Added command plug-ins, which add a top-level command to `dsc` without being part of the DSC + executable. A plug-in is defined by a plug-in manifest, a file whose name ends with + `.dsc.plugin.json`, `.dsc.plugin.yaml`, or `.dsc.plugin.yml`, or by the new `plugins` property + of a manifest list. When the first argument to `dsc` isn't a built-in command, DSC runs the + plug-in that defines the command with the remaining arguments and the standard streams of `dsc`, + and exits with the plug-in's exit code. Built-in commands always take precedence, and DSC only + searches for plug-ins when it needs them. The new `dsc plugin list` command lists the discovered + plug-ins, the top-level help lists plug-in commands after the built-in commands, + `dsc help ` displays the help of a plug-in command, and the scripts from + `dsc completer` complete the names of plug-in commands. DSC tells a plug-in how it was invoked + with the `DSC_EXE`, `DSC_VERSION`, `DSC_PLUGIN_COMMAND`, and `DSC_PLUGIN_DIR` environment + variables and makes the options specified before the command available as `DSC_TRACE_LEVEL`, + `DSC_TRACE_FORMAT`, and `DSC_PROGRESS_FORMAT`. The new `plugins.enabled` setting turns plug-ins + off, and the `dsc schema` command exposes the new `plugin-manifest` schema. + +
Related work items + + - Issues: [#461][#461] + - PRs: + +
+ - Added the `Microsoft.Windows/AuditPolicy`, `Microsoft.Windows/AuditPolicyList`, and `Microsoft.Windows/AuditPolicyOption` built-in resources for managing the Windows advanced audit policy natively, as a replacement for the `AuditPolicyDsc` PowerShell module. `AuditPolicy` @@ -116,6 +161,22 @@ changes since the last release, see the [diff on GitHub][unreleased]. ### Changed +- Schema validation errors now show the problem in place. When the input to a resource or a + resource's properties in a configuration document don't match the resource's schema, DSC reports + every violation at once with its JSON pointer, renders a snippet of the input or the document + that highlights each offending property or value, and suggests the closest known property name + for a property it doesn't recognize. For a configuration document the snippet points at the + resource's declaration in your document rather than at its evaluated properties. The snippet + isn't rendered for the `json` trace format, which keeps one line per error, and is never + rendered for values the schema identifies as secure. + +
Related work items + + - Issues: _None_ + - PRs: [#1768][#1768] + +
+ - Extensions that implement the `secret` capability must now define the `args` property for the `secret` command with exactly one secret name input argument (`nameArg`) and at most one vault input argument (`vaultArg`). Previously, an extension could omit the secret name input argument, @@ -1794,6 +1855,7 @@ Version `3.0.0` is the first generally available release of DSC. [#515]: https://github.com/PowerShell/DSC/issues/515 [#543]: https://github.com/PowerShell/DSC/issues/543 [#566]: https://github.com/PowerShell/DSC/issues/566 +[#591]: https://github.com/PowerShell/DSC/issues/591 [#57]: https://github.com/PowerShell/DSC/issues/57 [#598]: https://github.com/PowerShell/DSC/issues/598 [#628]: https://github.com/PowerShell/DSC/issues/628 @@ -1858,6 +1920,7 @@ Version `3.0.0` is the first generally available release of DSC. [#832]: https://github.com/PowerShell/DSC/issues/832 [#834]: https://github.com/PowerShell/DSC/issues/834 [#841]: https://github.com/PowerShell/DSC/issues/841 +[#842]: https://github.com/PowerShell/DSC/issues/842 [#843]: https://github.com/PowerShell/DSC/issues/843 [#844]: https://github.com/PowerShell/DSC/issues/844 [#845]: https://github.com/PowerShell/DSC/issues/845 @@ -2033,3 +2096,4 @@ Version `3.0.0` is the first generally available release of DSC. [#1755]: https://github.com/PowerShell/DSC/issues/1755 [#1758]: https://github.com/PowerShell/DSC/issues/1758 [#1759]: https://github.com/PowerShell/DSC/issues/1759 +[#1768]: https://github.com/PowerShell/DSC/issues/1768 diff --git a/Cargo.lock b/Cargo.lock index 5d5522668..c0d284181 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -822,6 +822,7 @@ dependencies = [ "dsc-lib-telemetry", "indicatif", "jsonschema", + "miette", "path-absolutize", "regex", "rmcp", @@ -888,12 +889,14 @@ dependencies = [ "registry", "rt-format", "rust-i18n", + "saphyr-parser", "schemars", "semver", "serde", "serde_json", "serde_yaml", "serial_test", + "tempfile", "test-case", "thiserror 2.0.21", "tokio", @@ -1083,6 +1086,23 @@ dependencies = [ "windows 0.62.2", ] +[[package]] +name = "dsclint" +version = "3.4.0-preview.3" +dependencies = [ + "clap", + "crossterm", + "dsc-lib", + "dsc-lib-telemetry", + "miette", + "rust-i18n", + "serde", + "serde_json", + "serde_yaml", + "syntect", + "tracing", +] + [[package]] name = "dsctest" version = "0.1.0" @@ -1683,11 +1703,32 @@ checksum = "25d29b57c17c6528d0dc4c2141cfa2d08200465acb537a3909c8b9b8a4a984c0" dependencies = [ "displaydoc", "litemap", + "serde", "tinystr", "writeable", "zerovec", ] +[[package]] +name = "icu_locale_fallback" +version = "2.4.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "406e9ae5b95d8aa37db6293405931b766a6172d1848daee22d70eb44aa78a43c" +dependencies = [ + "icu_locale_core", + "icu_locale_fallback_data", + "icu_provider", + "potential_utf", + "tinystr", + "zerovec", +] + +[[package]] +name = "icu_locale_fallback_data" +version = "2.4.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3632ebbd0c5c3637ba1a40bdf8288f2cb9d2268e73441a69d5fc73d7cfbaa87c" + [[package]] name = "icu_normalizer" version = "2.4.0" @@ -1737,6 +1778,8 @@ checksum = "413099315636c588ee8f011190b329b971a09a8fae9d157e01dcae8379468514" dependencies = [ "displaydoc", "icu_locale_core", + "serde", + "stable_deref_trait", "writeable", "yoke", "zerofrom", @@ -1744,6 +1787,29 @@ dependencies = [ "zerovec", ] +[[package]] +name = "icu_segmenter" +version = "2.4.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "61b195dc605f81aa181bec9b3bd14e26cc71b903e4644f6af578ca9772c7a710" +dependencies = [ + "icu_collections", + "icu_locale_fallback", + "icu_provider", + "icu_segmenter_data", + "potential_utf", + "smallvec", + "utf8_iter", + "zerotrie", + "zerovec", +] + +[[package]] +name = "icu_segmenter_data" +version = "2.4.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "913a7d294ad056474749152107309fdee3ec17d2482d1ab70fdc61a81c64b9a3" + [[package]] name = "ident_case" version = "1.0.1" @@ -1825,6 +1891,12 @@ version = "0.21.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "cf370abdafd54d13e54a620e8c3e1145f28e46cc9d704bc6d94414559df41763" +[[package]] +name = "is_ci" +version = "1.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7655c9839580ee829dfacba1d1278c2b7883e50a277ff7541299489d6bdfdc45" + [[package]] name = "is_elevated" version = "0.1.2" @@ -2015,6 +2087,12 @@ checksum = "5f98efec8807c63c752b5bd61f862c165c115b0a35685bdcfd9238c7aeb592b7" dependencies = [ "cfg-if", "miette-derive", + "owo-colors", + "supports-color", + "supports-hyperlinks", + "supports-unicode", + "terminal_size", + "textwrap", "unicode-width 0.1.14", ] @@ -2440,6 +2518,12 @@ version = "0.5.2" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "1a80800c0488c3a21695ea981a54918fbb37abf04f4d0720c453632255e2ff0e" +[[package]] +name = "owo-colors" +version = "4.4.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "13c45bb4a6ae1280ec0803b1ef9d3455eb50f01efbbe1447ab020f1d54fba9d8" + [[package]] name = "parking_lot" version = "0.12.5" @@ -2552,6 +2636,8 @@ version = "0.1.6" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "d83eb9bc6d8e5cf568e7a1101d60ee05e81ed50ea106026f3d18deeb046d7661" dependencies = [ + "serde_core", + "writeable", "zerovec", ] @@ -3052,6 +3138,16 @@ dependencies = [ "winapi-util", ] +[[package]] +name = "saphyr-parser" +version = "0.2.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6c6220586c3dacac54bfddec87af86ddb31862d29666e7e35cf4baf70a25d387" +dependencies = [ + "arraydeque", + "thiserror 2.0.21", +] + [[package]] name = "schemars" version = "1.2.3" @@ -3372,6 +3468,27 @@ dependencies = [ "syn 2.0.119", ] +[[package]] +name = "supports-color" +version = "3.0.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c64fc7232dd8d2e4ac5ce4ef302b1d81e0b80d055b9d77c7c4f51f6aa4c867d6" +dependencies = [ + "is_ci", +] + +[[package]] +name = "supports-hyperlinks" +version = "3.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e396b6523b11ccb83120b115a0b7366de372751aa6edf19844dfb13a6af97e91" + +[[package]] +name = "supports-unicode" +version = "3.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b7401a30af6cb5818bb64852270bb722533397edcfc7344954a38f420819ece2" + [[package]] name = "syn" version = "2.0.119" @@ -3463,6 +3580,16 @@ dependencies = [ "windows-sys", ] +[[package]] +name = "terminal_size" +version = "0.4.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "230a1b821ccbd75b185820a1f1ff7b14d21da1e442e22c0863ea5f08771a8874" +dependencies = [ + "rustix", + "windows-sys", +] + [[package]] name = "test-case" version = "3.4.0" @@ -3507,6 +3634,16 @@ dependencies = [ "serde_json", ] +[[package]] +name = "textwrap" +version = "0.16.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6ecfad6c3abc80a577f2b91c1e412ee57e7a060d430b553c1b0c940974ebcd49" +dependencies = [ + "icu_segmenter", + "unicode-width 0.2.3", +] + [[package]] name = "thiserror" version = "1.0.69" @@ -3593,6 +3730,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "abc840c238154d58f9e86b2792261edfde565a17696e1cda1a449de27979801a" dependencies = [ "displaydoc", + "serde_core", "zerovec", ] @@ -4739,6 +4877,7 @@ dependencies = [ "displaydoc", "yoke", "zerofrom", + "zerovec", ] [[package]] @@ -4747,6 +4886,7 @@ version = "0.11.9" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "f58a2799cfe9a1670c7ea8edbd52ef8edb93cb37a7f83bf31ba9e35d9dae6a40" dependencies = [ + "serde", "yoke", "zerofrom", "zerovec-derive", diff --git a/Cargo.toml b/Cargo.toml index 1025721d1..b74cc0200 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -5,6 +5,7 @@ resolver = "2" members = [ "dsc", "dsc-bicep-ext", + "dsclint", "grammars/tree-sitter-dscexpression", "grammars/tree-sitter-ssh-server-config", "lib/dsc-lib-jsonschema-macros", @@ -43,6 +44,7 @@ members = [ default-members = [ "dsc", "dsc-bicep-ext", + "dsclint", "grammars/tree-sitter-dscexpression", "grammars/tree-sitter-ssh-server-config", "lib/dsc-lib-jsonschema-macros", @@ -82,6 +84,7 @@ default-members = [ Windows = [ "dsc", "dsc-bicep-ext", + "dsclint", "grammars/tree-sitter-dscexpression", "grammars/tree-sitter-ssh-server-config", "lib/dsc-lib-jsonschema-macros", @@ -116,6 +119,7 @@ Windows = [ macOS = [ "dsc", "dsc-bicep-ext", + "dsclint", "grammars/tree-sitter-dscexpression", "grammars/tree-sitter-ssh-server-config", "lib/dsc-lib-jsonschema-macros", @@ -138,6 +142,7 @@ macOS = [ Linux = [ "dsc", "dsc-bicep-ext", + "dsclint", "grammars/tree-sitter-dscexpression", "grammars/tree-sitter-ssh-server-config", "lib/dsc-lib-jsonschema-macros", @@ -179,13 +184,13 @@ base64 = { version = "0.23.1" } bytes = { version = "1.12.1" } # dsc-lib, sshdconfig chrono = { version = "0.4.45" } -# dsc, dsc-lib, dsc-bicep-ext, dscecho, registry, runcommandonset, sshdconfig, dsctest, test_group_resource +# dsc, dsc-lib, dsc-bicep-ext, dsclint, dscecho, registry, runcommandonset, sshdconfig, dsctest, test_group_resource clap = { version = "4.6.7", features = ["derive"] } # dsc clap_complete = { version = "4.6.11" } # dsc-lib const-str = {version = "1.1" } -# dsc, registry, dsc-lib-registry, sshdconfig +# dsc, dsclint, registry, dsc-lib-registry, sshdconfig crossterm = { version = "0.29" } # dsc ctrlc = { version = "3.5.2" } @@ -205,7 +210,7 @@ jsonschema = { version = "0.60.2", default-features = false } referencing = { version = "0.60.2" } # dsc-lib linked-hash-map = { version = "0.5" } -# dsc-lib +# dsc, dsc-lib, dsclint miette = { version = "7.6.0" } # dsc-lib murmurhash64 = { version = "0.3" } @@ -237,15 +242,17 @@ rmcp = { version = "3.8.0" } roxmltree = { version = "0.21.2" } # dsc_lib rt-format = { version = "0.3" } -# dsc, dsc-lib, dsc-bicep-ext, dscecho, registry, dsc-lib-registry, runcommandonset, sshdconfig +# dsc, dsc-lib, dsc-bicep-ext, dsclint, dscecho, registry, dsc-lib-registry, runcommandonset, sshdconfig rust-i18n = { version = "4.2.4" } +# dsc-lib +saphyr-parser = { version = "0.2.2" } # dsc, dsc-lib, dscecho, registry, dsc-lib-registry, sshdconfig, dsctest, test_group_resource schemars = { version = "1.2.3", features = ["preserve_order"] } # dsc, dsc-lib semver = { version = "1.0.28" } # dsc-lib serial_test = { version = "4.0.1" } -# dsc, dsc-lib, dscecho, dsc-lib-osinfo, process, registry, dsc-lib-registry, runcommandonset, sshdconfig, dsctest, test_group_resource +# dsc, dsc-lib, dsclint, dscecho, dsc-lib-osinfo, process, registry, dsc-lib-registry, runcommandonset, sshdconfig, dsctest, test_group_resource serde = { version = "1.0.229", features = ["derive"] } # dsc, dsc-lib, dscecho, dsc-lib-osinfo, osinfo, process, registry, dsc-lib-registry, runcommandonset, sshdconfig, dsctest, test_group_resource, y2j serde_json = { version = "1.0.154", features = ["preserve_order"] } diff --git a/data.build.json b/data.build.json index 470b076a3..8321684a3 100644 --- a/data.build.json +++ b/data.build.json @@ -9,6 +9,8 @@ "dsc_default.settings.json", "dsc.settings.json", "dsc-bicep-ext", + "dsclint", + "dsclint.dsc.plugin.json", "dscecho", "echo.dsc.resource.json", "file_dsc", @@ -43,6 +45,8 @@ "dsc_default.settings.json", "dsc.settings.json", "dsc-bicep-ext", + "dsclint", + "dsclint.dsc.plugin.json", "dscecho", "echo.dsc.resource.json", "file_dsc", @@ -82,6 +86,8 @@ "dsc_default.settings.json", "dsc.settings.json", "dsc-bicep-ext.exe", + "dsclint.exe", + "dsclint.dsc.plugin.json", "dscecho.exe", "echo.dsc.resource.json", "file_dsc.exe", @@ -231,6 +237,20 @@ "dsc-bicep-ext" ] }, + { + "Name": "dsclint", + "Kind": "CLI", + "RelativePath": "dsclint", + "IsRust": true, + "Binaries": [ + "dsclint" + ], + "CopyFiles": { + "All": [ + "dsclint.dsc.plugin.json" + ] + } + }, { "Name": "extensions/appx", "Kind": "Extension", diff --git a/docs/reference/cli/completer/index.md b/docs/reference/cli/completer/index.md index b86ae1e57..1d8798cd3 100644 --- a/docs/reference/cli/completer/index.md +++ b/docs/reference/cli/completer/index.md @@ -31,6 +31,11 @@ given shell. DSC can generate completion scripts for the following shells: The output for this command is the script itself. To register completions for DSC, execute the script. +The script completes the commands that [plug-ins][06] provide in the same way as the built-in +commands. DSC discovers the available plug-ins when it generates the script, so generate the script +again after you install or remove a plug-in. The script doesn't complete the arguments of a plug-in +command, except for file paths. + > [!WARNING] > Always review scripts before executing them, especially in an elevated execution context. @@ -111,3 +116,4 @@ ShortSyntax : -h [03]: https://fishshell.com/ [04]: /powershell/scripting/overview [05]: https://zsh.sourceforge.io/ +[06]: ../index.md#plug-in-commands diff --git a/docs/reference/cli/index.md b/docs/reference/cli/index.md index f0823aa37..78ec4b0df 100644 --- a/docs/reference/cli/index.md +++ b/docs/reference/cli/index.md @@ -47,6 +47,15 @@ For more information, see [dsc function][03] The `mcp` command starts DSC as a Model Context Protocol (MCP) server. For more information, see [dsc mcp][04]. +### plugin + +The `plugin` command manages DSC command plug-ins. You can use it to: + +- List the available plug-ins with the command each one provides. + +For more information, see [dsc plugin][07]. For information about running a command that a plug-in +provides, see [Plug-in commands](#plug-in-commands). + ### resource The `resource` command manages a DSC Resource. You can use it to: @@ -80,6 +89,65 @@ gets the help for the `config set` subcommand. You can also use the [--help](#--help) option on a command to display the help information. For example, `dsc config --help` or `dsc config set --help`. +For a command that a plug-in provides, `dsc help ` runs the plug-in with the `--help` +option, so the plug-in displays its own help. + +## Plug-in commands + +A command plug-in adds a command to `dsc` without being part of the DSC executable. When the first +argument to `dsc` isn't one of the built-in [commands](#commands), DSC looks for a plug-in that +defines a command with that name and runs it: + +```sh +dsc [...] +``` + +For example, a plug-in that defines the command `lint` is invoked as `dsc lint`. Use the +[dsc plugin list][08] command to see the available plug-ins, or `dsc --help`, which lists the +plug-in commands after the built-in commands. + +A plug-in is defined by a plug-in manifest, a file whose name ends with `.dsc.plugin.json`, +`.dsc.plugin.yaml`, or `.dsc.plugin.yml`. DSC discovers plug-in manifests in the same directories +as resource and extension manifests. For more information about defining a plug-in, see the +[DSC plug-in manifest schema reference][09]. + +DSC runs a plug-in command with the following behavior: + +- Built-in commands always take precedence. A plug-in can't replace a built-in command. +- DSC only searches for plug-ins when the command isn't built in, when it displays the top-level + help, when it lists plug-ins, and when it generates a completion script. Built-in commands never + search for plug-ins. +- DSC passes every argument after the command name to the plug-in as written, including `--help`. + DSC doesn't interpret those arguments. +- The plug-in runs in the current working directory with the standard input, output, and error + streams of `dsc`. DSC doesn't modify the output of the plug-in. +- DSC exits with the exit code of the plug-in. The meaning of an exit code is defined by the + plug-in, not by the [exit codes](#exit-codes) of `dsc`. +- When more than one plug-in defines the same command name, the plug-in DSC discovers first + provides the command and DSC writes a warning that names the others, which are shadowed. To run + a specific plug-in, shadowed or not, specify its fully qualified type name instead of the command + name, like `dsc OpenDSC/Lint`. DSC doesn't write the warning when you use the type name. +- When no plug-in defines the command, DSC reports the command as an unrecognized subcommand and + exits with code `2`. + +Options for `dsc` itself, like [`--trace-level`](#--trace-level), must be specified before the +command name. DSC makes them available to the plug-in as environment variables. DSC sets the +following environment variables for a plug-in: + +| Environment variable | Value | +| :------------------------- | :---------------------------------------------------------------------------------- | +| `DSC_EXE` | The absolute path to the `dsc` executable that started the plug-in. | +| `DSC_VERSION` | The semantic version of that `dsc` executable. | +| `DSC_PLUGIN_COMMAND` | The name the plug-in was invoked as: its command name or its type name. | +| `DSC_PLUGIN_DIR` | The directory that contains the plug-in manifest. | +| `DSC_TRACE_LEVEL` | The trace level in effect, from the `--trace-level` option or the settings. | +| `DSC_TRACE_FORMAT` | The trace format in effect, from the `--trace-format` option or the settings. | +| `DSC_PROGRESS_FORMAT` | The value of the `--progress-format` option. Only set when the option is specified. | +| `DSC_IGNORE_SETTINGS_FILE` | Set to `1` or `0` when the `--ignore-settings-file` option is specified. | + +To prevent DSC from discovering and running plug-ins, set `plugins.enabled` to `false` in a DSC +settings file. When the setting is defined in the policy settings file, users can't override it. + ## Options ### -l, --trace-level @@ -230,7 +298,7 @@ trace level: The `dsc` command uses semantic exit codes. Each exit code represents a different result for the execution of the command. -| Exit Code | Meaning | +| Exit Code | Meaning | | :-------: | :------------------------------------------------------------------------------------------------------ | | `0` | The command executed successfully without any errors. | | `1` | The command failed because it received invalid arguments. | @@ -252,3 +320,6 @@ file, ensure that the input is encoded as UTF-8. [04]: server/index.md [05]: resource/index.md [06]: schema/index.md +[07]: plugin/index.md +[08]: plugin/list.md +[09]: ../schemas/plugin/manifest/root.md diff --git a/docs/reference/cli/plugin/index.md b/docs/reference/cli/plugin/index.md new file mode 100644 index 000000000..22e8d422b --- /dev/null +++ b/docs/reference/cli/plugin/index.md @@ -0,0 +1,72 @@ +--- +description: Command line reference for the 'dsc plugin' command +ms.date: 11/13/2026 +ms.topic: reference +title: dsc plugin +--- + +# dsc plugin + +## Synopsis + +Operations on DSC plug-ins. + +## Syntax + +```sh +dsc plugin [Options] +``` + +## Description + +The `dsc plugin` command contains a subcommand for listing DSC command plug-ins. + +A command plug-in adds a top-level command to DSC without being part of the DSC executable. For +example, a plug-in that defines the command `lint` is invoked as `dsc lint`. For more information +about how DSC discovers and runs plug-ins, see [Plug-in commands][01]. + +## Commands + +### list + +The `list` command returns the list of available DSC plug-ins with an optional filter. For more +information, see [dsc plugin list][02]. + +### help + +The `help` command returns help information for this command or a subcommand. + +To get the help for a command or subcommand, use the syntax: + +```sh +dsc plugin help [] +``` + +For example, `dsc plugin help` gets the help for this command. `dsc plugin help list` gets the help +for the `list` subcommand. + +You can also use the [--help](#--help) option on the command or subcommand to display the help +information. For example, `dsc plugin --help` or `dsc plugin list --help`. + +To get the help for a command that a plug-in provides, use `dsc help ` or +`dsc --help`. The plug-in prints its own help. + +## Options + +### -h, --help + + + + +Displays the help for the current command or subcommand. When you specify this option, the +application ignores all other options and arguments. + +```yaml +Type : boolean +Mandatory : false +LongSyntax : --help +ShortSyntax : -h +``` + +[01]: ../index.md#plug-in-commands +[02]: ./list.md diff --git a/docs/reference/cli/plugin/list.md b/docs/reference/cli/plugin/list.md new file mode 100644 index 000000000..bdffb1847 --- /dev/null +++ b/docs/reference/cli/plugin/list.md @@ -0,0 +1,206 @@ +--- +description: Command line reference for the 'dsc plugin list' command +ms.date: 11/13/2026 +ms.topic: reference +title: dsc plugin list +--- + +# dsc plugin list + +## Synopsis + +Retrieves the list of available DSC plug-ins with an optional filter. + +## Syntax + +```sh +dsc plugin list [Options] +``` + +## Description + +The `list` subcommand searches for available DSC command plug-ins and returns their information. +DSC discovers plug-ins by searching the directories it searches for resource and extension +manifests, which by default are the directory that contains `dsc` and the directories in the `PATH` +or [`DSC_RESOURCE_PATH`][01] environment variable. DSC reads the files whose name ends with +`.dsc.plugin.json`, `.dsc.plugin.yaml`, or `.dsc.plugin.yml`, and the `plugins` property of manifest +lists. + +DSC returns the plug-ins in the order it discovered them. That order decides which plug-in provides +a command when more than one plug-in defines the same command name: the plug-in DSC discovered first +provides the command, and the others are reported as shadowed. + +DSC doesn't return a plug-in when: + +- The manifest can't be loaded or isn't valid. DSC writes a warning that names the file. +- The `condition` of the manifest evaluates to `false`. +- The command name of the plug-in is the name or the alias of a built-in command, like `config`. + DSC writes a warning, because built-in commands always take precedence and the plug-in could + never run. +- Another manifest defines a higher version of the same plug-in type. + +When the `plugins.enabled` setting is `false`, DSC doesn't discover plug-ins and the command +returns nothing. + +If the command includes the `PLUGIN_NAME` argument, DSC filters the list of discovered plug-ins +before returning them. Filters are always applied after plug-in discovery. + +Listing plug-ins never runs a plug-in. + +## Examples + +### Example 1 - List all plug-ins + +Without any filters, the command returns every discovered DSC plug-in. + +```sh +dsc plugin list +``` + +```Output +Command Type Version Description +---------------------------------------------------------------------------------------------------- +lint Microsoft.DSC/Lint 3.4.0 Check configuration documents for problems without applying them +graph Contoso.DSC/Graph 0.1.0 Show the dependency graph of a configuration document +``` + +### Example 2 - List plug-ins with a matching type name + +When the `PLUGIN_NAME` argument includes a wildcard, the command returns every plug-in with a +matching type name. + +```sh +dsc plugin list Contoso* +``` + +```Output +Command Type Version Description +------------------------------------------------------------------------------------- +graph Contoso.DSC/Graph 0.1.0 Show the dependency graph of a configuration document +``` + +### Example 3 - Find out which plug-in provides a command + +When two plug-ins define the same command name, the summary table marks the plug-in that doesn't +provide the command as shadowed. + +```sh +dsc plugin list +``` + +```Output +Command Type Version Description +-------------------------------------------------------------------------------------------------- +graph Contoso.DSC/Graph 0.1.0 Show the dependency graph of a configuration document +graph (shadowed) Tailspin.DSC/Graph 1.2.0 Render a configuration document as a diagram +``` + +To see where each plug-in is defined, return the data as JSON or YAML. The `shadowed` property +indicates whether the plug-in is shadowed and the `path` property is the path to its manifest. + +To run a shadowed plug-in, specify its type name instead of the command name, like +`dsc Tailspin.DSC/Graph`. + +```sh +dsc plugin list --output-format yaml +``` + +## Arguments + +### PLUGIN_NAME + +Specifies an optional filter to apply for the type names of discovered DSC plug-ins. The filter can +include wildcards (`*`). The filter isn't case-sensitive. + +When this argument is specified, DSC filters the results to include only plug-ins where the plug-in +type name matches the filter. + +For example, specifying the filter `Microsoft.*` returns only the plug-ins published by Microsoft. +Specifying the filter `*Lint*` returns any plug-in with the string `Lint` in its type name, +regardless of the casing. + +The filter applies to the type name of the plug-in, not to the name of the command it provides. + +The value for this argument must be either a valid [fully qualified type name][02] or a valid +wildcard type name. + +```yaml +Type : string +Mandatory : false +``` + +## Options + +### -o, --output-format + + + + +The `--output-format` option controls which format DSC uses for the data the command returns. The +available formats are: + +- `json` to emit the data as a [JSON Line][03]. +- `pretty-json` to emit the data as JSON with newlines, indentation, and spaces for readability. +- `yaml` to emit the data as YAML. +- `table-no-truncate` to emit the data as a summary table without truncating each line to the + current console width. + +The default output format depends on whether DSC detects that the output is being redirected or +captured as a variable: + +- If the command isn't being redirected or captured, DSC displays the output as a summary table + described in the [Output](#output) section of this document. +- If the command output is redirected or captured, DSC emits the data as the `json` format to + stdout. + +When you use this option, DSC uses the specified format regardless of whether the command is being +redirected or captured. + +```yaml +Type : string +Mandatory : false +ValidValues : [json, pretty-json, yaml, table-no-truncate] +LongSyntax : --output-format +ShortSyntax : -o +``` + +### -h, --help + + + + +Displays the help for the current command or subcommand. When you specify this option, the +application ignores all other options and arguments. + +```yaml +Type : boolean +Mandatory : false +LongSyntax : --help +ShortSyntax : -h +``` + +## Output + +This command returns a formatted array containing an object for each plug-in that includes the +plug-in's type, version, command name, manifest, and other metadata. For more information, see +[dsc plugin list result schema][04]. + +If the output of the command isn't captured or redirected, it displays in the console by default as +a summary table for the returned plug-ins. The summary table includes the following columns, +displayed in the listed order: + +- **Command** - The name of the command the plug-in defines, which you specify after `dsc` to run + the plug-in. The name is followed by `(shadowed)` when a plug-in that DSC discovered earlier + provides the same command. +- **Type** - The fully qualified type name of the plug-in. +- **Version** - The semantic version of the plug-in. +- **Description** - The short description of the command. + +For more information about the formatting of the output data, see the +[--output-format option](#--output-format). + + +[01]: ../index.md#dsc_resource_path +[02]: ../../schemas/definitions/resourceType.md +[03]: https://jsonlines.org/ +[04]: ../../schemas/outputs/plugin/list.md diff --git a/docs/reference/cli/schema/index.md b/docs/reference/cli/schema/index.md index 1de0753b5..6968ab157 100644 --- a/docs/reference/cli/schema/index.md +++ b/docs/reference/cli/schema/index.md @@ -189,6 +189,7 @@ schema the application returns: - `get-result` ([reference documentation][ai]) - Represents the output from the `dsc resource get` command. - `include` ([reference documentation][aj]) - represents the instance schema for the built-in `Microsoft.DSC/Include` importer resource. - `manifest-list` ([reference documentation][ak]) +- `plugin-manifest` ([reference documentation][as]) - Validates a plug-in's manifest. DSC skips a plug-in with an invalid manifest and writes a warning. - `resolve-result` ([reference documentation][al]) - Represents the resolved form of the configuration document an `importer` resource emits. - `resource` ([reference documentation][am]) - Represents an instance of a resource in a configuration document. - `resource-manifest` ([reference documentation][an]) - Validates a command resource's manifest. If the manifest is invalid, DSC raises an error. @@ -211,6 +212,7 @@ ValidValues: [ get-result include manifest-list + plugin-manifest resolve-result resource-manifest restart-required @@ -297,3 +299,4 @@ For more information about the formatting of the output data, see the [ap]: ../../schemas/outputs/resource/set.md [aq]: ../../schemas/outputs/resource/test.md [ar]: https://jsonlines.org/ +[as]: ../../schemas/plugin/manifest/root.md diff --git a/docs/reference/schemas/outputs/lint/result.md b/docs/reference/schemas/outputs/lint/result.md new file mode 100644 index 000000000..d886f65af --- /dev/null +++ b/docs/reference/schemas/outputs/lint/result.md @@ -0,0 +1,96 @@ +--- +description: JSON schema reference for the data returned by the 'dsclint' command and the 'lint_dsc_config' MCP tool. +ms.date: 10/23/2026 +ms.topic: reference +title: dsclint result schema reference +--- + +# dsclint result schema reference + +## Synopsis + +The result output from the `dsclint` command and the `lint_dsc_config` tool of the DSC MCP server. + +## Metadata + +```yaml +SchemaDialect: https://json-schema.org/draft/2020-12/schema +SchemaID: https://raw.githubusercontent.com/PowerShell/DSC/main/schemas/v3.1.0/outputs/lint/result.json +Type: object +``` + +## Description + +The output from the `dsclint` command includes every diagnostic the rules reported, counts of the +diagnostics by level, whether the document is free of error-level diagnostics, and whether +resource discovery ran. The command always emits this object, including when the document doesn't +parse. The `dsc schema --type lint-result` command returns this schema. + +## Required properties + +The output always includes these properties: + +- [diagnostics](#diagnostics) +- [summary](#summary) +- [valid](#valid) +- [discovery](#discovery) + +## Properties + +### diagnostics + +The problems the rules reported, ordered by file, path, and rule. Each entry is an object with the +following properties: + +- `level` is the severity of the problem: `error`, `warning`, or `info`. +- `rule` is the identifier of the rule that reported the problem, such as `duplicate-resource`. +- `category` is the area of the document the rule inspects: `document`, `expression`, `parameter`, + `dependency`, `deprecation`, `environment`, or `resource`. +- `message` describes the problem. +- `path` is the JSON pointer ([RFC 6901][01]) to the location of the problem in the document. An + empty string identifies the document root. +- `file` is the file the document was read from. The property is omitted for documents passed + with `--input` or from stdin. +- `location` is the position of the problem in the source text as an object with `line` and + `column` properties, both starting at `1`. For a `path` the document doesn't spell out, like a + property that is missing, the location is that of the closest enclosing value. The property is + omitted when the linter is given an already parsed document, as the MCP server does. + +```yaml +Type: array +Required: true +``` + +### summary + +Counts of the diagnostics by level, as an object with the `errors`, `warnings`, and `info` +properties. + +```yaml +Type: object +Required: true +``` + +### valid + +Whether no error-level diagnostics were reported. This property doesn't depend on the `--fail-on` +option. + +```yaml +Type: boolean +Required: true +``` + +### discovery + +Whether resource discovery ran. When `false`, the rules that inspect installed resources were +skipped, either because of the `--no-discovery` option, because the document sets +`directives.resourceDiscovery` to `duringDeployment`, or because discovery failed. + +```yaml +Type: boolean +Required: true +``` + + +[01]: https://www.rfc-editor.org/rfc/rfc6901 diff --git a/docs/reference/schemas/outputs/lint/rules.md b/docs/reference/schemas/outputs/lint/rules.md new file mode 100644 index 000000000..72e5a4e6b --- /dev/null +++ b/docs/reference/schemas/outputs/lint/rules.md @@ -0,0 +1,110 @@ +--- +description: JSON schema reference for the data returned by the 'dsclint --list-rules' command. +ms.date: 10/23/2026 +ms.topic: reference +title: dsclint rules schema reference +--- + +# dsclint rules schema reference + +## Synopsis + +The output from the `dsclint --list-rules` command. + +## Metadata + +```yaml +SchemaDialect: https://json-schema.org/draft/2020-12/schema +SchemaID: https://raw.githubusercontent.com/PowerShell/DSC/main/schemas/v3.1.0/outputs/lint/rules.json +Type: object +``` + +## Description + +The `dsclint --list-rules` command returns an array of objects with this schema, one per rule, +sorted by rule identifier. Each object describes the rule and the level it reports at after the +settings file and any `--rule` overrides are applied. The `dsc schema --type lint-rules` command +returns this schema. + +## Required properties + +Each entry always includes these properties: + +- [id](#id) +- [category](#category) +- [tier](#tier) +- [defaultLevel](#defaultlevel) +- [level](#level) +- [fixed](#fixed) +- [description](#description) + +## Properties + +### id + +The identifier of the rule, such as `duplicate-resource`. + +```yaml +Type: string +Required: true +``` + +### category + +The area of the document the rule inspects: `document`, `expression`, `parameter`, `dependency`, +`deprecation`, `environment`, or `resource`. + +```yaml +Type: string +Required: true +``` + +### tier + +When the rule can run: `static` rules only need the document, `resource` rules need the installed +resources and only run when discovery is enabled. + +```yaml +Type: string +Required: true +ValidValues: [static, resource] +``` + +### defaultLevel + +The level the rule reports at unless it is overridden: `off`, `info`, `warning`, or `error`. + +```yaml +Type: string +Required: true +ValidValues: [off, info, warning, error] +``` + +### level + +The level the rule reports at in this run, after the settings file and any `--rule` overrides are +applied. + +```yaml +Type: string +Required: true +ValidValues: [off, info, warning, error] +``` + +### fixed + +Whether the level of the rule can't be changed. + +```yaml +Type: boolean +Required: true +``` + +### description + +What the rule checks. + +```yaml +Type: string +Required: true +``` diff --git a/docs/reference/schemas/outputs/plugin/list.md b/docs/reference/schemas/outputs/plugin/list.md new file mode 100644 index 000000000..8977c2b18 --- /dev/null +++ b/docs/reference/schemas/outputs/plugin/list.md @@ -0,0 +1,183 @@ +--- +description: JSON schema reference for the data returned by the 'dsc plugin list' command. +ms.date: 11/13/2026 +ms.topic: reference +title: dsc plugin list result schema reference +--- + +# dsc plugin list result schema reference + +## Synopsis + +The result output from the `dsc plugin list` command. + +## Metadata + +```yaml +SchemaDialect: https://json-schema.org/draft/2020-12/schema +SchemaID: https://raw.githubusercontent.com/PowerShell/DSC/main/schemas/v3.4.0/outputs/plugin/list.json +Type: object +``` + +## Description + +The output from the `dsc plugin list` command includes a representation of discovered DSC command +plug-ins as a series of [JSON Lines][01]. This schema describes the JSON object returned for each +plug-in. + +DSC returns the plug-ins in the order it discovered them, including the plug-ins that are +[shadowed](#shadowed). + +## Examples + +### Example 1 - Plug-in that provides a command + +This example shows the output for a plug-in defined in the file `/opt/dsc/dsclint.dsc.plugin.json`. + +```json +{ + "type": "Microsoft.DSC/Lint", + "version": "3.4.0", + "command": "lint", + "description": "Check configuration documents for problems without applying them", + "deprecationMessage": null, + "path": "/opt/dsc/dsclint.dsc.plugin.json", + "directory": "/opt/dsc", + "shadowed": false, + "manifest": { + "$schema": "https://aka.ms/dsc/schemas/v3/bundled/plugin/manifest.json", + "type": "Microsoft.DSC/Lint", + "version": "3.4.0", + "description": "Check configuration documents for problems without applying them", + "command": { + "name": "lint", + "executable": "dsclint" + } + } +} +``` + +## Required properties + +Each plug-in in the output always includes these properties: + +- [type](#type) +- [version](#version) +- [command](#command) +- [description](#description) +- [deprecationMessage](#deprecationmessage) +- [path](#path) +- [directory](#directory) +- [shadowed](#shadowed) +- [manifest](#manifest) + +## Properties + +### type + +Identifies the fully qualified type name of the plug-in. Plug-in type names use the same syntax as +resource type names. For more information, see +[DSC Resource fully qualified type name schema reference][02]. + +```yaml +Type: string +Required: true +Pattern: ^\w+(\.\w+)*\/\w+$ +``` + +### version + +Represents the current version of the plug-in as a valid semantic version (SemVer) string. When DSC +discovers more than one manifest with the same type, it only returns the plug-in with the highest +version. + +```yaml +Type: string +Required: true +Pattern: ^(0|[1-9]\d*)\.(0|[1-9]\d*)\.(0|[1-9]\d*)(?:-((?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?(?:\+([0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?$ +``` + +### command + +Indicates the name of the command the plug-in adds to DSC. For example, a plug-in with the command +`lint` is invoked as `dsc lint`. + +```yaml +Type: string +Required: true +Pattern: ^[a-z][a-z0-9]*(-[a-z0-9]+)*$ +``` + +### description + +Defines a short description of the command. The value is `null` when the manifest doesn't define a +description. + +```yaml +Type: [string, 'null'] +Required: true +``` + +### deprecationMessage + +Indicates that the plug-in is deprecated. The value is `null` when the plug-in isn't deprecated. +When the value is a string, DSC writes the message as a warning each time the command is invoked. + +```yaml +Type: [string, 'null'] +Required: true +``` + +### path + +Represents the path to the file that defines the plug-in manifest. For a plug-in defined in a +manifest list, this is the path to the manifest list. + +```yaml +Type: string +Required: true +``` + +### directory + +Represents the path to the folder that contains the plug-in manifest. DSC resolves the +manifest-relative path arguments of the plug-in against this folder and passes it to the plug-in in +the `DSC_PLUGIN_DIR` environment variable. + +```yaml +Type: string +Required: true +``` + +### shadowed + +Indicates whether another plug-in that DSC discovered earlier provides the same command. DSC never +runs a shadowed plug-in for its command name. + +Plug-ins are shadowed when plug-ins with different types define the same command name. DSC searches +the directories for manifests in a fixed order, and the plug-in in the first directory provides the +command. When both plug-ins are in the same directory, the plug-in whose type name sorts first +provides the command. + +To run a shadowed plug-in, specify its type name instead of the command name, like +`dsc OpenDSC/Lint`. + +```yaml +Type: boolean +Required: true +``` + +### manifest + +Represents the values defined in the plug-in manifest that defines the plug-in. For more +information on the value for this property, see [DSC plug-in manifest schema reference][03]. + +```yaml +Type: object +Required: true +``` + + +[01]: https://jsonlines.org/ +[02]: ../../definitions/resourceType.md +[03]: ../../plugin/manifest/root.md diff --git a/docs/reference/schemas/overview.md b/docs/reference/schemas/overview.md index 632d00182..39f93b682 100644 --- a/docs/reference/schemas/overview.md +++ b/docs/reference/schemas/overview.md @@ -51,6 +51,13 @@ can define: - [DSC extension manifest discover property schema reference][11] - [DSC extension discover operation stdout schema reference][12] +## Plug-in schemas + +The article [DSC plug-in manifest schema reference][59] describes the root JSON schema for plug-in +manifests. The following article describes the subschema for the command a plug-in defines: + +- [DSC plug-in manifest command property schema reference][60] + ## Resource schemas The article [DSC command resource manifest schema reference][13] describes the root JSON schema for @@ -104,6 +111,7 @@ DSC returns for its commands: | `dsc config test` | [dsc config test result schema reference][43] | | `dsc extension list` | [dsc extension list result schema reference][44] | | `dsc function list` | [dsc function list result schema reference][45] | +| `dsc plugin list` | [dsc plugin list result schema reference][61] | | `dsc resource get` | [dsc resource get result schema reference][46] | | `dsc resource list` | [dsc resource list result schema reference][47] | | `dsc resource set` | [dsc resource set result schema reference][48] | @@ -189,3 +197,6 @@ throughout DSC. [56]: ./config/function.md [57]: ./config/output.md [58]: ./config/executionInformation.md +[59]: ./plugin/manifest/root.md +[60]: ./plugin/manifest/command.md +[61]: ./outputs/plugin/list.md diff --git a/docs/reference/schemas/plugin/manifest/command.md b/docs/reference/schemas/plugin/manifest/command.md new file mode 100644 index 000000000..ff2938c31 --- /dev/null +++ b/docs/reference/schemas/plugin/manifest/command.md @@ -0,0 +1,147 @@ +--- +description: JSON schema reference for the 'command' property in a DSC plug-in manifest +ms.date: 11/13/2026 +ms.topic: reference +title: DSC plug-in manifest command property schema reference +--- + +# DSC plug-in manifest command property schema reference + +## Synopsis + +Defines the command a plug-in adds to DSC and how DSC runs it. + +## Metadata + +```yaml +SchemaDialect: https://json-schema.org/draft/2020-12/schema +SchemaID: https://raw.githubusercontent.com/PowerShell/DSC/main/schemas/v3.4.0/plugin/manifest.command.json +Type: object +``` + +## Description + +The `command` property of a plug-in manifest defines the name of the command the plug-in adds to +DSC and the executable that implements it. + +When the user specifies the command name as the first argument to `dsc`, DSC: + +1. Resolves the [executable](#executable). +1. Starts the executable in the current working directory with the standard input, output, and + error streams of `dsc`. DSC passes the arguments defined in [args](#args), followed by every + argument the user specified after the command name, exactly as the user specified them. +1. Waits for the executable to exit and exits with the same exit code. + +DSC sets environment variables that tell the plug-in how it was invoked, like `DSC_EXE`, +`DSC_VERSION`, `DSC_PLUGIN_COMMAND`, and `DSC_PLUGIN_DIR`. For the full list, see +[Plug-in commands][01]. + +A plug-in command must display its help and exit with code `0` when it receives the `--help` +option. DSC passes that option when the user requests help with `dsc help `. + +## Required properties + +The `command` definition must include these properties: + +- [name](#name) +- [executable](#executable) + +## Properties + +### name + +The `name` property defines the name of the command, which the user specifies after `dsc`. For +example, a plug-in with the command name `lint` is invoked as `dsc lint`. + +A command name must start with a lowercase ASCII letter and can only contain lowercase ASCII +letters, ASCII digits, and hyphens. It can't end with a hyphen or contain consecutive hyphens. DSC +compares command names case-sensitively. + +The name can't be the name or the alias of a built-in command, like `config`, `resource`, or +`help`. DSC always matches built-in commands first. When it discovers a plug-in that uses the name +of a built-in command, it writes a warning and skips the plug-in. + +When plug-ins with different [types][02] define the same command name, the plug-in DSC discovers +first provides the command. The `dsc plugin list` command reports the others as shadowed. A +shadowed plug-in can still be run by its type name, like `dsc OpenDSC/Lint`. + +A manifest defines exactly one command. A plug-in can implement nested commands, like +`dsc package build`, by parsing the arguments DSC passes to it. + +```yaml +Type: string +Required: true +Pattern: ^[a-z][a-z0-9]*(-[a-z0-9]+)*$ +``` + +### executable + +The `executable` property defines the name of the executable to run for the command. The value must +be one of the following: + +- The name of an executable discoverable in the system's `PATH` environment variable. +- The name of an executable in the same directory as the manifest. +- The full path to the executable. + +A file extension is only required when the operating system doesn't recognize the file as an +executable without it. + +To implement a plug-in as a script, define the interpreter as the executable and pass the script +in [args](#args) as a [manifest-relative path argument](#manifest-relative-path-argument). + +When the `resourcePath.restricted` setting is `true`, the executable must be in one of the +directories defined by the `resourcePath.directories` setting. + +```yaml +Type: string +Required: true +``` + +### args + +The `args` property defines the list of arguments to pass to the executable before the arguments +the user specified after the command name. Each item in the array can be a string representing a +static argument or a [manifest-relative path argument](#manifest-relative-path-argument) object. + +```yaml +Type: array +Required: false +ItemsType: [string, object(Manifest-relative path argument)] +``` + +#### String arguments + +Any item in the argument array can be a string representing a static argument to pass to the +executable, like `-NoProfile` or `-File`. DSC passes the string as written. + +```yaml +Type: string +``` + +#### Manifest-relative path argument + +Defines an argument that is the path to a file that ships with the plug-in. DSC replaces the item +with the absolute path to the file, resolved against the directory that contains the manifest. + +DSC runs a plug-in in the current working directory of the user, not in the directory that contains +the manifest, so a relative path defined as a string argument wouldn't resolve to a file next to +the manifest. Use this argument to pass such a file, like the script that implements the plug-in. + +A manifest-relative path argument is defined as a JSON object with the following properties: + +- `manifestRelativePath` (required) - The path to the file, relative to the directory that contains + the manifest, like `graph.ps1` or `scripts/graph.py`. The value can't be empty, an absolute path, + or a path that starts with a drive or a root. DSC doesn't load a manifest that defines such a + path. + +The object must not define any other properties. + +```yaml +Type: object +RequiredProperties: [manifestRelativePath] +AdditionalProperties: false +``` + + +[01]: ../../../cli/index.md#plug-in-commands +[02]: ./root.md#type diff --git a/docs/reference/schemas/plugin/manifest/root.md b/docs/reference/schemas/plugin/manifest/root.md new file mode 100644 index 000000000..1f898330b --- /dev/null +++ b/docs/reference/schemas/plugin/manifest/root.md @@ -0,0 +1,238 @@ +--- +description: JSON schema reference for a DSC plug-in manifest +ms.date: 11/13/2026 +ms.topic: reference +title: DSC plug-in manifest schema reference +--- + +# DSC plug-in manifest schema reference + +## Synopsis + +The data file that defines a DSC command plug-in. + +## Metadata + +```yaml +SchemaDialect: https://json-schema.org/draft/2020-12/schema +SchemaID: https://raw.githubusercontent.com/PowerShell/DSC/main/schemas/v3.4.0/plugin/manifest.json +Type: object +``` + +## Description + +A command plug-in adds a top-level command to DSC without being part of the DSC executable. Every +plug-in must have a manifest that defines: + +- The type name and version that identify the plug-in. +- The name of the command the plug-in adds to DSC. +- The executable DSC runs for the command, and any arguments DSC passes to it before the arguments + the user specified. + +When the first argument to `dsc` isn't a built-in command, DSC runs the plug-in that defines a +command with that name. For more information about how DSC runs a plug-in command, see +[Plug-in commands][01]. + +DSC discovers plug-in manifests in the same directories as resource and extension manifests. The +file name of a plug-in manifest must end with one of the following suffixes: + +- `.dsc.plugin.json` +- `.dsc.plugin.yaml` +- `.dsc.plugin.yml` + +You can also define plug-ins in the `plugins` property of a manifest list, a file whose name ends +with `.dsc.manifests.json`, `.dsc.manifests.yaml`, or `.dsc.manifests.yml`. The value of the +property is an array of plug-in manifests. + +DSC doesn't load a plug-in manifest that isn't valid for this schema. It writes a warning that names +the file when it discovers plug-ins. + +The rest of this document describes the manifest's schema. + +## Examples + +### Example 1 - Plug-in implemented as an executable + +This manifest defines the `lint` command, implemented by the `dsclint` executable. DSC finds the +executable in the `PATH` environment variable or in the same directory as the manifest. + +```json +{ + "$schema": "https://aka.ms/dsc/schemas/v3/bundled/plugin/manifest.json", + "type": "Microsoft.DSC/Lint", + "version": "3.4.0", + "description": "Check configuration documents for problems without applying them", + "command": { + "name": "lint", + "executable": "dsclint" + } +} +``` + +### Example 2 - Plug-in implemented as a script + +This manifest defines the `graph` command, implemented by a PowerShell script in the same directory +as the manifest. The `condition` makes the plug-in available only when `pwsh` is installed. The +`manifestRelativePath` argument passes the full path to the script, because DSC runs a plug-in in +the working directory of the user, not in the directory that contains the manifest. + +```json +{ + "$schema": "https://aka.ms/dsc/schemas/v3/bundled/plugin/manifest.json", + "type": "Contoso.DSC/Graph", + "version": "0.1.0", + "description": "Show the dependency graph of a configuration document", + "condition": "[not(equals(tryWhich('pwsh'), null()))]", + "command": { + "name": "graph", + "executable": "pwsh", + "args": [ + "-NoLogo", + "-NoProfile", + "-NonInteractive", + "-File", + { "manifestRelativePath": "graph.ps1" } + ] + } +} +``` + +## Required properties + +The manifest must include these properties: + +- [$schema](#schema) +- [type](#type) +- [version](#version) +- [command](#command) + +## Properties + +### $schema + +The `$schema` property indicates the canonical URI of this schema that the manifest validates +against. This property is mandatory. DSC uses this value to validate the manifest against the +correct JSON schema and doesn't load a manifest with a URI it doesn't recognize. + +For every version of the schema, there are three valid urls: + +- `.../plugin/manifest.json` + + The URL to the canonical non-bundled schema. When it's used for validation, the validating client + needs to retrieve this schema and every schema it references. + +- `.../bundled/plugin/manifest.json` + + The URL to the bundled schema. When it's used for validation, the validating client only needs to + retrieve this schema. + +- `.../bundled/plugin/manifest.vscode.json` + + The URL to the enhanced authoring schema. This schema is much larger than the other schemas, as + it includes additional definitions that provide contextual help and snippets that the others + don't include. + +```yaml +Type: string +Required: true +Format: URI +``` + +### type + +The `type` property represents the fully qualified type name of the plug-in. It identifies the +plug-in. When DSC discovers more than one manifest with the same type, it uses the manifest with +the highest [version](#version). + +The type name of a plug-in uses the same syntax as the type name of a resource. For more +information, see [DSC Resource fully qualified type name schema reference][02]. + +```yaml +Type: string +Required: true +Pattern: ^\w+(\.\w+)*\/\w+$ +``` + +### version + +The `version` property must be the current version of the plug-in as a valid semantic version +(semver) string. + +```yaml +Type: string +Required: true +Pattern: ^(0|[1-9]\d*)\.(0|[1-9]\d*)\.(0|[1-9]\d*)(?:-((?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?(?:\+([0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?$ +``` + +### condition + +The `condition` property defines a [DSC expression][03] that must evaluate to a boolean. When the +expression evaluates to `false`, DSC treats the plug-in as not installed. The command isn't listed +in the help or by the `dsc plugin list` command and can't be invoked. + +Use a condition for a plug-in that depends on a tool that might not be installed, like in +[Example 2](#example-2---plug-in-implemented-as-a-script). + +```yaml +Type: string +Required: false +``` + +### deprecationMessage + +The `deprecationMessage` property defines a message that indicates the plug-in is deprecated. When +this property is defined, DSC writes the message as a warning each time the command is invoked. + +```yaml +Type: string +Required: false +``` + +### description + +The `description` property defines a one-line description of the command. DSC displays the +description next to the command in the output of `dsc --help` and `dsc plugin list`, and in the +completion scripts that `dsc completer` generates. + +```yaml +Type: string +Required: false +``` + +### tags + +The `tags` property defines a list of searchable terms for the plug-in. + +```yaml +Type: array +Required: false +ItemsMustBeUnique: true +ItemsType: string +ItemsPattern: ^\w+$ +``` + +### command + +The `command` property defines the command the plug-in adds to DSC and how DSC runs it. For more +information, see [DSC plug-in manifest command property schema reference][04]. + +```yaml +Type: object +Required: true +``` + +### metadata + +The `metadata` property defines arbitrary data for the plug-in. DSC doesn't interpret the metadata. +It's returned with the manifest in the output of the `dsc plugin list` command. + +```yaml +Type: object +Required: false +``` + + +[01]: ../../../cli/index.md#plug-in-commands +[02]: ../../definitions/resourceType.md +[03]: ../../config/functions/overview.md +[04]: ./command.md diff --git a/docs/reference/tools/builtin.md b/docs/reference/tools/builtin.md index 0612dad88..4d1b57619 100644 --- a/docs/reference/tools/builtin.md +++ b/docs/reference/tools/builtin.md @@ -2,7 +2,7 @@ description: >- Lists the tools that ship with DSC, their purpose, and links to the reference documentation for each tool. -ms.date: 03/25/2025 +ms.date: 10/23/2026 ms.topic: reference title: osinfo --- @@ -15,10 +15,20 @@ feedback and functionality. The following table describes the tools included in current releases of DSC and the platforms those tools are available for: -| Executable | Platforms | Description | -|:-----------|:----------------------|:--------------------------------------------------------| -| `osinfo` | Linux, macOS, Windows | Returns information about the operating system as JSON. | -| `registry` | Windows | Manages registry keys and values. | +| Executable | Platforms | Description | +|:-----------|:----------------------|:-------------------------------------------------------------------| +| `dsclint` | Linux, macOS, Windows | Checks configuration documents for problems without applying them. | +| `osinfo` | Linux, macOS, Windows | Returns information about the operating system as JSON. | +| `registry` | Windows | Manages registry keys and values. | + +## dsclint + +The `dsclint` command checks a DSC configuration document for problems without applying it. It +reports duplicate resources, invalid expressions, unresolved dependencies, invalid parameters, +deprecated features, and, with resource discovery, resource types that aren't installed. `dsclint` +is available in the DSC release for all supported platforms. + +For more information about the command, see [dsclint command reference][05]. ## osinfo @@ -43,3 +53,4 @@ more information about the resource, see [Microsoft.Windows/Registry][04]. [02]: ../resources/Microsoft/OSInfo/index.md [03]: ./registry/index.md [04]: ../resources/Microsoft/Windows/Registry/index.md +[05]: ./dsclint/index.md diff --git a/docs/reference/tools/dsclint/index.md b/docs/reference/tools/dsclint/index.md new file mode 100644 index 000000000..b3ae54159 --- /dev/null +++ b/docs/reference/tools/dsclint/index.md @@ -0,0 +1,521 @@ +--- +description: Command line reference for the 'dsclint' command +ms.date: 10/23/2026 +ms.topic: reference +title: dsclint command reference +--- + +# dsclint command reference + +## Synopsis + +Checks a configuration document for problems without applying it. + +## Syntax + +### Configuration document from file + +```sh +dsclint [Options] --file +``` + +### Configuration document from option string + +```sh +dsclint [Options] --input +``` + +### Configuration document from stdin + +```sh +cat | dsclint [Options] --file - +``` + +### Several configuration documents + +```sh +dsclint [Options] --file --file +``` + +### List the rules + +```sh +dsclint --list-rules [--output-format ] +``` + +## Description + +The `dsclint` command analyzes a configuration document and reports problems as diagnostics. It +never invokes a resource operation, so it's safe to run on any machine, including CI agents. +`dsclint` ships with DSC and is installed alongside the `dsc` command. The same linter is available +to agents as the `lint_dsc_config` tool of the [DSC MCP server][05]. + +The command works in stages: + +1. It parses the document as JSON or YAML. A document that doesn't parse is reported with the + `parse-error` rule. +1. It validates the document against the configuration document JSON schema. Every violation is + reported with the `schema-invalid` rule and the JSON pointer to the problem. +1. It runs the static rules, which only need the document. These rules find duplicate resources, + expressions that don't parse or call unknown functions, undeclared parameters and variables, + unresolved and circular dependencies, invalid parameter constraints, deprecated features, and + more. +1. Unless you specify [--no-discovery](#--no-discovery), it discovers the resource types the + document uses and runs the resource rules, which report types that aren't installed, deprecated + types, and literal properties that don't match the resource's schema. + +Each diagnostic has a level: `error`, `warning`, or `info`. By default the command exits with code +`5` when it reports at least one error. Use [--fail-on](#--fail-on) to fail on warnings or +informational diagnostics instead. Use [--rule](#--rule) or the `lint` section of the DSC settings +file to change the level of a rule or turn it off. A few rules have a fixed level that can't be +changed: `parse-error`, `schema-invalid`, `unsupported-document-type`, and `discovery-failed`. + +For the list of rules and what each one checks, see [dsclint rules][01]. + +## Examples + +### Example 1 - Lint a configuration document + + + +The command reports a duplicate resource name and exits with code `5`. In the console, each +diagnostic is rendered as a report that shows the problem in the document. Secure values in the +document, like the default value of a `secureString` parameter, are shown as ``. + +```yaml +# example.dsc.config.yaml +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +resources: +- name: EnsureTest1Value + type: Microsoft.DSC.Debug/Echo + properties: + output: one +- name: EnsureTest1Value + type: Microsoft.DSC.Debug/Echo + properties: + output: two +``` + +```sh +dsclint --file ./example.dsc.config.yaml +``` + +```text +duplicate-resource + + × Resource 'EnsureTest1Value' of type 'Microsoft.DSC.Debug/Echo' is declared more than once + ╭─[./example.dsc.config.yaml:7:3] + 5 │ properties: + 6 │ output: one + 7 │ - name: EnsureTest1Value + · ────────────────────── + 8 │ type: Microsoft.DSC.Debug/Echo + 9 │ properties: + ╰──── + help: Resource names must be unique among resources of the same type at the same level. + +errors: 1, warnings: 0, info: 0 +``` + +When the output is redirected, or when you specify [--output-format](#--output-format), the +command emits the result as data instead: + +```sh +dsclint --file ./example.dsc.config.yaml --output-format yaml +``` + +```yaml +diagnostics: +- level: error + rule: duplicate-resource + category: document + message: Resource 'EnsureTest1Value' of type 'Microsoft.DSC.Debug/Echo' is declared more than once + path: /resources/1 + file: ./example.dsc.config.yaml + location: + line: 7 + column: 3 +summary: + errors: 1 + warnings: 0 + info: 0 +valid: false +discovery: true +``` + +### Example 2 - Fail on warnings in a pipeline + + + +The command exits with code `5` when any warning or error is reported, so a pipeline step fails on +an unused parameter as well as on an error. Discovery is skipped because the build agent doesn't +have the resources installed. + +```sh +dsclint --file ./example.dsc.config.yaml --no-discovery --fail-on warning +``` + +### Example 3 - Change the level of a rule + + + +The command turns off the `no-unused-parameters` rule and raises `missing-require-version`, which +is off by default, to a warning. + +```sh +dsclint --file ./example.dsc.config.yaml --rule no-unused-parameters=off --rule missing-require-version=warning +``` + +### Example 4 - Check a document that targets another system root + + + +When you pass `--system-root`, the `system-root-unused` rule warns if no expression in the document +calls the `systemRoot()` function, because the document wouldn't target the specified root. + +```sh +dsclint --file ./example.dsc.config.yaml --system-root D:\ +``` + +### Example 5 - List the rules + + + +The command lists every rule with its category, the level it reports at, whether it needs resource +discovery, and a description. + +```sh +dsclint --list-rules +``` + +## Options + +### -i, --input + + + + +Specifies the configuration document to lint. + +The document must be a string containing a JSON or YAML object. + +This option is mutually exclusive with the `--file` option. + +```yaml +Type : string +Mandatory : false +LongSyntax : --input +ShortSyntax : -i +``` + +### -f, --file + + + + +Defines the path to a configuration document to lint. Specify the option more than once to lint +several documents in one run; the diagnostics then include the `file` property. + +You can also use this option to pass a configuration document from stdin with `-` as the path. + +This option is mutually exclusive with the `--input` option. + +```yaml +Type : string +Mandatory : false +LongSyntax : --file +ShortSyntax : -f +``` + +### -p, --parameters + + + + +Specifies the parameter values that would be passed to the configuration, as a JSON or YAML +object. When you specify parameters, the command checks that every supplied value matches a +declared parameter, that every parameter without a default value has a value, and that each value +satisfies the parameter's type and constraints. + +```yaml +Type : string +Mandatory : false +LongSyntax : --parameters +ShortSyntax : -p +``` + +### --parameters-file + + + +Defines the path to a JSON or YAML file with the parameter values. Use `-` to read the parameters +from stdin. Values from `--parameters` take precedence over values from the file. + +```yaml +Type : string +Mandatory : false +LongSyntax : --parameters-file +``` + +### -r, --system-root + + + + +Specifies the operating system root path the configuration would target. The command doesn't +check that the path exists; it only enables the `system-root-unused` rule. + +```yaml +Type : string +Mandatory : false +LongSyntax : --system-root +ShortSyntax : -r +``` + +### --no-discovery + + + +Skips resource discovery. The rules that inspect installed resources don't run, and the output +reports `discovery: false`. Use this option on machines that don't have the resources installed, +or to lint faster. + +```yaml +Type : boolean +Mandatory : false +LongSyntax : --no-discovery +``` + +### --fail-on + + + +Defines the lowest diagnostic level that makes the command exit with code `5`. The default is +`error`. + +```yaml +Type : string +Mandatory : false +DefaultValue : error +ValidValues : [error, warning, info] +LongSyntax : --fail-on +``` + +### --rule + + + +Overrides the level of a rule for this run, as `=`. The level is one of `off`, +`info`, `warning`, or `error`. Specify the option more than once to override several rules. The +command exits with code `1` for an unknown rule. + +Overrides from this option take precedence over the `lint` section of the DSC settings file. The +policy settings file takes precedence over this option. + +```yaml +Type : string +Mandatory : false +LongSyntax : --rule +``` + +### --list-rules + + + +Lists the rules with the level each reports at in this run instead of linting a document. When +the output isn't redirected, the command shows a table; otherwise it emits the list in the format +specified by `--output-format`. For the schema of the list, see [dsclint rules schema][03]. + +This option is mutually exclusive with the `--input` and `--file` options. + +```yaml +Type : boolean +Mandatory : false +LongSyntax : --list-rules +``` + +### -o, --output-format + + + + +The `--output-format` option controls which format `dsclint` uses for the data the command returns. +The available formats are: + +- `json` to emit the data as a [JSON Line][02]. +- `pretty-json` to emit the data as JSON with newlines, indentation, and spaces for readability. +- `yaml` to emit the data as YAML. + +The default output depends on whether `dsclint` detects that the output is being redirected or +captured as a variable: + +- If the command isn't being redirected or captured, `dsclint` renders each diagnostic in the + console as a report that shows the problem in the document text, with the rule identifier as the + code and what the rule requires as the help, followed by the counts of diagnostics by level. +- If the command output is redirected or captured, `dsclint` emits the data as the `json` format to + stdout. + +Specify a format to get the data in the console too. + +```yaml +Type : string +Mandatory : false +ValidValues : [json, pretty-json, yaml] +LongSyntax : --output-format +ShortSyntax : -o +``` + +### -l, --trace-level + + + + +Defines the minimum message level `dsclint` should emit on stderr during a run. The levels are the +same as for the `dsc` command, from highest to lowest: `error`, `warn` (default), `info`, `debug`, +and `trace`. + +You can also set the trace level with the `DSC_TRACE_LEVEL` environment variable or the `tracing` +section of the DSC settings file. The option takes precedence over both unless the policy settings +file defines the tracing setting. + +```yaml +Type : string +Mandatory : false +DefaultValue : warn +ValidValues : [error, warn, info, debug, trace] +LongSyntax : --trace-level +ShortSyntax : -l +``` + +### -t, --trace-format + + + + +Defines the output format to use when emitting trace messages on stderr: + +- `default` - Emits the message with ANSI console coloring for the timestamp, message level, and + line number. +- `plaintext` - As `default` but without any console colors. +- `json` - Emits each message as a compressed JSON object with the timestamp, level, message, and + line number as properties. + +```yaml +Type : string +Mandatory : false +DefaultValue : default +ValidValues : [default, plaintext, json] +LongSyntax : --trace-format +ShortSyntax : -t +``` + +### --progress-format + + + +Defines the progress format to use when emitting progress messages on stderr during resource +discovery. This option has no short form because `-p` is used by [--parameters](#--parameters). + +- `default` - Shows a progress bar if `dsclint` detects that it's being called interactively. + Otherwise, no progress is shown. +- `none` - Doesn't show any progress. +- `json` - Emits progress as compressed JSON objects with the timestamp, level, message, and line + number as properties. + +```yaml +Type : string +Mandatory : false +DefaultValue : default +ValidValues : [default, none, json] +LongSyntax : --progress-format +``` + +### --ignore-settings-file + + + +Ignores the DSC settings file for this run. The `lint` and `tracing` sections of the settings file +aren't applied, so every rule reports at its default level unless you override it with +[--rule](#--rule). + +```yaml +Type : boolean +Mandatory : false +LongSyntax : --ignore-settings-file +``` + +### -V, --version + + + + +Displays the version of the application. When you specify this option, the application ignores all +options and arguments except for [--help](#--help), which overrides this option. + +```yaml +Type : boolean +Mandatory : false +LongSyntax : --version +ShortSyntax : -V +``` + +### -h, --help + + + + +Displays the help for the command. When you specify this option, the application ignores all other +options and arguments. + +```yaml +Type : boolean +Mandatory : false +LongSyntax : --help +ShortSyntax : -h +``` + +## Output + +This command returns the diagnostics for the document, counts of the diagnostics by level, whether +the document has no errors, and whether resource discovery ran. The command always writes the +result to stdout, including when the document doesn't parse. In the console it renders the +diagnostics as reports; otherwise, or when you specify [--output-format](#--output-format), it +emits the result as data. For more information, see [dsclint result schema][04]. + +The `dsc schema` command returns the JSON schemas for the output as the `lint-result` and +`lint-rules` schema types. + +## Exit codes + +| Exit Code | Meaning | +| :-------: | :------------------------------------------------------------------------------- | +| `0` | No diagnostic at or above the `--fail-on` level was reported. | +| `1` | The command received invalid arguments, such as an unknown rule. | +| `2` | The command failed for another reason, such as an unreadable settings file. | +| `4` | A document or the parameters couldn't be read. | +| `5` | At least one diagnostic at or above the `--fail-on` level was reported. | + +## Settings + +`dsclint` reads the DSC settings file (`dsc.settings.json`) and the policy settings file from the +folder that contains the `dsclint` executable, which is the folder `dsc` is installed in. The +`lint` section of the settings file sets the level of rules for every run. The same section in the +policy settings file takes precedence over the `--rule` option. + +```json +{ + "lint": { + "rules": { + "no-unused-parameters": { "level": "off" }, + "missing-require-version": { "level": "warning" } + } + } +} +``` + + +[01]: rules.md +[02]: https://jsonlines.org/ +[03]: ../../schemas/outputs/lint/rules.md +[04]: ../../schemas/outputs/lint/result.md +[05]: ../../cli/server/index.md diff --git a/docs/reference/tools/dsclint/rules.md b/docs/reference/tools/dsclint/rules.md new file mode 100644 index 000000000..e918b37bb --- /dev/null +++ b/docs/reference/tools/dsclint/rules.md @@ -0,0 +1,369 @@ +--- +description: Reference for the rules the 'dsclint' command checks configuration documents against +ms.date: 10/23/2026 +ms.topic: reference +title: dsclint rules +--- + +# dsclint rules + +The `dsclint` command checks a configuration document against the rules on this page. Each rule +has an identifier you can use with the `--rule` option or in the `lint` section of the settings +file, a category, a default level, and a tier: + +- `static` rules only need the document. +- `resource` rules need the installed resources and only run when discovery is enabled. + +Rules with a fixed level can't be changed or turned off. + +## Rules at a glance + +| Rule | Category | Default level | Tier | +| :------------------------------- | :---------- | :------------ | :------- | +| `parse-error` | document | error (fixed) | static | +| `schema-invalid` | document | error (fixed) | static | +| `unsupported-document-type` | document | error (fixed) | static | +| `duplicate-resource` | document | error | static | +| `duplicate-resource-name` | document | warning | static | +| `empty-resources` | document | warning | static | +| `unknown-microsoft-dsc-metadata` | document | warning | static | +| `api-version-alias` | document | info | static | +| `date-version` | document | warning | static | +| `missing-require-version` | document | off | static | +| `invalid-expression` | expression | error | static | +| `unknown-function` | expression | error | static | +| `invalid-argument-count` | expression | error | static | +| `invalid-argument-type` | expression | error | static | +| `undefined-parameter` | expression | error | static | +| `undefined-variable` | expression | error | static | +| `condition-not-expression` | expression | warning | static | +| `no-unused-parameters` | parameter | warning | static | +| `no-unused-variables` | parameter | warning | static | +| `parameter-default-type` | parameter | error | static | +| `invalid-parameter-constraints` | parameter | error | static | +| `secure-parameter-default` | parameter | warning | static | +| `prefer-secure-type` | parameter | warning | static | +| `secure-value-in-output` | parameter | warning | static | +| `use-parameter-description` | parameter | off | static | +| `undeclared-parameter-value` | parameter | error | static | +| `missing-parameter-value` | parameter | error | static | +| `parameter-value-type` | parameter | error | static | +| `unresolved-dependency` | dependency | error | static | +| `circular-dependency` | dependency | error | static | +| `reference-without-depends-on` | dependency | warning | static | +| `duplicate-depends-on` | dependency | info | static | +| `copy-loop-deprecated` | deprecation | warning | static | +| `copy-options-unsupported` | deprecation | error | static | +| `security-context-in-metadata` | deprecation | warning | static | +| `security-context-conflict` | deprecation | error | static | +| `dsc-version-unsatisfied` | deprecation | warning | static | +| `system-root-unused` | environment | warning | static | +| `hardcoded-system-path` | environment | off | static | +| `discovery-failed` | resource | warning (fixed) | resource | +| `unknown-resource-type` | resource | warning | resource | +| `unsatisfied-require-version` | resource | warning | resource | +| `deprecated-resource` | resource | warning | resource | +| `property-schema-mismatch` | resource | error | resource | +| `nested-resources-unsupported` | resource | warning | resource | +| `exist-false-unsupported` | resource | error | resource | +| `canonical-type-name` | resource | info | resource | + +## Document rules + +### parse-error + +The document must be valid JSON or YAML. The diagnostic includes the parser's message and, when +known, the line and column of the problem. An empty document is reported by this rule too. This +rule has a fixed level. + +### schema-invalid + +The document must validate against the configuration document JSON schema. The command reports +every violation with the JSON pointer to the problem, such as an unknown property, a missing +`$schema`, or a `dependsOn` entry that isn't a `resourceId()` expression. This rule has a fixed +level. + +### unsupported-document-type + +Only configuration documents can be linted. A document whose `$schema` identifies a resource or +extension manifest is rejected. This rule has a fixed level. + +### duplicate-resource + +Resource names must be unique among resources of the same type at the same level. DSC refuses to +run a document that declares the same resource twice. Types are compared without regard to case. + +```yaml +resources: +- name: Same + type: Microsoft.DSC.Debug/Echo +- name: Same # duplicate-resource + type: microsoft.dsc.debug/echo +``` + +### duplicate-resource-name + +Resources at the same level should have distinct names even when their types differ, because DSC +resolves dependencies by name first and then checks the type. + +### empty-resources + +The configuration should declare at least one resource. + +### unknown-microsoft-dsc-metadata + +Keys under `Microsoft.DSC` metadata must be ones DSC recognizes. DSC silently ignores unknown +keys, so a typo such as `securityContxt` has no effect. + +### api-version-alias + +Resources should use `requireVersion` rather than its alias `apiVersion`. + +### date-version + +Version requirements should be semantic. Date-based versions such as `2025-01-01` are deprecated +and only supported for compatibility. + +### missing-require-version + +Resources should pin the version they require with `requireVersion`. This rule is off by default; +enable it to require reproducible documents. + +## Expression rules + +The expression rules check strings in the places DSC evaluates them: parameter default values, +variables, output values and conditions, user-defined function bodies, and the name, condition, +`dependsOn` entries, copy settings, and properties of each resource. A parameter default value or +variable is only an expression when the whole value is a string; DSC uses object and array values +as written, so strings nested in them aren't checked. The properties of a group resource, such as +`Microsoft.DSC/Group`, hold an embedded configuration document that DSC evaluates separately, so +they aren't checked against the outer document. + +### invalid-expression + +Every string that starts with `[` is evaluated as an expression and must parse. To use a literal +string that starts with a bracket, escape it with `[[`. When the expression is the default value of +a `secureString` or `secureObject` parameter, the diagnostic omits the expression and the parser +error, because they can contain a secret. + +```yaml +properties: + output: "[concat('unterminated)]" # invalid-expression + literal: "[[not an expression]" # fine +``` + +### unknown-function + +Expressions may only call built-in functions or user-defined functions declared in `functions`. +Function names are case-sensitive; the diagnostic suggests the correct casing when the name +matches a built-in function except for case. + +### invalid-argument-count + +Function calls must pass the number of arguments the function accepts. The message is the same one +DSC raises at runtime. + +### invalid-argument-type + +Literal arguments must be of a kind the function accepts at that position. Arguments that are +nested expressions aren't checked, because their value is only known at runtime. + +### undefined-parameter + +`parameters()` must reference a parameter declared in `parameters`. Calls inside user-defined +function bodies refer to the function's own parameters and aren't checked. + +### undefined-variable + +`variables()` must reference a variable declared in `variables`. + +### condition-not-expression + +A `condition` on a resource or output must be an expression that evaluates to `true`. A plain +string such as `"true"` never does, so the item is always skipped. + +## Parameter rules + +### no-unused-parameters + +Declared parameters should be referenced by at least one expression. + +### no-unused-variables + +Declared variables should be referenced by at least one expression. + +### parameter-default-type + +A literal default value must satisfy the parameter's type, `allowedValues`, value limits, and +length limits. Default values that are expressions aren't checked. An escaped string such as +`"[[x]"` is checked as the value DSC uses, `"[x]"`. + +### invalid-parameter-constraints + +Parameter constraints must be consistent with each other and with the parameter type: `minValue` +can't exceed `maxValue`, `minLength` can't exceed `maxLength`, value limits only apply to `int` +parameters, length limits only apply to `string`, `secureString`, and `array` parameters, +`allowedValues` can't be empty, and every allowed value must be of the parameter's type. + +### secure-parameter-default + +A `secureString` or `secureObject` parameter shouldn't have a hard-coded default value, because +anyone with the document can read it. An empty string or an expression is allowed. + +### prefer-secure-type + +A `string` or `object` parameter whose name suggests a secret, such as `password`, `token`, or +`apiKey`, should use the `secureString` or `secureObject` type. + +### secure-value-in-output + +Outputs shouldn't use a secure parameter or a `secret()` value. DSC only withholds outputs whose +declared type is `secureString` or `secureObject`; it doesn't track where a value came from. An +output of any other type that extracts a plain value, such as +`[parameters('password').secureString]`, returns that value in the result. Outputs declared as +`secureString` or `secureObject` aren't reported, because DSC never returns them. + +### use-parameter-description + +Parameters should have a description. This rule is off by default. + +### undeclared-parameter-value + +When you pass parameters with `--parameters` or `--parameters-file`, every supplied value must +match a declared parameter. + +### missing-parameter-value + +When you pass parameters, every declared parameter without a default value must be supplied a +value. Without parameters this rule doesn't run, because a document can declare required +parameters on purpose. + +### parameter-value-type + +When you pass parameters, every supplied value must satisfy the parameter's type and constraints. +A `secureString` value must be supplied as a plain string, not in the wrapped +`{"secureString": "..."}` form that a default value can use. + +## Dependency rules + +### unresolved-dependency + +Every `dependsOn` entry and every `reference(resourceId(...))` call must point at a resource +declared at the same level. The diagnostic distinguishes a missing resource from a resource whose +type doesn't match. Resources in copy loops aren't checked, because their names are only known at +runtime. `dependsOn` entries match resource names as written, the way DSC resolves them. A +`reference()` call matches the name DSC evaluates, so it isn't reported when a resource at the same +level has a name that is an expression. + +```yaml +resources: +- name: first + type: Microsoft.DSC.Debug/Echo +- name: second + type: Microsoft.DSC.Debug/Echo + dependsOn: + - "[resourceId('Microsoft.DSC.Debug/Echo', 'missing')]" # unresolved-dependency +``` + +### circular-dependency + +Resource dependencies must not form a cycle, including a resource that depends on itself. The +message shows the cycle, like `a -> b -> a`. + +### reference-without-depends-on + +A resource that references another resource with `reference()` must list it in `dependsOn`. DSC +only makes the output of resources that have already run available to `reference()`, so the +reference fails when the referenced resource is declared later. + +### duplicate-depends-on + +A dependency should be listed once. + +## Deprecation rules + +### copy-loop-deprecated + +Copy loops are deprecated and will be removed in a future release. See +[issue #1429][01] for details. + +### copy-options-unsupported + +Copy loops don't support the `mode` and `batchSize` options; DSC rejects a document that sets +them. + +### security-context-in-metadata + +The security context should be set with `directives.securityContext` rather than the deprecated +`Microsoft.DSC` metadata. + +### security-context-conflict + +When both the metadata and the directive set a security context, they must agree; DSC refuses to +run the document otherwise. + +### dsc-version-unsatisfied + +The `directives.version` requirement should be satisfied by the version of DSC in use. The rule +checks the version of `dsclint`, which is the version of the `dsc` command it ships with. + +## Environment rules + +### system-root-unused + +When you pass `--system-root`, at least one expression in the document should call `systemRoot()`. +Otherwise the document doesn't target the specified root. + +### hardcoded-system-path + +Absolute operating system paths such as `C:\Windows` or `/etc` should be built with `systemRoot()` +so the document can target another root. This rule is off by default because many documents +intentionally target the running system. + +## Resource rules + +These rules run after resource discovery and are skipped with `--no-discovery`. + +### discovery-failed + +Resource discovery must succeed for the resource rules to run. When it fails, the command reports +this diagnostic and continues with the static results only. This rule has a fixed level. + +### unknown-resource-type + +Each resource type must be installed. This rule reports a warning rather than an error, because a +document can be valid on a machine that has the resource. + +### unsatisfied-require-version + +An installed version of the resource type must satisfy `requireVersion`. + +### deprecated-resource + +Resources shouldn't use resource types whose manifest marks them as deprecated. The diagnostic +includes the deprecation message from the manifest. + +### property-schema-mismatch + +Literal property values must validate against the resource's embedded schema. Escaped strings such +as `"[[x]"` are validated as the value DSC passes to the resource, `"[x]"`. Properties that hold +expressions aren't checked, because their value is only known at runtime. When an expression sits +inside an `anyOf` or `oneOf`, the rule only reports the failure if every alternative fails for a +reason the expression can't change, such as a literal sibling with the wrong type. The rule never +invokes the resource to retrieve its schema. + +### nested-resources-unsupported + +Only group and adapter resources can declare nested `resources`. + +### exist-false-unsupported + +A resource type must have the `delete` or `setHandlesExist` capability for `_exist: false` to +work. Without either, `dsc config set` fails for the resource. + +### canonical-type-name + +Resource types should be written with the casing their manifest uses. + + +[01]: https://github.com/PowerShell/DSC/issues/1429 diff --git a/dsc/Cargo.toml b/dsc/Cargo.toml index f65519c84..ea679aeab 100644 --- a/dsc/Cargo.toml +++ b/dsc/Cargo.toml @@ -11,6 +11,7 @@ crossterm = { workspace = true } ctrlc = { workspace = true } indicatif = { workspace = true } jsonschema = { workspace = true } +miette = { workspace = true, features = ["fancy-no-backtrace"] } path-absolutize = { workspace = true } regex = { workspace = true } rmcp = { workspace = true, features = [ diff --git a/dsc/dsc.settings.json b/dsc/dsc.settings.json index 19acd63ea..c472e8a5c 100644 --- a/dsc/dsc.settings.json +++ b/dsc/dsc.settings.json @@ -8,5 +8,11 @@ "level": "WARN", "format": "Default", "allowOverride": true + }, + "plugins": { + "enabled": true + }, + "lint": { + "rules": {} } } diff --git a/dsc/dsc_default.settings.json b/dsc/dsc_default.settings.json index fe37d458f..020df7b80 100644 --- a/dsc/dsc_default.settings.json +++ b/dsc/dsc_default.settings.json @@ -9,6 +9,12 @@ "level": "WARN", "format": "Default", "allowOverride": true + }, + "plugins": { + "enabled": true + }, + "lint": { + "rules": {} } } } diff --git a/dsc/locales/en-us.toml b/dsc/locales/en-us.toml index 94c4eba8f..85953a326 100644 --- a/dsc/locales/en-us.toml +++ b/dsc/locales/en-us.toml @@ -35,6 +35,8 @@ getAll = "Get all instances of the resource" resource = "The name of the resource to invoke" functionAbout = "Operations on DSC functions" listFunctionAbout = "List or find functions" +pluginAbout = "Operations on DSC plug-ins" +listPluginAbout = "List or find plug-ins" functionCategory = "Category to filter functions on; specify multiple times to require all categories" functionDescription = "Description to search for in the function description, accepts wildcards" version = "The version requirement for the resource, like '^1.2' or '>1.2.3, <1.3'" @@ -65,6 +67,20 @@ Press any key to close this window""" failedToStartServer = "Failed to start server: %{error}" failedToGetSystem = "Failed to get system information" +[plugin] +discoveryFailed = "Failed to discover plug-ins: %{error}" +shadowedPlugin = "Command '%{command}' is provided by plug-in '%{plugin}'. Plug-in '%{shadowed}' defines the same command and is shadowed. To run it, use 'dsc %{shadowed}'" +deprecationMessage = "Plug-in '%{plugin}' is deprecated: %{message}" +failedToResolve = "Failed to run command '%{command}' of plug-in '%{plugin}': %{error}" +failedToGetExePath = "Could not get the path of the DSC executable for the plug-in: %{error}" +invoking = "Running command '%{command}' of plug-in '%{plugin}' with executable '%{executable}'" +failedToStart = "Failed to start executable '%{executable}' for command '%{command}': %{error}" +helpHeadingCommands = "Commands" +helpHeadingPluginCommands = "Plug-in commands" +helpHeadingOptions = "Options" +helpOption = "Print help" +unrecognizedSubcommand = "unrecognized subcommand '%{command}'" + [resolve] processingInclude = "Processing Include input" invalidInclude = "Failed to deserialize Include input" @@ -121,6 +137,11 @@ whatIfNotSupported = "what_if is only supported for the 'set' and 'delete' opera actionNotSupported = "The type '%{resource}' is an action or adapter and cannot be invoked as a resource" invalidPropertiesJson = "Invalid properties JSON: %{error}" +[server.lint_dsc_config] +invalidParameters = "Invalid parameters" +failedConvertJson = "Failed to convert to JSON" +lintFailed = "Failed to lint the configuration" + [server.list_dsc_functions] invalidNamePattern = "Invalid function name pattern '%{pattern}'" invalidDescriptionPattern = "Invalid function description pattern '%{pattern}'" @@ -171,6 +192,7 @@ tableHeader_type = "Type" tableHeader_kind = "Kind" tableHeader_version = "Version" tableHeader_capabilities = "Capabilities" +tableHeader_command = "Command" tableHeader_adapter = "RequireAdapter" tableHeader_description = "Description" tableHeader_functionName = "Function" @@ -181,6 +203,8 @@ invalidFunctionDescriptionFilter = "Invalid function description filter" maxInt = "maxInt" invalidManifest = "Error in manifest for" jsonArrayNotSupported = "JSON array output format is only supported for `--all'" +failedDiscoverPlugins = "Failed to discover plug-ins" +pluginShadowed = "%{command} (shadowed)" yes = "Yes" no = "No" none = "None" @@ -200,6 +224,7 @@ failedToReadFile = "Failed to read input file" noInput = "No input provided" emptyInput = "Empty input provided" failedToParseInput = "Invalid JSON or YAML" +inputSourceName = "input" failedToAbsolutizePath = "Error making config path absolute" failedToGetParentPath = "Error reading config path parent" dscConfigRootAlreadySet = "The current value of DSC_CONFIG_ROOT env var will be overridden" diff --git a/dsc/src/args.rs b/dsc/src/args.rs index f95a8c50b..712600bc8 100644 --- a/dsc/src/args.rs +++ b/dsc/src/args.rs @@ -10,6 +10,7 @@ use dsc_lib::types::{FullyQualifiedTypeName, ResourceVersionReq, TypeNameFilter} use rust_i18n::t; use schemars::JsonSchema; use serde::Deserialize; +use std::ffi::OsString; #[derive(Debug, Clone, PartialEq, Eq, ValueEnum)] pub enum OutputFormat { @@ -123,6 +124,11 @@ pub enum SubCommand { #[clap(subcommand)] subcommand: FunctionSubCommand, }, + #[clap(name = "plugin", about = t!("args.pluginAbout").to_string())] + Plugin { + #[clap(subcommand)] + subcommand: PluginSubCommand, + }, #[clap(name = "server", alias = "mcp", about = t!("args.serverAbout").to_string())] Server, #[clap(name = "resource", about = t!("args.resourceAbout").to_string())] @@ -137,6 +143,11 @@ pub enum SubCommand { #[clap(short = 'o', long, help = t!("args.outputFormat").to_string(), value_enum)] output_format: Option, }, + /// A command that isn't built in, which a command plug-in may provide. The first item is the + /// command name and the rest are the arguments for the plug-in, exactly as the user specified + /// them. + #[clap(external_subcommand)] + External(Vec), } #[derive(Debug, PartialEq, Eq, Subcommand)] @@ -229,6 +240,18 @@ pub enum ExtensionSubCommand { }, } +#[derive(Debug, PartialEq, Eq, Subcommand)] +pub enum PluginSubCommand { + #[clap(name = "list", about = t!("args.listPluginAbout").to_string())] + List { + /// Optional plug-in type name to filter the list + #[clap(default_value_t)] + plugin_name: TypeNameFilter, + #[clap(short = 'o', long, help = t!("args.outputFormat").to_string())] + output_format: Option, + }, +} + #[derive(Debug, PartialEq, Eq, Subcommand)] pub enum FunctionSubCommand { #[clap(name = "list", about = t!("args.listFunctionAbout").to_string())] @@ -357,7 +380,10 @@ pub enum SchemaType { FunctionDefinition, GetResult, Include, + LintResult, + LintRules, ManifestList, + PluginManifest, ResolveResult, Resource, ResourceGetResult, diff --git a/dsc/src/main.rs b/dsc/src/main.rs index 7104e139c..1f2724b26 100644 --- a/dsc/src/main.rs +++ b/dsc/src/main.rs @@ -1,8 +1,7 @@ // Copyright (c) Microsoft Corporation. // Licensed under the MIT License. -use args::{Args, SubCommand}; -use clap::{CommandFactory, Parser}; +use args::SubCommand; use clap_complete::generate; use dsc_lib::{progress::ProgressFormat, settings::{DscSettings, DscSettingsScope, global as global_settings, sources::EnvironmentData}}; use server::start_server; @@ -19,6 +18,8 @@ use crossterm::event; use std::env; pub mod args; +pub mod plugin; +pub mod report; pub mod resolve; pub mod resource_command; pub mod server; @@ -76,7 +77,7 @@ fn dsc_main() -> Result<(), ExitCode> { error!("{}", t!("main.failedCtrlCHandler")); } - let args = Args::parse(); + let args = plugin::parse_args(); // Initialize with local settings; reload for global after tracing init // to emit messages for settings loading/resolution. @@ -115,35 +116,12 @@ fn dsc_main() -> Result<(), ExitCode> { match args.subcommand { SubCommand::Completer { shell } => { info!("{} {:?}", t!("main.generatingCompleter"), shell); - let mut cmd = Args::command(); + // Include the plug-in commands, so the script completes them like built-in commands. + let mut cmd = plugin::command_with_plugins(&plugin::discover()); generate(shell, &mut cmd, "dsc", &mut io::stdout()); }, SubCommand::Config { subcommand, parameters, parameters_file, system_root, as_group, as_assert, as_include } => { - let Ok(params) = get_input(None, parameters_file.as_ref()) else { - return Err(ExitCode::from(util::EXIT_INVALID_INPUT)); - }; - let file_params = if params.is_empty() { - None - } else { - Some(params) - }; - - let merged_parameters = match (file_params, parameters) { - (Some(file_content), Some(inline_content)) => { - info!("{}", t!("main.mergingParameters")); - match util::merge_parameters(&file_content, &inline_content) { - Ok(merged) => Some(merged), - Err(err) => { - error!("{}: {err}", t!("main.failedMergingParameters")); - return Err(ExitCode::from(util::EXIT_INVALID_INPUT)) ; - } - } - }, - (Some(file_content), None) => Some(file_content), - (None, Some(inline_content)) => Some(inline_content), - (None, None) => None, - }; - + let merged_parameters = merge_parameter_inputs(parameters, parameters_file.as_ref())?; subcommand::config(&subcommand, &merged_parameters, system_root.as_ref(), &as_group, &as_assert, &as_include, progress_format)?; }, SubCommand::Action { subcommand } => { @@ -155,6 +133,14 @@ fn dsc_main() -> Result<(), ExitCode> { SubCommand::Function { subcommand } => { subcommand::function(&subcommand)?; }, + SubCommand::Plugin { subcommand } => { + subcommand::plugin(&subcommand)?; + }, + SubCommand::External(external_args) => { + // Only returns when the plug-in couldn't be started; otherwise it exits with the + // plug-in's exit code. + plugin::dispatch(&external_args, args.progress_format)?; + }, SubCommand::Server => { if let Err(err) = start_server() { error!("{}", t!("main.failedToStartServer", error = err)); @@ -181,6 +167,48 @@ fn dsc_main() -> Result<(), ExitCode> { Ok(()) } +/// Combine the inline parameters with the parameters file, if either was given. +/// +/// # Arguments +/// +/// * `parameters` - The inline parameters, as JSON or YAML. +/// * `parameters_file` - The path to a parameters file, or `-` to read it from STDIN. +/// +/// # Returns +/// +/// The merged parameters as JSON, or `None` when neither was given. Inline values take precedence +/// over values from the file. +/// +/// # Errors +/// +/// Returns `EXIT_INVALID_INPUT` when the file can't be read or the two inputs can't be merged. +fn merge_parameter_inputs(parameters: Option, parameters_file: Option<&String>) -> Result, ExitCode> { + let Ok(file_params) = get_input(None, parameters_file) else { + return Err(ExitCode::from(util::EXIT_INVALID_INPUT)); + }; + let file_params = if file_params.is_empty() { + None + } else { + Some(file_params) + }; + + match (file_params, parameters) { + (Some(file_content), Some(inline_content)) => { + info!("{}", t!("main.mergingParameters")); + match util::merge_parameters(&file_content, &inline_content) { + Ok(merged) => Ok(Some(merged)), + Err(err) => { + error!("{}: {err}", t!("main.failedMergingParameters")); + Err(ExitCode::from(util::EXIT_INVALID_INPUT)) + } + } + }, + (Some(file_content), None) => Ok(Some(file_content)), + (None, Some(inline_content)) => Ok(Some(inline_content)), + (None, None) => Ok(None), + } +} + fn ctrlc_handler() { use std::process::exit; warn!("{}", t!("main.ctrlCReceived")); diff --git a/dsc/src/plugin.rs b/dsc/src/plugin.rs new file mode 100644 index 000000000..4f5e34830 --- /dev/null +++ b/dsc/src/plugin.rs @@ -0,0 +1,486 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +//! Command plug-in support for the `dsc` command line. +//! +//! A command plug-in adds a top-level command to `dsc` without being compiled into it. This module +//! parses the command line with the adjustments plug-in commands need, runs the plug-in that +//! provides a command, and adds plug-in commands to the top-level help and the completion scripts. +//! +//! Plug-ins are only discovered when the result is needed: when the first argument isn't a +//! built-in command, when the top-level help is shown, when plug-ins are listed, and when a +//! completion script is generated. A built-in command never triggers plug-in discovery. + +use crate::args::{Args, SubCommand}; +use crate::util::{current_dsc_version, EXIT_DSC_ERROR}; +use clap::{Arg, ArgAction, Command, CommandFactory, FromArgMatches, ValueEnum, ValueHint}; +use clap::builder::ValueParser; +use clap::error::ErrorKind; +use dsc_lib::plugins::discovery::{discover_plugins, find_plugin, find_plugin_by_type}; +use dsc_lib::plugins::dscplugin::DscPlugin; +use dsc_lib::progress::ProgressFormat; +use dsc_lib::settings::{global as global_settings, sources::EnvironmentData}; +use dsc_lib::util::get_exe_path; +use rust_i18n::t; +use std::env; +use std::ffi::OsString; +use std::fmt::Write as _; +use std::process::{exit, ExitCode, ExitStatus}; +use tracing::{debug, error, warn}; + +/// The environment variable that holds the absolute path of the `dsc` executable that started a +/// plug-in. +pub const DSC_EXE_ENV_VAR: &str = "DSC_EXE"; +/// The environment variable that holds the semantic version of the `dsc` executable that started +/// a plug-in. +pub const DSC_VERSION_ENV_VAR: &str = "DSC_VERSION"; +/// The environment variable that holds the command name a plug-in was invoked as. +pub const DSC_PLUGIN_COMMAND_ENV_VAR: &str = "DSC_PLUGIN_COMMAND"; +/// The environment variable that holds the directory that contains the manifest of a plug-in. +pub const DSC_PLUGIN_DIR_ENV_VAR: &str = "DSC_PLUGIN_DIR"; +/// The environment variable that holds the value of the `--progress-format` option, when the user +/// specified it. +pub const DSC_PROGRESS_FORMAT_ENV_VAR: &str = "DSC_PROGRESS_FORMAT"; + +/// The name of the help subcommand that clap adds to every command with subcommands. +const HELP_COMMAND: &str = "help"; +/// The exit code clap uses for a usage error, like an unrecognized subcommand. +const EXIT_USAGE_ERROR: i32 = 2; + +/// Parses the command line arguments of the current process. +/// +/// This function behaves like [`clap::Parser::parse()`], with two additions for plug-ins: +/// +/// - `dsc help ` for a command that isn't built in is parsed as the plug-in command +/// followed by `--help`, so the plug-in prints its own help. +/// - The top-level help lists the discovered plug-in commands after the built-in commands. +/// +/// Like `parse()`, this function prints the message and exits the process when the arguments are +/// invalid or ask for help or the version. +/// +/// # Returns +/// +/// The parsed arguments. An argument that isn't a built-in command is returned as +/// [`SubCommand::External`] without checking whether a plug-in provides it. [`dispatch()`] does +/// that. +pub fn parse_args() -> Args { + let raw_args: Vec = env::args_os().collect(); + + if let Some(args) = parse_plugin_help_request(&raw_args) { + return args; + } + + let mut command = Args::command(); + let mut matches = match command.try_get_matches_from_mut(&raw_args) { + Ok(matches) => matches, + Err(err) => { + if is_top_level_help(&err, &mut command) { + print_top_level_help_with_plugins(&raw_args); + } + err.exit() + } + }; + + match Args::from_arg_matches_mut(&mut matches) { + Ok(args) => args, + Err(err) => err.format(&mut command).exit(), + } +} + +/// Returns the names and aliases of the commands that are built into `dsc`. +/// +/// A plug-in can't use any of these names, because built-in commands are matched first. +/// +/// # Returns +/// +/// The name and every alias of each built-in command, and the `help` command that clap adds. +#[must_use] +pub fn reserved_commands() -> Vec { + let command = Args::command(); + let mut reserved: Vec = command + .get_subcommands() + .flat_map(|subcommand| { + std::iter::once(subcommand.get_name()).chain(subcommand.get_all_aliases()) + }) + .map(str::to_string) + .collect(); + reserved.push(HELP_COMMAND.to_string()); + reserved +} + +/// Discovers the plug-ins that are available to `dsc`. +/// +/// # Returns +/// +/// The discovered plug-ins, including shadowed plug-ins. When discovery fails, the function logs +/// the error as a warning and returns an empty list, so that a problem with the resource path +/// settings doesn't hide the built-in commands. +#[must_use] +pub fn discover() -> Vec { + match discover_plugins(&reserved_commands()) { + Ok(plugins) => plugins, + Err(err) => { + warn!("{}", t!("plugin.discoveryFailed", error = err)); + Vec::new() + } + } +} + +/// Returns the `dsc` command definition with a subcommand for every plug-in command. +/// +/// The returned definition describes plug-in commands the way it describes built-in commands, so +/// clap can suggest them for a mistyped command and include them in completion scripts. It doesn't +/// accept other external subcommands. It must not be used to run a plug-in, because the +/// subcommands only describe the name and description of each plug-in command, not its arguments. +/// +/// # Arguments +/// +/// * `plugins` - The discovered plug-ins. Shadowed plug-ins are ignored. +#[must_use] +pub fn command_with_plugins(plugins: &[DscPlugin]) -> Command { + // Clap turns external subcommands back on while it builds a command that has a value parser + // for them, so the value parser has to be reset together with the setting. + let mut command = Args::command() + .allow_external_subcommands(false) + .external_subcommand_value_parser(None::); + for plugin in plugins.iter().filter(|plugin| !plugin.shadowed) { + command = command.subcommand(plugin_subcommand(plugin)); + } + command +} + +/// Runs the plug-in that provides a command and exits with the plug-in's exit code. +/// +/// The plug-in runs in the caller's working directory with the standard streams of `dsc`. DSC +/// passes the arguments from the manifest followed by the arguments the user specified, and sets +/// the environment variables that tell the plug-in how it was invoked. +/// +/// When no plug-in provides the command, the function reports the same unrecognized subcommand +/// error as for any unknown command and exits with the exit code for a usage error. +/// +/// # Arguments +/// +/// * `external_args` - The command name followed by the arguments the user specified after it. +/// * `progress_format` - The value of the `--progress-format` option, if the user specified it. +/// +/// # Returns +/// +/// This function only returns when the plug-in couldn't be started. Otherwise it exits the +/// process. +/// +/// # Errors +/// +/// Returns the exit code for a DSC error when the executable of the plug-in can't be resolved or +/// started. +pub fn dispatch(external_args: &[OsString], progress_format: Option) -> Result<(), ExitCode> { + let plugins = discover(); + + let invoked_as = external_args.first().map(|name| name.to_string_lossy().into_owned()).unwrap_or_default(); + // A type name always contains a slash and a command name never does, so the name itself + // tells which of the two the user specified. + let by_type_name = invoked_as.contains('/'); + let found = if by_type_name { + find_plugin_by_type(&plugins, &invoked_as) + } else { + find_plugin(&plugins, &invoked_as) + }; + let Some(plugin) = found else { + exit_with_unrecognized_subcommand(&plugins); + }; + let user_args = &external_args[1..]; + + // Naming a plug-in by its type is unambiguous, so only warn when the command name was used. + if !by_type_name { + for shadowed in plugins.iter().filter(|other| other.shadowed && other.command == plugin.command) { + warn!("{}", t!( + "plugin.shadowedPlugin", + command = plugin.command, + plugin = plugin.type_name, + shadowed = shadowed.type_name + )); + } + } + + if let Some(message) = &plugin.deprecation_message { + warn!("{}", t!("plugin.deprecationMessage", plugin = plugin.type_name, message = message)); + } + + let invocation = match plugin.resolve_invocation() { + Ok(invocation) => invocation, + Err(err) => { + error!("{}", t!("plugin.failedToResolve", command = plugin.command, plugin = plugin.type_name, error = err)); + return Err(ExitCode::from(EXIT_DSC_ERROR)); + } + }; + + let mut process = std::process::Command::new(&invocation.executable); + process + .args(&invocation.args) + .args(user_args) + .env(DSC_VERSION_ENV_VAR, current_dsc_version().to_string()) + .env(DSC_PLUGIN_COMMAND_ENV_VAR, &invoked_as) + .env(DSC_PLUGIN_DIR_ENV_VAR, &plugin.directory) + .env( + EnvironmentData::DSC_TRACE_FORMAT_ENV_VAR, + global_settings::get_or_init().resolved().tracing.format.value.to_string() + ); + match get_exe_path() { + Ok(exe_path) => { process.env(DSC_EXE_ENV_VAR, exe_path); }, + Err(err) => debug!("{}", t!("plugin.failedToGetExePath", error = err)), + } + if let Some(name) = progress_format.and_then(|format| format.to_possible_value()) { + process.env(DSC_PROGRESS_FORMAT_ENV_VAR, name.get_name()); + } + + debug!("{}", t!( + "plugin.invoking", + command = plugin.command, + plugin = plugin.type_name, + executable = invocation.executable.display() + )); + match process.status() { + Ok(status) => exit(exit_code_for(status)), + Err(err) => { + error!("{}", t!( + "plugin.failedToStart", + command = plugin.command, + executable = invocation.executable.display(), + error = err + )); + Err(ExitCode::from(EXIT_DSC_ERROR)) + } + } +} + +/// Parses `dsc help ` for a command that isn't built in. +/// +/// Clap's `help` subcommand only knows the built-in commands. To recognize the request, this +/// function parses the arguments with the `help` subcommand disabled, which turns `help` into an +/// external subcommand whose arguments are the command path to show help for. +/// +/// # Arguments +/// +/// * `raw_args` - The arguments of the process, including the program name. +/// +/// # Returns +/// +/// The parsed arguments with the plug-in command as the subcommand and `--help` appended to its +/// arguments, or [`None`] when the arguments aren't a help request for a command that isn't built +/// in. Whether a plug-in provides the command is checked by [`dispatch()`]. +fn parse_plugin_help_request(raw_args: &[OsString]) -> Option { + // Building the command definition a second time isn't free, so only do it when the arguments + // can be a help request. + if !raw_args.iter().any(|arg| arg == HELP_COMMAND) { + return None; + } + + let mut command = Args::command().disable_help_subcommand(true); + let mut matches = command.try_get_matches_from_mut(raw_args).ok()?; + let mut args = Args::from_arg_matches_mut(&mut matches).ok()?; + + let SubCommand::External(external_args) = &args.subcommand else { + return None; + }; + let (first, command_path) = external_args.split_first()?; + if first != HELP_COMMAND { + return None; + } + let target = command_path.first()?.to_string_lossy(); + if reserved_commands().iter().any(|reserved| *reserved == target) { + return None; + } + + let mut plugin_args = command_path.to_vec(); + plugin_args.push(OsString::from("--help")); + args.subcommand = SubCommand::External(plugin_args); + Some(args) +} + +/// Indicates whether a parsing error is the request to show the top-level help of `dsc`. +/// +/// Clap reports a help request as an error that carries the rendered help, without saying which +/// command the help is for. This function compares that text to the help of the top-level command. +/// +/// # Arguments +/// +/// * `err` - The error that parsing the arguments returned. +/// * `command` - The command definition that parsed the arguments. It must be the same instance, +/// because parsing records the program name the usage line shows. +fn is_top_level_help(err: &clap::Error, command: &mut Command) -> bool { + if err.kind() != ErrorKind::DisplayHelp { + return false; + } + + let rendered = err.render().to_string(); + rendered == command.render_help().to_string() || rendered == command.render_long_help().to_string() +} + +/// Prints the top-level help with the plug-in commands listed after the built-in commands, then +/// exits the process. +/// +/// # Arguments +/// +/// * `raw_args` - The arguments of the process, including the program name. They are parsed again +/// so that clap renders the short or the long help, as requested. +/// +/// # Returns +/// +/// This function returns without printing anything when there are no plug-in commands to list, so +/// the caller can print the unmodified help. +fn print_top_level_help_with_plugins(raw_args: &[OsString]) { + // The arguments never got parsed, so the settings haven't been loaded yet. Tracing isn't + // enabled either, which keeps messages about invalid manifests out of the help. + let mut settings = global_settings::get_or_init(); + settings.load(); + settings.resolve_all(); + drop(settings); + + let plugins = discover(); + let mut provided: Vec<&DscPlugin> = plugins.iter().filter(|plugin| !plugin.shadowed).collect(); + if provided.is_empty() { + return; + } + provided.sort_by(|a, b| a.command.cmp(&b.command)); + + let command = Args::command(); + let template = top_level_help_template(&command, &provided); + if let Err(err) = command.help_template(template).try_get_matches_from_mut(raw_args) { + err.exit(); + } +} + +/// Builds the help template for the top-level command with a section for plug-in commands. +/// +/// The template reproduces clap's default layout and adds the plug-in commands between the +/// built-in commands and the options. +/// +/// # Arguments +/// +/// * `command` - The top-level command definition, for its styles and the names of its +/// subcommands. +/// * `plugins` - The plug-ins that provide a command, in the order to list them. +fn top_level_help_template(command: &Command, plugins: &[&DscPlugin]) -> String { + let styles = command.get_styles(); + let header = styles.get_header(); + let literal = styles.get_literal(); + + // Use the same column for the descriptions as the built-in commands when the names allow it. + let name_width = command + .get_subcommands() + .filter(|subcommand| !subcommand.is_hide_set()) + .map(|subcommand| subcommand.get_name().len()) + .chain(std::iter::once(HELP_COMMAND.len())) + .chain(plugins.iter().map(|plugin| plugin.command.len())) + .max() + .unwrap_or_default(); + + let mut plugin_lines = String::new(); + for plugin in plugins { + let name = plugin.command.as_ref(); + let description = plugin.description.as_deref().unwrap_or_default(); + let line = format!(" {literal}{name}{literal:#}{:width$} {description}", "", width = name_width - name.len()); + let _ = writeln!(plugin_lines, "{}", line.trim_end()); + } + + format!( + "{{before-help}}{{about-with-newline}}\n\ + {{usage-heading}} {{usage}}\n\ + \n\ + {header}{commands}:{header:#}\n\ + {{subcommands}}\n\ + \n\ + {header}{plugin_commands}:{header:#}\n\ + {plugin_lines}\ + \n\ + {header}{options}:{header:#}\n\ + {{options}}{{after-help}}", + commands = t!("plugin.helpHeadingCommands"), + plugin_commands = t!("plugin.helpHeadingPluginCommands"), + options = t!("plugin.helpHeadingOptions"), + ) +} + +/// Returns the subcommand definition that describes a plug-in command. +/// +/// The definition has the name and description of the plug-in command. It accepts `--help`, which +/// every plug-in must support, and any other arguments, completed as paths because DSC doesn't +/// know the arguments of the plug-in. +/// +/// # Arguments +/// +/// * `plugin` - The plug-in that provides the command. +fn plugin_subcommand(plugin: &DscPlugin) -> Command { + // Clap only takes names with a static lifetime unless its `string` feature is enabled. The + // process ends right after the command definition is used, so leaking a few short names is + // cheaper than enabling the feature for every command definition in `dsc`. + let name: &'static str = Box::leak(plugin.command.to_string().into_boxed_str()); + + let mut subcommand = Command::new(name) + .disable_help_flag(true) + .arg( + Arg::new("help") + .long("help") + .action(ArgAction::SetTrue) + .help(t!("plugin.helpOption").to_string()) + ) + .arg( + Arg::new("args") + .num_args(0..) + .trailing_var_arg(true) + .allow_hyphen_values(true) + .value_hint(ValueHint::AnyPath) + ); + if let Some(description) = &plugin.description { + subcommand = subcommand.about(description.clone()); + } + subcommand +} + +/// Reports the unrecognized subcommand error for the arguments of the process and exits. +/// +/// The arguments are parsed again with a command definition that doesn't accept external +/// subcommands, so clap reports the error exactly as it does for any unknown command. The +/// definition includes the plug-in commands so the suggestion in the error covers them. +/// +/// # Arguments +/// +/// * `plugins` - The discovered plug-ins. +fn exit_with_unrecognized_subcommand(plugins: &[DscPlugin]) -> ! { + let raw_args: Vec = env::args_os().collect(); + match command_with_plugins(plugins).try_get_matches_from_mut(&raw_args) { + Err(err) => err.exit(), + Ok(_) => { + // The arguments only parse when they name a built-in or plug-in command, and this + // function is only called when they don't. + let name = raw_args.get(1).map(|arg| arg.to_string_lossy().into_owned()).unwrap_or_default(); + error!("{}", t!("plugin.unrecognizedSubcommand", command = name)); + exit(EXIT_USAGE_ERROR) + } + } +} + +/// Returns the exit code for `dsc` to exit with after a plug-in finished. +/// +/// # Arguments +/// +/// * `status` - The exit status of the plug-in process. +/// +/// # Returns +/// +/// The exit code of the plug-in. When a signal ended the plug-in, the function returns `128` plus +/// the signal number, which is how shells report it. +fn exit_code_for(status: ExitStatus) -> i32 { + if let Some(code) = status.code() { + return code; + } + + #[cfg(unix)] + { + use std::os::unix::process::ExitStatusExt; + if let Some(signal) = status.signal() { + return 128 + signal; + } + } + + i32::from(EXIT_DSC_ERROR) +} diff --git a/dsc/src/report.rs b/dsc/src/report.rs new file mode 100644 index 000000000..704054175 --- /dev/null +++ b/dsc/src/report.rs @@ -0,0 +1,105 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +//! Render errors for the terminal. +//! +//! Errors from `dsc-lib` are [`miette::Diagnostic`]s. Most carry only a message, but the ones +//! found while validating a document or resource input also carry the text they apply to, with a +//! label on each problem and sometimes a suggestion. [`render`] turns such an error into a report +//! that shows the problem in place, in the style the trace format asks for. + +use std::io::{IsTerminal, stderr}; +use std::sync::OnceLock; + +use dsc_lib_telemetry::StderrFormat; +use miette::{Diagnostic, GraphicalReportHandler, GraphicalTheme}; + +/// The lines shown around each labeled line of a snippet. +const CONTEXT_LINES: usize = 2; + +static STDERR_FORMAT: OnceLock = OnceLock::new(); + +/// Record the trace format so that reports match the rest of the trace output. +/// +/// Only the first call takes effect. +/// +/// # Arguments +/// +/// * `format` - The format trace output is written in. +pub fn set_stderr_format(format: StderrFormat) { + let _ = STDERR_FORMAT.set(format); +} + +/// Render an error for the error log. +/// +/// An error that carries source text or help is rendered as a report with the text, the labels +/// on it, and the help; any other error is rendered as its message. When the trace format is +/// JSON the message alone is used, so that consumers get one line per error. +/// +/// # Arguments +/// +/// * `error` - The error to render. +#[must_use] +pub fn render(error: &dyn Diagnostic) -> String { + let format = STDERR_FORMAT.get().copied().unwrap_or(StderrFormat::Default); + let has_detail = error.source_code().is_some() || error.help().is_some(); + if !has_detail || format == StderrFormat::Json { + return error.to_string(); + } + + let handler = handler(format == StderrFormat::Default && stderr().is_terminal()); + let mut report = String::new(); + if handler.render_report(&mut report, error).is_err() { + return error.to_string(); + } + report.trim_end().to_string() +} + +/// A report handler for the terminal, drawing with unicode and, unless `NO_COLOR` asks +/// otherwise, ANSI colors. +/// +/// # Arguments +/// +/// * `colored` - Whether colors are wanted. +fn handler(colored: bool) -> GraphicalReportHandler { + let no_color = std::env::var("NO_COLOR").is_ok_and(|value| value != "0"); + let theme = if colored && !no_color { + GraphicalTheme::unicode() + } else { + GraphicalTheme::unicode_nocolor() + }; + GraphicalReportHandler::new_themed(theme).with_context_lines(CONTEXT_LINES) +} + +#[cfg(test)] +mod tests { + use super::*; + use dsc_lib::{diagnostics::SchemaValidationError, dscerror::DscError, dscresources::dscresource::validate_json_all}; + use serde_json::json; + + fn schema_error() -> DscError { + let schema = json!({"type": "object", "additionalProperties": false, "properties": {"output": {"type": "string"}}}); + let text = "{\n \"otput\": \"hi\"\n}"; + let violations = validate_json_all("test", &schema, &serde_json::from_str(text).unwrap()).unwrap(); + let error = SchemaValidationError::new("input for 'Test/Echo'", violations) + .with_schema(&schema) + .with_source(text, "input", ""); + DscError::SchemaValidation(Box::new(error)) + } + + #[test] + fn plain_errors_render_as_their_message() { + let error = DscError::Operation("failed".to_string()); + assert_eq!(render(&error), error.to_string()); + } + + #[test] + fn errors_with_source_render_a_snippet_and_help() { + let report = render(&schema_error()); + assert!(report.contains("input for 'Test/Echo'"), "{report}"); + assert!(report.contains("\"otput\""), "{report}"); + assert!(report.contains("'otput' isn't a known property"), "{report}"); + assert!(report.contains("Did you mean 'output'"), "{report}"); + assert!(report.contains("[input:2:3]"), "{report}"); + } +} diff --git a/dsc/src/resource_command.rs b/dsc/src/resource_command.rs index 50eaebd6b..e625c0604 100644 --- a/dsc/src/resource_command.rs +++ b/dsc/src/resource_command.rs @@ -2,6 +2,7 @@ // Licensed under the MIT License. use crate::args::{GetOutputFormat, OutputFormat}; +use crate::report::render; use crate::util::{EXIT_DSC_ERROR, EXIT_INVALID_ARGS, EXIT_JSON_ERROR, EXIT_DSC_RESOURCE_NOT_FOUND, write_object}; use dsc_lib::configure::config_doc::{Configuration, ExecutionKind}; use dsc_lib::configure::add_resource_export_results_to_configuration; @@ -64,7 +65,7 @@ pub fn get(dsc: &mut DscManager, resource_type: &FullyQualifiedTypeName, version write_object(&json, format, false)?; } Err(err) => { - error!("{err}"); + error!("{}", render(&err)); return Err(ExitCode::from(EXIT_DSC_ERROR)); } } @@ -88,7 +89,7 @@ pub fn get_all(dsc: &mut DscManager, resource_type: &FullyQualifiedTypeName, ver let export_result = match resource.export(&input, &context) { Ok(export) => { export } Err(err) => { - error!("{err}"); + error!("{}", render(&err)); return Err(ExitCode::from(EXIT_DSC_ERROR)); } }; @@ -169,13 +170,13 @@ pub fn set(dsc: &mut DscManager, resource_type: &FullyQualifiedTypeName, version Ok(GetResult::Resource(response)) => response.actual_state, Ok(_) => unreachable!(), Err(err) => { - error!("{err}"); + error!("{}", render(&err)); return Err(ExitCode::from(EXIT_DSC_ERROR)); } }; if let Err(err) = resource.delete(input, &context) { - error!("{err}"); + error!("{}", render(&err)); return Err(ExitCode::from(EXIT_DSC_ERROR)); } @@ -183,7 +184,7 @@ pub fn set(dsc: &mut DscManager, resource_type: &FullyQualifiedTypeName, version Ok(GetResult::Resource(response)) => response.actual_state, Ok(_) => unreachable!(), Err(err) => { - error!("{err}"); + error!("{}", render(&err)); return Err(ExitCode::from(EXIT_DSC_ERROR)); } }; @@ -221,7 +222,7 @@ pub fn set(dsc: &mut DscManager, resource_type: &FullyQualifiedTypeName, version write_object(&json, format, false)?; } Err(err) => { - error!("{err}"); + error!("{}", render(&err)); return Err(ExitCode::from(EXIT_DSC_ERROR)); } } @@ -259,7 +260,7 @@ pub fn test(dsc: &mut DscManager, resource_type: &FullyQualifiedTypeName, versio write_object(&json, format, false)?; } Err(err) => { - error!("{err}"); + error!("{}", render(&err)); return Err(ExitCode::from(EXIT_DSC_ERROR)); } } @@ -302,7 +303,7 @@ pub fn delete(dsc: &mut DscManager, resource_type: &FullyQualifiedTypeName, vers } }, Err(err) => { - error!("{err}"); + error!("{}", render(&err)); return Err(ExitCode::from(EXIT_DSC_ERROR)); } } @@ -334,7 +335,7 @@ pub fn schema(dsc: &mut DscManager, resource_type: &FullyQualifiedTypeName, vers write_object(&json, format, false)?; } Err(err) => { - error!("{err}"); + error!("{}", render(&err)); return Err(ExitCode::from(EXIT_DSC_ERROR)); } } @@ -358,7 +359,7 @@ pub fn export(dsc: &mut DscManager, resource_type: &FullyQualifiedTypeName, vers .ok() .and_then(|value| value.as_object().cloned()); if let Err(err) = add_resource_export_results_to_configuration(&*dsc_resource, &mut conf, input, &context) { - error!("{err}"); + error!("{}", render(&err)); return Err(ExitCode::from(EXIT_DSC_ERROR)); } diff --git a/dsc/src/server/lint_dsc_config.rs b/dsc/src/server/lint_dsc_config.rs new file mode 100644 index 000000000..a4e7e28e8 --- /dev/null +++ b/dsc/src/server/lint_dsc_config.rs @@ -0,0 +1,118 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +use std::collections::HashMap; + +use crate::server::mcp_server::McpServer; +use crate::util::current_dsc_version; +use dsc_lib::{ + dscerror::DscError, + lint::{LintError, LintOptions, LintResult, Linter, RuleLevel, RuleLevelOverride}, + progress::ProgressFormat, +}; +use rmcp::{handler::server::wrapper::Parameters, tool, tool_router, ErrorData as McpError, Json}; +use rust_i18n::t; +use schemars::JsonSchema; +use serde::{Deserialize, Serialize}; +use serde_json::Value; +use tokio::task; + +#[derive(Serialize, JsonSchema)] +pub struct LintDscConfigResponse { + pub result: LintResult, +} + +#[derive(Deserialize, JsonSchema)] +pub struct LintDscConfigRequest { + #[schemars(description = "The DSC configuration document as a JSON or YAML string")] + pub configuration: String, + #[schemars( + description = "Optional parameters to check against the configuration as a JSON or YAML string, with or without a top-level 'parameters' wrapper" + )] + pub parameters: Option, + #[schemars( + description = "When false, skip resource discovery and the rules that inspect installed resources. Defaults to true." + )] + #[serde(default)] + pub discovery: Option, + #[schemars(description = "Optional rule levels for this run, keyed by rule identifier")] + #[serde(default)] + pub rules: Option>, +} + +/// Parse the parameters of a lint request into the input format `dsc config` accepts. +/// +/// # Arguments +/// +/// * `parameters` - The parameters as JSON or YAML, with or without a top-level `parameters` wrapper. +/// +/// # Errors +/// +/// Returns an invalid-request error when the text isn't valid JSON or YAML. +fn parse_parameters(parameters: &str) -> Result { + let yaml = serde_yaml::from_str::(parameters) + .map_err(|e| McpError::invalid_request(format!("{}: {e}", t!("server.lint_dsc_config.invalidParameters")), None))?; + let json = serde_json::to_value(yaml) + .map_err(|e| McpError::invalid_request(format!("{}: {e}", t!("server.lint_dsc_config.failedConvertJson")), None))?; + if json.get("parameters").is_some_and(Value::is_object) { + Ok(json) + } else { + Ok(serde_json::json!({ "parameters": json })) + } +} + +/// Convert a linter error into the MCP error for it. +/// +/// # Arguments +/// +/// * `error` - The error from the linter. +fn lint_error(error: DscError) -> McpError { + let message = format!("{}: {error}", t!("server.lint_dsc_config.lintFailed")); + match error { + DscError::Lint(LintError::UnknownRule(_)) => McpError::invalid_params(message, None), + DscError::Lint(LintError::InvalidParameters(_)) => McpError::invalid_request(message, None), + _ => McpError::internal_error(message, None), + } +} + +#[tool_router(router = lint_dsc_config_router, vis = "pub")] +impl McpServer { + #[tool( + description = "Lint a DSC configuration document without applying it. Returns diagnostics for problems such as invalid expressions, undeclared parameters, unresolved dependencies, and, with discovery, property values that don't match the resource schema.", + annotations( + title = "Lint a DSC configuration document", + read_only_hint = true, + destructive_hint = false, + idempotent_hint = true, + open_world_hint = false, + ) + )] + pub async fn lint_dsc_config( + &self, + Parameters(LintDscConfigRequest { + configuration, + parameters, + discovery, + rules, + }): Parameters, + ) -> Result, McpError> { + let result = task::spawn_blocking(move || { + let parameters = parameters.as_deref().map(parse_parameters).transpose()?; + let rule_levels = rules.unwrap_or_default().into_iter().map(|(id, level)| RuleLevelOverride { id, level }).collect(); + let options = LintOptions { + discovery: discovery.unwrap_or(true), + rule_levels, + parameters, + dsc_version: Some(current_dsc_version()), + progress_format: ProgressFormat::None, + ..LintOptions::default() + }; + let mut linter = Linter::new(options).map_err(lint_error)?; + linter.lint(&configuration).map_err(lint_error) + }) + .await + .map_err(|e| McpError::internal_error(e.to_string(), None))??; + + Ok(Json(LintDscConfigResponse { result })) + } +} diff --git a/dsc/src/server/mcp_server.rs b/dsc/src/server/mcp_server.rs index b254314b9..d21f34ddf 100644 --- a/dsc/src/server/mcp_server.rs +++ b/dsc/src/server/mcp_server.rs @@ -27,6 +27,7 @@ impl McpServer { + Self::invoke_dsc_function_router() + Self::invoke_dsc_resource_router() + Self::list_dsc_actions_router() + + Self::lint_dsc_config_router() + Self::list_dsc_functions_router() + Self::list_dsc_resources_router() + Self::show_dsc_action_router() diff --git a/dsc/src/server/mod.rs b/dsc/src/server/mod.rs index 543509501..5d1124c4b 100644 --- a/dsc/src/server/mod.rs +++ b/dsc/src/server/mod.rs @@ -15,6 +15,7 @@ pub mod invoke_dsc_expression; pub mod invoke_dsc_function; pub mod invoke_dsc_resource; pub mod list_dsc_actions; +pub mod lint_dsc_config; pub mod list_dsc_functions; pub mod list_dsc_resources; pub mod mcp_server; diff --git a/dsc/src/subcommand.rs b/dsc/src/subcommand.rs index 3603f0294..b5b5f0b44 100644 --- a/dsc/src/subcommand.rs +++ b/dsc/src/subcommand.rs @@ -1,12 +1,15 @@ // Copyright (c) Microsoft Corporation. // Licensed under the MIT License. -use crate::args::{ConfigSubCommand, SchemaType, ActionSubCommand, ExtensionSubCommand, FunctionSubCommand, GetOutputFormat, ListOutputFormat, OutputFormat, ResourceSubCommand}; +use crate::args::{ConfigSubCommand, SchemaType, ActionSubCommand, ExtensionSubCommand, FunctionSubCommand, GetOutputFormat, ListOutputFormat, OutputFormat, PluginSubCommand, ResourceSubCommand}; +use crate::plugin::reserved_commands; +use crate::report::render; use crate::resolve::{get_contents, Include}; use crate::resource_command::{get_resource, self}; use crate::tablewriter::Table; -use crate::util::{current_dsc_version, get_input, get_schema, in_desired_state, set_dscconfigroot, write_object, DSC_CONFIG_ROOT, EXIT_DSC_ASSERTION_FAILED, EXIT_DSC_ERROR, EXIT_INVALID_ARGS, EXIT_INVALID_INPUT, EXIT_JSON_ERROR, EXIT_VALIDATION_FAILED}; +use crate::util::{current_dsc_version, get_input, get_input_with_source, get_schema, in_desired_state, set_dscconfigroot, write_object, DSC_CONFIG_ROOT, EXIT_DSC_ASSERTION_FAILED, EXIT_DSC_ERROR, EXIT_INVALID_ARGS, EXIT_INVALID_INPUT, EXIT_JSON_ERROR, EXIT_VALIDATION_FAILED}; use dsc_lib::actions::action_manifest::SupportedOperation; +use dsc_lib::diagnostics::DocumentSource; use dsc_lib::dscresources::dscresource::ResourceCapability; use dsc_lib::types::{FullyQualifiedTypeName, ResourceVersionReq, TypeNameFilter}; use dsc_lib::{configure::{config_doc::{Configuration, ExecutionKind, Resource}, @@ -18,6 +21,7 @@ use dsc_lib::{configure::{config_doc::{Configuration, ExecutionKind, Resource}, dscresources::dscresource::{validate_json, validate_properties}, extensions::dscextension::Capability as ExtensionCapability, functions::{FunctionCategory, FunctionDispatcher}, + plugins::discovery::discover_plugins, progress::ProgressFormat, resources::invoke_result::{TestResult, ValidateResult, ResolveResult}, util::convert_wildcard_to_regex, @@ -60,7 +64,7 @@ pub fn config_get(configurator: &mut Configurator, format: Option<&OutputFormat> } }, Err(err) => { - error!("{err}"); + error!("{}", render(&err)); return Err(ExitCode::from(EXIT_DSC_ERROR)); } } @@ -95,7 +99,7 @@ pub fn config_set(configurator: &mut Configurator, format: Option<&OutputFormat> } }, Err(err) => { - error!("Error: {err}"); + error!("Error: {}", render(&err)); return Err(ExitCode::from(EXIT_DSC_ERROR)); } } @@ -196,7 +200,7 @@ pub fn config_test(configurator: &mut Configurator, format: Option<&OutputFormat } }, Err(err) => { - error!("{err}"); + error!("{}", render(&err)); let exit_code = config_test_exit_code(&err); return Err(ExitCode::from(exit_code)); } @@ -206,7 +210,7 @@ pub fn config_test(configurator: &mut Configurator, format: Option<&OutputFormat fn config_test_exit_code(error: &DscError) -> u8 { match error { - DscError::Schema(_) => EXIT_VALIDATION_FAILED, + DscError::Schema(_) | DscError::SchemaValidation(_) => EXIT_VALIDATION_FAILED, _ => EXIT_DSC_ERROR, } } @@ -214,10 +218,13 @@ fn config_test_exit_code(error: &DscError) -> u8 { #[cfg(test)] mod tests { use super::{config_test_exit_code, DscError, EXIT_DSC_ERROR, EXIT_VALIDATION_FAILED}; + use dsc_lib::diagnostics::SchemaValidationError; #[test] fn schema_errors_use_validation_failed_exit_code() { assert_eq!(config_test_exit_code(&DscError::Schema("invalid value".to_string())), EXIT_VALIDATION_FAILED); + let violations = DscError::SchemaValidation(Box::new(SchemaValidationError::new("input for 'Test/Echo'", Vec::new()))); + assert_eq!(config_test_exit_code(&violations), EXIT_VALIDATION_FAILED); } #[test] @@ -248,7 +255,7 @@ pub fn config_export(configurator: &mut Configurator, format: Option<&OutputForm } }, Err(err) => { - error!("{err}"); + error!("{}", render(&err)); return Err(ExitCode::from(EXIT_DSC_ERROR)); } } @@ -288,14 +295,14 @@ fn initialize_config_root(path: Option<&String>) -> Result, ExitC #[allow(clippy::too_many_lines)] #[allow(clippy::too_many_arguments)] pub fn config(subcommand: &ConfigSubCommand, parameters: &Option, mounted_path: Option<&String>, as_group: &bool, as_assert: &bool, as_include: &bool, progress_format: ProgressFormat) -> Result<(), ExitCode> { - let (new_parameters, json_string) = match subcommand { + let (new_parameters, json_string, source) = match subcommand { ConfigSubCommand::Get { input, file, .. } | ConfigSubCommand::Set { input, file, .. } | ConfigSubCommand::Test { input, file, .. } | ConfigSubCommand::Validate { input, file, .. } | ConfigSubCommand::Export { input, file, .. } => { let new_path = initialize_config_root(file.as_ref())?; - let document = get_input(input.as_ref(), new_path.as_ref())?; + let (document, source) = get_input_with_source(input.as_ref(), new_path.as_ref())?; if *as_include { let (new_parameters, config_json) = match get_contents(&document) { Ok((parameters, config_json)) => (parameters, config_json), @@ -304,9 +311,9 @@ pub fn config(subcommand: &ConfigSubCommand, parameters: &Option, mounte return Err(ExitCode::from(EXIT_DSC_ERROR)); } }; - (new_parameters, config_json) + (new_parameters, config_json, None) } else { - (None, document) + (None, document, source) } }, ConfigSubCommand::Resolve { input, file, .. } => { @@ -319,17 +326,20 @@ pub fn config(subcommand: &ConfigSubCommand, parameters: &Option, mounte return Err(ExitCode::from(EXIT_DSC_ERROR)); } }; - (new_parameters, config_json) + (new_parameters, config_json, None) } }; let mut configurator = match Configurator::new_with_dsc_version(&json_string, progress_format, current_dsc_version()) { Ok(configurator) => configurator, Err(err) => { - error!("Error: {err}"); + error!("Error: {}", render(&err)); return Err(ExitCode::from(EXIT_DSC_ERROR)); } }; + if let Some(source) = source { + configurator.set_source(source); + } if let ConfigSubCommand::Set { what_if , .. } = subcommand && *what_if { configurator.context.execution_type = ExecutionKind::WhatIf; @@ -346,26 +356,7 @@ pub fn config(subcommand: &ConfigSubCommand, parameters: &Option, mounte }, Some(parameters) => { debug!("{}", t!("subcommand.parameters")); - match serde_json::from_str(parameters) { - Ok(json) => Some(json), - Err(_) => { - match serde_yaml::from_str::(parameters) { - Ok(yaml) => { - match serde_json::to_value(yaml) { - Ok(json) => Some(json), - Err(err) => { - error!("{}: {err}", t!("subcommand.failedConvertJson")); - return Err(ExitCode::from(EXIT_DSC_ERROR)); - } - } - }, - Err(err) => { - error!("{}: {err}", t!("subcommand.invalidParameters")); - return Err(ExitCode::from(EXIT_INVALID_INPUT)); - } - } - } - } + Some(parse_parameters(parameters)?) } }; @@ -416,12 +407,12 @@ pub fn config(subcommand: &ConfigSubCommand, parameters: &Option, mounte } } } else { - match validate_config(configurator.get_config(), progress_format) { + match validate_config(configurator.get_config(), configurator.source(), progress_format) { Ok(()) => { // valid, so do nothing }, Err(err) => { - error!("{err}"); + error!("{}", render(&err)); result.valid = false; } } @@ -472,11 +463,46 @@ pub fn config(subcommand: &ConfigSubCommand, parameters: &Option, mounte Ok(()) } +/// Parse parameter input as JSON, falling back to YAML. +/// +/// # Arguments +/// +/// * `parameters` - The parameters as JSON or YAML text. +/// +/// # Errors +/// +/// Returns `EXIT_INVALID_INPUT` when the text is neither JSON nor YAML, or `EXIT_DSC_ERROR` when +/// the YAML can't be converted to JSON. +fn parse_parameters(parameters: &str) -> Result { + match serde_json::from_str(parameters) { + Ok(json) => Ok(json), + Err(_) => { + match serde_yaml::from_str::(parameters) { + Ok(yaml) => { + match serde_json::to_value(yaml) { + Ok(json) => Ok(json), + Err(err) => { + error!("{}: {err}", t!("subcommand.failedConvertJson")); + Err(ExitCode::from(EXIT_DSC_ERROR)) + } + } + }, + Err(err) => { + error!("{}: {err}", t!("subcommand.invalidParameters")); + Err(ExitCode::from(EXIT_INVALID_INPUT)) + } + } + } + } +} + /// Validate configuration. /// /// # Arguments /// /// * `config` - The configuration to validate. +/// * `source` - The text the configuration was read from, so that errors point into it. +/// * `progress_format` - How to report progress during resource discovery. /// /// # Returns /// @@ -485,12 +511,12 @@ pub fn config(subcommand: &ConfigSubCommand, parameters: &Option, mounte /// # Errors /// /// * `DscError` - The error that occurred. -pub fn validate_config(config: &Configuration, progress_format: ProgressFormat) -> Result<(), DscError> { +pub fn validate_config(config: &Configuration, source: Option<&DocumentSource>, progress_format: ProgressFormat) -> Result<(), DscError> { // first validate against the config schema debug!("{}", t!("subcommand.validatingConfiguration")); let schema = serde_json::to_value(get_schema(SchemaType::Configuration))?; let config_value = serde_json::to_value(config)?; - validate_json("Configuration", &schema, &config_value)?; + validate_json("Configuration", &schema, &config_value).map_err(|err| anchor_error(err, source, ""))?; let mut dsc = DscManager::new(); // then validate each resource @@ -513,7 +539,7 @@ pub fn validate_config(config: &Configuration, progress_format: ProgressFormat) } dsc.find_resources(&resource_types, progress_format)?; - for resource_block in resources { + for (index, resource_block) in resources.iter().enumerate() { let Some(type_name) = resource_block["type"].as_str() else { return Err(DscError::Validation(t!("subcommand.resourceTypeNotSpecified").to_string())); }; @@ -531,7 +557,8 @@ pub fn validate_config(config: &Configuration, progress_format: ProgressFormat) }; let context = Context::new(); - validate_properties(&*resource, &resource_block["properties"], &context)?; + validate_properties(&*resource, &resource_block["properties"], &context) + .map_err(|err| anchor_error(err, source, &format!("/resources/{index}/properties")))?; } Ok(()) @@ -548,6 +575,22 @@ pub fn action(subcommand: &ActionSubCommand, progress_format: ProgressFormat) -> Ok(()) } +/// Point a schema validation error at the document it was found in. +/// +/// # Arguments +/// +/// * `error` - The error. +/// * `source` - The document text, when known. +/// * `base` - The pointer in the document that the error's paths are relative to. +fn anchor_error(error: DscError, source: Option<&DocumentSource>, base: &str) -> DscError { + match (error, source) { + (DscError::SchemaValidation(schema_error), Some(source)) => { + DscError::SchemaValidation(Box::new(schema_error.with_source(&source.text, &source.name, base))) + }, + (error, _) => error, + } +} + pub fn extension(subcommand: &ExtensionSubCommand, progress_format: ProgressFormat) -> Result<(), ExitCode> { let mut dsc = DscManager::new(); @@ -559,6 +602,24 @@ pub fn extension(subcommand: &ExtensionSubCommand, progress_format: ProgressForm Ok(()) } +/// Runs a subcommand of the `plugin` command. +/// +/// # Arguments +/// +/// * `subcommand` - The subcommand to run. +/// +/// # Errors +/// +/// Returns the exit code for the failure when the subcommand fails. +pub fn plugin(subcommand: &PluginSubCommand) -> Result<(), ExitCode> { + match subcommand { + PluginSubCommand::List { plugin_name, output_format } => { + list_plugins(plugin_name, output_format.as_ref())?; + }, + } + Ok(()) +} + pub fn function(subcommand: &FunctionSubCommand) -> Result<(), ExitCode> { let functions = FunctionDispatcher::new(); match subcommand { @@ -804,6 +865,84 @@ fn list_extensions(dsc: &mut DscManager, extension_name: &TypeNameFilter, format Ok(()) } +/// Lists the discovered command plug-ins. +/// +/// The plug-ins are listed in the order DSC found them, which is the order that decides which +/// plug-in provides a command when more than one defines it. A plug-in that doesn't provide its +/// command because of that is marked as shadowed. +/// +/// # Arguments +/// +/// * `plugin_name` - The filter for the type name of the plug-ins to list. +/// * `format` - The output format, if the user specified one. +/// +/// # Errors +/// +/// Returns the exit code for a DSC error when DSC can't resolve the directories to search, and the +/// exit code for a JSON error when a plug-in can't be serialized. +fn list_plugins(plugin_name: &TypeNameFilter, format: Option<&ListOutputFormat>) -> Result<(), ExitCode> { + let write_table = should_write_table(format); + + let mut table = Table::new(&[ + t!("subcommand.tableHeader_command").to_string().as_ref(), + t!("subcommand.tableHeader_type").to_string().as_ref(), + t!("subcommand.tableHeader_version").to_string().as_ref(), + t!("subcommand.tableHeader_description").to_string().as_ref(), + ]); + + let plugins = match discover_plugins(&reserved_commands()) { + Ok(plugins) => plugins, + Err(err) => { + error!("{}: {err}", t!("subcommand.failedDiscoverPlugins")); + return Err(ExitCode::from(EXIT_DSC_ERROR)); + } + }; + + let mut include_separator = false; + + for plugin in plugins.iter().filter(|plugin| plugin_name.is_match(&plugin.type_name)) { + if write_table { + let command = if plugin.shadowed { + t!("subcommand.pluginShadowed", command = plugin.command).to_string() + } else { + plugin.command.to_string() + }; + table.add_row(vec![ + command, + plugin.type_name.to_string(), + plugin.version.to_string(), + plugin.description.clone().unwrap_or_default() + ]); + } + else { + // convert to json + let json = match serde_json::to_string(plugin) { + Ok(json) => json, + Err(err) => { + error!("JSON: {err}"); + return Err(ExitCode::from(EXIT_JSON_ERROR)); + } + }; + let format = match format { + Some(ListOutputFormat::Json) => Some(OutputFormat::Json), + Some(ListOutputFormat::PrettyJson) => Some(OutputFormat::PrettyJson), + Some(ListOutputFormat::Yaml) => Some(OutputFormat::Yaml), + _ => None, + }; + write_object(&json, format.as_ref(), include_separator)?; + include_separator = true; + // insert newline separating instances if writing to console + if io::stdout().is_terminal() { println!(); } + } + } + + if write_table { + let truncate = format != Some(&ListOutputFormat::TableNoTruncate); + table.print(truncate); + } + Ok(()) +} + fn list_functions(functions: &FunctionDispatcher, function_name: Option<&String>, category: &[FunctionCategory], description: Option<&String>, output_format: Option<&ListOutputFormat>) -> Result<(), ExitCode> { let write_table = should_write_table(output_format); let mut table = Table::new(&[ diff --git a/dsc/src/util.rs b/dsc/src/util.rs index 874cdc4a0..d32cd5fc4 100644 --- a/dsc/src/util.rs +++ b/dsc/src/util.rs @@ -21,6 +21,7 @@ use dsc_lib::{ ResourceTestResult, }, }, + diagnostics::DocumentSource, discovery::{ command_discovery::ManifestList, Discovery, @@ -36,7 +37,9 @@ use dsc_lib::{ extension_manifest::ExtensionManifest, }, functions::FunctionDefinition, + plugins::plugin_manifest::PluginManifest, resources::invoke_result::{GetResult, ResolveResult, SetResult, TestResult}, + lint::{LintResult, LintRuleInfo}, schemas::dsc_repo::{DscRepoSchema, RecognizedSchemaVersion, SchemaForm, SchemaUriPrefix}, types::SemanticVersion, util::{ @@ -199,9 +202,18 @@ pub fn get_schema(schema: SchemaType) -> Schema { SchemaType::Include => { schema_for!(Include) }, + SchemaType::LintResult => { + repo_schema::() + }, + SchemaType::LintRules => { + repo_schema::() + }, SchemaType::ManifestList => { schema_for!(ManifestList) }, + SchemaType::PluginManifest => { + repo_schema::() + }, SchemaType::ResolveResult => { repo_schema::() }, @@ -360,6 +372,7 @@ pub fn enable_tracing(resolved_settings: &DscSettingsResolved) { tracing_level, stderr_format }.init_subscriber(); + crate::report::set_stderr_format(stderr_format); drop(default_guard); if tracing::subscriber::set_global_default(subscriber).is_err() { @@ -369,7 +382,80 @@ pub fn enable_tracing(resolved_settings: &DscSettingsResolved) { info!("Trace-level is {:?}", tracing_setting.level.value); } +/// Read an input document and convert it to a JSON string. +/// +/// # Arguments +/// +/// * `input` - The inline document, if provided. +/// * `file` - The path to the document, or `-` to read it from STDIN. +/// +/// # Returns +/// +/// The document as JSON, or an empty string when neither argument is provided. +/// +/// # Errors +/// +/// Returns `EXIT_INVALID_INPUT` when the document can't be read (see [`read_input`]) or isn't +/// valid JSON or YAML. pub fn get_input(input: Option<&String>, file: Option<&String>) -> Result { + get_input_with_source(input, file).map(|(json, _)| json) +} + +/// Read an input document, converting it to a JSON string and keeping the text it was read from. +/// +/// # Arguments +/// +/// * `input` - The inline document, if provided. +/// * `file` - The path to the document, or `-` to read it from STDIN. +/// +/// # Returns +/// +/// The document as JSON and, when there is a document, its text named after the file it was +/// read from, or `input` for inline text and STDIN. Both are empty when neither argument is +/// provided. +/// +/// # Errors +/// +/// Returns `EXIT_INVALID_INPUT` when the document can't be read (see [`read_input`]) or isn't +/// valid JSON or YAML. +pub fn get_input_with_source(input: Option<&String>, file: Option<&String>) -> Result<(String, Option), ExitCode> { + let text = read_input(input, file)?; + if text.is_empty() { + return Ok((text, None)); + } + + match parse_input_to_json(&text) { + Ok(json) => { + let name = match file { + Some(path) if path != "-" => path.clone(), + _ => t!("util.inputSourceName").to_string(), + }; + Ok((json, Some(DocumentSource::new(name, text)))) + }, + Err(err) => { + error!("{}: {err}", t!("util.failedToParseInput")); + Err(ExitCode::from(EXIT_INVALID_INPUT)) + } + } +} + +/// Read an input document as text without parsing it. +/// +/// # Arguments +/// +/// * `input` - The inline document, if provided. +/// * `file` - The path to the document, or `-` to read it from STDIN. Extensions with the +/// `import` capability get the first chance to handle a file path. +/// +/// # Returns +/// +/// The document text, or an empty string when neither argument is provided. +/// +/// # Errors +/// +/// Returns `EXIT_INVALID_INPUT` when the inline input is actually a path to an existing file, the +/// file can't be read, STDIN isn't valid UTF-8, or the document is empty. +pub fn read_input(input: Option<&String>, file: Option<&String>) -> Result { trace!("Input: {input:?}, File: {file:?}"); let value = if let Some(input) = input { debug!("{}", t!("util.readingInput")); @@ -438,13 +524,7 @@ pub fn get_input(input: Option<&String>, file: Option<&String>) -> Result Ok(json), - Err(err) => { - error!("{}: {err}", t!("util.failedToParseInput")); - Err(ExitCode::from(EXIT_INVALID_INPUT)) - } - } + Ok(value) } /// Sets `DSC_CONFIG_ROOT` env var and makes path absolute. diff --git a/dsc/tests/other/dsc_diagnostics.tests.ps1 b/dsc/tests/other/dsc_diagnostics.tests.ps1 new file mode 100644 index 000000000..d1f73fb36 --- /dev/null +++ b/dsc/tests/other/dsc_diagnostics.tests.ps1 @@ -0,0 +1,84 @@ +# Copyright (c) Microsoft Corporation. +# Licensed under the MIT License. + +Describe 'Schema validation diagnostics' { + BeforeAll { + $schemaUri = 'https://aka.ms/dsc/schemas/v3/bundled/config/document.json' + $configYaml = @" +`$schema: $schemaUri +resources: +- name: first + type: Microsoft.DSC.Debug/Echo + properties: + output: fine +- name: second + type: Microsoft.DSC.Debug/Echo + properties: + otput: hi +"@ + $configPath = Join-Path $TestDrive 'typo.dsc.yaml' + Set-Content -Path $configPath -Value $configYaml + } + + It 'shows the invalid property of resource input in a snippet with a suggestion' { + $resourceInput = @' +{ + "otput": "hi" +} +'@ + $null = dsc resource get --resource Microsoft.DSC.Debug/Echo --input $resourceInput 2> $TestDrive/error.log + $LASTEXITCODE | Should -Be 2 + $log = Get-Content -Path $TestDrive/error.log -Raw + $log | Should -Match "Schema: input for 'Microsoft.DSC.Debug/Echo'" -Because $log + $log | Should -Match "'otput' was unexpected" -Because $log + # the snippet opens at the first label, which marks the object missing its required property + $log | Should -Match '\[input:1:1\]' -Because $log + $log | Should -Match "'otput' isn't a known property" -Because $log + $log | Should -Match "missing required property 'output'" -Because $log + $log | Should -Match "Did you mean 'output' instead of 'otput'" -Because $log + } + + It 'points at the resource declaration in the configuration file for: ' -TestCases @( + @{ operation = 'get'; exitCode = 2 } + @{ operation = 'test'; exitCode = 5 } + ) { + param($operation, $exitCode) + + $null = dsc config $operation --file $configPath 2> $TestDrive/error.log + $LASTEXITCODE | Should -Be $exitCode + $log = Get-Content -Path $TestDrive/error.log -Raw + $log | Should -Match "Schema: input for 'Microsoft.DSC.Debug/Echo'" -Because $log + # the snippet is the author's YAML, not the evaluated properties + $log | Should -Match ([regex]::Escape($configPath) + ':9:3') -Because $log + $log | Should -Match '- name: second' -Because $log + $log | Should -Match 'otput: hi' -Because $log + $log | Should -Match "'otput' isn't a known property" -Because $log + $log | Should -Match "Did you mean 'output' instead of 'otput'" -Because $log + $log | Should -Not -Match 'name: first' -Because 'only the failing resource is shown' + } + + It 'names inline input as input in the snippet' { + $null = dsc config get --input $configYaml 2> $TestDrive/error.log + $LASTEXITCODE | Should -Be 2 + $log = Get-Content -Path $TestDrive/error.log -Raw + $log | Should -Match '\[input:9:3\]' -Because $log + } + + It 'keeps the message on one line for the json trace format' { + $null = dsc --trace-format json config get --file $configPath 2> $TestDrive/error.log + $LASTEXITCODE | Should -Be 2 + $errors = @(Get-Content -Path $TestDrive/error.log | Where-Object { $_ -like '*"level":"ERROR"*' }) + $errors.Count | Should -Be 1 -Because (Get-Content -Path $TestDrive/error.log -Raw) + $entry = $errors[0] | ConvertFrom-Json + $entry.fields.message | Should -Match "^Schema: input for 'Microsoft.DSC.Debug/Echo': Additional properties are not allowed \('otput' was unexpected\)" -Because $entry.fields.message + $entry.fields.message | Should -Not -Match 'otput: hi' -Because 'the snippet is only rendered for the default and plaintext formats' + } + + It 'renders the snippet without colors for the plaintext trace format' { + $null = dsc --trace-format plaintext config get --file $configPath 2> $TestDrive/error.log + $LASTEXITCODE | Should -Be 2 + $log = Get-Content -Path $TestDrive/error.log -Raw + $log | Should -Match "'otput' isn't a known property" -Because $log + $log | Should -Not -Match "`e\[" -Because 'plaintext output has no ANSI escape sequences' + } +} diff --git a/dsc/tests/other/dsc_plugin.tests.ps1 b/dsc/tests/other/dsc_plugin.tests.ps1 new file mode 100644 index 000000000..6c7a9f114 --- /dev/null +++ b/dsc/tests/other/dsc_plugin.tests.ps1 @@ -0,0 +1,657 @@ +# Copyright (c) Microsoft Corporation. +# Licensed under the MIT License. + +Describe 'Command plug-in tests' { + BeforeAll { + $script:oldPath = $env:PATH + $script:pathSeparator = [System.IO.Path]::PathSeparator + $script:manifestSchema = 'https://aka.ms/dsc/schemas/v3/bundled/plugin/manifest.json' + + # A plug-in that reports how it was invoked as one line of JSON. + $script:echoScript = @' +if ($args -contains '--help') { + "Usage: dsc $env:DSC_PLUGIN_COMMAND [OPTIONS]" + "HELP_ARGS: $($args -join '|')" + exit 0 +} + +$exitCode = 0 +$stdin = $null +for ($i = 0; $i -lt $args.Count; $i++) { + if ($args[$i] -eq '--exit-code') { $exitCode = [int]$args[$i + 1] } + if ($args[$i] -eq '--read-stdin') { $stdin = @($input) -join "`n" } +} + +[ordered]@{ + args = @($args) + # Report a marker file instead of the path, which can differ in form from the path the test + # used when a parent directory is a symbolic link. + cwdHasMarker = Test-Path -Path (Join-Path (Get-Location).Path 'plugin-cwd-marker.txt') + stdin = $stdin + env = [ordered]@{ + DSC_EXE = $env:DSC_EXE + DSC_VERSION = $env:DSC_VERSION + DSC_PLUGIN_COMMAND = $env:DSC_PLUGIN_COMMAND + DSC_PLUGIN_DIR = $env:DSC_PLUGIN_DIR + DSC_TRACE_LEVEL = $env:DSC_TRACE_LEVEL + DSC_TRACE_FORMAT = $env:DSC_TRACE_FORMAT + DSC_PROGRESS_FORMAT = $env:DSC_PROGRESS_FORMAT + DSC_IGNORE_SETTINGS_FILE = $env:DSC_IGNORE_SETTINGS_FILE + } +} | ConvertTo-Json -Compress -Depth 5 + +exit $exitCode +'@ + + function New-TestPluginManifest { + <# + .SYNOPSIS + Returns the manifest for a plug-in that runs the echo script next to the manifest. + #> + param( + [Parameter(Mandatory)][string]$Type, + [Parameter(Mandatory)][string]$Command, + [string]$Version = '1.0.0', + [string]$Description = 'Report how the plug-in was invoked', + [hashtable]$Extra = @{} + ) + + $manifest = [ordered]@{ + '$schema' = $script:manifestSchema + type = $Type + version = $Version + description = $Description + command = [ordered]@{ + name = $Command + executable = 'pwsh' + args = @( + '-NoLogo' + '-NoProfile' + '-NonInteractive' + '-File' + @{ manifestRelativePath = 'echo.ps1' } + ) + } + } + foreach ($key in $Extra.Keys) { + $manifest[$key] = $Extra[$key] + } + $manifest + } + + function New-TestPlugin { + <# + .SYNOPSIS + Writes a plug-in manifest and the echo script to a directory. + #> + param( + [Parameter(Mandatory)][string]$Directory, + [Parameter(Mandatory)][string]$Type, + [Parameter(Mandatory)][string]$Command, + [string]$Version = '1.0.0', + [string]$Description = 'Report how the plug-in was invoked', + [hashtable]$Extra = @{} + ) + + $null = New-Item -ItemType Directory -Force -Path $Directory + Set-Content -Path (Join-Path $Directory 'echo.ps1') -Value $script:echoScript + $fileName = '{0}.{1}.dsc.plugin.json' -f ($Type -replace '/', '.'), $Command + New-TestPluginManifest -Type $Type -Command $Command -Version $Version -Description $Description -Extra $Extra | + ConvertTo-Json -Depth 10 | + Set-Content -Path (Join-Path $Directory $fileName) + } + + function Use-PluginDirectory { + <# + .SYNOPSIS + Makes DSC search the directories for manifests, before every other directory. + #> + param([Parameter(Mandatory)][string[]]$Directory) + + $env:PATH = ($Directory + $script:oldPath) -join $script:pathSeparator + } + + $script:pluginDir = Join-Path $TestDrive 'plugins' + New-TestPlugin -Directory $script:pluginDir -Type 'Test/Echo' -Command 'echo' -Version '1.2.3' + Use-PluginDirectory -Directory $script:pluginDir + } + + AfterAll { + $env:PATH = $script:oldPath + } + + Context 'Listing plug-ins' { + It 'Lists a plug-in with its command and manifest' { + $out = dsc plugin list 'Test/Echo' 2> $TestDrive/error.txt | ConvertFrom-Json + $LASTEXITCODE | Should -Be 0 -Because (Get-Content -Raw $TestDrive/error.txt) + @($out).Count | Should -Be 1 + $out.type | Should -BeExactly 'Test/Echo' + $out.version | Should -BeExactly '1.2.3' + $out.command | Should -BeExactly 'echo' + $out.description | Should -BeExactly 'Report how the plug-in was invoked' + $out.deprecationMessage | Should -BeNullOrEmpty + $out.path | Should -BeExactly (Join-Path $pluginDir 'Test.Echo.echo.dsc.plugin.json') + $out.directory | Should -BeExactly $pluginDir + $out.shadowed | Should -BeFalse + $out.manifest.command.name | Should -BeExactly 'echo' + $out.manifest.command.executable | Should -BeExactly 'pwsh' + } + + It 'Filters the list with a wildcard: ' -TestCases @( + @{ filter = 'Test/*'; expected = 1 } + @{ filter = '*echo'; expected = 1 } + @{ filter = 'Test/DoesNotExist'; expected = 0 } + ) { + param($filter, $expected) + + $out = dsc plugin list $filter 2> $TestDrive/error.txt | ConvertFrom-Json + $LASTEXITCODE | Should -Be 0 -Because (Get-Content -Raw $TestDrive/error.txt) + @($out).Count | Should -Be $expected + } + + It 'Writes a table when asked to' { + $out = dsc plugin list 'Test/Echo' -o table-no-truncate 2> $TestDrive/error.txt | Out-String + $LASTEXITCODE | Should -Be 0 -Because (Get-Content -Raw $TestDrive/error.txt) + $out | Should -Match 'Command\s+Type\s+Version\s+Description' + $out | Should -Match 'echo\s+Test/Echo\s+1\.2\.3\s+Report how the plug-in was invoked' + } + + It 'Finds plug-ins in YAML manifests and manifest lists' { + $directory = Join-Path $TestDrive 'formats' + New-TestPlugin -Directory $directory -Type 'Test.Formats/Json' -Command 'formats-json' + @" +`$schema: $manifestSchema +type: Test.Formats/Yaml +version: 1.0.0 +command: + name: formats-yaml + executable: pwsh +"@ | Set-Content -Path (Join-Path $directory 'yaml.dsc.plugin.yaml') + @{ + plugins = @( + New-TestPluginManifest -Type 'Test.Formats/ListOne' -Command 'formats-list-one' + New-TestPluginManifest -Type 'Test.Formats/ListTwo' -Command 'formats-list-two' + ) + } | ConvertTo-Json -Depth 10 | Set-Content -Path (Join-Path $directory 'tools.dsc.manifests.json') + + try { + Use-PluginDirectory -Directory $directory + $out = dsc plugin list 'Test.Formats/*' 2> $TestDrive/error.txt | ConvertFrom-Json + $LASTEXITCODE | Should -Be 0 -Because (Get-Content -Raw $TestDrive/error.txt) + $out.command | Sort-Object | Should -Be @( + 'formats-json' + 'formats-list-one' + 'formats-list-two' + 'formats-yaml' + ) + + $result = dsc formats-list-two 2> $TestDrive/error.txt | ConvertFrom-Json + $LASTEXITCODE | Should -Be 0 -Because (Get-Content -Raw $TestDrive/error.txt) + $result.env.DSC_PLUGIN_COMMAND | Should -BeExactly 'formats-list-two' + } finally { + Use-PluginDirectory -Directory $pluginDir + } + } + } + + Context 'Running a plug-in command' { + It 'Passes the arguments after the command name to the plug-in as written' { + $out = dsc echo one --two 'three four' -l trace --help-me -- five 2> $TestDrive/error.txt | ConvertFrom-Json + $LASTEXITCODE | Should -Be 0 -Because (Get-Content -Raw $TestDrive/error.txt) + $out.args | Should -Be @('one', '--two', 'three four', '-l', 'trace', '--help-me', '--', 'five') + } + + It 'Runs the plug-in in the working directory of the caller' { + $out = dsc echo 2> $TestDrive/error.txt | ConvertFrom-Json + $LASTEXITCODE | Should -Be 0 -Because (Get-Content -Raw $TestDrive/error.txt) + $out.cwdHasMarker | Should -BeFalse + + $workingDirectory = New-Item -ItemType Directory -Force -Path (Join-Path $TestDrive 'working') + $null = New-Item -ItemType File -Force -Path (Join-Path $workingDirectory 'plugin-cwd-marker.txt') + Push-Location $workingDirectory + try { + $out = dsc echo 2> $TestDrive/error.txt | ConvertFrom-Json + $LASTEXITCODE | Should -Be 0 -Because (Get-Content -Raw $TestDrive/error.txt) + $out.cwdHasMarker | Should -BeTrue + } finally { + Pop-Location + } + } + + It 'Passes standard input to the plug-in' { + $out = 'hello from stdin' | dsc echo --read-stdin 2> $TestDrive/error.txt | ConvertFrom-Json + $LASTEXITCODE | Should -Be 0 -Because (Get-Content -Raw $TestDrive/error.txt) + $out.stdin | Should -BeExactly 'hello from stdin' + } + + It 'Exits with the exit code of the plug-in: ' -TestCases @( + @{ exitCode = 0 } + @{ exitCode = 1 } + @{ exitCode = 5 } + @{ exitCode = 42 } + ) { + param($exitCode) + + $null = dsc echo --exit-code $exitCode 2> $TestDrive/error.txt + $LASTEXITCODE | Should -Be $exitCode -Because (Get-Content -Raw $TestDrive/error.txt) + } + + It 'Exits with an exit code above 255 on Windows' -Skip:(!$IsWindows) { + $null = dsc echo --exit-code 1000 2> $TestDrive/error.txt + $LASTEXITCODE | Should -Be 1000 -Because (Get-Content -Raw $TestDrive/error.txt) + } + + It 'Tells the plug-in how it was invoked through environment variables' { + $out = dsc echo 2> $TestDrive/error.txt | ConvertFrom-Json + $LASTEXITCODE | Should -Be 0 -Because (Get-Content -Raw $TestDrive/error.txt) + $dscVersion = (dsc --version) -replace '^dsc\s+', '' + $out.env.DSC_EXE | Should -Not -BeNullOrEmpty + (Split-Path -Leaf $out.env.DSC_EXE) | Should -Match '^dsc(\.exe)?$' + Test-Path -Path $out.env.DSC_EXE -PathType Leaf | Should -BeTrue + $out.env.DSC_VERSION | Should -BeExactly $dscVersion + $out.env.DSC_PLUGIN_COMMAND | Should -BeExactly 'echo' + $out.env.DSC_PLUGIN_DIR | Should -BeExactly $pluginDir + # The values depend on the settings in effect, so only check that DSC defines them. + $out.env.DSC_TRACE_LEVEL | Should -Not -BeNullOrEmpty + $out.env.DSC_TRACE_FORMAT | Should -Not -BeNullOrEmpty + $out.env.DSC_PROGRESS_FORMAT | Should -BeNullOrEmpty + } + + It 'Hands the options before the command to the plug-in as environment variables' { + $out = dsc --trace-level info --trace-format json --progress-format none echo --trace-level error 2> $TestDrive/error.txt | + ConvertFrom-Json + $LASTEXITCODE | Should -Be 0 -Because (Get-Content -Raw $TestDrive/error.txt) + $out.args | Should -Be @('--trace-level', 'error') + $out.env.DSC_TRACE_LEVEL | Should -Be 'info' + $out.env.DSC_TRACE_FORMAT | Should -Be 'json' + $out.env.DSC_PROGRESS_FORMAT | Should -Be 'none' + } + + It 'Tells the plug-in to ignore the settings files when the user asked for that' { + $out = dsc --ignore-settings-file echo 2> $TestDrive/error.txt | ConvertFrom-Json + $LASTEXITCODE | Should -Be 0 -Because (Get-Content -Raw $TestDrive/error.txt) + $out.env.DSC_IGNORE_SETTINGS_FILE | Should -Be '1' + } + + It 'Fails when the executable of the plug-in cannot be found' { + $directory = Join-Path $TestDrive 'missing-executable' + $null = New-Item -ItemType Directory -Force -Path $directory + $manifest = New-TestPluginManifest -Type 'Test/MissingExecutable' -Command 'missing-executable' + $manifest.command.executable = 'dsc-test-executable-that-does-not-exist' + $manifest | ConvertTo-Json -Depth 10 | Set-Content -Path (Join-Path $directory 'missing.dsc.plugin.json') + + try { + Use-PluginDirectory -Directory $directory + $out = dsc missing-executable 2> $TestDrive/error.txt + $LASTEXITCODE | Should -Be 2 + $out | Should -BeNullOrEmpty + $stderr = Get-Content -Raw $TestDrive/error.txt + $stderr | Should -Match "missing-executable" + $stderr | Should -Match "dsc-test-executable-that-does-not-exist" + } finally { + Use-PluginDirectory -Directory $pluginDir + } + } + + It 'Warns when the plug-in is deprecated' { + $directory = Join-Path $TestDrive 'deprecated' + New-TestPlugin -Directory $directory -Type 'Test/Deprecated' -Command 'deprecated' -Extra @{ + deprecationMessage = 'Use the echo command instead.' + } + + try { + Use-PluginDirectory -Directory $directory + $out = dsc deprecated 2> $TestDrive/error.txt | ConvertFrom-Json + $LASTEXITCODE | Should -Be 0 -Because (Get-Content -Raw $TestDrive/error.txt) + $out.env.DSC_PLUGIN_COMMAND | Should -BeExactly 'deprecated' + $stderr = Get-Content -Raw $TestDrive/error.txt + $stderr | Should -Match 'WARN' + $stderr | Should -Match "Test/Deprecated.*Use the echo command instead\." + } finally { + Use-PluginDirectory -Directory $pluginDir + } + } + } + + Context 'Help' { + It 'Passes --help after the command name to the plug-in' { + $out = dsc echo --help 2> $TestDrive/error.txt + $LASTEXITCODE | Should -Be 0 -Because (Get-Content -Raw $TestDrive/error.txt) + $out[0] | Should -BeExactly 'Usage: dsc echo [OPTIONS]' + $out[1] | Should -BeExactly 'HELP_ARGS: --help' + } + + It 'Shows the help of the plug-in for: dsc help ' -TestCases @( + @{ arguments = @('echo'); expected = '--help' } + @{ arguments = @('echo', 'sub'); expected = 'sub|--help' } + @{ arguments = @('echo', 'sub', 'command'); expected = 'sub|command|--help' } + ) { + param($arguments, $expected) + + $out = dsc help @arguments 2> $TestDrive/error.txt + $LASTEXITCODE | Should -Be 0 -Because (Get-Content -Raw $TestDrive/error.txt) + $out[0] | Should -BeExactly 'Usage: dsc echo [OPTIONS]' + $out[1] | Should -BeExactly "HELP_ARGS: $expected" + } + + It 'Lists plug-in commands in the top-level help for: dsc ' -TestCases @( + @{ arguments = @('--help') } + @{ arguments = @('-h') } + @{ arguments = @('help') } + @{ arguments = @('--trace-level', 'info', '--help') } + ) { + param($arguments) + + $out = dsc @arguments 2> $TestDrive/error.txt + $LASTEXITCODE | Should -Be 0 -Because (Get-Content -Raw $TestDrive/error.txt) + Get-Content -Raw $TestDrive/error.txt | Should -BeNullOrEmpty + $text = $out | Out-String + $text | Should -Match '(?m)^Commands:\r?$' + $text | Should -Match '(?m)^Plug-in commands:\r?$' + $text | Should -Match '(?m)^ echo\s+Report how the plug-in was invoked\r?$' + $text | Should -Match '(?m)^Options:\r?$' + # The plug-in commands are listed between the built-in commands and the options. + $text.IndexOf('Commands:') | Should -BeLessThan $text.IndexOf('Plug-in commands:') + $text.IndexOf('Plug-in commands:') | Should -BeLessThan $text.IndexOf('Options:') + } + + It 'Keeps the help of built-in commands unchanged for: dsc ' -TestCases @( + @{ arguments = @('config', '--help') } + @{ arguments = @('help', 'config') } + @{ arguments = @('plugin', 'list', '--help') } + ) { + param($arguments) + + $out = dsc @arguments 2> $TestDrive/error.txt | Out-String + $LASTEXITCODE | Should -Be 0 -Because (Get-Content -Raw $TestDrive/error.txt) + $out | Should -Not -Match 'Plug-in commands:' + $out | Should -Match 'Usage: dsc(\.exe)? (config|plugin list)' + } + + It 'Lists the plugin command with the built-in commands' { + $out = dsc --help 2> $TestDrive/error.txt | Out-String + $LASTEXITCODE | Should -Be 0 -Because (Get-Content -Raw $TestDrive/error.txt) + $out | Should -Match '(?m)^ plugin\s+Operations on DSC plug-ins\r?$' + } + } + + Context 'Commands that no plug-in provides' { + It 'Reports an unrecognized subcommand and suggests a similar plug-in command' { + $out = dsc ech 2> $TestDrive/error.txt + $LASTEXITCODE | Should -Be 2 + $out | Should -BeNullOrEmpty + $stderr = Get-Content -Raw $TestDrive/error.txt + $stderr | Should -Match "error: unrecognized subcommand 'ech'" + $stderr | Should -Match "tip: a similar subcommand exists: 'echo'" + } + + It 'Reports an unrecognized subcommand and suggests a similar built-in command' { + $null = dsc confg get 2> $TestDrive/error.txt + $LASTEXITCODE | Should -Be 2 + $stderr = Get-Content -Raw $TestDrive/error.txt + $stderr | Should -Match "error: unrecognized subcommand 'confg'" + $stderr | Should -Match "tip: a similar subcommand exists: 'config'" + } + + It 'Reports an unrecognized subcommand for help on an unknown command' { + $null = dsc help does-not-exist 2> $TestDrive/error.txt + $LASTEXITCODE | Should -Be 2 + Get-Content -Raw $TestDrive/error.txt | Should -Match "error: unrecognized subcommand 'does-not-exist'" + } + + It 'Compares command names case-sensitively' { + $null = dsc Echo 2> $TestDrive/error.txt + $LASTEXITCODE | Should -Be 2 + Get-Content -Raw $TestDrive/error.txt | Should -Match "error: unrecognized subcommand 'Echo'" + } + } + + Context 'Completion scripts' { + BeforeAll { + # A command name that can't appear in a completion script for any other reason. + $script:completionDir = Join-Path $TestDrive 'completion' + New-TestPlugin -Directory $completionDir -Type 'Test/Completion' -Command 'completion-probe' -Description 'Probe for completion scripts' + Use-PluginDirectory -Directory $completionDir + } + + AfterAll { + Use-PluginDirectory -Directory $pluginDir + } + + It 'Includes plug-in commands in the completion script for ' -TestCases @( + @{ shell = 'bash' } + @{ shell = 'elvish' } + @{ shell = 'fish' } + @{ shell = 'powershell' } + @{ shell = 'zsh' } + ) { + param($shell) + + $out = dsc completer $shell 2> $TestDrive/error.txt | Out-String + $LASTEXITCODE | Should -Be 0 -Because (Get-Content -Raw $TestDrive/error.txt) + $out | Should -Match 'completion-probe' + } + + It 'Describes plug-in commands in the completion script' { + $out = dsc completer powershell 2> $TestDrive/error.txt | Out-String + $LASTEXITCODE | Should -Be 0 -Because (Get-Content -Raw $TestDrive/error.txt) + $out | Should -Match "'completion-probe', 'completion-probe', \[CompletionResultType\]::ParameterValue, 'Probe for completion scripts'" + } + } + + Context 'Manifests that are skipped' { + BeforeAll { + $script:skippedDir = Join-Path $TestDrive 'skipped' + New-TestPlugin -Directory $skippedDir -Type 'Test.Skipped/Reserved' -Command 'config' + New-TestPlugin -Directory $skippedDir -Type 'Test.Skipped/ReservedAlias' -Command 'mcp' + New-TestPlugin -Directory $skippedDir -Type 'Test.Skipped/ReservedHelp' -Command 'help' + New-TestPlugin -Directory $skippedDir -Type 'Test.Skipped/ConditionFalse' -Command 'condition-false' -Extra @{ + condition = '[equals(1, 2)]' + } + New-TestPlugin -Directory $skippedDir -Type 'Test.Skipped/ConditionTrue' -Command 'condition-true' -Extra @{ + condition = '[equals(1, 1)]' + } + '{ this is not a manifest' | Set-Content -Path (Join-Path $skippedDir 'broken.dsc.plugin.json') + Use-PluginDirectory -Directory $skippedDir + } + + AfterAll { + Use-PluginDirectory -Directory $pluginDir + } + + It 'Lists only the plug-ins that can provide a command' { + $out = dsc plugin list 'Test.Skipped/*' 2> $TestDrive/error.txt | ConvertFrom-Json + $LASTEXITCODE | Should -Be 0 -Because (Get-Content -Raw $TestDrive/error.txt) + @($out).Count | Should -Be 1 + $out.type | Should -BeExactly 'Test.Skipped/ConditionTrue' + } + + It 'Warns about a plug-in that uses the name of a built-in command: ' -TestCases @( + @{ command = 'config'; type = 'Test.Skipped/Reserved' } + @{ command = 'mcp'; type = 'Test.Skipped/ReservedAlias' } + @{ command = 'help'; type = 'Test.Skipped/ReservedHelp' } + ) { + param($command, $type) + + $null = dsc plugin list 2> $TestDrive/error.txt + $LASTEXITCODE | Should -Be 0 + $stderr = Get-Content -Raw $TestDrive/error.txt + $stderr | Should -Match "WARN.*'$type'.*'$command'.*built-in command" + } + + It 'Warns about a manifest that cannot be loaded' { + $null = dsc plugin list 2> $TestDrive/error.txt + $LASTEXITCODE | Should -Be 0 + Get-Content -Raw $TestDrive/error.txt | Should -Match 'WARN.*broken\.dsc\.plugin\.json' + } + + It 'Never runs a plug-in in place of a built-in command' { + $out = dsc config --help 2> $TestDrive/error.txt | Out-String + $LASTEXITCODE | Should -Be 0 + $out | Should -Match 'Apply a configuration document' + $out | Should -Not -Match 'HELP_ARGS' + } + + It 'Does not provide the command of a plug-in whose condition is false' { + $null = dsc condition-false 2> $TestDrive/error.txt + $LASTEXITCODE | Should -Be 2 + Get-Content -Raw $TestDrive/error.txt | Should -Match "unrecognized subcommand 'condition-false'" + } + + It 'Provides the command of a plug-in whose condition is true' { + $out = dsc condition-true 2> $TestDrive/error.txt | ConvertFrom-Json + $LASTEXITCODE | Should -Be 0 + $out.env.DSC_PLUGIN_COMMAND | Should -BeExactly 'condition-true' + } + + It 'Keeps warnings about manifests out of the top-level help' { + $out = dsc --help 2> $TestDrive/error.txt | Out-String + $LASTEXITCODE | Should -Be 0 + Get-Content -Raw $TestDrive/error.txt | Should -BeNullOrEmpty + $out | Should -Match '(?m)^ condition-true\s' + $out | Should -Not -Match '(?m)^ condition-false\s' + } + } + + Context 'Precedence between plug-ins' { + BeforeAll { + $script:firstDir = Join-Path $TestDrive 'first' + $script:secondDir = Join-Path $TestDrive 'second' + # The same type in two directories: the highest version wins, wherever it is. + New-TestPlugin -Directory $firstDir -Type 'Test.Precedence/Versioned' -Command 'versioned' -Version '1.0.0' + New-TestPlugin -Directory $secondDir -Type 'Test.Precedence/Versioned' -Command 'versioned' -Version '2.0.0' + # Different types with the same command: the first directory wins. + New-TestPlugin -Directory $firstDir -Type 'Test.Precedence/Zeta' -Command 'shared' -Version '1.0.0' + New-TestPlugin -Directory $secondDir -Type 'Test.Precedence/Alpha' -Command 'shared' -Version '9.0.0' + Use-PluginDirectory -Directory $firstDir, $secondDir + } + + AfterAll { + Use-PluginDirectory -Directory $pluginDir + } + + It 'Uses the highest version of a plug-in type' { + $out = dsc plugin list 'Test.Precedence/Versioned' 2> $TestDrive/error.txt | ConvertFrom-Json + $LASTEXITCODE | Should -Be 0 -Because (Get-Content -Raw $TestDrive/error.txt) + @($out).Count | Should -Be 1 + $out.version | Should -BeExactly '2.0.0' + $out.directory | Should -BeExactly $secondDir + + $result = dsc versioned 2> $TestDrive/error.txt | ConvertFrom-Json + $LASTEXITCODE | Should -Be 0 -Because (Get-Content -Raw $TestDrive/error.txt) + $result.env.DSC_PLUGIN_DIR | Should -BeExactly $secondDir + } + + It 'Reports the plug-in found later as shadowed' { + $out = dsc plugin list 'Test.Precedence/*' 2> $TestDrive/error.txt | ConvertFrom-Json + $LASTEXITCODE | Should -Be 0 -Because (Get-Content -Raw $TestDrive/error.txt) + ($out | Where-Object type -EQ 'Test.Precedence/Zeta').shadowed | Should -BeFalse + ($out | Where-Object type -EQ 'Test.Precedence/Alpha').shadowed | Should -BeTrue + } + + It 'Marks the shadowed plug-in in the table' { + $out = dsc plugin list 'Test.Precedence/*' -o table-no-truncate 2> $TestDrive/error.txt | Out-String + $LASTEXITCODE | Should -Be 0 -Because (Get-Content -Raw $TestDrive/error.txt) + $out | Should -Match 'shared \(shadowed\)\s+Test\.Precedence/Alpha' + $out | Should -Match 'shared\s+Test\.Precedence/Zeta' + } + + It 'Runs the plug-in found first and warns about the shadowed plug-in' { + $out = dsc shared 2> $TestDrive/error.txt | ConvertFrom-Json + $LASTEXITCODE | Should -Be 0 -Because (Get-Content -Raw $TestDrive/error.txt) + $out.env.DSC_PLUGIN_DIR | Should -BeExactly $firstDir + $out.env.DSC_PLUGIN_COMMAND | Should -BeExactly 'shared' + $stderr = Get-Content -Raw $TestDrive/error.txt + $stderr | Should -Match "WARN.*'Test\.Precedence/Zeta'.*'Test\.Precedence/Alpha'" + $stderr | Should -Match "'dsc Test\.Precedence/Alpha'" + } + + It 'Runs a shadowed plug-in by its type name without a warning' { + $out = dsc Test.Precedence/Alpha one --two 2> $TestDrive/error.txt | ConvertFrom-Json + $LASTEXITCODE | Should -Be 0 -Because (Get-Content -Raw $TestDrive/error.txt) + $out.args | Should -Be @('one', '--two') + $out.env.DSC_PLUGIN_DIR | Should -BeExactly $secondDir + $out.env.DSC_PLUGIN_COMMAND | Should -BeExactly 'Test.Precedence/Alpha' + Get-Content -Raw $TestDrive/error.txt | Should -Not -Match 'shadowed' + } + + It 'Runs the plug-in that provides the command by its type name without a warning' { + $out = dsc Test.Precedence/Zeta 2> $TestDrive/error.txt | ConvertFrom-Json + $LASTEXITCODE | Should -Be 0 -Because (Get-Content -Raw $TestDrive/error.txt) + $out.env.DSC_PLUGIN_DIR | Should -BeExactly $firstDir + Get-Content -Raw $TestDrive/error.txt | Should -Not -Match 'shadowed' + } + + It 'Compares the type name case-insensitively' { + $out = dsc test.precedence/ALPHA 2> $TestDrive/error.txt | ConvertFrom-Json + $LASTEXITCODE | Should -Be 0 -Because (Get-Content -Raw $TestDrive/error.txt) + $out.env.DSC_PLUGIN_DIR | Should -BeExactly $secondDir + } + + It 'Shows the help of a shadowed plug-in by its type name' { + $out = dsc help Test.Precedence/Alpha 2> $TestDrive/error.txt + $LASTEXITCODE | Should -Be 0 -Because (Get-Content -Raw $TestDrive/error.txt) + $out[0] | Should -BeExactly 'Usage: dsc Test.Precedence/Alpha [OPTIONS]' + $out[1] | Should -BeExactly 'HELP_ARGS: --help' + } + + It 'Reports an unrecognized subcommand for a type name that no plug-in has' { + $null = dsc Test.Precedence/DoesNotExist 2> $TestDrive/error.txt + $LASTEXITCODE | Should -Be 2 + Get-Content -Raw $TestDrive/error.txt | Should -Match "unrecognized subcommand 'Test\.Precedence/DoesNotExist'" + } + + It 'Lists a shadowed command once in the top-level help' { + $out = dsc --help 2> $TestDrive/error.txt + $LASTEXITCODE | Should -Be 0 -Because (Get-Content -Raw $TestDrive/error.txt) + @($out | Where-Object { $_ -match '^ shared\s' }).Count | Should -Be 1 + } + } + + Context 'Disabling plug-ins' { + BeforeAll { + # A settings file in the working directory is the workspace settings file. + $script:workspace = New-Item -ItemType Directory -Force -Path (Join-Path $TestDrive 'workspace') + @{ plugins = @{ enabled = $false } } | ConvertTo-Json | Set-Content -Path (Join-Path $workspace 'dsc.settings.json') + Push-Location $workspace + } + + AfterAll { + Pop-Location + } + + It 'Lists no plug-ins' { + $out = dsc plugin list 2> $TestDrive/error.txt + $LASTEXITCODE | Should -Be 0 -Because (Get-Content -Raw $TestDrive/error.txt) + $out | Should -BeNullOrEmpty + } + + It 'Reports a plug-in command as an unrecognized subcommand' { + $out = dsc echo 2> $TestDrive/error.txt + $LASTEXITCODE | Should -Be 2 + $out | Should -BeNullOrEmpty + $stderr = Get-Content -Raw $TestDrive/error.txt + $stderr | Should -Match "error: unrecognized subcommand 'echo'" + $stderr | Should -Not -Match 'HELP_ARGS' + } + + It 'Does not list plug-in commands in the top-level help' { + $out = dsc --help 2> $TestDrive/error.txt | Out-String + $LASTEXITCODE | Should -Be 0 -Because (Get-Content -Raw $TestDrive/error.txt) + $out | Should -Not -Match 'Plug-in commands:' + $out | Should -Match '(?m)^Commands:\r?$' + } + + It 'Does not include plug-in commands in the completion script' { + $out = dsc completer powershell 2> $TestDrive/error.txt | Out-String + $LASTEXITCODE | Should -Be 0 -Because (Get-Content -Raw $TestDrive/error.txt) + $out | Should -Not -Match "'echo'" + } + + It 'Runs plug-ins again when the settings files are ignored' { + $out = dsc --ignore-settings-file echo 2> $TestDrive/error.txt | ConvertFrom-Json + $LASTEXITCODE | Should -Be 0 -Because (Get-Content -Raw $TestDrive/error.txt) + $out.env.DSC_PLUGIN_COMMAND | Should -BeExactly 'echo' + } + } +} diff --git a/dsc/tests/other/dsc_schema.tests.ps1 b/dsc/tests/other/dsc_schema.tests.ps1 index d01bf13fa..9e66be0f7 100644 --- a/dsc/tests/other/dsc_schema.tests.ps1 +++ b/dsc/tests/other/dsc_schema.tests.ps1 @@ -48,6 +48,17 @@ Describe 'config schema tests' { $schema.'$schema' | Should -BeExactly 'https://json-schema.org/draft/2020-12/schema' } + It 'exposes the lint output schemas used by dsclint and the MCP server' { + $schema = dsc schema --type lint-result 2>$TestDrive/error.log | ConvertFrom-Json + $LASTEXITCODE | Should -Be 0 -Because (Get-Content $TestDrive/error.log -Raw) + $schema.properties.diagnostics | Should -Not -BeNullOrEmpty + $schema.properties.summary | Should -Not -BeNullOrEmpty + + $schema = dsc schema --type lint-rules 2>$TestDrive/error.log | ConvertFrom-Json + $LASTEXITCODE | Should -Be 0 -Because (Get-Content $TestDrive/error.log -Raw) + $schema.properties.defaultLevel | Should -Not -BeNullOrEmpty + } + It 'schema uses camelCase for ' -TestCases $schemaTestCases { param($type) diff --git a/dsc/tests/other/dsc_server.tests.ps1 b/dsc/tests/other/dsc_server.tests.ps1 index 857044957..0f3566f13 100644 --- a/dsc/tests/other/dsc_server.tests.ps1 +++ b/dsc/tests/other/dsc_server.tests.ps1 @@ -79,6 +79,7 @@ Describe 'Tests for DSC server' { 'invoke_dsc_function' = $false 'invoke_dsc_resource' = $false 'list_dsc_actions' = $false + 'lint_dsc_config' = $false 'list_dsc_functions' = $false 'list_dsc_resources' = $false 'show_dsc_action' = $false @@ -1185,4 +1186,63 @@ resources: $response.result.structuredContent.description | Should -BeExactly 'An action for testing.' $response.result.structuredContent.requireSecurityContext | Should -BeExactly 'current' } + + It 'Calling lint_dsc_config returns diagnostics' { + $configuration = @' +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +parameters: + unused: + type: string + defaultValue: x +resources: +- name: a + type: Microsoft.DSC.Debug/Echo + properties: + output: "[parameters('missing')]" +'@ + $mcpRequest = @{ + jsonrpc = "2.0" + id = 39 + method = "tools/call" + params = @{ + name = "lint_dsc_config" + arguments = @{ + configuration = $configuration + discovery = $false + rules = @{ 'no-unused-parameters' = 'info' } + } + } + } + + $response = Send-McpRequest -request $mcpRequest + $response.id | Should -Be 39 + $result = $response.result.structuredContent.result + $result.valid | Should -BeFalse -Because ($response | ConvertTo-Json -Depth 20 | Out-String) + $result.discovery | Should -BeFalse + $undefined = $result.diagnostics | Where-Object rule -EQ 'undefined-parameter' + $undefined.path | Should -BeExactly '/resources/0/properties/output' + $unused = $result.diagnostics | Where-Object rule -EQ 'no-unused-parameters' + $unused.level | Should -BeExactly 'info' + } + + It 'Calling lint_dsc_config with an unknown rule returns error' { + $mcpRequest = @{ + jsonrpc = "2.0" + id = 40 + method = "tools/call" + params = @{ + name = "lint_dsc_config" + arguments = @{ + configuration = '{"$schema": "https://aka.ms/dsc/schemas/v3/bundled/config/document.json", "resources": []}' + discovery = $false + rules = @{ 'no-such-rule' = 'off' } + } + } + } + + $response = Send-McpRequest -request $mcpRequest + $response.id | Should -Be 40 + $response.error.code | Should -Be -32602 + $response.error.message | Should -Match 'no-such-rule' + } } diff --git a/dsclint/.project.data.json b/dsclint/.project.data.json new file mode 100644 index 000000000..e2768e815 --- /dev/null +++ b/dsclint/.project.data.json @@ -0,0 +1,13 @@ +{ + "Name": "dsclint", + "Kind": "CLI", + "IsRust": true, + "Binaries": [ + "dsclint" + ], + "CopyFiles": { + "All": [ + "dsclint.dsc.plugin.json" + ] + } +} diff --git a/dsclint/Cargo.toml b/dsclint/Cargo.toml new file mode 100644 index 000000000..156f56f8a --- /dev/null +++ b/dsclint/Cargo.toml @@ -0,0 +1,24 @@ +[package] +name = "dsclint" +# Keep in lockstep with the `dsc` crate: the linter checks `directives.version` against this value. +version = "3.4.0-preview.3" +edition = "2024" + +[[bin]] +name = "dsclint" +path = "src/main.rs" + +[dependencies] +# external dependencies +clap = { workspace = true } +crossterm = { workspace = true } +miette = { workspace = true, features = ["fancy-no-backtrace"] } +rust-i18n = { workspace = true } +serde = { workspace = true } +serde_json = { workspace = true } +serde_yaml = { workspace = true } +syntect = { workspace = true } +tracing = { workspace = true } +# workspace crate dependencies +dsc-lib = { workspace = true } +dsc-lib-telemetry = { workspace = true } diff --git a/dsclint/build.rs b/dsclint/build.rs new file mode 100644 index 000000000..315b846a1 --- /dev/null +++ b/dsclint/build.rs @@ -0,0 +1,7 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +fn main() { + // Always rebuild if translations are updated. + println!("cargo::rerun-if-changed=locales"); +} diff --git a/dsclint/dsclint.dsc.plugin.json b/dsclint/dsclint.dsc.plugin.json new file mode 100644 index 000000000..a77d01297 --- /dev/null +++ b/dsclint/dsclint.dsc.plugin.json @@ -0,0 +1,10 @@ +{ + "$schema": "https://aka.ms/dsc/schemas/v3/bundled/plugin/manifest.json", + "type": "Microsoft.DSC/Lint", + "version": "3.4.0-preview.3", + "description": "Check configuration documents for problems without applying them", + "command": { + "name": "lint", + "executable": "dsclint" + } +} diff --git a/dsclint/locales/en-us.toml b/dsclint/locales/en-us.toml new file mode 100644 index 000000000..3dfbf0a13 --- /dev/null +++ b/dsclint/locales/en-us.toml @@ -0,0 +1,57 @@ +_version = 1 + +[args] +about = "Check a DSC configuration document for problems without applying it" +input = "The configuration document as JSON or YAML to lint" +file = "The path to a configuration document to lint; specify multiple times to lint several documents. Use '-' to read from STDIN." +parameters = "Parameters to pass to the configuration as JSON or YAML" +parametersFile = "Parameters to pass to the configuration as a JSON or YAML file" +systemRoot = "Specify the operating system root path if not targeting the current running OS" +noDiscovery = "Skip resource discovery and the rules that inspect installed resources" +failOn = "The lowest diagnostic level that makes the command fail (default: error)" +rule = "Override the level of a rule, like 'no-unused-parameters=off'; specify multiple times for several rules" +listRules = "List the lint rules and the level each reports at instead of linting" +outputFormat = "The output format to use" +traceFormat = "Trace format to use" +traceLevel = "Trace level to use" +progressFormat = "Progress format to use during resource discovery" +ignoreSettingsFile = "Ignore the settings file when running the command" + +[main] +failedToSpawnMain = "Failed to spawn dsclint main thread: %{error}" +failedToJoinMain = "Failed to join dsclint main thread: %{error}" +usingVersion = "Running dsclint version" + +[lint] +mergingParameters = "Merging inline parameters with parameters file (inline takes precedence)" +failedMergingParameters = "Failed to merge parameters" +invalidParameters = "Invalid parameters" +failedConvertJson = "Failed to convert YAML to JSON" +noInput = "No configuration document specified; use '--file' or '--input'" +lintFailed = "Linting failed" +invalidLintRule = "Invalid lint rule option" +tableHeader_rule = "Rule" +tableHeader_category = "Category" +tableHeader_level = "Level" +tableHeader_tier = "Tier" +tableHeader_description = "Description" + +[report] +lintSummary = "errors: %{errors}, warnings: %{warnings}, info: %{info}" + +[util] +failedToSetTracing = "Unable to set global default tracing subscriber. Tracing is disabled." +readingInput = "Reading input from command line parameter" +inputIsFile = "Document provided is a file path, use '--file' instead" +readingInputFromFile = "Reading input from file" +readingInputFromStdin = "Reading input from STDIN" +invalidUtf8 = "Invalid utf-8 input" +failedToReadStdin = "Failed to read input from STDIN" +failedToReadFile = "Failed to read input file" +noInput = "No input provided" +emptyInput = "Empty input provided" +removingUtf8Bom = "Removing UTF-8 BOM from input" +failedToParseFileParameters = "Failed to parse parameters file: %{error}" +failedToParseInlineParameters = "Failed to parse inline parameters: %{error}" +parametersNotObject = "Parameters must be an object" +invalidVersion = "Unable to parse the dsclint version from the cargo manifest: %{error}" diff --git a/dsclint/src/args.rs b/dsclint/src/args.rs new file mode 100644 index 000000000..ebf5bf39e --- /dev/null +++ b/dsclint/src/args.rs @@ -0,0 +1,59 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +use clap::{Parser, ValueEnum}; +use dsc_lib::dscresources::command_resource::TraceLevel; +use dsc_lib::lint::{LintLevel, RuleLevelOverride}; +use dsc_lib::progress::ProgressFormat; +use rust_i18n::t; +use serde::Deserialize; + +#[derive(Debug, Clone, PartialEq, Eq, ValueEnum)] +pub enum OutputFormat { + Json, + PrettyJson, + Yaml, +} + +#[derive(Debug, Clone, PartialEq, Eq, ValueEnum, Deserialize)] +pub enum TraceFormat { + Default, + Plaintext, + Json, + #[clap(hide = true)] + PassThrough, +} + +#[derive(Debug, Parser)] +#[clap(name = "dsclint", version = env!("CARGO_PKG_VERSION"), about = t!("args.about").to_string(), long_about = None)] +pub struct Args { + #[clap(short = 'i', long, help = t!("args.input").to_string(), conflicts_with = "file")] + pub input: Option, + #[clap(short = 'f', long, help = t!("args.file").to_string(), conflicts_with = "input")] + pub file: Vec, + #[clap(short = 'p', long, help = t!("args.parameters").to_string())] + pub parameters: Option, + #[clap(long, help = t!("args.parametersFile").to_string())] + pub parameters_file: Option, + #[clap(short = 'r', long, help = t!("args.systemRoot").to_string())] + pub system_root: Option, + #[clap(long, help = t!("args.noDiscovery").to_string())] + pub no_discovery: bool, + #[clap(long, help = t!("args.failOn").to_string(), value_enum)] + pub fail_on: Option, + #[clap(long = "rule", value_name = "RULE=LEVEL", help = t!("args.rule").to_string())] + pub rules: Vec, + #[clap(long, help = t!("args.listRules").to_string(), conflicts_with_all = ["input", "file"])] + pub list_rules: bool, + #[clap(short = 'o', long, help = t!("args.outputFormat").to_string(), value_enum)] + pub output_format: Option, + #[clap(short = 'l', long, help = t!("args.traceLevel").to_string(), value_enum)] + pub trace_level: Option, + #[clap(short = 't', long, help = t!("args.traceFormat").to_string(), value_enum)] + pub trace_format: Option, + // `-p` is taken by `--parameters`, as in `dsc config`, so this option has no short form. + #[clap(long, help = t!("args.progressFormat").to_string(), value_enum)] + pub progress_format: Option, + #[clap(long, help = t!("args.ignoreSettingsFile").to_string())] + pub ignore_settings_file: bool, +} diff --git a/dsclint/src/lint.rs b/dsclint/src/lint.rs new file mode 100644 index 000000000..5fced59e3 --- /dev/null +++ b/dsclint/src/lint.rs @@ -0,0 +1,197 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +//! The lint command: read the documents, run the linter, and write the result. + +use crate::args::{Args, OutputFormat}; +use crate::report::render_lint_result; +use crate::tablewriter::Table; +use crate::util::{ + EXIT_DSC_ERROR, EXIT_INVALID_ARGS, EXIT_INVALID_INPUT, EXIT_JSON_ERROR, EXIT_VALIDATION_FAILED, + current_version, merge_parameters, parse_parameters, read_document, read_input, write_object, +}; +use dsc_lib::dscerror::DscError; +use dsc_lib::lint::{LintError, LintLevel, LintOptions, LintResult, Linter}; +use dsc_lib::progress::ProgressFormat; +use rust_i18n::t; +use serde_json::Value; +use std::io::{self, IsTerminal}; +use std::path::PathBuf; +use std::process::ExitCode; +use tracing::{error, info}; + +/// Lint the configuration documents named by the arguments, or list the rules. +/// +/// # Arguments +/// +/// * `args` - The command-line arguments. +/// +/// # Errors +/// +/// Returns `EXIT_INVALID_ARGS` for an unknown rule or missing input, `EXIT_INVALID_INPUT` when a +/// document or the parameters can't be read, `EXIT_VALIDATION_FAILED` when a diagnostic at or +/// above the `--fail-on` level was reported, `EXIT_JSON_ERROR` when the result can't be +/// serialized, and `EXIT_DSC_ERROR` for other failures. A document that is empty or invalid is +/// read successfully and reported in the result, not as an error. +pub fn run(args: &Args) -> Result<(), ExitCode> { + let parameters = resolve_parameters(args.parameters.as_ref(), args.parameters_file.as_ref())?; + let options = LintOptions { + discovery: !args.no_discovery, + rule_levels: args.rules.clone(), + parameters, + system_root: args.system_root.as_ref().map(PathBuf::from), + dsc_version: Some(current_version()), + progress_format: args.progress_format.unwrap_or(ProgressFormat::Default), + ..LintOptions::default() + }; + let mut linter = match Linter::new(options) { + Ok(linter) => linter, + Err(DscError::Lint(err @ (LintError::UnknownRule(_) | LintError::InvalidRuleOverride(_) | LintError::InvalidRuleLevel(_)))) => { + error!("{}: {err}", t!("lint.invalidLintRule")); + return Err(ExitCode::from(EXIT_INVALID_ARGS)); + }, + Err(err) => { + error!("{}: {err}", t!("lint.lintFailed")); + return Err(ExitCode::from(EXIT_DSC_ERROR)); + }, + }; + + if args.list_rules { + return list_rules(&linter, args.output_format.as_ref()); + } + + let mut documents: Vec<(Option, String)> = Vec::new(); + if let Some(input) = &args.input { + documents.push((None, read_document(Some(input), None)?)); + } + for file in &args.file { + let document = read_document(None, Some(file))?; + let name = if file == "-" { None } else { Some(file.clone()) }; + documents.push((name, document)); + } + if documents.is_empty() { + error!("{}", t!("lint.noInput")); + return Err(ExitCode::from(EXIT_INVALID_ARGS)); + } + + let mut result = LintResult::from_diagnostics(Vec::new(), true); + for (file, document) in &documents { + linter.set_file(file.clone()); + let document_result = match linter.lint(document) { + Ok(document_result) => document_result, + Err(DscError::Lint(err @ LintError::InvalidParameters(_))) => { + error!("{}: {err}", t!("lint.lintFailed")); + return Err(ExitCode::from(EXIT_INVALID_INPUT)); + }, + Err(err) => { + error!("{}: {err}", t!("lint.lintFailed")); + return Err(ExitCode::from(EXIT_DSC_ERROR)); + }, + }; + result.merge(document_result); + } + + if args.output_format.is_none() && io::stdout().is_terminal() { + let descriptions = linter.rules().into_iter().map(|rule| (rule.id, rule.description)).collect(); + print!("{}", render_lint_result(&result, &documents, &descriptions, true)); + } else { + let json = match serde_json::to_string(&result) { + Ok(json) => json, + Err(err) => { + error!("JSON: {err}"); + return Err(ExitCode::from(EXIT_JSON_ERROR)); + } + }; + write_object(&json, args.output_format.as_ref())?; + } + + let threshold = args.fail_on.unwrap_or(LintLevel::Error); + if result.has_diagnostic_at_least(threshold) { + return Err(ExitCode::from(EXIT_VALIDATION_FAILED)); + } + Ok(()) +} + +/// Combine the inline parameters with the parameters file, if either was given. +/// +/// # Arguments +/// +/// * `parameters` - The inline parameters, as JSON or YAML. +/// * `parameters_file` - The path to a parameters file, or `-` to read it from STDIN. +/// +/// # Returns +/// +/// The parameters, or `None` when neither was given. Inline values take precedence over values +/// from the file. +/// +/// # Errors +/// +/// Returns `EXIT_INVALID_INPUT` when the file can't be read or either input isn't valid JSON or +/// YAML. +fn resolve_parameters(parameters: Option<&String>, parameters_file: Option<&String>) -> Result, ExitCode> { + let file_parameters = match parameters_file { + Some(path) => Some(read_input(None, Some(path))?), + None => None, + }; + + let parsed = match (file_parameters, parameters) { + (Some(file_content), Some(inline_content)) => { + info!("{}", t!("lint.mergingParameters")); + merge_parameters(&file_content, inline_content) + }, + (Some(file_content), None) => parse_parameters(&file_content, "FileParameters"), + (None, Some(inline_content)) => parse_parameters(inline_content, "InlineParameters"), + (None, None) => return Ok(None), + }; + + match parsed { + Ok(value) => Ok(Some(value)), + Err(err) => { + error!("{}: {err}", t!("lint.invalidParameters")); + Err(ExitCode::from(EXIT_INVALID_INPUT)) + } + } +} + +/// Print the lint rules with the level each reports at in this run. +/// +/// # Arguments +/// +/// * `linter` - The linter whose rule levels to show. +/// * `format` - The output format; a table when not given and stdout is a terminal. +/// +/// # Errors +/// +/// Returns `EXIT_JSON_ERROR` when the rules can't be serialized. +fn list_rules(linter: &Linter, format: Option<&OutputFormat>) -> Result<(), ExitCode> { + let rules = linter.rules(); + if format.is_none() && io::stdout().is_terminal() { + let mut table = Table::new(&[ + t!("lint.tableHeader_rule").to_string().as_ref(), + t!("lint.tableHeader_category").to_string().as_ref(), + t!("lint.tableHeader_level").to_string().as_ref(), + t!("lint.tableHeader_tier").to_string().as_ref(), + t!("lint.tableHeader_description").to_string().as_ref(), + ]); + for rule in rules { + table.add_row(vec![ + rule.id, + rule.category.to_string(), + rule.level.to_string(), + rule.tier.to_string(), + rule.description, + ]); + } + table.print(true); + return Ok(()); + } + + let json = match serde_json::to_string(&rules) { + Ok(json) => json, + Err(err) => { + error!("JSON: {err}"); + return Err(ExitCode::from(EXIT_JSON_ERROR)); + } + }; + write_object(&json, format) +} diff --git a/dsclint/src/main.rs b/dsclint/src/main.rs new file mode 100644 index 000000000..5ccf5e95e --- /dev/null +++ b/dsclint/src/main.rs @@ -0,0 +1,69 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +//! `dsclint` checks DSC configuration documents for problems without applying them. + +use args::Args; +use clap::Parser; +use dsc_lib::util::DSC_IGNORE_SETTINGS_FILE; +use rust_i18n::{i18n, t}; +use std::{env::set_var, process::ExitCode}; +use tracing::debug; + +mod args; +mod lint; +mod report; +mod tablewriter; +mod util; + +i18n!("locales", fallback = "en-us"); + +fn main() -> ExitCode { + #[cfg(windows)] + { + let handle = match std::thread::Builder::new() + .name("dsclint-main".to_string()) + .stack_size(2 * 1024 * 1024) // Default stack is too small on Windows causing overflow + .spawn(lint_main) + { + Ok(handle) => handle, + Err(err) => { + eprintln!("{}", t!("main.failedToSpawnMain", error = err)); + return ExitCode::from(util::EXIT_DSC_ERROR); + } + }; + + match handle.join() { + Ok(Ok(())) => ExitCode::from(util::EXIT_SUCCESS), + Ok(Err(code)) => code, + Err(err) => { + eprintln!("{}", t!("main.failedToJoinMain", error = err : {:?})); + ExitCode::from(util::EXIT_DSC_ERROR) + } + } + } + #[cfg(not(windows))] + { + match lint_main() { + Ok(()) => ExitCode::from(util::EXIT_SUCCESS), + Err(code) => code, + } + } +} + +fn lint_main() -> Result<(), ExitCode> { + let args = Args::parse(); + + // the settings file is read by `dsc-lib`, which honors this environment variable + if args.ignore_settings_file { + unsafe { + set_var(DSC_IGNORE_SETTINGS_FILE, "1"); + } + } + + util::enable_tracing(args.trace_level.as_ref(), args.trace_format.as_ref(), args.ignore_settings_file); + + debug!("{}: {}", t!("main.usingVersion"), env!("CARGO_PKG_VERSION")); + + lint::run(&args) +} diff --git a/dsclint/src/report.rs b/dsclint/src/report.rs new file mode 100644 index 000000000..6be3f3665 --- /dev/null +++ b/dsclint/src/report.rs @@ -0,0 +1,124 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +//! Render lint results for the terminal. +//! +//! Each [`dsc_lib::lint::LintDiagnostic`] is rendered as a [`miette`] report that shows the problem +//! in place in the document it was found in, followed by the counts by level. Secure values in +//! the document are replaced before it is shown, so a snippet never prints a secret. + +use std::collections::HashMap; + +use dsc_lib::diagnostics::SourceMap; +use dsc_lib::lint::{LintDiagnosticReport, LintResult, redacted_source_map}; +use miette::{GraphicalReportHandler, GraphicalTheme}; +use rust_i18n::t; + +/// The lines shown around each labeled line of a snippet. +const CONTEXT_LINES: usize = 2; + +/// Render the diagnostics of a lint run as reports that show each problem in its document, +/// followed by the counts by level. +/// +/// # Arguments +/// +/// * `result` - The lint result. +/// * `documents` - The text of each linted document, keyed by the file name recorded on its +/// diagnostics; `None` for inline input. Secure values in the text are shown as +/// ``; a document whose secure values can't be located is shown without a +/// snippet. +/// * `descriptions` - What each rule requires, by rule identifier, shown as the help text. +/// * `colored` - Whether to style the reports with ANSI colors. +#[must_use] +pub fn render_lint_result(result: &LintResult, documents: &[(Option, String)], descriptions: &HashMap, colored: bool) -> String { + let maps: HashMap<&Option, SourceMap> = documents.iter() + .filter_map(|(file, text)| redacted_source_map(text).map(|map| (file, map))) + .collect(); + let handler = handler(colored); + let mut output = String::new(); + for diagnostic in &result.diagnostics { + let report = LintDiagnosticReport::new(diagnostic, maps.get(&diagnostic.file), descriptions.get(&diagnostic.rule).cloned()); + if handler.render_report(&mut output, &report).is_err() { + output.push_str(&format!("{} {}: {}\n", diagnostic.level, diagnostic.rule, diagnostic.message)); + } + output.push('\n'); + } + output.push_str(&t!( + "report.lintSummary", + errors = result.summary.errors, + warnings = result.summary.warnings, + info = result.summary.info + )); + output.push('\n'); + output +} + +/// A report handler for the terminal, drawing with unicode and, unless `NO_COLOR` asks +/// otherwise, ANSI colors. +/// +/// # Arguments +/// +/// * `colored` - Whether colors are wanted. +fn handler(colored: bool) -> GraphicalReportHandler { + let no_color = std::env::var("NO_COLOR").is_ok_and(|value| value != "0"); + let theme = if colored && !no_color { + GraphicalTheme::unicode() + } else { + GraphicalTheme::unicode_nocolor() + }; + GraphicalReportHandler::new_themed(theme).with_context_lines(CONTEXT_LINES) +} + +#[cfg(test)] +mod tests { + use super::*; + use dsc_lib::lint::{LintCategory, LintDiagnostic, LintLevel}; + + #[test] + fn lint_results_render_each_diagnostic_in_its_document() { + let text = "resources:\n- name: a\n type: T/E\n- name: a\n type: T/E\n".to_string(); + let diagnostic = LintDiagnostic { + level: LintLevel::Warning, + rule: "duplicate-resource".to_string(), + category: LintCategory::Document, + message: "declared twice".to_string(), + path: "/resources/1".to_string(), + file: Some("demo.dsc.yaml".to_string()), + location: None, + }; + let result = LintResult::from_diagnostics(vec![diagnostic], false); + let descriptions = HashMap::from([("duplicate-resource".to_string(), "Names must be unique.".to_string())]); + let output = render_lint_result(&result, &[(Some("demo.dsc.yaml".to_string()), text)], &descriptions, false); + assert!(output.contains("duplicate-resource"), "{output}"); + assert!(output.contains("declared twice"), "{output}"); + assert!(output.contains("[demo.dsc.yaml:4:3]"), "{output}"); + assert!(output.contains("Names must be unique."), "{output}"); + assert!(output.contains("errors: 0, warnings: 1, info: 0"), "{output}"); + } + + #[test] + fn rendered_snippets_never_show_secure_values() { + let text = "parameters:\n password:\n type: secureString\n defaultValue: hunter2\n token:\n type: secureString\n defaultValue: \"[concat('letmein', 'unterminated)]\"\nresources:\n- name: a\n type: T/E\n properties:\n secret:\n secureString: topsecret\n output: \"[parameters('missing')]\"\n".to_string(); + let diagnostic = |rule: &str, level: LintLevel, path: &str| LintDiagnostic { + level, + rule: rule.to_string(), + category: LintCategory::Parameter, + message: format!("problem reported by {rule}"), + path: path.to_string(), + file: None, + location: None, + }; + let result = LintResult::from_diagnostics(vec![ + diagnostic("secure-parameter-default", LintLevel::Warning, "/parameters/password/defaultValue"), + diagnostic("invalid-expression", LintLevel::Error, "/parameters/token/defaultValue"), + diagnostic("undefined-parameter", LintLevel::Error, "/resources/0/properties/output"), + ], false); + let output = render_lint_result(&result, &[(None, text)], &HashMap::new(), false); + for secret in ["hunter2", "letmein", "topsecret"] { + assert!(!output.contains(secret), "{output}"); + } + assert!(output.contains(""), "{output}"); + assert!(output.contains("[input:4:5]"), "{output}"); + assert!(output.contains("parameters('missing')"), "{output}"); + } +} diff --git a/dsclint/src/tablewriter.rs b/dsclint/src/tablewriter.rs new file mode 100644 index 000000000..a9c6ce662 --- /dev/null +++ b/dsclint/src/tablewriter.rs @@ -0,0 +1,98 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +use crossterm::terminal::size; + +pub struct Table { + header: Vec, + rows: Vec>, + column_widths: Vec, +} + +impl Table { + /// Create a new table. + /// + /// # Arguments + /// + /// * `header` - The header row + /// + /// # Returns + /// + /// * `Table` - The new table + #[must_use] + pub fn new(header: &[&str]) -> Table { + let mut column_widths = Vec::new(); + for header_text in header { + column_widths.push(header_text.len()); + } + let header = header.iter().map(|s| (*s).to_string()).collect::>(); + Table { + header, + rows: Vec::new(), + column_widths, + } + } + + /// Add a row to the table. + /// + /// # Arguments + /// + /// * `row` - The row to add + pub fn add_row(&mut self, row: Vec) { + for (i, column) in row.iter().enumerate() { + if column.len() > self.column_widths[i] { + self.column_widths[i] = column.len(); + } + } + self.rows.push(row); + } + + /// Print the table to the console. + /// + /// # Arguments + /// + /// * `truncate` - Whether to truncate the table if it is too wide for the console + #[allow(clippy::format_push_string)] + pub fn print(&self, truncate: bool) { + let (width, _) = size().unwrap_or((80, 25)); + // make header bright green + println!("\x1b[1;32m"); + let mut header_row = String::with_capacity(width as usize); + let last_column = self.header.len() - 1; + for (i, column) in self.header.iter().enumerate() { + header_row.push_str(&format!("{: width as usize { + header_row.truncate(width as usize); + } + + println!("{header_row}"); + println!("{}\x1b[0m", "-".repeat(header_row.len())); + + // TODO: make this smarter by splitting long text to multiple lines + for row in &self.rows { + let mut row_str = String::with_capacity(header_row.len()); + for (i, column) in row.iter().enumerate() { + row_str.push_str(&format!("{: width as usize { + row_str.truncate(width as usize); + if !row_str.ends_with(' ') { + row_str.pop(); + row_str.push('…'); + } + } + + println!("{row_str}"); + } + } +} diff --git a/dsclint/src/util.rs b/dsclint/src/util.rs new file mode 100644 index 000000000..72ddd3231 --- /dev/null +++ b/dsclint/src/util.rs @@ -0,0 +1,431 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +//! Helpers shared by the command: exit codes, tracing setup, input reading, and output writing. +//! +//! These mirror the equivalents in the `dsc` crate so that `dsclint` behaves the same way as +//! `dsc` for settings, tracing, input, and output. + +use crate::args::{OutputFormat, TraceFormat}; +use dsc_lib::{ + discovery::Discovery, + dscerror::DscError, + dscresources::command_resource::TraceLevel, + extensions::dscextension::Capability, + types::SemanticVersion, + settings::{fields::TracingFormatField, global as global_settings, sources::{CommandLineData, EnvironmentData}}, +}; +use dsc_lib_telemetry::{StderrFormat, basic::BasicTracingOptions}; +use rust_i18n::t; +use serde_json::{Map, Value}; +use std::env; +use std::io::{IsTerminal, Read, Write, stdout}; +use std::path::Path; +use std::process::ExitCode; +use syntect::{ + easy::HighlightLines, + highlighting::ThemeSet, + parsing::SyntaxSet, + util::{LinesWithEndings, as_24_bit_terminal_escaped}, +}; +use tracing::{Level, debug, error, info, trace}; + +// The exit codes match the ones `dsc` uses for the same conditions. +pub const EXIT_SUCCESS: u8 = 0; +pub const EXIT_INVALID_ARGS: u8 = 1; +pub const EXIT_DSC_ERROR: u8 = 2; +pub const EXIT_JSON_ERROR: u8 = 3; +pub const EXIT_INVALID_INPUT: u8 = 4; +pub const EXIT_VALIDATION_FAILED: u8 = 5; + +/// Returns the version of `dsclint` as a semantic version. +/// +/// The crate version is kept in lockstep with `dsc`, so this is the DSC version that +/// `directives.version` requirements are checked against. +/// +/// # Panics +/// +/// Panics if the version defined in the cargo manifest isn't a valid semantic version. Cargo +/// rejects such a manifest at build time, so this can't happen for a built binary. +#[must_use] +pub fn current_version() -> SemanticVersion { + match SemanticVersion::parse(env!("CARGO_PKG_VERSION")) { + Ok(version) => version, + Err(err) => unreachable!("{}", t!("util.invalidVersion", error = err)), + } +} + +/// Write the JSON object to the console +/// +/// # Arguments +/// +/// * `json` - The JSON to write +/// * `format` - The format to use +/// +/// # Errors +/// +/// Returns `EXIT_JSON_ERROR` when the JSON can't be parsed or converted to the requested format. +pub fn write_object(json: &str, format: Option<&OutputFormat>) -> Result<(), ExitCode> { + let mut is_json = true; + let mut output_format = format; + let mut syntax_color = false; + if stdout().is_terminal() { + syntax_color = true; + if output_format.is_none() { + output_format = Some(&OutputFormat::Yaml); + } + } else if output_format.is_none() { + output_format = Some(&OutputFormat::Json); + } + + let output = match output_format { + Some(OutputFormat::Json) => json.to_string(), + Some(OutputFormat::PrettyJson) => { + let value: Value = match serde_json::from_str(json) { + Ok(value) => value, + Err(err) => { + error!("JSON: {err}"); + return Err(ExitCode::from(EXIT_JSON_ERROR)); + } + }; + match serde_json::to_string_pretty(&value) { + Ok(json) => json, + Err(err) => { + error!("JSON: {err}"); + return Err(ExitCode::from(EXIT_JSON_ERROR)); + } + } + }, + Some(OutputFormat::Yaml) | None => { + is_json = false; + let value: Value = match serde_json::from_str(json) { + Ok(value) => value, + Err(err) => { + error!("JSON: {err}"); + return Err(ExitCode::from(EXIT_JSON_ERROR)); + } + }; + match serde_yaml::to_string(&value) { + Ok(yaml) => yaml, + Err(err) => { + error!("YAML: {err}"); + return Err(ExitCode::from(EXIT_JSON_ERROR)); + } + } + } + }; + + if syntax_color { + let ps = SyntaxSet::load_defaults_newlines(); + let ts = ThemeSet::load_defaults(); + let Some(syntax) = (if is_json { + ps.find_syntax_by_extension("json") + } else { + ps.find_syntax_by_extension("yaml") + }) else { + println!("{json}"); + return Ok(()); + }; + + let mut h = HighlightLines::new(syntax, &ts.themes["base16-ocean.dark"]); + + for line in LinesWithEndings::from(&output) { + let Ok(ranges) = h.highlight_line(line, &ps) else { + continue; + }; + let escaped = as_24_bit_terminal_escaped(&ranges[..], false); + print!("{escaped}"); + } + } else { + let mut stdout_lock = stdout().lock(); + if writeln!(stdout_lock, "{output}").is_err() { + // likely caused by a broken pipe (e.g. 'head' command closed early) + return Ok(()); + } + } + + Ok(()) +} + +/// Enable tracing the same way `dsc` does, from the resolved DSC settings. +/// +/// The settings resolve the trace level and format from the policy file, the command-line +/// arguments, the `DSC_TRACE_LEVEL` and `DSC_TRACE_FORMAT` environment variables, and the settings +/// files, in that order of precedence. The environment variables are how `dsc` hands its own +/// options to `dsclint` when it runs as the `dsc lint` plug-in command. +/// +/// # Arguments +/// +/// * `trace_level_arg` - The trace level from the command line, if given. +/// * `trace_format_arg` - The trace format from the command line, if given. +/// * `ignore_settings_file` - Whether the command line asked to ignore the settings files. +pub fn enable_tracing(trace_level_arg: Option<&TraceLevel>, trace_format_arg: Option<&TraceFormat>, ignore_settings_file: bool) { + let default_guard = BasicTracingOptions::init_default_guard(); + + let command_line = CommandLineData { + trace_level: trace_level_arg.cloned().map(Into::into), + trace_format: trace_format_arg.map(|format| match format { + TraceFormat::Default => TracingFormatField::Default, + TraceFormat::Plaintext => TracingFormatField::Plaintext, + TraceFormat::Json | TraceFormat::PassThrough => TracingFormatField::Json, + }), + ignore_settings_file: ignore_settings_file.then(|| true.into()), + }; + let mut settings = global_settings::get_or_init_with_command_line(command_line); + settings.load(); + settings.resolve_all(); + let tracing_setting = settings.resolved().tracing.clone(); + drop(settings); + + let tracing_level: Level = tracing_setting.level.value.clone().into(); + let stderr_format = match tracing_setting.format.value { + TracingFormatField::Default => StderrFormat::Default, + TracingFormatField::Plaintext => StderrFormat::Plaintext, + TracingFormatField::Json => StderrFormat::Json, + }; + + // enable tracing + let subscriber = BasicTracingOptions { + tracing_level, + stderr_format + }.init_subscriber(); + + drop(default_guard); + if tracing::subscriber::set_global_default(subscriber).is_err() { + eprintln!("{}", t!("util.failedToSetTracing")); + } + + // set DSC_TRACE_LEVEL for child processes, like the resources run during discovery + unsafe { + env::set_var(EnvironmentData::DSC_TRACE_LEVEL_ENV_VAR, tracing_setting.level.value.to_string()); + } + info!("Trace-level is {:?}", tracing_setting.level.value); +} + +/// Read an input document as text without parsing it, rejecting an empty document. +/// +/// # Arguments +/// +/// * `input` - The inline document, if provided. +/// * `file` - The path to the document, or `-` to read it from STDIN. Extensions with the +/// `import` capability get the first chance to handle a file path. +/// +/// # Returns +/// +/// The document text, or an empty string when neither argument is provided. +/// +/// # Errors +/// +/// Returns `EXIT_INVALID_INPUT` when the inline input is actually a path to an existing file, the +/// file can't be read, STDIN isn't valid UTF-8, or the document is empty. +pub fn read_input(input: Option<&String>, file: Option<&String>) -> Result { + let value = read_document(input, file)?; + if (input.is_some() || file.is_some()) && value.trim().is_empty() { + error!("{}", t!("util.emptyInput")); + return Err(ExitCode::from(EXIT_INVALID_INPUT)); + } + Ok(value) +} + +/// Read an input document as text without parsing it, including an empty document. +/// +/// The linter reports an empty document as a diagnostic in its result, so the document is +/// returned as read. Use [`read_input`] for input that must not be empty, like parameters. +/// +/// # Arguments +/// +/// * `input` - The inline document, if provided. +/// * `file` - The path to the document, or `-` to read it from STDIN. Extensions with the +/// `import` capability get the first chance to handle a file path. +/// +/// # Returns +/// +/// The document text, or an empty string when neither argument is provided. +/// +/// # Errors +/// +/// Returns `EXIT_INVALID_INPUT` when the inline input is actually a path to an existing file, the +/// file can't be read, or STDIN isn't valid UTF-8. +pub fn read_document(input: Option<&String>, file: Option<&String>) -> Result { + trace!("Input: {input:?}, File: {file:?}"); + let value = if let Some(input) = input { + debug!("{}", t!("util.readingInput")); + + // see if user accidentally passed in a file path + if Path::new(input).exists() { + error!("{}", t!("util.inputIsFile")); + return Err(ExitCode::from(EXIT_INVALID_INPUT)); + } + input.clone() + } else if let Some(path) = file { + debug!("{} {path}", t!("util.readingInputFromFile")); + // check if need to read from STDIN + if path == "-" { + info!("{}", t!("util.readingInputFromStdin")); + let mut stdin = Vec::::new(); + match std::io::stdin().read_to_end(&mut stdin) { + Ok(_) => { + match String::from_utf8(stdin) { + Ok(input) => { + input + }, + Err(err) => { + error!("{}: {err}", t!("util.invalidUtf8")); + return Err(ExitCode::from(EXIT_INVALID_INPUT)); + } + } + }, + Err(err) => { + error!("{}: {err}", t!("util.failedToReadStdin")); + return Err(ExitCode::from(EXIT_INVALID_INPUT)); + } + } + } else { + // see if an extension should handle this file + let mut discovery = Discovery::new(); + let path_buf = Path::new(path); + for extension in discovery.get_extensions(&Capability::Import) { + if let Ok(content) = extension.import(path_buf) { + return Ok(content); + } + } + match std::fs::read_to_string(path) { + Ok(input) => { + // check if it contains UTF-8 BOM and remove it + if input.as_bytes().starts_with(&[0xEF, 0xBB, 0xBF]) { + info!("{}", t!("util.removingUtf8Bom")); + input[3..].to_string() + } else { + input + } + }, + Err(err) => { + error!("{}: {err}", t!("util.failedToReadFile")); + return Err(ExitCode::from(EXIT_INVALID_INPUT)); + } + } + } + } else { + debug!("{}", t!("util.noInput")); + String::new() + }; + + Ok(value) +} + +/// Parse parameter input as JSON, falling back to YAML. +/// +/// # Arguments +/// +/// * `input` - The parameters as JSON or YAML text. +/// * `context` - Which input is being parsed, for the error message: `FileParameters` or +/// `InlineParameters`. +/// +/// # Errors +/// +/// Returns an error if the input cannot be parsed as valid JSON or YAML. +pub fn parse_parameters(input: &str, context: &str) -> Result { + match serde_json::from_str(input) { + Ok(json) => Ok(json), + Err(_) => { + match serde_yaml::from_str::(input) { + Ok(yaml) => Ok(serde_json::to_value(yaml)?), + Err(err) => { + Err(DscError::Parser(t!(&format!("util.failedToParse{context}"), error = err.to_string()).to_string())) + } + } + } + } +} + +/// Convert parameter input to a map. +/// +/// # Arguments +/// +/// * `params` - Parameter string to convert (JSON or YAML format) +/// * `context` - Context string for error messages +/// +/// # Errors +/// +/// Returns an error if the input cannot be parsed or is not an object. +fn params_to_map(params: &str, context: &str) -> Result, DscError> { + let value = parse_parameters(params, context)?; + + let Some(map) = value.as_object().cloned() else { + return Err(DscError::Parser(t!("util.parametersNotObject").to_string())); + }; + + Ok(map) +} + +/// Merge two parameter sets, with inline parameters taking precedence over file parameters. +/// Top-level keys (like "parameters") are merged recursively, but parameter values themselves +/// are replaced (not merged) when specified inline. +/// +/// # Arguments +/// +/// * `file_params` - Parameters from file (JSON or YAML format) +/// * `inline_params` - Inline parameters (JSON or YAML format) that take precedence +/// +/// # Returns +/// +/// The merged parameters. +/// +/// # Errors +/// +/// Returns an error if either parameter set cannot be parsed as valid JSON or YAML or isn't an +/// object. +pub fn merge_parameters(file_params: &str, inline_params: &str) -> Result { + let mut file_map = params_to_map(file_params, "FileParameters")?; + let inline_map = params_to_map(inline_params, "InlineParameters")?; + + for (key, inline_value) in &inline_map { + if key == "parameters" { + // Special handling for "parameters" key - merge at parameter name level only. + // Within each parameter name, inline replaces (not merges). + if let Some(file_params_value) = file_map.get_mut("parameters") { + if let (Some(file_params_obj), Some(inline_params_obj)) = (file_params_value.as_object_mut(), inline_value.as_object()) { + for (param_name, param_value) in inline_params_obj { + file_params_obj.insert(param_name.clone(), param_value.clone()); + } + } else { + // If one is not an object, inline replaces completely + file_map.insert(key.clone(), inline_value.clone()); + } + } else { + file_map.insert(key.clone(), inline_value.clone()); + } + } else { + // For other top-level keys, inline value replaces file value + file_map.insert(key.clone(), inline_value.clone()); + } + } + + Ok(Value::Object(file_map)) +} + +#[cfg(test)] +mod tests { + use super::*; + use serde_json::json; + + #[test] + fn inline_parameters_replace_file_parameters_by_name() { + let file = r#"{"parameters": {"a": 1, "b": {"x": 1}}, "other": true}"#; + let inline = "parameters:\n b:\n y: 2\n c: 3\n"; + let merged = merge_parameters(file, inline).unwrap(); + assert_eq!(merged, json!({"parameters": {"a": 1, "b": {"y": 2}, "c": 3}, "other": true})); + } + + #[test] + fn parameters_must_be_an_object() { + let err = merge_parameters("[1, 2]", "{}").unwrap_err(); + assert!(err.to_string().contains("must be an object"), "{err}"); + } + + #[test] + fn invalid_parameters_name_the_input() { + let err = parse_parameters("{not: valid: yaml", "InlineParameters").unwrap_err(); + assert!(err.to_string().contains("inline parameters"), "{err}"); + } +} diff --git a/dsclint/tests/dsc_lint_plugin.tests.ps1 b/dsclint/tests/dsc_lint_plugin.tests.ps1 new file mode 100644 index 000000000..0186703fe --- /dev/null +++ b/dsclint/tests/dsc_lint_plugin.tests.ps1 @@ -0,0 +1,91 @@ +# Copyright (c) Microsoft Corporation. +# Licensed under the MIT License. + +Describe 'dsc lint plug-in command tests' { + BeforeAll { + $script:document = @' +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +parameters: + unused: + type: string + defaultValue: x +resources: +- name: a + type: Microsoft.DSC.Debug/Echo + properties: + output: "[parameters('missing')]" +'@ + $script:documentPath = Join-Path $TestDrive 'demo.dsc.yaml' + Set-Content -Path $documentPath -Value $document + } + + It 'Lists dsclint as the plug-in that provides the lint command' { + $out = dsc plugin list 'Microsoft.DSC/Lint' 2> $TestDrive/error.txt | ConvertFrom-Json + $LASTEXITCODE | Should -Be 0 -Because (Get-Content -Raw $TestDrive/error.txt) + $out.command | Should -BeExactly 'lint' + $out.shadowed | Should -BeFalse + $out.manifest.command.executable | Should -BeExactly 'dsclint' + $out.version | Should -BeExactly ((dsc --version) -replace '^dsc\s+', '') + } + + It 'Lists the lint command in the top-level help' { + $out = dsc --help 2> $TestDrive/error.txt | Out-String + $LASTEXITCODE | Should -Be 0 -Because (Get-Content -Raw $TestDrive/error.txt) + $out | Should -Match '(?m)^Plug-in commands:\r?$' + $out | Should -Match '(?m)^ lint\s+Check configuration documents for problems without applying them\r?$' + } + + It 'Returns the same result and exit code as dsclint' { + $expected = dsclint --file $documentPath --no-discovery --output-format json 2> $TestDrive/expected-error.txt + $expectedExitCode = $LASTEXITCODE + $actual = dsc lint --file $documentPath --no-discovery --output-format json 2> $TestDrive/error.txt + $LASTEXITCODE | Should -Be $expectedExitCode -Because (Get-Content -Raw $TestDrive/error.txt) + $expectedExitCode | Should -Be 5 + $actual | Should -BeExactly $expected + + $result = $actual | ConvertFrom-Json + $result.valid | Should -BeFalse + $result.diagnostics.rule | Should -Contain 'undefined-parameter' + $result.diagnostics.rule | Should -Contain 'no-unused-parameters' + } + + It 'Passes standard input to the linter' { + $out = $document | dsc lint --file - --no-discovery --output-format json 2> $TestDrive/error.txt | ConvertFrom-Json + $LASTEXITCODE | Should -Be 5 -Because (Get-Content -Raw $TestDrive/error.txt) + $out.diagnostics.rule | Should -Contain 'undefined-parameter' + } + + It 'Runs the linter by its type name' { + $out = dsc Microsoft.DSC/Lint --list-rules --output-format json 2> $TestDrive/error.txt | ConvertFrom-Json + $LASTEXITCODE | Should -Be 0 -Because (Get-Content -Raw $TestDrive/error.txt) + $out.id | Should -Contain 'no-unused-parameters' + } + + It 'Shows the help of the linter for: dsc ' -TestCases @( + @{ arguments = @('help', 'lint') } + @{ arguments = @('lint', '--help') } + ) { + param($arguments) + + $out = dsc @arguments 2> $TestDrive/error.txt | Out-String + $LASTEXITCODE | Should -Be 0 -Because (Get-Content -Raw $TestDrive/error.txt) + $out | Should -Match '--fail-on' + $out | Should -Match '--no-discovery' + } + + It 'Applies the trace level given before the command to the linter' { + $null = dsc --trace-level info lint --file $documentPath --no-discovery --output-format json 2> $TestDrive/error.txt + $LASTEXITCODE | Should -Be 5 + Get-Content -Raw $TestDrive/error.txt | Should -Match 'Trace-level is Info' + + $null = dsc lint --file $documentPath --no-discovery --output-format json 2> $TestDrive/error.txt + $LASTEXITCODE | Should -Be 5 + Get-Content -Raw $TestDrive/error.txt | Should -Not -Match 'Trace-level is' + } + + It 'Lets an option after the command override the trace level given before it' { + $null = dsc --trace-level error lint --trace-level info --file $documentPath --no-discovery --output-format json 2> $TestDrive/error.txt + $LASTEXITCODE | Should -Be 5 + Get-Content -Raw $TestDrive/error.txt | Should -Match 'Trace-level is Info' + } +} diff --git a/dsclint/tests/dsclint.tests.ps1 b/dsclint/tests/dsclint.tests.ps1 new file mode 100644 index 000000000..69bdc9072 --- /dev/null +++ b/dsclint/tests/dsclint.tests.ps1 @@ -0,0 +1,316 @@ +# Copyright (c) Microsoft Corporation. +# Licensed under the MIT License. + +Describe 'dsclint tests' { + BeforeAll { + $schemaUri = 'https://aka.ms/dsc/schemas/v3/bundled/config/document.json' + $cleanConfig = @" +`$schema: $schemaUri +parameters: + greeting: + type: string + defaultValue: hello +resources: +- name: Echo + type: Microsoft.DSC.Debug/Echo + properties: + output: "[parameters('greeting')]" +"@ + # The document from https://github.com/PowerShell/DSC/issues/842 + $duplicateConfig = @" +`$schema: $schemaUri +resources: +- name: EnsureTest1Value + type: Microsoft.DSC.Debug/Echo + properties: + output: one +- name: EnsureTest2Value + type: Microsoft.DSC.Debug/Echo + properties: + output: two +- name: EnsureTest1Value + type: Microsoft.DSC.Debug/Echo + properties: + output: three +"@ + $unusedParameterConfig = @" +`$schema: $schemaUri +parameters: + unused: + type: string + defaultValue: x +resources: +- name: Echo + type: Microsoft.DSC.Debug/Echo + properties: + output: hello +"@ + } + + It 'reports no diagnostics for a clean document' { + $result = $cleanConfig | dsclint --file - 2>$TestDrive/error.log | ConvertFrom-Json + $LASTEXITCODE | Should -Be 0 -Because (Get-Content $TestDrive/error.log -Raw) + $result.valid | Should -BeTrue + $result.discovery | Should -BeTrue + $result.diagnostics.Count | Should -Be 0 + $result.summary.errors | Should -Be 0 + $result.summary.warnings | Should -Be 0 + } + + It 'reports a duplicate resource and exits with code 5' { + $result = $duplicateConfig | dsclint --file - --no-discovery 2>$TestDrive/error.log | ConvertFrom-Json + $LASTEXITCODE | Should -Be 5 -Because (Get-Content $TestDrive/error.log -Raw) + $result.valid | Should -BeFalse + $result.discovery | Should -BeFalse + $result.summary.errors | Should -Be 1 + $result.diagnostics.Count | Should -Be 1 + $result.diagnostics[0].rule | Should -BeExactly 'duplicate-resource' + $result.diagnostics[0].level | Should -BeExactly 'error' + $result.diagnostics[0].category | Should -BeExactly 'document' + $result.diagnostics[0].path | Should -BeExactly '/resources/2' + $result.diagnostics[0].message | Should -BeLike "*EnsureTest1Value*" + # the third resource starts on line 11 of the document, after the leading `- ` + $result.diagnostics[0].location.line | Should -Be 11 + $result.diagnostics[0].location.column | Should -Be 3 + } + + It 'records the line and column of a diagnostic inside a value' { + $result = $unusedParameterConfig | dsclint --file - --no-discovery 2>$TestDrive/error.log | ConvertFrom-Json + $LASTEXITCODE | Should -Be 0 -Because (Get-Content $TestDrive/error.log -Raw) + $result.diagnostics[0].path | Should -BeExactly '/parameters/unused' + $result.diagnostics[0].location.line | Should -Be 3 + $result.diagnostics[0].location.column | Should -Be 3 + } + + It 'accepts the document with --input' { + $result = dsclint --input $duplicateConfig --no-discovery 2>$TestDrive/error.log | ConvertFrom-Json + $LASTEXITCODE | Should -Be 5 -Because (Get-Content $TestDrive/error.log -Raw) + $result.diagnostics[0].rule | Should -BeExactly 'duplicate-resource' + } + + It 'treats warnings as non-fatal unless --fail-on warning is set' { + $result = $unusedParameterConfig | dsclint --file - --no-discovery 2>$TestDrive/error.log | ConvertFrom-Json + $LASTEXITCODE | Should -Be 0 -Because (Get-Content $TestDrive/error.log -Raw) + $result.valid | Should -BeTrue + $result.summary.warnings | Should -Be 1 + $result.diagnostics[0].rule | Should -BeExactly 'no-unused-parameters' + $result.diagnostics[0].level | Should -BeExactly 'warning' + $result.diagnostics[0].path | Should -BeExactly '/parameters/unused' + + $null = $unusedParameterConfig | dsclint --file - --no-discovery --fail-on warning 2>$TestDrive/error.log + $LASTEXITCODE | Should -Be 5 -Because (Get-Content $TestDrive/error.log -Raw) + } + + It 'can change the level of a rule with --rule' { + $result = $unusedParameterConfig | dsclint --file - --no-discovery --rule no-unused-parameters=off 2>$TestDrive/error.log | ConvertFrom-Json + $LASTEXITCODE | Should -Be 0 -Because (Get-Content $TestDrive/error.log -Raw) + $result.diagnostics.Count | Should -Be 0 + + $result = $unusedParameterConfig | dsclint --file - --no-discovery --rule no-unused-parameters=error 2>$TestDrive/error.log | ConvertFrom-Json + $LASTEXITCODE | Should -Be 5 -Because (Get-Content $TestDrive/error.log -Raw) + $result.valid | Should -BeFalse + $result.diagnostics[0].level | Should -BeExactly 'error' + } + + It 'rejects an unknown rule with exit code 1' { + $null = $cleanConfig | dsclint --file - --no-discovery --rule no-such-rule=off 2>$TestDrive/error.log + $LASTEXITCODE | Should -Be 1 + (Get-Content $TestDrive/error.log -Raw) | Should -BeLike '*no-such-rule*' + } + + It 'reports a document that is not valid YAML as a diagnostic' { + $result = "resources:`n- name: a`n type: [unclosed" | dsclint --file - --no-discovery 2>$TestDrive/error.log | ConvertFrom-Json + $LASTEXITCODE | Should -Be 5 -Because (Get-Content $TestDrive/error.log -Raw) + $result.valid | Should -BeFalse + $result.diagnostics.Count | Should -Be 1 + $result.diagnostics[0].rule | Should -BeExactly 'parse-error' + $result.diagnostics[0].path | Should -BeExactly '' + $result.diagnostics[0].location.line | Should -BeGreaterThan 0 + } + + It 'reports an empty document as a diagnostic instead of failing to read it' { + $result = "`n" | dsclint --file - --no-discovery 2>$TestDrive/error.log | ConvertFrom-Json + $LASTEXITCODE | Should -Be 5 -Because (Get-Content $TestDrive/error.log -Raw) + $result.valid | Should -BeFalse + $result.diagnostics.Count | Should -Be 1 + $result.diagnostics[0].rule | Should -BeExactly 'parse-error' + + [IO.File]::WriteAllText("$TestDrive/empty.dsc.yaml", '') + $result = dsclint --file "$TestDrive/empty.dsc.yaml" --no-discovery 2>$TestDrive/error.log | ConvertFrom-Json + $LASTEXITCODE | Should -Be 5 -Because (Get-Content $TestDrive/error.log -Raw) + $result.diagnostics.Count | Should -Be 1 + $result.diagnostics[0].rule | Should -BeExactly 'parse-error' + $result.diagnostics[0].file | Should -BeLike '*empty.dsc.yaml' + } + + It 'reports schema violations with the path of the problem' { + $config = @" +`$schema: $schemaUri +resources: +- name: a + type: Microsoft.DSC.Debug/Echo + unknownKey: true +"@ + $result = $config | dsclint --file - --no-discovery 2>$TestDrive/error.log | ConvertFrom-Json + $LASTEXITCODE | Should -Be 5 -Because (Get-Content $TestDrive/error.log -Raw) + $violations = @($result.diagnostics | Where-Object rule -eq 'schema-invalid') + $violations.Count | Should -BeGreaterThan 0 + $violations.path | Should -Contain '/resources/0' + } + + It 'reports at ' -TestCases @( + @{ rule = 'undefined-parameter'; path = '/resources/0/properties/output'; property = "[parameters('missing')]" } + @{ rule = 'undefined-variable'; path = '/resources/0/properties/output'; property = "[variables('missing')]" } + @{ rule = 'unknown-function'; path = '/resources/0/properties/output'; property = "[noSuchFunction()]" } + @{ rule = 'invalid-argument-count'; path = '/resources/0/properties/output'; property = "[int('1', '2')]" } + @{ rule = 'invalid-argument-type'; path = '/resources/0/properties/output'; property = "[int(true)]" } + @{ rule = 'invalid-expression'; path = '/resources/0/properties/output'; property = "[concat('unterminated)]" } + @{ rule = 'unresolved-dependency'; path = '/resources/0/properties/output'; property = "[reference(resourceId('Microsoft.DSC.Debug/Echo', 'nobody'))]" } + ) { + param($rule, $path, $property) + $config = @" +`$schema: $schemaUri +resources: +- name: a + type: Microsoft.DSC.Debug/Echo + properties: + output: "$property" +"@ + $result = $config | dsclint --file - --no-discovery 2>$TestDrive/error.log | ConvertFrom-Json + $LASTEXITCODE | Should -Be 5 -Because (Get-Content $TestDrive/error.log -Raw) + $diagnostic = $result.diagnostics | Where-Object rule -eq $rule + $diagnostic | Should -Not -BeNullOrEmpty -Because ($result.diagnostics | ConvertTo-Json -Depth 5) + $diagnostic.path | Should -BeExactly $path + $diagnostic.level | Should -BeExactly 'error' + } + + It 'reports circular dependencies' { + $config = @" +`$schema: $schemaUri +resources: +- name: a + type: Microsoft.DSC.Debug/Echo + dependsOn: + - "[resourceId('Microsoft.DSC.Debug/Echo', 'b')]" +- name: b + type: Microsoft.DSC.Debug/Echo + dependsOn: + - "[resourceId('Microsoft.DSC.Debug/Echo', 'a')]" +"@ + $result = $config | dsclint --file - --no-discovery 2>$TestDrive/error.log | ConvertFrom-Json + $LASTEXITCODE | Should -Be 5 -Because (Get-Content $TestDrive/error.log -Raw) + $diagnostic = $result.diagnostics | Where-Object rule -eq 'circular-dependency' + $diagnostic.path | Should -BeExactly '/resources/0/dependsOn' + $diagnostic.message | Should -BeLike '*a -> b -> a*' + } + + It 'warns when --system-root is given but systemRoot() is not used' { + $result = $cleanConfig | dsclint --file - --no-discovery --system-root $TestDrive 2>$TestDrive/error.log | ConvertFrom-Json + $LASTEXITCODE | Should -Be 0 -Because (Get-Content $TestDrive/error.log -Raw) + $diagnostic = $result.diagnostics | Where-Object rule -eq 'system-root-unused' + $diagnostic | Should -Not -BeNullOrEmpty + $diagnostic.level | Should -BeExactly 'warning' + $diagnostic.path | Should -BeExactly '' + + $config = @" +`$schema: $schemaUri +resources: +- name: a + type: Microsoft.DSC.Debug/Echo + properties: + output: "[systemRoot()]" +"@ + $result = $config | dsclint --file - --no-discovery --system-root $TestDrive 2>$TestDrive/error.log | ConvertFrom-Json + $LASTEXITCODE | Should -Be 0 -Because (Get-Content $TestDrive/error.log -Raw) + $result.diagnostics | Where-Object rule -eq 'system-root-unused' | Should -BeNullOrEmpty + } + + It 'checks supplied parameter values' { + $config = @" +`$schema: $schemaUri +parameters: + count: + type: int + required: + type: string +resources: +- name: a + type: Microsoft.DSC.Debug/Echo + properties: + output: "[concat(string(parameters('count')), parameters('required'))]" +"@ + $result = $config | dsclint --file - --no-discovery --parameters '{"parameters": {"count": "five", "extra": 1}}' 2>$TestDrive/error.log | ConvertFrom-Json + $LASTEXITCODE | Should -Be 5 -Because (Get-Content $TestDrive/error.log -Raw) + $result.diagnostics.rule | Should -Contain 'parameter-value-type' + $result.diagnostics.rule | Should -Contain 'missing-parameter-value' + $result.diagnostics.rule | Should -Contain 'undeclared-parameter-value' + } + + It 'reports unknown resource types only when discovery runs' { + $config = @" +`$schema: $schemaUri +resources: +- name: a + type: Not.Installed/Anywhere +"@ + $result = $config | dsclint --file - 2>$TestDrive/error.log | ConvertFrom-Json + $LASTEXITCODE | Should -Be 0 -Because (Get-Content $TestDrive/error.log -Raw) + $result.discovery | Should -BeTrue + $diagnostic = $result.diagnostics | Where-Object rule -eq 'unknown-resource-type' + $diagnostic.level | Should -BeExactly 'warning' + $diagnostic.path | Should -BeExactly '/resources/0/type' + + $result = $config | dsclint --file - --no-discovery 2>$TestDrive/error.log | ConvertFrom-Json + $LASTEXITCODE | Should -Be 0 -Because (Get-Content $TestDrive/error.log -Raw) + $result.discovery | Should -BeFalse + $result.diagnostics | Where-Object rule -eq 'unknown-resource-type' | Should -BeNullOrEmpty + } + + It 'lints several files in one run and records the file on each diagnostic' { + Set-Content -Path "$TestDrive/clean.dsc.yaml" -Value $cleanConfig + Set-Content -Path "$TestDrive/duplicate.dsc.yaml" -Value $duplicateConfig + $result = dsclint --file "$TestDrive/clean.dsc.yaml" --file "$TestDrive/duplicate.dsc.yaml" --no-discovery 2>$TestDrive/error.log | ConvertFrom-Json + $LASTEXITCODE | Should -Be 5 -Because (Get-Content $TestDrive/error.log -Raw) + $result.diagnostics.Count | Should -Be 1 + $result.diagnostics[0].file | Should -BeLike '*duplicate.dsc.yaml' + } + + It 'rejects a resource manifest' { + $manifest = @" +`$schema: https://aka.ms/dsc/schemas/v3/bundled/resource/manifest.json +type: Test/Example +version: 1.0.0 +"@ + $result = $manifest | dsclint --file - --no-discovery 2>$TestDrive/error.log | ConvertFrom-Json + $LASTEXITCODE | Should -Be 5 -Because (Get-Content $TestDrive/error.log -Raw) + $result.diagnostics.Count | Should -Be 1 + $result.diagnostics[0].rule | Should -BeExactly 'unsupported-document-type' + } + + It 'lists the rules with their levels' { + $rules = dsclint --list-rules --output-format json 2>$TestDrive/error.log | ConvertFrom-Json + $LASTEXITCODE | Should -Be 0 -Because (Get-Content $TestDrive/error.log -Raw) + $rules.id | Should -Contain 'duplicate-resource' + $rules.id | Should -Contain 'parse-error' + ($rules | Where-Object id -eq 'parse-error').fixed | Should -BeTrue + $duplicate = $rules | Where-Object id -eq 'duplicate-resource' + $duplicate.defaultLevel | Should -BeExactly 'error' + $duplicate.level | Should -BeExactly 'error' + $duplicate.tier | Should -BeExactly 'static' + ($rules | Where-Object id -eq 'unknown-resource-type').tier | Should -BeExactly 'resource' + + $rules = dsclint --list-rules --rule duplicate-resource=warning --output-format json 2>$TestDrive/error.log | ConvertFrom-Json + ($rules | Where-Object id -eq 'duplicate-resource').level | Should -BeExactly 'warning' + } + + It 'lints the shipped example without errors: ' -TestCases ( + Get-ChildItem (Join-Path $PSScriptRoot '../../dsc/examples') -Filter '*.dsc.*' | + Where-Object { $_.Extension -in @('.yaml', '.json') -and $_.Name -ne 'invalid_schema.dsc.yaml' } | + ForEach-Object { @{ name = $_.Name; path = $_.FullName } } + ) { + param($name, $path) + $result = dsclint --file $path --no-discovery 2>$TestDrive/error.log | ConvertFrom-Json + $LASTEXITCODE | Should -Be 0 -Because ($result.diagnostics | ConvertTo-Json -Depth 5) + $result.summary.errors | Should -Be 0 + } +} diff --git a/helpers.build.psm1 b/helpers.build.psm1 index 8c9ff5e74..2282194c1 100644 --- a/helpers.build.psm1 +++ b/helpers.build.psm1 @@ -3066,6 +3066,9 @@ function Build-DscDebPackage{ $symlinkPath = Join-Path $debBuildRoot 'usr' 'bin' 'dsc-bicep-ext' New-Item -ItemType SymbolicLink -Path $symlinkPath -Target '/opt/dsc/dsc-bicep-ext' -Force > $null + $symlinkPath = Join-Path $debBuildRoot 'usr' 'bin' 'dsclint' + New-Item -ItemType SymbolicLink -Path $symlinkPath -Target '/opt/dsc/dsclint' -Force > $null + # Read the control template and replace placeholders $controlTemplate = Get-Content "$PSScriptRoot/packaging/deb/control" -Raw $controlContent = $controlTemplate.Replace('VERSION_PLACEHOLDER', $productVersion).Replace('ARCH_PLACEHOLDER', $debArch) diff --git a/lib/dsc-lib/Cargo.toml b/lib/dsc-lib/Cargo.toml index ea1de3f57..2cf177c4c 100644 --- a/lib/dsc-lib/Cargo.toml +++ b/lib/dsc-lib/Cargo.toml @@ -24,7 +24,7 @@ path-absolutize = { workspace = true } regex = { workspace = true } rt-format = { workspace = true } rust-i18n = { workspace = true } -# reqwest = { version = "0.12.8", features = ["native-tls"], default-features = false } +saphyr-parser = { workspace = true } schemars = { workspace = true } serde = { workspace = true } serde_json = { workspace = true } @@ -64,6 +64,8 @@ pretty_assertions = { workspace = true } test-case = { workspace = true } # Enables serializing test execution for shared context stubs serial_test = { workspace = true } +# Enables creating temporary directories for discovery tests +tempfile = { workspace = true } [target.'cfg(not(target_os = "windows"))'.dev-dependencies] nix = { workspace = true, features = ["user"] } diff --git a/lib/dsc-lib/locales/en-us.toml b/lib/dsc-lib/locales/en-us.toml index dc66170cc..4f9ea0ad2 100644 --- a/lib/dsc-lib/locales/en-us.toml +++ b/lib/dsc-lib/locales/en-us.toml @@ -124,6 +124,13 @@ importingParametersFromComplexInput = "Importing parameters from complex input" importingParametersFromInput = "Importing parameters from simple input" invalidParamsFormat = "Invalid parameters format: %{error}" +[diagnostics.schema] +unexpectedProperty = "'%{property}' isn't a known property" +missingProperty = "missing required property '%{property}'" +didYouMean = "Did you mean '%{suggestion}' instead of '%{property}'?" +knownProperties = "Known properties: %{properties}" +at = "at '%{path}'" + [discovery.commandDiscovery] appendingEnvPath = "Appending PATH to resourcePath" originalPath = "Original PATH: %{path}" @@ -247,6 +254,10 @@ processTrace = "PID %{process_id}: %{message}" processTraceWithTarget = "PID %{process_id}: %{target}: %{line_number}: %{message}" redactedSchemaErrorMessage = "'secureString' and 'secureObject' values are not compatible with the schema, use '.secureString' or '.secureObject' in an expression to convert to string or object" invokeNotSupportedForResources = "Operation argument 'invoke' is not supported for resources" +schemaSubjectInput = "input for '%{resource}'" +schemaSubjectOutput = "output from '%{resource}'" +schemaSourceInput = "input" +schemaSourceOutput = "output" [dscresources.dscresource] invokeGet = "Invoking get for '%{resource}'" @@ -280,7 +291,6 @@ resourceDoesNotImplementValidate = "Resource does not implement validation, usin resourceImplementsValidate = "Resource implements validation" resourceValidationFailed = "Resource failed validation" validatingSchema = "Validating against schema" -validationFailed = "Failed validation" adapterResourceNotFound = "Adapter resource '%{adapter}' not found" adapterManifestNotFound = "Adapter manifest for '%{adapter}' not found" adapterDoesNotSupportDelete = "Adapter '%{adapter}' does not support delete operation" @@ -934,6 +944,30 @@ parsingExpression = "Parsing expression" unknownExpressionType = "Unknown expression type: %{kind}" skippingExpressionProcessing = "Skipping expression processing" +[plugins.command_name] +invalidName = "invalid plug-in command name '%{name}': %{err}" +emptyName = "command name cannot be an empty string" +invalidFirstCharacter = "command name must start with a lowercase ASCII letter, but starts with '%{character}'" +invalidCharacters = "command name can only contain lowercase ASCII letters, ASCII digits, and hyphens, but contains '%{characters}'" +misplacedHyphen = "command name cannot end with a hyphen or contain consecutive hyphens" + +[plugins.discovery] +disabled = "Skipping plug-in discovery because the `%{setting}` setting is `false`" +directoryNotReadable = "Skipping directory '%{path}' because it can't be read" +failedLoadManifest = "Failed to load plug-in manifest '%{path}': %{err}" +failedLoadManifestList = "Skipping manifest list '%{path}' for plug-in discovery because it can't be parsed: %{err}" +pluginsNotArray = "Skipping the plug-ins in manifest list '%{path}' because the `plugins` property isn't an array" +conditionNotMet = "Condition '%{condition}' not met, skipping plug-in '%{plugin}' in manifest '%{path}'" +reservedCommand = "Skipping plug-in '%{plugin}' in manifest '%{path}' because its command name '%{command}' is the name of a built-in command" +newerVersionFound = "Using version %{version} of plug-in '%{plugin}' from manifest '%{path}'" +pluginFound = "Plug-in '%{plugin}' version %{version} found for command '%{command}'" +pluginShadowed = "Plug-in '%{plugin}' is shadowed because a plug-in found earlier provides the command '%{command}'" + +[plugins.plugin_manifest] +pluginManifestSchemaTitle = "Plug-in manifest schema URI" +pluginManifestSchemaDescription = "Defines the JSON Schema the plug-in manifest adheres to." +pathNotRelative = "The `manifestRelativePath` argument '%{path}' of plug-in '%{plugin}' must be a non-empty path relative to the directory that contains the manifest" + [dscerror] adapterNotFound = "Adapter not found" booleanConversion = "Function boolean argument conversion" @@ -1120,3 +1154,140 @@ invalidRegex = "wildcard type name '%{name}' could not be converted into a valid invalidTypeName = "invalid wildcard type name '%{text}': %{err}" missingNameSegment = "missing name segment when no wildcard in the pattern can match a name segment" noWildcard = "wildcard type name '%{text}' doesn't contain any wildcard characters (`*`)" + +[lint.mod] +unknownRule = "Unknown lint rule" +invalidRuleOverride = "Invalid rule level override, expected '='" +invalidRuleLevel = "Invalid rule level, expected 'off', 'info', 'warning', or 'error'" +invalidSettings = "The 'lint' setting is invalid" +invalidParameters = "The supplied parameters are invalid" +fixedRuleOverrideIgnored = "The level of lint rule '%{rule}' is fixed and can't be changed; the override was ignored" +unknownRuleInSettings = "The settings file configures unknown lint rule '%{rule}'; the entry was ignored" +parseError = "The document isn't valid JSON or YAML: %{error}" +emptyDocument = "The document is empty" +parseErrorDescription = "The document must be valid JSON or YAML." +schemaInvalidDescription = "The document must validate against the configuration document JSON schema." +documentInvalid = "The document couldn't be loaded as a configuration: %{error}" +unsupportedDocumentType = "Only configuration documents can be linted; the '$schema' identifies a %{kind}" +unsupportedDocumentTypeDescription = "The document must be a configuration document; manifests aren't supported yet." +discoveryFailed = "Resource discovery failed, so the rules that inspect installed resources were skipped: %{error}" +discoveryFailedDescription = "Resource discovery must succeed for the rules that inspect installed resources to run." + +[lint.options] +documentKindConfiguration = "configuration document" +documentKindResourceManifest = "resource manifest" +documentKindExtensionManifest = "extension manifest" + +[lint.report] +inputSourceName = "input" + +[lint.rules.document] +duplicateResourceDescription = "Resource names must be unique among resources of the same type at the same level." +duplicateResource = "Resource '%{name}' of type '%{type_name}' is declared more than once" +duplicateResourceNameDescription = "Resources at the same level should have distinct names even when their types differ, because dependencies are resolved by name first." +duplicateResourceName = "Resource name '%{name}' is used by both '%{first_type}' and '%{second_type}'; dependencies are resolved by name first, so use distinct names" +emptyResourcesDescription = "The configuration should declare at least one resource." +emptyResources = "The configuration doesn't declare any resources" +unknownMicrosoftDscMetadataDescription = "Keys under 'Microsoft.DSC' metadata must be ones DSC recognizes; unknown keys are silently ignored." +unknownMicrosoftDscMetadata = "Unknown key '%{key}' under 'Microsoft.DSC' metadata is ignored" +apiVersionAliasDescription = "Resources should use 'requireVersion' rather than its alias 'apiVersion'." +apiVersionAlias = "'apiVersion' is an alias of 'requireVersion'; use 'requireVersion'" +dateVersionDescription = "Version requirements should be semantic; date-based versions are deprecated." +dateVersion = "Date-based version requirement '%{version}' is deprecated; use a semantic version requirement" +missingRequireVersionDescription = "Resources should pin the version they require with 'requireVersion'." +missingRequireVersion = "Resource '%{name}' doesn't pin the version of '%{type_name}' with 'requireVersion'" + +[lint.rules.expression] +invalidExpressionDescription = "Strings that start with '[' must be valid expressions; escape a literal bracket with '[['." +invalidExpression = "Expression '%{expression}' failed to parse: %{error}" +invalidExpressionInSecureParameter = "The default value of secure parameter '%{name}' isn't a valid expression; the expression isn't shown because it can hold a secret" +unknownFunctionDescription = "Expressions may only call built-in functions or user-defined functions declared in 'functions'." +unknownFunction = "Unknown function '%{name}'" +unknownFunctionSuggestion = "Unknown function '%{name}'; function names are case-sensitive, did you mean '%{suggestion}'?" +unknownUserFunction = "Unknown user-defined function '%{name}'; it isn't declared in 'functions'" +invalidArgumentCountDescription = "Function calls must pass the number of arguments the function accepts." +invalidArgumentTypeDescription = "Literal arguments must be of a kind the function accepts at that position." +undefinedParameterDescription = "parameters() must reference a parameter declared in 'parameters'." +undefinedParameter = "Parameter '%{name}' isn't declared in 'parameters'" +undefinedVariableDescription = "variables() must reference a variable declared in 'variables'." +undefinedVariable = "Variable '%{name}' isn't declared in 'variables'" +conditionNotExpressionDescription = "A 'condition' must be an expression that evaluates to true; a plain string never does." +conditionNotExpression = "Condition '%{condition}' is a plain string, so it never evaluates to true and the item is always skipped" + +[lint.rules.parameter] +noUnusedParametersDescription = "Declared parameters should be referenced by at least one expression." +noUnusedParameters = "Parameter '%{name}' is declared but never used" +noUnusedVariablesDescription = "Declared variables should be referenced by at least one expression." +noUnusedVariables = "Variable '%{name}' is declared but never used" +parameterDefaultTypeDescription = "A literal default value must satisfy the parameter's type and constraints." +parameterDefaultType = "The default value of parameter '%{name}' is invalid: %{reason}" +valueNotOfType = "the value isn't of type '%{type_name}'" +secureStringNotPlain = "a 'secureString' value must be supplied as a plain string" +invalidParameterConstraintsDescription = "Parameter constraints must be consistent with each other and with the parameter type." +minValueExceedsMaxValue = "Parameter '%{name}' has a minValue of %{min} that is greater than its maxValue of %{max}" +minLengthExceedsMaxLength = "Parameter '%{name}' has a minLength of %{min} that is greater than its maxLength of %{max}" +valueLimitOnNonInt = "Parameter '%{name}' has '%{constraint}' but its type is '%{type_name}'; value limits only apply to 'int' parameters" +lengthLimitOnOtherType = "Parameter '%{name}' has '%{constraint}' but its type is '%{type_name}'; length limits only apply to 'string', 'secureString', and 'array' parameters" +negativeLength = "Parameter '%{name}' has a negative '%{constraint}'" +emptyAllowedValues = "Parameter '%{name}' has an empty 'allowedValues' list, so no value is allowed" +allowedValueTypeMismatch = "Parameter '%{name}' allows a value that isn't of type '%{type_name}'" +secureParameterDefaultDescription = "Secure parameters shouldn't have a hard-coded default value, because anyone with the document can read it." +secureParameterDefault = "Secure parameter '%{name}' has a hard-coded default value" +preferSecureTypeDescription = "Parameters whose names suggest a secret should use the 'secureString' or 'secureObject' type." +preferSecureType = "Parameter '%{name}' looks like a secret; use type '%{secure_type}' instead of '%{type_name}'" +secureValueInOutputDescription = "Outputs shouldn't use secure values; DSC only withholds outputs declared as 'secureString' or 'secureObject', so a plain value extracted into any other output is returned in the result." +secureValueInOutput = "Output '%{name}' uses the secure value from %{source}; because the output isn't declared as 'secureString' or 'secureObject', a plain value extracted from it is returned in the result" +useParameterDescriptionDescription = "Parameters should have a description." +useParameterDescription = "Parameter '%{name}' has no description" +undeclaredParameterValueDescription = "Supplied parameter values must match a declared parameter." +undeclaredParameterValue = "A value was supplied for parameter '%{name}', which isn't declared in 'parameters'" +missingParameterValueDescription = "Parameters without a default value must be supplied a value." +missingParameterValue = "Parameter '%{name}' has no default value and no value was supplied" +parameterValueTypeDescription = "Supplied parameter values must satisfy the parameter's type and constraints." +parameterValueType = "The supplied value for parameter '%{name}' is invalid: %{reason}" + +[lint.rules.dependency] +unresolvedDependencyDescription = "dependsOn entries and reference() calls must point at a resource declared at the same level." +unresolvedDependency = "Resource '%{name}' depends on '%{dependency_type}' named '%{dependency_name}', which isn't declared" +dependencyTypeMismatch = "Resource '%{name}' depends on '%{dependency_type}' named '%{dependency_name}', but that resource has type '%{actual_type}'" +circularDependencyDescription = "Resource dependencies must not form a cycle." +circularDependency = "Resource '%{name}' is part of a dependency cycle: %{cycle}" +referenceWithoutDependsOnDescription = "A resource that references another resource must list it in 'dependsOn' so the referenced resource runs first." +referenceWithoutDependsOn = "Resource '%{name}' references '%{target}' but doesn't list it in 'dependsOn'; the reference fails unless that resource has already run" +duplicateDependsOnDescription = "A dependency should be listed once." +duplicateDependsOn = "Resource '%{name}' lists the dependency '%{target}' more than once" + +[lint.rules.deprecation] +copyLoopDeprecatedDescription = "Copy loops are deprecated and will be removed in a future release." +copyLoopDeprecated = "Copy loop '%{name}' is deprecated and will be removed in a future release; see https://github.com/PowerShell/DSC/issues/1429" +copyOptionsUnsupportedDescription = "Copy loops don't support 'mode' or 'batchSize'." +copyOptionsUnsupported = "Copy loop option '%{option}' isn't supported" +securityContextInMetadataDescription = "The security context should be set with 'directives.securityContext' rather than 'Microsoft.DSC' metadata." +securityContextInMetadata = "Setting the security context in 'Microsoft.DSC' metadata is deprecated; use 'directives.securityContext'" +securityContextConflictDescription = "The security context in metadata and in directives must agree." +securityContextConflict = "The security context '%{metadata}' in metadata conflicts with '%{directive}' in directives" +dscVersionUnsatisfiedDescription = "The DSC version requirement in 'directives.version' should be satisfied by the version of DSC in use." +dscVersionUnsatisfied = "The document requires DSC version '%{required}', but this is DSC version '%{current}'" + +[lint.rules.environment] +systemRootUnusedDescription = "When a system root is specified, the document should use systemRoot() so its paths target that root." +systemRootUnused = "A system root of '%{path}' was specified, but no expression calls systemRoot(), so the document doesn't target it" +hardcodedSystemPathDescription = "Absolute operating system paths should be built with systemRoot() so the document can target another root." +hardcodedSystemPath = "'%{value}' is an absolute operating system path; build it with systemRoot() so the document can target another root" + +[lint.rules.resource] +unknownResourceTypeDescription = "Each resource type must be installed." +unknownResourceType = "Resource type '%{type_name}' isn't installed" +unsatisfiedRequireVersionDescription = "An installed version of the resource type must satisfy 'requireVersion'." +unsatisfiedRequireVersion = "No installed version of '%{type_name}' satisfies the requirement '%{version}'" +deprecatedResourceDescription = "Resources shouldn't use deprecated resource types." +deprecatedResource = "Resource type '%{type_name}' is deprecated: %{message}" +propertySchemaMismatchDescription = "Literal property values must validate against the resource's schema." +propertySchemaMismatch = "Property doesn't validate against the schema of '%{type_name}': %{message}" +propertySchemaCompileFailed = "The schema of '%{type_name}' couldn't be compiled, so its properties weren't checked: %{error}" +nestedResourcesUnsupportedDescription = "Only group and adapter resources can declare nested resources." +nestedResourcesUnsupported = "Resource '%{name}' declares nested resources, but '%{type_name}' is a '%{kind}' resource, not a group or adapter" +existFalseUnsupportedDescription = "A resource type must support delete or handle _exist itself for '_exist: false' to work." +existFalseUnsupported = "Resource type '%{type_name}' has neither the 'delete' nor the 'setHandlesExist' capability, so 'dsc config set' fails for '_exist: false'" +canonicalTypeNameDescription = "Resource types should be written with the casing their manifest uses." +canonicalTypeName = "Resource type '%{type_name}' is written as '%{canonical}' in its manifest" diff --git a/lib/dsc-lib/locales/schemas.definitions.yaml b/lib/dsc-lib/locales/schemas.definitions.yaml index 9fa0c5f17..57589ed41 100644 --- a/lib/dsc-lib/locales/schemas.definitions.yaml +++ b/lib/dsc-lib/locales/schemas.definitions.yaml @@ -87,6 +87,26 @@ schemas: Invalid exit code. Each exit code must be defined as a string representing a 32-bit signed integer, like `5` or `-2147024891`. + pluginCommandName: + title: + en-us: Plug-in command name + description: + en-us: >- + Defines the name of the command a plug-in adds to DSC. + markdownDescription: + en-us: |- + Defines the name of the command a plug-in adds to DSC. The user specifies the command + name after `dsc`, like `lint` in `dsc lint`. + + A command name must start with a lowercase ASCII letter and can only contain lowercase + ASCII letters, ASCII digits, and hyphens. It can't end with a hyphen or contain + consecutive hyphens. DSC compares command names case-sensitively. + patternErrorMessage: + en-us: >- + Invalid plug-in command name. A command name must start with a lowercase ASCII letter + and can only contain lowercase ASCII letters, ASCII digits, and hyphens, like `lint` or + `resource-kit`. It can't end with a hyphen or contain consecutive hyphens. + resourceType: title: en-us: Fully qualified type name diff --git a/lib/dsc-lib/locales/schemas.outputs.yaml b/lib/dsc-lib/locales/schemas.outputs.yaml new file mode 100644 index 000000000..c50a4b848 --- /dev/null +++ b/lib/dsc-lib/locales/schemas.outputs.yaml @@ -0,0 +1,126 @@ +_version: 2 +schemas: + outputs: + plugin: + list: + title: + en-us: Plug-in list output + description: + en-us: >- + Describes a command plug-in that DSC discovered. + markdownDescription: + en-us: |- + Describes a command plug-in that DSC discovered. The `dsc plugin list` command returns + one of these objects for every discovered plug-in, including plug-ins that are shadowed + by another plug-in that provides the same command. + + For more information, see the [plug-in list output reference][01]. + + [01]: https://learn.microsoft.com/powershell/dsc/reference/schemas/outputs/plugin/list + fields: + type: + title: + en-us: Type + description: + en-us: >- + The fully qualified type name that identifies the plug-in. + markdownDescription: + en-us: |- + Identifies the plug-in by its fully qualified type name, like `Microsoft.DSC/Lint`. + version: + title: + en-us: Version + description: + en-us: >- + The semantic version of the plug-in. + markdownDescription: + en-us: |- + Indicates the semantic version of the plug-in. When DSC finds more than one + manifest with the same type, it only returns the plug-in with the highest version. + command: + title: + en-us: Command + description: + en-us: >- + The name of the command the plug-in adds to DSC. + markdownDescription: + en-us: |- + Indicates the name of the command the plug-in adds to DSC. For example, a plug-in + with the command `lint` is invoked as `dsc lint`. + description: + title: + en-us: Description + description: + en-us: >- + The description of the command. + markdownDescription: + en-us: |- + Describes the command. The value is `null` when the manifest doesn't define a + description. + deprecationMessage: + title: + en-us: Deprecation message + description: + en-us: >- + The message that indicates the plug-in is deprecated. + markdownDescription: + en-us: |- + Indicates that the plug-in is deprecated. The value is `null` when the plug-in + isn't deprecated. DSC writes the message as a warning each time the command is + invoked. + path: + title: + en-us: Path + description: + en-us: >- + The path of the file that defines the plug-in manifest. + markdownDescription: + en-us: |- + Indicates the path of the file that defines the plug-in manifest. For a plug-in + defined in a manifest list, this is the path of the manifest list. + directory: + title: + en-us: Directory + description: + en-us: >- + The directory that contains the plug-in manifest. + markdownDescription: + en-us: |- + Indicates the directory that contains the plug-in manifest. DSC resolves the + `manifestRelativePath` arguments of the plug-in against this directory. + shadowed: + title: + en-us: Shadowed + description: + en-us: >- + Indicates whether another plug-in provides the same command. + markdownDescription: + en-us: |- + Indicates whether another plug-in that DSC discovered earlier provides the same + command. DSC never runs a shadowed plug-in for its command name. To run a + shadowed plug-in, specify its type name instead of the command name, like + `dsc OpenDSC/Lint`. + lint: + result: + title: + en-us: dsclint result + description: + en-us: >- + Describes the output of the `dsclint` command. + markdownDescription: + en-us: |- + Describes the output of the `dsclint` command: the diagnostics the lint rules + reported for the document, counts of the diagnostics by level, whether the document + is free of error-level diagnostics, and whether resource discovery ran. + rules: + title: + en-us: dsclint rule + description: + en-us: >- + Describes a lint rule and the level it reports at, as listed by `dsclint --list-rules`. + markdownDescription: + en-us: |- + Describes a lint rule and the level it reports at, as listed by + `dsclint --list-rules`. The `defaultLevel` is the level the rule reports at unless + it is overridden; `level` is the level in effect after the settings file and any + `--rule` overrides are applied. diff --git a/lib/dsc-lib/locales/schemas.plugin.yaml b/lib/dsc-lib/locales/schemas.plugin.yaml new file mode 100644 index 000000000..b97d6f21b --- /dev/null +++ b/lib/dsc-lib/locales/schemas.plugin.yaml @@ -0,0 +1,188 @@ +_version: 2 +schemas: + plugin: + manifest: + title: + en-us: Plug-in manifest + description: + en-us: >- + Defines a command plug-in: the command it adds to DSC and how DSC runs it. + markdownDescription: + en-us: |- + Defines a command plug-in. A plug-in adds a top-level command to DSC without being part + of the DSC executable. The manifest names the command and the executable that implements + it. + + DSC discovers plug-in manifests in the same directories as resource and extension + manifests. The file name of a plug-in manifest must end with `.dsc.plugin.json`, + `.dsc.plugin.yaml`, or `.dsc.plugin.yml`. A manifest list can define plug-ins in its + `plugins` property. + + When the first argument to DSC isn't a built-in command, DSC runs the plug-in that + provides the command with the remaining arguments and exits with the plug-in's exit code. + + For more information, see the [plug-in manifest reference][01]. + + [01]: https://learn.microsoft.com/powershell/dsc/reference/schemas/plugin/manifest/root + fields: + type: + title: + en-us: Type + description: + en-us: >- + The fully qualified type name that identifies the plug-in. + markdownDescription: + en-us: |- + Defines the fully qualified type name that identifies the plug-in, like + `Microsoft.DSC/Lint`. The name uses the same syntax as the type name of a resource + or an extension. + + When DSC finds more than one manifest with the same type, it uses the manifest with + the highest version. + version: + title: + en-us: Version + description: + en-us: >- + The semantic version of the plug-in. + markdownDescription: + en-us: |- + Defines the semantic version of the plug-in. When DSC finds more than one manifest + with the same type, it uses the manifest with the highest version. + condition: + title: + en-us: Condition + description: + en-us: >- + An expression that defines whether the plug-in is available. + markdownDescription: + en-us: |- + Defines a DSC expression that must evaluate to a boolean. When the expression + evaluates to `false`, DSC treats the plug-in as not installed: the command isn't + listed and can't be invoked. + + Use a condition for a plug-in that depends on a tool that might not be installed. For + example, the following condition makes a plug-in available only when `pwsh` is + discoverable: + + ```json + "condition": "[not(equals(tryWhich('pwsh'), null()))]" + ``` + deprecationMessage: + title: + en-us: Deprecation message + description: + en-us: >- + A message that indicates the plug-in is deprecated. + markdownDescription: + en-us: |- + Defines a message that indicates the plug-in is deprecated. When this property is + defined, DSC writes the message as a warning each time the command is invoked. + description: + title: + en-us: Description + description: + en-us: >- + A one-line description of the command. + markdownDescription: + en-us: |- + Defines a one-line description of the command. DSC shows the description next to the + command in `dsc --help` and in the output of `dsc plugin list`. + tags: + title: + en-us: Tags + description: + en-us: >- + Defines a list of searchable terms for the plug-in. + markdownDescription: + en-us: |- + Defines a list of searchable terms for the plug-in. + metadata: + title: + en-us: Metadata + description: + en-us: >- + Arbitrary data for the plug-in that DSC doesn't interpret. + markdownDescription: + en-us: |- + Defines arbitrary data for the plug-in. DSC doesn't interpret the metadata. It's + returned with the manifest in the output of `dsc plugin list`. + command: + title: + en-us: Command + description: + en-us: >- + Defines the command the plug-in adds to DSC and how DSC runs it. + markdownDescription: + en-us: |- + Defines the command the plug-in adds to DSC and how DSC runs it. + + DSC starts the executable in the caller's working directory, with the standard input, + output, and error streams of DSC. It passes the arguments defined in `args` followed by + the arguments the user specified after the command name. DSC exits with the exit code + of the executable. + + DSC sets environment variables that tell the plug-in how it was invoked, like + `DSC_EXE`, `DSC_VERSION`, `DSC_PLUGIN_COMMAND`, and `DSC_PLUGIN_DIR`. + + For more information, see the [command property reference][01]. + + [01]: https://learn.microsoft.com/powershell/dsc/reference/schemas/plugin/manifest/command + fields: + name: + title: + en-us: Command name + description: + en-us: >- + The name of the command, which the user specifies after `dsc`. + markdownDescription: + en-us: |- + Defines the name of the command, which the user specifies after `dsc`. For example, + a plug-in with the command name `lint` is invoked as `dsc lint`. + + The name can't be the name or the alias of a built-in command, like `config` or + `resource`. DSC always matches built-in commands first and skips a plug-in that + uses one of their names. + + When plug-ins with different types define the same command name, the plug-in DSC + discovers first provides the command. The others are reported as shadowed in the + output of `dsc plugin list` and can be run by their type name, like `dsc OpenDSC/Lint`. + executable: + title: + en-us: Executable + description: + en-us: >- + The name of the executable to run for the command. + markdownDescription: + en-us: |- + Defines the name of the executable to run for the command. The value must be the + name of an executable discoverable in the system's `PATH` environment variable, + the name of an executable in the same directory as the manifest, or the full path + to the executable. A file extension is only required when the operating system + doesn't recognize the file as an executable without it. + + To implement a plug-in as a script, define the interpreter as the executable and + pass the script in `args` with the `manifestRelativePath` argument kind. + args: + title: + en-us: Arguments + description: + en-us: >- + The arguments to pass to the executable before the arguments the user specified. + markdownDescription: + en-us: |- + Defines the list of arguments to pass to the executable before the arguments the + user specified after the command name. Each item is one of the following: + + - A string, which DSC passes as written. + - An object with the `manifestRelativePath` property. DSC replaces the item with + the absolute path of the file, resolved against the directory that contains the + manifest. The value must be a relative path. + + A plug-in runs in the caller's working directory, not in the directory that + contains the manifest. Use `manifestRelativePath` to pass a file that ships with + the plug-in, like a script: + + ```json + "args": ["-NoProfile", "-File", { "manifestRelativePath": "graph.ps1" }] + ``` diff --git a/lib/dsc-lib/locales/schemas.settings.yaml b/lib/dsc-lib/locales/schemas.settings.yaml index 4d522e7d1..0e0c9acbf 100644 --- a/lib/dsc-lib/locales/schemas.settings.yaml +++ b/lib/dsc-lib/locales/schemas.settings.yaml @@ -228,7 +228,29 @@ schemas: markdownDescription: en-us: | Append environment path to resource paths. If this is set to true, the environment path will be appended to the resource paths. - + plugins: + title: + en-us: "Plug-ins" + description: + en-us: "Settings related to command plug-ins" + markdownDescription: + en-us: | + Settings related to command plug-ins. A command plug-in adds a top-level command to + `dsc` without being compiled into it. + fields: + enabled: + title: + en-us: "Enabled" + description: + en-us: "Whether DSC discovers and runs command plug-ins" + markdownDescription: + en-us: | + Whether DSC discovers and runs command plug-ins. The default value is `true`. + + If this is set to `false`, DSC doesn't discover plug-in manifests. A command that + isn't built in fails as an unrecognized subcommand, the help doesn't list plug-in + commands, and listing the plug-ins returns nothing. + sources: default: title: @@ -343,6 +365,14 @@ schemas: markdownDescription: en-us: | Settings related to the resource path. This represents the values for the settings that control the resource path. + plugins: + title: + en-us: "Plug-ins" + description: + en-us: "Settings related to command plug-ins" + markdownDescription: + en-us: | + Settings related to command plug-ins. This represents the values for the settings that control whether DSC discovers and runs command plug-ins. policyFile: title: en-us: "Policy settings file" @@ -383,7 +413,15 @@ schemas: markdownDescription: en-us: | Settings related to the resource path. This represents the values for the settings that control the resource path. - + plugins: + title: + en-us: "Plug-ins" + description: + en-us: "Settings related to command plug-ins" + markdownDescription: + en-us: | + Settings related to command plug-ins. This represents the values for the settings that control whether DSC discovers and runs command plug-ins. A value defined as policy can't be overridden, so setting `enabled` to `false` keeps DSC to the built-in commands. + resolved: field: title: @@ -436,6 +474,14 @@ schemas: markdownDescription: en-us: | Settings related to the resource path. This represents the values for the settings that control the resource path. + plugins: + title: + en-us: "Plug-ins" + description: + en-us: "Settings related to command plug-ins" + markdownDescription: + en-us: | + Settings related to command plug-ins. This represents the values for the settings that control whether DSC discovers and runs command plug-ins. all: title: en-us: "DSC settings" @@ -483,6 +529,8 @@ schemas: directories: [] append_env_path: true restricted: false + plugins: + enabled: true ``` machine: title: diff --git a/lib/dsc-lib/src/configure/mod.rs b/lib/dsc-lib/src/configure/mod.rs index ae6c6870a..a2212cd19 100644 --- a/lib/dsc-lib/src/configure/mod.rs +++ b/lib/dsc-lib/src/configure/mod.rs @@ -6,6 +6,7 @@ use crate::configure::context::{Context, ProcessMode}; use crate::configure::parameters::{SecureObject, SecureString, import_parameters}; use crate::configure::{config_doc::{ExecutionKind, IntOrExpression, Metadata, Parameter, Resource, ResourceDiscoveryMode, RestartRequired, ValueOrCopy}}; use crate::discovery::discovery_trait::DiscoveryFilter; +use crate::diagnostics::{DocumentSource, JsonOrigin}; use crate::dscerror::DscError; use crate::dscresources::dscresource::ResourceCapability; use crate::dscresources::{ @@ -48,6 +49,7 @@ pub struct Configurator { discovery: Discovery, statement_parser: Statement, progress_format: ProgressFormat, + source: Option, } /// Invokes the [`Discovery::find_resource`] method to retrieve a specific resource or raise a @@ -347,7 +349,16 @@ fn add_metadata(dsc_resource: &dyn ResourceInvoke, mut properties: Option) -> Option { +/// Get the adapter a resource requires from its directives, if one is specified. +/// +/// # Arguments +/// +/// * `resource_directives` - The directives of the resource. +/// +/// # Returns +/// +/// The required adapter type name, or `None` when the directives don't require one. +pub(crate) fn get_require_adapter_from_directive(resource_directives: &Option) -> Option { if let Some(directives) = resource_directives && let Some(require_adapter) = &directives.require_adapter { return Some(require_adapter.clone()); @@ -564,6 +575,7 @@ impl Configurator { discovery: discovery.clone(), statement_parser: Statement::new()?, progress_format, + source: None, }; config.validate_config()?; for extension in discovery.extensions.values() { @@ -582,6 +594,57 @@ impl Configurator { &self.config } + /// Record the text the configuration was read from, so that errors found in a resource's + /// properties point at the author's document rather than at the evaluated properties. + /// + /// # Arguments + /// + /// * `source` - The document text and how to name it. + pub fn set_source(&mut self, source: DocumentSource) { + self.source = Some(source); + } + + /// The text the configuration was read from, when the host recorded it. + #[must_use] + pub fn source(&self) -> Option<&DocumentSource> { + self.source.as_ref() + } + + /// Point a schema validation error about a resource's input at the resource's declaration in + /// the document. + /// + /// Resources are validated after their expressions are evaluated, so the error arrives + /// pointing at the evaluated properties. When the document text is known and the resource + /// is declared in it directly, the error is re-anchored to the resource's `properties` in the + /// document. Resources that a copy loop expands keep the evaluated view. An error about what + /// the resource returned keeps pointing at that output, because the document doesn't hold the + /// offending value, and a redacted error stays without source text. Other errors pass + /// through unchanged. + /// + /// # Arguments + /// + /// * `error` - The error the resource operation returned. + /// * `resource` - The resource that was being invoked. + fn anchor_error(&self, error: DscError, resource: &Resource) -> DscError { + let DscError::SchemaValidation(schema_error) = error else { + return error; + }; + if schema_error.origin() == JsonOrigin::Output || schema_error.is_redacted() { + return DscError::SchemaValidation(schema_error); + } + let Some(source) = &self.source else { + return DscError::SchemaValidation(schema_error); + }; + let declared = self.config.resources.iter().position(|declared| { + declared.name == resource.name && declared.resource_type == resource.resource_type + }); + let Some(index) = declared else { + return DscError::SchemaValidation(schema_error); + }; + let base = format!("/resources/{index}/properties"); + DscError::SchemaValidation(Box::new(schema_error.with_source(&source.text, &source.name, &base))) + } + /// Get the discovery. /// /// # Returns @@ -656,7 +719,7 @@ impl Configurator { Err(e) => { progress.set_failure(get_failure_from_error(&e)); progress.write_increment(1); - return Err(e); + return Err(self.anchor_error(e, &resource)); }, }; let end_datetime = chrono::Local::now(); @@ -770,7 +833,7 @@ impl Configurator { Err(e) => { progress.set_failure(get_failure_from_error(&e)); progress.write_increment(1); - return Err(e); + return Err(self.anchor_error(e, &resource)); }, }; end_datetime = chrono::Local::now(); @@ -782,7 +845,7 @@ impl Configurator { Err(e) => { progress.set_failure(get_failure_from_error(&e)); progress.write_increment(1); - return Err(e); + return Err(self.anchor_error(e, &resource)); }, }; @@ -791,7 +854,7 @@ impl Configurator { Err(e) => { progress.set_failure(get_failure_from_error(&e)); progress.write_increment(1); - return Err(e); + return Err(self.anchor_error(e, &resource)); }, }; @@ -810,7 +873,7 @@ impl Configurator { Err(e) => { progress.set_failure(get_failure_from_error(&e)); progress.write_increment(1); - return Err(e); + return Err(self.anchor_error(e, &resource)); }, }; end_datetime = chrono::Local::now(); @@ -939,7 +1002,7 @@ impl Configurator { Err(e) => { progress.set_failure(get_failure_from_error(&e)); progress.write_increment(1); - return Err(e); + return Err(self.anchor_error(e, &resource)); }, }; let end_datetime = chrono::Local::now(); @@ -1033,7 +1096,7 @@ impl Configurator { Err(e) => { progress.set_failure(get_failure_from_error(&e)); progress.write_increment(1); - return Err(e); + return Err(self.anchor_error(e, resource)); }, }; self.context.references.insert(resource_id(&resource.resource_type, &evaluated_name), serde_json::to_value(&export_result.actual_state)?); @@ -1580,6 +1643,9 @@ fn get_failure_from_error(err: &DscError) -> Option { mod tests { use crate::dscresources::dscresource::DscResource; use super::*; + use crate::diagnostics::SchemaValidationError; + use crate::dscresources::dscresource::validate_json_all; + use miette::Diagnostic; use serde_json::json; fn resource_with_export(export: Value) -> DscResource { @@ -1657,4 +1723,60 @@ mod tests { fn filter_input_invalid_json_is_error() { assert!(parse_export_filter_input("not json").is_err()); } + + /// A document whose resources aren't discovered, so a configurator can be created in a test. + const ANCHOR_DOCUMENT: &str = "$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json\ndirectives:\n resourceDiscovery: duringDeployment\nresources:\n- name: a\n type: Test/Echo\n properties:\n output: hi\n verbose: nope\n"; + /// The evaluated properties of the resource in `ANCHOR_DOCUMENT`, as a resource sees them. + const EVALUATED_PROPERTIES: &str = "{\n \"output\": \"hi\",\n \"verbose\": \"nope\"\n}"; + + fn anchored_configurator() -> Configurator { + let json = serde_json::to_string(&serde_yaml::from_str::(ANCHOR_DOCUMENT).unwrap()).unwrap(); + let mut configurator = Configurator::new(&json, ProgressFormat::None).unwrap(); + configurator.set_source(DocumentSource::new("demo.dsc.yaml", ANCHOR_DOCUMENT)); + configurator + } + + /// The error a resource reports for `EVALUATED_PROPERTIES`, before any text is attached. + fn verbose_type_error() -> SchemaValidationError { + let schema = json!({ "type": "object", "properties": { "output": { "type": "string" }, "verbose": { "type": "boolean" } } }); + let violations = validate_json_all("Test/Echo", &schema, &serde_json::from_str(EVALUATED_PROPERTIES).unwrap()).unwrap(); + SchemaValidationError::new("input for 'Test/Echo'", violations) + } + + /// Run an error through `anchor_error` for the resource declared in `ANCHOR_DOCUMENT`. + fn anchored_schema_error(error: SchemaValidationError) -> SchemaValidationError { + let configurator = anchored_configurator(); + let resource = configurator.get_config().resources[0].clone(); + match configurator.anchor_error(DscError::SchemaValidation(Box::new(error)), &resource) { + DscError::SchemaValidation(error) => *error, + other => panic!("expected a schema validation error, got {other}"), + } + } + + fn first_label_offset(error: &SchemaValidationError) -> usize { + error.labels().unwrap().next().unwrap().offset() + } + + #[test] + fn anchor_error_points_input_errors_at_the_document() { + let error = anchored_schema_error(verbose_type_error().with_source(EVALUATED_PROPERTIES, "input", "")); + assert_eq!(error.origin(), JsonOrigin::Input); + assert_eq!(first_label_offset(&error), ANCHOR_DOCUMENT.find("nope").unwrap()); + } + + #[test] + fn anchor_error_keeps_output_errors_on_the_output() { + let error = anchored_schema_error(verbose_type_error().with_origin(JsonOrigin::Output).with_source(EVALUATED_PROPERTIES, "output", "")); + assert_eq!(error.origin(), JsonOrigin::Output); + assert_eq!(first_label_offset(&error), EVALUATED_PROPERTIES.find("\"nope\"").unwrap()); + } + + #[test] + fn anchor_error_keeps_redacted_errors_without_source() { + let error = anchored_schema_error(verbose_type_error().redacted("redacted")); + assert!(error.is_redacted()); + assert!(!error.has_source()); + assert!(error.labels().is_none()); + assert!(!format!("{error:?}").contains("nope")); + } } diff --git a/lib/dsc-lib/src/diagnostics/mod.rs b/lib/dsc-lib/src/diagnostics/mod.rs new file mode 100644 index 000000000..ec687a38e --- /dev/null +++ b/lib/dsc-lib/src/diagnostics/mod.rs @@ -0,0 +1,44 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +//! Diagnostics that point at the text they apply to. +//! +//! DSC reads documents and resource input as JSON or YAML text and validates the parsed value. +//! The types here keep the connection between a problem found in the value and the text it came +//! from, so a host can show the problem in place rather than only describe it: +//! +//! - [`SourceMap`] finds the byte span and line/column of any JSON pointer in JSON or YAML text. +//! - [`SchemaValidationError`] is a [`miette::Diagnostic`] for JSON schema violations, with a +//! label on each violation when its source text is attached, and a [`JsonOrigin`] that says +//! whether the value was input to a resource or output from one. +//! - [`DocumentSource`] carries the text and name of a document a host read, so library code +//! can anchor the problems it finds to the author's text. + +pub mod schema; +pub mod source_map; + +pub use schema::{JsonOrigin, SchemaValidationError}; +pub use source_map::SourceMap; + +/// The text a document was read from, for diagnostics that point into it. +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct DocumentSource { + /// How to name the text in a report: the file it was read from when known, otherwise a + /// description like `input`. + pub name: String, + /// The document text as the author wrote it, as JSON or YAML. + pub text: String, +} + +impl DocumentSource { + /// Create a document source. + /// + /// # Arguments + /// + /// * `name` - How to name the text in a report. + /// * `text` - The document text. + #[must_use] + pub fn new(name: impl Into, text: impl Into) -> Self { + Self { name: name.into(), text: text.into() } + } +} diff --git a/lib/dsc-lib/src/diagnostics/schema.rs b/lib/dsc-lib/src/diagnostics/schema.rs new file mode 100644 index 000000000..06075644e --- /dev/null +++ b/lib/dsc-lib/src/diagnostics/schema.rs @@ -0,0 +1,461 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +//! Schema validation failures that point at the text they were found in. + +// TODO: Remove when updating to rustc 1.94; false positive from thiserror + rust-i18n t!() macro +#![allow(unused_assignments)] + +use std::fmt::{self, Display}; + +use miette::{Diagnostic, LabeledSpan, NamedSource, SourceCode}; +use rust_i18n::t; +use serde_json::Value; + +use crate::dscresources::dscresource::{SchemaViolation, SchemaViolationDetail}; + +use super::source_map::SourceMap; + +/// Property names within this edit distance of an unknown property are suggested as corrections. +const MAX_SUGGESTION_DISTANCE: usize = 2; +/// The most property names to list when there is no close match. +const MAX_LISTED_PROPERTIES: usize = 12; + +/// Where the value that was validated against a schema came from. +/// +/// A host that knows the document a value was taken from can re-anchor an error about input to +/// the document; an error about what a resource returned has no place in the document. +#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)] +pub enum JsonOrigin { + /// The desired state, filter, or properties a caller passed to the resource. + #[default] + Input, + /// The state the resource returned. + Output, +} + +/// Every violation found when validating a value against a JSON schema. +/// +/// The error is a [`miette::Diagnostic`]: when source text is attached with +/// [`with_source`](Self::with_source), each violation becomes a label on the span of the text it +/// applies to, and unknown properties get a suggestion in the help text when the schema is +/// attached with [`with_schema`](Self::with_schema). Without source text the error still lists +/// every violation and its JSON pointer in its message. +/// +/// An error that was [`redacted`](Self::redacted) because the value holds secure data never +/// accepts source text again, so a host that re-anchors errors can't reintroduce the value. +#[derive(Debug)] +pub struct SchemaValidationError { + subject: String, + violations: Vec, + origin: JsonOrigin, + redacted: bool, + source: Option>, + labels: Vec, + help: Option, +} + +impl SchemaValidationError { + /// Create an error for a set of violations. + /// + /// The origin is [`JsonOrigin::Input`] until [`with_origin`](Self::with_origin) says + /// otherwise. + /// + /// # Arguments + /// + /// * `subject` - What was validated, as it should read in the message, like + /// `input for 'Microsoft.DSC.Debug/Echo'` or `Configuration`. + /// * `violations` - The violations found; must not be empty. + #[must_use] + pub fn new(subject: impl Into, violations: Vec) -> Self { + Self { + subject: subject.into(), + violations, + origin: JsonOrigin::default(), + redacted: false, + source: None, + labels: Vec::new(), + help: None, + } + } + + /// Record where the validated value came from. + /// + /// # Arguments + /// + /// * `origin` - Whether the value was input to a resource or output from one. + #[must_use] + pub fn with_origin(mut self, origin: JsonOrigin) -> Self { + self.origin = origin; + self + } + + /// Use the schema to suggest corrections for unknown properties in the help text. + /// + /// # Arguments + /// + /// * `schema` - The schema the value was validated against. + #[must_use] + pub fn with_schema(mut self, schema: &Value) -> Self { + let mut suggestions = Vec::new(); + let mut listed = Vec::new(); + for violation in &self.violations { + let SchemaViolationDetail::UnexpectedProperties(unexpected) = &violation.detail else { + continue; + }; + let Some(known) = known_properties(schema, &violation.schema_path) else { + continue; + }; + for property in unexpected { + match closest(property, &known) { + Some(suggestion) => suggestions.push(t!("diagnostics.schema.didYouMean", property = property, suggestion = suggestion).to_string()), + None if listed.is_empty() && !known.is_empty() && known.len() <= MAX_LISTED_PROPERTIES => { + listed.push(t!("diagnostics.schema.knownProperties", properties = known.join(", ")).to_string()); + }, + None => {}, + } + } + } + suggestions.extend(listed); + if !suggestions.is_empty() { + self.help = Some(suggestions.join("\n")); + } + self + } + + /// Attach the text the validated value came from, so the report can show each violation in + /// place. + /// + /// Replaces any text attached before, so a host that knows the document a value was taken + /// from can point at the document instead of the value. + /// + /// # Arguments + /// + /// * `text` - The document text, as JSON or YAML. + /// * `name` - How to name the text in the report, like `input` or a file path. + /// * `base` - The JSON pointer in `text` that the violations' instance paths are relative + /// to; empty when the text is the validated value itself. + /// + /// # Returns + /// + /// The error with the text attached, or unchanged when the text can't be parsed or the error + /// was [`redacted`](Self::redacted). + #[must_use] + pub fn with_source(mut self, text: &str, name: &str, base: &str) -> Self { + if self.redacted { + return self; + } + let Some(map) = SourceMap::parse(text) else { + return self; + }; + let mut labels = Vec::new(); + for violation in &self.violations { + labels.extend(labels_for(violation, &map, base)); + } + if labels.is_empty() { + return self; + } + self.labels = labels; + self.source = Some(NamedSource::new(name, map.text().to_string())); + self + } + + /// Replace every violation message and drop the source text, for a value that holds secure + /// data the report must not echo. + /// + /// The error stays redacted: a later [`with_source`](Self::with_source) is ignored. + /// + /// # Arguments + /// + /// * `message` - The message to show in place of each violation's. + #[must_use] + pub fn redacted(mut self, message: &str) -> Self { + for violation in &mut self.violations { + violation.message = message.to_string(); + violation.alternatives.clear(); + } + self.redacted = true; + self.source = None; + self.labels.clear(); + self.help = None; + self + } + + /// What was validated. + #[must_use] + pub fn subject(&self) -> &str { + &self.subject + } + + /// Where the validated value came from. + #[must_use] + pub fn origin(&self) -> JsonOrigin { + self.origin + } + + /// Whether the error was [`redacted`](Self::redacted) because the value holds secure data. + #[must_use] + pub fn is_redacted(&self) -> bool { + self.redacted + } + + /// The violations found. + #[must_use] + pub fn violations(&self) -> &[SchemaViolation] { + &self.violations + } + + /// Whether the report can show the violations in their source text. + #[must_use] + pub fn has_source(&self) -> bool { + self.source.is_some() + } +} + +impl Display for SchemaValidationError { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + write!(f, "{}: {}", t!("dscerror.schema"), self.subject)?; + for (index, violation) in self.violations.iter().enumerate() { + write!(f, "{} {}", if index == 0 { ":" } else { ";" }, violation.message)?; + if !violation.instance_path.is_empty() { + write!(f, " {}", t!("diagnostics.schema.at", path = violation.instance_path))?; + } + } + Ok(()) + } +} + +impl std::error::Error for SchemaValidationError {} + +impl Diagnostic for SchemaValidationError { + fn help<'a>(&'a self) -> Option> { + self.help.as_ref().map(|help| Box::new(help) as Box) + } + + fn source_code(&self) -> Option<&dyn SourceCode> { + self.source.as_ref().map(|source| source as &dyn SourceCode) + } + + fn labels(&self) -> Option + '_>> { + if self.labels.is_empty() { + None + } else { + Some(Box::new(self.labels.iter().cloned())) + } + } +} + +/// The labels that show a violation in the source text. +/// +/// # Arguments +/// +/// * `violation` - The violation. +/// * `map` - The source map of the text. +/// * `base` - The pointer the violation's instance path is relative to. +fn labels_for(violation: &SchemaViolation, map: &SourceMap, base: &str) -> Vec { + let pointer = format!("{base}{}", violation.instance_path); + match &violation.detail { + SchemaViolationDetail::UnexpectedProperties(unexpected) => { + unexpected.iter().filter_map(|property| { + let member = format!("{pointer}/{}", escape(property)); + let span = map.key_span(&member).or_else(|| map.span(&member))?; + Some(label(map, span, t!("diagnostics.schema.unexpectedProperty", property = property).to_string())) + }).collect() + }, + SchemaViolationDetail::MissingProperty(property) => { + let Some((_, span)) = map.nearest_span(&pointer) else { + return Vec::new(); + }; + vec![label(map, span, t!("diagnostics.schema.missingProperty", property = property).to_string())] + }, + SchemaViolationDetail::None => { + if let Some(span) = map.value_span(&pointer) { + return vec![label(map, span, violation.message.clone())]; + } + // The text doesn't spell the value out, so point at the closest enclosing value. + let Some((_, span)) = map.nearest_span(&pointer) else { + return Vec::new(); + }; + let text = if violation.instance_path.is_empty() { + violation.message.clone() + } else { + format!("{} {}", violation.message, t!("diagnostics.schema.at", path = violation.instance_path)) + }; + vec![label(map, span, text)] + }, + } +} + +/// A label over the first line of a span. +fn label(map: &SourceMap, span: std::ops::Range, text: String) -> LabeledSpan { + let first_line_end = map.text()[span.clone()].find('\n').map_or(span.end, |offset| span.start + offset); + let span = span.start..first_line_end.max(span.start); + LabeledSpan::new(Some(text), span.start, span.len()) +} + +/// Escape a member name as a JSON pointer reference token. +fn escape(key: &str) -> String { + key.replace('~', "~0").replace('/', "~1") +} + +/// The property names the schema declares at the object schema a keyword belongs to. +/// +/// # Arguments +/// +/// * `schema` - The root schema. +/// * `schema_path` - The pointer to the keyword that failed, like `/additionalProperties`. +fn known_properties(schema: &Value, schema_path: &str) -> Option> { + let parent = &schema_path[..schema_path.rfind('/')?]; + let properties = schema.pointer(&format!("{parent}/properties"))?.as_object()?; + Some(properties.keys().cloned().collect()) +} + +/// The known property that most resembles an unknown one, if any does. +/// +/// # Arguments +/// +/// * `property` - The unknown property. +/// * `known` - The known property names. +fn closest<'a>(property: &str, known: &'a [String]) -> Option<&'a str> { + if let Some(same) = known.iter().find(|candidate| candidate.eq_ignore_ascii_case(property)) { + return Some(same); + } + known.iter() + .map(|candidate| (edit_distance(&property.to_lowercase(), &candidate.to_lowercase()), candidate)) + .filter(|(distance, _)| *distance <= MAX_SUGGESTION_DISTANCE) + .min_by_key(|(distance, _)| *distance) + .map(|(_, candidate)| candidate.as_str()) +} + +/// The Levenshtein distance between two strings. +fn edit_distance(a: &str, b: &str) -> usize { + let a: Vec = a.chars().collect(); + let b: Vec = b.chars().collect(); + let mut previous: Vec = (0..=b.len()).collect(); + let mut current = vec![0; b.len() + 1]; + for (i, &ca) in a.iter().enumerate() { + current[0] = i + 1; + for (j, &cb) in b.iter().enumerate() { + let cost = usize::from(ca != cb); + current[j + 1] = (previous[j + 1] + 1).min(current[j] + 1).min(previous[j] + cost); + } + std::mem::swap(&mut previous, &mut current); + } + previous[b.len()] +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::dscresources::dscresource::validate_json_all; + use serde_json::json; + + fn echo_schema() -> Value { + json!({ + "type": "object", + "additionalProperties": false, + "required": ["output"], + "properties": { + "output": { "type": ["string", "object"] }, + "verbose": { "type": "boolean" } + } + }) + } + + fn error_for(text: &str) -> SchemaValidationError { + let schema = echo_schema(); + let value: Value = serde_yaml::from_str(text).unwrap(); + let violations = validate_json_all("test", &schema, &value).unwrap(); + SchemaValidationError::new("input for 'Test/Echo'", violations) + .with_schema(&schema) + .with_source(text, "input", "") + } + + #[test] + fn unknown_property_is_labeled_at_its_key_with_a_suggestion() { + let text = "{\n \"otput\": \"hi\",\n \"output\": \"x\"\n}"; + let error = error_for(text); + assert!(error.has_source()); + let labels: Vec = error.labels().unwrap().collect(); + assert_eq!(labels.len(), 1); + assert_eq!(&text[labels[0].offset()..labels[0].offset() + labels[0].len()], "\"otput\""); + assert_eq!(labels[0].label().unwrap(), "'otput' isn't a known property"); + assert_eq!(error.help().unwrap().to_string(), "Did you mean 'output' instead of 'otput'?"); + assert_eq!(error.to_string(), "Schema: input for 'Test/Echo': Additional properties are not allowed ('otput' was unexpected)"); + } + + #[test] + fn missing_property_is_labeled_at_the_enclosing_object() { + let text = "verbose: true\n"; + let error = error_for(text); + let labels: Vec = error.labels().unwrap().collect(); + assert_eq!(labels[0].label().unwrap(), "missing required property 'output'"); + assert_eq!(labels[0].offset(), 0); + assert!(error.help().is_none()); + } + + #[test] + fn wrong_type_is_labeled_at_the_value_and_named_in_the_message() { + let text = "output: hi\nverbose: nope\n"; + let error = error_for(text); + let labels: Vec = error.labels().unwrap().collect(); + assert_eq!(&text[labels[0].offset()..labels[0].offset() + labels[0].len()], "nope"); + assert!(error.to_string().ends_with("is not of type \"boolean\" at '/verbose'")); + } + + #[test] + fn base_pointer_anchors_violations_inside_a_document() { + let document = "resources:\n- name: a\n type: Test/Echo\n properties:\n otput: hi\n"; + let schema = echo_schema(); + let properties = json!({"otput": "hi"}); + let violations = validate_json_all("test", &schema, &properties).unwrap(); + let error = SchemaValidationError::new("input for 'Test/Echo'", violations) + .with_source(document, "demo.dsc.yaml", "/resources/0/properties"); + let labels: Vec = error.labels().unwrap().collect(); + let unknown = labels.iter().find(|label| label.label() == Some("'otput' isn't a known property")).unwrap(); + assert_eq!(&document[unknown.offset()..unknown.offset() + unknown.len()], "otput"); + } + + #[test] + fn redaction_drops_source_and_messages() { + let error = error_for("output: hi\nverbose: secret\n").redacted("redacted"); + assert!(error.is_redacted()); + assert!(!error.has_source()); + assert!(error.labels().is_none()); + assert!(error.help().is_none()); + assert_eq!(error.to_string(), "Schema: input for 'Test/Echo': redacted at '/verbose'"); + } + + #[test] + fn redaction_survives_attaching_source_again() { + let document = "resources:\n- name: a\n type: Test/Echo\n properties:\n output: hi\n verbose: secret\n"; + let error = error_for("output: hi\nverbose: secret\n") + .redacted("redacted") + .with_source(document, "demo.dsc.yaml", "/resources/0/properties"); + assert!(error.is_redacted()); + assert!(!error.has_source()); + assert!(error.labels().is_none()); + assert!(!format!("{error:?}").contains("secret")); + } + + #[test] + fn origin_defaults_to_input() { + let error = error_for("verbose: true\n"); + assert_eq!(error.origin(), JsonOrigin::Input); + assert_eq!(error.with_origin(JsonOrigin::Output).origin(), JsonOrigin::Output); + } + + #[test] + fn lists_known_properties_without_a_close_match() { + let error = error_for("output: hi\ncompletelyDifferent: 1\n"); + assert_eq!(error.help().unwrap().to_string(), "Known properties: output, verbose"); + } + + #[test] + fn edit_distance_counts_edits() { + assert_eq!(edit_distance("output", "otput"), 1); + assert_eq!(edit_distance("keyPath", "keypath"), 1); + assert_eq!(edit_distance("", "abc"), 3); + assert_eq!(edit_distance("same", "same"), 0); + } +} diff --git a/lib/dsc-lib/src/diagnostics/source_map.rs b/lib/dsc-lib/src/diagnostics/source_map.rs new file mode 100644 index 000000000..ac7e27929 --- /dev/null +++ b/lib/dsc-lib/src/diagnostics/source_map.rs @@ -0,0 +1,616 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +//! Locate JSON pointers in the text of a JSON or YAML document. + +use std::{collections::HashMap, ops::Range}; + +/// The spans of one location in a document. +#[derive(Clone, Debug, PartialEq, Eq)] +struct Entry { + /// The span of the member name, when the value is a member of an object. + key: Option>, + /// The span of the value. + value: Range, +} + +/// The text of a document together with the byte span of every location in it, indexed by JSON +/// pointer (RFC 6901). +/// +/// A source map lets a diagnostic point at the exact place in a document that a pointer +/// identifies, both as a byte range for rendering a snippet and as a line and column. +#[derive(Clone, Debug)] +pub struct SourceMap { + text: String, + entries: HashMap, + line_starts: Vec, +} + +impl SourceMap { + /// Build a source map from document text. + /// + /// Text that is valid JSON is mapped with the JSON grammar; anything else is mapped as YAML. + /// + /// # Arguments + /// + /// * `text` - The document text. + /// + /// # Returns + /// + /// The source map, or `None` when the text is neither valid JSON nor valid YAML. + #[must_use] + pub fn parse(text: &str) -> Option { + let entries = if serde_json::from_str::(text).is_ok() { + json::scan(text)? + } else { + yaml::scan(text)? + }; + Some(Self { + text: text.to_string(), + entries, + line_starts: line_starts(text), + }) + } + + /// The document text. + #[must_use] + pub fn text(&self) -> &str { + &self.text + } + + /// Whether the document has a value at the pointer. + /// + /// # Arguments + /// + /// * `pointer` - The JSON pointer; the empty pointer is the document root. + #[must_use] + pub fn contains(&self, pointer: &str) -> bool { + self.entries.contains_key(pointer) + } + + /// The span of the value at the pointer. + /// + /// # Arguments + /// + /// * `pointer` - The JSON pointer; the empty pointer is the document root. + #[must_use] + pub fn value_span(&self, pointer: &str) -> Option> { + self.entries.get(pointer).map(|entry| entry.value.clone()) + } + + /// The span of the member name of the value at the pointer, when it is a member of an object. + /// + /// # Arguments + /// + /// * `pointer` - The JSON pointer. + #[must_use] + pub fn key_span(&self, pointer: &str) -> Option> { + self.entries.get(pointer).and_then(|entry| entry.key.clone()) + } + + /// The span that best identifies the pointer: the member name when the value is a member of + /// an object, otherwise the value. + /// + /// # Arguments + /// + /// * `pointer` - The JSON pointer. + #[must_use] + pub fn span(&self, pointer: &str) -> Option> { + self.entries.get(pointer).map(|entry| entry.key.clone().unwrap_or_else(|| entry.value.clone())) + } + + /// The span of the nearest location that exists in the document, walking up from the pointer + /// to its ancestors. + /// + /// Useful for a pointer into a value the document doesn't spell out, like a property that is + /// missing or a character offset inside a string. + /// + /// # Arguments + /// + /// * `pointer` - The JSON pointer. + /// + /// # Returns + /// + /// The pointer that was found and its span. + #[must_use] + pub fn nearest_span(&self, pointer: &str) -> Option<(String, Range)> { + let mut current = pointer; + loop { + if let Some(span) = self.span(current) { + return Some((current.to_string(), span)); + } + let parent_end = current.rfind('/')?; + current = ¤t[..parent_end]; + } + } + + /// A map of the same document with the values at the given pointers replaced by a + /// placeholder, for text that is shown to a reader but holds values it must not show. + /// + /// The placeholder is written as a JSON string literal, which both grammars accept in place + /// of any value, so every other pointer in the document keeps its location. A pointer below + /// another listed pointer is covered by it. + /// + /// # Arguments + /// + /// * `pointers` - The JSON pointers of the values to replace. + /// * `placeholder` - The text shown in place of each value. + /// + /// # Returns + /// + /// The redacted map, or `None` when a pointer has no value in the document or the redacted + /// text can't be mapped, so the caller knows the text can't be shown safely. + #[must_use] + pub fn redacted(&self, pointers: &[String], placeholder: &str) -> Option { + let mut sorted: Vec<&str> = pointers.iter().map(String::as_str).collect(); + sorted.sort_unstable(); + sorted.dedup(); + let mut spans: Vec> = Vec::new(); + for pointer in &sorted { + let covered = sorted.iter().any(|other| { + other != pointer && pointer.starts_with(other) && pointer[other.len()..].starts_with('/') + }); + if !covered { + spans.push(self.value_span(pointer)?); + } + } + if spans.is_empty() { + return Some(self.clone()); + } + let replacement = serde_json::to_string(placeholder).ok()?; + spans.sort_by_key(|span| std::cmp::Reverse(span.start)); + let mut text = self.text.clone(); + for span in spans { + text.replace_range(span, &replacement); + } + Self::parse(&text) + } + + /// The line and column of a byte offset, both starting at 1. Columns count characters. + /// + /// # Arguments + /// + /// * `offset` - The byte offset into the text. + #[must_use] + pub fn location(&self, offset: usize) -> (usize, usize) { + let offset = offset.min(self.text.len()); + let line_index = match self.line_starts.binary_search(&offset) { + Ok(index) => index, + Err(index) => index.saturating_sub(1), + }; + let line_start = self.line_starts[line_index]; + let column = self.text[line_start..offset].chars().count() + 1; + (line_index + 1, column) + } +} + +/// The byte offset where each line starts. +fn line_starts(text: &str) -> Vec { + let mut starts = vec![0]; + for (index, byte) in text.bytes().enumerate() { + if byte == b'\n' { + starts.push(index + 1); + } + } + starts +} + +/// Escape a member name as a JSON pointer reference token. +fn escape(key: &str) -> String { + key.replace('~', "~0").replace('/', "~1") +} + +mod json { + use super::{Entry, escape}; + use std::{collections::HashMap, ops::Range}; + + /// Record the span of every value in valid JSON text. + pub(super) fn scan(text: &str) -> Option> { + let mut scanner = Scanner { text, bytes: text.as_bytes(), pos: 0, entries: HashMap::new() }; + scanner.value("", None)?; + Some(scanner.entries) + } + + struct Scanner<'a> { + text: &'a str, + bytes: &'a [u8], + pos: usize, + entries: HashMap, + } + + impl Scanner<'_> { + fn skip_whitespace(&mut self) { + while self.pos < self.bytes.len() && matches!(self.bytes[self.pos], b' ' | b'\t' | b'\n' | b'\r') { + self.pos += 1; + } + } + + fn expect(&mut self, byte: u8) -> Option<()> { + self.skip_whitespace(); + if self.bytes.get(self.pos) == Some(&byte) { + self.pos += 1; + Some(()) + } else { + None + } + } + + fn value(&mut self, pointer: &str, key: Option>) -> Option> { + self.skip_whitespace(); + let start = self.pos; + let span = match self.bytes.get(self.pos)? { + b'{' => self.object(pointer)?, + b'[' => self.array(pointer)?, + b'"' => self.string()?, + _ => { + while self.pos < self.bytes.len() && !matches!(self.bytes[self.pos], b',' | b'}' | b']' | b' ' | b'\t' | b'\n' | b'\r') { + self.pos += 1; + } + start..self.pos + }, + }; + self.entries.insert(pointer.to_string(), Entry { key, value: span.clone() }); + Some(span) + } + + fn object(&mut self, pointer: &str) -> Option> { + let start = self.pos; + self.pos += 1; + loop { + self.skip_whitespace(); + match self.bytes.get(self.pos)? { + b'}' => { + self.pos += 1; + return Some(start..self.pos); + }, + b',' => { + self.pos += 1; + }, + b'"' => { + let key_span = self.string()?; + let key: String = serde_json::from_str(&self.text[key_span.clone()]).ok()?; + self.expect(b':')?; + self.value(&format!("{pointer}/{}", escape(&key)), Some(key_span))?; + }, + _ => return None, + } + } + } + + fn array(&mut self, pointer: &str) -> Option> { + let start = self.pos; + self.pos += 1; + let mut index = 0; + loop { + self.skip_whitespace(); + match self.bytes.get(self.pos)? { + b']' => { + self.pos += 1; + return Some(start..self.pos); + }, + b',' => { + self.pos += 1; + }, + _ => { + self.value(&format!("{pointer}/{index}"), None)?; + index += 1; + }, + } + } + } + + fn string(&mut self) -> Option> { + let start = self.pos; + self.pos += 1; + while let Some(byte) = self.bytes.get(self.pos) { + match byte { + b'\\' => self.pos += 2, + b'"' => { + self.pos += 1; + return Some(start..self.pos); + }, + _ => self.pos += 1, + } + } + None + } + } +} + +mod yaml { + use super::{Entry, escape, line_starts}; + use saphyr_parser::{Event, Marker, Parser, Span}; + use std::{collections::HashMap, ops::Range}; + + /// A container whose members are being read. + enum Frame { + Mapping { + pointer: String, + key: Option>, + /// The member name the next value belongs to: `None` while expecting a key, and + /// `Some(None)` for a key that isn't a scalar and so can't be addressed by a pointer. + pending: Option)>>, + start: usize, + end: usize, + }, + Sequence { + pointer: String, + key: Option>, + index: usize, + start: usize, + end: usize, + }, + } + + /// Record the span of every node in the first document of YAML text. + pub(super) fn scan(text: &str) -> Option> { + let offsets = Offsets::new(text); + let mut entries = HashMap::new(); + let mut frames: Vec = Vec::new(); + // Nesting depth inside a key that is a collection, whose events are skipped. + let mut skipping = 0usize; + let mut root_done = false; + + for item in Parser::new_from_str(text) { + let (event, span) = item.ok()?; + let range = offsets.range(&span); + match event { + Event::StreamStart | Event::StreamEnd | Event::Nothing => {}, + Event::DocumentStart(_) => { + if root_done { + break; + } + }, + Event::DocumentEnd => root_done = true, + Event::Scalar(..) | Event::Alias(..) => { + if skipping > 0 { + continue; + } + let key_text = match &event { + Event::Scalar(value, ..) => Some(value.to_string()), + _ => None, + }; + match frames.last_mut() { + Some(Frame::Mapping { pending: pending @ None, .. }) => { + *pending = Some(key_text.map(|value| (value, range))); + }, + Some(frame) => { + let (pointer, key) = child(frame, range.end); + if let Some(pointer) = pointer { + entries.insert(pointer, Entry { key, value: range }); + } + }, + None => { + entries.insert(String::new(), Entry { key: None, value: range }); + }, + } + }, + Event::SequenceStart(..) | Event::MappingStart(..) => { + if skipping > 0 { + skipping += 1; + continue; + } + if let Some(Frame::Mapping { pending: pending @ None, .. }) = frames.last_mut() { + *pending = Some(None); + skipping = 1; + continue; + } + let (pointer, key) = match frames.last_mut() { + Some(frame) => child(frame, range.end), + None => (Some(String::new()), None), + }; + let Some(pointer) = pointer else { + skipping = 1; + continue; + }; + frames.push(if matches!(event, Event::SequenceStart(..)) { + Frame::Sequence { pointer, key, index: 0, start: range.start, end: range.end } + } else { + Frame::Mapping { pointer, key, pending: None, start: range.start, end: range.end } + }); + }, + Event::SequenceEnd | Event::MappingEnd => { + if skipping > 0 { + skipping -= 1; + continue; + } + let frame = frames.pop()?; + let (pointer, key, start, end) = match frame { + Frame::Mapping { pointer, key, start, end, .. } | Frame::Sequence { pointer, key, start, end, .. } => (pointer, key, start, end), + }; + // A flow collection's closing bracket has a span of its own; a block + // collection ends with its last member. + let end = if range.end > range.start { range.end.max(end) } else { end.max(start) }; + if let Some(parent) = frames.last_mut() { + extend(parent, end); + } + entries.insert(pointer, Entry { key, value: start..end }); + }, + } + } + Some(entries) + } + + /// Resolve the pointer and member-name span of the next value in a container, and advance + /// the container past it. + fn child(frame: &mut Frame, end: usize) -> (Option, Option>) { + match frame { + Frame::Mapping { pointer, pending, end: frame_end, .. } => { + *frame_end = (*frame_end).max(end); + match pending.take().flatten() { + Some((key, span)) => (Some(format!("{pointer}/{}", escape(&key))), Some(span)), + None => (None, None), + } + }, + Frame::Sequence { pointer, index, end: frame_end, .. } => { + *frame_end = (*frame_end).max(end); + let child = format!("{pointer}/{index}"); + *index += 1; + (Some(child), None) + }, + } + } + + fn extend(frame: &mut Frame, end: usize) { + match frame { + Frame::Mapping { end: frame_end, .. } | Frame::Sequence { end: frame_end, .. } => { + *frame_end = (*frame_end).max(end); + }, + } + } + + /// Converts the parser's line and column markers into byte offsets. + struct Offsets<'a> { + text: &'a str, + line_starts: Vec, + } + + impl<'a> Offsets<'a> { + fn new(text: &'a str) -> Self { + Self { text, line_starts: line_starts(text) } + } + + fn range(&self, span: &Span) -> Range { + let start = self.offset(&span.start); + let end = self.offset(&span.end).max(start); + start..end + } + + /// The byte offset of a marker, whose line starts at 1 and whose column starts at 0 and + /// counts characters. + fn offset(&self, marker: &Marker) -> usize { + let Some(&line_start) = self.line_starts.get(marker.line().saturating_sub(1)) else { + return self.text.len(); + }; + let line = &self.text[line_start..]; + let line = &line[..line.find('\n').unwrap_or(line.len())]; + let column_offset = line.char_indices().nth(marker.col()).map_or(line.len(), |(offset, _)| offset); + line_start + column_offset + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + + fn text_at(map: &SourceMap, span: Range) -> &str { + &map.text()[span] + } + + #[test] + fn json_locates_keys_values_and_nested_items() { + let map = SourceMap::parse(r#"{ + "a": 1, + "b": [true, {"c~/d": "x"}], + "e": {"f": null} +}"#).unwrap(); + assert_eq!(text_at(&map, map.key_span("/a").unwrap()), "\"a\""); + assert_eq!(text_at(&map, map.value_span("/a").unwrap()), "1"); + assert_eq!(text_at(&map, map.value_span("/b/0").unwrap()), "true"); + assert_eq!(text_at(&map, map.key_span("/b/1/c~0~1d").unwrap()), "\"c~/d\""); + assert_eq!(text_at(&map, map.value_span("/b/1/c~0~1d").unwrap()), "\"x\""); + assert_eq!(text_at(&map, map.value_span("/e").unwrap()), "{\"f\": null}"); + assert_eq!(text_at(&map, map.value_span("/b").unwrap()), "[true, {\"c~/d\": \"x\"}]"); + assert_eq!(map.value_span("").unwrap().start, 0); + assert!(map.value_span("/missing").is_none()); + assert_eq!(map.location(map.key_span("/b").unwrap().start), (3, 3)); + } + + #[test] + fn json_handles_escaped_strings() { + let map = SourceMap::parse(r#"{"k\"ey": "va\\lue", "n": -1.5e3}"#).unwrap(); + assert_eq!(text_at(&map, map.value_span("/k\"ey").unwrap()), r#""va\\lue""#); + assert_eq!(text_at(&map, map.value_span("/n").unwrap()), "-1.5e3"); + } + + #[test] + fn yaml_locates_block_mappings_and_sequences() { + let text = "$schema: https://example\nresources:\n- name: first\n type: Test/Echo\n properties:\n output: hi\n- name: second\n type: Test/Echo\n"; + let map = SourceMap::parse(text).unwrap(); + assert_eq!(text_at(&map, map.key_span("/resources/0/properties/output").unwrap()), "output"); + assert_eq!(text_at(&map, map.value_span("/resources/0/properties/output").unwrap()), "hi"); + assert_eq!(text_at(&map, map.value_span("/resources/1/name").unwrap()), "second"); + assert_eq!(text_at(&map, map.key_span("/resources").unwrap()), "resources"); + let item = map.value_span("/resources/0").unwrap(); + assert!(text_at(&map, item.clone()).starts_with("name: first")); + assert!(text_at(&map, item).ends_with("output: hi")); + assert_eq!(map.location(map.key_span("/resources/1/type").unwrap().start), (8, 3)); + } + + #[test] + fn yaml_locates_flow_collections_and_quoted_scalars() { + let text = "a: {b: [1, 2], 'c d': \"q\"}\n"; + let map = SourceMap::parse(text).unwrap(); + assert_eq!(text_at(&map, map.value_span("/a/b/1").unwrap()), "2"); + assert_eq!(text_at(&map, map.key_span("/a/c d").unwrap()), "'c d'"); + assert_eq!(text_at(&map, map.value_span("/a/c d").unwrap()), "\"q\""); + assert_eq!(text_at(&map, map.value_span("/a").unwrap()), "{b: [1, 2], 'c d': \"q\"}"); + } + + #[test] + fn yaml_counts_columns_in_characters() { + let text = "héllo: wörld\nnext: 1\n"; + let map = SourceMap::parse(text).unwrap(); + assert_eq!(text_at(&map, map.value_span("/héllo").unwrap()), "wörld"); + assert_eq!(text_at(&map, map.value_span("/next").unwrap()), "1"); + } + + #[test] + fn yaml_skips_complex_keys() { + let text = "? [a, b]\n: value\nplain: 1\n"; + let map = SourceMap::parse(text).unwrap(); + assert_eq!(text_at(&map, map.value_span("/plain").unwrap()), "1"); + } + + #[test] + fn nearest_span_walks_up_to_an_ancestor() { + let map = SourceMap::parse("resources:\n- name: a\n").unwrap(); + let (pointer, span) = map.nearest_span("/resources/0/properties/output").unwrap(); + assert_eq!(pointer, "/resources/0"); + assert_eq!(text_at(&map, span), "name: a"); + assert!(map.nearest_span("/resources/5").is_some()); + } + + #[test] + fn invalid_text_has_no_map() { + assert!(SourceMap::parse("a: [b").is_none()); + } + + #[test] + fn redacted_replaces_values_and_keeps_other_locations() { + let text = "parameters:\n password:\n type: secureString\n defaultValue: hunter2\nresources:\n- name: a\n properties:\n token:\n secureString: letmein\n output: hi\n"; + let map = SourceMap::parse(text).unwrap(); + let pointers = [ + "/parameters/password/defaultValue".to_string(), + "/resources/0/properties/token".to_string(), + "/resources/0/properties/token/secureString".to_string(), + "/resources/0/properties/token".to_string(), + ]; + let redacted = map.redacted(&pointers, "").unwrap(); + assert!(!redacted.text().contains("hunter2") && !redacted.text().contains("letmein"), "{}", redacted.text()); + assert_eq!(text_at(&redacted, redacted.value_span("/parameters/password/defaultValue").unwrap()), "\"\""); + assert_eq!(text_at(&redacted, redacted.value_span("/resources/0/properties/token").unwrap()), "\"\""); + assert_eq!(text_at(&redacted, redacted.value_span("/resources/0/properties/output").unwrap()), "hi"); + assert_eq!(redacted.location(redacted.key_span("/resources/0/properties/output").unwrap().start), (10, 5)); + assert_eq!(text_at(&map, map.value_span("/parameters/password/defaultValue").unwrap()), "hunter2"); + } + + #[test] + fn redacted_json_stays_json() { + let text = r#"{"parameters": {"password": {"type": "secureString", "defaultValue": {"secureString": "hunter2"}}}, "resources": []}"#; + let map = SourceMap::parse(text).unwrap(); + let redacted = map.redacted(&["/parameters/password/defaultValue".to_string()], "").unwrap(); + assert!(!redacted.text().contains("hunter2")); + assert!(serde_json::from_str::(redacted.text()).is_ok()); + assert_eq!(text_at(&redacted, redacted.value_span("/parameters/password/defaultValue").unwrap()), "\"\""); + assert!(redacted.value_span("/resources").is_some()); + } + + #[test] + fn redacted_needs_every_pointer_in_the_document() { + let map = SourceMap::parse("a: 1\n").unwrap(); + assert!(map.redacted(&["/missing".to_string()], "x").is_none()); + assert_eq!(map.redacted(&[], "x").unwrap().text(), "a: 1\n"); + } +} diff --git a/lib/dsc-lib/src/discovery/command_discovery.rs b/lib/dsc-lib/src/discovery/command_discovery.rs index b1a254f5d..56683463f 100644 --- a/lib/dsc-lib/src/discovery/command_discovery.rs +++ b/lib/dsc-lib/src/discovery/command_discovery.rs @@ -13,6 +13,7 @@ use crate::dscresources::command_resource::invoke_command; use crate::dscerror::DscError; use crate::extensions::dscextension::{self, DscExtension, Capability as ExtensionCapability}; use crate::extensions::extension_manifest::ExtensionManifest; +use crate::plugins::plugin_manifest::PluginManifest; use crate::progress::{ProgressBar, ProgressFormat}; use crate::schemas::dsc_repo::DscRepoSchema; use crate::schemas::transforms::idiomaticize_externally_tagged_enum; @@ -53,6 +54,10 @@ pub struct ManifestList { pub adapted_resources: Option>, pub resources: Option>, pub extensions: Option>, + /// The command plug-ins the list defines. Resource discovery doesn't load plug-ins. They are + /// discovered separately, only when a plug-in command is needed, by + /// [`discover_plugins()`](crate::plugins::discovery::discover_plugins). + pub plugins: Option>, } #[allow(clippy::large_enum_variant)] @@ -133,7 +138,7 @@ impl CommandDiscovery { /// - When joining the directories into a new `PATH` fails, which could /// occur due to invalid paths in `resourcePath.directories` or system /// limitations. - fn get_resource_paths() -> Result, DscError> { + pub(crate) fn get_resource_paths() -> Result, DscError> { // Check if the resource paths have already been resolved and cached; // if so, return the cached paths immediately to avoid redundant // computation and potentially mutating PATH repeatedly. @@ -776,7 +781,21 @@ fn insert_resource(resources: &mut DiscoveryCache, resource: Box) -> Result { +/// Evaluates the condition of a manifest. +/// +/// # Arguments +/// +/// * `condition` - The expression from the `condition` property of a manifest, if it defines one. +/// +/// # Returns +/// +/// The result of the expression, or `true` when the manifest doesn't define a condition. +/// +/// # Errors +/// +/// Returns a [`DscError`] when the expression can't be parsed or evaluated, or when its result +/// isn't a boolean. +pub(crate) fn evaluate_condition(condition: Option<&str>) -> Result { if let Some(cond) = condition { let mut statement = Statement::new()?; let result = statement.parse_and_execute(cond, &Context::new())?; diff --git a/lib/dsc-lib/src/dscerror.rs b/lib/dsc-lib/src/dscerror.rs index d97ebe78e..662abe9ad 100644 --- a/lib/dsc-lib/src/dscerror.rs +++ b/lib/dsc-lib/src/dscerror.rs @@ -122,6 +122,12 @@ pub enum DscError { #[error("{t}: {0}", t = t!("dscerror.operation"))] Operation(String), + #[error(transparent)] + PluginCommandName(#[from] crate::plugins::command_name::PluginCommandNameError), + + #[error(transparent)] + Lint(#[from] crate::lint::LintError), + #[error("{t}: {0}", t = t!("dscerror.parser"))] Parser(String), @@ -161,6 +167,10 @@ pub enum DscError { #[error("{t}: {0}", t = t!("dscerror.schema"))] Schema(String), + #[error(transparent)] + #[diagnostic(transparent)] + SchemaValidation(Box), + #[error("{t}: {0}", t = t!("dscerror.schemaNotAvailable"))] SchemaNotAvailable(String), diff --git a/lib/dsc-lib/src/dscresources/command_resource.rs b/lib/dsc-lib/src/dscresources/command_resource.rs index 01cd3f3ae..cc125f7fa 100644 --- a/lib/dsc-lib/src/dscresources/command_resource.rs +++ b/lib/dsc-lib/src/dscresources/command_resource.rs @@ -3,18 +3,17 @@ use clap::ValueEnum; use dsc_lib_security_context::{SecurityContext, get_security_context}; -use jsonschema::Validator; use rust_i18n::t; use serde::Deserialize; use serde_json::{Map, Value}; use std::{collections::HashMap, env, path::{Path, PathBuf}, process::Stdio}; -use crate::{configure::{config_doc::{ExecutionKind, Operation, SecurityContextKind}, config_result::{ResourceGetResult, ResourceTestResult}, context::Context, schema_cache::{RESOURCE_SCHEMAS, get_resource_schema}}, dscresources::resource_manifest::{ExportSchemaKind, ExportSchemaOrFiltering, SchemaArgKind}, settings::fields::ResourcePathResolvedSettings, types::ExitCodesMap, util::canonicalize_which}; +use crate::{diagnostics::{JsonOrigin, SchemaValidationError}, configure::{config_doc::{ExecutionKind, Operation, SecurityContextKind}, config_result::{ResourceGetResult, ResourceTestResult}, context::Context, schema_cache::{RESOURCE_SCHEMAS, get_resource_schema}}, dscresources::resource_manifest::{ExportSchemaKind, ExportSchemaOrFiltering, SchemaArgKind}, settings::fields::ResourcePathResolvedSettings, types::ExitCodesMap, util::canonicalize_which}; use crate::dscerror::DscError; use crate::locked_insert; use crate::resources::{invoke_result::{DeleteResult, DeleteResultKind, GetResult, SetResult, TestResult, ValidateResult, ResolveResult, ExportResult, ResourceGetResponse, ResourceSetResponse, ResourceTestResponse, get_in_desired_state}, resource_invoke::process_expression_arg}; use crate::settings::global as global_settings; use super::{ - dscresource::{get_diff, get_diff_with_schema, redact, DscResource}, + dscresource::{get_diff, get_diff_with_schema, redact, validate_json_all, DscResource}, resource_manifest::{ GetArgKind, SetDeleteArgKind, InputKind, Kind, ReturnKind, SchemaKind } @@ -51,7 +50,7 @@ pub fn invoke_get(resource: &DscResource, filter: &str, context: &Context) -> Re validate_security_context(context.target_resource.as_ref(), &get.require_security_context, &command_resource.type_name, &Operation::Get)?; let args = process_get_args(get.args.as_ref(), filter, command_resource, context)?; if !filter.is_empty() { - verify_json_from_manifest(resource, filter, context)?; + verify_json_from_manifest(resource, filter, context, JsonOrigin::Input)?; command_input = get_command_input(get.input.as_ref(), filter)?; } @@ -59,7 +58,7 @@ pub fn invoke_get(resource: &DscResource, filter: &str, context: &Context) -> Re let (_exit_code, stdout, stderr) = invoke_command(&get.executable, args, command_input.stdin.as_deref(), resource.directory.as_deref(), command_input.env, manifest.exit_codes.as_ref())?; if resource.kind == Kind::Resource { debug!("{}", t!("dscresources.commandResource.verifyOutputUsing", resource = &resource.type_name, executable = &get.executable)); - verify_json_from_manifest(resource, &stdout, context)?; + verify_json_from_manifest(resource, &stdout, context, JsonOrigin::Output)?; } let result: GetResult = if let Ok(group_response) = serde_json::from_str::>(&stdout) { @@ -135,7 +134,7 @@ pub fn invoke_set(resource: &DscResource, desired: &str, skip_test: bool, contex return Err(DscError::NotImplemented("set".to_string())); }; validate_security_context(context.target_resource.as_ref(), &set.require_security_context, &command_resource.type_name, &Operation::Set)?; - verify_json_from_manifest(resource, desired, context)?; + verify_json_from_manifest(resource, desired, context, JsonOrigin::Input)?; // if resource doesn't implement a pre-test, we execute test first to see if a set is needed if !skip_test && set.pre_test != Some(true) { @@ -188,7 +187,7 @@ pub fn invoke_set(resource: &DscResource, desired: &str, skip_test: bool, contex if resource.kind == Kind::Resource { debug!("{}", t!("dscresources.commandResource.setVerifyGet", resource = &resource.type_name, executable = &get.executable)); - verify_json_from_manifest(resource, &stdout, context)?; + verify_json_from_manifest(resource, &stdout, context, JsonOrigin::Output)?; } let pre_state_value: Value = if exit_code == 0 { @@ -238,7 +237,7 @@ pub fn invoke_set(resource: &DscResource, desired: &str, skip_test: bool, contex if resource.kind == Kind::Resource { debug!("{}", t!("dscresources.commandResource.setVerifyOutput", operation = operation_type, resource = &command_resource.type_name, executable = &set.executable)); - verify_json_from_manifest(resource, &stdout, context)?; + verify_json_from_manifest(resource, &stdout, context, JsonOrigin::Output)?; } let actual_value: Value = match serde_json::from_str(&stdout){ @@ -265,7 +264,8 @@ pub fn invoke_set(resource: &DscResource, desired: &str, skip_test: bool, contex }; if resource.kind == Kind::Resource { - verify_json_from_manifest(resource, actual_line, context)?; + debug!("{}", t!("dscresources.commandResource.setVerifyOutput", operation = operation_type, resource = &command_resource.type_name, executable = &set.executable)); + verify_json_from_manifest(resource, actual_line, context, JsonOrigin::Output)?; } let actual_value: Value = serde_json::from_str(actual_line)?; @@ -328,7 +328,7 @@ pub fn invoke_test(resource: &DscResource, expected: &str, context: &Context) -> return invoke_synthetic_test(resource, expected, context); }; - verify_json_from_manifest(resource, expected, context)?; + verify_json_from_manifest(resource, expected, context, JsonOrigin::Input)?; let command_resource = match context.target_resource.as_ref() { Some(r) => r, @@ -352,7 +352,7 @@ pub fn invoke_test(resource: &DscResource, expected: &str, context: &Context) -> Some(ReturnKind::State) => { if command_resource.kind == Kind::Resource { debug!("{}", t!("dscresources.commandResource.testVerifyOutput", resource = &command_resource.type_name, executable = &test.executable)); - verify_json_from_manifest(resource, &stdout, context)?; + verify_json_from_manifest(resource, &stdout, context, JsonOrigin::Output)?; } let actual_value: Value = match serde_json::from_str(&stdout){ @@ -380,7 +380,7 @@ pub fn invoke_test(resource: &DscResource, expected: &str, context: &Context) -> if resource.kind == Kind::Resource { debug!("{}", t!("dscresources.commandResource.testVerifyOutput", resource = &resource.type_name, executable = &test.executable)); - verify_json_from_manifest(resource, actual_value, context)?; + verify_json_from_manifest(resource, actual_value, context, JsonOrigin::Output)?; } let actual_value: Value = serde_json::from_str(actual_value)?; @@ -498,7 +498,7 @@ pub fn invoke_delete(resource: &DscResource, filter: &str, context: &Context) -> return Err(DscError::NotImplemented("delete".to_string())); }; - verify_json_from_manifest(resource, filter, context)?; + verify_json_from_manifest(resource, filter, context, JsonOrigin::Input)?; let command_resource = context.target_resource.as_ref().unwrap_or(resource); validate_security_context(context.target_resource.as_ref(), &delete.require_security_context, &command_resource.type_name, &Operation::Delete)?; @@ -647,17 +647,8 @@ fn verify_with_export_schema(input: &str, resource: &DscResource, context: &Cont } }; let schema = serde_json::from_str(&schema)?; - let compiled_schema = match Validator::new(&schema) { - Ok(schema) => schema, - Err(e) => { - return Err(DscError::Schema(e.to_string())); - }, - }; let json: Value = serde_json::from_str(input)?; - if let Err(err) = compiled_schema.validate(&json) { - return Err(DscError::Schema(err.to_string())); - } - Ok(()) + validate_against_schema(resource, &schema, &json, JsonOrigin::Input) } /// Invoke the export operation on a resource @@ -737,7 +728,7 @@ pub fn invoke_export(resource: &DscResource, input: Option<&str>, context: &Cont }; if resource.kind == Kind::Resource { debug!("{}", t!("dscresources.commandResource.exportVerifyOutput", resource = &resource.type_name, executable = &export.executable)); - verify_json_from_manifest(resource, line, context)?; + verify_json_from_manifest(resource, line, context, JsonOrigin::Output)?; } instances.push(instance); } @@ -1198,7 +1189,7 @@ fn get_command_input(input_kind: Option<&InputKind>, input: &str) -> Result Result<(), DscError> { +fn verify_json_from_manifest(resource: &DscResource, json: &str, context: &Context, origin: JsonOrigin) -> Result<(), DscError> { trace!("{}", t!("dscresources.commandResource.verifyJson", resource = resource.type_name, json = json)); let Some(manifest) = &resource.manifest else { return Err(DscError::MissingManifest(resource.type_name.to_string())); @@ -1223,23 +1214,55 @@ fn verify_json_from_manifest(resource: &DscResource, json: &str, context: &Conte // otherwise, use schema validation let schema = get_schema(resource, context)?; let schema: Value = serde_json::from_str(&schema)?; - let compiled_schema = match Validator::new(&schema) { - Ok(schema) => schema, - Err(e) => { - return Err(DscError::Schema(e.to_string())); - }, - }; let json: Value = serde_json::from_str(json)?; - if let Err(err) = compiled_schema.validate(&json) { - let mut error_message = err.to_string(); - // check if message contains a "secureString" or "secureObject" and redact it - if error_message.contains("\"secureString\"") || error_message.contains("\"secureObject\"") { - error_message = t!("dscresources.commandResource.redactedSchemaErrorMessage").to_string(); - } - return Err(DscError::Schema(error_message)); + validate_against_schema(resource, &schema, &json, origin) +} + +/// Validate JSON against a resource's schema, reporting every violation where it occurs. +/// +/// The error carries the origin, so a host can tell an error about its own input from one about +/// what the resource returned. The text shown with the error has every `secureString` and +/// `secureObject` value replaced, so a violation next to a secure value can't expose it; a +/// violation that quotes a secure value in its message is redacted entirely. +/// +/// # Arguments +/// +/// * `resource` - The resource whose schema it is +/// * `schema` - The schema +/// * `json` - The JSON to validate +/// * `origin` - Where the JSON came from, which the report names +/// +/// # Errors +/// +/// * `DscError::Validation` - The schema failed to compile +/// * `DscError::SchemaValidation` - The JSON violates the schema +fn validate_against_schema(resource: &DscResource, schema: &Value, json: &Value, origin: JsonOrigin) -> Result<(), DscError> { + let violations = validate_json_all(resource.type_name.as_ref(), schema, json)?; + if violations.is_empty() { + return Ok(()); } - Ok(()) + let (subject, name) = match origin { + JsonOrigin::Input => ( + t!("dscresources.commandResource.schemaSubjectInput", resource = resource.type_name).to_string(), + t!("dscresources.commandResource.schemaSourceInput").to_string(), + ), + JsonOrigin::Output => ( + t!("dscresources.commandResource.schemaSubjectOutput", resource = resource.type_name).to_string(), + t!("dscresources.commandResource.schemaSourceOutput").to_string(), + ), + }; + let mut error = SchemaValidationError::new(subject, violations).with_schema(schema).with_origin(origin); + // a message that quotes a "secureString" or "secureObject" value would expose it + let holds_secure_value = error.violations().iter().any(|violation| { + violation.message.contains("\"secureString\"") || violation.message.contains("\"secureObject\"") + }); + if holds_secure_value { + error = error.redacted(&t!("dscresources.commandResource.redactedSchemaErrorMessage")); + } else if let Ok(text) = serde_json::to_string_pretty(&redact(json)) { + error = error.with_source(&text, &name, ""); + } + Err(DscError::SchemaValidation(Box::new(error))) } fn json_to_hashmap(json: &str) -> Result, DscError> { @@ -1470,3 +1493,63 @@ struct Trace { #[serde(rename = "spans")] _spans: Option>>, } + +#[cfg(test)] +mod tests { + use super::*; + use crate::configure::parameters::SECURE_VALUE_REDACTED; + use miette::Diagnostic; + use serde_json::json; + + fn echo() -> DscResource { + let mut resource = DscResource::new(); + resource.type_name = "Test/Echo".parse().unwrap(); + resource + } + + /// The schema validation error inside a `DscError`. + fn schema_error(error: DscError) -> SchemaValidationError { + match error { + DscError::SchemaValidation(error) => *error, + other => panic!("expected a schema validation error, got {other}"), + } + } + + /// The whole text attached to an error. + fn attached_text(error: &SchemaValidationError) -> String { + let source = error.source_code().expect("the error has source text"); + let contents = source.read_span(&(0..1).into(), 0, 1000).unwrap(); + String::from_utf8_lossy(contents.data()).to_string() + } + + #[test] + fn attached_input_has_secure_values_redacted_even_when_they_are_valid() { + let schema = json!({ + "type": "object", + "additionalProperties": false, + "properties": { + "password": { "type": ["string", "object"] }, + "name": { "type": "string" } + } + }); + let input = json!({ "password": { "secureString": "hunter2" }, "nmae": "x" }); + let error = schema_error(validate_against_schema(&echo(), &schema, &input, JsonOrigin::Input).unwrap_err()); + assert_eq!(error.origin(), JsonOrigin::Input); + assert!(!error.is_redacted()); + let text = attached_text(&error); + assert!(!text.contains("hunter2"), "{text}"); + assert!(text.contains(SECURE_VALUE_REDACTED), "{text}"); + assert!(text.contains("nmae"), "{text}"); + } + + #[test] + fn secure_value_quoted_in_a_message_redacts_the_whole_error() { + let schema = json!({ "type": "object", "properties": { "password": { "type": "string" } } }); + let output = json!({ "password": { "secureString": "hunter2" } }); + let error = schema_error(validate_against_schema(&echo(), &schema, &output, JsonOrigin::Output).unwrap_err()); + assert_eq!(error.origin(), JsonOrigin::Output); + assert!(error.is_redacted()); + assert!(!error.has_source()); + assert!(!error.to_string().contains("hunter2"), "{error}"); + } +} diff --git a/lib/dsc-lib/src/dscresources/dscresource.rs b/lib/dsc-lib/src/dscresources/dscresource.rs index bf5323e2e..40050d2c4 100644 --- a/lib/dsc-lib/src/dscresources/dscresource.rs +++ b/lib/dsc-lib/src/dscresources/dscresource.rs @@ -3,6 +3,7 @@ use crate::{ actions::dscaction::DscAction, + diagnostics::SchemaValidationError, configure::{Configurator, config_doc::{Configuration, Resource}, context::{Context, ProcessMode}, parameters::{SECURE_VALUE_REDACTED}, schema_cache::get_resource_schema @@ -17,7 +18,7 @@ use crate::{ util::{contains_secure_value, is_secure_value}, }; use dscerror::DscError; -use jsonschema::Validator; +use jsonschema::{error::ValidationErrorKind, ValidationError, Validator}; use rust_i18n::t; use schemars::JsonSchema; use serde::{Deserialize, Serialize}; @@ -866,7 +867,7 @@ pub fn validate_properties(resource: &dyn ResourceInvoke, properties: &Value, co /// /// # Arguments /// -/// * `source` - The source of the JSON +/// * `source` - The source of the JSON, as it should read in the error message /// * `schema` - The schema to validate against /// * `json` - The JSON to validate /// @@ -876,10 +877,103 @@ pub fn validate_properties(resource: &dyn ResourceInvoke, properties: &Value, co /// /// # Errors /// -/// * `DscError` - The JSON is invalid +/// * `DscError::Validation` - The schema failed to compile +/// * `DscError::SchemaValidation` - The JSON is invalid; the error lists every violation pub fn validate_json(source: &str, schema: &Value, json: &Value) -> Result<(), DscError> { - debug!("{}: {source}", t!("dscresources.dscresource.validatingSchema")); trace!("{}", t!("dscresources.dscresource.json", json = json)); + let violations = validate_json_all(source, schema, json)?; + if violations.is_empty() { + return Ok(()); + } + Err(DscError::SchemaValidation(Box::new(SchemaValidationError::new(source, violations).with_schema(schema)))) +} + +/// Extra information for the keywords whose message alone doesn't locate the problem. +#[derive(Clone, Debug, Default, PartialEq, Eq)] +pub enum SchemaViolationDetail { + /// The message describes the problem fully. + #[default] + None, + /// For `additionalProperties` and `unevaluatedProperties`, the property names that aren't + /// allowed. + UnexpectedProperties(Vec), + /// For `required`, the property that is missing. + MissingProperty(String), +} + +/// A single violation found when validating JSON against a schema. +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct SchemaViolation { + /// JSON pointer (RFC 6901) to the location in the instance that failed validation. + pub instance_path: String, + /// JSON pointer to the keyword in the schema that the instance violated. + pub schema_path: String, + /// The keyword that failed, such as `type` or `anyOf`. + pub keyword: String, + /// The validation error message. + pub message: String, + /// What the keyword found, when the message alone doesn't locate the problem. + pub detail: SchemaViolationDetail, + /// For a failed `anyOf`, or a `oneOf` that no subschema matched, the violations of each + /// subschema in order; empty for every other keyword. + pub alternatives: Vec>, +} + +impl SchemaViolation { + /// Convert a validation error, including the violations of each `anyOf` or `oneOf` + /// subschema. + /// + /// # Arguments + /// + /// * `error` - The validation error. + fn from_error(error: &ValidationError<'_>) -> Self { + let alternatives = match error.kind() { + ValidationErrorKind::AnyOf { context } | ValidationErrorKind::OneOfNotValid { context } => { + context.iter().map(|branch| branch.iter().map(Self::from_error).collect()).collect() + }, + _ => Vec::new(), + }; + let detail = match error.kind() { + ValidationErrorKind::AdditionalProperties { unexpected } | ValidationErrorKind::UnevaluatedProperties { unexpected } => { + SchemaViolationDetail::UnexpectedProperties(unexpected.clone()) + }, + ValidationErrorKind::Required { property } => { + SchemaViolationDetail::MissingProperty(property.as_str().map_or_else(|| property.to_string(), str::to_string)) + }, + _ => SchemaViolationDetail::None, + }; + Self { + instance_path: error.instance_path().to_string(), + schema_path: error.schema_path().to_string(), + keyword: error.kind().keyword().to_string(), + message: error.to_string(), + detail, + alternatives, + } + } +} + +/// Validate the JSON against the schema and return every violation. +/// +/// Unlike [`validate_json`], which stops at the first error, this function collects all +/// violations so callers can report each one with its location. The JSON isn't traced because +/// it can hold secure values. +/// +/// # Arguments +/// +/// * `source` - The source of the JSON, used for logging +/// * `schema` - The schema to validate against +/// * `json` - The JSON to validate +/// +/// # Returns +/// +/// Every violation found; an empty list means the JSON is valid. +/// +/// # Errors +/// +/// * `DscError::Validation` - The schema failed to compile +pub fn validate_json_all(source: &str, schema: &Value, json: &Value) -> Result, DscError> { + debug!("{}: {source}", t!("dscresources.dscresource.validatingSchema")); trace!("{}", t!("dscresources.dscresource.schema", schema = schema)); let compiled_schema = match Validator::new(schema) { Ok(compiled_schema) => compiled_schema, @@ -888,11 +982,7 @@ pub fn validate_json(source: &str, schema: &Value, json: &Value) -> Result<(), D } }; - if let Err(err) = compiled_schema.validate(json) { - return Err(DscError::Validation(format!("{}: '{source}' {err}", t!("dscresources.dscresource.validationFailed")))); - } - - Ok(()) + Ok(compiled_schema.iter_errors(json).map(|error| SchemaViolation::from_error(&error)).collect()) } /// Compares two arrays independent of order diff --git a/lib/dsc-lib/src/functions/mod.rs b/lib/dsc-lib/src/functions/mod.rs index 36f51196a..f22344902 100644 --- a/lib/dsc-lib/src/functions/mod.rs +++ b/lib/dsc-lib/src/functions/mod.rs @@ -288,22 +288,7 @@ impl FunctionDispatcher { let metadata = function.get_metadata(); - // check if arg number are valid - let min_args = metadata.min_args; - let max_args = metadata.max_args; - if args.len() < min_args || args.len() > max_args { - if max_args == 0 { - return Err(DscError::Parser(t!("functions.noArgsAccepted", name = name).to_string())); - } - else if min_args == max_args { - return Err(DscError::Parser(t!("functions.invalidArgCount", name = name, count = min_args).to_string())); - } - else if max_args == usize::MAX { - return Err(DscError::Parser(t!("functions.minArgsRequired", name = name, count = min_args).to_string())); - } - - return Err(DscError::Parser(t!("functions.argCountRequired", name = name, min = min_args, max = max_args).to_string())); - } + Self::check_arg_count(name, &metadata, args.len())?; for (index, value) in args.iter().enumerate() { if index >= metadata.accepted_arg_ordered_types.len() { @@ -332,6 +317,98 @@ impl FunctionDispatcher { } } + /// Get the metadata of a built-in function. + /// + /// # Arguments + /// + /// * `name` - The name of the function; names are case-sensitive. + /// + /// # Returns + /// + /// The metadata of the function, or `None` when no built-in function has that name. + #[must_use] + pub fn get_metadata(&self, name: &str) -> Option { + self.functions.get(name).map(|function| function.get_metadata()) + } + + /// Validate that a built-in function accepts the given number of arguments without invoking it. + /// + /// # Arguments + /// + /// * `name` - The name of the function; names are case-sensitive. + /// * `count` - The number of arguments that would be passed. + /// + /// # Errors + /// + /// Returns `DscError::Parser` when the function is unknown or the count is outside the range + /// the function accepts. The messages are the same ones [`invoke`](Self::invoke) raises. + pub fn validate_arg_count(&self, name: &str, count: usize) -> Result<(), DscError> { + let Some(metadata) = self.get_metadata(name) else { + return Err(DscError::Parser(t!("functions.unknownFunction", name = name).to_string())); + }; + Self::check_arg_count(name, &metadata, count) + } + + /// Validate literal arguments against the kinds a built-in function accepts without invoking it. + /// + /// # Arguments + /// + /// * `name` - The name of the function; names are case-sensitive. + /// * `args` - One entry per argument in order. `None` marks an argument whose value isn't + /// known statically, such as a nested expression; those entries are skipped. + /// + /// # Errors + /// + /// Returns `DscError::Parser` when the function is unknown or a literal argument has a kind + /// the function doesn't accept at that position. The messages are the same ones + /// [`invoke`](Self::invoke) raises. + pub fn validate_literal_args(&self, name: &str, args: &[Option<&Value>]) -> Result<(), DscError> { + let Some(metadata) = self.get_metadata(name) else { + return Err(DscError::Parser(t!("functions.unknownFunction", name = name).to_string())); + }; + for (index, arg) in args.iter().enumerate() { + let Some(value) = arg else { + continue; + }; + if let Some(expected_types) = metadata.accepted_arg_ordered_types.get(index) { + Self::check_arg_against_expected_types(name, value, expected_types)?; + } else if let Some(remaining_types) = &metadata.remaining_arg_accepted_types { + Self::check_arg_against_expected_types(name, value, remaining_types)?; + } + } + Ok(()) + } + + /// Check that an argument count is within the range a function accepts. + /// + /// # Arguments + /// + /// * `name` - The name of the function, used in the error message. + /// * `metadata` - The metadata of the function. + /// * `count` - The number of arguments. + /// + /// # Errors + /// + /// Returns `DscError::Parser` describing the accepted range when `count` is outside it. + fn check_arg_count(name: &str, metadata: &FunctionMetadata, count: usize) -> Result<(), DscError> { + let min_args = metadata.min_args; + let max_args = metadata.max_args; + if count < min_args || count > max_args { + if max_args == 0 { + return Err(DscError::Parser(t!("functions.noArgsAccepted", name = name).to_string())); + } + else if min_args == max_args { + return Err(DscError::Parser(t!("functions.invalidArgCount", name = name, count = min_args).to_string())); + } + else if max_args == usize::MAX { + return Err(DscError::Parser(t!("functions.minArgsRequired", name = name, count = min_args).to_string())); + } + + return Err(DscError::Parser(t!("functions.argCountRequired", name = name, min = min_args, max = max_args).to_string())); + } + Ok(()) + } + fn check_arg_against_expected_types(name: &str, arg: &Value, expected_types: &[FunctionArgKind]) -> Result<(), DscError> { let is_lambda = arg.as_str().is_some_and(|s| s.starts_with("__lambda_")); diff --git a/lib/dsc-lib/src/lib.rs b/lib/dsc-lib/src/lib.rs index 20ce69d02..9fe250796 100644 --- a/lib/dsc-lib/src/lib.rs +++ b/lib/dsc-lib/src/lib.rs @@ -12,12 +12,15 @@ use rust_i18n::i18n; pub mod actions; pub mod configure; +pub mod diagnostics; pub mod discovery; pub mod dscerror; pub mod dscresources; pub mod extensions; pub mod functions; +pub mod lint; pub mod parser; +pub mod plugins; pub mod progress; pub mod resources; pub mod schemas; diff --git a/lib/dsc-lib/src/lint/context.rs b/lib/dsc-lib/src/lint/context.rs new file mode 100644 index 000000000..a018aea28 --- /dev/null +++ b/lib/dsc-lib/src/lint/context.rs @@ -0,0 +1,226 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +use std::collections::{HashMap, HashSet}; + +use serde_json::Value; + +use crate::{ + configure::config_doc::{Configuration, Parameter, Resource}, + dscerror::DscError, + dscresources::dscresource::DscResource, + functions::FunctionDispatcher, + schemas::dsc_repo::DscRepoSchema, +}; + +use super::{expression_index::ExpressionIndex, options::LintOptions, pointer::JsonPointer}; + +/// A resource declaration and where it sits in the document. +pub struct ResourceSite<'a> { + /// The pointer to the resource declaration. + pub pointer: JsonPointer, + /// The resource declaration. + pub resource: &'a Resource, + /// The index of the resource that declares this one in its nested `resources`, if any. + pub parent: Option, +} + +/// The built-in group resources. They receive an embedded configuration document as their +/// properties and evaluate its expressions themselves. +const GROUP_RESOURCE_TYPES: [&str; 2] = ["Microsoft.DSC/Group", "Microsoft.DSC/Assertion"]; + +impl ResourceSite<'_> { + /// Whether the resource's properties are an embedded configuration document, which DSC + /// passes to the resource without evaluating its expressions. + /// + /// Without discovery the resource kind isn't known, so this recognizes the built-in group + /// resources and any properties that declare a configuration document `$schema`. + #[must_use] + pub fn has_embedded_configuration(&self) -> bool { + let type_name: &str = self.resource.resource_type.as_ref(); + if GROUP_RESOURCE_TYPES.iter().any(|group| group.eq_ignore_ascii_case(type_name)) { + return true; + } + self.resource.properties.as_ref() + .and_then(|properties| properties.get("$schema")) + .and_then(Value::as_str) + .is_some_and(|uri| Configuration::is_recognized_schema_uri(&uri.to_string())) + } +} + +/// The outcome of discovering the resource type a declaration uses. +pub struct ResolvedResource { + /// The installed resource that satisfies the declaration, if one was found. + pub found: Option, + /// Whether any version of the resource type is installed, regardless of `requireVersion`. + pub type_exists: bool, +} + +/// Everything a rule can inspect. +pub struct LintContext<'a> { + /// The document as parsed, for checks that depend on how it was written. + pub raw: &'a Value, + /// The document as a typed configuration. + pub config: &'a Configuration, + /// Every resource declaration, depth first, with nested resources after their parent. + pub resources: Vec>, + /// Every expression in the document, parsed once. + pub expressions: ExpressionIndex, + /// The options of the lint run. + pub options: &'a LintOptions, + /// The built-in functions, for checking calls without evaluating them. + pub functions: &'a FunctionDispatcher, + /// The parameter values supplied for the run, when any were. + pub supplied_parameters: Option>, + /// The discovered resource for each entry of `resources`, when discovery ran. + pub discovered: Option>, +} + +impl LintContext<'_> { + /// Whether a string is an expression, as opposed to a plain or escaped (`[[`) string. + /// + /// # Arguments + /// + /// * `text` - The string to classify. + #[must_use] + pub fn is_expression(text: &str) -> bool { + text.starts_with('[') && !text.starts_with("[[") + } + + /// Group the resources that share a parent, in document order. + /// + /// # Returns + /// + /// The indexes into `resources` for each group, top-level resources first. + #[must_use] + pub fn sibling_groups(&self) -> Vec> { + let mut groups: Vec<(Option, Vec)> = Vec::new(); + for (index, site) in self.resources.iter().enumerate() { + match groups.iter_mut().find(|(parent, _)| *parent == site.parent) { + Some((_, members)) => members.push(index), + None => groups.push((site.parent, vec![index])), + } + } + groups.into_iter().map(|(_, members)| members).collect() + } + + /// The raw value of a resource declaration. + /// + /// # Arguments + /// + /// * `index` - The index into `resources`. + #[must_use] + pub fn raw_resource(&self, index: usize) -> Option<&Value> { + self.resources.get(index).and_then(|site| site.pointer.resolve(self.raw)) + } + + /// The declared parameter with the given name. + /// + /// # Arguments + /// + /// * `name` - The parameter name. + #[must_use] + pub fn declared_parameter(&self, name: &str) -> Option<&Parameter> { + self.config.parameters.as_ref().and_then(|parameters| parameters.get(name)) + } + + /// The fully qualified names (`namespace.member`) of the user-defined functions. + #[must_use] + pub fn user_function_names(&self) -> HashSet { + let mut names = HashSet::new(); + if let Some(functions) = &self.config.functions { + for function in functions { + for member in function.members.keys() { + names.insert(format!("{}.{member}", function.namespace)); + } + } + } + names + } + + /// The text of an error without the error-kind prefix, for use in a diagnostic message. + /// + /// # Arguments + /// + /// * `error` - The error to describe. + #[must_use] + pub fn error_message(error: &DscError) -> String { + match error { + DscError::Parser(message) | DscError::Validation(message) => message.clone(), + other => other.to_string(), + } + } +} + +/// Flatten the resources of a configuration into sites, depth first. +/// +/// # Arguments +/// +/// * `config` - The configuration. +#[must_use] +pub fn collect_resource_sites(config: &Configuration) -> Vec> { + let mut sites = Vec::new(); + collect_sites(&config.resources, &JsonPointer::root().push("resources"), None, &mut sites); + sites +} + +/// Append the sites for a list of resources and their nested resources. +/// +/// # Arguments +/// +/// * `resources` - The resources to add. +/// * `pointer` - The pointer to the `resources` array. +/// * `parent` - The index of the resource that declares this list, if any. +/// * `sites` - Where sites are appended. +fn collect_sites<'a>(resources: &'a [Resource], pointer: &JsonPointer, parent: Option, sites: &mut Vec>) { + for (index, resource) in resources.iter().enumerate() { + let resource_pointer = pointer.push_index(index); + let site_index = sites.len(); + sites.push(ResourceSite { + pointer: resource_pointer.clone(), + resource, + parent, + }); + if let Some(children) = &resource.resources { + collect_sites(children, &resource_pointer.push("resources"), Some(site_index), sites); + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn collects_nested_resources_depth_first() { + let config: Configuration = serde_yaml::from_str(r" +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +resources: +- name: group + type: Microsoft.DSC/Group + properties: + resources: + - name: inner + type: Microsoft.DSC.Debug/Echo + properties: + output: hi + resources: + - name: nested + type: Microsoft.DSC.Debug/Echo +- name: last + type: Microsoft.DSC.Debug/Echo +").unwrap(); + let sites = collect_resource_sites(&config); + let pointers: Vec<&str> = sites.iter().map(|site| site.pointer.as_str()).collect(); + assert_eq!(pointers, vec!["/resources/0", "/resources/0/resources/0", "/resources/1"]); + assert_eq!(sites[1].parent, Some(0)); + assert_eq!(sites[2].parent, None); + } + + #[test] + fn classifies_expressions() { + assert!(LintContext::is_expression("[parameters('x')]")); + assert!(!LintContext::is_expression("[[literal]")); + assert!(!LintContext::is_expression("plain")); + } +} diff --git a/lib/dsc-lib/src/lint/diagnostic.rs b/lib/dsc-lib/src/lint/diagnostic.rs new file mode 100644 index 000000000..01a20a742 --- /dev/null +++ b/lib/dsc-lib/src/lint/diagnostic.rs @@ -0,0 +1,234 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +use std::fmt::Display; + +use clap::ValueEnum; +use schemars::JsonSchema; +use serde::{Deserialize, Serialize}; + +use crate::schemas::{dsc_repo::DscRepoSchema, transforms::idiomaticize_string_enum}; + +/// The severity of a diagnostic. +#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash, Deserialize, Serialize, JsonSchema, ValueEnum)] +#[serde(rename_all = "camelCase")] +#[schemars(inline, transform = idiomaticize_string_enum)] +pub enum LintLevel { + /// A suggestion that doesn't affect whether the document works. + Info, + /// A problem that is likely to cause unexpected behavior. + Warning, + /// A problem that prevents the document from working. + Error, +} + +impl Display for LintLevel { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + let text = match self { + LintLevel::Info => "info", + LintLevel::Warning => "warning", + LintLevel::Error => "error", + }; + write!(f, "{text}") + } +} + +/// The area of a document that a rule inspects. +#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash, Deserialize, Serialize, JsonSchema)] +#[serde(rename_all = "camelCase")] +#[schemars(inline, transform = idiomaticize_string_enum)] +pub enum LintCategory { + /// The structure of the document and its resources. + Document, + /// Expressions and the functions they call. + Expression, + /// Parameter and variable declarations and their values. + Parameter, + /// Dependencies between resources. + Dependency, + /// Features that DSC has deprecated or doesn't support. + Deprecation, + /// The environment the document targets. + Environment, + /// Checks that need the installed resources. + Resource, +} + +impl Display for LintCategory { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + let text = match self { + LintCategory::Document => "document", + LintCategory::Expression => "expression", + LintCategory::Parameter => "parameter", + LintCategory::Dependency => "dependency", + LintCategory::Deprecation => "deprecation", + LintCategory::Environment => "environment", + LintCategory::Resource => "resource", + }; + write!(f, "{text}") + } +} + +/// A position in the source text. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Deserialize, Serialize, JsonSchema)] +#[serde(deny_unknown_fields)] +#[schemars(inline)] +pub struct SourceLocation { + /// The line number, starting at 1. + pub line: usize, + /// The column number, starting at 1. + pub column: usize, +} + +/// A problem found in a document. +#[derive(Debug, Clone, PartialEq, Eq, Deserialize, Serialize, JsonSchema)] +#[serde(deny_unknown_fields)] +#[schemars(inline)] +pub struct LintDiagnostic { + /// The severity of the problem. + pub level: LintLevel, + /// The identifier of the rule that reported the problem. + pub rule: String, + /// The category of the rule that reported the problem. + pub category: LintCategory, + /// A description of the problem. + pub message: String, + /// JSON pointer (RFC 6901) to the location of the problem in the document. An empty string + /// identifies the document root. + pub path: String, + /// The file the document was read from, when known. + #[serde(skip_serializing_if = "Option::is_none")] + pub file: Option, + /// The position of the problem in the source text, when known. + #[serde(skip_serializing_if = "Option::is_none")] + pub location: Option, +} + +/// Counts of diagnostics by level. +#[derive(Debug, Clone, Copy, Default, PartialEq, Eq, Deserialize, Serialize, JsonSchema)] +#[serde(deny_unknown_fields)] +#[schemars(inline)] +pub struct LintSummary { + /// The number of error-level diagnostics. + pub errors: usize, + /// The number of warning-level diagnostics. + pub warnings: usize, + /// The number of info-level diagnostics. + pub info: usize, +} + +/// The result of linting one or more documents. +#[derive(Debug, Clone, Default, PartialEq, Eq, Deserialize, Serialize, JsonSchema, DscRepoSchema)] +#[serde(deny_unknown_fields)] +#[dsc_repo_schema(base_name = "result", folder_path = "outputs/lint")] +#[schemars( + transform = LintResult::transform_export_schema_uris, + transform = LintResult::transform_schema_docs +)] +pub struct LintResult { + /// The problems found, ordered by path and rule. + pub diagnostics: Vec, + /// Counts of the problems found by level. + pub summary: LintSummary, + /// Whether no error-level problems were found. + pub valid: bool, + /// Whether resource discovery ran, which enables the rules that inspect installed resources. + pub discovery: bool, +} + +impl LintResult { + /// Build a result from a set of diagnostics, sorting them and computing the summary. + /// + /// # Arguments + /// + /// * `diagnostics` - The diagnostics found. + /// * `discovery` - Whether resource discovery ran. + #[must_use] + pub fn from_diagnostics(mut diagnostics: Vec, discovery: bool) -> Self { + diagnostics.sort_by(|a, b| a.file.cmp(&b.file).then_with(|| a.path.cmp(&b.path)).then_with(|| a.rule.cmp(&b.rule))); + let mut summary = LintSummary::default(); + for diagnostic in &diagnostics { + match diagnostic.level { + LintLevel::Error => summary.errors += 1, + LintLevel::Warning => summary.warnings += 1, + LintLevel::Info => summary.info += 1, + } + } + Self { + valid: summary.errors == 0, + discovery, + summary, + diagnostics, + } + } + + /// Combine another result into this one, for example when linting several files. + /// + /// # Arguments + /// + /// * `other` - The result to merge; its diagnostics are appended and re-sorted. + pub fn merge(&mut self, other: LintResult) { + let discovery = self.discovery && other.discovery; + let mut diagnostics = std::mem::take(&mut self.diagnostics); + diagnostics.extend(other.diagnostics); + *self = Self::from_diagnostics(diagnostics, discovery); + } + + /// Whether any diagnostic is at or above the given level. + /// + /// # Arguments + /// + /// * `level` - The threshold. + #[must_use] + pub fn has_diagnostic_at_least(&self, level: LintLevel) -> bool { + self.diagnostics.iter().any(|diagnostic| diagnostic.level >= level) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + fn diagnostic(level: LintLevel, rule: &str, path: &str) -> LintDiagnostic { + LintDiagnostic { + level, + rule: rule.to_string(), + category: LintCategory::Document, + message: String::new(), + path: path.to_string(), + file: None, + location: None, + } + } + + #[test] + fn from_diagnostics_sorts_and_summarizes() { + let result = LintResult::from_diagnostics(vec![ + diagnostic(LintLevel::Warning, "b", "/resources/1"), + diagnostic(LintLevel::Error, "a", "/resources/0"), + diagnostic(LintLevel::Info, "a", "/resources/1"), + ], true); + assert_eq!(result.summary, LintSummary { errors: 1, warnings: 1, info: 1 }); + assert!(!result.valid); + assert!(result.discovery); + let order: Vec<(&str, &str)> = result.diagnostics.iter().map(|d| (d.path.as_str(), d.rule.as_str())).collect(); + assert_eq!(order, vec![("/resources/0", "a"), ("/resources/1", "a"), ("/resources/1", "b")]); + } + + #[test] + fn merge_combines_results() { + let mut first = LintResult::from_diagnostics(vec![diagnostic(LintLevel::Warning, "a", "")], true); + let second = LintResult::from_diagnostics(vec![diagnostic(LintLevel::Error, "b", "")], false); + first.merge(second); + assert_eq!(first.diagnostics.len(), 2); + assert!(!first.valid); + assert!(!first.discovery); + assert!(first.has_diagnostic_at_least(LintLevel::Error)); + } + + #[test] + fn levels_are_ordered() { + assert!(LintLevel::Error > LintLevel::Warning); + assert!(LintLevel::Warning > LintLevel::Info); + } +} diff --git a/lib/dsc-lib/src/lint/expression_index.rs b/lib/dsc-lib/src/lint/expression_index.rs new file mode 100644 index 000000000..66fcf2715 --- /dev/null +++ b/lib/dsc-lib/src/lint/expression_index.rs @@ -0,0 +1,438 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +use std::collections::HashMap; + +use serde_json::Value; + +use crate::{ + dscerror::DscError, + parser::{expressions::{Accessor, Expression}, functions::{Function, FunctionArg}, ParsedStatement, Statement}, +}; + +use super::{context::ResourceSite, pointer::JsonPointer}; + +/// Where in the document an expression appears, which decides how its functions are resolved. +#[derive(Clone, Debug, PartialEq, Eq)] +pub enum ExpressionScope { + /// Inside the resource with the given index in the context's resource list. + Resource(usize), + /// Inside the default value of the named parameter. + Parameter(String), + /// Inside the named variable. + Variable(String), + /// Inside the named output. + Output(String), + /// Inside the body of a user-defined function, where `parameters()` refers to the + /// function's own parameters. + UserFunction, +} + +/// One expression found in the document. +pub struct ExpressionSite { + /// The pointer to the string that holds the expression. + pub pointer: JsonPointer, + /// The expression text as written. + pub text: String, + /// Where the expression appears. + pub scope: ExpressionScope, + /// The parsed expression, or the parse error. + pub parsed: Result, +} + +/// A function call found inside an expression. +pub struct FunctionCall<'a> { + /// The expression the call appears in. + pub site: &'a ExpressionSite, + /// The call. + pub function: &'a Function, +} + +/// Every expression in a document, parsed once so rules can inspect them without re-parsing. +pub struct ExpressionIndex { + sites: Vec, + by_pointer: HashMap, +} + +impl ExpressionIndex { + /// Find and parse every expression in a document. + /// + /// Strings are walked in the places DSC evaluates expressions: parameter default values, + /// variables, outputs, user-defined function bodies, and for each resource its name, + /// condition, `dependsOn` entries, copy counts, and properties. Parameter default values and + /// variables are only expressions when the whole value is a string; DSC uses object and + /// array values as written. The properties of a resource that holds an embedded + /// configuration are skipped, because that configuration is evaluated on its own. + /// + /// # Arguments + /// + /// * `raw` - The document. + /// * `resources` - The resource sites of the document, in the order the context lists them. + /// * `parser` - The parser to parse expressions with. + #[must_use] + pub fn build(raw: &Value, resources: &[ResourceSite<'_>], parser: &mut Statement) -> Self { + let mut index = Self { + sites: Vec::new(), + by_pointer: HashMap::new(), + }; + let root = JsonPointer::root(); + + if let Some(parameters) = raw.get("parameters").and_then(Value::as_object) { + let base = root.push("parameters"); + for (name, parameter) in parameters { + if let Some(default_value) = parameter.get("defaultValue") { + index.add_string(default_value, &base.push(name).push("defaultValue"), &ExpressionScope::Parameter(name.clone()), parser); + } + } + } + + if let Some(variables) = raw.get("variables").and_then(Value::as_object) { + let base = root.push("variables"); + for (name, variable) in variables { + index.add_string(variable, &base.push(name), &ExpressionScope::Variable(name.clone()), parser); + } + } + + if let Some(outputs) = raw.get("outputs").and_then(Value::as_object) { + let base = root.push("outputs"); + for (name, output) in outputs { + let scope = ExpressionScope::Output(name.clone()); + for key in ["value", "condition"] { + if let Some(member) = output.get(key) { + index.walk(member, &base.push(name).push(key), &scope, parser); + } + } + } + } + + if let Some(functions) = raw.get("functions").and_then(Value::as_array) { + let base = root.push("functions"); + for (position, function) in functions.iter().enumerate() { + let Some(members) = function.get("members").and_then(Value::as_object) else { + continue; + }; + for (member_name, member) in members { + if let Some(value) = member.get("output").and_then(|output| output.get("value")) { + let pointer = base.push_index(position).push("members").push(member_name).push("output").push("value"); + index.walk(value, &pointer, &ExpressionScope::UserFunction, parser); + } + } + } + } + + for (position, site) in resources.iter().enumerate() { + let Some(resource) = site.pointer.resolve(raw) else { + continue; + }; + let scope = ExpressionScope::Resource(position); + for key in ["name", "condition", "dependsOn", "properties"] { + if key == "properties" && site.has_embedded_configuration() { + continue; + } + if let Some(member) = resource.get(key) { + index.walk(member, &site.pointer.push(key), &scope, parser); + } + } + if let Some(copy) = resource.get("copy") { + for key in ["count", "batchSize"] { + if let Some(member) = copy.get(key) { + index.walk(member, &site.pointer.push("copy").push(key), &scope, parser); + } + } + } + } + + index + } + + /// Parse a value when it is a string that starts with `[`, without looking inside objects + /// or arrays. + /// + /// # Arguments + /// + /// * `value` - The value. + /// * `pointer` - The pointer to `value`. + /// * `scope` - The scope the expression belongs to. + /// * `parser` - The parser to parse expressions with. + fn add_string(&mut self, value: &Value, pointer: &JsonPointer, scope: &ExpressionScope, parser: &mut Statement) { + if let Value::String(text) = value && text.starts_with('[') { + self.by_pointer.insert(pointer.to_string(), self.sites.len()); + self.sites.push(ExpressionSite { + pointer: pointer.clone(), + text: text.clone(), + scope: scope.clone(), + parsed: parser.parse(text), + }); + } + } + + /// Walk a value, parsing every string that starts with `[`. + /// + /// # Arguments + /// + /// * `value` - The value to walk. + /// * `pointer` - The pointer to `value`. + /// * `scope` - The scope every expression found belongs to. + /// * `parser` - The parser to parse expressions with. + fn walk(&mut self, value: &Value, pointer: &JsonPointer, scope: &ExpressionScope, parser: &mut Statement) { + match value { + Value::String(_) => self.add_string(value, pointer, scope, parser), + Value::Array(items) => { + for (position, item) in items.iter().enumerate() { + self.walk(item, &pointer.push_index(position), scope, parser); + } + }, + Value::Object(members) => { + for (key, member) in members { + self.walk(member, &pointer.push(key), scope, parser); + } + }, + _ => {}, + } + } + + /// Every expression site in document order. + #[must_use] + pub fn sites(&self) -> &[ExpressionSite] { + &self.sites + } + + /// The expression site at a pointer, if the string there is an expression. + /// + /// # Arguments + /// + /// * `pointer` - The pointer to the string. + #[must_use] + pub fn site_at(&self, pointer: &JsonPointer) -> Option<&ExpressionSite> { + self.by_pointer.get(pointer.as_str()).map(|position| &self.sites[*position]) + } + + /// Every function call in every successfully parsed expression, outermost first. + #[must_use] + pub fn calls(&self) -> Vec> { + let mut calls = Vec::new(); + for site in &self.sites { + if let Ok(ParsedStatement::Expression(expression)) = &site.parsed { + visit_expression(expression, &mut |function| calls.push(FunctionCall { site, function })); + } + } + calls + } + + /// Every call of the named function; names are case-sensitive, as at runtime. + /// + /// # Arguments + /// + /// * `name` - The function name. + #[must_use] + pub fn calls_named(&self, name: &str) -> Vec> { + self.calls().into_iter().filter(|call| call.function.name() == name).collect() + } +} + +/// Visit a function and, depth first, every function nested in its arguments and accessors. +/// +/// # Arguments +/// +/// * `expression` - The expression to visit. +/// * `visit` - Called with each function found, outermost first. +pub fn visit_expression<'a>(expression: &'a Expression, visit: &mut dyn FnMut(&'a Function)) { + let function = expression.function(); + visit(function); + if let Some(args) = function.args() { + for arg in args { + match arg { + FunctionArg::Expression(nested) => visit_expression(nested, visit), + FunctionArg::Lambda(lambda) => visit_expression(&lambda.body, visit), + FunctionArg::Value(_) => {}, + } + } + } + for accessor in expression.accessors() { + if let Accessor::IndexExpression(nested) = accessor { + visit_expression(nested, visit); + } + } +} + +/// The literal string argument at a position of a call, if that argument is a literal string. +/// +/// # Arguments +/// +/// * `function` - The call. +/// * `position` - The zero-based argument position. +#[must_use] +pub fn literal_string_arg(function: &Function, position: usize) -> Option<&str> { + match function.args()?.get(position)? { + FunctionArg::Value(Value::String(text)) => Some(text), + _ => None, + } +} + +/// The literal values of a call's arguments, with `None` for arguments that are nested +/// expressions. +/// +/// # Arguments +/// +/// * `function` - The call. +#[must_use] +pub fn literal_args(function: &Function) -> Vec> { + function.args().map_or_else(Vec::new, |args| { + args.iter().map(|arg| match arg { + FunctionArg::Value(value) => Some(value), + _ => None, + }).collect() + }) +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::{configure::config_doc::Configuration, lint::context::collect_resource_sites}; + + fn index_for(yaml: &str) -> (Value, ExpressionIndex) { + let raw: Value = serde_yaml::from_str(yaml).unwrap(); + let config: Configuration = serde_json::from_value(raw.clone()).unwrap(); + let sites = collect_resource_sites(&config); + let mut parser = Statement::new().unwrap(); + let index = ExpressionIndex::build(&raw, &sites, &mut parser); + (raw, index) + } + + #[test] + fn indexes_expressions_in_every_scope() { + let (_, index) = index_for(r#" +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +parameters: + p: + type: string + defaultValue: "[concat('a', 'b')]" +variables: + v: "[parameters('p')]" +functions: +- namespace: my + members: + twice: + output: + type: string + value: "[concat(parameters('x'), parameters('x'))]" +outputs: + o: + type: string + value: "[variables('v')]" + condition: "[true()]" +resources: +- name: "[concat('n', '1')]" + type: Microsoft.DSC.Debug/Echo + condition: "[true()]" + dependsOn: + - "[resourceId('Microsoft.DSC.Debug/Echo', 'other')]" + properties: + output: + nested: "[parameters('p')]" + literal: "[[escaped]" +- name: other + type: Microsoft.DSC.Debug/Echo +"#); + let scopes: Vec<(&str, &ExpressionScope)> = index.sites().iter().map(|site| (site.pointer.as_str(), &site.scope)).collect(); + assert!(scopes.contains(&("/parameters/p/defaultValue", &ExpressionScope::Parameter("p".to_string())))); + assert!(scopes.contains(&("/variables/v", &ExpressionScope::Variable("v".to_string())))); + assert!(scopes.contains(&("/functions/0/members/twice/output/value", &ExpressionScope::UserFunction))); + assert!(scopes.contains(&("/outputs/o/value", &ExpressionScope::Output("o".to_string())))); + assert!(scopes.contains(&("/outputs/o/condition", &ExpressionScope::Output("o".to_string())))); + assert!(scopes.contains(&("/resources/0/name", &ExpressionScope::Resource(0)))); + assert!(scopes.contains(&("/resources/0/condition", &ExpressionScope::Resource(0)))); + assert!(scopes.contains(&("/resources/0/dependsOn/0", &ExpressionScope::Resource(0)))); + assert!(scopes.contains(&("/resources/0/properties/output/nested", &ExpressionScope::Resource(0)))); + // escaped strings are indexed so rules can tell them apart from literals + let escaped = index.site_at(&JsonPointer::from_pointer("/resources/0/properties/output/literal")).unwrap(); + assert!(matches!(escaped.parsed, Ok(ParsedStatement::Escaped(_)))); + assert!(index.sites().iter().all(|site| site.parsed.is_ok())); + } + + #[test] + fn collects_nested_calls() { + let (_, index) = index_for(r#" +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +resources: +- name: a + type: Microsoft.DSC.Debug/Echo + properties: + output: "[concat(parameters('p'), variables('v')[int('0')])]" +"#); + let names: Vec<&str> = index.calls().iter().map(|call| call.function.name()).collect(); + assert_eq!(names, vec!["concat", "parameters", "variables", "int"]); + let parameters = index.calls_named("parameters"); + assert_eq!(parameters.len(), 1); + assert_eq!(literal_string_arg(parameters[0].function, 0), Some("p")); + assert_eq!(literal_args(index.calls_named("concat")[0].function), vec![None, None]); + } + + #[test] + fn records_parse_failures() { + let (_, index) = index_for(r#" +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +resources: +- name: a + type: Microsoft.DSC.Debug/Echo + properties: + output: "[concat('unterminated)]" +"#); + assert_eq!(index.sites().len(), 1); + assert!(index.sites()[0].parsed.is_err()); + assert!(index.calls().is_empty()); + } + + #[test] + fn indexes_only_string_defaults_and_variables() { + let (_, index) = index_for(r#" +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +parameters: + p: + type: object + defaultValue: + nested: "[parameters('missing')]" +variables: + literal: + nested: "[parameters('missing')]" + list: + - "[parameters('missing')]" + evaluated: "[parameters('p')]" +resources: [] +"#); + let pointers: Vec<&str> = index.sites().iter().map(|site| site.pointer.as_str()).collect(); + assert_eq!(pointers, vec!["/variables/evaluated"]); + } + + #[test] + fn skips_embedded_configurations() { + let (_, index) = index_for(r#" +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +resources: +- name: "[concat('group', '1')]" + type: Microsoft.DSC/Group + properties: + $schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json + resources: + - name: inner + type: Microsoft.DSC.Debug/Echo + properties: + output: "[parameters('inner')]" +- name: other + type: Third.Party/Group + properties: + $schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json + resources: [] + parameters: + x: + type: string + defaultValue: "[parameters('inner')]" +- name: plain + type: Microsoft.DSC.Debug/Echo + properties: + output: "[parameters('outer')]" +"#); + let pointers: Vec<&str> = index.sites().iter().map(|site| site.pointer.as_str()).collect(); + assert_eq!(pointers, vec!["/resources/0/name", "/resources/2/properties/output"]); + } +} diff --git a/lib/dsc-lib/src/lint/mod.rs b/lib/dsc-lib/src/lint/mod.rs new file mode 100644 index 000000000..4c37a0b7a --- /dev/null +++ b/lib/dsc-lib/src/lint/mod.rs @@ -0,0 +1,866 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +//! Static analysis for DSC configuration documents. +//! +//! The linter parses a document, validates it against the configuration document schema, and +//! runs a set of rules that find problems before the document is applied. Rules never invoke +//! resource operations; with discovery enabled they only read the manifests of installed +//! resources. +//! +//! ```rust,ignore +//! use dsc_lib::lint::{Linter, LintOptions}; +//! +//! let mut linter = Linter::new(LintOptions::default())?; +//! let result = linter.lint(document_text)?; +//! for diagnostic in &result.diagnostics { +//! println!("{} {} {}: {}", diagnostic.level, diagnostic.rule, diagnostic.path, diagnostic.message); +//! } +//! ``` + +use std::collections::HashMap; + +use miette::Diagnostic; +use rust_i18n::t; +use serde_json::Value; +use thiserror::Error; +use tracing::{debug, warn}; + +use crate::{ + DscManager, + configure::{ + config_doc::{Configuration, Resource, ResourceDiscoveryMode}, + get_require_adapter_from_directive, + parameters::import_parameters, + }, + diagnostics::SourceMap, + discovery::discovery_trait::DiscoveryFilter, + dscerror::DscError, + dscresources::{dscresource::validate_json_all, resource_manifest::ResourceManifest}, + extensions::extension_manifest::ExtensionManifest, + parser::Statement, + progress::ProgressFormat, + schemas::dsc_repo::{DscRepoSchema, RecognizedSchemaVersion, SchemaForm, SchemaUriPrefix}, + settings::global as global_settings, + util::parse_input_to_value, +}; + +pub mod context; +pub mod diagnostic; +pub mod expression_index; +pub mod options; +pub mod pointer; +pub mod report; +pub mod rule; +pub mod rules; + +pub use diagnostic::{LintCategory, LintDiagnostic, LintLevel, LintResult, LintSummary, SourceLocation}; +pub use options::{DocumentKind, LintOptions, LintSettings, RuleLevel, RuleLevelOverride, RuleSetting}; +pub use pointer::JsonPointer; +pub use report::{LintDiagnosticReport, redacted_source_map}; +pub use rule::{CheckFn, DiagnosticSink, LintRuleInfo, Rule, RuleTier}; + +use context::{collect_resource_sites, LintContext, ResolvedResource}; +use expression_index::ExpressionIndex; + +/// The fixed rule that reports a document that isn't valid JSON or YAML. +pub const PARSE_ERROR_RULE: &str = "parse-error"; +/// The fixed rule that reports violations of the configuration document schema. +pub const SCHEMA_INVALID_RULE: &str = "schema-invalid"; +/// The fixed rule that reports a document kind the linter doesn't support yet. +pub const UNSUPPORTED_DOCUMENT_TYPE_RULE: &str = "unsupported-document-type"; +/// The fixed rule that reports a resource discovery failure. +pub const DISCOVERY_FAILED_RULE: &str = "discovery-failed"; + +/// The rules whose level can't be changed. +const FIXED_RULES: [&str; 4] = [PARSE_ERROR_RULE, SCHEMA_INVALID_RULE, UNSUPPORTED_DOCUMENT_TYPE_RULE, DISCOVERY_FAILED_RULE]; + +/// Errors raised by the linter itself, as opposed to problems found in a document. +#[derive(Debug, Error, Diagnostic)] +pub enum LintError { + /// A rule level override names a rule that doesn't exist. + #[error("{t}: '{0}'", t = t!("lint.mod.unknownRule"))] + UnknownRule(String), + /// A rule level override isn't written as `=`. + #[error("{t}: '{0}'", t = t!("lint.mod.invalidRuleOverride"))] + InvalidRuleOverride(String), + /// A rule level isn't `off`, `info`, `warning`, or `error`. + #[error("{t}: '{0}'", t = t!("lint.mod.invalidRuleLevel"))] + InvalidRuleLevel(String), + /// The `lint` section of the settings file can't be read. + #[error("{t}: {0}", t = t!("lint.mod.invalidSettings"))] + InvalidSettings(String), + /// The supplied parameters aren't in a format DSC accepts. + #[error("{t}: {0}", t = t!("lint.mod.invalidParameters"))] + InvalidParameters(String), +} + +/// A rule together with the level it reports at in this run. +struct ActiveRule { + rule: Rule, + level: LintLevel, +} + +/// Lints configuration documents. +/// +/// Create one with [`Linter::new`] and reuse it for several documents; it holds the compiled +/// expression parser and the effective rule levels. +pub struct Linter { + options: LintOptions, + rules: Vec, + levels: HashMap<&'static str, RuleLevel>, + parser: Statement, + config_schema: Value, +} + +impl Linter { + /// Create a linter. + /// + /// # Arguments + /// + /// * `options` - The options of the run. + /// + /// # Errors + /// + /// Returns [`LintError::UnknownRule`] when an override names an unknown rule, + /// [`LintError::InvalidSettings`] when the settings file's `lint` section is malformed, or + /// another `DscError` when the expression parser or the document schema can't be prepared. + pub fn new(options: LintOptions) -> Result { + let levels = effective_levels(&options)?; + let rules = rules::all_rules().into_iter().filter_map(|rule| { + let level = levels.get(rule.id).copied().unwrap_or(rule.default_level).as_lint_level()?; + Some(ActiveRule { rule, level }) + }).collect(); + let config_schema = serde_json::to_value(Configuration::generate_schema( + RecognizedSchemaVersion::default(), + SchemaForm::Bundled, + SchemaUriPrefix::AkaDotMs, + ))?; + Ok(Self { + options, + rules, + levels, + parser: Statement::new()?, + config_schema, + }) + } + + /// The options of the run. + #[must_use] + pub fn options(&self) -> &LintOptions { + &self.options + } + + /// Set the file name recorded on the diagnostics of the documents linted from now on. + /// + /// # Arguments + /// + /// * `file` - The file the next documents are read from, or `None` for inline input. + pub fn set_file(&mut self, file: Option) { + self.options.file = file; + } + + /// Describe every rule, fixed and configurable, with the level it reports at in this run. + #[must_use] + pub fn rules(&self) -> Vec { + let mut list = fixed_rule_infos(); + for rule in rules::all_rules() { + let level = self.levels.get(rule.id).copied().unwrap_or(rule.default_level); + list.push(LintRuleInfo::from_rule(&rule, level)); + } + list.sort_by(|a, b| a.id.cmp(&b.id)); + list + } + + /// Lint a document given as JSON or YAML text. + /// + /// Problems in the document are reported as diagnostics, including a document that doesn't + /// parse. Every diagnostic gets the line and column of its location in the text. + /// + /// # Arguments + /// + /// * `document` - The document text. + /// + /// # Errors + /// + /// Returns [`LintError::InvalidParameters`] when the supplied parameters can't be imported, + /// a `DscError::Validation` when the configuration document schema can't be compiled, or + /// another `DscError` when discovery itself fails in a way that isn't reported as a + /// diagnostic. + pub fn lint(&mut self, document: &str) -> Result { + match parse_document(document) { + Ok(raw) => { + let mut result = self.lint_parsed(&raw, Vec::new())?; + if let Some(map) = SourceMap::parse(document) { + fill_locations(&mut result.diagnostics, &map); + } + Ok(result) + }, + Err(diagnostic) => Ok(LintResult::from_diagnostics(stamp_file(vec![*diagnostic], &self.options.file), false)), + } + } + + /// Lint a document that is already parsed. + /// + /// # Arguments + /// + /// * `document` - The document. + /// + /// # Errors + /// + /// See [`lint`](Self::lint). + pub fn lint_value(&mut self, document: &Value) -> Result { + self.lint_parsed(document, Vec::new()) + } + + /// Run the pipeline on a parsed document. + /// + /// # Arguments + /// + /// * `raw` - The document. + /// * `diagnostics` - Diagnostics already collected for the document. + fn lint_parsed(&mut self, raw: &Value, mut diagnostics: Vec) -> Result { + if let Some(diagnostic) = self.check_document_kind(raw) { + diagnostics.push(diagnostic); + return Ok(LintResult::from_diagnostics(stamp_file(diagnostics, &self.options.file), false)); + } + + for violation in validate_json_all("Configuration", &self.config_schema, raw)? { + diagnostics.push(fixed_diagnostic( + SCHEMA_INVALID_RULE, + LintCategory::Document, + LintLevel::Error, + &JsonPointer::from_pointer(&violation.instance_path), + None, + violation.message, + )); + } + + let config: Configuration = match serde_json::from_value(raw.clone()) { + Ok(config) => config, + Err(err) => { + if diagnostics.is_empty() { + diagnostics.push(fixed_diagnostic( + SCHEMA_INVALID_RULE, + LintCategory::Document, + LintLevel::Error, + &JsonPointer::root(), + None, + t!("lint.mod.documentInvalid", error = err).to_string(), + )); + } + return Ok(LintResult::from_diagnostics(stamp_file(diagnostics, &self.options.file), false)); + }, + }; + + let supplied_parameters = match &self.options.parameters { + Some(parameters) => Some(import_parameters(parameters).map_err(|err| LintError::InvalidParameters(err.to_string()))?), + None => None, + }; + + let Self { options, rules, parser, .. } = self; + let sites = collect_resource_sites(&config); + let expressions = ExpressionIndex::build(raw, &sites, parser); + let mut context = LintContext { + raw, + config: &config, + resources: sites, + expressions, + options, + functions: parser.function_dispatcher(), + supplied_parameters, + discovered: None, + }; + + for active in rules.iter().filter(|active| active.rule.tier == RuleTier::Static) { + run_rule(active, &context, &mut diagnostics); + } + + let mut discovery_ran = false; + if options.discovery && !skips_discovery(&config) { + match discover(&context, options.progress_format) { + Ok(discovered) => { + context.discovered = Some(discovered); + discovery_ran = true; + for active in rules.iter().filter(|active| active.rule.tier == RuleTier::Resource) { + run_rule(active, &context, &mut diagnostics); + } + }, + Err(err) => { + debug!("{}", t!("lint.mod.discoveryFailed", error = err)); + diagnostics.push(fixed_diagnostic( + DISCOVERY_FAILED_RULE, + LintCategory::Resource, + LintLevel::Warning, + &JsonPointer::root(), + None, + t!("lint.mod.discoveryFailed", error = err).to_string(), + )); + }, + } + } + + Ok(LintResult::from_diagnostics(stamp_file(diagnostics, &options.file), discovery_ran)) + } + + /// Report a document whose kind the linter doesn't support. + /// + /// # Arguments + /// + /// * `raw` - The document. + fn check_document_kind(&self, raw: &Value) -> Option { + let kind = self.options.document_kind.or_else(|| detect_document_kind(raw))?; + if kind == DocumentKind::Configuration { + return None; + } + Some(fixed_diagnostic( + UNSUPPORTED_DOCUMENT_TYPE_RULE, + LintCategory::Document, + LintLevel::Error, + &JsonPointer::root().push("$schema"), + None, + t!("lint.mod.unsupportedDocumentType", kind = kind).to_string(), + )) + } +} + +/// Parse a document as JSON or YAML, reporting a failure as a diagnostic. +/// +/// Input that starts with `{` is reported with the JSON parser's message and position so the +/// message matches what the author wrote; everything else uses the YAML parser's. Text that is +/// empty or only whitespace is reported as empty rather than as whatever the YAML parser makes +/// of it. +/// +/// # Arguments +/// +/// * `document` - The document text. +/// +/// # Errors +/// +/// Returns the `parse-error` diagnostic when the text is empty or neither valid JSON nor valid +/// YAML. +fn parse_document(document: &str) -> Result> { + if document.trim().is_empty() { + return Err(Box::new(fixed_diagnostic( + PARSE_ERROR_RULE, + LintCategory::Document, + LintLevel::Error, + &JsonPointer::root(), + None, + t!("lint.mod.emptyDocument").to_string(), + ))); + } + let yaml_error = match parse_input_to_value(document) { + Ok(value) => return Ok(value), + Err(err) => err, + }; + let (message, location) = match (document.trim_start().starts_with('{'), serde_json::from_str::(document), &yaml_error) { + (true, Err(err), _) => (err.to_string(), Some(SourceLocation { line: err.line(), column: err.column() })), + (_, _, DscError::Yaml(err)) => (err.to_string(), err.location().map(|location| SourceLocation { line: location.line(), column: location.column() })), + (_, _, other) => (other.to_string(), None), + }; + Err(Box::new(fixed_diagnostic( + PARSE_ERROR_RULE, + LintCategory::Document, + LintLevel::Error, + &JsonPointer::root(), + location, + t!("lint.mod.parseError", error = message).to_string(), + ))) +} + +/// Detect the kind of document from its `$schema` URI. +/// +/// # Arguments +/// +/// * `raw` - The document. +/// +/// # Returns +/// +/// The kind, or `None` when there is no recognized `$schema`. +fn detect_document_kind(raw: &Value) -> Option { + let uri = raw.get("$schema")?.as_str()?.to_string(); + if Configuration::is_recognized_schema_uri(&uri) { + Some(DocumentKind::Configuration) + } else if ResourceManifest::is_recognized_schema_uri(&uri) { + Some(DocumentKind::ResourceManifest) + } else if ExtensionManifest::is_recognized_schema_uri(&uri) { + Some(DocumentKind::ExtensionManifest) + } else { + None + } +} + +/// Whether the document asks DSC to discover resources during deployment instead of up front. +/// +/// # Arguments +/// +/// * `config` - The configuration. +fn skips_discovery(config: &Configuration) -> bool { + config.directives.as_ref().and_then(|directives| directives.resource_discovery.as_ref()) == Some(&ResourceDiscoveryMode::DuringDeployment) +} + +/// The discovery filter for a resource declaration, matching what `dsc config` uses. +/// +/// # Arguments +/// +/// * `resource` - The resource declaration. +fn filter_for(resource: &Resource) -> DiscoveryFilter { + DiscoveryFilter::new( + &resource.resource_type, + resource.require_version.clone(), + get_require_adapter_from_directive(&resource.directives), + ) +} + +/// Discover the resource types the document uses. +/// +/// # Arguments +/// +/// * `context` - The lint context, for its resource declarations. +/// * `progress_format` - How to report discovery progress. +/// +/// # Returns +/// +/// One entry per resource declaration, in the context's order. +/// +/// # Errors +/// +/// Returns the discovery error when resources can't be discovered at all. +fn discover(context: &LintContext, progress_format: ProgressFormat) -> Result, DscError> { + let mut manager = DscManager::new(); + let mut filters: Vec = Vec::new(); + for site in &context.resources { + let filter = filter_for(site.resource); + if !filters.contains(&filter) { + filters.push(filter); + } + } + manager.find_resources(&filters, progress_format)?; + + let mut resolved = Vec::with_capacity(context.resources.len()); + for site in &context.resources { + let found = manager.find_resource(&filter_for(site.resource))?.and_then(|resource| resource.as_dsc_resource().cloned()); + let type_exists = if found.is_some() { + true + } else if site.resource.require_version.is_some() { + let any_version = DiscoveryFilter::new(&site.resource.resource_type, None, get_require_adapter_from_directive(&site.resource.directives)); + manager.find_resource(&any_version)?.is_some() + } else { + false + }; + resolved.push(ResolvedResource { found, type_exists }); + } + Ok(resolved) +} + +/// Run one rule, collecting what it reports. +/// +/// # Arguments +/// +/// * `active` - The rule and its level. +/// * `context` - The lint context. +/// * `diagnostics` - Where reported diagnostics are appended. +fn run_rule(active: &ActiveRule, context: &LintContext, diagnostics: &mut Vec) { + let mut sink = DiagnosticSink::new(active.rule.id, active.rule.category, active.level, diagnostics); + (active.rule.check)(context, &mut sink); +} + +/// Build a diagnostic for one of the fixed rules. +/// +/// # Arguments +/// +/// * `rule` - The rule identifier. +/// * `category` - The rule category. +/// * `level` - The level to report at. +/// * `path` - The location of the problem. +/// * `location` - The position in the source text, when known. +/// * `message` - A description of the problem. +fn fixed_diagnostic(rule: &str, category: LintCategory, level: LintLevel, path: &JsonPointer, location: Option, message: String) -> LintDiagnostic { + LintDiagnostic { + level, + rule: rule.to_string(), + category, + message, + path: path.to_string(), + file: None, + location, + } +} + +/// Record the source file on diagnostics that don't have one yet. +/// +/// # Arguments +/// +/// * `diagnostics` - The diagnostics. +/// * `file` - The file the document was read from, if known. +fn stamp_file(mut diagnostics: Vec, file: &Option) -> Vec { + for diagnostic in &mut diagnostics { + if diagnostic.file.is_none() { + diagnostic.file.clone_from(file); + } + } + diagnostics +} + +/// Record the line and column of each diagnostic's path on diagnostics that don't have a +/// location yet. +/// +/// A path the document doesn't spell out, like a property that is missing, gets the location of +/// its closest enclosing value. +/// +/// # Arguments +/// +/// * `diagnostics` - The diagnostics. +/// * `map` - The source map of the document text. +fn fill_locations(diagnostics: &mut [LintDiagnostic], map: &SourceMap) { + for diagnostic in diagnostics.iter_mut().filter(|diagnostic| diagnostic.location.is_none()) { + if let Some((_, span)) = map.nearest_span(&diagnostic.path) { + let (line, column) = map.location(span.start); + diagnostic.location = Some(SourceLocation { line, column }); + } + } +} + +/// Whether a rule's level is fixed. +/// +/// # Arguments +/// +/// * `id` - The rule identifier. +fn is_fixed_rule(id: &str) -> bool { + FIXED_RULES.contains(&id) +} + +/// Describe the fixed rules. +fn fixed_rule_infos() -> Vec { + let info = |id: &str, category: LintCategory, level: LintLevel, description: String| LintRuleInfo { + id: id.to_string(), + category, + tier: if category == LintCategory::Resource { RuleTier::Resource } else { RuleTier::Static }, + default_level: level.into(), + level: level.into(), + fixed: true, + description, + }; + vec![ + info(PARSE_ERROR_RULE, LintCategory::Document, LintLevel::Error, t!("lint.mod.parseErrorDescription").to_string()), + info(SCHEMA_INVALID_RULE, LintCategory::Document, LintLevel::Error, t!("lint.mod.schemaInvalidDescription").to_string()), + info(UNSUPPORTED_DOCUMENT_TYPE_RULE, LintCategory::Document, LintLevel::Error, t!("lint.mod.unsupportedDocumentTypeDescription").to_string()), + info(DISCOVERY_FAILED_RULE, LintCategory::Resource, LintLevel::Warning, t!("lint.mod.discoveryFailedDescription").to_string()), + ] +} + +/// Compute the level of every configurable rule for a run. +/// +/// Precedence, lowest first: the rule's default, the settings file, the overrides in the +/// options, and the policy file. +/// +/// # Arguments +/// +/// * `options` - The options of the run. +/// +/// # Errors +/// +/// Returns [`LintError::UnknownRule`] when an override names an unknown rule, or +/// [`LintError::InvalidSettings`] when the settings can't be read. +fn effective_levels(options: &LintOptions) -> Result, LintError> { + let mut levels: HashMap<&'static str, RuleLevel> = rules::all_rules().iter().map(|rule| (rule.id, rule.default_level)).collect(); + let (settings, policy) = if options.use_settings { load_settings() } else { (None, None) }; + if let Some(settings) = settings { + apply_settings(&mut levels, &settings); + } + for override_ in &options.rule_levels { + if is_fixed_rule(&override_.id) { + warn!("{}", t!("lint.mod.fixedRuleOverrideIgnored", rule = override_.id)); + continue; + } + let Some(level) = levels.get_mut(override_.id.as_str()) else { + return Err(LintError::UnknownRule(override_.id.clone())); + }; + *level = override_.level; + } + if let Some(policy) = policy { + apply_settings(&mut levels, &policy); + } + Ok(levels) +} + +/// Read the `lint` section of the settings and policy files. +/// +/// The machine, user, and workspace settings files are merged per rule, with the workspace file +/// taking precedence over the user file and the user file over the machine file. The settings +/// files aren't read when DSC is told to ignore them. The policy file is always read. +/// +/// A settings file whose `lint` section isn't valid fails to load as a whole. The settings module +/// reports that as a warning, and the file contributes no rule levels. +/// +/// # Returns +/// +/// The merged settings-file section and the policy-file section; either is `None` when not +/// configured. +fn load_settings() -> (Option, Option) { + let mut settings = global_settings::get_or_init(); + settings.load(); + + let mut merged: Option = None; + for scope in [&settings.machine, &settings.user, &settings.workspace] { + if let Some(lint) = scope.as_ref().and_then(|data| data.lint.as_ref()) { + merged.get_or_insert_with(LintSettings::default).rules.extend(lint.rules.clone()); + } + } + let policy = settings.policy.as_ref().and_then(|data| data.lint.clone()); + + (merged, policy) +} + +/// Apply the rule levels from a settings section, ignoring fixed and unknown rules. +/// +/// # Arguments +/// +/// * `levels` - The levels to update. +/// * `settings` - The settings section. +fn apply_settings(levels: &mut HashMap<&'static str, RuleLevel>, settings: &LintSettings) { + for (id, setting) in &settings.rules { + let Some(level) = setting.level else { + continue; + }; + if is_fixed_rule(id) { + warn!("{}", t!("lint.mod.fixedRuleOverrideIgnored", rule = id)); + continue; + } + match levels.get_mut(id.as_str()) { + Some(current) => *current = level, + None => warn!("{}", t!("lint.mod.unknownRuleInSettings", rule = id)), + } + } +} + +#[cfg(test)] +pub(crate) mod test_util { + use super::{LintOptions, LintResult, Linter}; + + /// Lint a document with discovery and the settings file disabled. + pub(crate) fn lint_yaml(document: &str) -> LintResult { + lint_with(document, LintOptions { + discovery: false, + use_settings: false, + ..Default::default() + }) + } + + /// Lint a document with the given options. + pub(crate) fn lint_with(document: &str, options: LintOptions) -> LintResult { + Linter::new(options).unwrap().lint(document).unwrap() + } + + /// The paths of the diagnostics a rule reported, in result order. + pub(crate) fn rule_paths(result: &LintResult, rule: &str) -> Vec { + result.diagnostics.iter().filter(|diagnostic| diagnostic.rule == rule).map(|diagnostic| diagnostic.path.clone()).collect() + } + + /// The messages of the diagnostics a rule reported, in result order. + pub(crate) fn rule_messages(result: &LintResult, rule: &str) -> Vec { + result.diagnostics.iter().filter(|diagnostic| diagnostic.rule == rule).map(|diagnostic| diagnostic.message.clone()).collect() + } +} + +#[cfg(test)] +mod tests { + use serde_json::json; + + use super::{test_util::{lint_with, lint_yaml, rule_paths}, *}; + + const CLEAN: &str = r#" +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +parameters: + greeting: + type: string + defaultValue: hello +resources: +- name: a + type: Microsoft.DSC.Debug/Echo + properties: + output: "[parameters('greeting')]" +"#; + + #[test] + fn clean_document_is_valid() { + let result = lint_yaml(CLEAN); + assert!(result.valid, "{:?}", result.diagnostics); + assert!(result.diagnostics.is_empty(), "{:?}", result.diagnostics); + assert!(!result.discovery); + } + + #[test] + fn reports_yaml_parse_errors_with_location() { + let result = lint_yaml("resources:\n- name: a\n type: [unclosed\n"); + assert_eq!(rule_paths(&result, PARSE_ERROR_RULE), vec![""]); + assert!(result.diagnostics[0].location.is_some()); + assert!(!result.valid); + } + + #[test] + fn reports_an_empty_document_as_a_parse_error() { + for document in ["", " \n\t\n"] { + let result = lint_yaml(document); + assert_eq!(rule_paths(&result, PARSE_ERROR_RULE), vec![""], "{document:?}"); + assert_eq!(result.diagnostics.len(), 1, "{document:?}"); + assert!(!result.valid); + } + } + + #[test] + fn reports_json_parse_errors_with_json_message() { + let result = lint_yaml("{\"resources\": [}"); + assert_eq!(rule_paths(&result, PARSE_ERROR_RULE), vec![""]); + let location = result.diagnostics[0].location.unwrap(); + assert_eq!(location.line, 1); + assert!(location.column > 1); + } + + #[test] + fn reports_schema_violations_with_paths() { + let result = lint_yaml(r#" +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +resources: +- name: a + type: Microsoft.DSC.Debug/Echo + unknownKey: true +"#); + let paths = rule_paths(&result, SCHEMA_INVALID_RULE); + assert!(paths.iter().any(|path| path == "/resources/0"), "{:?}", result.diagnostics); + assert!(!result.valid); + } + + #[test] + fn reports_missing_schema_uri() { + let result = lint_yaml("resources: []\n"); + assert!(!rule_paths(&result, SCHEMA_INVALID_RULE).is_empty(), "{:?}", result.diagnostics); + } + + #[test] + fn rejects_manifests() { + let result = lint_yaml(r#" +$schema: https://aka.ms/dsc/schemas/v3/bundled/resource/manifest.json +type: Test/Example +version: 1.0.0 +"#); + assert_eq!(rule_paths(&result, UNSUPPORTED_DOCUMENT_TYPE_RULE), vec!["/$schema"]); + assert_eq!(result.diagnostics.len(), 1); + } + + #[test] + fn overrides_change_levels_and_disable_rules() { + let document = r#" +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +parameters: + unused: + type: string + defaultValue: x +resources: +- name: a + type: Microsoft.DSC.Debug/Echo +"#; + let warned = lint_yaml(document); + assert_eq!(rule_paths(&warned, "no-unused-parameters"), vec!["/parameters/unused"]); + assert_eq!(warned.diagnostics[0].level, LintLevel::Warning); + + let raised = lint_with(document, LintOptions { + discovery: false, + use_settings: false, + rule_levels: vec!["no-unused-parameters=error".parse().unwrap()], + ..Default::default() + }); + assert_eq!(raised.diagnostics[0].level, LintLevel::Error); + assert!(!raised.valid); + + let silenced = lint_with(document, LintOptions { + discovery: false, + use_settings: false, + rule_levels: vec!["no-unused-parameters=off".parse().unwrap()], + ..Default::default() + }); + assert!(rule_paths(&silenced, "no-unused-parameters").is_empty()); + } + + #[test] + fn unknown_rule_override_is_an_error() { + let result = Linter::new(LintOptions { + use_settings: false, + rule_levels: vec!["no-such-rule=off".parse().unwrap()], + ..Default::default() + }); + assert!(matches!(result, Err(DscError::Lint(LintError::UnknownRule(id))) if id == "no-such-rule")); + } + + #[test] + fn fixed_rule_override_is_ignored() { + let linter = Linter::new(LintOptions { + use_settings: false, + rule_levels: vec!["parse-error=off".parse().unwrap()], + ..Default::default() + }).unwrap(); + let result = linter.rules(); + let parse_error = result.iter().find(|rule| rule.id == PARSE_ERROR_RULE).unwrap(); + assert!(parse_error.fixed); + assert_eq!(parse_error.level, RuleLevel::Error); + } + + #[test] + fn lists_rules_sorted_with_effective_levels() { + let linter = Linter::new(LintOptions { + use_settings: false, + rule_levels: vec!["duplicate-resource=info".parse().unwrap()], + ..Default::default() + }).unwrap(); + let rules = linter.rules(); + let ids: Vec<&str> = rules.iter().map(|rule| rule.id.as_str()).collect(); + let mut sorted = ids.clone(); + sorted.sort_unstable(); + assert_eq!(ids, sorted); + let duplicate = rules.iter().find(|rule| rule.id == "duplicate-resource").unwrap(); + assert_eq!(duplicate.default_level, RuleLevel::Error); + assert_eq!(duplicate.level, RuleLevel::Info); + assert!(!duplicate.fixed); + assert!(rules.iter().any(|rule| rule.tier == RuleTier::Resource)); + } + + #[test] + fn invalid_supplied_parameters_are_an_error() { + let result = Linter::new(LintOptions { + discovery: false, + use_settings: false, + parameters: Some(json!({"not": "parameters"})), + ..Default::default() + }).unwrap().lint(CLEAN); + assert!(matches!(result, Err(DscError::Lint(LintError::InvalidParameters(_))))); + } + + #[test] + fn records_the_file_on_diagnostics() { + let result = lint_with("resources: []\n", LintOptions { + discovery: false, + use_settings: false, + file: Some("config.yaml".to_string()), + ..Default::default() + }); + assert!(result.diagnostics.iter().all(|diagnostic| diagnostic.file.as_deref() == Some("config.yaml"))); + } + + #[test] + fn during_deployment_discovery_skips_the_resource_tier() { + let result = lint_with(r#" +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +directives: + resourceDiscovery: duringDeployment +resources: +- name: a + type: Not.Installed/Anywhere +"#, LintOptions { + use_settings: false, + ..Default::default() + }); + assert!(!result.discovery); + assert!(rule_paths(&result, "unknown-resource-type").is_empty()); + } +} diff --git a/lib/dsc-lib/src/lint/options.rs b/lib/dsc-lib/src/lint/options.rs new file mode 100644 index 000000000..26931783d --- /dev/null +++ b/lib/dsc-lib/src/lint/options.rs @@ -0,0 +1,222 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +use std::{collections::HashMap, fmt::Display, path::PathBuf, str::FromStr}; + +use clap::ValueEnum; +use rust_i18n::t; +use schemars::JsonSchema; +use serde::{Deserialize, Serialize}; +use serde_json::Value; + +use crate::{progress::ProgressFormat, schemas::transforms::idiomaticize_string_enum, types::SemanticVersion}; + +use super::{diagnostic::LintLevel, LintError}; + +/// The kind of DSC document being linted. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum DocumentKind { + /// A configuration document. + Configuration, + /// A resource manifest. + ResourceManifest, + /// An extension manifest. + ExtensionManifest, +} + +impl Display for DocumentKind { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + let text = match self { + DocumentKind::Configuration => t!("lint.options.documentKindConfiguration"), + DocumentKind::ResourceManifest => t!("lint.options.documentKindResourceManifest"), + DocumentKind::ExtensionManifest => t!("lint.options.documentKindExtensionManifest"), + }; + write!(f, "{text}") + } +} + +/// The level a rule reports at, or `off` to disable the rule. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Deserialize, Serialize, JsonSchema, ValueEnum)] +#[serde(rename_all = "camelCase")] +#[schemars(inline, transform = idiomaticize_string_enum)] +pub enum RuleLevel { + /// The rule doesn't run. + Off, + /// The rule reports at the info level. + Info, + /// The rule reports at the warning level. + Warning, + /// The rule reports at the error level. + Error, +} + +impl RuleLevel { + /// The diagnostic level for this rule level, or `None` when the rule is off. + #[must_use] + pub fn as_lint_level(self) -> Option { + match self { + RuleLevel::Off => None, + RuleLevel::Info => Some(LintLevel::Info), + RuleLevel::Warning => Some(LintLevel::Warning), + RuleLevel::Error => Some(LintLevel::Error), + } + } +} + +impl From for RuleLevel { + fn from(level: LintLevel) -> Self { + match level { + LintLevel::Info => RuleLevel::Info, + LintLevel::Warning => RuleLevel::Warning, + LintLevel::Error => RuleLevel::Error, + } + } +} + +impl Display for RuleLevel { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + let text = match self { + RuleLevel::Off => "off", + RuleLevel::Info => "info", + RuleLevel::Warning => "warning", + RuleLevel::Error => "error", + }; + write!(f, "{text}") + } +} + +impl FromStr for RuleLevel { + type Err = LintError; + + fn from_str(text: &str) -> Result { + match text.trim().to_ascii_lowercase().as_str() { + "off" => Ok(RuleLevel::Off), + "info" => Ok(RuleLevel::Info), + "warning" => Ok(RuleLevel::Warning), + "error" => Ok(RuleLevel::Error), + _ => Err(LintError::InvalidRuleLevel(text.to_string())), + } + } +} + +/// Overrides the level of one rule, written as `=` on the command line. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct RuleLevelOverride { + /// The identifier of the rule. + pub id: String, + /// The level the rule reports at. + pub level: RuleLevel, +} + +impl FromStr for RuleLevelOverride { + type Err = LintError; + + fn from_str(text: &str) -> Result { + let Some((id, level)) = text.split_once('=') else { + return Err(LintError::InvalidRuleOverride(text.to_string())); + }; + let id = id.trim(); + if id.is_empty() { + return Err(LintError::InvalidRuleOverride(text.to_string())); + } + Ok(Self { + id: id.to_string(), + level: level.parse()?, + }) + } +} + +impl Display for RuleLevelOverride { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + write!(f, "{}={}", self.id, self.level) + } +} + +/// The `lint` section of the DSC settings file. +#[derive(Debug, Clone, Default, PartialEq, Eq, Deserialize, Serialize, JsonSchema)] +#[serde(deny_unknown_fields, rename_all = "camelCase")] +#[schemars(inline)] +pub struct LintSettings { + /// Per-rule settings keyed by rule identifier. + #[serde(default)] + pub rules: HashMap, +} + +/// The settings for one rule in the DSC settings file. +#[derive(Debug, Clone, Default, PartialEq, Eq, Deserialize, Serialize, JsonSchema)] +#[serde(deny_unknown_fields)] +#[schemars(inline)] +pub struct RuleSetting { + /// The level the rule reports at. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub level: Option, +} + +/// Options that control a lint run. +#[derive(Debug, Clone)] +pub struct LintOptions { + /// The kind of document; detected from `$schema` when `None`. + pub document_kind: Option, + /// Whether to discover installed resources and run the rules that inspect them. + pub discovery: bool, + /// Whether to read rule levels from the DSC settings files. + pub use_settings: bool, + /// Rule level overrides, applied after the settings file and before the policy file. + pub rule_levels: Vec, + /// Parameter values that would be passed to the configuration, in either input format that + /// `dsc config` accepts. + pub parameters: Option, + /// The system root the configuration would be applied with, if not the running OS. + pub system_root: Option, + /// The DSC version to check `directives.version` against. + pub dsc_version: Option, + /// How to report progress during resource discovery. + pub progress_format: ProgressFormat, + /// The file the document was read from, recorded on each diagnostic. + pub file: Option, +} + +impl Default for LintOptions { + fn default() -> Self { + Self { + document_kind: None, + discovery: true, + use_settings: true, + rule_levels: Vec::new(), + parameters: None, + system_root: None, + dsc_version: None, + progress_format: ProgressFormat::None, + file: None, + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + use test_case::test_case; + + #[test_case("no-unused-parameters=off", "no-unused-parameters", RuleLevel::Off; "off")] + #[test_case("duplicate-resource = Warning", "duplicate-resource", RuleLevel::Warning; "spaces and case")] + #[test_case("x=ERROR", "x", RuleLevel::Error; "error")] + fn parses_rule_overrides(text: &str, id: &str, level: RuleLevel) { + let parsed: RuleLevelOverride = text.parse().unwrap(); + assert_eq!(parsed, RuleLevelOverride { id: id.to_string(), level }); + } + + #[test_case("no-equals"; "missing separator")] + #[test_case("=off"; "missing id")] + #[test_case("rule=loud"; "unknown level")] + fn rejects_invalid_rule_overrides(text: &str) { + assert!(text.parse::().is_err()); + } + + #[test] + fn settings_deserialize() { + let settings: LintSettings = serde_json::from_value(serde_json::json!({ + "rules": { "no-unused-variables": { "level": "info" } } + })).unwrap(); + assert_eq!(settings.rules["no-unused-variables"].level, Some(RuleLevel::Info)); + } +} diff --git a/lib/dsc-lib/src/lint/pointer.rs b/lib/dsc-lib/src/lint/pointer.rs new file mode 100644 index 000000000..2724f7df9 --- /dev/null +++ b/lib/dsc-lib/src/lint/pointer.rs @@ -0,0 +1,165 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +use std::fmt::Display; + +use serde_json::Value; + +/// A JSON pointer (RFC 6901) that identifies a location in a document. +/// +/// The empty pointer identifies the document root. +#[derive(Clone, Debug, Default, PartialEq, Eq, Hash, PartialOrd, Ord)] +pub struct JsonPointer(String); + +impl JsonPointer { + /// The pointer to the document root. + #[must_use] + pub fn root() -> Self { + Self(String::new()) + } + + /// Create a pointer from an existing pointer string, such as one produced by a JSON Schema + /// validator. + /// + /// # Arguments + /// + /// * `pointer` - The pointer text, which must be empty or start with `/`. + #[must_use] + pub fn from_pointer(pointer: &str) -> Self { + Self(pointer.to_string()) + } + + /// Return a pointer to the member `key` of the object at this pointer. + /// + /// # Arguments + /// + /// * `key` - The member name; `~` and `/` are escaped as the RFC requires. + #[must_use] + pub fn push(&self, key: &str) -> Self { + let escaped = key.replace('~', "~0").replace('/', "~1"); + Self(format!("{}/{escaped}", self.0)) + } + + /// Return a pointer to the element `index` of the array at this pointer. + /// + /// # Arguments + /// + /// * `index` - The zero-based element index. + #[must_use] + pub fn push_index(&self, index: usize) -> Self { + Self(format!("{}/{index}", self.0)) + } + + /// Return a pointer that appends another pointer to this one. + /// + /// # Arguments + /// + /// * `other` - The pointer text to append, which must be empty or start with `/`. + #[must_use] + pub fn join(&self, other: &str) -> Self { + Self(format!("{}{other}", self.0)) + } + + /// The pointer text. + #[must_use] + pub fn as_str(&self) -> &str { + &self.0 + } + + /// Whether this pointer identifies `other` or a location below it. + /// + /// # Arguments + /// + /// * `other` - The pointer to compare against. + #[must_use] + pub fn starts_with(&self, other: &JsonPointer) -> bool { + self.0 == other.0 || (self.0.starts_with(&other.0) && self.0[other.0.len()..].starts_with('/')) + } + + /// Resolve the pointer against a document. + /// + /// # Arguments + /// + /// * `document` - The document to look in. + /// + /// # Returns + /// + /// The value at this pointer, or `None` when the document has no such location. + #[must_use] + pub fn resolve<'a>(&self, document: &'a Value) -> Option<&'a Value> { + document.pointer(&self.0) + } +} + +impl Display for JsonPointer { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + write!(f, "{}", self.0) + } +} + +/// Call `visit` for every string value in a document. +/// +/// # Arguments +/// +/// * `value` - The value to walk. +/// * `pointer` - The pointer to `value`. +/// * `visit` - Called with the pointer and text of each string found. +pub fn for_each_string(value: &Value, pointer: &JsonPointer, visit: &mut dyn FnMut(&JsonPointer, &str)) { + match value { + Value::String(text) => visit(pointer, text), + Value::Array(items) => { + for (index, item) in items.iter().enumerate() { + for_each_string(item, &pointer.push_index(index), visit); + } + }, + Value::Object(members) => { + for (key, member) in members { + for_each_string(member, &pointer.push(key), visit); + } + }, + _ => {}, + } +} + +#[cfg(test)] +mod tests { + use super::*; + use serde_json::json; + + #[test] + fn push_escapes_special_characters() { + let pointer = JsonPointer::root().push("a/b").push("c~d"); + assert_eq!(pointer.as_str(), "/a~1b/c~0d"); + } + + #[test] + fn push_index_and_join() { + let pointer = JsonPointer::root().push("resources").push_index(2).join("/properties/path"); + assert_eq!(pointer.to_string(), "/resources/2/properties/path"); + } + + #[test] + fn starts_with_matches_self_and_descendants_only() { + let base = JsonPointer::root().push("resources").push_index(1); + assert!(base.starts_with(&base)); + assert!(base.push("name").starts_with(&base)); + assert!(base.starts_with(&JsonPointer::root())); + assert!(!JsonPointer::root().push("resources").push_index(10).starts_with(&base)); + } + + #[test] + fn resolve_finds_values() { + let document = json!({"resources": [{"name": "first"}]}); + let pointer = JsonPointer::root().push("resources").push_index(0).push("name"); + assert_eq!(pointer.resolve(&document), Some(&json!("first"))); + assert_eq!(JsonPointer::root().push("missing").resolve(&document), None); + } + + #[test] + fn for_each_string_visits_nested_strings() { + let document = json!({"a": ["x", {"b": "y"}], "c": 1}); + let mut found = Vec::new(); + for_each_string(&document, &JsonPointer::root(), &mut |pointer, text| found.push((pointer.to_string(), text.to_string()))); + assert_eq!(found, vec![("/a/0".to_string(), "x".to_string()), ("/a/1/b".to_string(), "y".to_string())]); + } +} diff --git a/lib/dsc-lib/src/lint/report.rs b/lib/dsc-lib/src/lint/report.rs new file mode 100644 index 000000000..7d89605d6 --- /dev/null +++ b/lib/dsc-lib/src/lint/report.rs @@ -0,0 +1,238 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +//! Present lint diagnostics as [`miette::Diagnostic`]s, so a host can show each problem in the +//! document text it was found in. + +use std::fmt::{self, Display}; + +use miette::{Diagnostic, LabeledSpan, NamedSource, Severity, SourceCode}; +use rust_i18n::t; +use serde_json::Value; + +use crate::{ + configure::parameters::SECURE_VALUE_REDACTED, + diagnostics::SourceMap, + util::{is_secure_value, parse_input_to_value}, +}; + +use super::{diagnostic::{LintDiagnostic, LintLevel}, pointer::JsonPointer}; + +/// Parse document text for showing diagnostics in it, with every secure value replaced. +/// +/// The default value of each `secureString` or `secureObject` parameter and every wrapped +/// `{"secureString": ...}` or `{"secureObject": ...}` value is replaced in the text by +/// ``, so a snippet, including the lines of context around the labeled line, never +/// shows a secret the author hard-coded. Diagnostics keep their paths, because the structure of +/// the document is unchanged. +/// +/// # Arguments +/// +/// * `text` - The document text, as JSON or YAML. +/// +/// # Returns +/// +/// The source map of the redacted text, or `None` when the text can't be mapped or a secure +/// value can't be located in it; a host then reports the diagnostics without a snippet. +#[must_use] +pub fn redacted_source_map(text: &str) -> Option { + let map = SourceMap::parse(text)?; + let raw = parse_input_to_value(text).ok()?; + let mut pointers = Vec::new(); + if let Some(parameters) = raw.get("parameters").and_then(Value::as_object) { + for (name, parameter) in parameters { + let secure_type = parameter.get("type").and_then(Value::as_str) + .is_some_and(|kind| kind == "secureString" || kind == "secureObject"); + if secure_type && parameter.get("defaultValue").is_some() { + pointers.push(JsonPointer::root().push("parameters").push(name).push("defaultValue").to_string()); + } + } + } + collect_secure_values(&raw, &JsonPointer::root(), &mut pointers); + map.redacted(&pointers, SECURE_VALUE_REDACTED) +} + +/// Record the pointer of every wrapped secure value at or below a value. +/// +/// # Arguments +/// +/// * `value` - The value to search. +/// * `pointer` - The pointer of `value` in the document. +/// * `pointers` - Where the pointers found are appended. +fn collect_secure_values(value: &Value, pointer: &JsonPointer, pointers: &mut Vec) { + if is_secure_value(value) { + pointers.push(pointer.to_string()); + return; + } + match value { + Value::Object(members) => { + for (key, member) in members { + collect_secure_values(member, &pointer.push(key), pointers); + } + }, + Value::Array(items) => { + for (index, item) in items.iter().enumerate() { + collect_secure_values(item, &pointer.push_index(index), pointers); + } + }, + _ => {}, + } +} + +/// A lint diagnostic together with the text it points into. +/// +/// The rule identifier is the diagnostic code, the level is the severity, and the description of +/// the rule is the help text. When the document text is available, the problem's location is +/// labeled in it. +#[derive(Debug)] +pub struct LintDiagnosticReport { + message: String, + rule: String, + severity: Severity, + help: Option, + source: Option>, + label: Option, +} + +impl LintDiagnosticReport { + /// Create a report for a diagnostic. + /// + /// # Arguments + /// + /// * `diagnostic` - The diagnostic. + /// * `source` - The source map of the document the diagnostic was found in, when the text is + /// available. + /// * `help` - What the rule requires, shown as the help text. + #[must_use] + pub fn new(diagnostic: &LintDiagnostic, source: Option<&SourceMap>, help: Option) -> Self { + let severity = match diagnostic.level { + LintLevel::Error => Severity::Error, + LintLevel::Warning => Severity::Warning, + LintLevel::Info => Severity::Advice, + }; + let located = source.and_then(|map| { + let (_, span) = map.nearest_span(&diagnostic.path)?; + let first_line_end = map.text()[span.clone()].find('\n').map_or(span.end, |offset| span.start + offset); + let span = span.start..first_line_end.max(span.start); + let name = diagnostic.file.clone().unwrap_or_else(|| t!("lint.report.inputSourceName").to_string()); + Some((NamedSource::new(name, map.text().to_string()), LabeledSpan::new(None, span.start, span.len()))) + }); + let (source, label) = match located { + Some((source, label)) => (Some(source), Some(label)), + None => (None, None), + }; + Self { + message: diagnostic.message.clone(), + rule: diagnostic.rule.clone(), + severity, + help, + source, + label, + } + } +} + +impl Display for LintDiagnosticReport { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + write!(f, "{}", self.message) + } +} + +impl std::error::Error for LintDiagnosticReport {} + +impl Diagnostic for LintDiagnosticReport { + fn code<'a>(&'a self) -> Option> { + Some(Box::new(&self.rule)) + } + + fn severity(&self) -> Option { + Some(self.severity) + } + + fn help<'a>(&'a self) -> Option> { + self.help.as_ref().map(|help| Box::new(help) as Box) + } + + fn source_code(&self) -> Option<&dyn SourceCode> { + self.source.as_ref().map(|source| source as &dyn SourceCode) + } + + fn labels(&self) -> Option + '_>> { + self.label.as_ref().map(|label| Box::new(std::iter::once(label.clone())) as Box + '_>) + } +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::lint::diagnostic::LintCategory; + + fn diagnostic(path: &str, level: LintLevel) -> LintDiagnostic { + LintDiagnostic { + level, + rule: "duplicate-resource".to_string(), + category: LintCategory::Document, + message: "declared twice".to_string(), + path: path.to_string(), + file: Some("demo.dsc.yaml".to_string()), + location: None, + } + } + + #[test] + fn report_labels_the_path_in_the_document() { + let text = "resources:\n- name: a\n type: T/E\n- name: a\n type: T/E\n"; + let map = SourceMap::parse(text).unwrap(); + let report = LintDiagnosticReport::new(&diagnostic("/resources/1", LintLevel::Error), Some(&map), Some("Names must be unique.".to_string())); + assert_eq!(report.code().unwrap().to_string(), "duplicate-resource"); + assert_eq!(report.severity(), Some(Severity::Error)); + assert_eq!(report.help().unwrap().to_string(), "Names must be unique."); + assert_eq!(report.to_string(), "declared twice"); + let label = report.labels().unwrap().next().unwrap(); + assert_eq!(&text[label.offset()..label.offset() + label.len()], "name: a"); + assert_eq!(label.offset(), text.rfind("name: a").unwrap()); + assert!(report.source_code().is_some()); + } + + #[test] + fn report_without_text_has_no_snippet() { + let report = LintDiagnosticReport::new(&diagnostic("/resources/0", LintLevel::Info), None, None); + assert_eq!(report.severity(), Some(Severity::Advice)); + assert!(report.labels().is_none()); + assert!(report.source_code().is_none()); + assert!(report.help().is_none()); + } + + #[test] + fn redacted_source_hides_secure_values_and_keeps_locations() { + let text = "parameters:\n password:\n type: secureString\n defaultValue: hunter2\n blob:\n type: secureObject\n defaultValue:\n key: letmein\n plain:\n type: string\n defaultValue: visible\nresources:\n- name: a\n type: T/E\n properties:\n token:\n secureString: topsecret\n output: hi\n"; + let map = redacted_source_map(text).unwrap(); + for secret in ["hunter2", "letmein", "topsecret"] { + assert!(!map.text().contains(secret), "{}", map.text()); + } + assert!(map.text().contains("visible")); + assert_eq!(&map.text()[map.value_span("/parameters/password/defaultValue").unwrap()], "\"\""); + assert_eq!(&map.text()[map.value_span("/parameters/blob/defaultValue").unwrap()], "\"\""); + assert_eq!(&map.text()[map.value_span("/resources/0/properties/token").unwrap()], "\"\""); + let report = LintDiagnosticReport::new(&diagnostic("/parameters/password/defaultValue", LintLevel::Warning), Some(&map), None); + let label = report.labels().unwrap().next().unwrap(); + assert_eq!(&map.text()[label.offset()..label.offset() + label.len()], "defaultValue"); + assert_eq!(map.location(label.offset()), (4, 5)); + } + + #[test] + fn redacted_source_handles_json_documents() { + let text = r#"{"parameters": {"password": {"type": "secureString", "defaultValue": "hunter2"}}, "resources": [{"name": "a", "type": "T/E", "properties": {"token": {"secureObject": {"k": "letmein"}}}}]}"#; + let map = redacted_source_map(text).unwrap(); + assert!(!map.text().contains("hunter2") && !map.text().contains("letmein"), "{}", map.text()); + assert!(map.value_span("/resources/0/properties/token").is_some()); + assert!(map.value_span("/resources/0/name").is_some()); + } + + #[test] + fn text_without_secure_values_is_unchanged_and_unmappable_text_has_no_map() { + let text = "resources:\n- name: a\n type: T/E\n"; + assert_eq!(redacted_source_map(text).unwrap().text(), text); + assert!(redacted_source_map("a: [b").is_none()); + } +} diff --git a/lib/dsc-lib/src/lint/rule.rs b/lib/dsc-lib/src/lint/rule.rs new file mode 100644 index 000000000..da6f06882 --- /dev/null +++ b/lib/dsc-lib/src/lint/rule.rs @@ -0,0 +1,158 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +use schemars::JsonSchema; +use serde::{Deserialize, Serialize}; + +use crate::schemas::{dsc_repo::DscRepoSchema, transforms::idiomaticize_string_enum}; + +use super::{ + context::LintContext, + diagnostic::{LintCategory, LintDiagnostic, LintLevel, SourceLocation}, + options::RuleLevel, + pointer::JsonPointer, +}; + +/// When a rule can run. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Deserialize, Serialize, JsonSchema)] +#[serde(rename_all = "camelCase")] +#[schemars(inline, transform = idiomaticize_string_enum)] +pub enum RuleTier { + /// The rule only needs the document. + Static, + /// The rule needs the installed resources, so it only runs when discovery is enabled. + Resource, +} + +impl std::fmt::Display for RuleTier { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + let text = match self { + RuleTier::Static => "static", + RuleTier::Resource => "resource", + }; + write!(f, "{text}") + } +} + +/// The signature of a rule's check function. +pub type CheckFn = fn(&LintContext<'_>, &mut DiagnosticSink<'_>); + +/// A lint rule. +pub struct Rule { + /// The kebab-case identifier of the rule, such as `duplicate-resource`. + pub id: &'static str, + /// The area of the document the rule inspects. + pub category: LintCategory, + /// When the rule can run. + pub tier: RuleTier, + /// The level the rule reports at unless overridden. + pub default_level: RuleLevel, + /// Returns the localized description of what the rule checks. + pub description: fn() -> String, + /// Runs the rule against a document, reporting problems to the sink. + pub check: CheckFn, +} + +/// Describes a rule and the level it is configured to report at. +#[derive(Debug, Clone, PartialEq, Eq, Deserialize, Serialize, JsonSchema, DscRepoSchema)] +#[serde(deny_unknown_fields, rename_all = "camelCase")] +#[dsc_repo_schema(base_name = "rules", folder_path = "outputs/lint")] +#[schemars( + transform = LintRuleInfo::transform_export_schema_uris, + transform = LintRuleInfo::transform_schema_docs +)] +pub struct LintRuleInfo { + /// The identifier of the rule. + pub id: String, + /// The area of the document the rule inspects. + pub category: LintCategory, + /// When the rule can run. + pub tier: RuleTier, + /// The level the rule reports at unless overridden. + pub default_level: RuleLevel, + /// The level the rule is configured to report at. + pub level: RuleLevel, + /// Whether the level of the rule can't be changed. + pub fixed: bool, + /// What the rule checks. + pub description: String, +} + +impl LintRuleInfo { + /// Describe a rule at its configured level. + /// + /// # Arguments + /// + /// * `rule` - The rule. + /// * `level` - The level the rule is configured to report at. + #[must_use] + pub fn from_rule(rule: &Rule, level: RuleLevel) -> Self { + Self { + id: rule.id.to_string(), + category: rule.category, + tier: rule.tier, + default_level: rule.default_level, + level, + fixed: false, + description: (rule.description)(), + } + } +} + +/// Collects the diagnostics one rule reports, stamping each with the rule's identity and level +/// so a rule can't report under another name. +pub struct DiagnosticSink<'a> { + rule: &'static str, + category: LintCategory, + level: LintLevel, + diagnostics: &'a mut Vec, +} + +impl<'a> DiagnosticSink<'a> { + /// Create a sink for one rule. + /// + /// # Arguments + /// + /// * `rule` - The identifier of the rule that reports through this sink. + /// * `category` - The category of the rule. + /// * `level` - The level the rule reports at. + /// * `diagnostics` - Where reported diagnostics are appended. + #[must_use] + pub fn new(rule: &'static str, category: LintCategory, level: LintLevel, diagnostics: &'a mut Vec) -> Self { + Self { + rule, + category, + level, + diagnostics, + } + } + + /// Report a problem at a location in the document. + /// + /// # Arguments + /// + /// * `path` - The location of the problem. + /// * `message` - A description of the problem. + pub fn report(&mut self, path: &JsonPointer, message: String) { + self.report_at(path, None, message); + } + + /// Report a problem at a location in the document with a position in the source text. + /// + /// # Arguments + /// + /// * `path` - The location of the problem. + /// * `location` - The position in the source text, when known. + /// * `message` - A description of the problem. + pub fn report_at(&mut self, path: &JsonPointer, location: Option, message: String) { + self.diagnostics.push(LintDiagnostic { + level: self.level, + rule: self.rule.to_string(), + category: self.category, + message, + path: path.to_string(), + file: None, + location, + }); + } +} diff --git a/lib/dsc-lib/src/lint/rules/dependency.rs b/lib/dsc-lib/src/lint/rules/dependency.rs new file mode 100644 index 000000000..b5adaa2d1 --- /dev/null +++ b/lib/dsc-lib/src/lint/rules/dependency.rs @@ -0,0 +1,543 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +//! Rules about the dependencies between resources. + +use std::collections::{HashMap, HashSet}; + +use rust_i18n::t; +use serde_json::Value; + +use crate::{ + lint::{ + context::LintContext, + diagnostic::LintCategory, + expression_index::ExpressionScope, + options::RuleLevel, + pointer::JsonPointer, + rule::{DiagnosticSink, Rule, RuleTier}, + }, + parser::{expressions::Expression, functions::FunctionArg, ParsedStatement}, +}; + +/// The rules in this module. +#[must_use] +pub fn rules() -> Vec { + vec![ + Rule { + id: "unresolved-dependency", + category: LintCategory::Dependency, + tier: RuleTier::Static, + default_level: RuleLevel::Error, + description: || t!("lint.rules.dependency.unresolvedDependencyDescription").to_string(), + check: unresolved_dependency, + }, + Rule { + id: "circular-dependency", + category: LintCategory::Dependency, + tier: RuleTier::Static, + default_level: RuleLevel::Error, + description: || t!("lint.rules.dependency.circularDependencyDescription").to_string(), + check: circular_dependency, + }, + Rule { + id: "reference-without-depends-on", + category: LintCategory::Dependency, + tier: RuleTier::Static, + default_level: RuleLevel::Warning, + description: || t!("lint.rules.dependency.referenceWithoutDependsOnDescription").to_string(), + check: reference_without_depends_on, + }, + Rule { + id: "duplicate-depends-on", + category: LintCategory::Dependency, + tier: RuleTier::Static, + default_level: RuleLevel::Info, + description: || t!("lint.rules.dependency.duplicateDependsOnDescription").to_string(), + check: duplicate_depends_on, + }, + ] +} + +/// Identifies a resource by type and name, as `resourceId()` does. +#[derive(Clone, Debug)] +struct ResourceKey { + /// The resource type as written; compared case-insensitively. + type_name: String, + /// The resource name; compared exactly. + name: String, +} + +impl ResourceKey { + /// Whether this key identifies the same resource as another. + /// + /// # Arguments + /// + /// * `other` - The key to compare with. + fn same_as(&self, other: &ResourceKey) -> bool { + self.name == other.name && self.type_name.eq_ignore_ascii_case(&other.type_name) + } + + /// The key as `type/name`, for messages. + fn display(&self) -> String { + format!("{} named '{}'", self.type_name, self.name) + } +} + +/// How a dependency key resolved against the sibling resources. +enum Resolution { + /// The resource at the given index matches by name and type. + Found(usize), + /// A resource at the given index has the name but a different type. + TypeMismatch(usize), + /// No sibling has the name. + NotFound, +} + +/// The key of a `resourceId('type', 'name')` expression with literal arguments and no accessors. +/// +/// # Arguments +/// +/// * `expression` - The expression. +fn resource_id_key(expression: &Expression) -> Option { + if !expression.accessors().is_empty() || expression.function().name() != "resourceId" { + return None; + } + let args = expression.function().args()?; + match args { + [FunctionArg::Value(Value::String(type_name)), FunctionArg::Value(Value::String(name))] => Some(ResourceKey { + type_name: type_name.clone(), + name: name.clone(), + }), + _ => None, + } +} + +/// The literal `dependsOn` entries of a resource with their pointers. +/// +/// Entries that aren't a literal `resourceId()` call are skipped; the schema already rejects +/// entries that aren't `resourceId()` expressions. +/// +/// # Arguments +/// +/// * `context` - The lint context. +/// * `index` - The index of the resource in the context's resource list. +fn literal_dependencies(context: &LintContext, index: usize) -> Vec<(JsonPointer, ResourceKey)> { + let site = &context.resources[index]; + let Some(depends_on) = &site.resource.depends_on else { + return Vec::new(); + }; + let base = site.pointer.push("dependsOn"); + (0..depends_on.len()).filter_map(|position| { + let pointer = base.push_index(position); + let parsed = context.expressions.site_at(&pointer)?.parsed.as_ref().ok()?; + let ParsedStatement::Expression(expression) = parsed else { + return None; + }; + resource_id_key(expression).map(|key| (pointer, key)) + }).collect() +} + +/// The resources a resource references through `reference(resourceId('type', 'name'))`, with +/// the pointer of each referencing expression. +/// +/// # Arguments +/// +/// * `context` - The lint context. +/// * `index` - The index of the resource in the context's resource list. +fn reference_targets(context: &LintContext, index: usize) -> Vec<(JsonPointer, ResourceKey)> { + context.expressions.calls_named("reference").into_iter() + .filter(|call| call.site.scope == ExpressionScope::Resource(index)) + .filter_map(|call| { + let Some([FunctionArg::Expression(argument), ..]) = call.function.args() else { + return None; + }; + resource_id_key(argument).map(|key| (call.site.pointer.clone(), key)) + }) + .collect() +} + +/// Resolve a dependency key against a group of sibling resources by their names as written, +/// which is how DSC resolves `dependsOn`. +/// +/// # Arguments +/// +/// * `context` - The lint context. +/// * `group` - The indexes of the sibling resources. +/// * `key` - The dependency key. +fn resolve(context: &LintContext, group: &[usize], key: &ResourceKey) -> Resolution { + resolve_by(context, group, key, |index| Some(context.resources[index].resource.name.clone())) +} + +/// Resolve a `reference()` key against a group of sibling resources by their evaluated names, +/// which is how DSC resolves references. +/// +/// # Arguments +/// +/// * `context` - The lint context. +/// * `group` - The indexes of the sibling resources. +/// * `key` - The referenced key. +/// +/// # Returns +/// +/// The resolution, or `None` when the key isn't found and a sibling's name is an expression +/// that could evaluate to it. +fn resolve_reference(context: &LintContext, group: &[usize], key: &ResourceKey) -> Option { + let resolution = resolve_by(context, group, key, |index| evaluated_name(context, index)); + let has_dynamic_name = group.iter().any(|&index| evaluated_name(context, index).is_none()); + match resolution { + Resolution::Found(_) => Some(resolution), + _ if has_dynamic_name => None, + _ => Some(resolution), + } +} + +/// The name a resource has at runtime, when it is known without evaluating an expression. +/// +/// # Arguments +/// +/// * `context` - The lint context. +/// * `index` - The index of the resource in the context's resource list. +/// +/// # Returns +/// +/// The name as written, the name without the escape character when it starts with `[[`, or +/// `None` when the name is an expression. +fn evaluated_name(context: &LintContext, index: usize) -> Option { + let name = &context.resources[index].resource.name; + if LintContext::is_expression(name) { + None + } else if let Some(unescaped) = name.strip_prefix('[') { + Some(unescaped.to_string()) + } else { + Some(name.clone()) + } +} + +/// Resolve a dependency key against a group of sibling resources. +/// +/// # Arguments +/// +/// * `context` - The lint context. +/// * `group` - The indexes of the sibling resources. +/// * `key` - The dependency key. +/// * `name_of` - The name to match for a sibling, or `None` when it isn't known. +fn resolve_by(context: &LintContext, group: &[usize], key: &ResourceKey, name_of: impl Fn(usize) -> Option) -> Resolution { + let by_name: Vec = group.iter().copied().filter(|&index| name_of(index).is_some_and(|name| name == key.name)).collect(); + if let Some(&index) = by_name.iter().find(|&&index| context.resources[index].resource.resource_type.as_ref().eq_ignore_ascii_case(&key.type_name)) { + return Resolution::Found(index); + } + match by_name.first() { + Some(&index) => Resolution::TypeMismatch(index), + None => Resolution::NotFound, + } +} + +/// Report `dependsOn` entries and `reference()` calls whose target isn't a sibling resource. +/// +/// Resources in copy loops are skipped, and a missing target isn't reported when a sibling is +/// a copy loop, because copy loops produce names that are only known at runtime. For the same +/// reason a `reference()` call isn't reported when a sibling's name is an expression. +fn unresolved_dependency(context: &LintContext, sink: &mut DiagnosticSink) { + for group in context.sibling_groups() { + let has_copy = group.iter().any(|&index| context.resources[index].resource.copy.is_some()); + for &index in &group { + let site = &context.resources[index]; + if site.resource.copy.is_some() { + continue; + } + let dependencies = literal_dependencies(context, index).into_iter().map(|(pointer, key)| { + let resolution = resolve(context, &group, &key); + (pointer, key, Some(resolution)) + }); + let references = reference_targets(context, index).into_iter().map(|(pointer, key)| { + let resolution = resolve_reference(context, &group, &key); + (pointer, key, resolution) + }); + for (pointer, key, resolution) in dependencies.chain(references) { + let Some(resolution) = resolution else { + continue; + }; + match resolution { + Resolution::Found(_) => {}, + Resolution::TypeMismatch(actual) => sink.report( + &pointer, + t!( + "lint.rules.dependency.dependencyTypeMismatch", + name = site.resource.name, + dependency_type = key.type_name, + dependency_name = key.name, + actual_type = context.resources[actual].resource.resource_type + ).to_string(), + ), + Resolution::NotFound => { + if !has_copy { + sink.report( + &pointer, + t!( + "lint.rules.dependency.unresolvedDependency", + name = site.resource.name, + dependency_type = key.type_name, + dependency_name = key.name + ).to_string(), + ); + } + }, + } + } + } + } +} + +/// Report resources whose `dependsOn` entries form a cycle. +fn circular_dependency(context: &LintContext, sink: &mut DiagnosticSink) { + for group in context.sibling_groups() { + let mut edges: HashMap> = HashMap::new(); + for &index in &group { + if context.resources[index].resource.copy.is_some() { + continue; + } + let targets = literal_dependencies(context, index).into_iter().filter_map(|(_, key)| match resolve(context, &group, &key) { + Resolution::Found(target) => Some(target), + _ => None, + }).collect(); + edges.insert(index, targets); + } + + let mut reported: HashSet> = HashSet::new(); + for &start in &group { + let Some(cycle) = find_cycle(start, &edges) else { + continue; + }; + let mut members = cycle.clone(); + members.sort_unstable(); + if !reported.insert(members) { + continue; + } + let site = &context.resources[start]; + let path: Vec<&str> = cycle.iter().chain(std::iter::once(&start)).map(|&index| context.resources[index].resource.name.as_str()).collect(); + sink.report( + &site.pointer.push("dependsOn"), + t!("lint.rules.dependency.circularDependency", name = site.resource.name, cycle = path.join(" -> ")).to_string(), + ); + } + } +} + +/// Find a dependency path that leads from a resource back to itself. +/// +/// # Arguments +/// +/// * `start` - The resource to start from. +/// * `edges` - The resolved dependencies of each resource. +/// +/// # Returns +/// +/// The resources on the cycle, starting with `start`, or `None` when there is no cycle. +fn find_cycle(start: usize, edges: &HashMap>) -> Option> { + let mut path = Vec::new(); + let mut visited = HashSet::new(); + if walk(start, start, edges, &mut path, &mut visited) { Some(path) } else { None } +} + +/// Depth-first search for a path from `node` back to `start`. +/// +/// # Arguments +/// +/// * `node` - The current resource. +/// * `start` - The resource the search is looking for. +/// * `edges` - The resolved dependencies of each resource. +/// * `path` - The resources on the current path; holds the cycle when the search succeeds. +/// * `visited` - The resources already explored. +fn walk(node: usize, start: usize, edges: &HashMap>, path: &mut Vec, visited: &mut HashSet) -> bool { + path.push(node); + visited.insert(node); + for &next in edges.get(&node).map(Vec::as_slice).unwrap_or(&[]) { + if next == start { + return true; + } + if !visited.contains(&next) && walk(next, start, edges, path, visited) { + return true; + } + } + path.pop(); + false +} + +/// Report `reference()` calls to resources that aren't listed in `dependsOn`. +fn reference_without_depends_on(context: &LintContext, sink: &mut DiagnosticSink) { + for index in 0..context.resources.len() { + let site = &context.resources[index]; + let dependencies = literal_dependencies(context, index); + for (pointer, target) in reference_targets(context, index) { + if !dependencies.iter().any(|(_, dependency)| dependency.same_as(&target)) { + sink.report( + &pointer, + t!("lint.rules.dependency.referenceWithoutDependsOn", name = site.resource.name, target = target.display()).to_string(), + ); + } + } + } +} + +/// Report `dependsOn` entries that repeat an earlier entry. +fn duplicate_depends_on(context: &LintContext, sink: &mut DiagnosticSink) { + for index in 0..context.resources.len() { + let site = &context.resources[index]; + let mut seen: Vec = Vec::new(); + for (pointer, key) in literal_dependencies(context, index) { + if seen.iter().any(|earlier| earlier.same_as(&key)) { + sink.report( + &pointer, + t!("lint.rules.dependency.duplicateDependsOn", name = site.resource.name, target = key.display()).to_string(), + ); + } else { + seen.push(key); + } + } + } +} + +#[cfg(test)] +mod tests { + use crate::lint::test_util::{lint_yaml, rule_messages, rule_paths}; + + #[test] + fn reports_unresolved_and_mismatched_dependencies() { + let result = lint_yaml(r#" +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +resources: +- name: first + type: Microsoft.DSC.Debug/Echo +- name: second + type: Microsoft.DSC.Debug/Echo + dependsOn: + - "[resourceId('microsoft.dsc.debug/echo', 'first')]" + - "[resourceId('Microsoft.DSC.Debug/Echo', 'missing')]" + - "[resourceId('Microsoft/OSInfo', 'first')]" + properties: + output: "[reference(resourceId('Microsoft.DSC.Debug/Echo', 'nobody'))]" +"#); + assert_eq!( + rule_paths(&result, "unresolved-dependency"), + vec!["/resources/1/dependsOn/1", "/resources/1/dependsOn/2", "/resources/1/properties/output"] + ); + assert!(rule_messages(&result, "unresolved-dependency")[1].contains("Microsoft.DSC.Debug/Echo")); + } + + #[test] + fn references_resolve_against_evaluated_names() { + let result = lint_yaml(r#" +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +parameters: + name: + type: string + defaultValue: source +resources: +- name: "[parameters('name')]" + type: Microsoft.DSC.Debug/Echo +- name: consumer + type: Microsoft.DSC.Debug/Echo + dependsOn: + - "[resourceId('Microsoft.DSC.Debug/Echo', 'source')]" + properties: + output: "[reference(resourceId('Microsoft.DSC.Debug/Echo', 'source'))]" +"#); + // dependsOn matches names as written, so only it is reported + assert_eq!(rule_paths(&result, "unresolved-dependency"), vec!["/resources/1/dependsOn/0"]); + + let result = lint_yaml(r#" +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +resources: +- name: "[[bracketed]" + type: Microsoft.DSC.Debug/Echo +- name: consumer + type: Microsoft.DSC.Debug/Echo + properties: + found: "[reference(resourceId('Microsoft.DSC.Debug/Echo', '[bracketed]'))]" + missing: "[reference(resourceId('Microsoft.DSC.Debug/Echo', '[[bracketed]'))]" +"#); + assert_eq!(rule_paths(&result, "unresolved-dependency"), vec!["/resources/1/properties/missing"]); + } + + #[test] + fn copy_loops_suppress_unresolved_dependencies() { + let result = lint_yaml(r#" +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +resources: +- name: "[concat('item', copyIndex())]" + type: Microsoft.DSC.Debug/Echo + copy: + name: items + count: 2 +- name: consumer + type: Microsoft.DSC.Debug/Echo + dependsOn: + - "[resourceId('Microsoft.DSC.Debug/Echo', 'item0')]" +"#); + assert!(rule_paths(&result, "unresolved-dependency").is_empty()); + } + + #[test] + fn reports_cycles_once() { + let result = lint_yaml(r#" +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +resources: +- name: a + type: Microsoft.DSC.Debug/Echo + dependsOn: + - "[resourceId('Microsoft.DSC.Debug/Echo', 'b')]" +- name: b + type: Microsoft.DSC.Debug/Echo + dependsOn: + - "[resourceId('Microsoft.DSC.Debug/Echo', 'a')]" +- name: self + type: Microsoft.DSC.Debug/Echo + dependsOn: + - "[resourceId('Microsoft.DSC.Debug/Echo', 'self')]" +- name: fine + type: Microsoft.DSC.Debug/Echo + dependsOn: + - "[resourceId('Microsoft.DSC.Debug/Echo', 'a')]" +"#); + assert_eq!(rule_paths(&result, "circular-dependency"), vec!["/resources/0/dependsOn", "/resources/2/dependsOn"]); + assert!(rule_messages(&result, "circular-dependency")[0].contains("a -> b -> a")); + } + + #[test] + fn reports_references_without_depends_on() { + let result = lint_yaml(r#" +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +resources: +- name: source + type: Microsoft.DSC.Debug/Echo +- name: listed + type: Microsoft.DSC.Debug/Echo + dependsOn: + - "[resourceId('Microsoft.DSC.Debug/Echo', 'source')]" + properties: + output: "[reference(resourceId('Microsoft.DSC.Debug/Echo', 'source')).output]" +- name: unlisted + type: Microsoft.DSC.Debug/Echo + properties: + output: "[reference(resourceId('Microsoft.DSC.Debug/Echo', 'source')).output]" +"#); + assert_eq!(rule_paths(&result, "reference-without-depends-on"), vec!["/resources/2/properties/output"]); + } + + #[test] + fn reports_duplicate_depends_on_entries() { + let result = lint_yaml(r#" +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +resources: +- name: source + type: Microsoft.DSC.Debug/Echo +- name: twice + type: Microsoft.DSC.Debug/Echo + dependsOn: + - "[resourceId('Microsoft.DSC.Debug/Echo', 'source')]" + - "[resourceId('Microsoft.DSC.Debug/Echo', 'source')]" +"#); + assert_eq!(rule_paths(&result, "duplicate-depends-on"), vec!["/resources/1/dependsOn/1"]); + } +} diff --git a/lib/dsc-lib/src/lint/rules/deprecation.rs b/lib/dsc-lib/src/lint/rules/deprecation.rs new file mode 100644 index 000000000..070bdec7b --- /dev/null +++ b/lib/dsc-lib/src/lint/rules/deprecation.rs @@ -0,0 +1,206 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +//! Rules about features DSC has deprecated or doesn't support. + +use rust_i18n::t; + +use crate::{ + configure::config_doc::ValueOrCopy, + lint::{ + context::LintContext, + diagnostic::LintCategory, + options::RuleLevel, + pointer::JsonPointer, + rule::{DiagnosticSink, Rule, RuleTier}, + }, +}; + +/// The rules in this module. +#[must_use] +pub fn rules() -> Vec { + vec![ + Rule { + id: "copy-loop-deprecated", + category: LintCategory::Deprecation, + tier: RuleTier::Static, + default_level: RuleLevel::Warning, + description: || t!("lint.rules.deprecation.copyLoopDeprecatedDescription").to_string(), + check: copy_loop_deprecated, + }, + Rule { + id: "copy-options-unsupported", + category: LintCategory::Deprecation, + tier: RuleTier::Static, + default_level: RuleLevel::Error, + description: || t!("lint.rules.deprecation.copyOptionsUnsupportedDescription").to_string(), + check: copy_options_unsupported, + }, + Rule { + id: "security-context-in-metadata", + category: LintCategory::Deprecation, + tier: RuleTier::Static, + default_level: RuleLevel::Warning, + description: || t!("lint.rules.deprecation.securityContextInMetadataDescription").to_string(), + check: security_context_in_metadata, + }, + Rule { + id: "security-context-conflict", + category: LintCategory::Deprecation, + tier: RuleTier::Static, + default_level: RuleLevel::Error, + description: || t!("lint.rules.deprecation.securityContextConflictDescription").to_string(), + check: security_context_conflict, + }, + Rule { + id: "dsc-version-unsatisfied", + category: LintCategory::Deprecation, + tier: RuleTier::Static, + default_level: RuleLevel::Warning, + description: || t!("lint.rules.deprecation.dscVersionUnsatisfiedDescription").to_string(), + check: dsc_version_unsatisfied, + }, + ] +} + +/// Report copy loops on resources and outputs. +fn copy_loop_deprecated(context: &LintContext, sink: &mut DiagnosticSink) { + for site in &context.resources { + if let Some(copy) = &site.resource.copy { + sink.report(&site.pointer.push("copy"), t!("lint.rules.deprecation.copyLoopDeprecated", name = copy.name).to_string()); + } + } + if let Some(outputs) = &context.config.outputs { + for (name, output) in outputs { + if let ValueOrCopy::Copy(copy) = &output.value_or_copy { + sink.report( + &JsonPointer::root().push("outputs").push(name).push("copy"), + t!("lint.rules.deprecation.copyLoopDeprecated", name = copy.name).to_string(), + ); + } + } + } +} + +/// Report copy loop options that DSC rejects. +fn copy_options_unsupported(context: &LintContext, sink: &mut DiagnosticSink) { + for site in &context.resources { + let Some(copy) = &site.resource.copy else { + continue; + }; + let base = site.pointer.push("copy"); + if copy.mode.is_some() { + sink.report(&base.push("mode"), t!("lint.rules.deprecation.copyOptionsUnsupported", option = "mode").to_string()); + } + if copy.batch_size.is_some() { + sink.report(&base.push("batchSize"), t!("lint.rules.deprecation.copyOptionsUnsupported", option = "batchSize").to_string()); + } + } +} + +/// Report a security context set through `Microsoft.DSC` metadata. +fn security_context_in_metadata(context: &LintContext, sink: &mut DiagnosticSink) { + if metadata_security_context(context).is_some() { + sink.report( + &JsonPointer::root().push("metadata").push("Microsoft.DSC").push("securityContext"), + t!("lint.rules.deprecation.securityContextInMetadata").to_string(), + ); + } +} + +/// Report a security context in metadata that contradicts the directive. +fn security_context_conflict(context: &LintContext, sink: &mut DiagnosticSink) { + let directive = context.config.directives.as_ref().and_then(|directives| directives.security_context.as_ref()); + if let (Some(metadata), Some(directive)) = (metadata_security_context(context), directive) + && metadata != directive { + sink.report( + &JsonPointer::root().push("directives").push("securityContext"), + t!("lint.rules.deprecation.securityContextConflict", metadata = metadata, directive = directive).to_string(), + ); + } +} + +/// Report a `directives.version` requirement the linting DSC doesn't satisfy. +fn dsc_version_unsatisfied(context: &LintContext, sink: &mut DiagnosticSink) { + let Some(current) = &context.options.dsc_version else { + return; + }; + let Some(required) = context.config.directives.as_ref().and_then(|directives| directives.version.as_ref()) else { + return; + }; + if !required.matches(current) { + sink.report( + &JsonPointer::root().push("directives").push("version"), + t!("lint.rules.deprecation.dscVersionUnsatisfied", required = required, current = current).to_string(), + ); + } +} + +/// The security context set through `Microsoft.DSC` metadata, if any. +/// +/// # Arguments +/// +/// * `context` - The lint context. +fn metadata_security_context<'a>(context: &'a LintContext<'_>) -> Option<&'a crate::configure::config_doc::SecurityContextKind> { + context.config.metadata.as_ref()?.microsoft.as_ref()?.security_context.as_ref() +} + +#[cfg(test)] +mod tests { + use crate::{lint::{options::LintOptions, test_util::{lint_with, lint_yaml, rule_paths}}, types::SemanticVersion}; + + #[test] + fn reports_copy_loops_and_their_options() { + let result = lint_yaml(r#" +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +outputs: + items: + type: array + copy: + name: outer + count: 2 +resources: +- name: "[concat('a', copyIndex())]" + type: Microsoft.DSC.Debug/Echo + copy: + name: loop + count: 2 + mode: serial + batchSize: 1 +"#); + assert_eq!(rule_paths(&result, "copy-loop-deprecated"), vec!["/outputs/items/copy", "/resources/0/copy"]); + assert_eq!(rule_paths(&result, "copy-options-unsupported"), vec!["/resources/0/copy/batchSize", "/resources/0/copy/mode"]); + } + + #[test] + fn reports_security_context_in_metadata_and_conflicts() { + let result = lint_yaml(r#" +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +metadata: + Microsoft.DSC: + securityContext: elevated +directives: + securityContext: restricted +resources: [] +"#); + assert_eq!(rule_paths(&result, "security-context-in-metadata"), vec!["/metadata/Microsoft.DSC/securityContext"]); + assert_eq!(rule_paths(&result, "security-context-conflict"), vec!["/directives/securityContext"]); + } + + #[test] + fn reports_unsatisfied_dsc_version() { + let options = LintOptions { + discovery: false, + use_settings: false, + dsc_version: Some(SemanticVersion::parse("3.1.0").unwrap()), + ..Default::default() + }; + let result = lint_with(r#" +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +directives: + version: ">=4.0.0" +resources: [] +"#, options); + assert_eq!(rule_paths(&result, "dsc-version-unsatisfied"), vec!["/directives/version"]); + } +} diff --git a/lib/dsc-lib/src/lint/rules/document.rs b/lib/dsc-lib/src/lint/rules/document.rs new file mode 100644 index 000000000..05820ecbf --- /dev/null +++ b/lib/dsc-lib/src/lint/rules/document.rs @@ -0,0 +1,364 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +//! Rules about the structure of a configuration document and its resource declarations. + +use rust_i18n::t; +use serde_json::Value; + +use crate::{ + lint::{ + context::LintContext, + diagnostic::LintCategory, + options::RuleLevel, + pointer::JsonPointer, + rule::{DiagnosticSink, Rule, RuleTier}, + }, + types::ResourceVersionReq, +}; + +/// The keys DSC recognizes under `Microsoft.DSC` metadata. +const MICROSOFT_DSC_METADATA_KEYS: [&str; 9] = [ + "duration", + "endDatetime", + "executionType", + "operation", + "restartRequired", + "copyLoops", + "securityContext", + "startDatetime", + "version", +]; + +/// The rules in this module. +#[must_use] +pub fn rules() -> Vec { + vec![ + Rule { + id: "duplicate-resource", + category: LintCategory::Document, + tier: RuleTier::Static, + default_level: RuleLevel::Error, + description: || t!("lint.rules.document.duplicateResourceDescription").to_string(), + check: duplicate_resource, + }, + Rule { + id: "duplicate-resource-name", + category: LintCategory::Document, + tier: RuleTier::Static, + default_level: RuleLevel::Warning, + description: || t!("lint.rules.document.duplicateResourceNameDescription").to_string(), + check: duplicate_resource_name, + }, + Rule { + id: "empty-resources", + category: LintCategory::Document, + tier: RuleTier::Static, + default_level: RuleLevel::Warning, + description: || t!("lint.rules.document.emptyResourcesDescription").to_string(), + check: empty_resources, + }, + Rule { + id: "unknown-microsoft-dsc-metadata", + category: LintCategory::Document, + tier: RuleTier::Static, + default_level: RuleLevel::Warning, + description: || t!("lint.rules.document.unknownMicrosoftDscMetadataDescription").to_string(), + check: unknown_microsoft_dsc_metadata, + }, + Rule { + id: "api-version-alias", + category: LintCategory::Document, + tier: RuleTier::Static, + default_level: RuleLevel::Info, + description: || t!("lint.rules.document.apiVersionAliasDescription").to_string(), + check: api_version_alias, + }, + Rule { + id: "date-version", + category: LintCategory::Document, + tier: RuleTier::Static, + default_level: RuleLevel::Warning, + description: || t!("lint.rules.document.dateVersionDescription").to_string(), + check: date_version, + }, + Rule { + id: "missing-require-version", + category: LintCategory::Document, + tier: RuleTier::Static, + default_level: RuleLevel::Off, + description: || t!("lint.rules.document.missingRequireVersionDescription").to_string(), + check: missing_require_version, + }, + ] +} + +/// Report resources that share a name and a type with an earlier sibling. +/// +/// Resources in copy loops are skipped because their names are only known at runtime. +fn duplicate_resource(context: &LintContext, sink: &mut DiagnosticSink) { + for group in context.sibling_groups() { + for (position, &later) in group.iter().enumerate() { + let later_site = &context.resources[later]; + if later_site.resource.copy.is_some() { + continue; + } + let duplicate = group[..position].iter().any(|&earlier| { + let earlier_site = &context.resources[earlier]; + earlier_site.resource.copy.is_none() + && earlier_site.resource.name == later_site.resource.name + && earlier_site.resource.resource_type == later_site.resource.resource_type + }); + if duplicate { + sink.report( + &later_site.pointer, + t!( + "lint.rules.document.duplicateResource", + name = later_site.resource.name, + type_name = later_site.resource.resource_type + ).to_string(), + ); + } + } + } +} + +/// Report resources that share a name with an earlier sibling of a different type. +fn duplicate_resource_name(context: &LintContext, sink: &mut DiagnosticSink) { + for group in context.sibling_groups() { + for (position, &later) in group.iter().enumerate() { + let later_site = &context.resources[later]; + if later_site.resource.copy.is_some() { + continue; + } + let same_name = group[..position].iter().map(|&earlier| &context.resources[earlier]).find(|earlier_site| { + earlier_site.resource.copy.is_none() + && earlier_site.resource.name == later_site.resource.name + && earlier_site.resource.resource_type != later_site.resource.resource_type + }); + if let Some(earlier_site) = same_name { + sink.report( + &later_site.pointer, + t!( + "lint.rules.document.duplicateResourceName", + name = later_site.resource.name, + first_type = earlier_site.resource.resource_type, + second_type = later_site.resource.resource_type + ).to_string(), + ); + } + } + } +} + +/// Report a configuration without resources. +fn empty_resources(context: &LintContext, sink: &mut DiagnosticSink) { + if context.config.resources.is_empty() { + sink.report(&JsonPointer::root().push("resources"), t!("lint.rules.document.emptyResources").to_string()); + } +} + +/// Report unrecognized keys under `Microsoft.DSC` metadata on the document and on each resource. +fn unknown_microsoft_dsc_metadata(context: &LintContext, sink: &mut DiagnosticSink) { + check_microsoft_dsc_metadata(context.raw, &JsonPointer::root(), sink); + for (index, site) in context.resources.iter().enumerate() { + if let Some(resource) = context.raw_resource(index) { + check_microsoft_dsc_metadata(resource, &site.pointer, sink); + } + } +} + +/// Report the unrecognized keys under the `Microsoft.DSC` metadata of one node. +/// +/// # Arguments +/// +/// * `node` - The document or resource node. +/// * `base` - The pointer to `node`. +/// * `sink` - Where to report. +fn check_microsoft_dsc_metadata(node: &Value, base: &JsonPointer, sink: &mut DiagnosticSink) { + let Some(metadata) = node.get("metadata").and_then(|metadata| metadata.get("Microsoft.DSC")).and_then(Value::as_object) else { + return; + }; + let pointer = base.push("metadata").push("Microsoft.DSC"); + for key in metadata.keys() { + if !MICROSOFT_DSC_METADATA_KEYS.contains(&key.as_str()) { + sink.report(&pointer.push(key), t!("lint.rules.document.unknownMicrosoftDscMetadata", key = key).to_string()); + } + } +} + +/// Report resources written with `apiVersion` instead of `requireVersion`. +fn api_version_alias(context: &LintContext, sink: &mut DiagnosticSink) { + for (index, site) in context.resources.iter().enumerate() { + if context.raw_resource(index).is_some_and(|resource| resource.get("apiVersion").is_some()) { + sink.report(&site.pointer.push("apiVersion"), t!("lint.rules.document.apiVersionAlias").to_string()); + } + } +} + +/// Report resources that require a date-based version. +fn date_version(context: &LintContext, sink: &mut DiagnosticSink) { + for (index, site) in context.resources.iter().enumerate() { + if let Some(version @ ResourceVersionReq::Date(_)) = &site.resource.require_version { + sink.report(&version_pointer(context, index), t!("lint.rules.document.dateVersion", version = version).to_string()); + } + } +} + +/// Report resources that don't pin a version. +fn missing_require_version(context: &LintContext, sink: &mut DiagnosticSink) { + for site in &context.resources { + if site.resource.require_version.is_none() { + sink.report( + &site.pointer, + t!( + "lint.rules.document.missingRequireVersion", + name = site.resource.name, + type_name = site.resource.resource_type + ).to_string(), + ); + } + } +} + +/// The pointer to the version requirement of a resource, whichever key the author used. +/// +/// # Arguments +/// +/// * `context` - The lint context. +/// * `index` - The index of the resource in the context's resource list. +fn version_pointer(context: &LintContext, index: usize) -> JsonPointer { + let site = &context.resources[index]; + if context.raw_resource(index).is_some_and(|resource| resource.get("apiVersion").is_some()) { + site.pointer.push("apiVersion") + } else { + site.pointer.push("requireVersion") + } +} + +#[cfg(test)] +mod tests { + use crate::lint::test_util::{lint_yaml, rule_paths}; + + #[test] + fn reports_duplicate_resources_case_insensitively() { + let result = lint_yaml(r#" +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +resources: +- name: Same + type: Microsoft.DSC.Debug/Echo +- name: Same + type: microsoft.dsc.debug/echo +- name: Other + type: Microsoft.DSC.Debug/Echo +"#); + assert_eq!(rule_paths(&result, "duplicate-resource"), vec!["/resources/1"]); + assert!(rule_paths(&result, "duplicate-resource-name").is_empty()); + } + + #[test] + fn skips_copy_loops_for_duplicates() { + let result = lint_yaml(r#" +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +resources: +- name: "[concat('a', copyIndex())]" + type: Microsoft.DSC.Debug/Echo + copy: + name: loop + count: 2 +- name: "[concat('a', copyIndex())]" + type: Microsoft.DSC.Debug/Echo + copy: + name: loop + count: 2 +"#); + assert!(rule_paths(&result, "duplicate-resource").is_empty()); + } + + #[test] + fn reports_duplicate_names_across_types() { + let result = lint_yaml(r#" +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +resources: +- name: Same + type: Microsoft.DSC.Debug/Echo +- name: Same + type: Microsoft/OSInfo +"#); + assert_eq!(rule_paths(&result, "duplicate-resource-name"), vec!["/resources/1"]); + assert!(rule_paths(&result, "duplicate-resource").is_empty()); + } + + #[test] + fn nested_resources_are_separate_groups() { + let result = lint_yaml(r#" +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +resources: +- name: Same + type: Microsoft.DSC/Group + resources: + - name: Same + type: Microsoft.DSC/Group +"#); + assert!(rule_paths(&result, "duplicate-resource").is_empty()); + } + + #[test] + fn reports_empty_resources() { + let result = lint_yaml(r#" +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +resources: [] +"#); + assert_eq!(rule_paths(&result, "empty-resources"), vec!["/resources"]); + } + + #[test] + fn reports_unknown_microsoft_dsc_metadata_keys() { + let result = lint_yaml(r#" +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +metadata: + Microsoft.DSC: + securityContext: elevated + customKey: 1 + Other: + anything: true +resources: +- name: a + type: Microsoft.DSC.Debug/Echo + metadata: + Microsoft.DSC: + nope: 2 +"#); + assert_eq!( + rule_paths(&result, "unknown-microsoft-dsc-metadata"), + vec!["/metadata/Microsoft.DSC/customKey", "/resources/0/metadata/Microsoft.DSC/nope"] + ); + } + + #[test] + fn reports_api_version_alias_and_date_versions() { + let result = lint_yaml(r#" +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +resources: +- name: a + type: Microsoft.DSC.Debug/Echo + apiVersion: "2025-01-01" +- name: b + type: Microsoft.DSC.Debug/Echo + requireVersion: "^1.0" +"#); + assert_eq!(rule_paths(&result, "api-version-alias"), vec!["/resources/0/apiVersion"]); + assert_eq!(rule_paths(&result, "date-version"), vec!["/resources/0/apiVersion"]); + } + + #[test] + fn missing_require_version_is_off_by_default() { + let result = lint_yaml(r#" +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +resources: +- name: a + type: Microsoft.DSC.Debug/Echo +"#); + assert!(rule_paths(&result, "missing-require-version").is_empty()); + } +} diff --git a/lib/dsc-lib/src/lint/rules/environment.rs b/lib/dsc-lib/src/lint/rules/environment.rs new file mode 100644 index 000000000..ae77eb3b5 --- /dev/null +++ b/lib/dsc-lib/src/lint/rules/environment.rs @@ -0,0 +1,124 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +//! Rules about the environment a configuration document targets. + +use std::sync::LazyLock; + +use regex::Regex; +use rust_i18n::t; + +use crate::lint::{ + context::LintContext, + diagnostic::LintCategory, + options::RuleLevel, + pointer::{for_each_string, JsonPointer}, + rule::{DiagnosticSink, Rule, RuleTier}, +}; + +/// Absolute paths rooted in an operating system drive or system directory. +static SYSTEM_PATH_PATTERN: LazyLock = LazyLock::new(|| { + Regex::new(r"^(?:[A-Za-z]:[\\/]|/(?:etc|usr|var|opt|bin|sbin|lib|home)/)") + .expect("the system path pattern is a valid regex") +}); + +/// The rules in this module. +#[must_use] +pub fn rules() -> Vec { + vec![ + Rule { + id: "system-root-unused", + category: LintCategory::Environment, + tier: RuleTier::Static, + default_level: RuleLevel::Warning, + description: || t!("lint.rules.environment.systemRootUnusedDescription").to_string(), + check: system_root_unused, + }, + Rule { + id: "hardcoded-system-path", + category: LintCategory::Environment, + tier: RuleTier::Static, + default_level: RuleLevel::Off, + description: || t!("lint.rules.environment.hardcodedSystemPathDescription").to_string(), + check: hardcoded_system_path, + }, + ] +} + +/// Report a document that doesn't call `systemRoot()` although a system root was specified. +fn system_root_unused(context: &LintContext, sink: &mut DiagnosticSink) { + let Some(system_root) = &context.options.system_root else { + return; + }; + if context.expressions.calls_named("systemRoot").is_empty() { + sink.report( + &JsonPointer::root(), + t!("lint.rules.environment.systemRootUnused", path = system_root.display()).to_string(), + ); + } +} + +/// Report literal absolute operating system paths in resource properties. +fn hardcoded_system_path(context: &LintContext, sink: &mut DiagnosticSink) { + for (index, site) in context.resources.iter().enumerate() { + let Some(properties) = context.raw_resource(index).and_then(|resource| resource.get("properties")) else { + continue; + }; + for_each_string(properties, &site.pointer.push("properties"), &mut |pointer, text| { + if !text.starts_with('[') && SYSTEM_PATH_PATTERN.is_match(text) { + sink.report(pointer, t!("lint.rules.environment.hardcodedSystemPath", value = text).to_string()); + } + }); + } +} + +#[cfg(test)] +mod tests { + use std::path::PathBuf; + + use crate::lint::{options::{LintOptions, RuleLevel, RuleLevelOverride}, test_util::{lint_with, rule_paths}}; + + fn options_with_root() -> LintOptions { + LintOptions { + discovery: false, + use_settings: false, + system_root: Some(PathBuf::from("/mnt/image")), + rule_levels: vec![RuleLevelOverride { id: "hardcoded-system-path".to_string(), level: RuleLevel::Info }], + ..Default::default() + } + } + + #[test] + fn reports_unused_system_root_and_hardcoded_paths() { + let result = lint_with(r#" +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +resources: +- name: a + type: Microsoft.DSC.Debug/Echo + properties: + output: + windows: 'C:\Windows\System32' + unix: /etc/ssh/sshd_config + relative: config/sshd_config + escaped: "[[C:\\not-an-expression]" +"#, options_with_root()); + assert_eq!(rule_paths(&result, "system-root-unused"), vec![""]); + assert_eq!( + rule_paths(&result, "hardcoded-system-path"), + vec!["/resources/0/properties/output/unix", "/resources/0/properties/output/windows"] + ); + } + + #[test] + fn system_root_call_satisfies_the_rule() { + let result = lint_with(r#" +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +resources: +- name: a + type: Microsoft.DSC.Debug/Echo + properties: + output: "[concat(systemRoot(), 'etc')]" +"#, options_with_root()); + assert!(rule_paths(&result, "system-root-unused").is_empty()); + } +} diff --git a/lib/dsc-lib/src/lint/rules/expression.rs b/lib/dsc-lib/src/lint/rules/expression.rs new file mode 100644 index 000000000..70ff62bff --- /dev/null +++ b/lib/dsc-lib/src/lint/rules/expression.rs @@ -0,0 +1,382 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +//! Rules about expressions and the functions they call. + +use rust_i18n::t; + +use crate::configure::config_doc::DataType; +use crate::lint::{ + context::LintContext, + diagnostic::LintCategory, + expression_index::{literal_args, literal_string_arg, ExpressionScope, FunctionCall}, + options::RuleLevel, + pointer::JsonPointer, + rule::{DiagnosticSink, Rule, RuleTier}, +}; + +/// The rules in this module. +#[must_use] +pub fn rules() -> Vec { + vec![ + Rule { + id: "invalid-expression", + category: LintCategory::Expression, + tier: RuleTier::Static, + default_level: RuleLevel::Error, + description: || t!("lint.rules.expression.invalidExpressionDescription").to_string(), + check: invalid_expression, + }, + Rule { + id: "unknown-function", + category: LintCategory::Expression, + tier: RuleTier::Static, + default_level: RuleLevel::Error, + description: || t!("lint.rules.expression.unknownFunctionDescription").to_string(), + check: unknown_function, + }, + Rule { + id: "invalid-argument-count", + category: LintCategory::Expression, + tier: RuleTier::Static, + default_level: RuleLevel::Error, + description: || t!("lint.rules.expression.invalidArgumentCountDescription").to_string(), + check: invalid_argument_count, + }, + Rule { + id: "invalid-argument-type", + category: LintCategory::Expression, + tier: RuleTier::Static, + default_level: RuleLevel::Error, + description: || t!("lint.rules.expression.invalidArgumentTypeDescription").to_string(), + check: invalid_argument_type, + }, + Rule { + id: "undefined-parameter", + category: LintCategory::Expression, + tier: RuleTier::Static, + default_level: RuleLevel::Error, + description: || t!("lint.rules.expression.undefinedParameterDescription").to_string(), + check: undefined_parameter, + }, + Rule { + id: "undefined-variable", + category: LintCategory::Expression, + tier: RuleTier::Static, + default_level: RuleLevel::Error, + description: || t!("lint.rules.expression.undefinedVariableDescription").to_string(), + check: undefined_variable, + }, + Rule { + id: "condition-not-expression", + category: LintCategory::Expression, + tier: RuleTier::Static, + default_level: RuleLevel::Warning, + description: || t!("lint.rules.expression.conditionNotExpressionDescription").to_string(), + check: condition_not_expression, + }, + ] +} + +/// Whether a call is to `lambda()`, which the runtime dispatches specially with no arguments. +/// +/// # Arguments +/// +/// * `call` - The call. +fn is_lambda(call: &FunctionCall) -> bool { + call.function.name().eq_ignore_ascii_case("lambda") +} + +/// Whether a call can be checked against built-in function metadata. +/// +/// User-defined functions (`namespace.member`) and `lambda()` are excluded. +/// +/// # Arguments +/// +/// * `context` - The lint context. +/// * `call` - The call. +fn is_checkable_builtin(context: &LintContext, call: &FunctionCall) -> bool { + let name = call.function.name(); + !name.contains('.') && !is_lambda(call) && context.functions.get_metadata(name).is_some() +} + +/// The name of the secure parameter whose default value holds an expression, if any. +/// +/// # Arguments +/// +/// * `context` - The lint context. +/// * `scope` - Where the expression appears. +fn secure_parameter_scope<'a>(context: &LintContext, scope: &'a ExpressionScope) -> Option<&'a str> { + let ExpressionScope::Parameter(name) = scope else { + return None; + }; + let secure = context.declared_parameter(name) + .is_some_and(|parameter| matches!(parameter.parameter_type, DataType::SecureString | DataType::SecureObject)); + secure.then_some(name.as_str()) +} + +/// Report expressions that fail to parse. +/// +/// The expression and the parser error, which repeats it, are left out of the message when the +/// expression is the default value of a secure parameter, because it can hold a secret. +fn invalid_expression(context: &LintContext, sink: &mut DiagnosticSink) { + for site in context.expressions.sites() { + let Err(err) = &site.parsed else { + continue; + }; + let message = match secure_parameter_scope(context, &site.scope) { + Some(name) => t!("lint.rules.expression.invalidExpressionInSecureParameter", name = name), + None => t!( + "lint.rules.expression.invalidExpression", + expression = site.text, + error = LintContext::error_message(err) + ), + }; + sink.report(&site.pointer, message.to_string()); + } +} + +/// Report calls to functions that are neither built in nor declared in `functions`. +fn unknown_function(context: &LintContext, sink: &mut DiagnosticSink) { + let user_functions = context.user_function_names(); + for call in context.expressions.calls() { + let name = call.function.name(); + if name.contains('.') { + if !user_functions.contains(name) { + sink.report(&call.site.pointer, t!("lint.rules.expression.unknownUserFunction", name = name).to_string()); + } + continue; + } + if is_lambda(&call) || context.functions.get_metadata(name).is_some() { + continue; + } + let suggestion = context.functions.list().into_iter().map(|function| function.name).find(|known| known.eq_ignore_ascii_case(name)); + let message = match suggestion { + Some(suggestion) => t!("lint.rules.expression.unknownFunctionSuggestion", name = name, suggestion = suggestion), + None => t!("lint.rules.expression.unknownFunction", name = name), + }; + sink.report(&call.site.pointer, message.to_string()); + } +} + +/// Report calls that pass a number of arguments the function doesn't accept. +fn invalid_argument_count(context: &LintContext, sink: &mut DiagnosticSink) { + for call in context.expressions.calls() { + if !is_checkable_builtin(context, &call) { + continue; + } + let count = call.function.args().map_or(0, <[_]>::len); + if let Err(err) = context.functions.validate_arg_count(call.function.name(), count) { + sink.report(&call.site.pointer, LintContext::error_message(&err)); + } + } +} + +/// Report calls whose literal arguments have a kind the function doesn't accept. +fn invalid_argument_type(context: &LintContext, sink: &mut DiagnosticSink) { + for call in context.expressions.calls() { + if !is_checkable_builtin(context, &call) { + continue; + } + let args = literal_args(call.function); + if let Err(err) = context.functions.validate_literal_args(call.function.name(), &args) { + sink.report(&call.site.pointer, LintContext::error_message(&err)); + } + } +} + +/// Report `parameters()` calls that name an undeclared parameter. +/// +/// Calls inside user-defined function bodies refer to the function's own parameters and are +/// skipped. +fn undefined_parameter(context: &LintContext, sink: &mut DiagnosticSink) { + for call in context.expressions.calls_named("parameters") { + if call.site.scope == ExpressionScope::UserFunction { + continue; + } + let Some(name) = literal_string_arg(call.function, 0) else { + continue; + }; + if context.declared_parameter(name).is_none() { + sink.report(&call.site.pointer, t!("lint.rules.expression.undefinedParameter", name = name).to_string()); + } + } +} + +/// Report `variables()` calls that name an undeclared variable. +fn undefined_variable(context: &LintContext, sink: &mut DiagnosticSink) { + for call in context.expressions.calls_named("variables") { + let Some(name) = literal_string_arg(call.function, 0) else { + continue; + }; + let declared = context.config.variables.as_ref().is_some_and(|variables| variables.contains_key(name)); + if !declared { + sink.report(&call.site.pointer, t!("lint.rules.expression.undefinedVariable", name = name).to_string()); + } + } +} + +/// Report resource and output conditions that are plain strings rather than expressions. +fn condition_not_expression(context: &LintContext, sink: &mut DiagnosticSink) { + for site in &context.resources { + if let Some(condition) = &site.resource.condition + && !LintContext::is_expression(condition) { + sink.report( + &site.pointer.push("condition"), + t!("lint.rules.expression.conditionNotExpression", condition = condition).to_string(), + ); + } + } + if let Some(outputs) = &context.config.outputs { + for (name, output) in outputs { + if let Some(condition) = &output.condition + && !LintContext::is_expression(condition) { + sink.report( + &JsonPointer::root().push("outputs").push(name).push("condition"), + t!("lint.rules.expression.conditionNotExpression", condition = condition).to_string(), + ); + } + } + } +} + +#[cfg(test)] +mod tests { + use crate::lint::test_util::{lint_yaml, rule_messages, rule_paths}; + + #[test] + fn reports_invalid_expressions() { + let result = lint_yaml(r#" +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +resources: +- name: a + type: Microsoft.DSC.Debug/Echo + properties: + output: "[concat('unterminated)]" + fine: "[[not an expression]" +"#); + assert_eq!(rule_paths(&result, "invalid-expression"), vec!["/resources/0/properties/output"]); + } + + #[test] + fn redacts_invalid_expressions_in_secure_defaults() { + let result = lint_yaml(r#" +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +parameters: + password: + type: secureString + defaultValue: "[concat('hunter2', 'unterminated)]" + blob: + type: secureObject + defaultValue: "[createObject('key', 'hunter2'))]" + plain: + type: string + defaultValue: "[concat('visible', 'unterminated)]" +resources: +- name: a + type: Microsoft.DSC.Debug/Echo + properties: + output: "[concat(parameters('password'), parameters('blob'), parameters('plain'))]" +"#); + assert_eq!( + rule_paths(&result, "invalid-expression"), + vec!["/parameters/blob/defaultValue", "/parameters/password/defaultValue", "/parameters/plain/defaultValue"] + ); + let messages = rule_messages(&result, "invalid-expression"); + assert!(messages[0].contains("'blob'") && !messages[0].contains("hunter2"), "{}", messages[0]); + assert!(messages[1].contains("'password'") && !messages[1].contains("hunter2"), "{}", messages[1]); + assert!(messages[2].contains("visible"), "{}", messages[2]); + } + + #[test] + fn reports_unknown_functions_with_case_suggestion() { + let result = lint_yaml(r#" +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +functions: +- namespace: my + members: + hello: + output: + type: string + value: "[concat('hi', parameters('who'))]" +resources: +- name: a + type: Microsoft.DSC.Debug/Echo + properties: + output: + a: "[Concat('a', 'b')]" + b: "[nosuchfunction()]" + c: "[my.hello('x')]" + d: "[my.missing('x')]" + e: "[map(createArray(1), lambda('item', lambdaVariables('item')))]" +"#); + assert_eq!( + rule_paths(&result, "unknown-function"), + vec!["/resources/0/properties/output/a", "/resources/0/properties/output/b", "/resources/0/properties/output/d"] + ); + let messages = rule_messages(&result, "unknown-function"); + assert!(messages[0].contains("'concat'"), "expected a case suggestion, got {}", messages[0]); + assert!(rule_paths(&result, "undefined-parameter").is_empty(), "user function parameters must not be checked"); + } + + #[test] + fn reports_argument_count_and_type() { + let result = lint_yaml(r#" +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +resources: +- name: a + type: Microsoft.DSC.Debug/Echo + properties: + output: + count: "[int('1', '2')]" + kind: "[int(true)]" + nested: "[int(concat('1', '2'))]" +"#); + assert_eq!(rule_paths(&result, "invalid-argument-count"), vec!["/resources/0/properties/output/count"]); + assert_eq!(rule_paths(&result, "invalid-argument-type"), vec!["/resources/0/properties/output/kind"]); + } + + #[test] + fn reports_undefined_parameters_and_variables() { + let result = lint_yaml(r#" +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +parameters: + known: + type: string + defaultValue: x +variables: + v: 1 +resources: +- name: a + type: Microsoft.DSC.Debug/Echo + properties: + output: + p1: "[parameters('known')]" + p2: "[parameters('unknown')]" + v1: "[variables('v')]" + v2: "[variables('w')]" +"#); + assert_eq!(rule_paths(&result, "undefined-parameter"), vec!["/resources/0/properties/output/p2"]); + assert_eq!(rule_paths(&result, "undefined-variable"), vec!["/resources/0/properties/output/v2"]); + } + + #[test] + fn reports_plain_string_conditions() { + let result = lint_yaml(r#" +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +outputs: + o: + type: string + value: x + condition: "true" +resources: +- name: a + type: Microsoft.DSC.Debug/Echo + condition: "true" +- name: b + type: Microsoft.DSC.Debug/Echo + condition: "[true()]" +"#); + assert_eq!(rule_paths(&result, "condition-not-expression"), vec!["/outputs/o/condition", "/resources/0/condition"]); + } +} diff --git a/lib/dsc-lib/src/lint/rules/mod.rs b/lib/dsc-lib/src/lint/rules/mod.rs new file mode 100644 index 000000000..d6bb0645b --- /dev/null +++ b/lib/dsc-lib/src/lint/rules/mod.rs @@ -0,0 +1,49 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +//! The lint rules, grouped by the area of the document they inspect. + +use super::rule::Rule; + +pub mod dependency; +pub mod deprecation; +pub mod document; +pub mod environment; +pub mod expression; +pub mod parameter; +pub mod resource; + +/// Every rule, grouped by category. +#[must_use] +pub fn all_rules() -> Vec { + let mut rules = Vec::new(); + rules.extend(document::rules()); + rules.extend(expression::rules()); + rules.extend(parameter::rules()); + rules.extend(dependency::rules()); + rules.extend(deprecation::rules()); + rules.extend(environment::rules()); + rules.extend(resource::rules()); + rules +} + +#[cfg(test)] +mod tests { + use std::collections::HashSet; + + use super::*; + + #[test] + fn rule_ids_are_unique_and_kebab_case() { + let mut seen = HashSet::new(); + for rule in all_rules() { + assert!(seen.insert(rule.id), "duplicate rule id {}", rule.id); + assert!( + rule.id.chars().all(|c| c.is_ascii_lowercase() || c == '-'), + "rule id {} must be kebab-case", + rule.id + ); + assert!(!(rule.description)().is_empty(), "rule {} has no description", rule.id); + } + } +} diff --git a/lib/dsc-lib/src/lint/rules/parameter.rs b/lib/dsc-lib/src/lint/rules/parameter.rs new file mode 100644 index 000000000..bd81201d6 --- /dev/null +++ b/lib/dsc-lib/src/lint/rules/parameter.rs @@ -0,0 +1,729 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +//! Rules about parameter and variable declarations and the values supplied for them. + +use std::{collections::HashSet, sync::LazyLock}; + +use regex::Regex; +use rust_i18n::t; +use serde_json::Value; + +use crate::{ + configure::{ + config_doc::{DataType, Parameter, ValueOrCopy}, + constraints::{check_allowed_values, check_length, check_number_limits}, + }, + lint::{ + context::LintContext, + diagnostic::LintCategory, + expression_index::{literal_string_arg, ExpressionScope}, + options::RuleLevel, + pointer::JsonPointer, + rule::{DiagnosticSink, Rule, RuleTier}, + }, + parser::ParsedStatement, +}; + +/// Parameter names that suggest the value is a secret. +static SECRET_NAME_PATTERN: LazyLock = LazyLock::new(|| { + Regex::new(r"(?i)(password|passwd|secret|token|apikey|api[-_]?key|credential|connectionstring)") + .expect("the secret name pattern is a valid regex") +}); + +/// The rules in this module. +#[must_use] +pub fn rules() -> Vec { + vec![ + Rule { + id: "no-unused-parameters", + category: LintCategory::Parameter, + tier: RuleTier::Static, + default_level: RuleLevel::Warning, + description: || t!("lint.rules.parameter.noUnusedParametersDescription").to_string(), + check: no_unused_parameters, + }, + Rule { + id: "no-unused-variables", + category: LintCategory::Parameter, + tier: RuleTier::Static, + default_level: RuleLevel::Warning, + description: || t!("lint.rules.parameter.noUnusedVariablesDescription").to_string(), + check: no_unused_variables, + }, + Rule { + id: "parameter-default-type", + category: LintCategory::Parameter, + tier: RuleTier::Static, + default_level: RuleLevel::Error, + description: || t!("lint.rules.parameter.parameterDefaultTypeDescription").to_string(), + check: parameter_default_type, + }, + Rule { + id: "invalid-parameter-constraints", + category: LintCategory::Parameter, + tier: RuleTier::Static, + default_level: RuleLevel::Error, + description: || t!("lint.rules.parameter.invalidParameterConstraintsDescription").to_string(), + check: invalid_parameter_constraints, + }, + Rule { + id: "secure-parameter-default", + category: LintCategory::Parameter, + tier: RuleTier::Static, + default_level: RuleLevel::Warning, + description: || t!("lint.rules.parameter.secureParameterDefaultDescription").to_string(), + check: secure_parameter_default, + }, + Rule { + id: "prefer-secure-type", + category: LintCategory::Parameter, + tier: RuleTier::Static, + default_level: RuleLevel::Warning, + description: || t!("lint.rules.parameter.preferSecureTypeDescription").to_string(), + check: prefer_secure_type, + }, + Rule { + id: "secure-value-in-output", + category: LintCategory::Parameter, + tier: RuleTier::Static, + default_level: RuleLevel::Warning, + description: || t!("lint.rules.parameter.secureValueInOutputDescription").to_string(), + check: secure_value_in_output, + }, + Rule { + id: "use-parameter-description", + category: LintCategory::Parameter, + tier: RuleTier::Static, + default_level: RuleLevel::Off, + description: || t!("lint.rules.parameter.useParameterDescriptionDescription").to_string(), + check: use_parameter_description, + }, + Rule { + id: "undeclared-parameter-value", + category: LintCategory::Parameter, + tier: RuleTier::Static, + default_level: RuleLevel::Error, + description: || t!("lint.rules.parameter.undeclaredParameterValueDescription").to_string(), + check: undeclared_parameter_value, + }, + Rule { + id: "missing-parameter-value", + category: LintCategory::Parameter, + tier: RuleTier::Static, + default_level: RuleLevel::Error, + description: || t!("lint.rules.parameter.missingParameterValueDescription").to_string(), + check: missing_parameter_value, + }, + Rule { + id: "parameter-value-type", + category: LintCategory::Parameter, + tier: RuleTier::Static, + default_level: RuleLevel::Error, + description: || t!("lint.rules.parameter.parameterValueTypeDescription").to_string(), + check: parameter_value_type, + }, + ] +} + +/// The names passed as the literal first argument of every call of a function, outside +/// user-defined function bodies. +/// +/// # Arguments +/// +/// * `context` - The lint context. +/// * `function_name` - The function, such as `parameters` or `variables`. +fn referenced_names(context: &LintContext, function_name: &str) -> HashSet { + context.expressions.calls_named(function_name).into_iter() + .filter(|call| call.site.scope != ExpressionScope::UserFunction) + .filter_map(|call| literal_string_arg(call.function, 0).map(String::from)) + .collect() +} + +/// The pointer to a declared parameter. +/// +/// # Arguments +/// +/// * `name` - The parameter name. +fn parameter_pointer(name: &str) -> JsonPointer { + JsonPointer::root().push("parameters").push(name) +} + +/// Whether a value is acceptable for a parameter type. +/// +/// Secure types accept the plain value as well as the wrapped form (`{"secureString": ...}`), +/// because DSC wraps plain values when it imports parameters. +/// +/// # Arguments +/// +/// * `value` - The value. +/// * `data_type` - The parameter type. +fn value_matches_type(value: &Value, data_type: &DataType) -> bool { + match data_type { + DataType::String => value.is_string(), + DataType::SecureString => value.is_string() || wrapped_secure_value(value, "secureString").is_some_and(Value::is_string), + DataType::Int => value.is_i64(), + DataType::Bool => value.is_boolean(), + DataType::Object => value.is_object(), + DataType::SecureObject => value.is_object(), + DataType::Array => value.is_array(), + } +} + +/// The inner value of a wrapped secure value such as `{"secureString": "..."}`. +/// +/// # Arguments +/// +/// * `value` - The value to unwrap. +/// * `key` - The wrapper key, `secureString` or `secureObject`. +fn wrapped_secure_value<'a>(value: &'a Value, key: &str) -> Option<&'a Value> { + let object = value.as_object()?; + if object.len() == 1 { object.get(key) } else { None } +} + +/// Check a literal value against a parameter's type and constraints. +/// +/// # Arguments +/// +/// * `name` - The parameter name. +/// * `value` - The value. +/// * `parameter` - The parameter declaration. +/// +/// # Errors +/// +/// Returns the reason the value is invalid. +fn check_value(name: &str, value: &Value, parameter: &Parameter) -> Result<(), String> { + if !value_matches_type(value, ¶meter.parameter_type) { + return Err(t!("lint.rules.parameter.valueNotOfType", type_name = parameter.parameter_type).to_string()); + } + check_allowed_values(name, value, parameter).map_err(|err| LintContext::error_message(&err))?; + if parameter.parameter_type == DataType::Int { + check_number_limits(name, value, parameter).map_err(|err| LintContext::error_message(&err))?; + } + if (value.is_string() || value.is_array()) && has_length_type(parameter) { + check_length(name, value, parameter).map_err(|err| LintContext::error_message(&err))?; + } + Ok(()) +} + +/// Whether length constraints apply to the parameter's type. +/// +/// # Arguments +/// +/// * `parameter` - The parameter declaration. +fn has_length_type(parameter: &Parameter) -> bool { + matches!(parameter.parameter_type, DataType::String | DataType::SecureString | DataType::Array) +} + +/// Whether a parameter has a secure type. +/// +/// # Arguments +/// +/// * `parameter` - The parameter declaration. +fn is_secure(parameter: &Parameter) -> bool { + matches!(parameter.parameter_type, DataType::SecureString | DataType::SecureObject) +} + +/// Report declared parameters that no expression references. +fn no_unused_parameters(context: &LintContext, sink: &mut DiagnosticSink) { + let Some(parameters) = &context.config.parameters else { + return; + }; + let referenced = referenced_names(context, "parameters"); + for name in parameters.keys() { + if !referenced.contains(name) { + sink.report(¶meter_pointer(name), t!("lint.rules.parameter.noUnusedParameters", name = name).to_string()); + } + } +} + +/// Report declared variables that no expression references. +fn no_unused_variables(context: &LintContext, sink: &mut DiagnosticSink) { + let Some(variables) = &context.config.variables else { + return; + }; + let referenced = referenced_names(context, "variables"); + for name in variables.keys() { + if !referenced.contains(name) { + sink.report(&JsonPointer::root().push("variables").push(name), t!("lint.rules.parameter.noUnusedVariables", name = name).to_string()); + } + } +} + +/// Report literal default values that don't satisfy the parameter's type or constraints. +/// +/// An escaped string default (`[[...`) is checked with the escape character removed, which is +/// the value DSC uses. +fn parameter_default_type(context: &LintContext, sink: &mut DiagnosticSink) { + let Some(parameters) = &context.config.parameters else { + return; + }; + for (name, parameter) in parameters { + let Some(value) = ¶meter.default_value else { + continue; + }; + if value.as_str().is_some_and(LintContext::is_expression) { + continue; + } + let pointer = parameter_pointer(name).push("defaultValue"); + let unescaped = match context.expressions.site_at(&pointer).map(|site| &site.parsed) { + Some(Ok(ParsedStatement::Escaped(text))) => Value::String(text.clone()), + _ => value.clone(), + }; + if let Err(reason) = check_value(name, &unescaped, parameter) { + sink.report(&pointer, t!("lint.rules.parameter.parameterDefaultType", name = name, reason = reason).to_string()); + } + } +} + +/// Report constraints that contradict each other or don't apply to the parameter's type. +fn invalid_parameter_constraints(context: &LintContext, sink: &mut DiagnosticSink) { + let Some(parameters) = &context.config.parameters else { + return; + }; + for (name, parameter) in parameters { + let base = parameter_pointer(name); + let type_name = ¶meter.parameter_type; + if let (Some(min), Some(max)) = (parameter.min_value, parameter.max_value) + && min > max { + sink.report(&base.push("minValue"), t!("lint.rules.parameter.minValueExceedsMaxValue", name = name, min = min, max = max).to_string()); + } + if let (Some(min), Some(max)) = (parameter.min_length, parameter.max_length) + && min > max { + sink.report(&base.push("minLength"), t!("lint.rules.parameter.minLengthExceedsMaxLength", name = name, min = min, max = max).to_string()); + } + if *type_name != DataType::Int { + for (constraint, limit) in [("minValue", parameter.min_value), ("maxValue", parameter.max_value)] { + if limit.is_some() { + sink.report(&base.push(constraint), t!("lint.rules.parameter.valueLimitOnNonInt", name = name, constraint = constraint, type_name = type_name).to_string()); + } + } + } + for (constraint, limit) in [("minLength", parameter.min_length), ("maxLength", parameter.max_length)] { + let Some(limit) = limit else { + continue; + }; + if !has_length_type(parameter) { + sink.report(&base.push(constraint), t!("lint.rules.parameter.lengthLimitOnOtherType", name = name, constraint = constraint, type_name = type_name).to_string()); + } else if limit < 0 { + sink.report(&base.push(constraint), t!("lint.rules.parameter.negativeLength", name = name, constraint = constraint).to_string()); + } + } + if let Some(allowed_values) = ¶meter.allowed_values { + if allowed_values.is_empty() { + sink.report(&base.push("allowedValues"), t!("lint.rules.parameter.emptyAllowedValues", name = name).to_string()); + } + for (position, allowed) in allowed_values.iter().enumerate() { + if !value_matches_type(allowed, type_name) { + sink.report( + &base.push("allowedValues").push_index(position), + t!("lint.rules.parameter.allowedValueTypeMismatch", name = name, type_name = type_name).to_string(), + ); + } + } + } + } +} + +/// Report secure parameters with a hard-coded default value. +fn secure_parameter_default(context: &LintContext, sink: &mut DiagnosticSink) { + let Some(parameters) = &context.config.parameters else { + return; + }; + for (name, parameter) in parameters { + if !is_secure(parameter) { + continue; + } + let Some(value) = ¶meter.default_value else { + continue; + }; + let harmless = value.as_str().is_some_and(|text| text.is_empty() || LintContext::is_expression(text)); + if !harmless { + sink.report(¶meter_pointer(name).push("defaultValue"), t!("lint.rules.parameter.secureParameterDefault", name = name).to_string()); + } + } +} + +/// Report parameters whose names suggest a secret but whose type isn't secure. +fn prefer_secure_type(context: &LintContext, sink: &mut DiagnosticSink) { + let Some(parameters) = &context.config.parameters else { + return; + }; + for (name, parameter) in parameters { + let secure_type = match parameter.parameter_type { + DataType::String => DataType::SecureString, + DataType::Object => DataType::SecureObject, + _ => continue, + }; + if SECRET_NAME_PATTERN.is_match(name) { + sink.report( + ¶meter_pointer(name), + t!("lint.rules.parameter.preferSecureType", name = name, secure_type = secure_type, type_name = parameter.parameter_type).to_string(), + ); + } + } +} + +/// Report outputs whose value references a secure parameter or a secret. +/// +/// Outputs declared as `secureString` or `secureObject` are skipped, because DSC withholds them +/// from the result. +fn secure_value_in_output(context: &LintContext, sink: &mut DiagnosticSink) { + let Some(outputs) = &context.config.outputs else { + return; + }; + for (name, output) in outputs { + if !matches!(output.value_or_copy, ValueOrCopy::Value(_)) || matches!(output.r#type, DataType::SecureString | DataType::SecureObject) { + continue; + } + let pointer = JsonPointer::root().push("outputs").push(name).push("value"); + let calls = context.expressions.calls().into_iter().filter(|call| call.site.pointer == pointer); + for call in calls { + let source = match call.function.name() { + "parameters" => { + let Some(parameter_name) = literal_string_arg(call.function, 0) else { + continue; + }; + if !context.declared_parameter(parameter_name).is_some_and(is_secure) { + continue; + } + format!("parameters('{parameter_name}')") + }, + "secret" => "secret()".to_string(), + _ => continue, + }; + sink.report(&pointer, t!("lint.rules.parameter.secureValueInOutput", name = name, source = source).to_string()); + break; + } + } +} + +/// Report parameters without a description. +fn use_parameter_description(context: &LintContext, sink: &mut DiagnosticSink) { + let Some(parameters) = &context.config.parameters else { + return; + }; + for (name, parameter) in parameters { + if parameter.description.as_ref().is_none_or(|description| description.trim().is_empty()) { + sink.report(¶meter_pointer(name), t!("lint.rules.parameter.useParameterDescription", name = name).to_string()); + } + } +} + +/// Report supplied parameter values that don't match a declared parameter. +fn undeclared_parameter_value(context: &LintContext, sink: &mut DiagnosticSink) { + let Some(supplied) = &context.supplied_parameters else { + return; + }; + for name in supplied.keys() { + if context.declared_parameter(name).is_none() { + sink.report(&JsonPointer::root().push("parameters"), t!("lint.rules.parameter.undeclaredParameterValue", name = name).to_string()); + } + } +} + +/// Report declared parameters without a default that weren't supplied a value. +fn missing_parameter_value(context: &LintContext, sink: &mut DiagnosticSink) { + let Some(supplied) = &context.supplied_parameters else { + return; + }; + let Some(parameters) = &context.config.parameters else { + return; + }; + for (name, parameter) in parameters { + if parameter.default_value.is_none() && !supplied.contains_key(name) { + sink.report(¶meter_pointer(name), t!("lint.rules.parameter.missingParameterValue", name = name).to_string()); + } + } +} + +/// Check a supplied value against a parameter's type and constraints. +/// +/// Unlike a default value, a supplied `secureString` value must be a plain string; DSC wraps it +/// itself and rejects the wrapped form. +/// +/// # Arguments +/// +/// * `name` - The parameter name. +/// * `value` - The supplied value. +/// * `parameter` - The parameter declaration. +/// +/// # Errors +/// +/// Returns the reason the value is invalid. +fn check_supplied_value(name: &str, value: &Value, parameter: &Parameter) -> Result<(), String> { + if parameter.parameter_type == DataType::SecureString && !value.is_string() { + return Err(t!("lint.rules.parameter.secureStringNotPlain").to_string()); + } + check_value(name, value, parameter) +} + +/// Report supplied parameter values that don't satisfy the parameter's type or constraints. +fn parameter_value_type(context: &LintContext, sink: &mut DiagnosticSink) { + let Some(supplied) = &context.supplied_parameters else { + return; + }; + for (name, value) in supplied { + let Some(parameter) = context.declared_parameter(name) else { + continue; + }; + if let Err(reason) = check_supplied_value(name, value, parameter) { + sink.report(¶meter_pointer(name), t!("lint.rules.parameter.parameterValueType", name = name, reason = reason).to_string()); + } + } +} + +#[cfg(test)] +mod tests { + use serde_json::json; + + use crate::lint::{options::LintOptions, test_util::{lint_with, lint_yaml, rule_paths}}; + + #[test] + fn reports_unused_parameters_and_variables() { + let result = lint_yaml(r#" +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +parameters: + used: + type: string + defaultValue: a + unused: + type: string + defaultValue: b +variables: + used: 1 + unused: 2 +resources: +- name: a + type: Microsoft.DSC.Debug/Echo + properties: + output: "[concat(parameters('used'), string(variables('used')))]" +"#); + assert_eq!(rule_paths(&result, "no-unused-parameters"), vec!["/parameters/unused"]); + assert_eq!(rule_paths(&result, "no-unused-variables"), vec!["/variables/unused"]); + } + + #[test] + fn reports_invalid_default_values() { + let result = lint_yaml(r#" +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +parameters: + wrongType: + type: int + defaultValue: ten + notAllowed: + type: string + defaultValue: c + allowedValues: [a, b] + tooSmall: + type: int + defaultValue: 1 + minValue: 5 + tooShort: + type: string + defaultValue: ab + minLength: 3 + expression: + type: int + defaultValue: "[int('1')]" + secureOk: + type: secureString + defaultValue: "" + fine: + type: array + defaultValue: [1] +resources: [] +"#); + assert_eq!( + rule_paths(&result, "parameter-default-type"), + vec![ + "/parameters/notAllowed/defaultValue", + "/parameters/tooShort/defaultValue", + "/parameters/tooSmall/defaultValue", + "/parameters/wrongType/defaultValue", + ] + ); + } + + #[test] + fn checks_escaped_defaults_without_the_escape_character() { + let result = lint_yaml(r#" +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +parameters: + fits: + type: string + defaultValue: "[[x]" + maxLength: 3 + allowedValues: ["[x]"] + tooLong: + type: string + defaultValue: "[[abc]" + maxLength: 4 +resources: [] +"#); + assert_eq!(rule_paths(&result, "parameter-default-type"), vec!["/parameters/tooLong/defaultValue"]); + } + + #[test] + fn reports_wrapped_supplied_secure_strings() { + let options = LintOptions { + discovery: false, + use_settings: false, + parameters: Some(json!({"parameters": {"wrapped": {"secureString": "x"}, "plain": "x"}})), + ..Default::default() + }; + let result = lint_with(r#" +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +parameters: + wrapped: + type: secureString + plain: + type: secureString + defaulted: + type: secureString + defaultValue: + secureString: "[[not a secret]" +resources: +- name: a + type: Microsoft.DSC.Debug/Echo + properties: + output: "[concat(parameters('wrapped'), parameters('plain'), parameters('defaulted'))]" +"#, options); + assert_eq!(rule_paths(&result, "parameter-value-type"), vec!["/parameters/wrapped"]); + assert!(rule_paths(&result, "parameter-default-type").is_empty()); + } + + #[test] + fn reports_invalid_constraints() { + let result = lint_yaml(r#" +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +parameters: + flipped: + type: int + minValue: 10 + maxValue: 1 + lengthOnInt: + type: int + minLength: 1 + valueOnString: + type: string + maxValue: 1 + negative: + type: string + maxLength: -1 + empty: + type: string + allowedValues: [] + mixed: + type: int + allowedValues: [1, two] +resources: [] +"#); + assert_eq!( + rule_paths(&result, "invalid-parameter-constraints"), + vec![ + "/parameters/empty/allowedValues", + "/parameters/flipped/minValue", + "/parameters/lengthOnInt/minLength", + "/parameters/mixed/allowedValues/1", + "/parameters/negative/maxLength", + "/parameters/valueOnString/maxValue", + ] + ); + } + + #[test] + fn reports_secure_defaults_and_secret_like_names() { + let result = lint_yaml(r#" +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +parameters: + adminPassword: + type: string + defaultValue: "" + token: + type: secureString + defaultValue: hardcoded + apiKey: + type: secureString + config: + type: object + defaultValue: {} +resources: [] +"#); + assert_eq!(rule_paths(&result, "secure-parameter-default"), vec!["/parameters/token/defaultValue"]); + assert_eq!(rule_paths(&result, "prefer-secure-type"), vec!["/parameters/adminPassword"]); + } + + #[test] + fn reports_secure_values_in_outputs() { + let result = lint_yaml(r#" +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +parameters: + password: + type: secureString + plain: + type: string + defaultValue: x +outputs: + leaked: + type: string + value: "[parameters('password').secureString]" + viaSecret: + type: string + value: "[secret('vault')]" + fine: + type: string + value: "[parameters('plain')]" + withheld: + type: secureString + value: "[parameters('password')]" +resources: [] +"#); + assert_eq!(rule_paths(&result, "secure-value-in-output"), vec!["/outputs/leaked/value", "/outputs/viaSecret/value"]); + } + + #[test] + fn checks_supplied_parameter_values() { + let options = LintOptions { + discovery: false, + use_settings: false, + parameters: Some(json!({"parameters": {"count": "five", "extra": 1}})), + ..Default::default() + }; + let result = lint_with(r#" +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +parameters: + count: + type: int + required: + type: string + optional: + type: string + defaultValue: x +resources: +- name: a + type: Microsoft.DSC.Debug/Echo + properties: + output: "[concat(string(parameters('count')), parameters('required'), parameters('optional'))]" +"#, options); + assert_eq!(rule_paths(&result, "undeclared-parameter-value"), vec!["/parameters"]); + assert_eq!(rule_paths(&result, "missing-parameter-value"), vec!["/parameters/required"]); + assert_eq!(rule_paths(&result, "parameter-value-type"), vec!["/parameters/count"]); + } + + #[test] + fn parameter_value_rules_need_supplied_parameters() { + let result = lint_yaml(r#" +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +parameters: + required: + type: string +resources: +- name: a + type: Microsoft.DSC.Debug/Echo + properties: + output: "[parameters('required')]" +"#); + assert!(rule_paths(&result, "missing-parameter-value").is_empty()); + } +} diff --git a/lib/dsc-lib/src/lint/rules/resource.rs b/lib/dsc-lib/src/lint/rules/resource.rs new file mode 100644 index 000000000..9b9fc30c6 --- /dev/null +++ b/lib/dsc-lib/src/lint/rules/resource.rs @@ -0,0 +1,544 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +//! Rules that inspect the installed resources a configuration document uses. They only run +//! when discovery is enabled and never invoke a resource operation. + +use rust_i18n::t; +use serde_json::Value; + +use crate::{ + dscresources::{ + dscresource::{validate_json_all, ResourceCapability as Capability, DscResource, ImplementedAs, SchemaViolation}, + resource_manifest::{Kind, SchemaKind}, + }, + lint::{ + context::{LintContext, ResolvedResource}, + diagnostic::LintCategory, + expression_index::ExpressionScope, + options::RuleLevel, + pointer::JsonPointer, + rule::{DiagnosticSink, Rule, RuleTier}, + }, + parser::ParsedStatement, +}; + +/// The rules in this module. +#[must_use] +pub fn rules() -> Vec { + vec![ + Rule { + id: "unknown-resource-type", + category: LintCategory::Resource, + tier: RuleTier::Resource, + default_level: RuleLevel::Warning, + description: || t!("lint.rules.resource.unknownResourceTypeDescription").to_string(), + check: unknown_resource_type, + }, + Rule { + id: "unsatisfied-require-version", + category: LintCategory::Resource, + tier: RuleTier::Resource, + default_level: RuleLevel::Warning, + description: || t!("lint.rules.resource.unsatisfiedRequireVersionDescription").to_string(), + check: unsatisfied_require_version, + }, + Rule { + id: "deprecated-resource", + category: LintCategory::Resource, + tier: RuleTier::Resource, + default_level: RuleLevel::Warning, + description: || t!("lint.rules.resource.deprecatedResourceDescription").to_string(), + check: deprecated_resource, + }, + Rule { + id: "property-schema-mismatch", + category: LintCategory::Resource, + tier: RuleTier::Resource, + default_level: RuleLevel::Error, + description: || t!("lint.rules.resource.propertySchemaMismatchDescription").to_string(), + check: property_schema_mismatch, + }, + Rule { + id: "nested-resources-unsupported", + category: LintCategory::Resource, + tier: RuleTier::Resource, + default_level: RuleLevel::Warning, + description: || t!("lint.rules.resource.nestedResourcesUnsupportedDescription").to_string(), + check: nested_resources_unsupported, + }, + Rule { + id: "exist-false-unsupported", + category: LintCategory::Resource, + tier: RuleTier::Resource, + default_level: RuleLevel::Error, + description: || t!("lint.rules.resource.existFalseUnsupportedDescription").to_string(), + check: exist_false_unsupported, + }, + Rule { + id: "canonical-type-name", + category: LintCategory::Resource, + tier: RuleTier::Resource, + default_level: RuleLevel::Info, + description: || t!("lint.rules.resource.canonicalTypeNameDescription").to_string(), + check: canonical_type_name, + }, + ] +} + +/// The resource declarations paired with what discovery found for them. +/// +/// # Arguments +/// +/// * `context` - The lint context. +/// +/// # Returns +/// +/// One entry per resource declaration, or an empty list when discovery didn't run. +fn resolved<'a>(context: &'a LintContext<'_>) -> Vec<(usize, &'a ResolvedResource)> { + context.discovered.as_ref().map_or_else(Vec::new, |discovered| discovered.iter().enumerate().collect()) +} + +/// The pointer to the version requirement of a resource, whichever key the author used. +/// +/// # Arguments +/// +/// * `context` - The lint context. +/// * `index` - The index of the resource in the context's resource list. +fn version_pointer(context: &LintContext, index: usize) -> JsonPointer { + let site = &context.resources[index]; + if context.raw_resource(index).is_some_and(|resource| resource.get("apiVersion").is_some()) { + site.pointer.push("apiVersion") + } else { + site.pointer.push("requireVersion") + } +} + +/// The schema for a resource's properties without invoking the resource: the schema discovery +/// loaded, or the schema embedded in the manifest. +/// +/// # Arguments +/// +/// * `resource` - The discovered resource. +fn embedded_schema(resource: &DscResource) -> Option { + if let Some(schema) = &resource.schema { + return Some(Value::Object(schema.clone())); + } + if let Some(SchemaKind::Embedded(schema)) = resource.manifest.as_ref().and_then(|manifest| manifest.schema.as_ref()) { + return Some(schema.clone()); + } + None +} + +/// Report resource types that aren't installed. +fn unknown_resource_type(context: &LintContext, sink: &mut DiagnosticSink) { + for (index, resolved) in resolved(context) { + if resolved.found.is_none() && !resolved.type_exists { + let site = &context.resources[index]; + sink.report(&site.pointer.push("type"), t!("lint.rules.resource.unknownResourceType", type_name = site.resource.resource_type).to_string()); + } + } +} + +/// Report version requirements that no installed version of the type satisfies. +fn unsatisfied_require_version(context: &LintContext, sink: &mut DiagnosticSink) { + for (index, resolved) in resolved(context) { + if resolved.found.is_none() && resolved.type_exists + && let Some(version) = &context.resources[index].resource.require_version { + sink.report( + &version_pointer(context, index), + t!( + "lint.rules.resource.unsatisfiedRequireVersion", + type_name = context.resources[index].resource.resource_type, + version = version + ).to_string(), + ); + } + } +} + +/// Report resources whose type is deprecated. +fn deprecated_resource(context: &LintContext, sink: &mut DiagnosticSink) { + for (index, resolved) in resolved(context) { + if let Some(message) = resolved.found.as_ref().and_then(|found| found.deprecation_message.as_ref()) { + let site = &context.resources[index]; + sink.report( + &site.pointer.push("type"), + t!("lint.rules.resource.deprecatedResource", type_name = site.resource.resource_type, message = message).to_string(), + ); + } + } +} + +/// Keywords whose outcome depends only on the shape of a value (its type, member names, or +/// length) and not on the values nested in it, so an expression nested in the value can't +/// change it. +const STRUCTURAL_KEYWORDS: [&str; 9] = [ + "type", + "required", + "additionalProperties", + "propertyNames", + "minProperties", + "maxProperties", + "dependentRequired", + "minItems", + "maxItems", +]; + +/// The properties of a resource as DSC passes them to the resource, as far as that is known +/// statically. +/// +/// # Arguments +/// +/// * `context` - The lint context. +/// * `index` - The index of the resource in the context's resource list. +/// * `properties` - The properties as written. +/// +/// # Returns +/// +/// The properties with escaped strings (`[[...`) replaced by their value, and the pointers of +/// the expressions, whose values are only known at runtime. +fn runtime_properties(context: &LintContext, index: usize, properties: &Value) -> (Value, Vec) { + let base = context.resources[index].pointer.push("properties"); + let mut value = properties.clone(); + let mut unknown = Vec::new(); + for site in context.expressions.sites() { + if site.scope != ExpressionScope::Resource(index) || !site.pointer.starts_with(&base) { + continue; + } + match &site.parsed { + Ok(ParsedStatement::Escaped(text)) => { + let relative = &site.pointer.as_str()[base.as_str().len()..]; + if let Some(slot) = value.pointer_mut(relative) { + *slot = Value::String(text.clone()); + } + }, + Ok(ParsedStatement::Literal(_)) => {}, + Ok(ParsedStatement::Expression(_)) | Err(_) => unknown.push(site.pointer.clone()), + } + } + (value, unknown) +} + +/// Whether a violation holds whatever the expressions in the properties evaluate to. +/// +/// A violation at or below an expression is unknown until runtime. Above an expression, only +/// violations of a structural keyword are definite. A failed `anyOf` or `oneOf` is definite only +/// when every subschema has a definite violation, because otherwise the runtime values could +/// satisfy one of them. +/// +/// # Arguments +/// +/// * `violation` - The violation. +/// * `base` - The pointer to the properties the violation paths are relative to. +/// * `unknown` - The pointers of the expressions in the properties. +fn is_definite(violation: &SchemaViolation, base: &JsonPointer, unknown: &[JsonPointer]) -> bool { + if !violation.alternatives.is_empty() { + return violation.alternatives.iter().all(|alternative| alternative.iter().any(|nested| is_definite(nested, base, unknown))); + } + let pointer = base.join(&violation.instance_path); + if unknown.iter().any(|expression| pointer.starts_with(expression)) { + return false; + } + let contains_expression = unknown.iter().any(|expression| expression.starts_with(&pointer)); + !contains_expression || STRUCTURAL_KEYWORDS.contains(&violation.keyword.as_str()) +} + +/// Report property values that don't validate against the resource's embedded schema. +/// +/// The properties are validated as DSC passes them to the resource, with escaped strings +/// unescaped. Violations that an expression's runtime value could resolve are dropped; see +/// [`is_definite`]. +fn property_schema_mismatch(context: &LintContext, sink: &mut DiagnosticSink) { + for (index, resolved) in resolved(context) { + let Some(found) = &resolved.found else { + continue; + }; + if found.implemented_as != Some(ImplementedAs::Command) { + continue; + } + let Some(schema) = embedded_schema(found) else { + continue; + }; + let site = &context.resources[index]; + let base = site.pointer.push("properties"); + let Some(properties) = base.resolve(context.raw) else { + continue; + }; + let (properties, unknown) = runtime_properties(context, index, properties); + let violations = match validate_json_all(found.type_name.as_ref(), &schema, &properties) { + Ok(violations) => violations, + Err(err) => { + sink.report( + &base, + t!("lint.rules.resource.propertySchemaCompileFailed", type_name = site.resource.resource_type, error = LintContext::error_message(&err)).to_string(), + ); + continue; + }, + }; + for violation in violations { + if !is_definite(&violation, &base, &unknown) { + continue; + } + sink.report( + &base.join(&violation.instance_path), + t!("lint.rules.resource.propertySchemaMismatch", type_name = site.resource.resource_type, message = violation.message).to_string(), + ); + } + } +} + +/// Report nested resources under a type that isn't a group or adapter. +fn nested_resources_unsupported(context: &LintContext, sink: &mut DiagnosticSink) { + for (index, resolved) in resolved(context) { + let site = &context.resources[index]; + if site.resource.resources.is_none() { + continue; + } + let Some(found) = &resolved.found else { + continue; + }; + if !matches!(found.kind, Kind::Group | Kind::Adapter) { + sink.report( + &site.pointer.push("resources"), + t!( + "lint.rules.resource.nestedResourcesUnsupported", + name = site.resource.name, + type_name = site.resource.resource_type, + kind = format!("{:?}", found.kind).to_lowercase() + ).to_string(), + ); + } + } +} + +/// Report `_exist: false` on a type that can't delete instances. +fn exist_false_unsupported(context: &LintContext, sink: &mut DiagnosticSink) { + for (index, resolved) in resolved(context) { + let site = &context.resources[index]; + let exists = site.resource.properties.as_ref().and_then(|properties| properties.get("_exist")); + if exists != Some(&Value::Bool(false)) { + continue; + } + let Some(found) = &resolved.found else { + continue; + }; + let can_delete = found.capabilities.contains(&Capability::Delete) || found.capabilities.contains(&Capability::SetHandlesExist); + if !can_delete { + sink.report( + &site.pointer.push("properties").push("_exist"), + t!("lint.rules.resource.existFalseUnsupported", type_name = site.resource.resource_type).to_string(), + ); + } + } +} + +/// Report resource types written with different casing than their manifest. +fn canonical_type_name(context: &LintContext, sink: &mut DiagnosticSink) { + for (index, resolved) in resolved(context) { + let Some(found) = &resolved.found else { + continue; + }; + let site = &context.resources[index]; + let written: &str = site.resource.resource_type.as_ref(); + let canonical: &str = found.type_name.as_ref(); + if written != canonical { + sink.report( + &site.pointer.push("type"), + t!("lint.rules.resource.canonicalTypeName", type_name = written, canonical = canonical).to_string(), + ); + } + } +} + +#[cfg(test)] +mod tests { + use serde_json::{json, Value}; + + use crate::{ + configure::config_doc::Configuration, + dscresources::dscresource::{ResourceCapability as Capability, DscResource, ImplementedAs}, + dscresources::resource_manifest::Kind, + lint::{ + context::{collect_resource_sites, LintContext, ResolvedResource}, + diagnostic::{LintDiagnostic, LintLevel}, + expression_index::ExpressionIndex, + options::LintOptions, + rule::DiagnosticSink, + }, + parser::Statement, + types::FullyQualifiedTypeName, + }; + + use super::rules; + + fn echo(kind: Kind, capabilities: Vec, schema: Option) -> DscResource { + DscResource { + type_name: FullyQualifiedTypeName::parse("Microsoft.DSC.Debug/Echo").unwrap(), + kind, + capabilities, + implemented_as: Some(ImplementedAs::Command), + schema: schema.and_then(|schema| schema.as_object().cloned()), + ..DscResource::new() + } + } + + fn run(document: &str, discovered: Vec, rule_id: &str) -> Vec { + let raw: Value = serde_yaml::from_str(document).unwrap(); + let config: Configuration = serde_json::from_value(raw.clone()).unwrap(); + let options = LintOptions::default(); + let mut parser = Statement::new().unwrap(); + let sites = collect_resource_sites(&config); + let expressions = ExpressionIndex::build(&raw, &sites, &mut parser); + let context = LintContext { + raw: &raw, + config: &config, + resources: sites, + expressions, + options: &options, + functions: parser.function_dispatcher(), + supplied_parameters: None, + discovered: Some(discovered), + }; + let rule = rules().into_iter().find(|rule| rule.id == rule_id).unwrap(); + let mut diagnostics = Vec::new(); + let mut sink = DiagnosticSink::new(rule.id, rule.category, LintLevel::Error, &mut diagnostics); + (rule.check)(&context, &mut sink); + diagnostics + } + + fn paths(diagnostics: &[LintDiagnostic]) -> Vec<&str> { + diagnostics.iter().map(|diagnostic| diagnostic.path.as_str()).collect() + } + + const DOCUMENT: &str = r#" +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +resources: +- name: a + type: microsoft.dsc.debug/echo + requireVersion: "^9.0" + properties: + output: literal + dynamic: "[parameters('x')]" + _exist: false + resources: + - name: nested + type: Microsoft.DSC.Debug/Echo +"#; + + #[test] + fn reports_unknown_types_and_unsatisfied_versions() { + let discovered = vec![ + ResolvedResource { found: None, type_exists: true }, + ResolvedResource { found: None, type_exists: false }, + ]; + assert_eq!(paths(&run(DOCUMENT, discovered, "unknown-resource-type")), vec!["/resources/0/resources/0/type"]); + let discovered = vec![ + ResolvedResource { found: None, type_exists: true }, + ResolvedResource { found: None, type_exists: false }, + ]; + assert_eq!(paths(&run(DOCUMENT, discovered, "unsatisfied-require-version")), vec!["/resources/0/requireVersion"]); + } + + #[test] + fn reports_deprecated_resources_and_casing() { + let mut deprecated = echo(Kind::Resource, vec![Capability::Get], None); + deprecated.deprecation_message = Some("use something else".to_string()); + let discovered = vec![ + ResolvedResource { found: Some(deprecated), type_exists: true }, + ResolvedResource { found: Some(echo(Kind::Resource, vec![Capability::Get], None)), type_exists: true }, + ]; + assert_eq!(paths(&run(DOCUMENT, discovered, "deprecated-resource")), vec!["/resources/0/type"]); + let discovered = vec![ + ResolvedResource { found: Some(echo(Kind::Resource, vec![Capability::Get], None)), type_exists: true }, + ResolvedResource { found: Some(echo(Kind::Resource, vec![Capability::Get], None)), type_exists: true }, + ]; + assert_eq!(paths(&run(DOCUMENT, discovered, "canonical-type-name")), vec!["/resources/0/type"]); + } + + #[test] + fn reports_schema_violations_outside_expressions() { + let schema = json!({ + "type": "object", + "additionalProperties": false, + "properties": { + "output": { "type": "integer" }, + "dynamic": { "type": "integer" }, + "_exist": { "type": "boolean" } + } + }); + let discovered = vec![ + ResolvedResource { found: Some(echo(Kind::Resource, vec![Capability::Get], Some(schema))), type_exists: true }, + ResolvedResource { found: Some(echo(Kind::Resource, vec![Capability::Get], None)), type_exists: true }, + ]; + assert_eq!(paths(&run(DOCUMENT, discovered, "property-schema-mismatch")), vec!["/resources/0/properties/output"]); + } + + #[test] + fn combinators_allow_expressions_but_keep_definite_errors() { + let document = r#" +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +resources: +- name: a + type: Microsoft.DSC.Debug/Echo + properties: + items: + - name: dynamic + scope: "[parameters('scope')]" + - name: literal + scope: nowhere + - name: brokenSibling + scope: "[parameters('scope')]" + count: notAnInt + count: "[[5]" + mode: "[[x]" +"#; + let branch = |scope: &str| json!({ + "type": "object", + "properties": { + "name": { "type": "string" }, + "scope": { "const": scope }, + "count": { "type": "integer" } + } + }); + let schema = json!({ + "type": "object", + "properties": { + "items": { "type": "array", "items": { "anyOf": [branch("currentUser"), branch("machine")] } }, + "count": { "type": "integer" }, + "mode": { "enum": ["[x]"] } + } + }); + let discovered = vec![ + ResolvedResource { found: Some(echo(Kind::Resource, vec![Capability::Get], Some(schema))), type_exists: true }, + ]; + let diagnostics = run(document, discovered, "property-schema-mismatch"); + let mut reported = paths(&diagnostics); + reported.sort_unstable(); + assert_eq!(reported, vec!["/resources/0/properties/count", "/resources/0/properties/items/1", "/resources/0/properties/items/2"]); + } + + #[test] + fn reports_nested_resources_and_exist_false_on_unsupported_types() { + let discovered = vec![ + ResolvedResource { found: Some(echo(Kind::Resource, vec![Capability::Get, Capability::Set], None)), type_exists: true }, + ResolvedResource { found: Some(echo(Kind::Resource, vec![Capability::Get], None)), type_exists: true }, + ]; + assert_eq!(paths(&run(DOCUMENT, discovered, "nested-resources-unsupported")), vec!["/resources/0/resources"]); + let discovered = vec![ + ResolvedResource { found: Some(echo(Kind::Resource, vec![Capability::Get, Capability::Set], None)), type_exists: true }, + ResolvedResource { found: Some(echo(Kind::Resource, vec![Capability::Get], None)), type_exists: true }, + ]; + assert_eq!(paths(&run(DOCUMENT, discovered, "exist-false-unsupported")), vec!["/resources/0/properties/_exist"]); + let discovered = vec![ + ResolvedResource { found: Some(echo(Kind::Group, vec![Capability::Get, Capability::Delete], None)), type_exists: true }, + ResolvedResource { found: Some(echo(Kind::Resource, vec![Capability::Get], None)), type_exists: true }, + ]; + assert!(run(DOCUMENT, discovered, "nested-resources-unsupported").is_empty()); + let discovered = vec![ + ResolvedResource { found: Some(echo(Kind::Group, vec![Capability::Get, Capability::Delete], None)), type_exists: true }, + ResolvedResource { found: Some(echo(Kind::Resource, vec![Capability::Get], None)), type_exists: true }, + ]; + assert!(run(DOCUMENT, discovered, "exist-false-unsupported").is_empty()); + } +} diff --git a/lib/dsc-lib/src/parser/expressions.rs b/lib/dsc-lib/src/parser/expressions.rs index 12ceee3ba..760bf5734 100644 --- a/lib/dsc-lib/src/parser/expressions.rs +++ b/lib/dsc-lib/src/parser/expressions.rs @@ -204,4 +204,16 @@ impl Expression { Ok(value) } } + + /// Get the function the expression invokes. + #[must_use] + pub fn function(&self) -> &Function { + &self.function + } + + /// Get the accessors applied to the function result, in the order they are applied. + #[must_use] + pub fn accessors(&self) -> &[Accessor] { + &self.accessors + } } diff --git a/lib/dsc-lib/src/parser/functions.rs b/lib/dsc-lib/src/parser/functions.rs index 68c68c74b..194cc599b 100644 --- a/lib/dsc-lib/src/parser/functions.rs +++ b/lib/dsc-lib/src/parser/functions.rs @@ -147,6 +147,12 @@ impl Function { pub fn name(&self) -> &str { &self.name } + + /// Get the arguments passed to the function, if any were written. + #[must_use] + pub fn args(&self) -> Option<&[FunctionArg]> { + self.args.as_deref() + } } fn convert_args_node(statement_bytes: &[u8], args: Option<&Node>) -> Result>, DscError> { diff --git a/lib/dsc-lib/src/parser/mod.rs b/lib/dsc-lib/src/parser/mod.rs index eb67e3dce..790df9177 100644 --- a/lib/dsc-lib/src/parser/mod.rs +++ b/lib/dsc-lib/src/parser/mod.rs @@ -19,6 +19,17 @@ pub struct Statement { function_dispatcher: FunctionDispatcher, } +/// The result of parsing a statement without evaluating it. +#[derive(Clone, Debug)] +pub enum ParsedStatement { + /// The statement is a plain string that doesn't contain an expression. + Literal(String), + /// The statement started with `[[`; the value has the escape character removed. + Escaped(String), + /// The statement is an expression that can be invoked. + Expression(Expression), +} + impl Statement { /// Create a new `StatementParser` instance. /// @@ -56,6 +67,29 @@ impl Statement { return Ok(Value::String(statement.to_string())); } + match self.parse(statement)? { + ParsedStatement::Literal(value) | ParsedStatement::Escaped(value) => Ok(Value::String(value)), + ParsedStatement::Expression(expression) => expression.invoke(&self.function_dispatcher, context), + } + } + + /// Parse a statement without evaluating it. + /// + /// Use this to inspect the expression tree of a statement, for example to find which + /// functions it calls, without needing a [`Context`] and without side effects. + /// + /// # Arguments + /// + /// * `statement` - The statement to parse. + /// + /// # Returns + /// + /// The parsed statement: a literal, an escaped literal, or an expression tree. + /// + /// # Errors + /// + /// This function will return an error if the statement fails to parse. + pub fn parse(&mut self, statement: &str) -> Result { let Some(tree) = &mut self.parser.parse(statement, None) else { return Err(DscError::Parser(t!("parser.failedToParse", statement = statement).to_string())); }; @@ -68,19 +102,19 @@ impl Statement { } let statement_bytes = statement.as_bytes(); let mut cursor = root_node.walk(); - let mut return_value = Value::Null; + let mut parsed = ParsedStatement::Literal(statement.to_string()); for child_node in root_node.named_children(&mut cursor) { if child_node.is_error() { return Err(DscError::Parser(t!("parser.failedToParse", statement = statement).to_string())); } - match child_node.kind() { + parsed = match child_node.kind() { "stringLiteral" => { let Ok(value) = child_node.utf8_text(statement_bytes) else { return Err(DscError::Parser(t!("parser.failedToParseStringLiteral").to_string())); }; debug!("{}", t!("parser.parsingStringLiteral", value = value.to_string())); - return_value = Value::String(value.to_string()); + ParsedStatement::Literal(value.to_string()) }, "escapedStringLiteral" => { // need to remove the first character: [[ => [ @@ -88,20 +122,25 @@ impl Statement { return Err(DscError::Parser(t!("parser.failedToParseEscapedStringLiteral").to_string())); }; debug!("{}", t!("parser.parsingEscapedStringLiteral", value = value[1..].to_string())); - return_value = Value::String(value[1..].to_string()); + ParsedStatement::Escaped(value[1..].to_string()) }, "expression" => { debug!("{}", t!("parser.parsingExpression")); - let expression = Expression::new(statement_bytes, &child_node)?; - return_value = expression.invoke(&self.function_dispatcher, context)?; + ParsedStatement::Expression(Expression::new(statement_bytes, &child_node)?) }, _ => { return Err(DscError::Parser(t!("parser.unknownExpressionType", kind = child_node.kind()).to_string())); } - } + }; } - Ok(return_value) + Ok(parsed) + } + + /// Get the function dispatcher used to evaluate expressions. + #[must_use] + pub fn function_dispatcher(&self) -> &FunctionDispatcher { + &self.function_dispatcher } } @@ -141,6 +180,39 @@ mod tests { assert!(result.is_err()); } + #[test] + fn parse_literal_without_evaluating() { + let mut parser = Statement::new().unwrap(); + let result = parser.parse("plain text").unwrap(); + assert!(matches!(result, ParsedStatement::Literal(value) if value == "plain text")); + } + + #[test] + fn parse_escaped_literal_without_evaluating() { + let mut parser = Statement::new().unwrap(); + let result = parser.parse("[[not an expression]").unwrap(); + assert!(matches!(result, ParsedStatement::Escaped(value) if value == "[not an expression]")); + } + + #[test] + fn parse_expression_without_evaluating() { + let mut parser = Statement::new().unwrap(); + // `parameters('x')` would fail to evaluate without a context, but parsing must succeed + let result = parser.parse("[parameters('x').member]").unwrap(); + let ParsedStatement::Expression(expression) = result else { + panic!("expected an expression"); + }; + assert_eq!(expression.function().name(), "parameters"); + assert_eq!(expression.function().args().map(<[_]>::len), Some(1)); + assert_eq!(expression.accessors().len(), 1); + } + + #[test] + fn parse_invalid_expression_fails() { + let mut parser = Statement::new().unwrap(); + assert!(parser.parse("[concat('abc)]").is_err()); + } + #[test] fn invalid_function() { let mut parser = Statement::new().unwrap(); diff --git a/lib/dsc-lib/src/plugins/command_name.rs b/lib/dsc-lib/src/plugins/command_name.rs new file mode 100644 index 000000000..19f8e4080 --- /dev/null +++ b/lib/dsc-lib/src/plugins/command_name.rs @@ -0,0 +1,308 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +use std::{fmt::{Display, Formatter}, ops::Deref, str::FromStr}; + +use miette::Diagnostic; +use rust_i18n::t; +use schemars::JsonSchema; +use serde::{Deserialize, Serialize}; +use thiserror::Error; + +use crate::schemas::dsc_repo::{DscRepoSchema, schema_i18n}; + +/// Defines the name of the command a plug-in adds to a DSC host. +/// +/// A command name is what the user types after the name of the host, like `lint` in `dsc lint`. +/// A valid command name: +/// +/// - Starts with a lowercase ASCII letter. +/// - Contains only lowercase ASCII letters, ASCII digits, and hyphens. +/// - Doesn't end with a hyphen or contain consecutive hyphens. +/// +/// These rules are the same as the [`VALIDATING_PATTERN`]. DSC compares command names +/// case-sensitively, and because a valid name never contains an uppercase letter, two names are +/// equal only when their text is identical. +/// +/// Every constructor validates these rules, and deserializing a name that breaks them fails, so an +/// instance is always valid. +/// +/// # Examples +/// +/// ```rust +/// use dsc_lib::plugins::command_name::PluginCommandName; +/// +/// let name = PluginCommandName::parse("lint").unwrap(); +/// assert_eq!(name, "lint"); +/// +/// assert!(PluginCommandName::parse("Lint").is_err()); +/// assert!(PluginCommandName::parse("resource-kit").is_ok()); +/// assert!(PluginCommandName::parse("resource--kit").is_err()); +/// ``` +/// +/// [`VALIDATING_PATTERN`]: PluginCommandName::VALIDATING_PATTERN +#[derive( + Clone, + Debug, + PartialEq, + Eq, + Hash, + PartialOrd, + Ord, + Serialize, + Deserialize, + JsonSchema, + DscRepoSchema +)] +#[dsc_repo_schema(base_name = "pluginCommandName", folder_path = "definitions")] +#[serde(try_from = "String", into = "String")] +#[schemars( + transform = PluginCommandName::transform_export_schema_uris, + transform = PluginCommandName::transform_schema_docs_strict, + extend( + "pattern" = PluginCommandName::VALIDATING_PATTERN, + "patternErrorMessage" = schema_i18n!("patternErrorMessage"), + ) +)] +pub struct PluginCommandName(String); + +/// Defines the errors that can occur when parsing a [`PluginCommandName`]. +#[derive(Debug, Clone, PartialEq, Eq, Error, Diagnostic)] +pub enum PluginCommandNameError { + /// Indicates that the text isn't a valid command name for one or more reasons. + /// + /// The `errors` field lists every rule the text breaks, so the author of a plug-in manifest + /// can review all of them at once instead of discovering them one at a time. + #[error("{t}", t = t!( + "plugins.command_name.invalidName", + name = text, + err = errors.iter().map(ToString::to_string).collect::>().join(", ") + ))] + InvalidName { + /// The text that failed to parse as a command name. + text: String, + /// The specific rules the text breaks. + #[related] + errors: Vec, + }, + + /// Indicates that the text is an empty string. + /// + /// Validation raises this error in the `errors` field of the main [`InvalidName`] error. + /// + /// [`InvalidName`]: Self::InvalidName + #[error("{t}", t = t!("plugins.command_name.emptyName"))] + EmptyName, + + /// Indicates that the text doesn't start with a lowercase ASCII letter. + /// + /// Validation raises this error in the `errors` field of the main [`InvalidName`] error. + /// + /// [`InvalidName`]: Self::InvalidName + #[error("{t}", t = t!("plugins.command_name.invalidFirstCharacter", character = character))] + InvalidFirstCharacter { + /// The first character of the text. + character: char, + }, + + /// Indicates that the text contains characters other than lowercase ASCII letters, ASCII + /// digits, and hyphens. + /// + /// Validation raises this error in the `errors` field of the main [`InvalidName`] error. + /// + /// [`InvalidName`]: Self::InvalidName + #[error("{t}", t = t!("plugins.command_name.invalidCharacters", characters = characters))] + InvalidCharacters { + /// The distinct invalid characters, in the order they first appear in the text. + characters: String, + }, + + /// Indicates that the text ends with a hyphen or contains consecutive hyphens. + /// + /// Validation raises this error in the `errors` field of the main [`InvalidName`] error. + /// + /// [`InvalidName`]: Self::InvalidName + #[error("{t}", t = t!("plugins.command_name.misplacedHyphen"))] + MisplacedHyphen, +} + +impl PluginCommandName { + /// Defines the regular expression that a valid command name matches. + /// + /// The name must start with a lowercase ASCII letter, followed by any number of lowercase + /// ASCII letters and ASCII digits. A hyphen may separate groups of letters and digits, but the + /// name can't end with a hyphen or contain consecutive hyphens. + /// + /// [`parse()`] checks the same rules without the regular expression so it can report every + /// rule a name breaks. The pattern is published in the JSON Schema for the type. + /// + /// [`parse()`]: Self::parse + pub const VALIDATING_PATTERN: &str = r"^[a-z][a-z0-9]*(-[a-z0-9]+)*$"; + + /// Parses a string as a command name. + /// + /// # Arguments + /// + /// * `text` - The text to parse as a command name. + /// + /// # Errors + /// + /// Returns [`PluginCommandNameError::InvalidName`] when the text breaks any of the rules for + /// a command name. The error lists every rule the text breaks. + pub fn parse(text: &str) -> Result { + Self::validate(text)?; + Ok(Self(text.to_string())) + } + + /// Validates a string as a command name without creating an instance. + /// + /// # Arguments + /// + /// * `text` - The text to validate as a command name. + /// + /// # Errors + /// + /// Returns [`PluginCommandNameError::InvalidName`] when the text breaks any of the rules for + /// a command name. The error lists every rule the text breaks: + /// + /// * [`PluginCommandNameError::EmptyName`] when the text is an empty string. No other rule is + /// checked for an empty string. + /// * [`PluginCommandNameError::InvalidFirstCharacter`] when the text doesn't start with a + /// lowercase ASCII letter. + /// * [`PluginCommandNameError::InvalidCharacters`] when the text contains characters other + /// than lowercase ASCII letters, ASCII digits, and hyphens. + /// * [`PluginCommandNameError::MisplacedHyphen`] when the text ends with a hyphen or contains + /// consecutive hyphens. + pub fn validate(text: &str) -> Result<(), PluginCommandNameError> { + let mut errors = Vec::new(); + + match text.chars().next() { + None => errors.push(PluginCommandNameError::EmptyName), + Some(first) => { + if !first.is_ascii_lowercase() { + errors.push(PluginCommandNameError::InvalidFirstCharacter { character: first }); + } + + let mut invalid_characters = String::new(); + for character in text.chars() { + let is_valid = character.is_ascii_lowercase() + || character.is_ascii_digit() + || character == '-'; + if !is_valid && !invalid_characters.contains(character) { + invalid_characters.push(character); + } + } + if !invalid_characters.is_empty() { + errors.push(PluginCommandNameError::InvalidCharacters { + characters: invalid_characters, + }); + } + + if text.ends_with('-') || text.contains("--") { + errors.push(PluginCommandNameError::MisplacedHyphen); + } + }, + } + + if errors.is_empty() { + Ok(()) + } else { + Err(PluginCommandNameError::InvalidName { text: text.to_string(), errors }) + } + } +} + +// Enables using command names in `format!()` and similar macros. +impl Display for PluginCommandName { + fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result { + write!(f, "{}", self.0) + } +} + +// Enables passing a command name as `&str`. +impl AsRef for PluginCommandName { + fn as_ref(&self) -> &str { + self.0.as_ref() + } +} + +// Enables directly accessing string methods on a command name, like `starts_with()`. +impl Deref for PluginCommandName { + type Target = str; + + fn deref(&self) -> &Self::Target { + self.0.deref() + } +} + +impl FromStr for PluginCommandName { + type Err = PluginCommandNameError; + + fn from_str(s: &str) -> Result { + Self::parse(s) + } +} + +// Enables converting from a `String` and raising the appropriate error for an invalid name. Serde +// uses this implementation to validate a name while deserializing it. +impl TryFrom for PluginCommandName { + type Error = PluginCommandNameError; + + fn try_from(value: String) -> Result { + Self::validate(&value)?; + Ok(Self(value)) + } +} + +// Enables converting from a string slice and raising the appropriate error for an invalid name. +impl TryFrom<&str> for PluginCommandName { + type Error = PluginCommandNameError; + + fn try_from(value: &str) -> Result { + Self::parse(value) + } +} + +// Enables infallibly converting a command name into the underlying string. +impl From for String { + fn from(value: PluginCommandName) -> Self { + value.0 + } +} + +// Enable comparing the newtype with strings in either direction. +impl PartialEq for PluginCommandName { + fn eq(&self, other: &String) -> bool { + &self.0 == other + } +} + +impl PartialEq for String { + fn eq(&self, other: &PluginCommandName) -> bool { + self == &other.0 + } +} + +impl PartialEq for PluginCommandName { + fn eq(&self, other: &str) -> bool { + self.0 == other + } +} + +impl PartialEq for str { + fn eq(&self, other: &PluginCommandName) -> bool { + self == other.0 + } +} + +impl PartialEq<&str> for PluginCommandName { + fn eq(&self, other: &&str) -> bool { + self.0 == *other + } +} + +impl PartialEq for &str { + fn eq(&self, other: &PluginCommandName) -> bool { + *self == other.0 + } +} diff --git a/lib/dsc-lib/src/plugins/discovery.rs b/lib/dsc-lib/src/plugins/discovery.rs new file mode 100644 index 000000000..e5c39da7d --- /dev/null +++ b/lib/dsc-lib/src/plugins/discovery.rs @@ -0,0 +1,338 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +use rust_i18n::t; +use serde_json::Value; +use std::collections::HashSet; +use std::ffi::OsStr; +use std::fs::read_to_string; +use std::path::{Path, PathBuf}; +use tracing::{debug, trace, warn}; + +use crate::discovery::command_discovery::{ + CommandDiscovery, + DSC_MANIFEST_LIST_EXTENSIONS, + evaluate_condition, +}; +use crate::dscerror::DscError; +use crate::plugins::command_name::PluginCommandName; +use crate::plugins::dscplugin::DscPlugin; +use crate::plugins::plugin_manifest::PluginManifest; +use crate::settings::global as global_settings; + +/// The file name suffixes that identify a plug-in manifest. +pub const DSC_PLUGIN_EXTENSIONS: [&str; 3] = [".dsc.plugin.json", ".dsc.plugin.yaml", ".dsc.plugin.yml"]; + +/// Discovers the command plug-ins that are available to a DSC host. +/// +/// DSC searches the same directories it searches for resource and extension manifests, in the +/// same order, and applies the rules that [`discover_plugins_in()`] describes. +/// +/// When the `plugins.enabled` setting is `false`, the function doesn't search and returns an empty +/// list. +/// +/// # Arguments +/// +/// * `reserved_commands` - The names and aliases of the commands that are built into the host. +/// +/// # Returns +/// +/// The discovered plug-ins in the order DSC found them, including shadowed plug-ins. +/// +/// # Errors +/// +/// Returns a [`DscError`] when DSC can't resolve the directories to search, for example when the +/// `resourcePath.restricted` setting is `true` and `resourcePath.directories` is empty. +pub fn discover_plugins>(reserved_commands: &[S]) -> Result, DscError> { + if !global_settings::get_or_init().resolved().plugins.enabled.value { + debug!("{}", t!("plugins.discovery.disabled", setting = "plugins.enabled")); + return Ok(Vec::new()); + } + + let directories = CommandDiscovery::get_resource_paths()?; + Ok(discover_plugins_in(&directories, reserved_commands)) +} + +/// Discovers the command plug-ins defined in a list of directories. +/// +/// DSC reads the files whose name ends with one of the [`DSC_PLUGIN_EXTENSIONS`] and the `plugins` +/// property of manifest lists. Subdirectories aren't searched. Discovery never runs a plug-in and +/// never asks a discovery extension for manifests. +/// +/// The function applies these rules to what it finds: +/// +/// 1. A manifest that fails to load or validate is skipped with a warning. +/// 1. A manifest whose `condition` evaluates to `false` is skipped. +/// 1. A manifest whose command name is one of the `reserved_commands` is skipped with a warning, +/// because the host matches its own commands first and the plug-in could never run. +/// 1. When the same type is found more than once, the highest version is kept. +/// 1. When plug-ins of different types use the same command name, the one found first provides +/// the command and the others are marked as [`shadowed`]. Within one directory, the type name +/// that sorts first is found first. +/// +/// Unlike [`discover_plugins()`], this function doesn't consult the DSC settings. +/// +/// # Arguments +/// +/// * `directories` - The directories to search, in the order to search them. +/// * `reserved_commands` - The names and aliases of the commands that are built into the host. +/// +/// # Returns +/// +/// The discovered plug-ins in the order DSC found them, including shadowed plug-ins. +/// +/// [`shadowed`]: DscPlugin::shadowed +pub fn discover_plugins_in>(directories: &[PathBuf], reserved_commands: &[S]) -> Vec { + let mut plugins: Vec = Vec::new(); + for directory in directories { + let mut found = load_plugins_in_directory(directory); + found.sort_by(|a, b| a.type_name.cmp(&b.type_name)); + + for plugin in found { + if reserved_commands.iter().any(|reserved| &*plugin.command == reserved.as_ref()) { + warn!("{}", t!( + "plugins.discovery.reservedCommand", + plugin = plugin.type_name, + path = plugin.path.display(), + command = plugin.command + )); + continue; + } + + if let Some(existing) = plugins.iter_mut().find(|p| p.type_name == plugin.type_name) { + if plugin.version > existing.version { + trace!("{}", t!( + "plugins.discovery.newerVersionFound", + plugin = plugin.type_name, + version = plugin.version, + path = plugin.path.display() + )); + *existing = plugin; + } + continue; + } + + trace!("{}", t!( + "plugins.discovery.pluginFound", + plugin = plugin.type_name, + version = plugin.version, + command = plugin.command + )); + plugins.push(plugin); + } + } + + let mut provided: HashSet = HashSet::new(); + for plugin in &mut plugins { + plugin.shadowed = !provided.insert(plugin.command.clone()); + if plugin.shadowed { + debug!("{}", t!( + "plugins.discovery.pluginShadowed", + plugin = plugin.type_name, + command = plugin.command + )); + } + } + + plugins +} + +/// Finds the plug-in that provides a command. +/// +/// # Arguments +/// +/// * `plugins` - The plug-ins that [`discover_plugins()`] returned. +/// * `command` - The command name to look up. The comparison is case-sensitive. +/// +/// # Returns +/// +/// The plug-in that provides the command, or [`None`] when no plug-in does. A shadowed plug-in is +/// never returned. +#[must_use] +pub fn find_plugin<'a>(plugins: &'a [DscPlugin], command: &str) -> Option<&'a DscPlugin> { + plugins.iter().find(|plugin| !plugin.shadowed && &*plugin.command == command) +} + +/// Finds a plug-in by its fully qualified type name. +/// +/// Unlike [`find_plugin()`], this function also returns a shadowed plug-in, which is how a user +/// runs a plug-in that doesn't provide its command. +/// +/// # Arguments +/// +/// * `plugins` - The plug-ins that [`discover_plugins()`] returned. +/// * `type_name` - The type name to look up, like `OpenDSC/Lint`. The comparison is +/// case-insensitive. +/// +/// # Returns +/// +/// The plug-in with the type name, or [`None`] when no plug-in has it. +#[must_use] +pub fn find_plugin_by_type<'a>(plugins: &'a [DscPlugin], type_name: &str) -> Option<&'a DscPlugin> { + plugins.iter().find(|plugin| plugin.type_name == type_name) +} + +/// Loads every plug-in defined in a directory. +/// +/// Manifests that can't be loaded, don't validate, or whose condition isn't met are skipped. A +/// directory that can't be read yields no plug-ins. +/// +/// # Arguments +/// +/// * `directory` - The directory to search. Subdirectories aren't searched. +fn load_plugins_in_directory(directory: &Path) -> Vec { + let Ok(entries) = directory.read_dir() else { + trace!("{}", t!("plugins.discovery.directoryNotReadable", path = directory.display())); + return Vec::new(); + }; + + // Sort the candidates so the result doesn't depend on the order the file system lists them. + let mut candidates: Vec<(PathBuf, bool)> = entries + .filter_map(Result::ok) + .filter_map(|entry| { + let path = entry.path(); + let file_name = path.file_name().and_then(OsStr::to_str)?.to_lowercase(); + if DSC_PLUGIN_EXTENSIONS.iter().any(|ext| file_name.ends_with(ext)) { + Some((path, false)) + } else if DSC_MANIFEST_LIST_EXTENSIONS.iter().any(|ext| file_name.ends_with(ext)) { + Some((path, true)) + } else { + None + } + }) + .collect(); + candidates.sort(); + + let mut plugins = Vec::new(); + for (path, is_manifest_list) in candidates { + let manifests = if is_manifest_list { + load_manifests_from_list(&path) + } else { + match load_manifest_file(&path) { + Ok(manifest) => vec![manifest], + Err(err) => { + warn!("{}", t!("plugins.discovery.failedLoadManifest", path = path.display(), err = err)); + continue; + } + } + }; + + for manifest in manifests { + match load_plugin(&path, manifest) { + Ok(Some(plugin)) => plugins.push(plugin), + Ok(None) => {}, + Err(err) => { + warn!("{}", t!("plugins.discovery.failedLoadManifest", path = path.display(), err = err)); + } + } + } + } + + plugins +} + +/// Reads a plug-in manifest from a file. +/// +/// # Arguments +/// +/// * `path` - The path of a file whose name ends with one of the [`DSC_PLUGIN_EXTENSIONS`]. +/// +/// # Errors +/// +/// Returns a [`DscError`] when the file can't be read or its content isn't a valid plug-in +/// manifest. +fn load_manifest_file(path: &Path) -> Result { + let contents = read_to_string(path)?; + if is_json_file(path) { + Ok(serde_json::from_str::(&contents)?) + } else { + Ok(serde_yaml::from_str::(&contents)?) + } +} + +/// Reads the plug-in manifests from the `plugins` property of a manifest list. +/// +/// Only the `plugins` property is interpreted, so an invalid resource or extension in the same +/// list doesn't hide the plug-ins. Resource discovery reports a list that can't be parsed, so this +/// function only logs that at the debug level. An invalid entry in `plugins` is skipped with a +/// warning. +/// +/// # Arguments +/// +/// * `path` - The path of a manifest list file. +fn load_manifests_from_list(path: &Path) -> Vec { + let list = read_to_string(path).map_err(DscError::from).and_then(|contents| { + if is_json_file(path) { + Ok(serde_json::from_str::(&contents)?) + } else { + Ok(serde_yaml::from_str::(&contents)?) + } + }); + let list = match list { + Ok(list) => list, + Err(err) => { + debug!("{}", t!("plugins.discovery.failedLoadManifestList", path = path.display(), err = err)); + return Vec::new(); + } + }; + + let Some(entries) = list.get("plugins").filter(|plugins| !plugins.is_null()) else { + return Vec::new(); + }; + let Some(entries) = entries.as_array() else { + warn!("{}", t!("plugins.discovery.pluginsNotArray", path = path.display())); + return Vec::new(); + }; + + let mut manifests = Vec::with_capacity(entries.len()); + for entry in entries { + match serde_json::from_value::(entry.clone()) { + Ok(manifest) => manifests.push(manifest), + Err(err) => { + warn!("{}", t!("plugins.discovery.failedLoadManifest", path = path.display(), err = err)); + } + } + } + + manifests +} + +/// Validates a manifest and evaluates its condition. +/// +/// # Arguments +/// +/// * `path` - The path of the file that defines the manifest. +/// * `manifest` - The manifest to turn into a plug-in. +/// +/// # Returns +/// +/// The plug-in, or [`None`] when the condition of the manifest evaluates to `false`. +/// +/// # Errors +/// +/// Returns a [`DscError`] when the manifest doesn't validate or its condition can't be evaluated +/// as a boolean. +fn load_plugin(path: &Path, manifest: PluginManifest) -> Result, DscError> { + manifest.validate()?; + + if !evaluate_condition(manifest.condition.as_deref())? { + debug!("{}", t!( + "plugins.discovery.conditionNotMet", + path = path.display(), + condition = manifest.condition.as_deref().unwrap_or_default(), + plugin = manifest.r#type + )); + return Ok(None); + } + + Ok(Some(DscPlugin::from_manifest(path, manifest))) +} + +/// Indicates whether a manifest file is JSON, as opposed to YAML, based on its file extension. +/// +/// # Arguments +/// +/// * `path` - The path of the manifest file. +fn is_json_file(path: &Path) -> bool { + path.extension().is_some_and(|ext| ext.eq_ignore_ascii_case("json")) +} diff --git a/lib/dsc-lib/src/plugins/dscplugin.rs b/lib/dsc-lib/src/plugins/dscplugin.rs new file mode 100644 index 000000000..03a11e239 --- /dev/null +++ b/lib/dsc-lib/src/plugins/dscplugin.rs @@ -0,0 +1,153 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +use schemars::JsonSchema; +use serde::{Deserialize, Serialize}; +use std::ffi::OsString; +use std::path::{Path, PathBuf}; + +use crate::dscerror::DscError; +use crate::dscresources::command_resource::verify_executable_is_invokable; +use crate::plugins::command_name::PluginCommandName; +use crate::plugins::plugin_manifest::PluginManifest; +use crate::schemas::dsc_repo::{DscRepoSchema, schema_i18n}; +use crate::types::{FullyQualifiedTypeName, SemanticVersion}; +use crate::util::canonicalize_which; + +/// Describes a discovered command plug-in. +/// +/// This is the data a host reports when it lists the available plug-ins, and what it needs to run +/// the plug-in's command. +#[derive(Clone, Debug, PartialEq, Deserialize, Serialize, JsonSchema, DscRepoSchema)] +#[serde(deny_unknown_fields, rename_all = "camelCase")] +#[dsc_repo_schema(base_name = "list", folder_path = "outputs/plugin")] +#[schemars( + transform = DscPlugin::transform_export_schema_uris, + transform = DscPlugin::transform_schema_docs +)] +pub struct DscPlugin { + /// The namespaced name that identifies the plug-in. + #[serde(rename = "type")] + #[schemars(extend( + "title" = schema_i18n!("fields.type.title"), + "description" = schema_i18n!("fields.type.description"), + "markdownDescription" = schema_i18n!("fields.type.markdownDescription"), + ))] + pub type_name: FullyQualifiedTypeName, + /// The version of the plug-in. + #[schemars(extend( + "title" = schema_i18n!("fields.version.title"), + "description" = schema_i18n!("fields.version.description"), + "markdownDescription" = schema_i18n!("fields.version.markdownDescription"), + ))] + pub version: SemanticVersion, + /// The name of the command the plug-in adds. + #[schemars(extend( + "title" = schema_i18n!("fields.command.title"), + "description" = schema_i18n!("fields.command.description"), + "markdownDescription" = schema_i18n!("fields.command.markdownDescription"), + ))] + pub command: PluginCommandName, + /// The description of the command. + #[schemars(extend( + "title" = schema_i18n!("fields.description.title"), + "description" = schema_i18n!("fields.description.description"), + "markdownDescription" = schema_i18n!("fields.description.markdownDescription"), + ))] + pub description: Option, + /// An optional message indicating the plug-in is deprecated. + #[schemars(extend( + "title" = schema_i18n!("fields.deprecationMessage.title"), + "description" = schema_i18n!("fields.deprecationMessage.description"), + "markdownDescription" = schema_i18n!("fields.deprecationMessage.markdownDescription"), + ))] + pub deprecation_message: Option, + /// The path of the file that defines the plug-in manifest. + #[schemars(extend( + "title" = schema_i18n!("fields.path.title"), + "description" = schema_i18n!("fields.path.description"), + "markdownDescription" = schema_i18n!("fields.path.markdownDescription"), + ))] + pub path: PathBuf, + /// The directory that contains the plug-in manifest. + #[schemars(extend( + "title" = schema_i18n!("fields.directory.title"), + "description" = schema_i18n!("fields.directory.description"), + "markdownDescription" = schema_i18n!("fields.directory.markdownDescription"), + ))] + pub directory: PathBuf, + /// Indicates whether another plug-in that DSC found earlier provides the same command. A + /// shadowed plug-in is never run for its command. + #[schemars(extend( + "title" = schema_i18n!("fields.shadowed.title"), + "description" = schema_i18n!("fields.shadowed.description"), + "markdownDescription" = schema_i18n!("fields.shadowed.markdownDescription"), + ))] + pub shadowed: bool, + /// The manifest of the plug-in. + pub manifest: PluginManifest, +} + +/// Defines the executable and the leading arguments to run for a plug-in command. +/// +/// The host appends the arguments the user specified after [`args`] before it starts +/// [`executable`]. +/// +/// [`args`]: Self::args +/// [`executable`]: Self::executable +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct PluginInvocation { + /// The resolved path of the executable to run. + pub executable: PathBuf, + /// The arguments from the manifest, to pass before the arguments the user specified. + pub args: Vec, +} + +impl DscPlugin { + /// Creates the description of a plug-in from its manifest. + /// + /// The plug-in isn't marked as shadowed. Discovery decides that after it has found every + /// plug-in. + /// + /// # Arguments + /// + /// * `manifest_path` - The path of the file that defines the manifest. + /// * `manifest` - The manifest of the plug-in. + #[must_use] + pub fn from_manifest(manifest_path: &Path, manifest: PluginManifest) -> Self { + Self { + type_name: manifest.r#type.clone(), + version: manifest.version.clone(), + command: manifest.command.name.clone(), + description: manifest.description.clone(), + deprecation_message: manifest.deprecation_message.clone(), + path: manifest_path.to_path_buf(), + directory: manifest_path.parent().map(Path::to_path_buf).unwrap_or_default(), + shadowed: false, + manifest, + } + } + + /// Resolves the executable and the leading arguments to run for the plug-in's command. + /// + /// DSC resolves the executable the way it does for resource and extension manifests: first in + /// the `PATH` environment variable, then relative to the directory that contains the + /// manifest. + /// + /// # Errors + /// + /// * [`DscError::CommandOperation`] when the executable can't be found. + /// * [`DscError::CommandNotInRestrictedDirs`] when the `resourcePath.restricted` setting is + /// `true` and the executable isn't in one of the `resourcePath.directories`. + pub fn resolve_invocation(&self) -> Result { + let executable = PathBuf::from( + canonicalize_which(&self.manifest.command.executable, Some(&self.directory))? + ); + verify_executable_is_invokable(&executable)?; + + Ok(PluginInvocation { + executable, + args: self.manifest.command.to_command_args(&self.directory), + }) + } +} diff --git a/lib/dsc-lib/src/plugins/mod.rs b/lib/dsc-lib/src/plugins/mod.rs new file mode 100644 index 000000000..54a58a382 --- /dev/null +++ b/lib/dsc-lib/src/plugins/mod.rs @@ -0,0 +1,24 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +//! Defines command plug-ins for DSC. +//! +//! A command plug-in adds a top-level command to a DSC host, like the `dsc` CLI, without being +//! compiled into the host. A plug-in is described by a [`PluginManifest`], which names the command +//! and the executable that implements it. The host discovers the manifests with +//! [`discover_plugins()`], finds the plug-in that provides a command with [`find_plugin()`], and +//! runs the executable that [`DscPlugin::resolve_invocation()`] returns. +//! +//! Plug-ins differ from extensions. DSC calls an extension with a fixed contract while it +//! processes an operation. A person or a script calls a plug-in command, and the plug-in defines +//! its own arguments and output. +//! +//! [`PluginManifest`]: plugin_manifest::PluginManifest +//! [`discover_plugins()`]: discovery::discover_plugins +//! [`find_plugin()`]: discovery::find_plugin +//! [`DscPlugin::resolve_invocation()`]: dscplugin::DscPlugin::resolve_invocation + +pub mod command_name; +pub mod discovery; +pub mod dscplugin; +pub mod plugin_manifest; diff --git a/lib/dsc-lib/src/plugins/plugin_manifest.rs b/lib/dsc-lib/src/plugins/plugin_manifest.rs new file mode 100644 index 000000000..768cfde8b --- /dev/null +++ b/lib/dsc-lib/src/plugins/plugin_manifest.rs @@ -0,0 +1,215 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +use rust_i18n::t; +use schemars::JsonSchema; +use serde::{Deserialize, Serialize}; +use serde_json::{Map, Value}; +use std::ffi::OsString; +use std::path::{Component, Path, PathBuf}; + +use crate::dscerror::DscError; +use crate::plugins::command_name::PluginCommandName; +use crate::schemas::dsc_repo::{DscRepoSchema, schema_i18n}; +use crate::types::{FullyQualifiedTypeName, SemanticVersion, TagList}; + +/// Defines a command plug-in: the command it adds to a DSC host and how to run it. +/// +/// DSC discovers plug-in manifests in files whose name ends with `.dsc.plugin.json`, +/// `.dsc.plugin.yaml`, or `.dsc.plugin.yml`, and in the `plugins` property of a manifest list. +#[derive(Debug, Clone, PartialEq, Deserialize, Serialize, JsonSchema, DscRepoSchema)] +#[serde(deny_unknown_fields, rename_all = "camelCase")] +#[schemars( + transform = PluginManifest::transform_export_schema_uris, + transform = PluginManifest::transform_schema_docs +)] +#[dsc_repo_schema( + base_name = "manifest", + folder_path = "plugin", + should_bundle = true, + schema_field( + name = schema_version, + title = t!("plugins.plugin_manifest.pluginManifestSchemaTitle"), + description = t!("plugins.plugin_manifest.pluginManifestSchemaDescription"), + ) +)] +pub struct PluginManifest { + /// The version of the plug-in manifest schema. + #[serde(rename = "$schema")] + #[schemars(schema_with = "PluginManifest::recognized_schema_uris_subschema")] + pub schema_version: String, + /// The namespaced name that identifies the plug-in. + #[serde(rename = "type")] + #[schemars(extend( + "title" = schema_i18n!("fields.type.title"), + "description" = schema_i18n!("fields.type.description"), + "markdownDescription" = schema_i18n!("fields.type.markdownDescription"), + ))] + pub r#type: FullyQualifiedTypeName, + /// The version of the plug-in using semantic versioning. + #[schemars(extend( + "title" = schema_i18n!("fields.version.title"), + "description" = schema_i18n!("fields.version.description"), + "markdownDescription" = schema_i18n!("fields.version.markdownDescription"), + ))] + pub version: SemanticVersion, + /// An optional condition for the plug-in to be available. If the condition evaluates to + /// false, DSC treats the plug-in as not installed. + #[serde(skip_serializing_if = "Option::is_none")] + #[schemars(extend( + "title" = schema_i18n!("fields.condition.title"), + "description" = schema_i18n!("fields.condition.description"), + "markdownDescription" = schema_i18n!("fields.condition.markdownDescription"), + ))] + pub condition: Option, + /// An optional message indicating the plug-in is deprecated. If provided, DSC shows the + /// message as a warning each time the command is invoked. + #[serde(skip_serializing_if = "Option::is_none")] + #[schemars(extend( + "title" = schema_i18n!("fields.deprecationMessage.title"), + "description" = schema_i18n!("fields.deprecationMessage.description"), + "markdownDescription" = schema_i18n!("fields.deprecationMessage.markdownDescription"), + ))] + pub deprecation_message: Option, + /// A one-line description of the command, shown in help and in the list of plug-ins. + #[serde(skip_serializing_if = "Option::is_none")] + #[schemars(extend( + "title" = schema_i18n!("fields.description.title"), + "description" = schema_i18n!("fields.description.description"), + "markdownDescription" = schema_i18n!("fields.description.markdownDescription"), + ))] + pub description: Option, + /// Tags for the plug-in. + #[serde(default, skip_serializing_if = "TagList::is_empty")] + #[schemars(extend( + "title" = schema_i18n!("fields.tags.title"), + "description" = schema_i18n!("fields.tags.description"), + "markdownDescription" = schema_i18n!("fields.tags.markdownDescription"), + ))] + pub tags: TagList, + /// The command the plug-in adds and how DSC runs it. + pub command: PluginCommand, + /// Arbitrary data for the plug-in that DSC doesn't interpret. + #[serde(skip_serializing_if = "Option::is_none")] + #[schemars(extend( + "title" = schema_i18n!("fields.metadata.title"), + "description" = schema_i18n!("fields.metadata.description"), + "markdownDescription" = schema_i18n!("fields.metadata.markdownDescription"), + ))] + pub metadata: Option>, +} + +impl PluginManifest { + /// Validates the rules for a plug-in manifest that deserialization doesn't enforce. + /// + /// # Errors + /// + /// * [`DscError::UnrecognizedSchemaUri`] when the `$schema` property isn't one of the + /// recognized URIs for the plug-in manifest schema. + /// * [`DscError::InvalidManifest`] when a `manifestRelativePath` argument is empty or isn't a + /// relative path. + pub fn validate(&self) -> Result<(), DscError> { + self.validate_schema_uri()?; + + for arg in self.command.args.iter().flatten() { + if let PluginArgKind::ManifestRelativePath { manifest_relative_path } = arg + && !is_manifest_relative(manifest_relative_path) { + return Err(DscError::InvalidManifest(t!( + "plugins.plugin_manifest.pathNotRelative", + plugin = self.r#type, + path = manifest_relative_path.display() + ).to_string())); + } + } + + Ok(()) + } +} + +/// Defines the command a plug-in adds to a DSC host and how to run it. +#[derive(Debug, Clone, PartialEq, Eq, Deserialize, Serialize, JsonSchema, DscRepoSchema)] +#[serde(deny_unknown_fields, rename_all = "camelCase")] +#[schemars( + transform = PluginCommand::transform_export_schema_uris, + transform = PluginCommand::transform_schema_docs +)] +#[dsc_repo_schema(base_name = "manifest.command", folder_path = "plugin")] +pub struct PluginCommand { + /// The name of the command, which the user types after the name of the host. + #[schemars(extend( + "title" = schema_i18n!("fields.name.title"), + "description" = schema_i18n!("fields.name.description"), + "markdownDescription" = schema_i18n!("fields.name.markdownDescription"), + ))] + pub name: PluginCommandName, + /// The executable to run for the command. + #[schemars(extend( + "title" = schema_i18n!("fields.executable.title"), + "description" = schema_i18n!("fields.executable.description"), + "markdownDescription" = schema_i18n!("fields.executable.markdownDescription"), + ))] + pub executable: String, + /// The arguments to pass to the executable before the arguments the user specified. + #[serde(skip_serializing_if = "Option::is_none")] + #[schemars(extend( + "title" = schema_i18n!("fields.args.title"), + "description" = schema_i18n!("fields.args.description"), + "markdownDescription" = schema_i18n!("fields.args.markdownDescription"), + ))] + pub args: Option>, +} + +impl PluginCommand { + /// Resolves the argument definitions into the arguments to pass to the executable. + /// + /// Static string arguments are passed as written. A `manifestRelativePath` argument is + /// replaced with the path of the file joined to the manifest directory. + /// + /// # Arguments + /// + /// * `manifest_directory` - The directory that contains the plug-in manifest. + #[must_use] + pub fn to_command_args(&self, manifest_directory: &Path) -> Vec { + self.args.iter().flatten().map(|arg| match arg { + PluginArgKind::String(text) => OsString::from(text), + PluginArgKind::ManifestRelativePath { manifest_relative_path } => { + manifest_directory.join(manifest_relative_path).into_os_string() + }, + }).collect() + } +} + +/// Defines an argument that DSC passes to the executable of a plug-in. +#[derive(Debug, Clone, PartialEq, Eq, Hash, Deserialize, Serialize, JsonSchema)] +#[serde(untagged, deny_unknown_fields)] +#[schemars(inline)] +pub enum PluginArgKind { + /// A static string argument to pass to the executable as written, like `-NoProfile`. + String(String), + /// A file that ships with the plug-in. DSC passes the absolute path of the file, resolved + /// against the directory that contains the manifest. A plug-in runs in the caller's working + /// directory, not in the manifest directory, so this is how a manifest names a script that + /// sits next to it. + ManifestRelativePath { + /// The path of the file relative to the directory that contains the manifest, like + /// `graph.ps1` or `scripts/graph.py`. + #[serde(rename = "manifestRelativePath")] + manifest_relative_path: PathBuf, + }, +} + +/// Indicates whether a path can be resolved against the manifest directory. +/// +/// A path is relative to the manifest directory when it isn't empty and has neither a root, like +/// `/tools/graph.ps1`, nor a prefix, like `C:graph.ps1`. Joining a path with a root or a prefix to +/// the manifest directory would discard part or all of the manifest directory. +/// +/// # Arguments +/// +/// * `path` - The path from a `manifestRelativePath` argument. +fn is_manifest_relative(path: &Path) -> bool { + !path.as_os_str().is_empty() + && !path.components().any(|component| { + matches!(component, Component::Prefix(_) | Component::RootDir) + }) +} diff --git a/lib/dsc-lib/src/settings/dsc_settings.rs b/lib/dsc-lib/src/settings/dsc_settings.rs index 9abce5f36..776b70b79 100644 --- a/lib/dsc-lib/src/settings/dsc_settings.rs +++ b/lib/dsc-lib/src/settings/dsc_settings.rs @@ -123,6 +123,8 @@ pub struct DscSettings { /// directories: [] /// append_env_path: true /// restricted: false + /// plugins: + /// enabled: true /// ``` #[schemars( title = schema_i18n!("fields.default.title"), @@ -1319,6 +1321,7 @@ impl DscSettings { resolve_field!(tracing => [level, format]); resolve_field!(resource_path => [append_env_path, directories, restricted]); + resolve_field!(plugins => [enabled]); } /// Resolves the policy settings and applies them to the resolved settings. @@ -1371,6 +1374,7 @@ impl DscSettings { // Resolve remaining fields as normal. resolve_field!(tracing => [level, format]); resolve_field!(resource_path => [append_env_path, directories, restricted]); + resolve_field!(plugins => [enabled]); } /// Indicates whether the folder containing the DSC policy settings file has diff --git a/lib/dsc-lib/src/settings/fields/mod.rs b/lib/dsc-lib/src/settings/fields/mod.rs index 66049f3d6..e302c0cbe 100644 --- a/lib/dsc-lib/src/settings/fields/mod.rs +++ b/lib/dsc-lib/src/settings/fields/mod.rs @@ -201,6 +201,8 @@ mod forbid_ignore_settings_file; pub use forbid_ignore_settings_file::*; mod ignore_settings_file; pub use ignore_settings_file::*; +mod plugins; +pub use plugins::*; mod resource_path; pub use resource_path::*; mod tracing; diff --git a/lib/dsc-lib/src/settings/fields/plugins.rs b/lib/dsc-lib/src/settings/fields/plugins.rs new file mode 100644 index 000000000..96b294afb --- /dev/null +++ b/lib/dsc-lib/src/settings/fields/plugins.rs @@ -0,0 +1,99 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +use crate::schemas::dsc_repo::DscRepoSchema; +use super::macros::*; + +define_container_field!{ + base_name: "plugins", + folder_path: "settings/fields", + field_definition: { + /// Defines the available settings for command plug-ins in DSC. + /// + /// This struct is used to represent the settings as they are defined + /// in both [`PolicyFileData`] and [`PreferenceFileData`]. + /// + /// A command plug-in adds a top-level command to the `dsc` CLI + /// without being compiled into it. When the first argument to `dsc` + /// isn't a built-in command, DSC looks for a plug-in manifest that + /// provides the command and runs the executable that the manifest + /// names. + /// + /// The plug-in settings define whether DSC uses plug-ins at all: + /// + /// - When [`enabled`] is resolved as `true`, DSC discovers plug-in + /// manifests, lists plug-in commands in the help, and dispatches + /// unknown commands to the plug-in that provides them. + /// - When [`enabled`] is resolved as `false`, DSC doesn't discover + /// plug-in manifests. A command that isn't built in fails as an + /// unrecognized subcommand, the help doesn't list plug-in commands, + /// and listing the plug-ins returns nothing. + /// + /// To configure whether DSC uses plug-ins, users can define the + /// `plugins.enabled` field in the [`Policy`], [`Machine`], [`User`], + /// or [`Workspace`] settings files. The setting can't be defined as an + /// environment variable or a command line argument. + /// + /// Defining the field in the [`Policy`] settings file gives an + /// administrator one switch to keep a machine to the built-in + /// commands, because no other scope can override a policy setting. + /// + /// The directories that DSC searches for plug-in manifests are the + /// same directories it searches for every other manifest. To limit + /// where plug-ins can be discovered instead of disabling them, see + /// [`ResourcePathFileData`]. + /// + /// [`PolicyFileData`]: crate::settings::sources::PolicyFileData + /// [`PreferenceFileData`]: crate::settings::sources::PreferenceFileData + /// [`ResourcePathFileData`]: crate::settings::fields::ResourcePathFileData + /// [`enabled`]: Self::enabled + /// [`Policy`]: crate::settings::DscSettingsScope::Policy + /// [`Machine`]: crate::settings::DscSettingsScope::Machine + /// [`User`]: crate::settings::DscSettingsScope::User + /// [`Workspace`]: crate::settings::DscSettingsScope::Workspace + #[schemars(extend("unevaluatedProperties" = false))] + struct PluginsFileData { + /// Whether DSC discovers and runs command plug-ins. + /// + /// If set to `false`, DSC doesn't discover plug-in manifests and + /// only the built-in commands are available. The default value is + /// `true`. + enabled: bool = true, + } + }, + resolved_container_type: { + /// Defines the resolved settings for command plug-ins in DSC. + /// + /// For more information about the available settings, see + /// [`PluginsFileData`]. + PluginsResolvedSettings + }, + code_default_type: { + /// Defines the structure of the code defaults for the plug-in settings + /// in DSC. + /// + /// For more information about the available settings, see + /// [`PluginsFileData`]. For more information about the code defaults, + /// see [`CODE_DEFAULT_PLUGINS`]. + PluginsCodeDefaults + }, + code_default_const: { + /// Defines the default values for the plug-in settings in DSC. + /// + /// The following snippet shows the effective code defaults as YAML + /// data: + /// + /// ```yaml + /// plugins: + /// enabled: true + /// ``` + /// + /// With these code defaults, DSC discovers plug-in manifests and + /// dispatches commands that aren't built in to the plug-in that + /// provides them. + /// + /// For more information about defining the settings, see + /// [`PluginsFileData`]. + CODE_DEFAULT_PLUGINS + } +} diff --git a/lib/dsc-lib/src/settings/mod.rs b/lib/dsc-lib/src/settings/mod.rs index e08ed4354..28ef91d8f 100644 --- a/lib/dsc-lib/src/settings/mod.rs +++ b/lib/dsc-lib/src/settings/mod.rs @@ -284,6 +284,27 @@ //! manifest, DSC can only invoke commands that exist in the specified directories. Attempting //! to invoke commands outside of the specified directories raises an error. //! +//! ## Plug-in settings +//! +//! A command plug-in adds a top-level command to the `dsc` CLI without being compiled into it. +//! When the first argument to `dsc` isn't a built-in command, DSC looks for a plug-in manifest +//! that provides the command and runs the executable that the manifest names. By default, DSC +//! discovers plug-in manifests in the same directories it searches for every other manifest. +//! +//! Users can control whether DSC uses plug-ins by defining the [`plugins.enabled`] field in the +//! [`Policy`], [`Machine`], [`User`], or [`Workspace`] settings files. The setting can't be +//! defined as an environment variable or a command line argument. +//! +//! When the resolved value for [`plugins.enabled`] is `false`, DSC doesn't discover plug-in +//! manifests. A command that isn't built in fails as an unrecognized subcommand, the help doesn't +//! list plug-in commands, and listing the plug-ins returns nothing. +//! +//! Defining the field in the [`Policy`] settings file gives an administrator one switch to keep a +//! machine to the built-in commands, because no other scope can override a policy setting. +//! +//! The resource path settings limit where DSC discovers plug-in manifests in the same way they +//! limit the discovery of every other manifest. +//! //! [scopes]: DscSettingsScope //! [`Default`]: DscSettingsScope::Default //! [`Machine`]: DscSettingsScope::Machine @@ -316,6 +337,7 @@ //! [`append_env_path`]: fields::ResourcePathResolvedSettings::append_env_path //! [`resource_path.directories`]: fields::ResourcePathResolvedSettings::directories //! [`resource_path.restricted`]: fields::ResourcePathResolvedSettings::restricted +//! [`plugins.enabled`]: fields::PluginsFileData::enabled //! [`new_with_command_line()`]: DscSettings::new_with_command_line //! [`load()`]: DscSettings::load //! [`try_load()`]: DscSettings::try_load diff --git a/lib/dsc-lib/src/settings/resolved/settings.rs b/lib/dsc-lib/src/settings/resolved/settings.rs index 31aa1bb00..7e7eed659 100644 --- a/lib/dsc-lib/src/settings/resolved/settings.rs +++ b/lib/dsc-lib/src/settings/resolved/settings.rs @@ -197,6 +197,35 @@ pub struct DscSettingsResolved { ) )] pub resource_path: ResourcePathResolvedSettings, + /// Indicates whether DSC should discover and run command plug-ins during + /// command execution. + /// + /// These settings control whether DSC looks for plug-in manifests, lists + /// plug-in commands in the help, and dispatches commands that aren't + /// built in to the plug-in that provides them. + /// + /// The following snippet shows the effective code defaults as YAML data: + /// + /// ```yaml + /// plugins: + /// enabled: true + /// ``` + /// + /// These settings can only be defined in the [policy settings file] and + /// the [preference settings files]. For more information on defining + /// them, see [`PluginsFileData`]. + /// + /// [`PluginsFileData`]: crate::settings::fields::PluginsFileData + /// [policy settings file]: crate::settings::sources::PolicyFileData + /// [preference settings files]: crate::settings::sources::PreferenceFileData + #[schemars( + title = schema_i18n!("plugins.title"), + description = schema_i18n!("plugins.description"), + extend( + "markdownDescription" = schema_i18n!("plugins.markdownDescription") + ) + )] + pub plugins: PluginsResolvedSettings, } impl Default for DscSettingsResolved { @@ -206,6 +235,7 @@ impl Default for DscSettingsResolved { ignore_settings_file: DscSettingsResolvedField::for_code_default(CODE_DEFAULT_IGNORE_SETTINGS_FILE), tracing: TracingResolvedSettings::default(), resource_path: ResourcePathResolvedSettings::default(), + plugins: PluginsResolvedSettings::default(), } } } diff --git a/lib/dsc-lib/src/settings/sources/code_defaults.rs b/lib/dsc-lib/src/settings/sources/code_defaults.rs index cb37940a2..012ffdee0 100644 --- a/lib/dsc-lib/src/settings/sources/code_defaults.rs +++ b/lib/dsc-lib/src/settings/sources/code_defaults.rs @@ -128,6 +128,29 @@ pub struct DscSettingsCodeDefaults { /// [`Environment`]: crate::settings::DscSettingsScope::Environment /// [`EnvironmentData`]: crate::settings::sources::EnvironmentData pub resource_path: ResourcePathCodeDefaults, + /// Defines whether DSC discovers and runs command plug-ins. + /// + /// The code default for this container field is defined by + /// [`CODE_DEFAULT_PLUGINS`]. + /// + /// You can override this setting in the file-based scopes only. It can't + /// be defined in the [`Environment`] or [`CommandLine`] scopes. For more + /// information, see: + /// + /// - [`PolicyFileData::plugins`] to define this setting in the [`Policy`] + /// scope. + /// - [`PreferenceFileData::plugins`] to define this setting as a + /// preference in the [`Machine`], [`User`], and [`Workspace`] scopes. + /// + /// [`Environment`]: crate::settings::DscSettingsScope::Environment + /// [`CommandLine`]: crate::settings::DscSettingsScope::CommandLine + /// [`Policy`]: crate::settings::DscSettingsScope::Policy + /// [`Machine`]: crate::settings::DscSettingsScope::Machine + /// [`User`]: crate::settings::DscSettingsScope::User + /// [`Workspace`]: crate::settings::DscSettingsScope::Workspace + /// [`PolicyFileData::plugins`]: crate::settings::sources::PolicyFileData::plugins + /// [`PreferenceFileData::plugins`]: crate::settings::sources::PreferenceFileData::plugins + pub plugins: PluginsCodeDefaults, } impl JsonSchema for DscSettingsCodeDefaults { @@ -161,12 +184,15 @@ impl JsonSchema for DscSettingsCodeDefaults { /// append_env_path: true /// directories: [] /// restricted: false +/// plugins: +/// enabled: true /// ``` pub const DSC_SETTINGS_CODE_DEFAULTS: DscSettingsCodeDefaults = DscSettingsCodeDefaults { forbid_ignore_settings_file: CODE_DEFAULT_FORBID_IGNORE_SETTINGS_FILE, ignore_settings_file: CODE_DEFAULT_IGNORE_SETTINGS_FILE, tracing: CODE_DEFAULT_TRACING, resource_path: CODE_DEFAULT_RESOURCE_PATH, + plugins: CODE_DEFAULT_PLUGINS, }; impl Default for DscSettingsCodeDefaults { diff --git a/lib/dsc-lib/src/settings/sources/policy_file.rs b/lib/dsc-lib/src/settings/sources/policy_file.rs index bcd5d47bb..78722ac16 100644 --- a/lib/dsc-lib/src/settings/sources/policy_file.rs +++ b/lib/dsc-lib/src/settings/sources/policy_file.rs @@ -136,6 +136,30 @@ pub struct PolicyFileData { ), )] pub resource_path: Option, + /// Defines settings that affect whether DSC discovers and runs command + /// plug-ins. + /// + /// You can control whether DSC dispatches commands that aren't built in + /// to a plug-in. Defining `plugins.enabled` as `false` in the policy file + /// keeps DSC to the built-in commands for every user, because no other + /// scope can override a policy setting. + /// + /// For more information, see [`PluginsFileData`]. + #[serde(skip_serializing_if = "Option::is_none")] + #[schemars( + title = schema_i18n!("plugins.title"), + description = schema_i18n!("plugins.description"), + extend( + "markdownDescription" = schema_i18n!("plugins.markdownDescription") + ), + )] + pub plugins: Option, + /// Defines the levels of the lint rules. + /// + /// The linter reads this section itself. It isn't part of the resolved settings, because the + /// levels are merged per rule and combined with the overrides of a lint run. + #[serde(skip_serializing_if = "Option::is_none")] + pub lint: Option, } /// Public API @@ -209,6 +233,7 @@ impl PolicyFileData { pub(crate) fn any_defined(&self) -> bool { self.forbid_ignore_settings_file.is_some() || self.ignore_settings_file.is_some() || + self.plugins.is_some() || self.resource_path.is_some() || self.tracing.is_some() } diff --git a/lib/dsc-lib/src/settings/sources/preference_file.rs b/lib/dsc-lib/src/settings/sources/preference_file.rs index 7d3113873..7ec2b1dcc 100644 --- a/lib/dsc-lib/src/settings/sources/preference_file.rs +++ b/lib/dsc-lib/src/settings/sources/preference_file.rs @@ -104,6 +104,28 @@ pub struct PreferenceFileData { ), )] pub resource_path: Option, + /// Defines settings that affect whether DSC discovers and runs command + /// plug-ins. + /// + /// You can control whether DSC dispatches commands that aren't built in + /// to a plug-in. + /// + /// For more information, see [`PluginsFileData`]. + #[serde(skip_serializing_if = "Option::is_none")] + #[schemars( + title = schema_i18n!("plugins.title"), + description = schema_i18n!("plugins.description"), + extend( + "markdownDescription" = schema_i18n!("plugins.markdownDescription") + ), + )] + pub plugins: Option, + /// Defines the levels of the lint rules. + /// + /// The linter reads this section itself. It isn't part of the resolved settings, because the + /// levels are merged per rule and combined with the overrides of a lint run. + #[serde(skip_serializing_if = "Option::is_none")] + pub lint: Option, } /// Public API @@ -175,6 +197,7 @@ impl PreferenceFileData { /// /// `true` if any of the file data fields are populated, otherwise `false`. pub(crate) fn any_defined(&self) -> bool { + self.plugins.is_some() || self.resource_path.is_some() || self.tracing.is_some() } diff --git a/lib/dsc-lib/src/tests/settings/sources/policy_file.rs b/lib/dsc-lib/src/tests/settings/sources/policy_file.rs index a7e4524d8..566193b6b 100644 --- a/lib/dsc-lib/src/tests/settings/sources/policy_file.rs +++ b/lib/dsc-lib/src/tests/settings/sources/policy_file.rs @@ -5,6 +5,7 @@ use crate::settings::DscSettingsError; use crate::settings::fields::{ ForbidIgnoreSettingsFileField, IgnoreSettingsFileField, + PluginsFileData, TracingFileData, TracingLevelField, TracingFormatField, @@ -162,6 +163,9 @@ mod from_file { "appendEnvPath": true, "directories": ["dir1", "dir2"], "restricted": false, + }, + "plugins": { + "enabled": false } }); let json_str = serde_json::to_string_pretty(&json).unwrap(); @@ -192,6 +196,9 @@ mod from_file { ]), restricted: Some(false), }), + plugins: Some(PluginsFileData { + enabled: Some(false), + }), ..Default::default() }; diff --git a/lib/dsc-lib/src/tests/settings/sources/preference_file.rs b/lib/dsc-lib/src/tests/settings/sources/preference_file.rs index 3722da09f..887cc4a88 100644 --- a/lib/dsc-lib/src/tests/settings/sources/preference_file.rs +++ b/lib/dsc-lib/src/tests/settings/sources/preference_file.rs @@ -3,6 +3,7 @@ use crate::settings::DscSettingsError; use crate::settings::fields::{ + PluginsFileData, TracingFileData, TracingLevelField, TracingFormatField, @@ -168,6 +169,9 @@ use super::*; "appendEnvPath": true, "directories": ["dir1", "dir2"], "restricted": false, + }, + "plugins": { + "enabled": false } }); let json_str = serde_json::to_string_pretty(&json).unwrap(); @@ -196,6 +200,9 @@ use super::*; ]), restricted: Some(false), }), + plugins: Some(PluginsFileData { + enabled: Some(false), + }), ..Default::default() }; diff --git a/lib/dsc-lib/src/util.rs b/lib/dsc-lib/src/util.rs index 421d7f6a4..72203da0b 100644 --- a/lib/dsc-lib/src/util.rs +++ b/lib/dsc-lib/src/util.rs @@ -12,6 +12,36 @@ use std::{ }; use which::which; +pub const DSC_IGNORE_SETTINGS_FILE: &str = "DSC_IGNORE_SETTINGS_FILE"; + +/// Parse a JSON or YAML string into a JSON value. +/// +/// JSON is tried first; if that fails the input is parsed as YAML. +/// +/// # Arguments +/// +/// * `value` - A string slice that holds the input value +/// +/// # Returns +/// +/// The parsed value +/// +/// # Errors +/// +/// Returns `DscError::Yaml` with the YAML parser error when the input is neither valid JSON nor +/// valid YAML. +pub fn parse_input_to_value(value: &str) -> Result { + match serde_json::from_str::(value) { + Ok(json) => Ok(json), + Err(_) => { + match serde_yaml::from_str::(value) { + Ok(yaml) => Ok(yaml), + Err(err) => Err(DscError::Yaml(err)), + } + } + } +} + /// Return JSON string whether the input is JSON or YAML /// /// # Arguments diff --git a/lib/dsc-lib/tests/integration/main.rs b/lib/dsc-lib/tests/integration/main.rs index 00003360a..87117ce9f 100644 --- a/lib/dsc-lib/tests/integration/main.rs +++ b/lib/dsc-lib/tests/integration/main.rs @@ -36,6 +36,7 @@ pub(crate) mod macros { mod command_resource; mod extensions; +mod plugins; #[macro_use] pub(crate) mod schemas; mod settings; mod types; diff --git a/lib/dsc-lib/tests/integration/plugins/command_name.rs b/lib/dsc-lib/tests/integration/plugins/command_name.rs new file mode 100644 index 000000000..436122009 --- /dev/null +++ b/lib/dsc-lib/tests/integration/plugins/command_name.rs @@ -0,0 +1,309 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +#[cfg(test)] +mod methods { + #[cfg(test)] + mod parse { + use dsc_lib::plugins::command_name::{PluginCommandName, PluginCommandNameError}; + use test_case::test_case; + + #[test_case("lint"; "lowercase letters")] + #[test_case("a"; "single letter")] + #[test_case("lint2"; "letters followed by a digit")] + #[test_case("resource-kit"; "hyphen between two groups")] + #[test_case("a-1-b2"; "several hyphenated groups")] + fn valid(text: &str) { + let name = PluginCommandName::parse(text).expect("name should be valid"); + + pretty_assertions::assert_eq!(name.to_string(), text); + } + + #[test_case("" => + vec![PluginCommandNameError::EmptyName]; + "empty string" + )] + #[test_case("Lint" => + vec![ + PluginCommandNameError::InvalidFirstCharacter { character: 'L' }, + PluginCommandNameError::InvalidCharacters { characters: "L".to_string() }, + ]; + "uppercase first letter" + )] + #[test_case("2lint" => + vec![PluginCommandNameError::InvalidFirstCharacter { character: '2' }]; + "digit as first character" + )] + #[test_case("-lint" => + vec![PluginCommandNameError::InvalidFirstCharacter { character: '-' }]; + "hyphen as first character" + )] + #[test_case("lint_docs" => + vec![PluginCommandNameError::InvalidCharacters { characters: "_".to_string() }]; + "underscore" + )] + #[test_case("lint docs" => + vec![PluginCommandNameError::InvalidCharacters { characters: " ".to_string() }]; + "space" + )] + #[test_case("lintDOCS.DOC" => + vec![PluginCommandNameError::InvalidCharacters { characters: "DOCS.".to_string() }]; + "lists each invalid character once" + )] + #[test_case("lint-" => + vec![PluginCommandNameError::MisplacedHyphen]; + "trailing hyphen" + )] + #[test_case("lint--docs" => + vec![PluginCommandNameError::MisplacedHyphen]; + "consecutive hyphens" + )] + #[test_case("L_i--" => + vec![ + PluginCommandNameError::InvalidFirstCharacter { character: 'L' }, + PluginCommandNameError::InvalidCharacters { characters: "L_".to_string() }, + PluginCommandNameError::MisplacedHyphen, + ]; + "reports every rule the name breaks" + )] + fn invalid(text: &str) -> Vec { + match PluginCommandName::parse(text).unwrap_err() { + PluginCommandNameError::InvalidName { text: actual_text, errors } => { + pretty_assertions::assert_eq!(actual_text, text); + errors + }, + other => panic!("expected the collecting error, got: {other:?}"), + } + } + } + + #[cfg(test)] + mod validate { + use dsc_lib::plugins::command_name::PluginCommandName; + use regex::Regex; + use test_case::test_case; + + #[test_case("lint")] + #[test_case("a")] + #[test_case("lint2")] + #[test_case("resource-kit")] + #[test_case("a-1-b2")] + #[test_case("")] + #[test_case("Lint")] + #[test_case("2lint")] + #[test_case("-lint")] + #[test_case("lint_docs")] + #[test_case("lint docs")] + #[test_case("lint-")] + #[test_case("lint--docs")] + #[test_case("-")] + #[test_case("lint\n")] + fn agrees_with_validating_pattern(text: &str) { + let pattern = Regex::new(PluginCommandName::VALIDATING_PATTERN).expect("pattern is valid"); + + pretty_assertions::assert_eq!( + PluginCommandName::validate(text).is_ok(), + pattern.is_match(text), + "validation and the published pattern disagree for '{}'", + text + ); + } + } +} + +crate::macros::test_dsc_repo_schema! { + define_statics: { + dsc_lib::plugins::command_name::PluginCommandName + }, + test_meta_schema: {}, + test_docs: { + title => "title", + description => "description", + markdown_description => "markdownDescription", + pattern_error_message => "patternErrorMessage" + }, + test_validation: { + lowercase_name_is_valid => { + input_json: json!("lint"), + expected_valid: true, + }, + hyphenated_name_is_valid => { + input_json: json!("resource-kit"), + expected_valid: true, + }, + name_with_digits_is_valid => { + input_json: json!("lint2"), + expected_valid: true, + }, + empty_string_is_invalid => { + input_json: json!(""), + expected_valid: false, + }, + uppercase_name_is_invalid => { + input_json: json!("Lint"), + expected_valid: false, + }, + name_with_leading_digit_is_invalid => { + input_json: json!("2lint"), + expected_valid: false, + }, + name_with_trailing_hyphen_is_invalid => { + input_json: json!("lint-"), + expected_valid: false, + }, + name_with_consecutive_hyphens_is_invalid => { + input_json: json!("lint--docs"), + expected_valid: false, + }, + name_with_underscore_is_invalid => { + input_json: json!("lint_docs"), + expected_valid: false, + }, + boolean_is_invalid => { + input_json: json!(true), + expected_valid: false, + }, + number_is_invalid => { + input_json: json!(1), + expected_valid: false, + }, + object_is_invalid => { + input_json: json!({"name": "lint"}), + expected_valid: false, + }, + array_is_invalid => { + input_json: json!(["lint"]), + expected_valid: false, + }, + null_is_invalid => { + input_json: json!(null), + expected_valid: false, + } + } +} + +#[cfg(test)] +mod serde { + use dsc_lib::plugins::command_name::PluginCommandName; + use serde_json::{json, Value}; + use test_case::test_case; + + #[test_case("lint"; "lowercase name")] + #[test_case("resource-kit"; "hyphenated name")] + fn serializing(name: &str) { + let actual = serde_json::to_string(&PluginCommandName::parse(name).expect("name should be valid")) + .expect("serialization should never fail"); + + pretty_assertions::assert_eq!(actual, format!(r#""{name}""#)); + } + + #[test_case(json!("lint") => matches Ok(_); "lowercase string value is valid")] + #[test_case(json!("resource-kit") => matches Ok(_); "hyphenated string value is valid")] + #[test_case(json!("Lint") => matches Err(_); "uppercase string value raises error")] + #[test_case(json!("") => matches Err(_); "empty string value raises error")] + #[test_case(json!(true) => matches Err(_); "boolean value raises error")] + #[test_case(json!(1) => matches Err(_); "integer value raises error")] + #[test_case(json!({"name": "lint"}) => matches Err(_); "object value raises error")] + #[test_case(json!(["lint"]) => matches Err(_); "array value raises error")] + #[test_case(Value::Null => matches Err(_); "null value raises error")] + fn deserializing(value: Value) -> Result { + serde_json::from_value::(value) + } + + #[test] + fn deserializing_error_names_the_broken_rule() { + let err = serde_json::from_value::(json!("lint-")).unwrap_err(); + + assert!( + err.to_string().contains("lint-"), + "expected the error to name the invalid text, got: {err}" + ); + } +} + +#[cfg(test)] +mod traits { + use std::str::FromStr; + + use dsc_lib::plugins::command_name::{PluginCommandName, PluginCommandNameError}; + use test_case::test_case; + + fn name() -> PluginCommandName { + PluginCommandName::parse("resource-kit").expect("name should be valid") + } + + #[test] + fn display() { + pretty_assertions::assert_eq!(format!("dsc {}", name()), "dsc resource-kit"); + } + + #[test] + fn as_ref() { + let name = name(); + let text: &str = name.as_ref(); + + pretty_assertions::assert_eq!(text, "resource-kit"); + } + + #[test] + fn deref() { + let name = name(); + + assert!(name.starts_with("resource")); + pretty_assertions::assert_eq!(name.len(), 12); + } + + #[test_case("lint" => matches Ok(_); "valid name")] + #[test_case("Lint" => matches Err(PluginCommandNameError::InvalidName { .. }); "invalid name")] + fn from_str(text: &str) -> Result { + PluginCommandName::from_str(text) + } + + #[test_case("lint" => matches Ok(_); "valid name")] + #[test_case("Lint" => matches Err(PluginCommandNameError::InvalidName { .. }); "invalid name")] + fn try_from_string(text: &str) -> Result { + PluginCommandName::try_from(text.to_string()) + } + + #[test_case("lint" => matches Ok(_); "valid name")] + #[test_case("Lint" => matches Err(PluginCommandNameError::InvalidName { .. }); "invalid name")] + fn try_from_str(text: &str) -> Result { + PluginCommandName::try_from(text) + } + + #[test] + fn into_string() { + let text: String = name().into(); + + pretty_assertions::assert_eq!(text, "resource-kit".to_string()); + } + + #[test] + fn partial_eq() { + let name = name(); + + assert!(name == "resource-kit"); + assert!("resource-kit" == name); + assert!(name == *"resource-kit"); + assert!(*"resource-kit" == name); + assert!(name == "resource-kit".to_string()); + assert!("resource-kit".to_string() == name); + assert!(name != "Resource-Kit"); + assert!(name != "lint"); + } + + #[test] + fn ord() { + let mut names = vec![ + PluginCommandName::parse("lint").unwrap(), + PluginCommandName::parse("graph").unwrap(), + PluginCommandName::parse("lint2").unwrap(), + ]; + names.sort(); + + pretty_assertions::assert_eq!( + names.iter().map(ToString::to_string).collect::>(), + vec!["graph".to_string(), "lint".to_string(), "lint2".to_string()] + ); + } +} diff --git a/lib/dsc-lib/tests/integration/plugins/discovery.rs b/lib/dsc-lib/tests/integration/plugins/discovery.rs new file mode 100644 index 000000000..d75f8e844 --- /dev/null +++ b/lib/dsc-lib/tests/integration/plugins/discovery.rs @@ -0,0 +1,538 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +mod helpers { + //! Defines helper functions for writing plug-in manifests to temporary directories. + + use std::fs; + use std::path::{Path, PathBuf}; + + use dsc_lib::plugins::discovery::discover_plugins_in; + use dsc_lib::plugins::dscplugin::DscPlugin; + use serde_json::{json, Value}; + use tempfile::TempDir; + + pub(super) use super::super::helpers::manifest_json; + + /// The commands the tests treat as built into the host. + pub(super) const RESERVED: [&str; 3] = ["config", "resource", "help"]; + + /// Creates a temporary directory to search for plug-ins. + pub(super) fn directory() -> TempDir { + tempfile::tempdir().expect("temporary directory should be created") + } + + /// Writes a JSON value to a file in the directory and returns the path of the file. + pub(super) fn write_json(directory: &Path, file_name: &str, value: &Value) -> PathBuf { + let path = directory.join(file_name); + fs::write(&path, serde_json::to_string_pretty(value).expect("value should serialize")) + .expect("file should be written"); + path + } + + /// Writes text to a file in the directory and returns the path of the file. + pub(super) fn write_text(directory: &Path, file_name: &str, text: &str) -> PathBuf { + let path = directory.join(file_name); + fs::write(&path, text).expect("file should be written"); + path + } + + /// Writes a manifest with only the required properties as `.dsc.plugin.json`. + pub(super) fn write_manifest(directory: &Path, type_name: &str, version: &str, command: &str) -> PathBuf { + let file_name = format!("{}.{command}.dsc.plugin.json", type_name.replace('/', ".")); + write_json(directory, &file_name, &manifest_json(type_name, version, command)) + } + + /// Returns a manifest with a condition. + pub(super) fn manifest_with_condition(type_name: &str, command: &str, condition: &str) -> Value { + let mut manifest = manifest_json(type_name, "0.1.0", command); + manifest["condition"] = json!(condition); + manifest + } + + /// Discovers the plug-ins in the directories with the reserved commands of the tests. + pub(super) fn discover(directories: &[&Path]) -> Vec { + let directories: Vec = directories.iter().map(|directory| directory.to_path_buf()).collect(); + discover_plugins_in(&directories, &RESERVED) + } + + /// Returns the type, version, command, and shadowed state of each plug-in, for comparisons. + pub(super) fn summarize(plugins: &[DscPlugin]) -> Vec<(String, String, String, bool)> { + plugins.iter().map(|plugin| ( + plugin.type_name.to_string(), + plugin.version.to_string(), + plugin.command.to_string(), + plugin.shadowed, + )).collect() + } + + /// Returns the expected summary entry for a plug-in. + pub(super) fn entry(type_name: &str, version: &str, command: &str, shadowed: bool) -> (String, String, String, bool) { + (type_name.to_string(), version.to_string(), command.to_string(), shadowed) + } +} + +#[cfg(test)] +mod discover_plugins_in { + use std::fs; + + use serde_json::json; + use test_case::test_case; + + use super::helpers::*; + + #[test] + fn empty_directory_has_no_plugins() { + let directory = directory(); + + assert!(discover(&[directory.path()]).is_empty()); + } + + #[test] + fn missing_directory_has_no_plugins() { + let directory = directory(); + let missing = directory.path().join("missing"); + + assert!(discover(&[&missing]).is_empty()); + } + + #[test_case("graph.dsc.plugin.json"; "json manifest")] + #[test_case("Graph.DSC.Plugin.JSON"; "file name suffix in another casing")] + fn finds_json_manifest(file_name: &str) { + let directory = directory(); + let path = write_json(directory.path(), file_name, &manifest_json("Test/Graph", "0.1.0", "graph")); + + let plugins = discover(&[directory.path()]); + + pretty_assertions::assert_eq!(summarize(&plugins), vec![entry("Test/Graph", "0.1.0", "graph", false)]); + pretty_assertions::assert_eq!(plugins[0].path, path); + pretty_assertions::assert_eq!(plugins[0].directory, directory.path()); + } + + #[test_case("graph.dsc.plugin.yaml"; "yaml extension")] + #[test_case("graph.dsc.plugin.yml"; "yml extension")] + fn finds_yaml_manifest(file_name: &str) { + let directory = directory(); + write_text(directory.path(), file_name, concat!( + "$schema: https://aka.ms/dsc/schemas/v3/bundled/plugin/manifest.json\n", + "type: Test/Graph\n", + "version: 0.1.0\n", + "command:\n", + " name: graph\n", + " executable: pwsh\n", + " args:\n", + " - -File\n", + " - manifestRelativePath: graph.ps1\n", + )); + + let plugins = discover(&[directory.path()]); + + pretty_assertions::assert_eq!(summarize(&plugins), vec![entry("Test/Graph", "0.1.0", "graph", false)]); + } + + #[test] + fn ignores_other_files_and_subdirectories() { + let directory = directory(); + write_json(directory.path(), "graph.json", &manifest_json("Test/Json", "0.1.0", "json")); + write_json(directory.path(), "graph.dsc.extension.json", &manifest_json("Test/Extension", "0.1.0", "extension")); + write_json(directory.path(), "graph.dsc.plugin.json.bak", &manifest_json("Test/Backup", "0.1.0", "backup")); + let subdirectory = directory.path().join("nested"); + fs::create_dir(&subdirectory).expect("subdirectory should be created"); + write_manifest(&subdirectory, "Test/Nested", "0.1.0", "nested"); + + assert!(discover(&[directory.path()]).is_empty()); + } + + #[test] + fn finds_plugins_in_manifest_list() { + let directory = directory(); + let path = write_json(directory.path(), "tools.dsc.manifests.json", &json!({ + "plugins": [ + manifest_json("Test/Graph", "0.1.0", "graph"), + manifest_json("Test/Explain", "0.2.0", "explain"), + ] + })); + + let plugins = discover(&[directory.path()]); + + pretty_assertions::assert_eq!( + summarize(&plugins), + vec![ + entry("Test/Explain", "0.2.0", "explain", false), + entry("Test/Graph", "0.1.0", "graph", false), + ] + ); + pretty_assertions::assert_eq!(plugins[0].path, path); + } + + #[test] + fn finds_plugins_in_yaml_manifest_list() { + let directory = directory(); + write_text(directory.path(), "tools.dsc.manifests.yaml", concat!( + "plugins:\n", + " - $schema: https://aka.ms/dsc/schemas/v3/bundled/plugin/manifest.json\n", + " type: Test/Graph\n", + " version: 0.1.0\n", + " command:\n", + " name: graph\n", + " executable: pwsh\n", + )); + + pretty_assertions::assert_eq!( + summarize(&discover(&[directory.path()])), + vec![entry("Test/Graph", "0.1.0", "graph", false)] + ); + } + + #[test] + fn manifest_list_with_invalid_resource_still_yields_plugins() { + let directory = directory(); + write_json(directory.path(), "tools.dsc.manifests.json", &json!({ + "resources": [{ "type": "not a resource manifest" }], + "plugins": [manifest_json("Test/Graph", "0.1.0", "graph")] + })); + + pretty_assertions::assert_eq!( + summarize(&discover(&[directory.path()])), + vec![entry("Test/Graph", "0.1.0", "graph", false)] + ); + } + + #[test_case(json!({ "resources": [] }); "list without plugins")] + #[test_case(json!({ "plugins": null }); "plugins defined as null")] + #[test_case(json!({ "plugins": {} }); "plugins defined as an object")] + #[test_case(json!({ "plugins": [] }); "plugins defined as an empty array")] + fn manifest_list_without_plugins_has_no_plugins(list: serde_json::Value) { + let directory = directory(); + write_json(directory.path(), "tools.dsc.manifests.json", &list); + + assert!(discover(&[directory.path()]).is_empty()); + } + + #[test] + fn skips_invalid_entry_in_manifest_list() { + let directory = directory(); + write_json(directory.path(), "tools.dsc.manifests.json", &json!({ + "plugins": [ + { "type": "Test/Broken" }, + manifest_json("Test/Graph", "0.1.0", "graph"), + ] + })); + + pretty_assertions::assert_eq!( + summarize(&discover(&[directory.path()])), + vec![entry("Test/Graph", "0.1.0", "graph", false)] + ); + } + + #[test] + fn skips_manifest_that_is_not_valid_json() { + let directory = directory(); + write_text(directory.path(), "broken.dsc.plugin.json", "{ not json"); + write_manifest(directory.path(), "Test/Graph", "0.1.0", "graph"); + + pretty_assertions::assert_eq!( + summarize(&discover(&[directory.path()])), + vec![entry("Test/Graph", "0.1.0", "graph", false)] + ); + } + + #[test] + fn skips_manifest_with_invalid_command_name() { + let directory = directory(); + write_json(directory.path(), "broken.dsc.plugin.json", &manifest_json("Test/Broken", "0.1.0", "Not-Valid")); + + assert!(discover(&[directory.path()]).is_empty()); + } + + #[test] + fn skips_manifest_with_unknown_property() { + let directory = directory(); + let mut manifest = manifest_json("Test/Broken", "0.1.0", "broken"); + manifest["discover"] = json!({ "executable": "pwsh" }); + write_json(directory.path(), "broken.dsc.plugin.json", &manifest); + + assert!(discover(&[directory.path()]).is_empty()); + } + + #[test] + fn skips_manifest_with_unrecognized_schema_uri() { + let directory = directory(); + let mut manifest = manifest_json("Test/Broken", "0.1.0", "broken"); + manifest["$schema"] = json!("https://example.com/not-a-dsc-schema.json"); + write_json(directory.path(), "broken.dsc.plugin.json", &manifest); + + assert!(discover(&[directory.path()]).is_empty()); + } + + #[test] + fn skips_manifest_with_path_that_is_not_relative() { + let directory = directory(); + let mut manifest = manifest_json("Test/Broken", "0.1.0", "broken"); + manifest["command"]["args"] = json!([{ "manifestRelativePath": "/tools/broken.ps1" }]); + write_json(directory.path(), "broken.dsc.plugin.json", &manifest); + + assert!(discover(&[directory.path()]).is_empty()); + } + + #[test] + fn keeps_manifest_whose_condition_is_true() { + let directory = directory(); + write_json( + directory.path(), + "graph.dsc.plugin.json", + &manifest_with_condition("Test/Graph", "graph", "[equals(1, 1)]") + ); + + pretty_assertions::assert_eq!( + summarize(&discover(&[directory.path()])), + vec![entry("Test/Graph", "0.1.0", "graph", false)] + ); + } + + #[test] + fn skips_manifest_whose_condition_is_false() { + let directory = directory(); + write_json( + directory.path(), + "graph.dsc.plugin.json", + &manifest_with_condition("Test/Graph", "graph", "[equals(1, 2)]") + ); + + assert!(discover(&[directory.path()]).is_empty()); + } + + #[test] + fn skips_manifest_whose_condition_is_not_a_boolean() { + let directory = directory(); + write_json( + directory.path(), + "graph.dsc.plugin.json", + &manifest_with_condition("Test/Graph", "graph", "[concat('a', 'b')]") + ); + + assert!(discover(&[directory.path()]).is_empty()); + } + + #[test_case("config"; "name of a built-in command")] + #[test_case("help"; "help command")] + fn skips_manifest_with_reserved_command(command: &str) { + let directory = directory(); + write_manifest(directory.path(), "Test/Reserved", "0.1.0", command); + write_manifest(directory.path(), "Test/Graph", "0.1.0", "graph"); + + pretty_assertions::assert_eq!( + summarize(&discover(&[directory.path()])), + vec![entry("Test/Graph", "0.1.0", "graph", false)] + ); + } + + #[test] + fn reserved_command_is_compared_case_sensitively() { + let directory = directory(); + write_manifest(directory.path(), "Test/Graph", "0.1.0", "graph"); + + let plugins = dsc_lib::plugins::discovery::discover_plugins_in( + &[directory.path().to_path_buf()], + &["GRAPH"] + ); + + pretty_assertions::assert_eq!(summarize(&plugins), vec![entry("Test/Graph", "0.1.0", "graph", false)]); + } + + #[test] + fn keeps_highest_version_of_a_type_found_in_later_directory() { + let first = directory(); + let second = directory(); + write_manifest(first.path(), "Test/Graph", "1.0.0", "graph"); + let newer = write_manifest(second.path(), "Test/Graph", "1.1.0", "graph"); + + let plugins = discover(&[first.path(), second.path()]); + + pretty_assertions::assert_eq!(summarize(&plugins), vec![entry("Test/Graph", "1.1.0", "graph", false)]); + pretty_assertions::assert_eq!(plugins[0].path, newer); + } + + #[test] + fn keeps_highest_version_of_a_type_found_in_earlier_directory() { + let first = directory(); + let second = directory(); + let newer = write_manifest(first.path(), "Test/Graph", "1.1.0", "graph"); + write_manifest(second.path(), "Test/Graph", "1.0.0", "graph"); + + let plugins = discover(&[first.path(), second.path()]); + + pretty_assertions::assert_eq!(summarize(&plugins), vec![entry("Test/Graph", "1.1.0", "graph", false)]); + pretty_assertions::assert_eq!(plugins[0].path, newer); + } + + #[test] + fn keeps_highest_version_of_a_type_within_one_directory() { + let directory = directory(); + write_json(directory.path(), "a.dsc.plugin.json", &manifest_json("Test/Graph", "1.0.0", "graph")); + write_json(directory.path(), "b.dsc.plugin.json", &manifest_json("Test/Graph", "2.0.0", "graph")); + write_json(directory.path(), "c.dsc.plugin.json", &manifest_json("Test/Graph", "1.5.0", "graph")); + + pretty_assertions::assert_eq!( + summarize(&discover(&[directory.path()])), + vec![entry("Test/Graph", "2.0.0", "graph", false)] + ); + } + + #[test] + fn type_names_are_compared_case_insensitively() { + let first = directory(); + let second = directory(); + write_manifest(first.path(), "Test/Graph", "1.0.0", "graph"); + write_manifest(second.path(), "test/graph", "1.1.0", "graph"); + + let plugins = discover(&[first.path(), second.path()]); + + pretty_assertions::assert_eq!(plugins.len(), 1); + pretty_assertions::assert_eq!(plugins[0].version.to_string(), "1.1.0"); + } + + #[test] + fn first_directory_provides_a_command_claimed_by_different_types() { + let first = directory(); + let second = directory(); + write_manifest(first.path(), "Zeta/Graph", "0.1.0", "graph"); + write_manifest(second.path(), "Alpha/Graph", "9.0.0", "graph"); + + pretty_assertions::assert_eq!( + summarize(&discover(&[first.path(), second.path()])), + vec![ + entry("Zeta/Graph", "0.1.0", "graph", false), + entry("Alpha/Graph", "9.0.0", "graph", true), + ] + ); + } + + #[test] + fn type_that_sorts_first_provides_a_command_within_one_directory() { + let directory = directory(); + write_json(directory.path(), "a.dsc.plugin.json", &manifest_json("Zeta/Graph", "0.1.0", "graph")); + write_json(directory.path(), "b.dsc.plugin.json", &manifest_json("Alpha/Graph", "0.1.0", "graph")); + + pretty_assertions::assert_eq!( + summarize(&discover(&[directory.path()])), + vec![ + entry("Alpha/Graph", "0.1.0", "graph", false), + entry("Zeta/Graph", "0.1.0", "graph", true), + ] + ); + } + + #[test] + fn returns_plugins_in_search_order() { + let first = directory(); + let second = directory(); + write_manifest(first.path(), "Test/Lint", "0.1.0", "lint"); + write_manifest(first.path(), "Test/Graph", "0.1.0", "graph"); + write_manifest(second.path(), "Test/Explain", "0.1.0", "explain"); + + pretty_assertions::assert_eq!( + summarize(&discover(&[first.path(), second.path()])), + vec![ + entry("Test/Graph", "0.1.0", "graph", false), + entry("Test/Lint", "0.1.0", "lint", false), + entry("Test/Explain", "0.1.0", "explain", false), + ] + ); + } +} + +#[cfg(test)] +mod find_plugin_by_type { + use dsc_lib::plugins::discovery::find_plugin_by_type; + use test_case::test_case; + + use super::helpers::*; + + #[test_case("Alpha/Graph"; "type name as defined")] + #[test_case("alpha/GRAPH"; "type name in another casing")] + fn returns_a_shadowed_plugin(type_name: &str) { + let first = directory(); + let second = directory(); + write_manifest(first.path(), "Zeta/Graph", "0.1.0", "graph"); + write_manifest(second.path(), "Alpha/Graph", "0.1.0", "graph"); + let plugins = discover(&[first.path(), second.path()]); + + let plugin = find_plugin_by_type(&plugins, type_name).expect("plug-in should be found"); + + pretty_assertions::assert_eq!(plugin.type_name.to_string(), "Alpha/Graph"); + assert!(plugin.shadowed); + } + + #[test] + fn returns_the_plugin_that_provides_its_command() { + let directory = directory(); + write_manifest(directory.path(), "Test/Graph", "0.1.0", "graph"); + let plugins = discover(&[directory.path()]); + + let plugin = find_plugin_by_type(&plugins, "Test/Graph").expect("plug-in should be found"); + + assert!(!plugin.shadowed); + } + + #[test_case("Test/Lint"; "type that is not installed")] + #[test_case("graph"; "command name")] + fn returns_none_for_unknown_type(type_name: &str) { + let directory = directory(); + write_manifest(directory.path(), "Test/Graph", "0.1.0", "graph"); + let plugins = discover(&[directory.path()]); + + assert!(find_plugin_by_type(&plugins, type_name).is_none()); + } +} + +#[cfg(test)] +mod find_plugin { + use dsc_lib::plugins::discovery::find_plugin; + + use super::helpers::*; + + #[test] + fn returns_the_plugin_that_provides_the_command() { + let directory = directory(); + write_manifest(directory.path(), "Test/Graph", "0.1.0", "graph"); + write_manifest(directory.path(), "Test/Lint", "0.1.0", "lint"); + let plugins = discover(&[directory.path()]); + + let plugin = find_plugin(&plugins, "lint").expect("plug-in should be found"); + + pretty_assertions::assert_eq!(plugin.type_name.to_string(), "Test/Lint"); + } + + #[test] + fn never_returns_a_shadowed_plugin() { + let first = directory(); + let second = directory(); + write_manifest(first.path(), "Zeta/Graph", "0.1.0", "graph"); + write_manifest(second.path(), "Alpha/Graph", "0.1.0", "graph"); + let plugins = discover(&[first.path(), second.path()]); + + let plugin = find_plugin(&plugins, "graph").expect("plug-in should be found"); + + pretty_assertions::assert_eq!(plugin.type_name.to_string(), "Zeta/Graph"); + assert!(!plugin.shadowed); + } + + #[test] + fn returns_none_for_unknown_command() { + let directory = directory(); + write_manifest(directory.path(), "Test/Graph", "0.1.0", "graph"); + let plugins = discover(&[directory.path()]); + + assert!(find_plugin(&plugins, "lint").is_none()); + } + + #[test] + fn compares_the_command_case_sensitively() { + let directory = directory(); + write_manifest(directory.path(), "Test/Graph", "0.1.0", "graph"); + let plugins = discover(&[directory.path()]); + + assert!(find_plugin(&plugins, "Graph").is_none()); + } +} diff --git a/lib/dsc-lib/tests/integration/plugins/dscplugin.rs b/lib/dsc-lib/tests/integration/plugins/dscplugin.rs new file mode 100644 index 000000000..394b29790 --- /dev/null +++ b/lib/dsc-lib/tests/integration/plugins/dscplugin.rs @@ -0,0 +1,206 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +#[cfg(test)] +mod methods { + #[cfg(test)] + mod from_manifest { + use std::path::Path; + + use dsc_lib::plugins::dscplugin::DscPlugin; + use dsc_lib::plugins::plugin_manifest::PluginManifest; + use serde_json::json; + + use super::super::super::helpers::{manifest, manifest_json}; + + #[test] + fn copies_the_identity_of_the_manifest() { + let manifest = manifest("Test/Graph", "1.2.3", "graph"); + let path = Path::new("plugins").join("graph.dsc.plugin.json"); + + let plugin = DscPlugin::from_manifest(&path, manifest.clone()); + + pretty_assertions::assert_eq!(plugin.type_name.to_string(), "Test/Graph"); + pretty_assertions::assert_eq!(plugin.version.to_string(), "1.2.3"); + pretty_assertions::assert_eq!(plugin.command.to_string(), "graph"); + pretty_assertions::assert_eq!(plugin.description, None); + pretty_assertions::assert_eq!(plugin.deprecation_message, None); + pretty_assertions::assert_eq!(plugin.path, path); + pretty_assertions::assert_eq!(plugin.directory, Path::new("plugins")); + pretty_assertions::assert_eq!(plugin.manifest, manifest); + } + + #[test] + fn copies_the_description_and_deprecation_message() { + let mut json = manifest_json("Test/Graph", "1.2.3", "graph"); + json["description"] = json!("Show the dependency graph of a configuration document"); + json["deprecationMessage"] = json!("Use 'dsc diagram' instead."); + let manifest: PluginManifest = serde_json::from_value(json).expect("manifest should deserialize"); + + let plugin = DscPlugin::from_manifest(Path::new("graph.dsc.plugin.json"), manifest); + + pretty_assertions::assert_eq!( + plugin.description.as_deref(), + Some("Show the dependency graph of a configuration document") + ); + pretty_assertions::assert_eq!(plugin.deprecation_message.as_deref(), Some("Use 'dsc diagram' instead.")); + } + + #[test] + fn is_not_shadowed() { + let plugin = DscPlugin::from_manifest( + Path::new("graph.dsc.plugin.json"), + manifest("Test/Graph", "1.2.3", "graph") + ); + + assert!(!plugin.shadowed); + } + } +} + +#[cfg(test)] +mod serde { + use std::path::Path; + + use dsc_lib::plugins::dscplugin::DscPlugin; + use serde_json::json; + + use super::super::helpers::{manifest, manifest_json}; + + #[test] + fn serializing_uses_the_documented_property_names() { + let path = Path::new("plugins").join("graph.dsc.plugin.json"); + let plugin = DscPlugin::from_manifest(&path, manifest("Test/Graph", "1.2.3", "graph")); + + pretty_assertions::assert_eq!( + serde_json::to_value(&plugin).expect("serialization should never fail"), + json!({ + "type": "Test/Graph", + "version": "1.2.3", + "command": "graph", + "description": null, + "deprecationMessage": null, + "path": path, + "directory": Path::new("plugins"), + "shadowed": false, + "manifest": manifest_json("Test/Graph", "1.2.3", "graph") + }) + ); + } + + #[test] + fn round_trips() { + let plugin = DscPlugin::from_manifest( + &Path::new("plugins").join("graph.dsc.plugin.json"), + manifest("Test/Graph", "1.2.3", "graph") + ); + + let json = serde_json::to_string(&plugin).expect("serialization should never fail"); + let actual: DscPlugin = serde_json::from_str(&json).expect("plug-in should deserialize"); + + pretty_assertions::assert_eq!(actual, plugin); + } +} + +crate::macros::test_dsc_repo_schema! { + define_statics: { + dsc_lib::plugins::dscplugin::DscPlugin, + properties: { + TYPE_SCHEMA => "type", + VERSION_SCHEMA => "version", + COMMAND_SCHEMA => "command", + DESCRIPTION_SCHEMA => "description", + DEPRECATION_MESSAGE_SCHEMA => "deprecationMessage", + PATH_SCHEMA => "path", + DIRECTORY_SCHEMA => "directory", + SHADOWED_SCHEMA => "shadowed", + } + }, + test_meta_schema: {}, + test_docs: { + title => "title", + description => "description", + markdown_description => "markdownDescription", + type_title => (TYPE_SCHEMA, "title"), + type_description => (TYPE_SCHEMA, "description"), + type_markdown_description => (TYPE_SCHEMA, "markdownDescription"), + version_title => (VERSION_SCHEMA, "title"), + version_description => (VERSION_SCHEMA, "description"), + version_markdown_description => (VERSION_SCHEMA, "markdownDescription"), + command_title => (COMMAND_SCHEMA, "title"), + command_description => (COMMAND_SCHEMA, "description"), + command_markdown_description => (COMMAND_SCHEMA, "markdownDescription"), + description_title => (DESCRIPTION_SCHEMA, "title"), + description_description => (DESCRIPTION_SCHEMA, "description"), + description_markdown_description => (DESCRIPTION_SCHEMA, "markdownDescription"), + deprecation_message_title => (DEPRECATION_MESSAGE_SCHEMA, "title"), + deprecation_message_description => (DEPRECATION_MESSAGE_SCHEMA, "description"), + deprecation_message_markdown_description => (DEPRECATION_MESSAGE_SCHEMA, "markdownDescription"), + path_title => (PATH_SCHEMA, "title"), + path_description => (PATH_SCHEMA, "description"), + path_markdown_description => (PATH_SCHEMA, "markdownDescription"), + directory_title => (DIRECTORY_SCHEMA, "title"), + directory_description => (DIRECTORY_SCHEMA, "description"), + directory_markdown_description => (DIRECTORY_SCHEMA, "markdownDescription"), + shadowed_title => (SHADOWED_SCHEMA, "title"), + shadowed_description => (SHADOWED_SCHEMA, "description"), + shadowed_markdown_description => (SHADOWED_SCHEMA, "markdownDescription") + }, + test_validation: { + plugin_with_every_property_is_valid => { + input_json: json!({ + "type": "Test/Graph", + "version": "1.2.3", + "command": "graph", + "description": "Show the dependency graph of a configuration document", + "deprecationMessage": null, + "path": "/opt/plugins/graph.dsc.plugin.json", + "directory": "/opt/plugins", + "shadowed": false, + "manifest": { + "$schema": "https://aka.ms/dsc/schemas/v3/bundled/plugin/manifest.json", + "type": "Test/Graph", + "version": "1.2.3", + "command": { "name": "graph", "executable": "pwsh" } + } + }), + expected_valid: true, + }, + plugin_without_manifest_is_invalid => { + input_json: json!({ + "type": "Test/Graph", + "version": "1.2.3", + "command": "graph", + "description": null, + "deprecationMessage": null, + "path": "/opt/plugins/graph.dsc.plugin.json", + "directory": "/opt/plugins", + "shadowed": false + }), + expected_valid: false, + }, + plugin_with_invalid_command_is_invalid => { + input_json: json!({ + "type": "Test/Graph", + "version": "1.2.3", + "command": "Graph", + "description": null, + "deprecationMessage": null, + "path": "/opt/plugins/graph.dsc.plugin.json", + "directory": "/opt/plugins", + "shadowed": false, + "manifest": { + "$schema": "https://aka.ms/dsc/schemas/v3/bundled/plugin/manifest.json", + "type": "Test/Graph", + "version": "1.2.3", + "command": { "name": "graph", "executable": "pwsh" } + } + }), + expected_valid: false, + }, + empty_object_is_invalid => { + input_json: json!({}), + expected_valid: false, + } + } +} diff --git a/lib/dsc-lib/tests/integration/plugins/mod.rs b/lib/dsc-lib/tests/integration/plugins/mod.rs new file mode 100644 index 000000000..07ddfed71 --- /dev/null +++ b/lib/dsc-lib/tests/integration/plugins/mod.rs @@ -0,0 +1,42 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +//! Defines integration tests for the public types in the `plugins` module of [`dsc-lib`]. + +#[cfg(test)] +mod command_name; +#[cfg(test)] +mod discovery; +#[cfg(test)] +mod dscplugin; +#[cfg(test)] +mod plugin_manifest; + +pub(crate) mod helpers { + //! Defines helper functions for creating plug-in manifests. + + use dsc_lib::plugins::plugin_manifest::PluginManifest; + use serde_json::{json, Value}; + + /// The default URI for the `$schema` property of a plug-in manifest. + pub(crate) const MANIFEST_SCHEMA_URI: &str = "https://aka.ms/dsc/schemas/v3/bundled/plugin/manifest.json"; + + /// Returns the JSON for a manifest with only the required properties. + pub(crate) fn manifest_json(type_name: &str, version: &str, command: &str) -> Value { + json!({ + "$schema": MANIFEST_SCHEMA_URI, + "type": type_name, + "version": version, + "command": { + "name": command, + "executable": "plugin-executable" + } + }) + } + + /// Returns a manifest with only the required properties. + pub(crate) fn manifest(type_name: &str, version: &str, command: &str) -> PluginManifest { + serde_json::from_value(manifest_json(type_name, version, command)) + .expect("manifest should be valid") + } +} diff --git a/lib/dsc-lib/tests/integration/plugins/plugin_manifest.rs b/lib/dsc-lib/tests/integration/plugins/plugin_manifest.rs new file mode 100644 index 000000000..3835976c1 --- /dev/null +++ b/lib/dsc-lib/tests/integration/plugins/plugin_manifest.rs @@ -0,0 +1,466 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +#[cfg(test)] +mod methods { + #[cfg(test)] + mod validate { + use dsc_lib::dscerror::DscError; + use dsc_lib::plugins::plugin_manifest::PluginManifest; + use dsc_lib::schemas::dsc_repo::DscRepoSchema; + use serde_json::json; + use test_case::test_case; + + use super::super::super::helpers::{manifest_json, MANIFEST_SCHEMA_URI}; + + /// Returns a manifest whose command passes a single `manifestRelativePath` argument. + fn manifest_with_path(path: &str) -> PluginManifest { + let mut manifest = manifest_json("Test/Graph", "0.1.0", "graph"); + manifest["command"]["args"] = json!(["-File", { "manifestRelativePath": path }]); + serde_json::from_value(manifest).expect("manifest should deserialize") + } + + #[test] + fn default_schema_uri_is_the_documented_uri() { + pretty_assertions::assert_eq!(PluginManifest::default_schema_id_uri(), MANIFEST_SCHEMA_URI); + } + + #[test] + fn manifest_with_required_properties_is_valid() { + let manifest: PluginManifest = serde_json::from_value(manifest_json("Test/Graph", "0.1.0", "graph")) + .expect("manifest should deserialize"); + + assert!(manifest.validate().is_ok()); + } + + #[test] + fn unrecognized_schema_uri_is_invalid() { + let mut manifest = manifest_json("Test/Graph", "0.1.0", "graph"); + manifest["$schema"] = json!("https://example.com/not-a-dsc-schema.json"); + let manifest: PluginManifest = serde_json::from_value(manifest).expect("manifest should deserialize"); + + assert!(matches!(manifest.validate(), Err(DscError::UnrecognizedSchemaUri(_)))); + } + + #[test_case("graph.ps1"; "file next to the manifest")] + #[test_case("scripts/graph.py"; "file in a subdirectory")] + #[test_case("../shared/graph.ps1"; "file in a sibling directory")] + fn relative_path_is_valid(path: &str) { + assert!(manifest_with_path(path).validate().is_ok()); + } + + #[test_case(""; "empty path")] + #[test_case("/tools/graph.ps1"; "path with a root")] + #[cfg_attr(windows, test_case(r"C:\tools\graph.ps1"; "path with a drive and a root"))] + #[cfg_attr(windows, test_case("C:graph.ps1"; "path with a drive and no root"))] + #[cfg_attr(windows, test_case(r"\\server\share\graph.ps1"; "UNC path"))] + fn path_that_is_not_relative_is_invalid(path: &str) { + let err = manifest_with_path(path).validate().unwrap_err(); + + assert!(matches!(err, DscError::InvalidManifest(_)), "unexpected error: {err:?}"); + assert!(err.to_string().contains("Test/Graph"), "error should name the plug-in: {err}"); + } + } + + #[cfg(test)] + mod to_command_args { + use std::ffi::OsString; + use std::path::Path; + + use dsc_lib::plugins::plugin_manifest::PluginManifest; + use serde_json::json; + + use super::super::super::helpers::manifest_json; + + #[test] + fn without_args_returns_no_arguments() { + let manifest: PluginManifest = serde_json::from_value(manifest_json("Test/Graph", "0.1.0", "graph")) + .expect("manifest should deserialize"); + + assert!(manifest.command.to_command_args(Path::new("plugins")).is_empty()); + } + + #[test] + fn passes_strings_as_written_and_resolves_manifest_relative_paths() { + let mut manifest = manifest_json("Test/Graph", "0.1.0", "graph"); + manifest["command"]["args"] = json!([ + "-NoProfile", + "-File", + { "manifestRelativePath": "graph.ps1" }, + "./not-resolved.txt" + ]); + let manifest: PluginManifest = serde_json::from_value(manifest).expect("manifest should deserialize"); + let directory = Path::new("plugins").join("graph"); + + pretty_assertions::assert_eq!( + manifest.command.to_command_args(&directory), + vec![ + OsString::from("-NoProfile"), + OsString::from("-File"), + directory.join("graph.ps1").into_os_string(), + OsString::from("./not-resolved.txt"), + ] + ); + } + } +} + +#[cfg(test)] +mod serde { + use std::path::PathBuf; + + use dsc_lib::plugins::plugin_manifest::{PluginArgKind, PluginManifest}; + use serde_json::{json, Value}; + use test_case::test_case; + + use super::super::helpers::manifest_json; + + /// Returns the minimal manifest with one property replaced or removed. + fn manifest_with(property: &str, value: Option) -> Value { + let mut manifest = manifest_json("Test/Graph", "0.1.0", "graph"); + let object = manifest.as_object_mut().expect("manifest is an object"); + match value { + Some(value) => { object.insert(property.to_string(), value); }, + None => { object.remove(property); }, + } + manifest + } + + #[test] + fn deserializing_minimal_manifest() { + let manifest: PluginManifest = serde_json::from_value(manifest_json("Test/Graph", "0.1.0", "graph")) + .expect("manifest should deserialize"); + + pretty_assertions::assert_eq!(manifest.r#type.to_string(), "Test/Graph"); + pretty_assertions::assert_eq!(manifest.version.to_string(), "0.1.0"); + pretty_assertions::assert_eq!(manifest.command.name.to_string(), "graph"); + pretty_assertions::assert_eq!(manifest.command.executable, "plugin-executable"); + assert!(manifest.command.args.is_none()); + assert!(manifest.condition.is_none()); + assert!(manifest.deprecation_message.is_none()); + assert!(manifest.description.is_none()); + assert!(manifest.tags.is_empty()); + assert!(manifest.metadata.is_none()); + } + + #[test] + fn deserializing_manifest_with_every_property() { + let mut manifest = manifest_json("Test/Graph", "0.1.0", "graph"); + manifest["condition"] = json!("[equals(1, 1)]"); + manifest["deprecationMessage"] = json!("Use 'dsc diagram' instead."); + manifest["description"] = json!("Show the dependency graph of a configuration document"); + manifest["tags"] = json!(["graph", "authoring"]); + manifest["metadata"] = json!({ "owner": "tests" }); + manifest["command"]["args"] = json!(["-File", { "manifestRelativePath": "graph.ps1" }]); + + let manifest: PluginManifest = serde_json::from_value(manifest).expect("manifest should deserialize"); + + pretty_assertions::assert_eq!(manifest.condition.as_deref(), Some("[equals(1, 1)]")); + pretty_assertions::assert_eq!(manifest.deprecation_message.as_deref(), Some("Use 'dsc diagram' instead.")); + pretty_assertions::assert_eq!( + manifest.description.as_deref(), + Some("Show the dependency graph of a configuration document") + ); + pretty_assertions::assert_eq!(manifest.tags.len(), 2); + pretty_assertions::assert_eq!(manifest.metadata.expect("metadata is defined")["owner"], json!("tests")); + pretty_assertions::assert_eq!( + manifest.command.args, + Some(vec![ + PluginArgKind::String("-File".to_string()), + PluginArgKind::ManifestRelativePath { manifest_relative_path: PathBuf::from("graph.ps1") }, + ]) + ); + } + + #[test] + fn deserializing_yaml_manifest() { + let yaml = r#" + $schema: https://aka.ms/dsc/schemas/v3/bundled/plugin/manifest.json + type: Test/Graph + version: 0.1.0 + command: + name: graph + executable: pwsh + args: + - -File + - manifestRelativePath: graph.ps1 + "#; + + let manifest: PluginManifest = serde_yaml::from_str(yaml).expect("manifest should deserialize"); + + pretty_assertions::assert_eq!(manifest.command.name.to_string(), "graph"); + pretty_assertions::assert_eq!(manifest.command.args.map(|args| args.len()), Some(2)); + } + + #[test_case("$schema"; "schema")] + #[test_case("type"; "type")] + #[test_case("version"; "version")] + #[test_case("command"; "command")] + fn deserializing_without_required_property_fails(property: &str) { + let manifest = manifest_with(property, None); + + assert!(serde_json::from_value::(manifest).is_err()); + } + + #[test_case("type", json!("NotATypeName"); "type that is not a fully qualified type name")] + #[test_case("version", json!("1.0"); "version that is not a semantic version")] + #[test_case("command", json!("graph"); "command that is not an object")] + #[test_case("command", json!({ "name": "graph" }); "command without an executable")] + #[test_case("command", json!({ "executable": "pwsh" }); "command without a name")] + #[test_case("command", json!({ "name": "Graph", "executable": "pwsh" }); "command with an invalid name")] + #[test_case( + "command", + json!({ "name": "graph", "executable": "pwsh", "input": "stdin" }); + "command with an unknown property" + )] + #[test_case( + "command", + json!({ "name": "graph", "executable": "pwsh", "args": [1] }); + "argument that is a number" + )] + #[test_case( + "command", + json!({ "name": "graph", "executable": "pwsh", "args": [{ "jsonInputArg": "--input" }] }); + "argument kind of another manifest" + )] + #[test_case( + "command", + json!({ "name": "graph", "executable": "pwsh", "args": [{ "manifestRelativePath": "a", "other": "b" }] }); + "argument object with an extra property" + )] + #[test_case("discover", json!({ "executable": "pwsh" }); "property of an extension manifest")] + fn deserializing_invalid_property_fails(property: &str, value: Value) { + let manifest = manifest_with(property, Some(value)); + + assert!(serde_json::from_value::(manifest).is_err()); + } + + #[test] + fn serializing_omits_undefined_optional_properties() { + let json = manifest_json("Test/Graph", "0.1.0", "graph"); + let manifest: PluginManifest = serde_json::from_value(json.clone()).expect("manifest should deserialize"); + + pretty_assertions::assert_eq!( + serde_json::to_value(&manifest).expect("serialization should never fail"), + json + ); + } + + #[test] + fn serializing_round_trips_arguments() { + let mut json = manifest_json("Test/Graph", "0.1.0", "graph"); + json["command"]["args"] = json!(["-File", { "manifestRelativePath": "graph.ps1" }]); + let manifest: PluginManifest = serde_json::from_value(json.clone()).expect("manifest should deserialize"); + + pretty_assertions::assert_eq!( + serde_json::to_value(&manifest).expect("serialization should never fail"), + json + ); + } +} + +#[cfg(test)] +mod manifest { + crate::macros::test_dsc_repo_schema! { + define_statics: { + dsc_lib::plugins::plugin_manifest::PluginManifest, + properties: { + TYPE_SCHEMA => "type", + VERSION_SCHEMA => "version", + CONDITION_SCHEMA => "condition", + DEPRECATION_MESSAGE_SCHEMA => "deprecationMessage", + DESCRIPTION_SCHEMA => "description", + TAGS_SCHEMA => "tags", + METADATA_SCHEMA => "metadata", + } + }, + test_meta_schema: {}, + test_docs: { + title => "title", + description => "description", + markdown_description => "markdownDescription", + type_title => (TYPE_SCHEMA, "title"), + type_description => (TYPE_SCHEMA, "description"), + type_markdown_description => (TYPE_SCHEMA, "markdownDescription"), + version_title => (VERSION_SCHEMA, "title"), + version_description => (VERSION_SCHEMA, "description"), + version_markdown_description => (VERSION_SCHEMA, "markdownDescription"), + condition_title => (CONDITION_SCHEMA, "title"), + condition_description => (CONDITION_SCHEMA, "description"), + condition_markdown_description => (CONDITION_SCHEMA, "markdownDescription"), + deprecation_message_title => (DEPRECATION_MESSAGE_SCHEMA, "title"), + deprecation_message_description => (DEPRECATION_MESSAGE_SCHEMA, "description"), + deprecation_message_markdown_description => (DEPRECATION_MESSAGE_SCHEMA, "markdownDescription"), + description_title => (DESCRIPTION_SCHEMA, "title"), + description_description => (DESCRIPTION_SCHEMA, "description"), + description_markdown_description => (DESCRIPTION_SCHEMA, "markdownDescription"), + tags_title => (TAGS_SCHEMA, "title"), + tags_description => (TAGS_SCHEMA, "description"), + tags_markdown_description => (TAGS_SCHEMA, "markdownDescription"), + metadata_title => (METADATA_SCHEMA, "title"), + metadata_description => (METADATA_SCHEMA, "description"), + metadata_markdown_description => (METADATA_SCHEMA, "markdownDescription") + }, + test_validation: { + manifest_with_required_properties_is_valid => { + input_json: json!({ + "$schema": "https://aka.ms/dsc/schemas/v3/bundled/plugin/manifest.json", + "type": "Test/Graph", + "version": "0.1.0", + "command": { "name": "graph", "executable": "pwsh" } + }), + expected_valid: true, + }, + manifest_with_every_property_is_valid => { + input_json: json!({ + "$schema": "https://aka.ms/dsc/schemas/v3/bundled/plugin/manifest.json", + "type": "Test/Graph", + "version": "0.1.0", + "condition": "[not(equals(tryWhich('pwsh'), null()))]", + "deprecationMessage": "Use 'dsc diagram' instead.", + "description": "Show the dependency graph of a configuration document", + "tags": ["graph", "authoring"], + "metadata": { "owner": "tests" }, + "command": { + "name": "graph", + "executable": "pwsh", + "args": ["-NoProfile", "-File", { "manifestRelativePath": "graph.ps1" }] + } + }), + expected_valid: true, + }, + manifest_without_schema_is_invalid => { + input_json: json!({ + "type": "Test/Graph", + "version": "0.1.0", + "command": { "name": "graph", "executable": "pwsh" } + }), + expected_valid: false, + }, + manifest_with_unrecognized_schema_is_invalid => { + input_json: json!({ + "$schema": "https://example.com/not-a-dsc-schema.json", + "type": "Test/Graph", + "version": "0.1.0", + "command": { "name": "graph", "executable": "pwsh" } + }), + expected_valid: false, + }, + manifest_without_command_is_invalid => { + input_json: json!({ + "$schema": "https://aka.ms/dsc/schemas/v3/bundled/plugin/manifest.json", + "type": "Test/Graph", + "version": "0.1.0" + }), + expected_valid: false, + }, + manifest_with_invalid_command_name_is_invalid => { + input_json: json!({ + "$schema": "https://aka.ms/dsc/schemas/v3/bundled/plugin/manifest.json", + "type": "Test/Graph", + "version": "0.1.0", + "command": { "name": "Graph", "executable": "pwsh" } + }), + expected_valid: false, + }, + manifest_with_unknown_property_is_invalid => { + input_json: json!({ + "$schema": "https://aka.ms/dsc/schemas/v3/bundled/plugin/manifest.json", + "type": "Test/Graph", + "version": "0.1.0", + "command": { "name": "graph", "executable": "pwsh" }, + "discover": { "executable": "pwsh" } + }), + expected_valid: false, + }, + manifest_with_unknown_argument_kind_is_invalid => { + input_json: json!({ + "$schema": "https://aka.ms/dsc/schemas/v3/bundled/plugin/manifest.json", + "type": "Test/Graph", + "version": "0.1.0", + "command": { + "name": "graph", + "executable": "pwsh", + "args": [{ "jsonInputArg": "--input" }] + } + }), + expected_valid: false, + }, + string_is_invalid => { + input_json: json!("Test/Graph"), + expected_valid: false, + } + } + } +} + +#[cfg(test)] +mod command { + crate::macros::test_dsc_repo_schema! { + define_statics: { + dsc_lib::plugins::plugin_manifest::PluginCommand, + properties: { + NAME_SCHEMA => "name", + EXECUTABLE_SCHEMA => "executable", + ARGS_SCHEMA => "args", + } + }, + test_meta_schema: {}, + test_docs: { + title => "title", + description => "description", + markdown_description => "markdownDescription", + name_title => (NAME_SCHEMA, "title"), + name_description => (NAME_SCHEMA, "description"), + name_markdown_description => (NAME_SCHEMA, "markdownDescription"), + executable_title => (EXECUTABLE_SCHEMA, "title"), + executable_description => (EXECUTABLE_SCHEMA, "description"), + executable_markdown_description => (EXECUTABLE_SCHEMA, "markdownDescription"), + args_title => (ARGS_SCHEMA, "title"), + args_description => (ARGS_SCHEMA, "description"), + args_markdown_description => (ARGS_SCHEMA, "markdownDescription") + }, + test_validation: { + command_with_name_and_executable_is_valid => { + input_json: json!({ "name": "graph", "executable": "pwsh" }), + expected_valid: true, + }, + command_with_string_arguments_is_valid => { + input_json: json!({ "name": "graph", "executable": "pwsh", "args": ["-NoProfile", "-File"] }), + expected_valid: true, + }, + command_with_manifest_relative_path_argument_is_valid => { + input_json: json!({ + "name": "graph", + "executable": "pwsh", + "args": ["-File", { "manifestRelativePath": "graph.ps1" }] + }), + expected_valid: true, + }, + command_with_empty_arguments_is_valid => { + input_json: json!({ "name": "graph", "executable": "pwsh", "args": [] }), + expected_valid: true, + }, + command_without_name_is_invalid => { + input_json: json!({ "executable": "pwsh" }), + expected_valid: false, + }, + command_without_executable_is_invalid => { + input_json: json!({ "name": "graph" }), + expected_valid: false, + }, + command_with_numeric_argument_is_invalid => { + input_json: json!({ "name": "graph", "executable": "pwsh", "args": [1] }), + expected_valid: false, + }, + command_with_unknown_property_is_invalid => { + input_json: json!({ "name": "graph", "executable": "pwsh", "input": "stdin" }), + expected_valid: false, + }, + empty_object_is_invalid => { + input_json: json!({}), + expected_valid: false, + } + } + } +} diff --git a/lib/dsc-lib/tests/integration/schemas/schema_for.rs b/lib/dsc-lib/tests/integration/schemas/schema_for.rs index 19a99a8a6..58e083297 100644 --- a/lib/dsc-lib/tests/integration/schemas/schema_for.rs +++ b/lib/dsc-lib/tests/integration/schemas/schema_for.rs @@ -131,6 +131,17 @@ macro_rules! test_schema_for { } } + #[cfg(test)] mod lint { + #[allow(unused_must_use)] + #[cfg(test)] mod diagnostic { + test_schema_for!(dsc_lib::lint::diagnostic::LintResult); + } + #[allow(unused_must_use)] + #[cfg(test)] mod rule { + test_schema_for!(dsc_lib::lint::rule::LintRuleInfo); + } + } + #[cfg(test)] mod extensions { #[allow(unused_must_use)] #[cfg(test)] mod discover { @@ -158,6 +169,23 @@ macro_rules! test_schema_for { test_schema_for!(dsc_lib::extensions::secret::SecretMethod); } } + + #[cfg(test)] mod plugins { + #[allow(unused_must_use)] + #[cfg(test)] mod command_name { + test_schema_for!(dsc_lib::plugins::command_name::PluginCommandName); + } + #[allow(unused_must_use)] + #[cfg(test)] mod dscplugin { + test_schema_for!(dsc_lib::plugins::dscplugin::DscPlugin); + } + #[allow(unused_must_use)] + #[cfg(test)] mod plugin_manifest { + test_schema_for!(dsc_lib::plugins::plugin_manifest::PluginArgKind); + test_schema_for!(dsc_lib::plugins::plugin_manifest::PluginCommand); + test_schema_for!(dsc_lib::plugins::plugin_manifest::PluginManifest); + } + } mod settings { test_schema_for!(dsc_lib::settings::DscSettings); test_schema_for!(dsc_lib::settings::DscSettingsScope); @@ -165,6 +193,8 @@ macro_rules! test_schema_for { mod fields { test_schema_for!(dsc_lib::settings::fields::ForbidIgnoreSettingsFileField); test_schema_for!(dsc_lib::settings::fields::IgnoreSettingsFileField); + test_schema_for!(dsc_lib::settings::fields::PluginsFileData); + test_schema_for!(dsc_lib::settings::fields::PluginsResolvedSettings); test_schema_for!(dsc_lib::settings::fields::ResourcePathFileData); test_schema_for!(dsc_lib::settings::fields::ResourcePathResolvedSettings); test_schema_for!(dsc_lib::settings::fields::TracingFileData); diff --git a/lib/dsc-lib/tests/integration/settings/builders.rs b/lib/dsc-lib/tests/integration/settings/builders.rs index 8ea7781ac..779419670 100644 --- a/lib/dsc-lib/tests/integration/settings/builders.rs +++ b/lib/dsc-lib/tests/integration/settings/builders.rs @@ -157,12 +157,56 @@ impl ResourcePathFileDataBuilder { } } +/// Builder for [`PluginsFileData`] +pub struct PluginsFileDataBuilder { + enabled: Option, +} + +impl PluginsFileDataBuilder { + /// Creates a new instance of the builder with all fields undefined. + /// + /// # Returns + /// + /// An instance of [`PluginsFileDataBuilder`]. + pub fn new() -> Self { + Self { + enabled: None, + } + } + + /// Adds the specified enabled flag to the builder. + /// + /// # Arguments + /// + /// - `enabled` - A boolean indicating whether DSC discovers and runs command plug-ins. + /// + /// # Returns + /// + /// The updated instance of [`PluginsFileDataBuilder`]. + pub fn with_enabled(mut self, enabled: bool) -> Self { + self.enabled = Some(enabled); + self + } + + /// Builds the [`PluginsFileData`] instance from the specified fields. + /// + /// # Returns + /// + /// An instance of [`PluginsFileData`]. + pub fn build(self) -> PluginsFileData { + PluginsFileData { + enabled: self.enabled, + } + } +} + /// Builder for [`PolicyFileData`] pub struct PolicyFileDataBuilder { pub forbid_ignore_settings_file: Option, pub ignore_settings_file: Option, pub tracing: Option, pub resource_path: Option, + pub plugins: Option, } impl PolicyFileDataBuilder { @@ -177,6 +221,7 @@ impl PolicyFileDataBuilder { ignore_settings_file: None, tracing: None, resource_path: None, + plugins: None, } } /// Adds the specified forbid ignore settings file flag to the builder. @@ -231,6 +276,19 @@ impl PolicyFileDataBuilder { self.resource_path = Some(resource_path); self } + /// Adds the specified plug-in configuration to the builder. + /// + /// # Arguments + /// + /// - `plugins` - An instance of [`PluginsFileData`]. + /// + /// # Returns + /// + /// The updated instance of [`PolicyFileDataBuilder`]. + pub fn with_plugins(mut self, plugins: PluginsFileData) -> Self { + self.plugins = Some(plugins); + self + } /// Builds the [`PolicyFileData`] instance from the specified fields. /// /// # Returns @@ -242,6 +300,8 @@ impl PolicyFileDataBuilder { ignore_settings_file: self.ignore_settings_file, tracing: self.tracing, resource_path: self.resource_path, + plugins: self.plugins, + lint: None, } } } @@ -250,6 +310,7 @@ impl PolicyFileDataBuilder { pub struct PreferenceFileDataBuilder { pub tracing: Option, pub resource_path: Option, + pub plugins: Option, } impl PreferenceFileDataBuilder { /// Creates a new instance of the [`PreferenceFileDataBuilder`] with default values. @@ -261,6 +322,7 @@ impl PreferenceFileDataBuilder { Self { tracing: None, resource_path: None, + plugins: None, } } @@ -292,6 +354,20 @@ impl PreferenceFileDataBuilder { self } + /// Adds the specified plug-in configuration to the builder. + /// + /// # Arguments + /// + /// - `plugins` - An instance of [`PluginsFileData`]. + /// + /// # Returns + /// + /// The updated instance of [`PreferenceFileDataBuilder`]. + pub fn with_plugins(mut self, plugins: PluginsFileData) -> Self { + self.plugins = Some(plugins); + self + } + /// Builds the [`PreferenceFileData`] instance from the specified fields. /// /// # Returns @@ -301,6 +377,8 @@ impl PreferenceFileDataBuilder { PreferenceFileData { tracing: self.tracing, resource_path: self.resource_path, + plugins: self.plugins, + lint: None, } } } @@ -773,12 +851,60 @@ impl TracingResolvedSettingsBuilder { } } +/// Builder for constructing instances of [`PluginsResolvedSettings`]. +pub struct PluginsResolvedSettingsBuilder { + pub enabled: Option>, +} + +impl PluginsResolvedSettingsBuilder { + /// Creates a new instance of the builder with default values. + /// + /// # Returns + /// + /// The newly created instance of [`PluginsResolvedSettingsBuilder`]. + pub fn new() -> Self { + Self { + enabled: None, + } + } + + /// Adds the specified enabled setting to the builder. + /// + /// # Arguments + /// + /// - `value` - The value to set for the enabled setting. + /// - `scope` - The scope of the setting. + /// + /// # Returns + /// + /// The updated instance of [`PluginsResolvedSettingsBuilder`]. + pub fn with_enabled(mut self, value: bool, scope: DscSettingsScope) -> Self { + self.enabled = Some(DscSettingsResolvedField::new(value, scope)); + self + } + + /// Builds the final instance of [`PluginsResolvedSettings`] from the builder. + /// + /// # Returns + /// + /// The constructed instance of [`PluginsResolvedSettings`]. + pub fn build(self) -> PluginsResolvedSettings { + let mut resolved = PluginsResolvedSettings::default(); + if let Some(enabled) = self.enabled { + resolved.enabled = enabled; + } + + resolved + } +} + /// Builder for constructing instances of [`DscSettingsResolved`]. pub struct DscSettingsResolvedBuilder { pub forbid_ignore_settings_file: Option>, pub ignore_settings_file: Option>, pub tracing: Option, pub resource_path: Option, + pub plugins: Option, } impl DscSettingsResolvedBuilder { @@ -793,6 +919,7 @@ impl DscSettingsResolvedBuilder { ignore_settings_file: None, tracing: None, resource_path: None, + plugins: None, } } @@ -853,6 +980,20 @@ impl DscSettingsResolvedBuilder { self.resource_path = Some(resource_path); self } + + /// Adds the specified plug-in settings to the builder. + /// + /// # Arguments + /// + /// - `plugins` - The plug-in settings to set. + /// + /// # Returns + /// + /// The updated instance of [`DscSettingsResolvedBuilder`]. + pub fn with_plugins(mut self, plugins: PluginsResolvedSettings) -> Self { + self.plugins = Some(plugins); + self + } /// Builds the final instance of [`DscSettingsResolved`] using the settings provided to the builder. /// /// # Returns @@ -872,6 +1013,9 @@ impl DscSettingsResolvedBuilder { if let Some(resource_path) = self.resource_path { settings.resource_path = resource_path; } + if let Some(plugins) = self.plugins { + settings.plugins = plugins; + } settings } diff --git a/lib/dsc-lib/tests/integration/settings/dsc_settings.rs b/lib/dsc-lib/tests/integration/settings/dsc_settings.rs index dd78bf7e2..da890bd7a 100644 --- a/lib/dsc-lib/tests/integration/settings/dsc_settings.rs +++ b/lib/dsc-lib/tests/integration/settings/dsc_settings.rs @@ -453,6 +453,167 @@ mod methods { } } + mod resolved_plugins { + use super::*; + use dsc_lib::settings::sources::{PolicyFileData, PreferenceFileData}; + use test_case::test_case; + + /// Builds preference file data that only defines `plugins.enabled`. + /// + /// # Arguments + /// + /// - `enabled` - The value for the `plugins.enabled` field. + /// + /// # Returns + /// + /// An instance of [`PreferenceFileData`] with only `plugins.enabled` defined. + fn preference(enabled: bool) -> PreferenceFileData { + PreferenceFileDataBuilder::new() + .with_plugins(PluginsFileDataBuilder::new().with_enabled(enabled).build()) + .build() + } + + /// Builds policy file data that only defines `plugins.enabled`. + /// + /// # Arguments + /// + /// - `enabled` - The value for the `plugins.enabled` field. + /// + /// # Returns + /// + /// An instance of [`PolicyFileData`] with only `plugins.enabled` defined. + fn policy(enabled: bool) -> PolicyFileData { + PolicyFileDataBuilder::new() + .with_plugins(PluginsFileDataBuilder::new().with_enabled(enabled).build()) + .build() + } + + /// Builds settings from the optional `plugins.enabled` value for each file-based scope. + /// + /// # Arguments + /// + /// - `machine` - The value for the machine preference file, if it defines one. + /// - `user` - The value for the user preference file, if it defines one. + /// - `workspace` - The value for the workspace preference file, if it defines one. + /// - `policy_value` - The value for the policy file, if it defines one. + /// + /// # Returns + /// + /// An instance of [`DscSettings`] with the specified scopes loaded. + fn settings_for( + machine: Option, + user: Option, + workspace: Option, + policy_value: Option, + ) -> DscSettings { + let mut builder = DscSettingsBuilder::new(); + if let Some(enabled) = machine { + builder = builder.with_machine(preference(enabled)); + } + if let Some(enabled) = user { + builder = builder.with_user(preference(enabled)); + } + if let Some(enabled) = workspace { + builder = builder.with_workspace(preference(enabled)); + } + if let Some(enabled) = policy_value { + builder = builder.with_policy(policy(enabled)); + } + + builder.build() + } + + #[test_case(None, None, None, None => (true, DscSettingsScope::Default); "code default enables plugins")] + #[test_case(Some(false), None, None, None => (false, DscSettingsScope::Machine); "machine overrides code default")] + #[test_case(Some(false), Some(true), None, None => (true, DscSettingsScope::User); "user overrides machine")] + #[test_case(Some(true), Some(false), None, None => (false, DscSettingsScope::User); "user disables after machine enables")] + #[test_case(Some(false), Some(false), Some(true), None => (true, DscSettingsScope::Workspace); "workspace overrides user and machine")] + #[test_case(None, None, Some(false), None => (false, DscSettingsScope::Workspace); "workspace overrides code default")] + #[test_case(None, None, None, Some(false) => (false, DscSettingsScope::Policy); "policy overrides code default")] + #[test_case(Some(true), Some(true), Some(true), Some(false) => (false, DscSettingsScope::Policy); "policy disabling overrides every preference")] + #[test_case(Some(false), Some(false), Some(false), Some(true) => (true, DscSettingsScope::Policy); "policy enabling overrides every preference")] + fn enabled( + machine: Option, + user: Option, + workspace: Option, + policy_value: Option, + ) -> (bool, DscSettingsScope) { + let settings = &mut settings_for(machine, user, workspace, policy_value); + let resolved = settings.resolved().plugins.enabled.clone(); + + (resolved.value, resolved.scope) + } + + #[test] fn only_plugins_differs_from_code_defaults() { + let settings = &mut settings_for(None, Some(false), None, None); + let expected = DscSettingsResolvedBuilder::new() + .with_plugins( + PluginsResolvedSettingsBuilder::new() + .with_enabled(false, DscSettingsScope::User) + .build() + ) + .build(); + let actual = settings.resolved().clone(); + pretty_assertions::assert_eq!(actual, expected) + } + + #[test] fn container_without_enabled_keeps_lower_scope_value() { + let settings = &mut DscSettingsBuilder::new() + .with_machine(preference(false)) + .with_user( + PreferenceFileDataBuilder::new() + .with_plugins(PluginsFileDataBuilder::new().build()) + .build() + ) + .build(); + let resolved = settings.resolved().plugins.enabled.clone(); + pretty_assertions::assert_eq!(resolved.value, false); + pretty_assertions::assert_eq!(resolved.scope, DscSettingsScope::Machine); + } + + #[test] fn ignoring_settings_files_skips_preferences() { + let settings = &mut DscSettingsBuilder::new() + .with_machine(preference(false)) + .with_user(preference(false)) + .with_workspace(preference(false)) + .with_environment( + EnvironmentDataBuilder::new() + .with_ignore_settings_file(true) + .build() + ) + .build(); + let resolved = settings.resolved().plugins.enabled.clone(); + pretty_assertions::assert_eq!(resolved.value, true); + pretty_assertions::assert_eq!(resolved.scope, DscSettingsScope::Default); + } + + #[test] fn ignoring_settings_files_keeps_policy() { + let settings = &mut DscSettingsBuilder::new() + .with_user(preference(true)) + .with_policy(policy(false)) + .with_command_line( + CommandLineDataBuilder::new() + .with_ignore_settings_file(true) + .build() + ) + .build(); + let resolved = settings.resolved().plugins.enabled.clone(); + pretty_assertions::assert_eq!(resolved.value, false); + pretty_assertions::assert_eq!(resolved.scope, DscSettingsScope::Policy); + } + + #[test] fn environment_and_command_line_cannot_override() { + let settings = &mut DscSettingsBuilder::new() + .with_user(preference(false)) + .with_environment(ENVIRONMENT_DATA.clone()) + .with_command_line(COMMAND_LINE_DATA.clone()) + .build(); + let resolved = settings.resolved().plugins.enabled.clone(); + pretty_assertions::assert_eq!(resolved.value, false); + pretty_assertions::assert_eq!(resolved.scope, DscSettingsScope::User); + } + } + #[test] fn resolve_all() { let settings = &mut DscSettingsBuilder::new() .with_machine(MACHINE_DATA.clone()) diff --git a/lib/dsc-lib/tests/integration/settings/fields/plugins.rs b/lib/dsc-lib/tests/integration/settings/fields/plugins.rs new file mode 100644 index 000000000..2053b6e03 --- /dev/null +++ b/lib/dsc-lib/tests/integration/settings/fields/plugins.rs @@ -0,0 +1,167 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +crate::macros::test_dsc_repo_schema! { + define_statics: { + dsc_lib::settings::fields::PluginsFileData, + properties: { + ENABLED_SCHEMA => "enabled", + } + }, + test_meta_schema: {}, + test_docs: { + title => "title", + description => "description", + markdown_description => "markdownDescription", + enabled_title => (ENABLED_SCHEMA, "title"), + enabled_description => (ENABLED_SCHEMA, "description"), + enabled_markdown_description => (ENABLED_SCHEMA, "markdownDescription"), + }, + test_validation: { + empty_object_is_valid => { + input_json: json!({}), + expected_valid: true, + }, + with_enabled_as_true_is_valid => { + input_json: json!({"enabled": true}), + expected_valid: true, + }, + with_enabled_as_false_is_valid => { + input_json: json!({"enabled": false}), + expected_valid: true, + }, + with_enabled_as_string_is_invalid => { + input_json: json!({"enabled": "true"}), + expected_valid: false, + }, + with_enabled_as_integer_is_invalid => { + input_json: json!({"enabled": 1}), + expected_valid: false, + }, + with_enabled_as_null_is_valid => { + input_json: json!({"enabled": null}), + expected_valid: true, + }, + with_an_unexpected_field_is_invalid => { + input_json: json!({"unexpectedField": true}), + expected_valid: false, + }, + array_value_is_invalid => { + input_json: json!([]), + expected_valid: false, + }, + string_value_is_invalid => { + input_json: json!("invalid"), + expected_valid: false, + }, + null_value_is_invalid => { + input_json: json!(null), + expected_valid: false, + }, + boolean_value_is_invalid => { + input_json: json!(true), + expected_valid: false, + }, + integer_value_is_invalid => { + input_json: json!(123), + expected_valid: false, + }, + }, +} + +#[cfg(test)] +mod serde { + use dsc_lib::settings::fields::PluginsFileData; + use serde_json::{json, Value}; + use test_case::test_case; + + #[test_case( + PluginsFileData::default() => json!({}); + "default serializes as empty object" + )] + #[test_case( + PluginsFileData { + enabled: Some(true), + } => json!({ + "enabled": true + }); "enabled as true serializes as boolean" + )] + #[test_case( + PluginsFileData { + enabled: Some(false), + } => json!({ + "enabled": false + }); "enabled as false serializes as boolean" + )] + fn serializing(data: PluginsFileData) -> Value { + serde_json::to_value(data).expect("serialization should never fail") + } + + #[test_case(&json!("string"); "string value")] + #[test_case(&json!(123); "integer value")] + #[test_case(&json!(1.2); "number value")] + #[test_case(&json!(true); "boolean value")] + #[test_case(&json!(null); "null value")] + #[test_case(&json!([{"enabled": true}]); "array value")] + #[test_case(&json!({"enabled": "invalid"}); "object with enabled as string value")] + #[test_case(&json!({"enabled": 123}); "object with enabled as number value")] + #[test_case(&json!({"enabled": {"nested": "object"}}); "object with enabled as object value")] + #[test_case(&json!({"enabled": [false]}); "object with enabled as array value")] + fn deserializing_invalid(input_value: &Value) { + serde_json::from_value::(input_value.clone()) + .expect_err(&format!("json value '{input_value}' should be invalid")); + } + + #[test_case( + &json!({}) => PluginsFileData::default(); + "empty object deserializes as default" + )] + #[test_case( + &json!({ + "enabled": null + }) => PluginsFileData::default(); + "enabled as null deserializes as undefined" + )] + #[test_case( + &json!({ + "enabled": true + }) => PluginsFileData { + enabled: Some(true), + }; + "enabled as true deserializes as boolean" + )] + #[test_case( + &json!({ + "enabled": false + }) => PluginsFileData { + enabled: Some(false), + }; + "enabled as false deserializes as boolean" + )] + fn deserializing_valid(input_value: &Value) -> PluginsFileData { + serde_json::from_value::(input_value.clone()) + .expect(&format!( + "expected input to serialize but failed for input: {}", + serde_json::to_string_pretty(input_value).unwrap(), + )) + } +} + +#[cfg(test)] +mod defaults { + use dsc_lib::settings::{ + DscSettingsScope, + fields::{CODE_DEFAULT_PLUGINS, PluginsCodeDefaults, PluginsResolvedSettings}, + }; + + #[test] fn code_default_enables_plugins() { + pretty_assertions::assert_eq!(CODE_DEFAULT_PLUGINS.enabled, true); + pretty_assertions::assert_eq!(PluginsCodeDefaults::default(), CODE_DEFAULT_PLUGINS); + } + + #[test] fn resolved_default_is_enabled_from_default_scope() { + let resolved = PluginsResolvedSettings::default(); + pretty_assertions::assert_eq!(resolved.enabled.value, true); + pretty_assertions::assert_eq!(resolved.enabled.scope, DscSettingsScope::Default); + } +} diff --git a/lib/dsc-lib/tests/integration/settings/mod.rs b/lib/dsc-lib/tests/integration/settings/mod.rs index 17b3d7a32..4e6a88764 100644 --- a/lib/dsc-lib/tests/integration/settings/mod.rs +++ b/lib/dsc-lib/tests/integration/settings/mod.rs @@ -6,6 +6,7 @@ mod builders; mod fields { mod ignore_settings_file; mod forbid_ignore_settings_file; + mod plugins; mod resource_path; mod tracing; } diff --git a/lib/dsc-lib/tests/integration/settings/resolved/settings.rs b/lib/dsc-lib/tests/integration/settings/resolved/settings.rs index 11a78b4c1..54728f8bb 100644 --- a/lib/dsc-lib/tests/integration/settings/resolved/settings.rs +++ b/lib/dsc-lib/tests/integration/settings/resolved/settings.rs @@ -9,6 +9,7 @@ crate::macros::test_dsc_repo_schema! { IGNORE_SETTINGS_FILE_SCHEMA => "ignoreSettingsFile", TRACING_SCHEMA => "tracing", RESOURCE_PATH_SCHEMA => "resourcePath", + PLUGINS_SCHEMA => "plugins", } }, test_meta_schema: {}, @@ -28,6 +29,9 @@ crate::macros::test_dsc_repo_schema! { resource_path_title => (RESOURCE_PATH_SCHEMA, "title"), resource_path_description => (RESOURCE_PATH_SCHEMA, "description"), resource_path_markdown_description => (RESOURCE_PATH_SCHEMA, "markdownDescription"), + plugins_title => (PLUGINS_SCHEMA, "title"), + plugins_description => (PLUGINS_SCHEMA, "description"), + plugins_markdown_description => (PLUGINS_SCHEMA, "markdownDescription"), }, test_validation: { with_all_defined_is_valid => { @@ -63,9 +67,55 @@ crate::macros::test_dsc_repo_schema! { "value": true, "scope": "policy" }, + }, + "plugins": { + "enabled": { + "value": false, + "scope": "policy" + }, } }), expected_valid: true, + }, + with_plugins_enabled_as_bare_boolean_is_invalid => { + input_json: json!({ + "forbidIgnoreSettingsFile": { + "value": false, + "scope": "default" + }, + "ignoreSettingsFile": { + "value": false, + "scope": "default" + }, + "tracing": { + "level": { + "value": "warn", + "scope": "default" + }, + "format": { + "value": "default", + "scope": "default" + } + }, + "resourcePath": { + "appendEnvPath": { + "value": true, + "scope": "default" + }, + "directories": { + "value": [], + "scope": "default" + }, + "restricted": { + "value": false, + "scope": "default" + }, + }, + "plugins": { + "enabled": false + } + }), + expected_valid: false, } } } @@ -92,10 +142,38 @@ mod serde { directories: DscSettingsResolvedField::for_policy(vec!["/dsc/resources".into(), "/dsc/extensions".into()]), restricted: DscSettingsResolvedField::for_policy(true.into()), }, + plugins: PluginsResolvedSettings { + enabled: DscSettingsResolvedField::for_policy(false), + }, }; - serde_json::to_value(&settings) + let value = serde_json::to_value(&settings) + .expect("serialization should never fail"); + + pretty_assertions::assert_eq!( + value["plugins"], + serde_json::json!({ + "enabled": { + "value": false, + "scope": "policy" + } + }) + ); + } + + #[test] fn default_enables_plugins_from_code_defaults() { + let value = serde_json::to_value(DscSettingsResolved::default()) .expect("serialization should never fail"); + + pretty_assertions::assert_eq!( + value["plugins"], + serde_json::json!({ + "enabled": { + "value": true, + "scope": "default" + } + }) + ); } // No tests for deserializing, as the type only implements Serialize. diff --git a/lib/dsc-lib/tests/integration/settings/sources/policy_file.rs b/lib/dsc-lib/tests/integration/settings/sources/policy_file.rs index 3b78eb436..a6c6ffa36 100644 --- a/lib/dsc-lib/tests/integration/settings/sources/policy_file.rs +++ b/lib/dsc-lib/tests/integration/settings/sources/policy_file.rs @@ -19,6 +19,7 @@ crate::macros::test_dsc_repo_schema! { IGNORE_PROPERTY_SCHEMA => "ignoreSettingsFile", TRACING_PROPERTY_SCHEMA => "tracing", RESOURCE_PATH_PROPERTY_SCHEMA => "resourcePath", + PLUGINS_PROPERTY_SCHEMA => "plugins", } }, test_meta_schema: {}, @@ -38,6 +39,9 @@ crate::macros::test_dsc_repo_schema! { tracing_title => (TRACING_PROPERTY_SCHEMA, "title"), tracing_description => (TRACING_PROPERTY_SCHEMA, "description"), tracing_markdown_description => (TRACING_PROPERTY_SCHEMA, "markdownDescription"), + plugins_title => (PLUGINS_PROPERTY_SCHEMA, "title"), + plugins_description => (PLUGINS_PROPERTY_SCHEMA, "description"), + plugins_markdown_description => (PLUGINS_PROPERTY_SCHEMA, "markdownDescription"), }, test_validation: { empty_object_is_valid => { @@ -76,12 +80,28 @@ crate::macros::test_dsc_repo_schema! { input_json: json!({"resourcePath": {"appendEnvPath": "invalid"}}), expected_valid: false, }, + plugins_field_as_empty_object_is_valid => { + input_json: json!({"plugins": {}}), + expected_valid: true, + }, + plugins_field_with_valid_data_is_valid => { + input_json: json!({"plugins": {"enabled": false}}), + expected_valid: true, + }, + plugins_field_with_invalid_data_is_invalid => { + input_json: json!({"plugins": {"enabled": "invalid"}}), + expected_valid: false, + }, + plugins_field_as_boolean_is_invalid => { + input_json: json!({"plugins": false}), + expected_valid: false, + }, } } #[cfg(test)] mod serde { - use dsc_lib::settings::{fields::{ForbidIgnoreSettingsFileField, ResourcePathFileData}, sources::PolicyFileData}; + use dsc_lib::settings::{fields::{ForbidIgnoreSettingsFileField, PluginsFileData, ResourcePathFileData}, sources::PolicyFileData}; use std::path::PathBuf; use serde_json::{json, Value}; use test_case::test_case; @@ -98,7 +118,9 @@ mod serde { PathBuf::from("/dsc/extensions") ]), ..Default::default() - }) + }), + plugins: None, + lint: None, }, json!({ "forbidIgnoreSettingsFile": true, @@ -108,6 +130,18 @@ mod serde { }); "with defined fields only includes those defined fields" )] + #[test_case( + &PolicyFileData { + plugins: Some(PluginsFileData { enabled: Some(false) }), + ..Default::default() + }, + json!({ + "plugins": { + "enabled": false + } + }); + "with plugins defined only includes plugins" + )] fn serializing(data: &PolicyFileData, expected: Value) { let serialized = serde_json::to_value(data) .expect("serialization should never fail"); @@ -118,6 +152,8 @@ mod serde { #[test_case(json!({"ignoreSettingsFile": "invalid"}); "ignore_settings_file as non-boolean is invalid")] #[test_case(json!({"resourcePath": {"appendEnvPath": "invalid"}}); "resource_path with invalid data is invalid")] #[test_case(json!({"tracing": {"level": "invalid"}}); "tracing with invalid data is invalid")] + #[test_case(json!({"plugins": {"enabled": "invalid"}}); "plugins with invalid data is invalid")] + #[test_case(json!({"plugins": false}); "plugins as boolean is invalid")] fn derserializing_invalid(input_value: Value) { let result = serde_json::from_value::(input_value.clone()); assert!(result.is_err(), "expected deserialization to fail for input: {:?}", input_value); @@ -127,6 +163,7 @@ mod serde { #[test_case(json!({"ignoreSettingsFile": true}); "ignore_settings_file as boolean is valid")] #[test_case(json!({"resourcePath": {"directories": ["/dsc/resources"]}}); "resource_path with valid data is valid")] #[test_case(json!({"tracing": {"level": "info"}}); "tracing with valid data is valid")] + #[test_case(json!({"plugins": {"enabled": false}}); "plugins with valid data is valid")] fn deserializing_valid(input_value: Value) { let result = serde_json::from_value::(input_value.clone()); assert!( diff --git a/lib/dsc-lib/tests/integration/settings/sources/preference_file.rs b/lib/dsc-lib/tests/integration/settings/sources/preference_file.rs index 84ec08526..4847be49f 100644 --- a/lib/dsc-lib/tests/integration/settings/sources/preference_file.rs +++ b/lib/dsc-lib/tests/integration/settings/sources/preference_file.rs @@ -17,6 +17,7 @@ crate::macros::test_dsc_repo_schema!{ properties: { TRACING_PROPERTY_SCHEMA => "tracing", RESOURCE_PATH_PROPERTY_SCHEMA => "resourcePath", + PLUGINS_PROPERTY_SCHEMA => "plugins", } }, test_meta_schema: {}, @@ -30,6 +31,9 @@ crate::macros::test_dsc_repo_schema!{ resource_path_title => (RESOURCE_PATH_PROPERTY_SCHEMA, "title"), resource_path_description => (RESOURCE_PATH_PROPERTY_SCHEMA, "description"), resource_path_markdown_description => (RESOURCE_PATH_PROPERTY_SCHEMA, "markdownDescription"), + plugins_title => (PLUGINS_PROPERTY_SCHEMA, "title"), + plugins_description => (PLUGINS_PROPERTY_SCHEMA, "description"), + plugins_markdown_description => (PLUGINS_PROPERTY_SCHEMA, "markdownDescription"), }, test_validation: { empty_object_is_valid => { @@ -52,12 +56,28 @@ crate::macros::test_dsc_repo_schema!{ input_json: json!({"resourcePath": {"appendEnvPath": "invalid"}}), expected_valid: false, }, + plugins_field_as_empty_object_is_valid => { + input_json: json!({"plugins": {}}), + expected_valid: true, + }, + plugins_field_with_valid_data_is_valid => { + input_json: json!({"plugins": {"enabled": false}}), + expected_valid: true, + }, + plugins_field_with_invalid_data_is_invalid => { + input_json: json!({"plugins": {"enabled": "invalid"}}), + expected_valid: false, + }, + plugins_field_as_boolean_is_invalid => { + input_json: json!({"plugins": false}), + expected_valid: false, + }, } } #[cfg(test)] mod serde { - use dsc_lib::settings::{fields::ResourcePathFileData, sources::PreferenceFileData}; + use dsc_lib::settings::{fields::{PluginsFileData, ResourcePathFileData}, sources::PreferenceFileData}; use std::path::PathBuf; use serde_json::{json, Value}; use test_case::test_case; @@ -72,7 +92,9 @@ mod serde { PathBuf::from("/dsc/extensions") ]), ..Default::default() - }) + }), + plugins: None, + lint: None, }, json!({ "resourcePath": { @@ -81,6 +103,18 @@ mod serde { }); "with defined fields only includes those defined fields" )] + #[test_case( + &PreferenceFileData { + plugins: Some(PluginsFileData { enabled: Some(false) }), + ..Default::default() + }, + json!({ + "plugins": { + "enabled": false + } + }); + "with plugins defined only includes plugins" + )] fn serializing(data: &PreferenceFileData, expected: Value) { let serialized = serde_json::to_value(data) .expect("serialization should never fail"); @@ -89,6 +123,8 @@ mod serde { #[test_case(json!({"tracing": {"level": "invalid"}}); "tracing with invalid data is invalid")] #[test_case(json!({"resourcePath": {"appendEnvPath": "invalid"}}); "resource_path with invalid data is invalid")] + #[test_case(json!({"plugins": {"enabled": "invalid"}}); "plugins with invalid data is invalid")] + #[test_case(json!({"plugins": false}); "plugins as boolean is invalid")] fn derserializing_invalid(input_value: Value) { let result = serde_json::from_value::(input_value.clone()); assert!(result.is_err(), "expected deserialization to fail for input: {:?}", input_value); @@ -96,6 +132,7 @@ mod serde { #[test_case(json!({}); "empty object is valid")] #[test_case(json!({"resourcePath": {"directories": ["/dsc/resources"]}}); "resource_path with valid data is valid")] #[test_case(json!({"tracing": {"level": "info"}}); "tracing with valid data is valid")] + #[test_case(json!({"plugins": {"enabled": false}}); "plugins with valid data is valid")] fn deserializing_valid(input_value: Value) { let result = serde_json::from_value::(input_value.clone()); assert!( diff --git a/packaging/msix/AppxManifest.xml b/packaging/msix/AppxManifest.xml index 8877ddaa9..aed4d67b2 100644 --- a/packaging/msix/AppxManifest.xml +++ b/packaging/msix/AppxManifest.xml @@ -52,6 +52,17 @@ + + + + + + + + + + + diff --git a/packaging/rpm/dsc.spec b/packaging/rpm/dsc.spec index 76267d725..980614116 100644 --- a/packaging/rpm/dsc.spec +++ b/packaging/rpm/dsc.spec @@ -29,11 +29,13 @@ cp -r $RPM_SOURCE_DIR/dsc_files/* $RPM_BUILD_ROOT/opt/dsc/ # Create symlink to make dsc available in PATH ln -s /opt/dsc/dsc $RPM_BUILD_ROOT/usr/bin/dsc ln -s /opt/dsc/dsc-bicep-ext $RPM_BUILD_ROOT/usr/bin/dsc-bicep-ext +ln -s /opt/dsc/dsclint $RPM_BUILD_ROOT/usr/bin/dsclint %files /opt/dsc/* /usr/bin/dsc /usr/bin/dsc-bicep-ext +/usr/bin/dsclint %changelog * Wed Oct 22 2025 Microsoft Corporation diff --git a/xtask/src/schemas/export.rs b/xtask/src/schemas/export.rs index 04c51644c..8467e22f8 100644 --- a/xtask/src/schemas/export.rs +++ b/xtask/src/schemas/export.rs @@ -149,11 +149,18 @@ pub(crate) fn export_schemas( dsc_lib::extensions::extension_manifest::ExtensionManifest, dsc_lib::extensions::import::ImportMethod, dsc_lib::extensions::secret::SecretMethod, + dsc_lib::plugins::command_name::PluginCommandName, + dsc_lib::plugins::dscplugin::DscPlugin, + dsc_lib::plugins::plugin_manifest::PluginCommand, + dsc_lib::plugins::plugin_manifest::PluginManifest, dsc_lib::functions::FunctionArgKind, dsc_lib::functions::FunctionCategory, dsc_lib::functions::FunctionDefinition, + dsc_lib::lint::diagnostic::LintResult, + dsc_lib::lint::rule::LintRuleInfo, dsc_lib::settings::fields::ForbidIgnoreSettingsFileField, dsc_lib::settings::fields::IgnoreSettingsFileField, + dsc_lib::settings::fields::PluginsFileData, dsc_lib::settings::fields::ResourcePathFileData, dsc_lib::settings::fields::TracingFileData, dsc_lib::settings::fields::TracingFormatField,