Skip to content

fix: hand a list from a map source to a slice flag in its serialized form - #2524

Open
paoValle wants to merge 3 commits into
urfave:mainfrom
paoValle:fix/map-value-source-slices
Open

paoValle wants to merge 3 commits into
urfave:mainfrom
paoValle:fix/map-value-source-slices

Conversation

@paoValle

@paoValle paoValle commented Nov 3, 2026

Copy link
Copy Markdown

What type of PR is this?

  • bug

What this PR does / why we need it:

A slice flag splits its value on commas, and mapValueSource.Lookup formatted whatever it found with %+v. A list in the map therefore reached the flag as "[80 443]": the IntSliceFlag from the issue fails to parse it, and a StringSliceFlag keeps the whole string as one element without reporting an error.

A comma-joined value would fix the reported case and break the next ones: an element that contains a comma is silently split, and SliceFlagSeparator/DisableSliceFlagSeparator change the separator the flag expects. So Lookup returns slices and arrays in the serialized form SliceBase.Set already parses, which bypasses splitting entirely. Scalars format as before.

Which issue(s) this PR fixes:

Fixes #2516

Tests

TestMapValueSourceFeedsSliceFlags drives a command with an IntSliceFlag and a StringSliceFlag fed from a map source, which is the reproduction in the issue, and checks the parsed values. TestMapValueSourceListSeparators covers an element containing a comma, a command with SliceFlagSeparator: ";" and one with the separator disabled. Reverting the Lookup change fails them with the error from the issue.

Release Notes

A list in a map value source now reaches a slice flag intact, instead of failing on `"[80 443]"` or silently keeping one element.

…form

A slice flag splits its value on commas, and the map value source formatted
whatever it found with %+v. A list therefore arrived as "[80 443]": the
IntSliceFlag from the issue failed to parse it, and a StringSliceFlag kept the
whole string as one element without an error.

The value source now returns a slice or array in the serialized form SliceBase
already parses, so the flag stops splitting it: an element that contains a
comma, a custom SliceFlagSeparator and a disabled separator all arrive intact.
@paoValle
paoValle requested a review from a team as a code owner November 3, 2026 13:01

@yxshee yxshee left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I opened #2516, so I ran this against a few more config shapes. Keeping commas inside elements and honoring a custom or disabled separator is a real improvement over just joining the items with commas. The catch is that the serialized path skips the per-element parsing, and SliceBase.Set drops the json.Unmarshal error, so when the JSON types don't match the flag's element type the flag ends up with zero values and no error:

map value flag this PR joining with ,
[]any{80, 443} StringSliceFlag ["" ""] ["80" "443"]
[]string{"80", "443"} IntSliceFlag [0 0] [80 443]
[]any{"a", 2, true} StringSliceFlag ["a" "" ""] ["a" "2" "true"]
[]any{-1} UintSliceFlag [0] parse error
[]string{"x", "y"} StringFlag sl:::1793718754028232588:::["x","y"] x,y

cmd.Run returned nil for all of these on this branch. The first row is probably the common one, since YAML like ports: [80, 443] decodes to []any of ints, and a string slice is a reasonable flag for it. The last row is the internal prefix leaking into a non-slice flag.

A couple of ways to keep the separator fix without this: in the serialized branch, unmarshal into []string (formatting each element with %+v on the value source side) and pass each item through the flag's value parser, the same way the split path does. At a minimum, returning the json.Unmarshal error there would turn the silent zeros into an error.

The tests in the PR pass for me, as do go test ./..., gofmt and go vet.

@paoValle

paoValle commented Nov 6, 2026

Copy link
Copy Markdown
Author

Both shapes are fixed in e0a31644. The serialized branch now decodes the array and hands each element to the same parser the split path uses, so the first four rows behave like your joining column: []any{80,443} into a StringSliceFlag gives ["80" "443"], []string{"80","443"} into an IntSliceFlag gives [80 443], the mixed row gives ["a" "2" "true"], and []any{-1} into a UintSliceFlag returns a parse error instead of [0]. The last row is handled before Set: a scalar flag gets the elements comma-joined (x,y), never the sl::: prefix. TestMapValueSourceListsWithMismatchedTypes pins all five, and the suite and vet are green.

@paoValle
paoValle force-pushed the fix/map-value-source-slices branch from e0a3164 to 91614ff Compare November 6, 2026 17:09
@paoValle

paoValle commented Nov 6, 2026

Copy link
Copy Markdown
Author

Replaced e0a31644 with 91614ffc: same behaviour, but the serialized array is now scanned instead of decoded.

Decoding into any concrete type ([]string, []any, []json.RawMessage) links the whole encoding/json decoder here, which took the example binary from 2.0 MB to 2.4 MB and failed the binary-size check. A serialized slice is a flat array of scalars, so the scan only finds the element boundaries; TestMapValueSourceListElementsSurviveTheSerializedForm pins commas, quotes, backslashes, newlines, empty strings and brackets.

The five rows behave as described above, everything is green, and the binary is back at its 2.0 MB.

A list from a map source arrives serialized, and Set decoded it straight into
the flag's slice with json.Unmarshal, dropping the error. A JSON number in a
string slice became "", a JSON string in an int slice became 0, and a negative
in an unsigned slice silently became 0 instead of failing.

The serialized branch now splits the array and hands each element to the same
value parser the split path uses, so mismatched types parse or report instead
of turning into zero values. A scalar flag no longer sees the internal prefix:
its elements are comma-joined before Set.

The split scans the array rather than decoding it: a decode into any concrete
type links the whole encoding/json decoder, which adds 300 KB to the example
binary and fails `make check-binary-size`. A serialized slice is a flat array
of scalars, so the scan covers every element Serialize can write.
@paoValle
paoValle force-pushed the fix/map-value-source-slices branch from 91614ff to 458cde6 Compare November 6, 2026 17:25
@paoValle

paoValle commented Nov 6, 2026

Copy link
Copy Markdown
Author

The branch head is 458cde65 now: it adds tests for the scanner's refusals (malformed array, unterminated string, bad escape, nested value, missing comma), which is what the patch-coverage check was flagging, plus a check that a malformed value reaches the caller as an error.

@yxshee

yxshee commented Nov 15, 2026

Copy link
Copy Markdown
Contributor

One follow-up to my earlier suggestion: routing every serialized element through the flag's parser changes existing Serialize/Set round trips when IntegerConfig.Base is not decimal.

I ran the same public-API program on main d5c06cb and this head 458cde65:

IntegerConfig.Base Original value Main restores This PR restores
10 [16] [16] [16]
16 [16] [16] [22], no error
2 [16] [16] error parsing "16"

Minimal hexadecimal case:

var original, restored []int
value := cli.IntSlice{}.Create([]int{16}, &original, cli.IntegerConfig{Base: 16})
encoded := value.(cli.Serializer).Serialize()
receiver := cli.IntSlice{}.Create(nil, &restored, cli.IntegerConfig{Base: 16})
err := receiver.Set(encoded)
fmt.Println(restored, err) // main: [16] <nil>; PR: [22] <nil>

Serialize writes the number as decimal JSON 16, but flag_slice_base.go:73 now reparses that token as hexadecimal 16. The serialized round trip needs to retain the numeric value independently of the flag's text-input base. A non-decimal configuration regression would catch this while keeping the mixed-type map-source coverage added in response to the earlier review.

Tested on macOS arm64 with Go 1.26.5. This is a focused reproduction, not a full approval of the PR.

`Serialize` writes canonical JSON, so a serialized number is always decimal.
Feeding each token back through the flag's own parser made the round trip
depend on `IntegerConfig.Base`: `16` came back as `22` with base 16, and as a
parse error with base 2.

A quoted token is text and still goes to the parser, which keeps the mixed-type
cases the previous commit added. A bare number or bool is a value, decoded in
`T`, so the input base never reads it. A `null` or empty element is now refused
by the scanner instead of becoming a zero value or a panic, and a rejected
element leaves the current list untouched.

Text input is unchanged: with base 16, `--ports 10` still reads 16.
@paoValle

Copy link
Copy Markdown
Author

Fixed on 0683c24d. A serialized token is canonical JSON, so a bare number now decodes into the element type instead of going back through the parser; a quoted token is still text and still goes through it, so the input base keeps working where it should.

Your three rows are covered by the new round-trip test over bases 2 and 16, with a uint variant and a check that with base 16 the text 10 still reads 16. The full suite is green and the binary size is unchanged.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

NewMapValueSource passes a list to slice flags as "[a b]"

2 participants