Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
7 changes: 7 additions & 0 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -238,6 +238,13 @@ change the source contract, template, or post-processing step and regenerate onl
The generated client is not a deployment-aware public facade. Public operations should select deployment/auth through
SDK configuration, then call the appropriate generated transport internally.

Before generation, `scripts/prepare_open_enums.py` expands enum uses to a union of the existing enum and its primitive
type in a temporary copy of the spec. Known values remain enum members. Unfamiliar values remain strings or integers
and serialize unchanged. The checked-in spec and enum class names stay unchanged. Singleton data enums remain open;
constants, discriminator fields, and enums marked `x-ms-enum.modelAsString: false` remain closed. Compatible arrays of
enum alternatives share an item union so every known enum is tried before accepting an unfamiliar value. The templates
check array containers and primitive types. SDK operations that select behavior must handle unsupported values explicitly.

## Test Architecture

Root pytest configuration enables xdist, disables non-local sockets, injects fake credentials, and sets timeouts.
Expand Down
8 changes: 8 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
integrations and task types 7, 9 and 12, are kept so reads from those servers keep working.
- CrewAI run IDs are now derived from SHA-256 (as RFC 4122 version 4 UUIDs) instead of MD5, resolving a
SAST finding. IDs for the same inputs differ from earlier releases.
- Generated API parsers preserve unfamiliar string and integer enum values from newer servers,
including nested run creator roles and permission actions. Known values remain enum members.
Unknown values remain raw strings or integers and serialize unchanged. Generated type hints
include those primitive alternatives. Constants and explicitly closed enums stay strict.
- Evaluator lookup reports unsupported scorer types explicitly instead of treating them as built-in evaluators.
- Preserve known categorical roll-up members before using raw string fallbacks, keep singleton data enums extensible,
and reject malformed containers in enum arrays. Preserve enum behavior across inline, inferred, nullable, and wrapped
schema definitions during client generation.

## [0.6.1]

Expand Down
11 changes: 11 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -690,6 +690,17 @@ with splunk_ao_context(
...
```

### API enum compatibility

API enum fields can contain values added after your SDK version was released. Known values remain enum members.
Unfamiliar values remain strings or integers, and generated type hints include those alternatives. Reading and
serializing these values preserves them exactly. You can also send new enum values as their raw strings or integers
for the service to validate. Enums with one known value remain extensible. Constants, discriminator fields, and
explicitly closed enums retain their fixed alternatives. Array fields still require arrays.

SDK operations that need a specific implementation can still reject an unsupported value. For example, evaluator
lookup reports an unsupported scorer type and asks you to upgrade instead of guessing its implementation.

## Contributing

See [CONTRIBUTING.md](CONTRIBUTING.md) for development setup and guidelines and
Expand Down
85 changes: 85 additions & 0 deletions codegen_templates/property_templates/list_property.py.jinja
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
{% macro find_enums(property, ns) %}
{% if property.template == "enum_property.py.jinja" %}
{% set ns.has_enum = true %}
{% elif property.template == "union_property.py.jinja" %}
{% for inner_property in property.inner_properties %}
{% set _ = find_enums(inner_property, ns) %}
{% endfor %}
{% elif property.template == "list_property.py.jinja" %}
{% set _ = find_enums(property.inner_property, ns) %}
{% endif %}
{% endmacro %}

{% macro _check_type(property, source) %}
{% set ns = namespace(has_enum=false) %}
{% set _ = find_enums(property.inner_property, ns) %}
{% if ns.has_enum %}
if not isinstance({{ source }}, list):
raise TypeError("Unexpected value type for {{ property.python_name }}")
{% endif %}
{% endmacro %}

{% macro construct(property, source) %}
{% set inner_property = property.inner_property %}
{% import "property_templates/" + inner_property.template as inner_template %}
{% if inner_template.construct %}
{% set inner_source = inner_property.python_name + "_data" %}
{% if property.required %}
{{ property.python_name }} = []
_{{ property.python_name }} = {{ source }}
{{ _check_type(property, "_" + property.python_name) }}
for {{ inner_source }} in (_{{ property.python_name }}):
{{ inner_template.construct(inner_property, inner_source) | indent(4) }}
{{ property.python_name }}.append({{ inner_property.python_name }})
{% else %}
_{{ property.python_name }} = {{ source }}
{{ property.python_name }}: {{ property.get_type_string() }} = UNSET
if _{{ property.python_name }} is not UNSET:
{{ _check_type(property, "_" + property.python_name) | indent(4) }}
{{ property.python_name }} = []
for {{ inner_source }} in _{{ property.python_name }}:
{{ inner_template.construct(inner_property, inner_source) | indent(8) }}
{{ property.python_name }}.append({{ inner_property.python_name }})
{% endif %}
{% else %}
{{ property.python_name }} = cast({{ property.get_type_string(no_optional=True) }}, {{ source }})
{% endif %}
{% endmacro %}

{% macro _transform(property, source, destination, transform_method) %}
{% set inner_property = property.inner_property %}
{% import "property_templates/" + inner_property.template as inner_template %}
{% if inner_template.transform %}
{% set inner_source = inner_property.python_name + "_data" %}
{{ _check_type(property, source) }}
{{ destination }} = []
for {{ inner_source }} in {{ source }}:
{{ inner_template.transform(inner_property, inner_source, inner_property.python_name, transform_method) | indent(4) }}
{{ destination }}.append({{ inner_property.python_name }})
{% else %}
{{ destination }} = {{ source }}
{% endif %}
{% endmacro %}

{% macro check_type_for_construct(property, source) %}isinstance({{ source }}, list){% endmacro %}

{% macro transform(property, source, destination, declare_type=True, skip_unset=False) %}
{% set inner_property = property.inner_property %}
{% set type_string = property.get_type_string(json=True) %}
{% if property.required %}
{{ _transform(property, source, destination, "to_dict") }}
{% else %}
{% if not skip_unset %}{{ destination }}{% if declare_type %}: {{ type_string }}{% endif %} = UNSET{% endif +%}
if not isinstance({{ source }}, Unset):
{{ _transform(property, source, destination, "to_dict") | indent(4)}}
{% endif %}
{% endmacro %}

{% macro multipart(property, source, destination) %}
{% set inner_property = property.inner_property %}
{% import "property_templates/" + inner_property.template as inner_template %}
{% set inner_source = inner_property.python_name + "_element" %}
{{ _check_type(property, source) }}
for {{ inner_source }} in {{ source }}:
{{ inner_template.multipart(inner_property, inner_source, destination) | indent(4) }}
{% endmacro %}
33 changes: 32 additions & 1 deletion codegen_templates/property_templates/union_property.py.jinja
Original file line number Diff line number Diff line change
@@ -1,3 +1,19 @@
{% macro _check_enum_array_type(property, source) %}
{% import "property_templates/list_property.py.jinja" as list_template %}
{% set array_ns = namespace(has_enum=false, only_arrays=true) %}
{% for inner_property in property.inner_properties %}
{% if inner_property.template == "list_property.py.jinja" %}
{% set _ = list_template.find_enums(inner_property.inner_property, array_ns) %}
{% elif inner_property.get_instance_type_string() not in ["None", "Unset"] %}
{% set array_ns.only_arrays = false %}
{% endif %}
{% endfor %}
{% if array_ns.has_enum and array_ns.only_arrays %}
if not isinstance({{ source }}, list){% if "None" in property.get_type_strings_in_union(json=True) %} and {{ source }} is not None{% endif %}{% if not property.required %} and not isinstance({{ source }}, Unset){% endif %}:
raise TypeError("Unexpected value type for {{ property.python_name }}")
{% endif %}
{% endmacro %}

{% macro construct(property, source) %}
def _parse_{{ property.python_name }}(data: object) -> {{ property.get_type_string() }}:
{% if "None" in property.get_type_strings_in_union(json=True) %}
Expand All @@ -8,6 +24,7 @@ def _parse_{{ property.python_name }}(data: object) -> {{ property.get_type_stri
if isinstance(data, Unset):
return data
{% endif %}
{{ _check_enum_array_type(property, "data") | indent(4) }}
{# Ad Hoc check for Extended*Record types #}
{# Ref.: https://app.shortcut.com/galileo/story/46169/error-when-fetching-spans-using-search-get-spans-via-python-sdk #}
{% set has_extended_records = false %}
Expand Down Expand Up @@ -66,11 +83,20 @@ def _parse_{{ property.python_name }}(data: object) -> {{ property.get_type_stri

# Fallback to standard union parsing
{% endif %}
{% set ns = namespace(contains_unmodified_properties = false) %}
{% set ns = namespace(contains_unmodified_properties=false, has_enum=false, raw_types=[], can_check_raw=true) %}
{% for inner_property in property.inner_properties %}
{% import "property_templates/" + inner_property.template as inner_template %}
{% if inner_property.template == "enum_property.py.jinja" %}
{% set ns.has_enum = true %}
{% endif %}
{% if not inner_template.construct %}
{% set ns.contains_unmodified_properties = true %}
{% set raw_type = inner_property.get_instance_type_string() %}
{% if raw_type in ["str", "int", "float", "bool"] %}
{% set ns.raw_types = ns.raw_types + [raw_type] %}
{% elif raw_type not in ["None", "Unset"] %}
{% set ns.can_check_raw = false %}
{% endif %}
{% continue %}
{% endif %}
{% if has_extended_records %}{# For Extended*Record unions, always wrap in try-except #}
Expand Down Expand Up @@ -109,6 +135,10 @@ def _parse_{{ property.python_name }}(data: object) -> {{ property.get_type_stri
{% endif %}
{% endfor %}
{% if ns.contains_unmodified_properties %}
{% if ns.has_enum and ns.can_check_raw and ns.raw_types %}
if type(data) not in ({{ ns.raw_types | unique | join(", ") }},):
raise TypeError("Unexpected value type for {{ property.python_name }}")
{% endif %}
return cast({{ property.get_type_string() }}, data)
{% elif has_extended_records %}
# If we reach here, none of the parsers succeeded
Expand All @@ -121,6 +151,7 @@ def _parse_{{ property.python_name }}(data: object) -> {{ property.get_type_stri

{% macro transform(property, source, destination, declare_type=True) %}
{% set ns = namespace(contains_properties_without_transform = false, contains_modified_properties = not property.required, has_if = false) %}
{{ _check_enum_array_type(property, source) }}
{% if declare_type %}{{ destination }}: {{ property.get_type_string(json=True) }}{% endif %}

{% if not property.required %}
Expand Down
7 changes: 6 additions & 1 deletion scripts/auto-generate-api-client.sh
Original file line number Diff line number Diff line change
Expand Up @@ -17,11 +17,16 @@ echo "$HOME_DIR"

cd "$HOME_DIR"

GENERATION_DIR=$(mktemp -d "${TMPDIR:-/tmp}/splunk-ao-open-enums.XXXXXX")
GENERATION_SPEC="$GENERATION_DIR/openapi.json"
trap 'rm -r "$GENERATION_DIR"' EXIT
poetry run python "$HOME_DIR/prepare_open_enums.py" "$OPENAPI_SPEC_PATH" "$GENERATION_SPEC"

# Backup the current output path
mv "$OUTPUT_PATH" "$OUTPUT_PATH"_backup

# Run the OpenAPI Python client generator
poetry run openapi-python-client generate --meta none --path "$OPENAPI_SPEC_PATH" --output-path "$OUTPUT_PATH" --custom-template-path ../codegen_templates --config "$CONFIG_PATH" --overwrite
poetry run openapi-python-client generate --meta none --path "$GENERATION_SPEC" --output-path "$OUTPUT_PATH" --custom-template-path ../codegen_templates --config "$CONFIG_PATH" --overwrite

# Remove the backup directory
rm -r "$OUTPUT_PATH"_backup
Expand Down
Loading
Loading