@@ -9793,53 +9793,30 @@ components:
97939793 - data_source
97949794 - query
97959795 type: object
9796- MonitorFormulaAndFunctionDataJobsMonitorOptions:
9797- description: Monitor configuration options for data jobs queries.
9798- properties:
9799- alert_on_every_failure:
9800- description: |-
9801- Alert separately for each failed run instead of on the count of trailing failures.
9802- Only supported for `failed_runs` monitors with the `aastra` query dialect. It requires
9803- a `> 0` threshold with no warning threshold, an evaluation window of at most two days,
9804- and renotification disabled.
9805- example: true
9806- type: boolean
9807- type: object
98089796 MonitorFormulaAndFunctionDataJobsQueryDefinition:
98099797 description: A formula and functions data jobs query.
98109798 properties:
98119799 job_type:
98129800 description: |-
98139801 The type of job being monitored. Valid values include:
9814- `databricks.job`, `spark.application`, `airflow.dag`, `dbt.job`, `glue.job`, `adf.pipeline`.
9815- The `aastra` dialect also accepts `airflow .job`, `dbt.model`, and `dbt.test`.
9802+ `databricks.job`, `spark.application`, `airflow.dag`,
9803+ `dbt .job`, `dbt.model`, `dbt.test`, `glue.job `.
98169804 Custom job types are supported with the `custom.ol.` prefix.
98179805 example: "databricks.job"
98189806 type: string
98199807 jobs_query:
9820- description: |-
9821- Filter expression used to select the jobs to monitor. Its syntax depends on `query_dialect`:
9822- a metric tag filter such as `job_name:smoke*` for `metric`, or an Aastra entity search
9823- such as ``search for databricks_job where `@team:data-eng` `` for `aastra`.
9808+ description: Filter expression used to select the jobs to monitor.
98249809 example: "job_name:smoke*"
98259810 type: string
9826- monitor_options:
9827- $ref: "#/components/schemas/MonitorFormulaAndFunctionDataJobsMonitorOptions"
98289811 name:
98299812 description: Name of the query for use in formulas. Must be `run_query`.
98309813 example: "run_query"
98319814 type: string
98329815 query_dialect:
98339816 description: |-
9834- Query dialect for data jobs queries. Valid values are `metric` and `aastra` .
9817+ Query dialect for data jobs queries. Currently only `metric` is supported .
98359818 example: "metric"
98369819 type: string
9837- run_filter:
9838- description: |-
9839- Optional filter on individual runs, using metrics filter syntax.
9840- Only supported with the `aastra` query dialect.
9841- example: "env:prod"
9842- type: string
98439820 required:
98449821 - name
98459822 - jobs_query
@@ -9854,18 +9831,6 @@ components:
98549831 type: string
98559832 x-enum-varnames:
98569833 - DATA_QUALITY_METRICS
9857- MonitorFormulaAndFunctionDataQualityDetectionType:
9858- description: |-
9859- How a `data-jobs alert` job duration monitor detects a breach. Must match the
9860- `.detection()` call in the monitor query. Required for job duration monitors.
9861- enum:
9862- - static
9863- - anomaly
9864- example: "anomaly"
9865- type: string
9866- x-enum-varnames:
9867- - STATIC
9868- - ANOMALY
98699834 MonitorFormulaAndFunctionDataQualityDiffFunction:
98709835 description: |-
98719836 Function applied to the measure before it is compared against the predicted bounds.
@@ -9926,8 +9891,7 @@ components:
99269891 The data quality measure to query. Common values include:
99279892 `bytes`, `cardinality`, `custom`, `freshness`, `max`, `mean`, `min`,
99289893 `nullness`, `percent_negative`, `percent_zero`, `row_count`, `stddev`,
9929- `sum`, `uniqueness`. `data-jobs alert` job duration monitors use `duration`.
9930- Additional values may be supported.
9894+ `sum`, `uniqueness`. Additional values may be supported.
99319895 example: "row_count"
99329896 type: string
99339897 MonitorFormulaAndFunctionDataQualityModelBoundsOverride:
@@ -9997,14 +9961,6 @@ components:
99979961 MonitorFormulaAndFunctionDataQualityMonitorOptions:
99989962 description: Monitor configuration options for data quality queries.
99999963 properties:
10000- consecutive_runs:
10001- description: |-
10002- Number of consecutive runs that must breach before the monitor alerts.
10003- Only supported by `data-jobs alert` job duration monitors. Defaults to `1`.
10004- example: 1
10005- format: int64
10006- minimum: 1
10007- type: integer
100089964 crontab_override:
100099965 description: Crontab expression to override the default schedule.
100109966 example: "* * * 10"
@@ -10017,8 +9973,6 @@ components:
100179973 description: Custom WHERE clause for the query.
100189974 example: "USER_ID = 123"
100199975 type: string
10020- detection_type:
10021- $ref: "#/components/schemas/MonitorFormulaAndFunctionDataQualityDetectionType"
100229976 group_by_columns:
100239977 description: Columns to group results by.
100249978 example: ["col1", "col2"]
@@ -10066,12 +10020,6 @@ components:
1006610020 description: Name of the query for use in formulas.
1006710021 example: "query1"
1006810022 type: string
10069- run_filter:
10070- description: |-
10071- Optional filter on individual runs, using metrics filter syntax.
10072- Only supported by `data-jobs alert` job duration monitors.
10073- example: "env:prod"
10074- type: string
1007510023 schema_version:
1007610024 description: Schema version for the data quality query.
1007710025 example: "0.0.1"
@@ -37331,22 +37279,6 @@ paths:
3733137279 - `time_window` #m (between 1 and 2880), #h (between 1 and 48).
3733237280 - `operator` `<`, `<=`, `>`, `>=`, `==`, or `!=`.
3733337281 - `#` an integer or decimal number used to set the threshold.
37334-
37335- **Data Jobs Alert Query**
37336-
37337- Job status example: `formula("failed_runs(run_query)").by(group_keys).last(time_window) > #`
37338-
37339- Job duration examples: `formula("job_duration(run_query)").detection(static).by(group_keys) > duration`
37340- and `formula("job_duration(run_query)").detection(anomaly).by(group_keys) > 0`
37341-
37342- - `run_query`: The name of the single query variable in `options.variables`. Job status monitors
37343- use a data jobs query; job duration monitors use a data quality query with the `duration` measure.
37344- - `group_keys`: The tags to group by, such as `entity_id,name,workspace_name`.
37345- - `time_window`: Number of minutes (#m), hours (#h), or days (#d), up to `10d`.
37346- - `#`: The number of trailing failed runs to alert above, as a non-negative integer.
37347- - `duration`: The duration to alert above, such as `2h`, `30m`, or a number of seconds.
37348- - `options.thresholds.critical`: Must equal the query threshold, in seconds for job duration.
37349- Anomaly job duration monitors omit `options.thresholds`.
3735037282 operationId: CreateMonitor
3735137283 requestBody:
3735237284 content:
0 commit comments