diff --git a/docs/user_guides/fs/feature_view/deployment.md b/docs/user_guides/fs/feature_view/deployment.md new file mode 100644 index 0000000000..085753b351 --- /dev/null +++ b/docs/user_guides/fs/feature_view/deployment.md @@ -0,0 +1,190 @@ +--- +description: Documentation on how to deploy a feature view as an online endpoint that returns transformed feature vectors. +--- + +# How To Deploy A Feature View { #feature-view-deployment } + +## Introduction + +In this guide, you will learn how to serve a feature view without a model. +A feature view deployment answers a prediction-style request with the transformed feature vector a model would receive. +It uses the same request contract, feature lookup, transformations, logging, and monitoring as a model deployment served by the default predictor. +See the [Deployment Schema Guide][deployment-schema] for the request contract and the error codes, which are shared with model deployments. + +Use it to serve features to a model that runs outside Hopsworks, to test transformations online before a model exists, or to give a feature vector API to another team. + +!!! warning "Serving identity" + The deployment looks up features as the project's serving identity, not as the caller. + Anyone allowed to call the deployment can obtain the transformed features of any entity the feature view can serve. + +## Code + +### Step 1: Connect to Hopsworks + +=== "Python" + + ```python + import hopsworks + + + project = hopsworks.login() + + fs = project.get_feature_store() + ``` + +### Step 2: Pin a training dataset + +Model-dependent transformations that need statistics, such as `min_max_scaler`, take them from a training dataset. +The deployment uses the training dataset you last read or created in this session, or the one you pass to `deploy()`. + +=== "Python" + + ```python + feature_view = fs.get_feature_view("transactions", version=1) + + # reading or creating a training dataset records it as the one to serve with + X_train, X_test, y_train, y_test = feature_view.train_test_split(test_size=0.2) + ``` + +If the feature view has such a transformation and no training dataset was read or created, `deploy()` refuses and names the transformation, because its statistics cannot be computed. + +### Step 3: Deploy the feature view + +=== "Python" + + ```python + deployment = feature_view.deploy( + name="transactionsfv", + passed_features=["amount"], # features the client sends with each request + ) + deployment.start(await_running=600) + ``` + +The deployment name defaults to the feature view name and version without special characters. +The client publishes the deployment schema before the deployment is created, so `deployment.schema` describes the request immediately: + +=== "Python" + + ```python + deployment.schema.describe() + print(deployment.schema.names) # the order of positional rows + ``` + +### Step 4: Request feature vectors + +Each row carries the serving keys, the passed features, the request parameters of on-demand transformations, and any extra logging columns. +The response carries one transformed vector per row and the column names. + +=== "Python" + + ```python + response = deployment.predict( + inputs=[{"cc_num": 4473593503484549, "amount": 12.5}] + ) + print(response["columns"]) # ["amount_scaled", "age_days", ...] + print(response["predictions"]) # [[0.31, -1.2, ...]] + ``` + +Rows can also be arrays in `deployment.schema.names` order. +When every stored feature of the view is passed, the schema has no serving keys and the deployment only computes the on-demand features and applies the model-dependent transformations; see [Deployments without lookups][deployment-schema-no-lookup]. +Invalid rows are refused before any feature is read; see [Errors][deployment-schema-errors]. + +### Step 5: Inspect the deployment + +=== "Python" + + ```python + print(deployment.has_feature_view) # True + print(deployment.feature_view_name, deployment.feature_view_version) + print(deployment.training_dataset_version) # the pinned version + feature_view = deployment.get_feature_view() # the FeatureView object + print(deployment.get_model()) # None + ``` + +## Feature logging + +When logging is enabled on the feature view, every request is logged with the untransformed and transformed features, the request id, the training dataset version, and the reserved deployment columns `deployment_name`, `deployment_version`, `deployment_schema_id`, and `request_row`, when the logging feature group declares them. +The model columns of the log are null, because there is no model. +See [Feature logging in the Deployment Schema Guide][deployment-schema-feature-logging] for how requests are logged and for the reserved columns. + +=== "Python" + + ```python + feature_view.enable_logging( + extra_log_columns=[ + {"name": "deployment_name", "type": "string"}, + {"name": "deployment_version", "type": "int"}, + {"name": "deployment_schema_id", "type": "string"}, + {"name": "request_row", "type": "int"}, + ] + ) + deployment = feature_view.deploy(name="transactionsfv", passed_features=["amount"]) + ``` + +## Feature monitoring + +A feature view deployment has no model, so `deployment.create_model_monitoring()` raises. +Use `deployment.create_feature_monitoring()`, which attaches a feature monitoring configuration to the logging feature group of the view: + +=== "Python" + + ```python + config = ( + deployment.create_feature_monitoring(name="amount_drift") + .with_detection_window(time_offset="1d", window_length="1d") + .with_reference_window(time_offset="8d", window_length="7d") + .compare_on(metric="MEAN", threshold=10.0, feature_name="amount") + .save() + ) + deployment.get_monitoring_configs() + ``` + +A distribution comparison (`compare_on_distribution`) over rolling windows needs KLL statistics on the logging feature group, which it does not keep by default; enable them in the logging feature group's statistics configuration first. + +Two deployments of the same feature view version log to the same feature group; their rows are told apart by the reserved deployment columns, but a monitoring configuration sees both. +Deploy a separate version of the feature view when the statistics of one deployment must not include another's traffic. + +## Custom predictor script + +To post-process the vectors or to change how they are looked up, subclass the default predictor and pass the script to `deploy()`. +The script must end with the hand-over to the serving wrapper: + +=== "Python" + + ```python + from hsml.default_predictor import DefaultPredict, run_kserve_wrapper + + + class Predict(DefaultPredict): + def model_predict(self, feature_vectors): + # a feature view deployment has no model: return the vectors + return feature_vectors.round(3) + + + if __name__ == "__main__": + run_kserve_wrapper() + ``` + +Backends that support the `SERVING_SCRIPT_KIND=predictor` marker set by `deploy()` start the serving wrapper directly and never run the `__main__` block. +Older backends start the script with `python`, and the block hands over to the wrapper. +`deploy(script_file=...)` refuses a local script without it; a script already in HopsFS is not checked client-side. + +## REST access + +The deployment answers on the KServe V1 route of the Istio ingress, `/v1/models/:predict`, and through the Hopsworks REST API at `/project//inference/serving/:predict`. +See the [REST API Guide][hopsworks-model-serving-rest-api] for authentication and the base URL. +`deployment.get_inference_url()` returns the Istio URL, or `None` when the Istio ingress is not configured for external access. +Use the Hopsworks REST API path above when it does. + +## CLI + +```bash +hops fv deploy transactions --passed-feature amount +hops deployment schema transactionsfv --openapi +``` + +## API Reference + +`hsfs.feature_view.FeatureView.deploy` + +[`Deployment`][hsml.deployment.Deployment] diff --git a/docs/user_guides/mlops/registry/model_schema.md b/docs/user_guides/mlops/registry/model_schema.md index b1f21260e7..1867f88d82 100644 --- a/docs/user_guides/mlops/registry/model_schema.md +++ b/docs/user_guides/mlops/registry/model_schema.md @@ -6,10 +6,21 @@ description: Documentation on how to attach a model schema to a model. ## Introduction +!!! warning "Deprecated" + `ModelSchema` is deprecated and will be removed in a future release. + A model registered with `create_model(feature_view=...)` gets its input and output schema from the feature view's training dataset, and a deployment describes its requests with the [deployment schema][deployment-schema]. + The default predictor still reads a legacy model schema to select the model's input columns; a `DefaultPredict` subclass that overrides `model_predict` replaces that. + For a model without a feature view, name its input columns with `passed_features=` on `deploy()`. + In this guide you will learn how to attach a model schema to your model. A model schema, describes the type and shape of inputs and outputs (predictions) for your model. Attaching a model schema to your model will give other users a better understanding of what data it expects. +!!! info "Model schema and deployment schema" + A model registered with `feature_view=` gets its model schema inferred from the feature view's training dataset schema when it is saved. + The default predictor checks at pod start that every model input column is served by the feature view, and the deployment schema describes what clients send. + See the [Deployment Schema Guide][deployment-schema]. + ## Code ### Step 1: Connect to Hopsworks diff --git a/docs/user_guides/mlops/serving/deployment-schema.md b/docs/user_guides/mlops/serving/deployment-schema.md new file mode 100644 index 0000000000..d28ef0167a --- /dev/null +++ b/docs/user_guides/mlops/serving/deployment-schema.md @@ -0,0 +1,368 @@ +--- +description: Documentation on the deployment schema, the request contract of a model deployment, and the default predictor that serves a model without a predictor script. +--- + +# How To Use A Deployment Schema { #deployment-schema } + +## Introduction + +In this guide, you will learn how a deployment describes the prediction requests it accepts, how clients read that contract, and how to serve a model without writing a predictor script. + +A deployment schema lists the fields a client sends with each request, with their types, nullability, and order, plus the shape of the response. +It is inferred from the feature view the model was registered with: the serving keys, the features you pass with the request, the request parameters of on-demand transformations, and the extra columns of feature logging. +It is published as a JSON Schema and an OpenAPI document, so clients in any language can validate requests before sending them. +Every REST V1 request is validated in the pod before any predictor code runs, and rejected with a structured error when it does not match. +Enforcement covers the KServe REST V1 protocol only: a gRPC deployment is served without it, and the pod logs a warning at startup. + +The **default predictor** is the library class that serves such a deployment: it looks up and transforms the features by serving key, runs the model, and logs the request when the feature view has logging enabled. +A [feature view can be deployed on its own][feature-view-deployment] with the same class and the same contract, returning the transformed feature vector instead of a prediction. + +## Code + +### Step 1: Connect to Hopsworks + +=== "Python" + + ```python + import hopsworks + + + project = hopsworks.login() + + fs = project.get_feature_store() + mr = project.get_model_registry() + ``` + +### Step 2: Register the model with its feature view + +Register the model with `feature_view=` so the deployment knows where its features come from. +The training dataset version is taken from the training dataset you last read or created with that feature view in this session, and the model schema is inferred from the feature view's training dataset schema. + +=== "Python" + + ```python + feature_view = fs.get_feature_view("transactions", version=1) + X_train, X_test, y_train, y_test = feature_view.train_test_split(test_size=0.2) + + # ... train and pickle the model into model_dir ... + + model = mr.python.create_model( + name="fraud", + feature_view=feature_view, + ) + model.save("model_dir") + ``` + +The default predictor loads a single `.pkl`, `.pickle`, or `.joblib` file from the model directory. + +### Step 3: Deploy without a predictor script + +=== "Python" + + ```python + deployment = model.deploy( + name="fraud", + passed_features=["amount"], # features the client sends with each request + ) + deployment.start(await_running=600) + ``` + +The default predictor is used when the model is a Python model registered with a feature view, no `script_file` or transformer is given, and the deployment uses KServe over REST. +Pass `default_predictor=True` to force it, for instance for a scikit-learn model, or `default_predictor=False` to keep the plain model server. + +At pod start the predictor checks that every input column of the model schema is served by the feature view, with a compatible type. +A mismatch fails the deployment with the offending columns in `deployment.get_logs()`, instead of serving wrong predictions. + +If the feature view has a transformation that needs training dataset statistics, such as `min_max_scaler`, and the model has no training dataset version, `deploy()` refuses and names the transformation. + +### Step 4: Read the contract + +=== "Python" + + ```python + schema = deployment.schema + schema.describe() # one row per field: group, name, type, nullable + + print(schema.names) # the order of positional rows + print(schema.unresolved) # fields whose type is not known, such as request parameters + + json_schema = schema.to_json_schema() # {"request": ..., "response": ...} + openapi = schema.to_openapi("fraud", url=deployment.get_inference_url()) + ``` + +Fields belong to one of four groups: + +| group | source | required | +| --- | --- | --- | +| serving keys | the feature view's serving keys, present only when a feature is looked up | yes, non-null | +| passed features | `passed_features=` | yes, nullable when the feature is | +| request parameters | arguments of on-demand transformations that are not features | yes | +| extra logging features | extra columns of the feature view's logging, minus the reserved ones | no | + +Request parameter types come from the annotations of the transformation function's arguments: `def amount_ratio(amount: float, budget: float)` records `budget` as `double`. +An unannotated argument is reported as unresolved, and any value is accepted for it unless you refine the schema (Step 6). + +### Step 5: Send requests + +Rows are objects keyed by field name, or arrays in `schema.names` order. +A request holds one to `schema.max_batch_rows` rows (default 512). +The limit is part of the schema, so the published JSON Schema, the client, the transformer, and the predictor all apply the same one. +Set `SERVING_MAX_BATCH_ROWS` in `env_vars=` to change it; the change publishes a new schema id. +A request carries either `instances` or `inputs`, never both. + +=== "Python" + + ```python + deployment.predict(inputs=[{"cc_num": 4473593503484549, "amount": 12.5}]) + deployment.predict(inputs=[[4473593503484549, 12.5]]) + ``` + +The client validates the rows against the schema before sending and raises `ModelServingException` with every problem found; pass `validate=False` to skip that and let the pod answer. +A batch is all objects or all arrays, in the published JSON Schema as in the pod. + +=== "curl" + + ```bash + # INFERENCE_URL is deployment.get_inference_url(), also shown on the deployment page + curl -X POST "$INFERENCE_URL" \ + -H "Authorization: ApiKey $API_KEY" -H "Content-Type: application/json" \ + -d '{"instances": [{"cc_num": 4473593503484549, "amount": 12.5}]}' + ``` + +### Step 6: Refine or replace the schema + +Pass `schema=` to `deploy()` to refine the inferred schema, for instance to give a request parameter a type. +A refinement keeps the inferred fields; adding or removing one is refused. + +=== "Python" + + ```python + from hsml.deployment_schema import DeploymentSchema + + deployment = model.deploy(name="fraud", passed_features=["amount"]) + inferred = deployment.schema + refined = DeploymentSchema( + serving_keys=inferred.serving_keys, + passed_features=inferred.passed_features, + request_parameters=[{"name": "rate", "type": "double", "nullable": False}], + extra_logging_features=inferred.extra_logging_features, + feature_view=inferred.feature_view, + training_dataset_version=inferred.training_dataset_version, + output=inferred.output, + ) + deployment.schema = refined + deployment.save() + ``` + +A custom predictor script deployed with `schema=` (or `passed_features=`) gets the same validation in the pod, before its `predict()` is called, for REST V1 requests. +The client validates REST requests only, so a gRPC client is not checked on either side. + +### Step 7: Republish after changing the feature view + +The served contract does not change when you enable logging, add logging columns, or change the feature view. +Re-infer and save to publish the new contract as a new revision: + +=== "Python" + + ```python + deployment.reinfer_schema() + deployment.save() + ``` + +## Revisions { #deployment-schema-revisions } + +Every schema is content-addressed: `deployment.schema_id` is a hash of its content, and equal schemas have equal ids. +The client writes the schema and its JSON Schema and OpenAPI renderings to `/Deployments//resources/schema/.*` before the deployment is created or updated, and records the id in the environment variable `SERVING_SCHEMA_ID` of the predictor and of the transformer, when there is one. +`SERVING_SCHEMA_ENFORCER` on the same components records which of the two validates requests for that revision. +The files are never modified or removed while the deployment exists. + +A deployment revision therefore always enforces the exact schema it was created with, and answers discovery consistently with what it enforces. +The pod also takes its model, feature view, and training dataset version from its own revision, and refuses to start when the schema was published for another training dataset version than the one it would serve. +Updating the schema rolls the instances: old pods keep the old contract until they are replaced. +Rolling back is `deployment.schema = previous_schema; deployment.save()`, which points the revision at a file that is still there. + +## Discovery for non-Python clients + +The Hopsworks REST API serves the three documents to any client with an API key that has the `SERVING` scope, so prediction access implies discovery access: + +```bash +SERVING_ID=$(curl -s -H "Authorization: ApiKey $API_KEY" \ + "https://$HOST/hopsworks-api/api/project/$PROJECT_ID/serving?name=fraud" | jq .id) + +curl -s -H "Authorization: ApiKey $API_KEY" \ + "https://$HOST/hopsworks-api/api/project/$PROJECT_ID/serving/$SERVING_ID/schema?format=openapi" +``` + +`format` is `schema` (default), `jsonschema`, or `openapi`. +`schemaId=` returns the documents of an earlier revision. +A deployment without a schema, or an unknown id, answers `404` with error code `240037`. + +## Type encoding + +The JSON Schema fragment, the accepted JSON values, and the encoding the Python client applies follow one table. + +| feature type | JSON Schema | accepted JSON | Python client sends | +| --- | --- | --- | --- | +| `tinyint`, `smallint`, `int` | `{"type": "integer"}` | integer | `int` | +| `bigint` | integer or decimal string | integer, or a decimal string for values beyond 2^53 | `int` | +| `float`, `double` | `{"type": "number"}` | finite number | `int`, `float` | +| `decimal(p,s)` | number or string | number, or decimal string | `Decimal` as string | +| `string`, `varchar(n)`, `char(n)` | `{"type": "string"}` | string | `str` | +| `boolean` | `{"type": "boolean"}` | boolean | `bool` | +| `timestamp` | RFC 3339 string or integer | RFC 3339 string, or epoch milliseconds | `datetime` as RFC 3339 UTC | +| `date` | date string or integer | `YYYY-MM-DD`, or days since epoch | `date` as `YYYY-MM-DD` | +| `binary` | base64 string | base64 string | `bytes` as base64 | +| `array` | array of `T` | array | list | +| `struct<...>` | object with exactly those fields | object | dict | +| `map` | object with values of `V` | object | dict | +| unresolved | `{}` | anything | unchanged | + +## Errors { #deployment-schema-errors } + +Errors raised by the default predictor and by the schema enforcement carry a structured `detail`: + +```json +{"detail": { + "code": "SCHEMA_VALIDATION", + "message": "Prediction request does not match the deployment schema of 'fraud'.", + "schema_id": "3f9a1c2b7d4e6f80", + "errors": [ + {"row": 0, "field": "amount", "reason": "missing"}, + {"row": 2, "field": "cc_num", "reason": "must not be null"} + ]}} +``` + +| status | code | when | +| --- | --- | --- | +| 400 | `SCHEMA_VALIDATION` | the request does not match the schema; `errors` names every row and field | +| 400 | `FEATURE_LOOKUP_FAILED` | the feature store rejected the lookup for a reason other than a missing entity | +| 404 | `ENTITY_NOT_FOUND` | at least one row's serving keys match no entity; the batch is rejected and `errors` names the rows | +| 413 | `BATCH_TOO_LARGE` | more than `SERVING_MAX_BATCH_ROWS` rows | +| 422 | `TRANSFORMATION_FAILED` | a transformation raised; `field` is the transformation name | +| 500 | `MODEL_FAILED` | the model raised | +| 500 | `CONTRACT_VIOLATION` | the pod produced a result that does not match the contract: a different number of vectors or predictions than rows, or other feature vector columns than published | +| 503 | `FEATURE_STORE_UNAVAILABLE` | the online store or the feature store API could not be reached | + +A request is all or nothing: either every row gets a prediction, in request order, or the whole request fails and no row is logged. +Error responses never include feature values or exception text: a failure names the exception type only, and `detail.request_id` carries the correlation id (the `x-request-id` header, or one generated for the request) under which the pod log holds the full error. +The Hopsworks REST inference proxy does not forward `x-request-id`; send it through the Istio URL when the id must be yours. +Responses with a 5xx status may be retried; a retry may read newer features and always produces another log row, so reuse the `x-request-id` header to tie the rows together. + +## Feature logging and monitoring { #deployment-schema-feature-logging } + +When the feature view has logging enabled, the default predictor logs every request with the untransformed and transformed features, the predictions, the request id, the training dataset version, and the model name and version, so `deployment.create_model_monitoring()` works with no extra code. + +Declare the reserved extra logging columns on the feature view and the predictor fills them, which tells deployments and revisions apart in the log: + +| column | type | value | +| --- | --- | --- | +| `deployment_name` | `string` | the deployment name | +| `deployment_version` | `int` | the deployment version | +| `deployment_schema_id` | `string` | the schema id of the revision that served the request | +| `request_row` | `int` | the row's index in its request | + +Any other extra logging column becomes a request field that clients may send. + +Logging is asynchronous: the request is answered immediately, the logging frame is built on a background thread of the predictor, and the rows are handed to the pod's inference-logger sidecar from there. +A logging failure never fails a request, and both buffers are bounded by `FEATURE_LOGGER_QUEUE_SIZE` rows (default 1000: rows waiting for the predictor's logging thread, and rows waiting in the sidecar logger); beyond it a request's rows are dropped and counted, so a slow logger cannot exhaust the pod's memory. +Both buffers count rows rather than requests, because one request carries a whole batch. +The predictor's own backlog admits one request whatever its size when it is empty, so a deployment whose batches are larger than the buffer logs instead of dropping every request; its peak is then that single batch, itself capped by the schema's batch limit. +There is no synchronous mode: a prediction is never delayed by its log write. + +## Custom predictor scripts + +Subclass the default predictor when the model needs another loader or the predictions need post-processing, and deploy with `default_predictor=True` so the schema is still inferred: + +=== "Python" + + ```python + from hsml.default_predictor import DefaultPredict + + + class Predict(DefaultPredict): + def load_model(self, model_files_path): ... + + def model_predict(self, feature_vectors): + return self.model.predict_proba(feature_vectors[self.model_input_columns]) + ``` + +The serving wrapper imports a model deployment's script itself, so the script needs no `__main__` block. +Only a [feature view deployment][feature-view-deployment] script, which may be started as a plain script, hands over to the wrapper. +Any predictor script, subclass or not, is protected by the serving wrapper when the deployment carries a schema: invalid rows and oversize batches are refused before `predict()` runs. +With a transformer, the transformer validates the request, whether or not it implements `preprocess()`, and the predictor trusts the transformer's output. +Each pod reads that role from its own revision, so a predictor created before a transformer was added keeps validating until it is replaced. +This needs an inference environment built from a Hopsworks 5.1 or later base image; an older image serves the deployment without checking. + +## Deployments without lookups { #deployment-schema-no-lookup } + +Nothing is looked up in the online store when every stored feature of the view arrives with the request, or when the model has no feature view. +In both cases the schema has no serving keys. + +### Every feature passed + +Pass every non-label stored feature of the view in `passed_features=`; on-demand features are computed from the request parameters, so they are never looked up. +The view still computes the on-demand features and applies the model-dependent transformations with the pinned training dataset's statistics, and feature logging and monitoring work as for any other deployment. + +=== "Python" + + ```python + stored = [ + f.name + for f in feature_view.features + if not f.label and f.on_demand_transformation_function is None + ] + deployment = model.deploy(name="fraud_passed", passed_features=stored) + print(deployment.schema.serving_keys) # [] + ``` + +The same applies to `feature_view.deploy(passed_features=stored)`, which then returns the transformed vectors of the passed features. + +### A model without a feature view + +A Python model registered without a feature view deploys with the default predictor when you pass `default_predictor=True` and name its input columns with `passed_features=`, in the order the model expects. +Those columns are the whole request: the schema has no serving keys and no request parameters, nothing is looked up or transformed, and there is no feature logging or monitoring because there is no feature view. +The types are unresolved, so any JSON value is accepted for them; refine the schema with `schema=` to pin them down. + +=== "Python" + + ```python + model = mr.python.create_model(name="fraud_plain") + model.save("model_dir") + + deployment = model.deploy( + default_predictor=True, + passed_features=["amount", "age_days"], # the model's input columns, in its order + ) + deployment.schema.describe() # passed features only, types unresolved + deployment.predict(inputs=[{"amount": 12.5, "age_days": 41}]) + ``` + +## Access control + +Prediction through the Hopsworks REST API and through the Istio ingress requires an API key with the `SERVING` scope and the Data Owner or Data Scientist role in the project. +The pod looks up features as the project's serving identity, not as the caller. +Anyone allowed to call `:predict` can therefore obtain the transformed features of any entity the feature view can serve, and a feature view deployment returns those features directly. +Log rows contain feature values and are governed by the logging feature group's permissions. + +## Environment variables + +| variable | set by | meaning | +| --- | --- | --- | +| `SERVING_SCHEMA_ID` | the client | the schema the revision serves | +| `SERVING_FEATURE_VIEW_NAME`, `SERVING_FEATURE_VIEW_VERSION` | the client, feature view deployments | the feature view served | +| `SERVING_TRAINING_DATASET_VERSION` | the client, feature view deployments | the pinned training dataset | +| `SERVING_SCHEMA_ENFORCER` | the client | `predictor` or `transformer`: the component of the revision that validates requests | +| `SERVING_MAX_BATCH_ROWS` | you, through `env_vars=` | rows accepted per request, default 512; recorded in the schema at publication | +| `FEATURE_LOGGER_QUEUE_SIZE` | you, through `env_vars=` | rows the predictor's logging thread and the async logger each buffer before dropping, default 1000; a value that is not a positive integer is ignored | + +The `SERVING_*` names are reserved and refused in `env_vars=`, except `SERVING_MAX_BATCH_ROWS`. + +## API Reference + +`hsml.deployment_schema.DeploymentSchema` + +`hsml.default_predictor.DefaultPredict` + +[`Model.deploy`][hsml.model.Model.deploy] + +[`Deployment`][hsml.deployment.Deployment] diff --git a/docs/user_guides/mlops/serving/deployment.md b/docs/user_guides/mlops/serving/deployment.md index ace3a9239f..bd1ebc90ea 100644 --- a/docs/user_guides/mlops/serving/deployment.md +++ b/docs/user_guides/mlops/serving/deployment.md @@ -165,6 +165,57 @@ This will create a deployment for your model with default values. !!! info "Predictor script and server configuration file" You can provide a predictor script and a server configuration file directly in the `.deploy()` method using the `script_file` and `config_file` parameters. See the [Predictor Guide](predictor.md) for more details. +### Step 3b: Deploy a model with its feature view + +A Python model registered with `feature_view=` deploys without a predictor script. +The default predictor looks up and transforms the features by serving key, runs the model, logs the request when the feature view has logging enabled, and validates every request against the deployment schema, which the client infers from the feature view. +Clients send only the serving keys and the features named in `passed_features`. + +=== "Python" + + ```python + fs = project.get_feature_store() + feature_view = fs.get_feature_view("transactions", version=1) + + # register the trained model with the feature view it was trained on + fraud_model = mr.python.create_model(name="fraud", feature_view=feature_view) + fraud_model.save("model_dir") # one .pkl or .joblib file inside + + fraud_deployment = fraud_model.deploy( + name="fraud", + passed_features=["amount"], # sent by the client, not read from the online store + ) + fraud_deployment.start(await_running=600) + + fraud_deployment.schema.describe() # the request contract + fraud_deployment.predict(inputs=[{"cc_num": 4473593503484549, "amount": 12.5}]) + ``` + +See the [Deployment Schema Guide][deployment-schema] for the request contract, the error codes, feature logging, and custom predictor scripts that subclass the default predictor. + +### Step 3c: Deploy a feature view without a model + +A feature view deploys on its own and returns the transformed feature vectors, for callers that run the model elsewhere. +The deployment pins the training dataset whose statistics the transformations use. + +=== "Python" + + ```python + feature_view = fs.get_feature_view("transactions", version=1) + X_train, X_test, y_train, y_test = feature_view.train_test_split(test_size=0.2) + + fv_deployment = feature_view.deploy( + name="transactionsfv", + passed_features=["amount"], + ) + fv_deployment.start(await_running=600) + + response = fv_deployment.predict(inputs=[{"cc_num": 4473593503484549, "amount": 12.5}]) + print(response["columns"], response["predictions"]) + ``` + +See the [Feature View Deployment Guide][feature-view-deployment]. + ### API Reference [`ModelServing`][hsml.model_serving.ModelServing] @@ -196,6 +247,9 @@ Each deployment tracks its artifact files through a ==deployment version== — a Inside a model deployment, the local path to the artifact files is stored in the `ARTIFACT_FILES_PATH` environment variable (see [environment variables](../serving/predictor.md#environment-variables)). +Deployments with a deployment schema also keep the schema documents under `/Deployments//resources/schema/`, one set of files per schema content id. +These files are never modified, so a deployment revision always finds the schema it was created with; see [Revisions in the Deployment Schema Guide][deployment-schema-revisions]. + !!! warning All files under `/Models` and `/Deployments` are managed by Hopsworks. Manual changes to these files cannot be reverted and can have an impact on existing model deployments. diff --git a/docs/user_guides/mlops/serving/index.md b/docs/user_guides/mlops/serving/index.md index 2b0eda9853..2e693af0a4 100644 --- a/docs/user_guides/mlops/serving/index.md +++ b/docs/user_guides/mlops/serving/index.md @@ -9,6 +9,11 @@ Refer to the [Deployment Creation Guide](deployment.md) for step-by-step instruc !!! tip "Python deployments" If you want to deploy a Python script without a model artifact, see the [Python Deployments](../../projects/python-deployment/python-deployment.md) page. +### Deployment Schema and default predictor + +Describe the prediction request a deployment accepts, validate requests against it, and serve a model without writing a predictor script, see the [Deployment Schema Guide][deployment-schema]. +A feature view can be deployed on its own with the same contract, see the [Feature View Deployment Guide][feature-view-deployment]. + ### Predictor (KServe component) Predictors are responsible for running a model server that loads a trained model, handles inference requests and returns predictions, see the [Predictor Guide](predictor.md). diff --git a/docs/user_guides/mlops/serving/predictor.md b/docs/user_guides/mlops/serving/predictor.md index 522e63c5d2..f3564c14a4 100644 --- a/docs/user_guides/mlops/serving/predictor.md +++ b/docs/user_guides/mlops/serving/predictor.md @@ -162,9 +162,47 @@ Once you are done with the changes, click on `Create new deployment` at the bott ms = project.get_model_serving() ``` +### Step 2: Choose the predictor + +A Python model registered with `feature_view=` needs no predictor script when it takes a feature vector of the feature view and is stored as a single pickle or joblib file. +The library's default predictor validates the request against the deployment schema, looks up and transforms the features, runs the model, and logs the request when the feature view has logging enabled. + +=== "Python" + + ```python + my_model = mr.get_model("my_model", version=1) + + my_deployment = my_model.deploy(passed_features=["amount"]) + my_deployment.schema.describe() + ``` + +To customise it, subclass it in your own script and deploy with `default_predictor=True`, so the schema is still inferred: + +=== "Python" + + ```python + from hsml.default_predictor import DefaultPredict + + + class Predict(DefaultPredict): + def load_model(self, model_files_path): + # anything the default loader does not handle + ... + + def model_predict(self, feature_vectors): + return self.model.predict_proba(feature_vectors[self.model_input_columns]) + ``` + +The serving wrapper imports a model deployment's script itself, so the script needs no `__main__` block. +Only a [feature view deployment][feature-view-deployment] script, which may be started as a plain script, hands over to the wrapper. +See the [Deployment Schema Guide][deployment-schema] for the request contract, the error codes, and the feature logging guarantees. + +To serve the model with your own code instead, implement a predictor script (Steps 2.1 and 2.2). + ### Step 2.1 (Optional): Implement a predictor script -For Python model deployments, you need implement a predictor script that loads and serve your model. +For Python model deployments that the default predictor does not cover, implement a predictor script that loads and serves your model. +A script deployed with `schema=` or `passed_features=` still gets every request validated against the deployment schema by the serving wrapper before `predict()` is called. === "Predictor" diff --git a/docs/user_guides/mlops/serving/rest-api.md b/docs/user_guides/mlops/serving/rest-api.md index 550bdf1aee..382abdb7a5 100644 --- a/docs/user_guides/mlops/serving/rest-api.md +++ b/docs/user_guides/mlops/serving/rest-api.md @@ -81,6 +81,20 @@ For model deployments using Python, KServe sklearnserver, or TensorFlow Serving, inference_url = deployment.get_inference_url() ``` +### Deployment schema discovery + +Deployments that carry a deployment schema describe their request and response contract as JSON Schema and OpenAPI, and validate every request against it before any predictor code runs. +The documents are served by the Hopsworks REST API (not the Istio ingress), with an API key that has the `SERVING` scope: + +!!! example "" + **`GET https:///hopsworks-api/api/project//serving//schema?format=openapi`** + +`format` is `schema` (default), `jsonschema`, or `openapi`; `schemaId=` returns the contract of an earlier revision. +The serving id is the `id` field of `GET .../project//serving?name=`. +A deployment without a schema, or an unknown id, answers `404` with error code `240037`. +Feature view deployments answer on the same `/v1/models/:predict` route as Python model deployments. +See the [Deployment Schema Guide][deployment-schema]. + ### OpenAI-compatible ==vLLM deployments== provide an OpenAI API-compatible endpoint at `/v1/`, allowing you to send any standard OpenAI API request to the vLLM server. diff --git a/mkdocs.yml b/mkdocs.yml index 04adf56a22..90d711a913 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -109,6 +109,7 @@ nav: - Spines: user_guides/fs/feature_view/spine-query.md - Feature Monitoring: user_guides/fs/feature_view/feature_monitoring.md - Feature Logging: user_guides/fs/feature_view/feature_logging.md + - Deployment: user_guides/fs/feature_view/deployment.md - Vector Similarity Search: user_guides/fs/vector_similarity_search.md - Transformation Functions: user_guides/fs/transformation_functions.md - Compute Engines: user_guides/fs/compute_engines.md @@ -236,6 +237,7 @@ nav: - Model Deployment: - Deployment Creation: user_guides/mlops/serving/deployment.md - Deployment State: user_guides/mlops/serving/deployment-state.md + - Deployment Schema: user_guides/mlops/serving/deployment-schema.md - Predictor (KServe): user_guides/mlops/serving/predictor.md - Transformer (KServe): user_guides/mlops/serving/transformer.md - Inference Logger: user_guides/mlops/serving/inference-logger.md