Skip to content

EvaluatingApi.md

Latest commit

 

History

History
348 lines (236 loc) · 12.3 KB

EvaluatingApi.md

File metadata and controls

348 lines (236 loc) · 12.3 KB
Raw

cms_client.EvaluatingApi

All URIs are relative to http://localhost

Method HTTP request Description
get_annotation_stats POST /annotation-stats Get Annotation Stats
get_concatenated_trainer_exports POST /concat_trainer_exports Get Concatenated Trainer Exports
get_inter_annotator_agreement_scores POST /iaa-scores Get Inter Annotator Agreement Scores
get_sanity_check_with_trainer_export POST /sanity-check Get Sanity Check With Trainer Export

get_annotation_stats

get_annotation_stats(trainer_export, tracking_id=tracking_id)

Get Annotation Stats

Get annotation stats of trainer export files

Example

  • OAuth Authentication (OAuth2PasswordBearer):
  • Api Key Authentication (APIKeyCookie):
import cms_client
from cms_client.rest import ApiException
from pprint import pprint

# Defining the host is optional and defaults to http://localhost
# See configuration.py for a list of all supported configuration parameters.
configuration = cms_client.Configuration(
    host = "http://localhost"
)

# The client must configure the authentication and authorization parameters
# in accordance with the API server security policy.
# Examples for each auth method are provided below, use the example that
# satisfies your auth use case.

configuration.access_token = os.environ["ACCESS_TOKEN"]

# Configure API key authorization: APIKeyCookie
configuration.api_key['APIKeyCookie'] = os.environ["API_KEY"]

# Uncomment below to setup prefix (e.g. Bearer) for API key, if needed
# configuration.api_key_prefix['APIKeyCookie'] = 'Bearer'

# Enter a context with an instance of the API client
with cms_client.ApiClient(configuration) as api_client:
    # Create an instance of the API class
    api_instance = cms_client.EvaluatingApi(api_client)
    trainer_export = None # List[bytearray] | One or more trainer export files to be uploaded
    tracking_id = 'tracking_id_example' # str | The tracking ID of the requested task (optional)

    try:
        # Get Annotation Stats
        api_instance.get_annotation_stats(trainer_export, tracking_id=tracking_id)
    except Exception as e:
        print("Exception when calling EvaluatingApi->get_annotation_stats: %s\n" % e)

Parameters

Name Type Description Notes
trainer_export List[bytearray] One or more trainer export files to be uploaded
tracking_id str The tracking ID of the requested task [optional]

Return type

void (empty response body)

Authorization

OAuth2PasswordBearer, APIKeyCookie

HTTP request headers

  • Content-Type: multipart/form-data
  • Accept: application/json

HTTP response details

Status code Description Response headers
200 Successful Response -
422 Validation Error -

[Back to top] [Back to API list] [Back to Model list] [Back to README]

get_concatenated_trainer_exports

object get_concatenated_trainer_exports(trainer_export, tracking_id=tracking_id)

Get Concatenated Trainer Exports

Concatenate multiple trainer export files into a single file for download

Example

  • OAuth Authentication (OAuth2PasswordBearer):
  • Api Key Authentication (APIKeyCookie):
import cms_client
from cms_client.rest import ApiException
from pprint import pprint

# Defining the host is optional and defaults to http://localhost
# See configuration.py for a list of all supported configuration parameters.
configuration = cms_client.Configuration(
    host = "http://localhost"
)

# The client must configure the authentication and authorization parameters
# in accordance with the API server security policy.
# Examples for each auth method are provided below, use the example that
# satisfies your auth use case.

configuration.access_token = os.environ["ACCESS_TOKEN"]

# Configure API key authorization: APIKeyCookie
configuration.api_key['APIKeyCookie'] = os.environ["API_KEY"]

# Uncomment below to setup prefix (e.g. Bearer) for API key, if needed
# configuration.api_key_prefix['APIKeyCookie'] = 'Bearer'

# Enter a context with an instance of the API client
with cms_client.ApiClient(configuration) as api_client:
    # Create an instance of the API class
    api_instance = cms_client.EvaluatingApi(api_client)
    trainer_export = None # List[bytearray] | A list of trainer export files to be concatenated
    tracking_id = 'tracking_id_example' # str | The tracking ID of the requested task (optional)

    try:
        # Get Concatenated Trainer Exports
        api_response = api_instance.get_concatenated_trainer_exports(trainer_export, tracking_id=tracking_id)
        print("The response of EvaluatingApi->get_concatenated_trainer_exports:\n")
        pprint(api_response)
    except Exception as e:
        print("Exception when calling EvaluatingApi->get_concatenated_trainer_exports: %s\n" % e)

Parameters

Name Type Description Notes
trainer_export List[bytearray] A list of trainer export files to be concatenated
tracking_id str The tracking ID of the requested task [optional]

Return type

object

Authorization

OAuth2PasswordBearer, APIKeyCookie

HTTP request headers

  • Content-Type: multipart/form-data
  • Accept: application/json

HTTP response details

Status code Description Response headers
200 Successful Response -
422 Validation Error -

[Back to top] [Back to API list] [Back to Model list] [Back to README]

get_inter_annotator_agreement_scores

get_inter_annotator_agreement_scores(annotator_a_project_id, annotator_b_project_id, scope, trainer_export, tracking_id=tracking_id)

Get Inter Annotator Agreement Scores

Calculate inter annotator agreement scores between two projects

Example

  • OAuth Authentication (OAuth2PasswordBearer):
  • Api Key Authentication (APIKeyCookie):
import cms_client
from cms_client.rest import ApiException
from pprint import pprint

# Defining the host is optional and defaults to http://localhost
# See configuration.py for a list of all supported configuration parameters.
configuration = cms_client.Configuration(
    host = "http://localhost"
)

# The client must configure the authentication and authorization parameters
# in accordance with the API server security policy.
# Examples for each auth method are provided below, use the example that
# satisfies your auth use case.

configuration.access_token = os.environ["ACCESS_TOKEN"]

# Configure API key authorization: APIKeyCookie
configuration.api_key['APIKeyCookie'] = os.environ["API_KEY"]

# Uncomment below to setup prefix (e.g. Bearer) for API key, if needed
# configuration.api_key_prefix['APIKeyCookie'] = 'Bearer'

# Enter a context with an instance of the API client
with cms_client.ApiClient(configuration) as api_client:
    # Create an instance of the API class
    api_instance = cms_client.EvaluatingApi(api_client)
    annotator_a_project_id = 56 # int | The project ID from one annotator
    annotator_b_project_id = 56 # int | The project ID from another annotator
    scope = 'scope_example' # str | The scope for which the score will be calculated, e.g., per_concept, per_document or per_span
    trainer_export = None # List[bytearray] | A list of trainer export files to be uploaded
    tracking_id = 'tracking_id_example' # str | The tracking ID of the requested task (optional)

    try:
        # Get Inter Annotator Agreement Scores
        api_instance.get_inter_annotator_agreement_scores(annotator_a_project_id, annotator_b_project_id, scope, trainer_export, tracking_id=tracking_id)
    except Exception as e:
        print("Exception when calling EvaluatingApi->get_inter_annotator_agreement_scores: %s\n" % e)

Parameters

Name Type Description Notes
annotator_a_project_id int The project ID from one annotator
annotator_b_project_id int The project ID from another annotator
scope str The scope for which the score will be calculated, e.g., per_concept, per_document or per_span
trainer_export List[bytearray] A list of trainer export files to be uploaded
tracking_id str The tracking ID of the requested task [optional]

Return type

void (empty response body)

Authorization

OAuth2PasswordBearer, APIKeyCookie

HTTP request headers

  • Content-Type: multipart/form-data
  • Accept: application/json

HTTP response details

Status code Description Response headers
200 Successful Response -
422 Validation Error -

[Back to top] [Back to API list] [Back to Model list] [Back to README]

get_sanity_check_with_trainer_export

get_sanity_check_with_trainer_export(trainer_export, tracking_id=tracking_id)

Get Sanity Check With Trainer Export

Sanity check the model being served with a trainer export

Example

  • OAuth Authentication (OAuth2PasswordBearer):
  • Api Key Authentication (APIKeyCookie):
import cms_client
from cms_client.rest import ApiException
from pprint import pprint

# Defining the host is optional and defaults to http://localhost
# See configuration.py for a list of all supported configuration parameters.
configuration = cms_client.Configuration(
    host = "http://localhost"
)

# The client must configure the authentication and authorization parameters
# in accordance with the API server security policy.
# Examples for each auth method are provided below, use the example that
# satisfies your auth use case.

configuration.access_token = os.environ["ACCESS_TOKEN"]

# Configure API key authorization: APIKeyCookie
configuration.api_key['APIKeyCookie'] = os.environ["API_KEY"]

# Uncomment below to setup prefix (e.g. Bearer) for API key, if needed
# configuration.api_key_prefix['APIKeyCookie'] = 'Bearer'

# Enter a context with an instance of the API client
with cms_client.ApiClient(configuration) as api_client:
    # Create an instance of the API class
    api_instance = cms_client.EvaluatingApi(api_client)
    trainer_export = None # List[bytearray] | One or more trainer export files to be uploaded
    tracking_id = 'tracking_id_example' # str | The tracking ID of the requested task (optional)

    try:
        # Get Sanity Check With Trainer Export
        api_instance.get_sanity_check_with_trainer_export(trainer_export, tracking_id=tracking_id)
    except Exception as e:
        print("Exception when calling EvaluatingApi->get_sanity_check_with_trainer_export: %s\n" % e)

Parameters

Name Type Description Notes
trainer_export List[bytearray] One or more trainer export files to be uploaded
tracking_id str The tracking ID of the requested task [optional]

Return type

void (empty response body)

Authorization

OAuth2PasswordBearer, APIKeyCookie

HTTP request headers

  • Content-Type: multipart/form-data
  • Accept: application/json

HTTP response details

Status code Description Response headers
200 Successful Response -
422 Validation Error -

[Back to top] [Back to API list] [Back to Model list] [Back to README]