Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 0 additions & 4 deletions docs/02_concepts/01_actor_lifecycle.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,3 @@ Before triggering a reboot, persist any essential state externally (e.g., to the
Update the status only when the user's understanding of progress changes - avoid frequent updates for every processed item. Detailed information should go to logs or storages (dataset, key-value store) instead.

<RunnableCodeBlock className="language-python" language="python">{StatusMessageExample}</RunnableCodeBlock>

## Conclusion

This page has presented the full Actor lifecycle: initialization, execution, error handling, rebooting, shutdown and status messages. You've seen how the SDK supports both context-based and manual control patterns. For deeper dives, explore the <ApiLink to="class/Actor">`Actor`</ApiLink> API reference, [guides](../guides/beautifulsoup-httpx), and [platform documentation](https://docs.apify.com/platform).
6 changes: 0 additions & 6 deletions docs/02_concepts/02_actor_input.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -37,9 +37,3 @@ Actors commonly receive a list of URLs to process via their input. The <ApiLink
The Apify platform supports [secret input fields](https://docs.apify.com/platform/actors/development/secret-input) that are encrypted before being stored. When you mark an input field as `"isSecret": true` in your Actor's [input schema](https://docs.apify.com/platform/actors/development/input-schema), the platform encrypts the value with the Actor's public key.

No special handling is needed in your code — when you call <ApiLink to="class/Actor#get_input">`Actor.get_input`</ApiLink>, encrypted fields are automatically decrypted using the Actor's private key, which is provided by the platform via environment variables. You receive the plaintext values directly.

## Conclusion

This page has shown how to read Actor input with <ApiLink to="class/Actor#get_input">`Actor.get_input`</ApiLink>, how to load URL sources with <ApiLink to="class/ApifyRequestList">`ApifyRequestList`</ApiLink>, and how secret input fields are decrypted automatically when you read them.

For more details on Actor input and how to define input schemas, see the [Actor input](https://docs.apify.com/platform/actors/running/input) and [input schema](https://docs.apify.com/platform/actors/development/input-schema) documentation on the Apify platform.
11 changes: 4 additions & 7 deletions docs/02_concepts/03_storages.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,10 @@ import KvsIteratingExample from '!!raw-loader!roa-loader!./code/03_kvs_iterating
import KvsPublicRecordExample from '!!raw-loader!roa-loader!./code/03_kvs_public_url.py';
import RqExample from '!!raw-loader!roa-loader!./code/03_rq.py';

The `Actor` class provides methods to work either with the default storages of the Actor, or with any other storage, named or unnamed.
The <ApiLink to="class/Actor">`Actor`</ApiLink> class provides methods to work either with the default storages of the Actor, or with any other storage, named or unnamed.

For details on storage types, see [Storage](https://docs.apify.com/platform/storage).


## Types of storages

Expand Down Expand Up @@ -186,9 +189,3 @@ To check if all the requests in the queue are handled, you can use the <ApiLink
## Storage clients

Behind the scenes, the SDK uses storage clients to communicate with the storage backend. The appropriate client is selected automatically based on the runtime environment. On the Apify platform, data is persisted via the Apify API, while local runs use the filesystem. For most use cases, you don't need to think about storage clients at all. To learn about the available implementations, how to switch between a single and shared request queue, or how to configure a custom client, see [Storage clients](./storage-clients). For a deeper look at how storage clients work internally, see the [Crawlee storage clients guide](https://crawlee.dev/python/docs/guides/storage-clients).

## Conclusion

This page has covered the three storage types (datasets, key-value stores, and request queues): how they are emulated on the local filesystem, how to open named and unnamed storages, and how to read from and write to each through the `Actor` shortcuts and the storage classes. When multiple Actor runs need to process the same request queue, choose between a [single and shared request queue](./storage-clients#single-vs-shared-request-queue).

For comprehensive information about storage on the Apify platform, see the [storage documentation](https://docs.apify.com/platform/storage), including the pages on [datasets](https://docs.apify.com/platform/storage/dataset), [key-value stores](https://docs.apify.com/platform/storage/key-value-store), and [request queues](https://docs.apify.com/platform/storage/request-queue).
10 changes: 3 additions & 7 deletions docs/02_concepts/04_actor_events.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
id: actor-events
title: Actor events & state persistence
title: Actor events and state persistence
description: Handle platform events like state persistence and graceful shutdown in your Actors.
---

Expand All @@ -12,6 +12,8 @@ import ApiLink from '@theme/ApiLink';

During its runtime, the Actor receives Actor events sent by the Apify platform or generated by the Apify SDK itself.

For details, see also [System events](https://docs.apify.com/platform/actors/development/programming-interface/system-events) and [State persistence](https://docs.apify.com/platform/actors/development/state-persistence).

## Event types

A listener can optionally receive a single argument, a Pydantic model with the event's data. The following table lists the events, the type of that data object, and when each event is emitted.
Expand Down Expand Up @@ -102,9 +104,3 @@ You can optionally specify a `key` (the key-value store key under which the stat
<RunnableCodeBlock className="language-python" language="python">
{UseStateExample}
</RunnableCodeBlock>

## Conclusion

This page has described the events emitted during a run (`SYSTEM_INFO`, `MIGRATING`, `ABORTING`, `PERSIST_STATE`, and `EXIT`): how to handle them with <ApiLink to="class/Actor#on">`Actor.on`</ApiLink>, and how to persist state automatically with <ApiLink to="class/Actor#use_state">`Actor.use_state`</ApiLink>.

For more details on platform events and state persistence, see the [system events](https://docs.apify.com/platform/actors/development/programming-interface/system-events) and [state persistence](https://docs.apify.com/platform/actors/development/state-persistence) documentation on the Apify platform.
6 changes: 0 additions & 6 deletions docs/02_concepts/05_proxy_management.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -121,9 +121,3 @@ Make sure you have the `httpx` library installed:
```bash
pip install httpx
```

## Conclusion

This page has explained how to manage proxies with the <ApiLink to="class/ProxyConfiguration">`ProxyConfiguration`</ApiLink> class: using Apify Proxy or your own servers, keeping sessions sticky across requests, configuring tiered proxy rotation, and feeding proxy settings from Actor input.

For full details on proxy configuration options, see the <ApiLink to="class/ProxyConfiguration">`ProxyConfiguration`</ApiLink> API reference and the [Apify Proxy documentation](https://docs.apify.com/proxy).
9 changes: 3 additions & 6 deletions docs/02_concepts/06_interacting_with_other_actors.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,9 @@ import ApiLink from '@theme/ApiLink';

The Apify SDK lets you start, call, and transform (metamorph) other Actors directly from your Actor code. This is useful for composing complex workflows from smaller, reusable Actors.

For the full list of methods for interacting with other Actors, see the <ApiLink to="class/Actor">`Actor`</ApiLink> class in the API reference. For details on running Actors and Actor tasks on the platform, see [Actors](https://docs.apify.com/platform/actors) and [Actor tasks](https://docs.apify.com/platform/actors/tasks).


## Actor start

The <ApiLink to="class/Actor#start">`Actor.start`</ApiLink> method starts another Actor on the Apify platform, and immediately returns the details of the started Actor run.
Expand Down Expand Up @@ -62,9 +65,3 @@ When you set `gracefully=True`, the platform sends `ABORTING` and `PERSIST_STATE
<RunnableCodeBlock className="language-python" language="python">
{InteractingAbortExample}
</RunnableCodeBlock>

## Conclusion

This page has shown how to interact with other Actors from your code: starting a run with <ApiLink to="class/Actor#start">`Actor.start`</ApiLink>, waiting for it to finish with <ApiLink to="class/Actor#call">`Actor.call`</ApiLink> or <ApiLink to="class/Actor#call_task">`Actor.call_task`</ApiLink>, transforming a run with <ApiLink to="class/Actor#metamorph">`Actor.metamorph`</ApiLink>, and stopping one with <ApiLink to="class/Actor#abort">`Actor.abort`</ApiLink>.

For the full list of methods for interacting with other Actors, see the <ApiLink to="class/Actor">`Actor`</ApiLink> API reference. For more details on running Actors and Actor tasks on the platform, see the [Actors](https://docs.apify.com/platform/actors) and [Actor tasks](https://docs.apify.com/platform/actors/tasks) documentation.
10 changes: 1 addition & 9 deletions docs/02_concepts/07_webhooks.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,7 @@ import ApiLink from '@theme/ApiLink';
import WebhookExample from '!!raw-loader!roa-loader!./code/07_webhook.py';
import WebhookPreventingExample from '!!raw-loader!roa-loader!./code/07_webhook_preventing.py';

Webhooks allow you to configure the Apify platform to perform an action when a certain event occurs. For example, you can use them to start another Actor when the current run finishes or fails.

You can learn more in the [documentation for webhooks](https://docs.apify.com/platform/integrations/webhooks).
[Webhooks](https://docs.apify.com/platform/integrations/webhooks) allow you to configure the Apify platform to perform an action when a certain event occurs. For example, you can use them to start another Actor when the current run finishes or fails.

## Creating an ad-hoc webhook dynamically

Expand All @@ -31,9 +29,3 @@ To ensure that duplicate ad-hoc webhooks won't get created in a case of Actor re
<RunnableCodeBlock className="language-python" language="python">
{WebhookPreventingExample}
</RunnableCodeBlock>

## Conclusion

This page has shown how to create ad-hoc webhooks dynamically with <ApiLink to="class/Actor#add_webhook">`Actor.add_webhook`</ApiLink> and how to avoid creating duplicates across Actor restarts using the `idempotency_key` parameter.

For more information about webhooks, including event types and payloads, see the [Apify webhooks documentation](https://docs.apify.com/platform/integrations/webhooks).
6 changes: 0 additions & 6 deletions docs/02_concepts/08_access_apify_api.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -35,9 +35,3 @@ To create a completely new instance of the client (for example, to use a differe
</RunnableCodeBlock>

This is useful when the Actor's own token is not the right one, for instance when acting on behalf of another user, or when you want different retry and timeout behavior than the shared `Actor.apify_client` instance.

## Conclusion

This page has shown how to reach the Apify API directly for platform features the SDK does not wrap. You can use the built-in client exposed by <ApiLink to="class/Actor#apify_client">`Actor.apify_client`</ApiLink>, or a freshly configured one created with <ApiLink to="class/Actor#new_client">`Actor.new_client`</ApiLink>.

For the full API client documentation, see the [Apify API Client for Python](https://docs.apify.com/api/client/python).
4 changes: 0 additions & 4 deletions docs/02_concepts/09_logging.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,3 @@ You can further decide whether you want to redirect just new logs of the ongoing
<RunnableCodeBlock className="language-python" language="python">
{RedirectLogExistingRun}
</RunnableCodeBlock>

## Conclusion

This page has covered how the SDK logs through the `apify` logger: setting the log level, formatting output with <ApiLink to="class/ActorLogFormatter">`ActorLogFormatter`</ApiLink>, using the logger at each level, and redirecting logs from other Actor runs back into the parent run. For more on the underlying logging machinery, see Python's standard [`logging`](https://docs.python.org/3/library/logging.html) module documentation.
8 changes: 1 addition & 7 deletions docs/02_concepts/10_configuration.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ import ApiLink from '@theme/ApiLink';

The <ApiLink to="class/Actor">`Actor`</ApiLink> class is configured through the <ApiLink to="class/Configuration">`Configuration`</ApiLink> class, which reads its settings from environment variables. When running on the Apify platform or through the Apify CLI, configuration is automatic — manual setup is only needed for custom requirements.

If you need some special configuration, you can adjust it either through the `Configuration` class directly, or by setting environment variables when running the Actor locally.
If you need some special configuration, you can adjust it either through the `Configuration` class directly, or by setting [environment variables](https://docs.apify.com/platform/actors/development/programming-interface/environment-variables) when running the Actor locally.

To see the full list of configuration options, check the `Configuration` class or the list of environment variables that the Actor understands.

Expand Down Expand Up @@ -65,9 +65,3 @@ The <ApiLink to="class/Actor#is_at_home">`Actor.is_at_home`</ApiLink> method ret
<RunnableCodeBlock className="language-python" language="python">
{PlatformDetectionExample}
</RunnableCodeBlock>

## Conclusion

This page has shown how to configure an Actor through the <ApiLink to="class/Configuration">`Configuration`</ApiLink> class or environment variables, which options you are most likely to set yourself, and how to inspect the runtime environment with <ApiLink to="class/Actor#get_env">`Actor.get_env`</ApiLink> and <ApiLink to="class/Actor#is_at_home">`Actor.is_at_home`</ApiLink>.

For the full list of configuration options, see the <ApiLink to="class/Configuration">`Configuration`</ApiLink> API reference. For a complete list of environment variables available on the platform, see the [environment variables](https://docs.apify.com/platform/actors/development/programming-interface/environment-variables) documentation.
8 changes: 2 additions & 6 deletions docs/02_concepts/12_storage_clients.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ import CustomStorageClientExample from '!!raw-loader!roa-loader!./code/12_custom

Storage clients are the components that read and write your [storages](./storages): datasets, key-value stores, and request queues. The Apify SDK selects an appropriate client automatically based on where the Actor runs. For most Actors you never need to think about them. This page explains the available clients and how to customize them when you do.

For details on how storage clients work and how to write your own, see the [Crawlee storage clients guide](https://crawlee.dev/python/docs/guides/storage-clients).

## How the Actor selects a storage client

By default, the Actor uses a <ApiLink to="class/SmartApifyStorageClient">`SmartApifyStorageClient`</ApiLink>, a hybrid client that delegates to one of two underlying clients depending on the environment:
Expand Down Expand Up @@ -69,9 +71,3 @@ You can replace either of the underlying clients, for example to keep all local
The Actor's storage client must be a `SmartApifyStorageClient`. Setting a bare `ApifyStorageClient` or `MemoryStorageClient` directly in the service locator raises an error. Wrap it in a `SmartApifyStorageClient` as shown above.

:::

## Conclusion

This page has explained how the Actor selects a storage client, the clients available in the `apify.storage_clients` module, the difference between the single and shared request-queue clients, and how to customize the client through the service locator.

For a deeper look at how storage clients work and how to write your own, see the [Crawlee storage clients guide](https://crawlee.dev/python/docs/guides/storage-clients).
4 changes: 0 additions & 4 deletions docs/02_concepts/13_exceptions.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,3 @@ To detect the limit, check the `event_charge_limit_reached` field on the `Charge
## Errors while crawling

If your Actor runs a [Crawlee](https://crawlee.dev/python) crawler, failures inside request handlers surface as Crawlee exceptions. Crawlee handles the retries and session rotation around them, so a single failing request doesn't stop the crawl. API calls you make from inside a handler still raise <ApiLink to="class/ApifyApiError">`ApifyApiError`</ApiLink>. For how to handle those errors, see [Errors from the Apify API](#errors-from-the-apify-api).

## Conclusion

Most failures you handle at runtime are <ApiLink to="class/ApifyApiError">`ApifyApiError`</ApiLink> from the API client. Catch it to cover any API failure, and reach for a subclass or the HTTP `status_code` when you need finer control. The standard [`RuntimeError`](https://docs.python.org/3/library/exceptions.html#RuntimeError), [`ValueError`](https://docs.python.org/3/library/exceptions.html#ValueError), and [`TypeError`](https://docs.python.org/3/library/exceptions.html#TypeError) signal a bug or bad input, so correct the call rather than catch them. After <ApiLink to="class/Actor#call">`Actor.call`</ApiLink>, check `run.status` to react to a failed run, and let Crawlee handle the errors raised inside a crawler.
Loading