Merge remote-tracking branch 'origin/master' into fix/rq-cache-commit-on-success

# Conflicts:
#	tests/unit/storage_clients/test_apify_request_queue_client.py

Author: vdusek

CHANGELOG.md

@@ -11,12 +11,19 @@ All notable changes to this project will be documented in this file.
 - **scrapy:** Skip a request that fails to convert instead of crashing the run ([#952](https://github.com/apify/apify-sdk-python/pull/952)) ([db9444f](https://github.com/apify/apify-sdk-python/commit/db9444faeb0158c29aa394121cf733ff2e843f28)) by [@vdusek](https://github.com/vdusek)
 - **scrapy:** [**breaking**] Serialize requests and HTTP cache as JSON instead of pickle ([#951](https://github.com/apify/apify-sdk-python/pull/951)) ([a87e8d1](https://github.com/apify/apify-sdk-python/commit/a87e8d1597478b4f12fd5bb9b379f65f637d8e96)) by [@vdusek](https://github.com/vdusek)
 - **scrapy:** Make logging configuration idempotent ([#954](https://github.com/apify/apify-sdk-python/pull/954)) ([2cc5602](https://github.com/apify/apify-sdk-python/commit/2cc5602b741b93c81f264d4e09e0d9bcfc7200f2)) by [@vdusek](https://github.com/vdusek)
+- Bump typing-extensions floor to 4.4.0 ([#960](https://github.com/apify/apify-sdk-python/pull/960)) ([b7e7d9c](https://github.com/apify/apify-sdk-python/commit/b7e7d9c9a053992d4281a8a19897957d2bff67e1)) by [@vdusek](https://github.com/vdusek)
+- Preserve decorated symbol types in docs_group and docs_name ([#964](https://github.com/apify/apify-sdk-python/pull/964)) ([6c359a7](https://github.com/apify/apify-sdk-python/commit/6c359a714b089b24c1be7fa9a07c9b9899cc5e11)) by [@vdusek](https://github.com/vdusek)
+- Exit already-entered contexts when Actor or event manager init fails ([#969](https://github.com/apify/apify-sdk-python/pull/969)) ([c1a15a7](https://github.com/apify/apify-sdk-python/commit/c1a15a7425eab74f3fe8848160576741c036c4fa)) by [@vdusek](https://github.com/vdusek)
+- **scrapy:** Dump pydantic models in JSON mode when serializing requests ([#961](https://github.com/apify/apify-sdk-python/pull/961)) ([f2ccae1](https://github.com/apify/apify-sdk-python/commit/f2ccae16d046e9586281e1f2ffc531014e05fe51)) by [@vdusek](https://github.com/vdusek)
+- Accept arbitrary JSON userData in ApifyRequestList ([#966](https://github.com/apify/apify-sdk-python/pull/966)) ([2cfd8a5](https://github.com/apify/apify-sdk-python/commit/2cfd8a590ae52a38b7efe39bb8a3a1570437b2fc)) by [@vdusek](https://github.com/vdusek)
+- Coerce null stats in Apify request queue metadata ([#974](https://github.com/apify/apify-sdk-python/pull/974)) ([63eb771](https://github.com/apify/apify-sdk-python/commit/63eb771e0234575ae3842cba9441938c3e4116fc)) by [@vdusek](https://github.com/vdusek)
 
 ### 🚜 Refactor
 
 - [**breaking**] Remove deprecated APIs ([#918](https://github.com/apify/apify-sdk-python/pull/918)) ([3e5728d](https://github.com/apify/apify-sdk-python/commit/3e5728d94cb8fd879d5a76e33a03d55792d835d5)) by [@vdusek](https://github.com/vdusek), closes [#635](https://github.com/apify/apify-sdk-python/issues/635)
 - [**breaking**] Mark secondary arguments as keyword-only ([#917](https://github.com/apify/apify-sdk-python/pull/917)) ([eb94c99](https://github.com/apify/apify-sdk-python/commit/eb94c992ec4aba1cd7cf4dfd7a98731cb304651b)) by [@vdusek](https://github.com/vdusek), closes [#881](https://github.com/apify/apify-sdk-python/issues/881)
 - [**breaking**] Adapt to apify-client v3 ([#719](https://github.com/apify/apify-sdk-python/pull/719)) ([10203bc](https://github.com/apify/apify-sdk-python/commit/10203bc51e67590c97938b37d81614376bc3d29a)) by [@vdusek](https://github.com/vdusek), closes [#697](https://github.com/apify/apify-sdk-python/issues/697), [#736](https://github.com/apify/apify-sdk-python/issues/736), [#770](https://github.com/apify/apify-sdk-python/issues/770), [#853](https://github.com/apify/apify-sdk-python/issues/853)
+- [**breaking**] Remove wait_for_finish from Actor.start ([#962](https://github.com/apify/apify-sdk-python/pull/962)) ([0daca28](https://github.com/apify/apify-sdk-python/commit/0daca2818860cbcbd0778efa4d73622783ce63cf)) by [@vdusek](https://github.com/vdusek)
 
 ### ⚙️ Miscellaneous Tasks
 

CONTRIBUTING.md

@@ -4,7 +4,7 @@ Here you'll find a contributing guide to get started with development.
 
 ## Environment
 
-For local development, it is required to have Python 3.10 (or a later version) installed.
+For local development, it is required to have Python 3.11 (or a later version) installed.
 
 We use [uv](https://docs.astral.sh/uv/) for project management. Install it and set up your IDE accordingly.
 

README.md

@@ -9,7 +9,16 @@ import CodeBlock from '@theme/CodeBlock';
 
 import IntroductionExample from '!!raw-loader!./code/01_introduction.py';
 
-The Apify SDK for Python is the official library for creating [Apify Actors](https://docs.apify.com/platform/actors) in Python. It provides useful features like Actor lifecycle management, local storage emulation, and Actor event handling.
+The Apify SDK for Python is the official library for creating [Apify Actors](https://docs.apify.com/platform/actors) in Python. It provides everything you need to build an Actor and run it both locally and on the [Apify platform](https://docs.apify.com/platform). With the SDK, you can:
+
+- Manage the Actor lifecycle: initialization, graceful shutdown, status messages, rebooting, and metamorphing.
+- Work with datasets, key-value stores, and request queues, with automatic local emulation when running outside the platform.
+- Read the Actor input, including automatic decryption of secret fields.
+- React to platform events (system info, migration, abort) and persist state across migrations and restarts.
+- Manage proxies, both [Apify Proxy](https://docs.apify.com/platform/proxy) and your own, with session and tiered-proxy support.
+- Start, call, and abort Actors and tasks, create webhooks, and reach the full Apify API client.
+- Charge users with the pay-per-event pricing model.
+- Integrate with [Crawlee](../guides/crawlee) and [Scrapy](../guides/scrapy), with guides for [Playwright](../guides/playwright) and others.
 
 <CodeBlock className="language-python">
     {IntroductionExample}
@@ -29,7 +38,7 @@ Explore the Guides section in the sidebar for a deeper understanding of the SDK'
 
 ## Installation
 
-The Apify SDK for Python requires Python version 3.10 or above. It is typically installed when you create a new Actor project using the [Apify CLI](https://docs.apify.com/cli). To install it manually in an existing project, use:
+The Apify SDK for Python requires Python version 3.11 or above. It is typically installed when you create a new Actor project using the [Apify CLI](https://docs.apify.com/cli). To install it manually in an existing project, use:
 
 ```bash
 pip install apify

docs/01_introduction/index.mdx

@@ -17,11 +17,11 @@ import UnderscoreMainExample from '!!raw-loader!./code/actor_structure/__main__.
 
 ## Step 1: Create Actors
 
-To create and run Actors in Apify Console, refer to the [Console documentation](/platform/actors/development/quick-start/web-ide).
+To create and run Actors in [Apify Console](https://docs.apify.com/platform/console), refer to the [Console documentation](/platform/actors/development/quick-start/web-ide).
 
 To create a new Apify Actor on your computer, you can use the [Apify CLI](/cli), and select one of the [Python Actor templates](https://apify.com/templates?category=python).
 
-For example, to create an Actor from the "[beta] Python SDK" template, you can use the [`apify create` command](/cli/docs/reference#apify-create-actorname).
+For example, to create an Actor from the "Getting started with Python" template, you can use the [`apify create` command](/cli/docs/reference#apify-create-actorname).
 
 ```bash
 apify create my-first-actor --template python-start
@@ -59,15 +59,15 @@ The Actor's runtime dependencies are specified in the `requirements.txt` file, w
 The Actor's source code is in the `src` folder. This folder contains two important files:
 
 - `main.py` - which contains the main function of the Actor
-- `__main__.py` - which is the entrypoint of the Actor package setting up the Actor [logger](../concepts/logging) and executing the Actor's main function via [`asyncio.run()`](https://docs.python.org/3/library/asyncio-runner.html#asyncio.run).
+- `__main__.py` - which is the entrypoint of the Actor package, executing the Actor's main function via [`asyncio.run()`](https://docs.python.org/3/library/asyncio-runner.html#asyncio.run).
 
 <Tabs>
     <TabItem value="main.py" label="main.py" default>
         <CodeBlock className="language-python">
             {MainExample}
         </CodeBlock>
     </TabItem>
-    <TabItem value="__main__.py" label="__main.py__">
+    <TabItem value="__main__.py" label="__main__.py">
         <CodeBlock className="language-python">
             {UnderscoreMainExample}
         </CodeBlock>
@@ -79,13 +79,15 @@ We recommend keeping the entrypoint for the Actor in the `src/__main__.py` file.
 
 ## Next steps
 
+Now that you can create and run an Actor locally, explore the rest of the SDK's features and its framework integrations.
+
 ### Concepts
 
 To learn more about the features of the Apify SDK and how to use them, check out the Concepts section in the sidebar:
 
 - [Actor lifecycle](../concepts/actor-lifecycle)
 - [Actor input](../concepts/actor-input)
-- [Working with storages](../concepts/storages)
+- [Storages](../concepts/storages)
 - [Actor events & state persistence](../concepts/actor-events)
 - [Proxy management](../concepts/proxy-management)
 - [Interacting with other Actors](../concepts/interacting-with-other-actors)
@@ -94,6 +96,7 @@ To learn more about the features of the Apify SDK and how to use them, check out
 - [Logging](../concepts/logging)
 - [Actor configuration](../concepts/actor-configuration)
 - [Pay-per-event monetization](../concepts/pay-per-event)
+- [Storage clients](../concepts/storage-clients)
 
 ### Guides
 

docs/01_introduction/quick-start.mdx

@@ -21,7 +21,7 @@ import RebootExample from '!!raw-loader!roa-loader!./code/01_reboot.py';
 
 import StatusMessageExample from '!!raw-loader!roa-loader!./code/01_status_message.py';
 
-This guide explains how an **Apify Actor** starts, runs, and shuts down, describing the complete Actor lifecycle. For information about the core concepts such as Actors, the Apify Console, storages, and events, check out the [Apify platform documentation](https://docs.apify.com/platform).
+This guide explains how an **Apify Actor** starts, runs, and shuts down, describing the complete Actor lifecycle. For information about the core concepts such as Actors, the [Apify Console](https://docs.apify.com/platform/console), storages, and events, check out the [Apify platform documentation](https://docs.apify.com/platform).
 
 ## Actor initialization
 
@@ -106,4 +106,4 @@ Update the status only when the user's understanding of progress changes - avoid
 
 ## Conclusion
 
-This page has presented the full Actor lifecycle: initialization, execution, error handling, rebooting, shutdown and status messages. You've seen how the SDK supports both context-based and manual control patterns. For deeper dives, explore the <ApiLink to="">reference docs</ApiLink>, [guides](https://docs.apify.com/sdk/python/docs/guides/beautifulsoup-httpx), and [platform documentation](https://docs.apify.com/platform).
+This page has presented the full Actor lifecycle: initialization, execution, error handling, rebooting, shutdown and status messages. You've seen how the SDK supports both context-based and manual control patterns. For deeper dives, explore the <ApiLink to="class/Actor">`Actor`</ApiLink> API reference, [guides](../guides/beautifulsoup-httpx), and [platform documentation](https://docs.apify.com/platform).

docs/02_concepts/01_actor_lifecycle.mdx

@@ -12,7 +12,7 @@ import ApiLink from '@theme/ApiLink';
 
 The Actor gets its [input](https://docs.apify.com/platform/actors/running/input) from the input record in its default [key-value store](https://docs.apify.com/platform/storage/key-value-store).
 
-To access it, instead of reading the record manually, you can use the <ApiLink to="class/Actor#get_input">`Actor.get_input`</ApiLink> convenience method. It will get the input record key from the Actor configuration, read the record from the default key-value store,and decrypt any [secret input fields](https://docs.apify.com/platform/actors/development/secret-input).
+To access it, instead of reading the record manually, you can use the <ApiLink to="class/Actor#get_input">`Actor.get_input`</ApiLink> convenience method. It gets the input record key from the Actor configuration, reads the record from the default key-value store, and decrypts any [secret input fields](https://docs.apify.com/platform/actors/development/secret-input).
 
 For example, if an Actor received a JSON input with two fields, `{ "firstNumber": 1, "secondNumber": 2 }`, this is how you might process it:
 
@@ -38,4 +38,8 @@ The Apify platform supports [secret input fields](https://docs.apify.com/platfor
 
 No special handling is needed in your code — when you call <ApiLink to="class/Actor#get_input">`Actor.get_input`</ApiLink>, encrypted fields are automatically decrypted using the Actor's private key, which is provided by the platform via environment variables. You receive the plaintext values directly.
 
+## Conclusion
+
+This page has shown how to read Actor input with <ApiLink to="class/Actor#get_input">`Actor.get_input`</ApiLink>, how to load URL sources with <ApiLink to="class/ApifyRequestList">`ApifyRequestList`</ApiLink>, and how secret input fields are decrypted automatically when you read them.
+
 For more details on Actor input and how to define input schemas, see the [Actor input](https://docs.apify.com/platform/actors/running/input) and [input schema](https://docs.apify.com/platform/actors/development/input-schema) documentation on the Apify platform.

docs/02_concepts/02_actor_input.mdx

@@ -1,6 +1,6 @@
 ---
 id: storages
-title: Working with storages
+title: Storages
 description: Use datasets, key-value stores, and request queues to persist Actor data.
 ---
 
@@ -45,11 +45,11 @@ Each dataset item, key-value store record, or request in a request queue is then
 
 When developing locally, opening any storage will by default use local storage. To change this behavior and to use remote storage you have to use `force_cloud=True` argument in <ApiLink to="class/Actor#open_dataset">`Actor.open_dataset`</ApiLink>, <ApiLink to="class/Actor#open_request_queue">`Actor.open_request_queue`</ApiLink> or <ApiLink to="class/Actor#open_key_value_store">`Actor.open_key_value_store`</ApiLink>. Proper use of this argument allows you to work with both local and remote storages.
 
-Calling another remote Actor and accessing its default storage is typical use-case for using `force-cloud=True` argument to open remote Actor's storages.
+Calling another remote Actor and accessing its default storage is a typical use-case for using `force_cloud=True` argument to open remote Actor's storages.
 
 ### Local storage persistence
 
-By default, the storage contents are persisted across multiple Actor runs. To clean up the Actor storages before the running the Actor, use the `--purge` flag of the [`apify run`](https://docs.apify.com/cli/docs/reference#apify-run) command of the Apify CLI.
+By default, the storage contents are persisted across multiple Actor runs. To clean up the Actor storages before running the Actor, use the `--purge` flag of the [`apify run`](https://docs.apify.com/cli/docs/reference#apify-run) command of the Apify CLI.
 
 ```bash
 apify run --purge
@@ -106,8 +106,8 @@ To get an iterator of the data, you can use the <ApiLink to="class/Dataset#itera
 ### Exporting items
 
 You can also export the dataset items into a key-value store, as either a CSV or a JSON record,
-using the <ApiLink to="class/Dataset#export_to_csv">`Dataset.export_to_csv`</ApiLink>
-or <ApiLink to="class/Dataset#export_to_json">`Dataset.export_to_json`</ApiLink> method.
+using the <ApiLink to="class/Dataset#export_to">`Dataset.export_to`</ApiLink> method with the
+`content_type` argument set to `'csv'` or `'json'`.
 
 <RunnableCodeBlock className="language-python" language="python">
     {DatasetExportsExample}
@@ -183,6 +183,10 @@ To check if all the requests in the queue are handled, you can use the <ApiLink
 
 ## Storage clients
 
-Behind the scenes, the SDK uses storage clients to communicate with the storage backend. The appropriate client is selected automatically based on the runtime environment — on the Apify platform, data is persisted via the Apify API, while local runs use the filesystem. For most use cases, you don't need to think about storage clients at all. If you want to learn more about how storage clients work, the available implementations, or how to configure them, see the [Crawlee storage clients guide](https://crawlee.dev/python/docs/guides/storage-clients). The Apify-specific clients are available in the `apify.storage_clients` module.
+Behind the scenes, the SDK uses storage clients to communicate with the storage backend. The appropriate client is selected automatically based on the runtime environment. On the Apify platform, data is persisted via the Apify API, while local runs use the filesystem. For most use cases, you don't need to think about storage clients at all. To learn about the available implementations, how to switch between a single and shared request queue, or how to configure a custom client, see [Storage clients](./storage-clients). For a deeper look at how storage clients work internally, see the [Crawlee storage clients guide](https://crawlee.dev/python/docs/guides/storage-clients).
+
+## Conclusion
+
+This page has covered the three storage types (datasets, key-value stores, and request queues): how they are emulated on the local filesystem, how to open named and unnamed storages, and how to read from and write to each through the `Actor` shortcuts and the storage classes.
 
 For comprehensive information about storage on the Apify platform, see the [storage documentation](https://docs.apify.com/platform/storage), including the pages on [datasets](https://docs.apify.com/platform/storage/dataset), [key-value stores](https://docs.apify.com/platform/storage/key-value-store), and [request queues](https://docs.apify.com/platform/storage/request-queue).