Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 19 additions & 9 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,8 @@ graph TD
DAZ["DAZ Studio<br/>(DzScript engine, scene graph)"]

Client -->|HTTP POST /execute| SLT
SLT -->|BlockingQueuedConnection| Pane
SLT -->|sync + DS4 async submit:<br/>BlockingQueuedConnection| Pane
SLT -->|DS6 async: validate + enqueue| ARM
Pane --> Auth
Pane --> Rate
Pane --> WL
Expand All @@ -51,7 +52,7 @@ graph TD
RV --> RP
RP -->|Main thread| DAZ
RP --> ARM
ARM -->|BlockingQueuedConnection| Pane
ARM -->|QueuedConnection| Pane
Auth --> SR
JB -.-> Pane
SS -.-> Services
Expand Down Expand Up @@ -86,11 +87,16 @@ sequenceDiagram
**Rules that must never be broken:**

1. `DzScript`, `QScriptEngine`, and all DAZ API calls on the **main thread only**.
2. HTTP handlers (running on `std::thread`s) must emit signals with
`Qt::BlockingQueuedConnection` to cross into the main thread.
2. Synchronous execution crosses from HTTP workers with
`Qt::BlockingQueuedConnection`. On Studio 6, async script submission uses
only local, reentrant Qt Core values and mutex-protected services so it can
enqueue while the main thread is busy. Studio 4 retains the blocking crossing
for submission because its JSON parser uses `QScriptEngine`.
3. `DzScript` objects must be created **and** destroyed on the main thread.
4. `killRender()` must be invoked via a signal on the main thread, not from
the HTTP thread.
5. HTTP workers must not read GUI-owned mutable settings; async handlers receive
immutable snapshots captured before the server starts.

---

Expand Down Expand Up @@ -236,13 +242,17 @@ Qt 4.8 (the SDK's Qt) has `QHttp` but it is deprecated and removed in later
Qt versions. `cpp-httplib` is header-only, zero-dependency, and well-maintained.
It runs on a dedicated thread managed by `ServerListenThread`.

### Why BlockingQueuedConnection for every request?
### Why different crossings for synchronous and asynchronous execution?

The DAZ Studio SDK explicitly requires all scene-graph and script-engine
operations on the main thread. `BlockingQueuedConnection` gives us the main-
thread guarantee while letting the HTTP thread block until the result is ready
(for synchronous `/execute`) or until the request is accepted into the queue
(for async endpoints).
operations on the main thread. Synchronous `/execute` therefore uses
`BlockingQueuedConnection` and waits for the result. On Studio 6, async script
endpoints validate request value data on the HTTP worker and submit directly to
the mutex-protected `AsyncRequestManager`; blocking on the main thread here
would prevent submission and queued cancellation while a prior script is
running. Studio 4 keeps submission on the main thread because its JSON fallback
uses `QScriptEngine`. In both builds the manager posts execution to the main
thread, where `DzScript` is loaded and run.

### Why session-only script registry?

Expand Down
19 changes: 19 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,25 @@ All notable changes to DazScript Server are documented here.

## [Unreleased]

### Async script-file jobs

`POST /execute/async` now implements the `scriptFile` form already described
by the API schema. File-backed requests keep the path in the queue and call
`DzScript::loadFromFile()` when execution starts, preserving
`getScriptFileName()` and relative `include()` behavior instead of flattening
the file to inline source. `DazClient` and `AsyncDazClient` expose this as
`execute_file_async_submit()`; the existing status/result/list/cancel methods
manage the returned request id.

On Studio 6, the result-capturing `evaluate()` wrapper now forwards the
request's explicit filename; Studio otherwise treats the wrapper as anonymous
source and returns an empty `getScriptFileName()`. Script enqueue
handlers also submit directly from the HTTP worker into the mutex-protected
queue on Studio 6, so a new async request returns while Daz's main thread is
occupied and can be cancelled before it starts. Studio 4 retains main-thread
submission because its JSON fallback uses `QScriptEngine`. UI logging remains
queued to the main thread.

## [2.9.0] - 2026-08-16

### Added
Expand Down
19 changes: 15 additions & 4 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,10 +53,21 @@ reuses a stale CMake cache from the other SDK.

### Threading Model (CRITICAL)

- **Main Qt thread**: GUI, script execution via `DzScript`, all Qt/DAZ API calls
- **HTTP thread**: `ServerListenThread` blocks on `httplib::Server::listen()`

**IMPORTANT:** HTTP handlers run on raw `std::thread`s (not Qt threads). Handlers must do minimal work (parse body), then invoke `handleExecuteRequest()` on main thread via `Qt::BlockingQueuedConnection`. All `QScriptEngine`, `DzScript`, and Qt operations MUST happen on the main thread.
- **Main Qt thread**: GUI, script execution via `DzScript`, DAZ API calls, and
access to thread-affine `QObject` instances
- **HTTP workers**: middleware plus synchronous parsing/validation; Studio 6
async queue submission may use local instances of documented reentrant Qt
Core value classes

**IMPORTANT:** HTTP handlers run on raw `std::thread`s (not Qt threads).
Synchronous execution crosses to `handleExecuteRequest()` via
`Qt::BlockingQueuedConnection`. On Studio 6, async script submission may parse
and validate with local reentrant Qt Core values, then submit to mutex-protected
services without blocking on the main thread. Studio 4 must keep submission on
the main thread because `JsonStd::parseObject()` uses `QScriptEngine` there.
Neither path may touch GUI objects, `DzScript`, the DAZ API, or shared mutable
state from a worker. Settings used by an async handler are snapshotted before
the server starts.

### Request Flow (POST /execute)

Expand Down
11 changes: 6 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -239,7 +239,7 @@ Async frameworks (FastAPI, FastMCP, ComfyUI custom nodes, asyncio/Temporal
workflows) previously had to wrap every `DazClient` call in
`asyncio.to_thread()` or hand-roll an `httpx.AsyncClient` wrapper.
`dazpy.aio.AsyncDazClient` mirrors `DazClient`'s full method surface —
`execute`, `execute_file`, async submit/status/result/list/cancel, render
`execute`, `execute_file`, inline/file async submit/status/result/list/cancel, render
submit/batch/animation, USD export, and server-health endpoints — as
native `async def` methods backed by `httpx.AsyncClient`, including the
same `retry_on_busy`/`max_wait` backoff semantics (via `asyncio.sleep`
Expand All @@ -250,8 +250,9 @@ from dazpy.aio import AsyncDazClient

async def main():
async with AsyncDazClient() as client:
result = await client.execute("1 + 1;")
print(result.value)
request_id = await client.execute_file_async_submit("C:/jobs/pose-probe.dsa")
result = await client.get_request_result(request_id, wait=True)
print(result)
```

Requires the optional `httpx` dependency: `pip install dazpy[aio]`.
Expand Down Expand Up @@ -784,7 +785,7 @@ See the [full migration guide](MIGRATION.md) for upgrade steps and rollback inst

Long-running operations (renders, exports, batch jobs) no longer need to block the HTTP connection:

- **`POST /execute/async`** — Submit any inline script asynchronously; returns a `request_id` immediately
- **`POST /execute/async`** — Submit inline `script` or host-side `scriptFile` work asynchronously; returns a `request_id` immediately
- **`POST /scripts/:id/async`** — Submit a registered script asynchronously
- **`GET /requests/:id/status`** — Poll for progress (`queued`, `running`, `completed`, `failed`, `cancelled`)
- **`GET /requests/:id/result`** — Fetch the final result; supports `?wait=true` to long-poll until complete
Expand Down Expand Up @@ -1768,7 +1769,7 @@ result = requests.get(f"{BASE}/requests/{req_id}/result?wait=true&timeout=300",

### `POST /execute/async`

**Purpose:** Submit an inline script for asynchronous execution
**Purpose:** Submit an inline script or host-side script file for asynchronous execution
**Authentication:** Required (if enabled)

**Request Body:** Same as `POST /execute`
Expand Down
21 changes: 21 additions & 0 deletions dazpy/_client.py
Original file line number Diff line number Diff line change
Expand Up @@ -300,6 +300,27 @@ def _do():

return self._with_busy_retry(_do, retry_on_busy, max_wait)

def execute_file_async_submit(
self, script_file: str, args: object = None, *, retry_on_busy: bool = False, max_wait: float = 30.0
) -> str:
"""Submit a host-side ``.dsa`` file for asynchronous execution.

The file is loaded by DAZ Studio when the queued job starts, preserving
its filename for ``getScriptFileName()`` and relative ``include()``
calls. Returns the server-assigned request id immediately; use the
request status/result/cancel methods to manage its lifecycle.
"""
payload: dict = {"scriptFile": script_file}
if args is not None:
payload["args"] = args

def _do():
resp = self._post("/execute/async", payload)
_raise_for_error(resp)
return resp.json().get("request_id", "")

return self._with_busy_retry(_do, retry_on_busy, max_wait)

def get_request_status(self, request_id: str) -> dict:
"""Return the current status of an async request.

Expand Down
18 changes: 18 additions & 0 deletions dazpy/_client_aio.py
Original file line number Diff line number Diff line change
Expand Up @@ -141,6 +141,24 @@ async def _do():

return await self._with_busy_retry(_do, retry_on_busy, max_wait)

async def execute_file_async_submit(
self, script_file: str, args: object = None, *, retry_on_busy: bool = False, max_wait: float = 30.0
) -> str:
"""Submit a host-side ``.dsa`` file asynchronously.

See :meth:`dazpy.DazClient.execute_file_async_submit`.
"""
payload: dict = {"scriptFile": script_file}
if args is not None:
payload["args"] = args

async def _do():
resp = await self._post("/execute/async", payload)
_raise_for_error(resp)
return resp.json().get("request_id", "")

return await self._with_busy_retry(_do, retry_on_busy, max_wait)

async def get_request_status(self, request_id: str) -> dict:
"""See :meth:`dazpy.DazClient.get_request_status`."""
resp = await self._get(f"/requests/{request_id}/status")
Expand Down
10 changes: 6 additions & 4 deletions include/AsyncRequestManager.h
Original file line number Diff line number Diff line change
Expand Up @@ -59,8 +59,8 @@ class AsyncRequestManager {

// Enqueue a new async request. Returns SubmitResult::accepted=false when
// the queue is at capacity or too many requests are tracked.
SubmitResult submit(const QString& scriptText, const QVariantMap& args,
const QString& idPrefix);
SubmitResult submit(const QString& scriptText, const QString& scriptFile,
const QVariantMap& args, const QString& idPrefix);

// Enqueue a render job. Same as submit() but tags the request as
// REQUEST_TYPE_RENDER so cancel dispatch calls killRender() correctly.
Expand All @@ -79,9 +79,10 @@ class AsyncRequestManager {

// ── Main-thread API ────────────────────────────────────────────────────

// Dequeue next QUEUED request into outId/outScript/outArgs.
// Dequeue next QUEUED request into outId/outScript/outScriptFile/outArgs.
// Sets m_currentId and returns true if work was found; false otherwise.
bool dequeueNext(QString& outId, QString& outScript, QVariantMap& outArgs);
bool dequeueNext(QString& outId, QString& outScript, QString& outScriptFile,
QVariantMap& outArgs);

// Mark the running request as RUNNING (sets startedAt).
void markRunning(const QString& id);
Expand Down Expand Up @@ -144,6 +145,7 @@ class AsyncRequestManager {
RequestStatus status;
RequestType requestType;
QString scriptText;
QString scriptFile;
QVariantMap args;
QVariant scriptResult;
QStringList outputLines;
Expand Down
26 changes: 17 additions & 9 deletions include/DzScriptServerPane.h
Original file line number Diff line number Diff line change
Expand Up @@ -116,13 +116,19 @@ public slots:
Q_INVOKABLE HttpResult handleRegisterScript(const QByteArray& jsonBody, const QByteArray& clientIP);
Q_INVOKABLE HttpResult handleRegistryExecuteRequest(const QByteArray& scriptText, const QByteArray& scriptId, const QByteArray& requestBody, const QByteArray& clientIP);

// Async enqueue helpers — called on main thread via BlockingQueuedConnection from
// AsyncExecuteHandler / AsyncScriptHandler so that all DzScript/Qt work
// stays on the Qt main thread.
Q_INVOKABLE HttpResult handleAsyncExecuteEnqueue(const QByteArray& jsonBody);
// On Studio 6 these are called directly from HTTP workers and use only local,
// reentrant Qt Core values plus mutex-protected services. Studio 4 routes them
// through BlockingQueuedConnection because its JSON parser uses QScriptEngine.
// Neither path touches GUI, DzScript, or DAZ SDK state off the main thread.
// The script limit is an immutable snapshot captured before server start.
Q_INVOKABLE HttpResult handleAsyncExecuteEnqueue(const QByteArray& jsonBody,
const QByteArray& clientIP,
int maxScriptLengthKB);
Q_INVOKABLE HttpResult handleAsyncScriptEnqueue(const QByteArray& scriptBytes,
const QByteArray& scriptIdBytes,
const QByteArray& bodyBytes);

// Render enqueue still builds Daz/Qt render work on the main thread.
Q_INVOKABLE HttpResult handleAsyncRenderEnqueue(const QByteArray& jsonBody);
Q_INVOKABLE HttpResult handleAsyncRenderBatchEnqueue(const QByteArray& jsonBody);
Q_INVOKABLE HttpResult handleAsyncRenderAnimationEnqueue(const QByteArray& jsonBody);
Expand Down Expand Up @@ -152,8 +158,10 @@ public slots:
bool lookupRegistryScript(const std::string& id, std::string& outScript) const;

// Async request management — called from HTTP threads (delegated to AsyncRequestManager)
// enqueueAsyncRequest still takes Qt types (used by Tier-1 async handlers — fix pending)
QString enqueueAsyncRequest(const QString& scriptText, const QVariantMap& args,
// Qt value types here are protected by AsyncRequestManager's mutex and are
// only called directly from documented reentrant Studio 6 worker paths.
QString enqueueAsyncRequest(const QString& scriptText, const QString& scriptFile,
const QVariantMap& args,
const QString& idPrefix, qint64& outSubmittedAt,
QString& outError);
std::pair<int, std::string> getAsyncStatusJson(const std::string& requestId) const;
Expand Down Expand Up @@ -222,9 +230,9 @@ private slots:

// Persistent DzScript instance, reused (via clear()) across every script
// execution instead of constructing/destroying one per request. All
// execution is already serialized onto this (the main) thread via
// Qt::BlockingQueuedConnection — DazScript isn't thread-safe — so reuse
// here is sequential, not concurrent. See runDazScript() in the .cpp.
// execution is serialized onto this (the main) thread — synchronously for
// sync requests and by a queued wake-up for async requests — so reuse is
// sequential, not concurrent. See runDazScript() in the .cpp.
DzScript* m_pPersistentScript;

// ── Settings service ─────────────────────────────────────────────────────
Expand Down
3 changes: 2 additions & 1 deletion include/RequestHandler.h
Original file line number Diff line number Diff line change
Expand Up @@ -180,10 +180,11 @@ class ScriptExecuteHandler : public IRequestHandler {

class AsyncExecuteHandler : public IRequestHandler {
public:
explicit AsyncExecuteHandler(DzScriptServerPane* pane);
AsyncExecuteHandler(DzScriptServerPane* pane, int maxScriptLengthKB);
void handle(HttpContext& ctx) override;
private:
DzScriptServerPane* m_pPane;
const int m_maxScriptLengthKB;
};

class AsyncScriptHandler : public IRequestHandler {
Expand Down
2 changes: 1 addition & 1 deletion openapi.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -271,7 +271,7 @@ paths:
/execute/async:
post:
tags: [Async]
summary: Submit an inline script for asynchronous execution
summary: Submit an inline script or host-side script file for asynchronous execution
operationId: executeAsync
security:
- apiToken: []
Expand Down
8 changes: 6 additions & 2 deletions src/AsyncRequestManager.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,8 @@ AsyncRequestManager::AsyncRequestManager(QObject* notifyTarget)
// ─── HTTP-thread API ──────────────────────────────────────────────────────────

AsyncRequestManager::SubmitResult AsyncRequestManager::submit(
const QString& scriptText, const QVariantMap& args, const QString& idPrefix)
const QString& scriptText, const QString& scriptFile,
const QVariantMap& args, const QString& idPrefix)
{
SubmitResult r;
r.accepted = false;
Expand All @@ -44,6 +45,7 @@ AsyncRequestManager::SubmitResult AsyncRequestManager::submit(
AsyncRequest req;
req.id = MetricsCollector::generateAsyncId(idPrefix);
req.scriptText = scriptText;
req.scriptFile = scriptFile;
req.args = args;
req.submittedAt = QDateTime::currentMSecsSinceEpoch();

Expand Down Expand Up @@ -385,7 +387,8 @@ int AsyncRequestManager::getTotalTracked() const

// ─── Main-thread API ──────────────────────────────────────────────────────────

bool AsyncRequestManager::dequeueNext(QString& outId, QString& outScript, QVariantMap& outArgs)
bool AsyncRequestManager::dequeueNext(QString& outId, QString& outScript,
QString& outScriptFile, QVariantMap& outArgs)
{
QMutexLocker locker(&m_mutex);
if (!m_currentId.isEmpty()) return false;
Expand All @@ -397,6 +400,7 @@ bool AsyncRequestManager::dequeueNext(QString& outId, QString& outScript, QVaria
const AsyncRequest& req = m_requests.value(id);
outId = id;
outScript = req.scriptText;
outScriptFile = req.scriptFile;
outArgs = req.args;
return true;
}
Expand Down
Loading
Loading