diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 84a6711..34b63db 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -42,7 +42,8 @@ graph TD DAZ["DAZ Studio
(DzScript engine, scene graph)"] Client -->|HTTP POST /execute| SLT - SLT -->|BlockingQueuedConnection| Pane + SLT -->|sync + DS4 async submit:
BlockingQueuedConnection| Pane + SLT -->|DS6 async: validate + enqueue| ARM Pane --> Auth Pane --> Rate Pane --> WL @@ -51,7 +52,7 @@ graph TD RV --> RP RP -->|Main thread| DAZ RP --> ARM - ARM -->|BlockingQueuedConnection| Pane + ARM -->|QueuedConnection| Pane Auth --> SR JB -.-> Pane SS -.-> Services @@ -86,11 +87,16 @@ sequenceDiagram **Rules that must never be broken:** 1. `DzScript`, `QScriptEngine`, and all DAZ API calls on the **main thread only**. -2. HTTP handlers (running on `std::thread`s) must emit signals with - `Qt::BlockingQueuedConnection` to cross into the main thread. +2. Synchronous execution crosses from HTTP workers with + `Qt::BlockingQueuedConnection`. On Studio 6, async script submission uses + only local, reentrant Qt Core values and mutex-protected services so it can + enqueue while the main thread is busy. Studio 4 retains the blocking crossing + for submission because its JSON parser uses `QScriptEngine`. 3. `DzScript` objects must be created **and** destroyed on the main thread. 4. `killRender()` must be invoked via a signal on the main thread, not from the HTTP thread. +5. HTTP workers must not read GUI-owned mutable settings; async handlers receive + immutable snapshots captured before the server starts. --- @@ -236,13 +242,17 @@ Qt 4.8 (the SDK's Qt) has `QHttp` but it is deprecated and removed in later Qt versions. `cpp-httplib` is header-only, zero-dependency, and well-maintained. It runs on a dedicated thread managed by `ServerListenThread`. -### Why BlockingQueuedConnection for every request? +### Why different crossings for synchronous and asynchronous execution? The DAZ Studio SDK explicitly requires all scene-graph and script-engine -operations on the main thread. `BlockingQueuedConnection` gives us the main- -thread guarantee while letting the HTTP thread block until the result is ready -(for synchronous `/execute`) or until the request is accepted into the queue -(for async endpoints). +operations on the main thread. Synchronous `/execute` therefore uses +`BlockingQueuedConnection` and waits for the result. On Studio 6, async script +endpoints validate request value data on the HTTP worker and submit directly to +the mutex-protected `AsyncRequestManager`; blocking on the main thread here +would prevent submission and queued cancellation while a prior script is +running. Studio 4 keeps submission on the main thread because its JSON fallback +uses `QScriptEngine`. In both builds the manager posts execution to the main +thread, where `DzScript` is loaded and run. ### Why session-only script registry? diff --git a/CHANGELOG.md b/CHANGELOG.md index 8f9b515..90e639b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,25 @@ All notable changes to DazScript Server are documented here. ## [Unreleased] +### Async script-file jobs + +`POST /execute/async` now implements the `scriptFile` form already described +by the API schema. File-backed requests keep the path in the queue and call +`DzScript::loadFromFile()` when execution starts, preserving +`getScriptFileName()` and relative `include()` behavior instead of flattening +the file to inline source. `DazClient` and `AsyncDazClient` expose this as +`execute_file_async_submit()`; the existing status/result/list/cancel methods +manage the returned request id. + +On Studio 6, the result-capturing `evaluate()` wrapper now forwards the +request's explicit filename; Studio otherwise treats the wrapper as anonymous +source and returns an empty `getScriptFileName()`. Script enqueue +handlers also submit directly from the HTTP worker into the mutex-protected +queue on Studio 6, so a new async request returns while Daz's main thread is +occupied and can be cancelled before it starts. Studio 4 retains main-thread +submission because its JSON fallback uses `QScriptEngine`. UI logging remains +queued to the main thread. + ## [2.9.0] - 2026-08-16 ### Added diff --git a/CLAUDE.md b/CLAUDE.md index f641bf6..be7bc57 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -53,10 +53,21 @@ reuses a stale CMake cache from the other SDK. ### Threading Model (CRITICAL) -- **Main Qt thread**: GUI, script execution via `DzScript`, all Qt/DAZ API calls -- **HTTP thread**: `ServerListenThread` blocks on `httplib::Server::listen()` - -**IMPORTANT:** HTTP handlers run on raw `std::thread`s (not Qt threads). Handlers must do minimal work (parse body), then invoke `handleExecuteRequest()` on main thread via `Qt::BlockingQueuedConnection`. All `QScriptEngine`, `DzScript`, and Qt operations MUST happen on the main thread. +- **Main Qt thread**: GUI, script execution via `DzScript`, DAZ API calls, and + access to thread-affine `QObject` instances +- **HTTP workers**: middleware plus synchronous parsing/validation; Studio 6 + async queue submission may use local instances of documented reentrant Qt + Core value classes + +**IMPORTANT:** HTTP handlers run on raw `std::thread`s (not Qt threads). +Synchronous execution crosses to `handleExecuteRequest()` via +`Qt::BlockingQueuedConnection`. On Studio 6, async script submission may parse +and validate with local reentrant Qt Core values, then submit to mutex-protected +services without blocking on the main thread. Studio 4 must keep submission on +the main thread because `JsonStd::parseObject()` uses `QScriptEngine` there. +Neither path may touch GUI objects, `DzScript`, the DAZ API, or shared mutable +state from a worker. Settings used by an async handler are snapshotted before +the server starts. ### Request Flow (POST /execute) diff --git a/README.md b/README.md index 5b9c3d2..3924ea5 100644 --- a/README.md +++ b/README.md @@ -239,7 +239,7 @@ Async frameworks (FastAPI, FastMCP, ComfyUI custom nodes, asyncio/Temporal workflows) previously had to wrap every `DazClient` call in `asyncio.to_thread()` or hand-roll an `httpx.AsyncClient` wrapper. `dazpy.aio.AsyncDazClient` mirrors `DazClient`'s full method surface — -`execute`, `execute_file`, async submit/status/result/list/cancel, render +`execute`, `execute_file`, inline/file async submit/status/result/list/cancel, render submit/batch/animation, USD export, and server-health endpoints — as native `async def` methods backed by `httpx.AsyncClient`, including the same `retry_on_busy`/`max_wait` backoff semantics (via `asyncio.sleep` @@ -250,8 +250,9 @@ from dazpy.aio import AsyncDazClient async def main(): async with AsyncDazClient() as client: - result = await client.execute("1 + 1;") - print(result.value) + request_id = await client.execute_file_async_submit("C:/jobs/pose-probe.dsa") + result = await client.get_request_result(request_id, wait=True) + print(result) ``` Requires the optional `httpx` dependency: `pip install dazpy[aio]`. @@ -784,7 +785,7 @@ See the [full migration guide](MIGRATION.md) for upgrade steps and rollback inst Long-running operations (renders, exports, batch jobs) no longer need to block the HTTP connection: -- **`POST /execute/async`** — Submit any inline script asynchronously; returns a `request_id` immediately +- **`POST /execute/async`** — Submit inline `script` or host-side `scriptFile` work asynchronously; returns a `request_id` immediately - **`POST /scripts/:id/async`** — Submit a registered script asynchronously - **`GET /requests/:id/status`** — Poll for progress (`queued`, `running`, `completed`, `failed`, `cancelled`) - **`GET /requests/:id/result`** — Fetch the final result; supports `?wait=true` to long-poll until complete @@ -1768,7 +1769,7 @@ result = requests.get(f"{BASE}/requests/{req_id}/result?wait=true&timeout=300", ### `POST /execute/async` -**Purpose:** Submit an inline script for asynchronous execution +**Purpose:** Submit an inline script or host-side script file for asynchronous execution **Authentication:** Required (if enabled) **Request Body:** Same as `POST /execute` diff --git a/dazpy/_client.py b/dazpy/_client.py index b53ba4b..9a60a0c 100644 --- a/dazpy/_client.py +++ b/dazpy/_client.py @@ -300,6 +300,27 @@ def _do(): return self._with_busy_retry(_do, retry_on_busy, max_wait) + def execute_file_async_submit( + self, script_file: str, args: object = None, *, retry_on_busy: bool = False, max_wait: float = 30.0 + ) -> str: + """Submit a host-side ``.dsa`` file for asynchronous execution. + + The file is loaded by DAZ Studio when the queued job starts, preserving + its filename for ``getScriptFileName()`` and relative ``include()`` + calls. Returns the server-assigned request id immediately; use the + request status/result/cancel methods to manage its lifecycle. + """ + payload: dict = {"scriptFile": script_file} + if args is not None: + payload["args"] = args + + def _do(): + resp = self._post("/execute/async", payload) + _raise_for_error(resp) + return resp.json().get("request_id", "") + + return self._with_busy_retry(_do, retry_on_busy, max_wait) + def get_request_status(self, request_id: str) -> dict: """Return the current status of an async request. diff --git a/dazpy/_client_aio.py b/dazpy/_client_aio.py index 9d25497..c6050bd 100644 --- a/dazpy/_client_aio.py +++ b/dazpy/_client_aio.py @@ -141,6 +141,24 @@ async def _do(): return await self._with_busy_retry(_do, retry_on_busy, max_wait) + async def execute_file_async_submit( + self, script_file: str, args: object = None, *, retry_on_busy: bool = False, max_wait: float = 30.0 + ) -> str: + """Submit a host-side ``.dsa`` file asynchronously. + + See :meth:`dazpy.DazClient.execute_file_async_submit`. + """ + payload: dict = {"scriptFile": script_file} + if args is not None: + payload["args"] = args + + async def _do(): + resp = await self._post("/execute/async", payload) + _raise_for_error(resp) + return resp.json().get("request_id", "") + + return await self._with_busy_retry(_do, retry_on_busy, max_wait) + async def get_request_status(self, request_id: str) -> dict: """See :meth:`dazpy.DazClient.get_request_status`.""" resp = await self._get(f"/requests/{request_id}/status") diff --git a/include/AsyncRequestManager.h b/include/AsyncRequestManager.h index 82a9979..d04c384 100644 --- a/include/AsyncRequestManager.h +++ b/include/AsyncRequestManager.h @@ -59,8 +59,8 @@ class AsyncRequestManager { // Enqueue a new async request. Returns SubmitResult::accepted=false when // the queue is at capacity or too many requests are tracked. - SubmitResult submit(const QString& scriptText, const QVariantMap& args, - const QString& idPrefix); + SubmitResult submit(const QString& scriptText, const QString& scriptFile, + const QVariantMap& args, const QString& idPrefix); // Enqueue a render job. Same as submit() but tags the request as // REQUEST_TYPE_RENDER so cancel dispatch calls killRender() correctly. @@ -79,9 +79,10 @@ class AsyncRequestManager { // ── Main-thread API ──────────────────────────────────────────────────── - // Dequeue next QUEUED request into outId/outScript/outArgs. + // Dequeue next QUEUED request into outId/outScript/outScriptFile/outArgs. // Sets m_currentId and returns true if work was found; false otherwise. - bool dequeueNext(QString& outId, QString& outScript, QVariantMap& outArgs); + bool dequeueNext(QString& outId, QString& outScript, QString& outScriptFile, + QVariantMap& outArgs); // Mark the running request as RUNNING (sets startedAt). void markRunning(const QString& id); @@ -144,6 +145,7 @@ class AsyncRequestManager { RequestStatus status; RequestType requestType; QString scriptText; + QString scriptFile; QVariantMap args; QVariant scriptResult; QStringList outputLines; diff --git a/include/DzScriptServerPane.h b/include/DzScriptServerPane.h index 9fe146c..418593c 100644 --- a/include/DzScriptServerPane.h +++ b/include/DzScriptServerPane.h @@ -116,13 +116,19 @@ public slots: Q_INVOKABLE HttpResult handleRegisterScript(const QByteArray& jsonBody, const QByteArray& clientIP); Q_INVOKABLE HttpResult handleRegistryExecuteRequest(const QByteArray& scriptText, const QByteArray& scriptId, const QByteArray& requestBody, const QByteArray& clientIP); - // Async enqueue helpers — called on main thread via BlockingQueuedConnection from - // AsyncExecuteHandler / AsyncScriptHandler so that all DzScript/Qt work - // stays on the Qt main thread. - Q_INVOKABLE HttpResult handleAsyncExecuteEnqueue(const QByteArray& jsonBody); + // On Studio 6 these are called directly from HTTP workers and use only local, + // reentrant Qt Core values plus mutex-protected services. Studio 4 routes them + // through BlockingQueuedConnection because its JSON parser uses QScriptEngine. + // Neither path touches GUI, DzScript, or DAZ SDK state off the main thread. + // The script limit is an immutable snapshot captured before server start. + Q_INVOKABLE HttpResult handleAsyncExecuteEnqueue(const QByteArray& jsonBody, + const QByteArray& clientIP, + int maxScriptLengthKB); Q_INVOKABLE HttpResult handleAsyncScriptEnqueue(const QByteArray& scriptBytes, const QByteArray& scriptIdBytes, const QByteArray& bodyBytes); + + // Render enqueue still builds Daz/Qt render work on the main thread. Q_INVOKABLE HttpResult handleAsyncRenderEnqueue(const QByteArray& jsonBody); Q_INVOKABLE HttpResult handleAsyncRenderBatchEnqueue(const QByteArray& jsonBody); Q_INVOKABLE HttpResult handleAsyncRenderAnimationEnqueue(const QByteArray& jsonBody); @@ -152,8 +158,10 @@ public slots: bool lookupRegistryScript(const std::string& id, std::string& outScript) const; // Async request management — called from HTTP threads (delegated to AsyncRequestManager) - // enqueueAsyncRequest still takes Qt types (used by Tier-1 async handlers — fix pending) - QString enqueueAsyncRequest(const QString& scriptText, const QVariantMap& args, + // Qt value types here are protected by AsyncRequestManager's mutex and are + // only called directly from documented reentrant Studio 6 worker paths. + QString enqueueAsyncRequest(const QString& scriptText, const QString& scriptFile, + const QVariantMap& args, const QString& idPrefix, qint64& outSubmittedAt, QString& outError); std::pair getAsyncStatusJson(const std::string& requestId) const; @@ -222,9 +230,9 @@ private slots: // Persistent DzScript instance, reused (via clear()) across every script // execution instead of constructing/destroying one per request. All - // execution is already serialized onto this (the main) thread via - // Qt::BlockingQueuedConnection — DazScript isn't thread-safe — so reuse - // here is sequential, not concurrent. See runDazScript() in the .cpp. + // execution is serialized onto this (the main) thread — synchronously for + // sync requests and by a queued wake-up for async requests — so reuse is + // sequential, not concurrent. See runDazScript() in the .cpp. DzScript* m_pPersistentScript; // ── Settings service ───────────────────────────────────────────────────── diff --git a/include/RequestHandler.h b/include/RequestHandler.h index 951babd..96bc0b1 100644 --- a/include/RequestHandler.h +++ b/include/RequestHandler.h @@ -180,10 +180,11 @@ class ScriptExecuteHandler : public IRequestHandler { class AsyncExecuteHandler : public IRequestHandler { public: - explicit AsyncExecuteHandler(DzScriptServerPane* pane); + AsyncExecuteHandler(DzScriptServerPane* pane, int maxScriptLengthKB); void handle(HttpContext& ctx) override; private: DzScriptServerPane* m_pPane; + const int m_maxScriptLengthKB; }; class AsyncScriptHandler : public IRequestHandler { diff --git a/openapi.yaml b/openapi.yaml index 37524da..b10d169 100644 --- a/openapi.yaml +++ b/openapi.yaml @@ -271,7 +271,7 @@ paths: /execute/async: post: tags: [Async] - summary: Submit an inline script for asynchronous execution + summary: Submit an inline script or host-side script file for asynchronous execution operationId: executeAsync security: - apiToken: [] diff --git a/src/AsyncRequestManager.cpp b/src/AsyncRequestManager.cpp index 263043d..1a99e67 100644 --- a/src/AsyncRequestManager.cpp +++ b/src/AsyncRequestManager.cpp @@ -20,7 +20,8 @@ AsyncRequestManager::AsyncRequestManager(QObject* notifyTarget) // ─── HTTP-thread API ────────────────────────────────────────────────────────── AsyncRequestManager::SubmitResult AsyncRequestManager::submit( - const QString& scriptText, const QVariantMap& args, const QString& idPrefix) + const QString& scriptText, const QString& scriptFile, + const QVariantMap& args, const QString& idPrefix) { SubmitResult r; r.accepted = false; @@ -44,6 +45,7 @@ AsyncRequestManager::SubmitResult AsyncRequestManager::submit( AsyncRequest req; req.id = MetricsCollector::generateAsyncId(idPrefix); req.scriptText = scriptText; + req.scriptFile = scriptFile; req.args = args; req.submittedAt = QDateTime::currentMSecsSinceEpoch(); @@ -385,7 +387,8 @@ int AsyncRequestManager::getTotalTracked() const // ─── Main-thread API ────────────────────────────────────────────────────────── -bool AsyncRequestManager::dequeueNext(QString& outId, QString& outScript, QVariantMap& outArgs) +bool AsyncRequestManager::dequeueNext(QString& outId, QString& outScript, + QString& outScriptFile, QVariantMap& outArgs) { QMutexLocker locker(&m_mutex); if (!m_currentId.isEmpty()) return false; @@ -397,6 +400,7 @@ bool AsyncRequestManager::dequeueNext(QString& outId, QString& outScript, QVaria const AsyncRequest& req = m_requests.value(id); outId = id; outScript = req.scriptText; + outScriptFile = req.scriptFile; outArgs = req.args; return true; } diff --git a/src/DzScriptServerPane.cpp b/src/DzScriptServerPane.cpp index 1e683c2..c4da053 100644 --- a/src/DzScriptServerPane.cpp +++ b/src/DzScriptServerPane.cpp @@ -107,7 +107,10 @@ struct ScriptRunResult { QStringList output; // SDK6 only — see below; SDK4 still uses the debugMsg signal capture }; -static ScriptRunResult runDazScript(DzScript* script, const QVariantMap& argsMap) +static ScriptRunResult runDazScript( + DzScript* script, + const QVariantMap& argsMap, + const QString& scriptFilename) { ScriptRunResult r; #if DAZ_SDK_MAJOR_VERSION >= 6 @@ -123,22 +126,32 @@ static ScriptRunResult runDazScript(DzScript* script, const QVariantMap& argsMap QVariantList argsList; argsList << QVariant(argsMap); QString argsJson = QString::fromStdString(JsonStd::variantToJson(QVariant(argsList))); + // evaluate() runs the wrapper as anonymous source on SDK6, so the engine's + // built-in getScriptFileName() sees no filename even after loadFromFile(). + // Carry the request's explicit filename into the evaluated program. Do not + // read it back from the reused DzScript instance: the SDK does not document + // whether clear() resets a filename retained by a preceding file-backed job. + // This also keeps file-backed scripts' relative-loader helpers working while + // guaranteeing that an inline job reports an empty filename. + QString filenameJson = QString::fromStdString( + JsonStd::variantToJson(QVariant(scriptFilename))); QString codeLiteral = QString::fromStdString( "\"" + JsonStd::escape(JsonStd::qstrToStd(script->getCode())) + "\""); QString shimmed = QString( "var __dss_output = [];\n" "function print(msg) { __dss_output.push(String(msg)); }\n" "function getArguments(){ return %1; }\n" + "function getScriptFileName(){ return %2; }\n" "var __dss_result = null, __dss_error = null, __dss_errorLine = 0;\n" "try {\n" - " __dss_result = eval(%2);\n" + " __dss_result = eval(%3);\n" "} catch (e) {\n" " __dss_error = (e && e.message !== undefined) ? String(e.message) : String(e);\n" " __dss_errorLine = (e && e.lineNumber) ? e.lineNumber : 0;\n" "}\n" "JSON.stringify({ result: __dss_result, output: __dss_output, " "error: __dss_error, errorLine: __dss_errorLine });" - ).arg(argsJson, codeLiteral); + ).arg(argsJson, filenameJson, codeLiteral); QJSValue evalResult = script->evaluate(shimmed); if (evalResult.isError()) { @@ -173,8 +186,12 @@ static ScriptRunResult runDazScript(DzScript* script, const QVariantMap& argsMap QVariantList argsList; argsList << QVariant(argsMap); QString argsJson = QString::fromStdString(JsonStd::variantToJson(QVariant(argsList))); - QString shimmed = QString("function getArguments(){ return %1; }\n%2") - .arg(argsJson, script->getCode()); + QString filenameJson = QString::fromStdString( + JsonStd::variantToJson(QVariant(scriptFilename))); + QString shimmed = QString( + "function getArguments(){ return %1; }\n" + "function getScriptFileName(){ return %2; }\n%3") + .arg(argsJson, filenameJson, script->getCode()); script->setCode(shimmed); if (script->execute()) { r.success = true; @@ -882,9 +899,12 @@ static void applyContext(const HttpContext& ctx, httplib::Response& res) // ─── Route setup ────────────────────────────────────────────────────────────── // // THREADING RULE: httplib invokes these handlers on raw std::threads (not QThreads). -// Handlers must do NO Qt work beyond calling handler/middleware methods that are -// themselves designed for HTTP-thread use (mutex-protected data, QueuedConnection logs). -// All DzScript execution happens on the main thread via Qt::BlockingQueuedConnection. +// Sync handlers cross to the main thread with BlockingQueuedConnection. On +// Studio 6, async script enqueue handlers may build local reentrant Qt Core +// values and call mutex-protected services; Studio 4 crosses to the main thread +// because its JSON parser uses QScriptEngine. Execution later reaches the main +// thread through a queued wake-up. No worker may touch GUI, DzScript, DAZ SDK, +// or shared mutable pane state. void DzScriptServerPane::setupRoutes() { @@ -912,7 +932,9 @@ void DzScriptServerPane::setupRoutes() m_pScriptListHandler.reset(new ScriptListHandler(this)); m_pScriptDeleteHandler.reset(new ScriptDeleteHandler(this)); m_pScriptExecHandler.reset(new ScriptExecuteHandler(this)); - m_pAsyncExecHandler.reset(new AsyncExecuteHandler(this)); + // Snapshot the GUI-owned limit before httplib starts its worker threads. + // The corresponding control is disabled while the server is running. + m_pAsyncExecHandler.reset(new AsyncExecuteHandler(this, m_nMaxScriptLengthKB)); m_pAsyncScriptHandler.reset(new AsyncScriptHandler(this)); m_pAsyncStatusHandler.reset(new AsyncStatusHandler(this)); m_pAsyncResultHandler.reset(new AsyncResultHandler(this)); @@ -1442,12 +1464,14 @@ bool DzScriptServerPane::lookupRegistryScript(const std::string& id, std::string // ─── Async Request public API (called from HTTP threads) ────────────────────── QString DzScriptServerPane::enqueueAsyncRequest(const QString& scriptText, + const QString& scriptFile, const QVariantMap& args, const QString& idPrefix, qint64& outSubmittedAt, QString& outError) { - AsyncRequestManager::SubmitResult r = m_pAsyncMgr->submit(scriptText, args, idPrefix); + AsyncRequestManager::SubmitResult r = m_pAsyncMgr->submit( + scriptText, scriptFile, args, idPrefix); outSubmittedAt = r.submittedAt; outError = r.error; if (!r.accepted) { @@ -1570,7 +1594,7 @@ HttpResult DzScriptServerPane::handleExecuteRequest(const QByteArray& jsonBody, // Args are accessible in scripts via getArguments()[0], since // DzScriptContext methods are available as globals in every DzScript. - ScriptRunResult runResult = runDazScript(m_pPersistentScript, argsMap); + ScriptRunResult runResult = runDazScript(m_pPersistentScript, argsMap, scriptFile); #if DAZ_SDK_MAJOR_VERSION >= 6 m_aCapturedLogLines = runResult.output; // SDK6: evaluate() bypasses the debugMsg signal #endif @@ -1703,7 +1727,7 @@ HttpResult DzScriptServerPane::handleRegistryExecuteRequest( ensurePersistentScript()->clear(); m_pPersistentScript->setCode(QString::fromUtf8(scriptText.constData(), scriptText.size())); - ScriptRunResult runResult = runDazScript(m_pPersistentScript, argsMap); + ScriptRunResult runResult = runDazScript(m_pPersistentScript, argsMap, QString()); #if DAZ_SDK_MAJOR_VERSION >= 6 m_aCapturedLogLines = runResult.output; // SDK6: evaluate() bypasses the debugMsg signal #endif @@ -1752,7 +1776,10 @@ static HttpResult buildQueuedResponse(const QString& requestId, qint64 submitted return HttpResult(200, QByteArray(resp.c_str(), (int)resp.size())); } -HttpResult DzScriptServerPane::handleAsyncExecuteEnqueue(const QByteArray& jsonBody) +HttpResult DzScriptServerPane::handleAsyncExecuteEnqueue( + const QByteArray& jsonBody, + const QByteArray& clientIP, + int maxScriptLengthKB) { QVariantMap body; std::string parseErrDetail; @@ -1760,21 +1787,34 @@ HttpResult DzScriptServerPane::handleAsyncExecuteEnqueue(const QByteArray& jsonB return HttpResult(400, stdToQBA(ErrorResponse::build(ErrorCode::INVALID_JSON))); } - QString scriptText = body.value("script").toString(); + QString scriptFile = body.value("scriptFile").toString(); + QString scriptText = body.value("script").toString(); + const bool bothProvided = !scriptFile.isEmpty() && !scriptText.isEmpty(); - ValidationResult vr = RequestValidator::validateRequiredField(scriptText, "script"); + ValidationResult vr = RequestValidator::validateExecuteFields( + scriptFile, scriptText, maxScriptLengthKB); if (!vr.valid) return HttpResult(vr.httpStatus(), stdToQBA(vr.toErrorJson())); qint64 submittedAt = 0; QString enqueueError; QString requestId = enqueueAsyncRequest( - scriptText, body.value("args").toMap(), "execute", submittedAt, enqueueError); + scriptText, scriptFile, body.value("args").toMap(), "execute", + submittedAt, enqueueError); if (requestId.isEmpty()) return HttpResult(503, stdToQBA(ErrorResponse::build( ErrorCode::SERVER_UNAVAILABLE, JsonStd::qstrToStd(enqueueError)))); + if (bothProvided) { + std::string logLine = "[" + JsonStd::currentTime() + "] [" + + std::string(clientIP.constData(), clientIP.size()) + "] [WARN] [" + + JsonStd::qstrToStd(requestId) + + "] Both scriptFile and script provided; using scriptFile"; + QMetaObject::invokeMethod(this, "appendLogBytes", Qt::QueuedConnection, + Q_ARG(QByteArray, QByteArray(logLine.c_str(), (int)logLine.size()))); + } + return buildQueuedResponse(requestId, submittedAt); } @@ -1797,15 +1837,17 @@ HttpResult DzScriptServerPane::handleAsyncScriptEnqueue( qint64 submittedAt = 0; QString enqueueError; QString requestId = enqueueAsyncRequest( - scriptText, argsMap, "script", submittedAt, enqueueError); + scriptText, QString(), argsMap, "script", submittedAt, enqueueError); if (requestId.isEmpty()) return HttpResult(503, stdToQBA(ErrorResponse::build( ErrorCode::SERVER_UNAVAILABLE, JsonStd::qstrToStd(enqueueError)))); - appendLog(QString("[%1] [ASYNC QUEUED] script:%2 -> %3") - .arg(QDateTime::currentDateTime().toString("HH:mm:ss")) - .arg(scriptId).arg(requestId)); + std::string logLine = "[" + JsonStd::currentTime() + + "] [ASYNC QUEUED] script:" + JsonStd::qstrToStd(scriptId) + + " -> " + JsonStd::qstrToStd(requestId); + QMetaObject::invokeMethod(this, "appendLogBytes", Qt::QueuedConnection, + Q_ARG(QByteArray, QByteArray(logLine.c_str(), (int)logLine.size()))); return buildQueuedResponse(requestId, submittedAt); } @@ -2242,7 +2284,7 @@ HttpResult DzScriptServerPane::handleAsyncRenderEnqueue(const QByteArray& jsonBo ensurePersistentScript()->clear(); m_pPersistentScript->setCode(validateScript); - ScriptRunResult valRunResult = runDazScript(m_pPersistentScript, QVariantMap()); + ScriptRunResult valRunResult = runDazScript(m_pPersistentScript, QVariantMap(), QString()); if (valRunResult.success) { QString valResult = valRunResult.result.toString(); if (valResult.startsWith("NOT_FOUND:")) { @@ -2429,7 +2471,7 @@ HttpResult DzScriptServerPane::handleAsyncRenderBatchEnqueue(const QByteArray& j "})()"; ensurePersistentScript()->clear(); m_pPersistentScript->setCode(validateScript); - ScriptRunResult valRunResult = runDazScript(m_pPersistentScript, QVariantMap()); + ScriptRunResult valRunResult = runDazScript(m_pPersistentScript, QVariantMap(), QString()); if (valRunResult.success) { QString valResult = valRunResult.result.toString(); if (valResult.startsWith("NOT_FOUND:")) { @@ -2559,7 +2601,7 @@ HttpResult DzScriptServerPane::handleAsyncRenderAnimationEnqueue(const QByteArra "})()"; ensurePersistentScript()->clear(); m_pPersistentScript->setCode(validateScript); - ScriptRunResult valRunResult = runDazScript(m_pPersistentScript, QVariantMap()); + ScriptRunResult valRunResult = runDazScript(m_pPersistentScript, QVariantMap(), QString()); if (valRunResult.success && valRunResult.result.toString() == "NOT_FOUND") return HttpResult(400, stdToQBA(ErrorResponse::build(ErrorCode::INVALID_FIELD, "Camera not found in scene: " + JsonStd::qstrToStd(camera)))); @@ -2885,9 +2927,8 @@ std::string DzScriptServerPane::mainThreadBusyMessage() const // ─── Async Execution (main thread) ─────────────────────────────────────────── -// Called on the main thread via Qt::QueuedConnection (connected to -// AsyncRequestManager::requestEnqueued signal) and self-reposted after each -// execution completes to drain the queue. +// Called on the main thread by AsyncRequestManager's queued wake-up and +// self-reposted after each execution completes to drain the queue. // // Blocks the main thread (Qt event loop) for the full duration of each script. // That is intentional — DAZ Studio's DzScript API is not thread-safe. HTTP @@ -2895,9 +2936,9 @@ std::string DzScriptServerPane::mainThreadBusyMessage() const // mutex-protected map and are unaffected. void DzScriptServerPane::processNextAsyncRequest() { - QString id, scriptText; + QString id, scriptText, scriptFile; QVariantMap args; - if (!m_pAsyncMgr->dequeueNext(id, scriptText, args)) return; + if (!m_pAsyncMgr->dequeueNext(id, scriptText, scriptFile, args)) return; // Request may have been cancelled while queued if (m_pAsyncMgr->isCancelRequested(id)) { @@ -2923,9 +2964,14 @@ void DzScriptServerPane::processNextAsyncRequest() Qt::DirectConnection); ensurePersistentScript()->clear(); - m_pPersistentScript->setCode(scriptText); - - ScriptRunResult runResult = runDazScript(m_pPersistentScript, args); + ScriptRunResult runResult; + if (!scriptFile.isEmpty() && !m_pPersistentScript->loadFromFile(scriptFile)) { + runResult.errorMessage = QString("Failed to load script file: %1").arg(scriptFile); + } else { + if (scriptFile.isEmpty()) + m_pPersistentScript->setCode(scriptText); + runResult = runDazScript(m_pPersistentScript, args, scriptFile); + } #if DAZ_SDK_MAJOR_VERSION >= 6 m_aCapturedLogLines = runResult.output; // SDK6: evaluate() bypasses the debugMsg signal #endif diff --git a/src/RequestHandlers.cpp b/src/RequestHandlers.cpp index 70e631e..382b136 100644 --- a/src/RequestHandlers.cpp +++ b/src/RequestHandlers.cpp @@ -229,17 +229,33 @@ void ScriptExecuteHandler::handle(HttpContext& ctx) // ───────────────────────────────────────────────────────────────────────────── -AsyncExecuteHandler::AsyncExecuteHandler(DzScriptServerPane* pane) : m_pPane(pane) {} +AsyncExecuteHandler::AsyncExecuteHandler(DzScriptServerPane* pane, int maxScriptLengthKB) + : m_pPane(pane), m_maxScriptLengthKB(maxScriptLengthKB) {} void AsyncExecuteHandler::handle(HttpContext& ctx) { - if (respondIfMainThreadBusy(m_pPane, ctx)) return; QByteArray bodyBytes(ctx.body.c_str(), (int)ctx.body.size()); +#if DAZ_SDK_MAJOR_VERSION >= 6 + // Enqueue is deliberately worker-thread-safe. Sending this through a + // BlockingQueuedConnection makes an "async" submit wait behind the very + // main-thread job it is meant to queue after, which also makes queued + // cancellation impossible while Daz is busy. + QByteArray clientIPBytes(ctx.remoteAddr.c_str(), (int)ctx.remoteAddr.size()); + HttpResult result = m_pPane->handleAsyncExecuteEnqueue( + bodyBytes, clientIPBytes, m_maxScriptLengthKB); +#else + // Qt 4's JsonStd parser uses QScriptEngine, so Studio 4 must retain the + // main-thread crossing even though Studio 6 can enqueue directly. + if (respondIfMainThreadBusy(m_pPane, ctx)) return; + QByteArray clientIPBytes(ctx.remoteAddr.c_str(), (int)ctx.remoteAddr.size()); HttpResult result; QMetaObject::invokeMethod(m_pPane, "handleAsyncExecuteEnqueue", Qt::BlockingQueuedConnection, Q_RETURN_ARG(HttpResult, result), - Q_ARG(QByteArray, bodyBytes)); + Q_ARG(QByteArray, bodyBytes), + Q_ARG(QByteArray, clientIPBytes), + Q_ARG(int, m_maxScriptLengthKB)); +#endif ctx.respond(result.first, std::string(result.second.constData(), result.second.size())); } @@ -249,7 +265,6 @@ AsyncScriptHandler::AsyncScriptHandler(DzScriptServerPane* pane) : m_pPane(pane) void AsyncScriptHandler::handle(HttpContext& ctx) { - if (respondIfMainThreadBusy(m_pPane, ctx)) return; std::string scriptText; if (!m_pPane->lookupRegistryScript(ctx.urlMatch, scriptText)) { ctx.respond(404, ErrorResponse::build(ErrorCode::SCRIPT_NOT_FOUND, ctx.urlMatch)); @@ -259,6 +274,11 @@ void AsyncScriptHandler::handle(HttpContext& ctx) QByteArray scriptBytes(scriptText.c_str(), (int)scriptText.size()); QByteArray scriptIdBytes(ctx.urlMatch.c_str(), (int)ctx.urlMatch.size()); QByteArray bodyBytes(ctx.body.c_str(), (int)ctx.body.size()); +#if DAZ_SDK_MAJOR_VERSION >= 6 + HttpResult result = m_pPane->handleAsyncScriptEnqueue( + scriptBytes, scriptIdBytes, bodyBytes); +#else + if (respondIfMainThreadBusy(m_pPane, ctx)) return; HttpResult result; QMetaObject::invokeMethod(m_pPane, "handleAsyncScriptEnqueue", Qt::BlockingQueuedConnection, @@ -266,6 +286,7 @@ void AsyncScriptHandler::handle(HttpContext& ctx) Q_ARG(QByteArray, scriptBytes), Q_ARG(QByteArray, scriptIdBytes), Q_ARG(QByteArray, bodyBytes)); +#endif ctx.respond(result.first, std::string(result.second.constData(), result.second.size())); } diff --git a/tests/test_api.py b/tests/test_api.py index 6ccac8c..b59226f 100644 --- a/tests/test_api.py +++ b/tests/test_api.py @@ -16,6 +16,7 @@ import os import sys +import tempfile import threading import time import unittest @@ -278,11 +279,13 @@ def test_both_script_and_scriptfile_uses_scriptfile(self): # ─── Async execution ───────────────────────────────────────────────────────── -def async_execute(script=None, args=None, headers=None): +def async_execute(script=None, script_file=None, args=None, headers=None): """POST /execute/async and return the raw response.""" payload = {} if script is not None: payload["script"] = script + if script_file is not None: + payload["scriptFile"] = script_file if args is not None: payload["args"] = args h = auth_headers() if headers is None else headers @@ -355,6 +358,118 @@ def test_async_args_accessible(self): body = result_r.json() self.assertEqual(body.get("result"), 99) + def test_queued_request_can_be_cancelled_while_main_thread_runs(self): + blocker = async_execute(script=iife( + "var start = Date.now(); " + "while (Date.now() - start < 4000) {} " + "return 'blocker-complete';" + )) + self.assertEqual(blocker.status_code, 200, blocker.text) + blocker_id = blocker.json()["request_id"] + + submit_started = time.monotonic() + queued = async_execute(script=iife("return 'must-not-run';")) + submit_seconds = time.monotonic() - submit_started + self.assertEqual(queued.status_code, 200, queued.text) + queued_id = queued.json()["request_id"] + + status = requests.get( + f"{BASE_URL}/requests/{queued_id}/status", + headers=auth_headers(), timeout=5, + ).json() + cancelled = requests.delete( + f"{BASE_URL}/requests/{queued_id}", + headers=auth_headers(), timeout=5, + ) + final = poll_status(queued_id, timeout=10) + blocker_result = get_result(blocker_id, wait=True, timeout=10).json() + + self.assertLess(submit_seconds, 2.0) + self.assertEqual(status.get("status"), "queued") + self.assertEqual(cancelled.status_code, 200) + self.assertEqual(final.get("status"), "cancelled") + self.assertEqual(blocker_result.get("status"), "completed") + + def test_async_scriptfile_preserves_file_identity(self): + script_path = "" + try: + with tempfile.NamedTemporaryFile( + mode="w", suffix=".dsa", delete=False, encoding="utf-8" + ) as script_file: + script_file.write(iife("return getScriptFileName();")) + script_path = os.path.abspath(script_file.name) + + r = async_execute(script_file=script_path) + self.assertEqual(r.status_code, 200) + request_id = r.json()["request_id"] + final = poll_status(request_id, timeout=20) + self.assertEqual(final["status"], "completed") + + body = get_result(request_id).json() + self.assertTrue(body.get("success")) + self.assertEqual( + os.path.normcase(os.path.normpath(body.get("result"))), + os.path.normcase(os.path.normpath(script_path)), + ) + finally: + if script_path and os.path.exists(script_path): + os.unlink(script_path) + + def test_async_inline_after_scriptfile_has_empty_file_identity(self): + script_path = "" + try: + with tempfile.NamedTemporaryFile( + mode="w", suffix=".dsa", delete=False, encoding="utf-8" + ) as script_file: + script_file.write(iife("return getScriptFileName();")) + script_path = os.path.abspath(script_file.name) + + file_request = async_execute( + script=iife("return 'inline-should-not-run';"), + script_file=script_path, + ) + self.assertEqual(file_request.status_code, 200, file_request.text) + file_result = get_result( + file_request.json()["request_id"], wait=True, timeout=20 + ).json() + self.assertTrue(file_result.get("success"), file_result.get("error")) + self.assertEqual( + os.path.normcase(os.path.normpath(file_result.get("result"))), + os.path.normcase(os.path.normpath(script_path)), + ) + + inline_request = async_execute( + script=iife("return getScriptFileName();") + ) + self.assertEqual(inline_request.status_code, 200, inline_request.text) + inline_result = get_result( + inline_request.json()["request_id"], wait=True, timeout=20 + ).json() + self.assertTrue(inline_result.get("success"), inline_result.get("error")) + self.assertEqual(inline_result.get("result"), "") + finally: + if script_path and os.path.exists(script_path): + os.unlink(script_path) + + def test_sync_scriptfile_preserves_file_identity(self): + script_path = "" + try: + with tempfile.NamedTemporaryFile( + mode="w", suffix=".dsa", delete=False, encoding="utf-8" + ) as script_file: + script_file.write(iife("return getScriptFileName();")) + script_path = os.path.abspath(script_file.name) + + body = execute(script_file=script_path).json() + self.assertTrue(body.get("success")) + self.assertEqual( + os.path.normcase(os.path.normpath(body.get("result"))), + os.path.normcase(os.path.normpath(script_path)), + ) + finally: + if script_path and os.path.exists(script_path): + os.unlink(script_path) + def test_async_script_error_gives_failed_status(self): r = async_execute(script="this is not valid dazscript !!!") request_id = r.json()["request_id"] diff --git a/tests/test_dazpy.py b/tests/test_dazpy.py index ce3bb9b..0172687 100644 --- a/tests/test_dazpy.py +++ b/tests/test_dazpy.py @@ -63,6 +63,36 @@ from dazpy import exceptions +class TestDazClientAsyncFileSubmit(unittest.TestCase): + def test_posts_script_file_to_async_endpoint_and_returns_request_id(self): + client = DazClient(token="") + original_session = client._session + client._session = MagicMock() + response = MagicMock() + response.status_code = 200 + response.json.return_value = {"request_id": "execute-file-123", "status": "queued"} + response.headers = {} + client._session.post.return_value = response + + try: + request_id = client.execute_file_async_submit( + "C:/scripts/pose-probe.dsa", args={"mode": "probe"} + ) + finally: + original_session.close() + + self.assertEqual(request_id, "execute-file-123") + client._session.post.assert_called_once_with( + "http://127.0.0.1:18811/execute/async", + json={ + "scriptFile": "C:/scripts/pose-probe.dsa", + "args": {"mode": "probe"}, + }, + headers={}, + timeout=30.0, + ) + + def _make_client(return_value=None, output=None): client = MagicMock(spec=DazClient) client.execute.return_value = ExecutionResult( diff --git a/tests/test_dazpy_aio.py b/tests/test_dazpy_aio.py index 86f456c..1c0674e 100644 --- a/tests/test_dazpy_aio.py +++ b/tests/test_dazpy_aio.py @@ -72,6 +72,23 @@ async def test_execute_file(self): _, kwargs = mock_http.post.call_args assert kwargs["json"] == {"scriptFile": "C:/scripts/foo.dsa"} + @pytest.mark.asyncio + async def test_execute_file_async_submit(self): + client, mock_http = _client_with_mock_http() + mock_http.post.return_value = _mock_resp( + json_data={"request_id": "execute-file-123", "status": "queued"} + ) + request_id = await client.execute_file_async_submit( + "C:/scripts/foo.dsa", args={"mode": "probe"} + ) + assert request_id == "execute-file-123" + args, kwargs = mock_http.post.call_args + assert args[0] == "http://127.0.0.1:18811/execute/async" + assert kwargs["json"] == { + "scriptFile": "C:/scripts/foo.dsa", + "args": {"mode": "probe"}, + } + @pytest.mark.asyncio async def test_auth_error_401(self): client, mock_http = _client_with_mock_http(token="bad")