Skip to content

thingworx-field-work/Python-Edge-Machine-Learning

Repository files navigation

Edge ML interface: ThingWorx + OPC UA + PMML

A complete Python application for edge ML interface that:

  1. Subscribes to OPC UA data (Kepware or compatible server)
  2. Loads and runs a trained PMML model (via sklearn-pmml-model or pypmml)
  3. Writes predictions back to a ThingWorx Platform Thing

Quick Start

Prerequisites

  • Python 3.11+ (3.12 recommended)
  • ThingWorx Platform instance (local or remote)
  • OPC UA server (Kepware, Prosys OPC UA Simulation Server, etc.)
  • A trained PMML model file (.xml)

Installation

  1. Create and activate a Python virtual environment:

    cd c:\Dev\Python\thingworx-python
    python -m venv .venv
    .venv\Scripts\Activate.ps1
  2. Install Python dependencies:

    pip install -r ML/requirements.txt
  3. Install the ThingWorx Python wrapper (editable): From the repository root:

    pip install -e .
  4. (Optional) Install pypmml + Java runtime Only required if your PMML model is not supported by sklearn-pmml-model (e.g. KNIME, SAS, R caret models):

    pip install pypmml

    Also install a Java 8+ JRE and make sure java is on PATH.

  5. Build and locate the ThingWorx C SDK:

    • The SDK library (twCSdk.dll on Windows, libtwCSdk.so on Linux) must be available
    • Set the TWCSDK_LIB_PATH environment variable to its directory, or place it in the current working directory:
      $env:TWCSDK_LIB_PATH = "C:\path\to\sdk\lib"
    • The repo already ships a Windows copy at ML/twCSdk.dll alongside libcrypto-3-x64.dll and libssl-3-x64.dll; running from the ML/ directory will pick those up automatically.

Configuration

Edit config.json with your environment details:

{
  "thingworx": {
    "host": "localhost",              // ThingWorx server hostname
    "port": 8016,                     // 8016 (HTTP), 443 (HTTPS)
    "app_key": "YOUR-APP-KEY",        // Application key for auth
    "thing_name": "EdgeMLThing",      // Thing name on platform
    "encryption": false,              // Disable for port 8016
    "input_properties": [             // Read from platform and pass to model
      {
        "name": "OperatingMode",
        "type": "STRING",
        "model_input_field": "operating_mode",
        "read_interval_s": 5
      }
    ],
    "output_properties": [            // Write model predictions here
      {
        "name": "ElectricalOutput_mo",
        "type": "NUMBER",
        "model_output_field": "ElectricalOutput_mo",
        "push_type": "ALWAYS"
      }
    ]
  },
  "opcua": {
    "server_url": "opc.tcp://server:49320",
    "subscription_ms": 500,
    "keepalive_interval_sec": 30,
    "initial_reconnect_delay": 5.0,
    "max_reconnect_delay": 60.0,
    "reconnect_backoff_multiplier": 1.5,
    "security": {
      "mode": "username",            // "none" | "username" | "cert" | "username_cert"
      "username": "OPCUAClient",
      "password": "secret"
    },
    "nodes": [
      {
        "node_id": "ns=2;s=SIG.SN0001.Temperature",
        "name": "AmbientTemperature",
        "type": "NUMBER",
        "mode": "subscription"        // "subscription" (default) or "polling"
      },
      {
        "node_id": "ns=2;s=SIG.SN0001.Pressure",
        "name": "AmbientPressure",
        "type": "NUMBER",
        "mode": "polling",
        "polling_interval_ms": 1000
      }
    ]
  },
  "model": {
    "pmml_file": "ccpp_model.xml",
    "backend": "auto",               // "auto", "sklearn-pmml-model", "pypmml"
    "trigger_on": "all",             // "any" or "all" (all OPC nodes first)
    "input_mapping": {               // OPC UA node name → PMML field name
      "AmbientTemperature": "AmbientTemperature"
    },
    "output_mapping": {              // PMML output → ThingWorx property
      "ElectricalOutput_mo": "ElectricalOutput_mo"
    }
  }
}

Running

Start the inference engine:

python edge_inference.py [--config config.json]

Monitor output:

2026-04-15T09:12:02 [INFO ] edge: Starting edge_inference  config=config.json  Python=3.12.0
2026-04-15T09:12:02 [INFO ] edge.model: Loaded PMML model via backend=sklearn-pmml-model
2026-04-15T09:12:02 [INFO ] edge.thingworx: Connected to ThingWorx localhost:8016  thing=EdgeMLThing
2026-04-15T09:12:03 [INFO ] edge.opcua: OPC UA connected: opc.tcp://server:49320/Discovery
2026-04-15T09:12:03 [INFO ] edge.inference: Subscribed to 4 OPC UA node(s) at 500ms interval
2026-04-15T09:12:03 [INFO ] edge: Edge inference running. Press Ctrl+C or type 'q'/'exit' to stop.

Stop gracefully:

  • Press Ctrl+C, or
  • Type q or exit and press Enter

Data Flow

OPC UA Server (Kepware)
    ↓ asyncio subscription (datachange_notification)
    ↓
OpcUaManager
    ↓ asyncio.create_task()
    ↓
InferenceEngine._run_inference()
    ├─ Merge OPC UA values + ThingWorx input properties
    ├─ Apply model.predict(inputs) via sklearn-pmml-model
    ├─ Map output fields to ThingWorx property names
    ↓
ThingWorxManager.write_properties()
    ├─ Run in executor (C SDK is sync, event loop safe)
    ↓
ThingWorx Platform Thing
    └─ Properties updated on platform (subscribed clients notified)

Diagnostic Tools

Test ThingWorx Connection

Validate Thing configuration on the platform:

python test_connection.py [--config config.json]

Output: Writes sentinel values to each output property, reads them back, reads input properties.

Test PMML Model

Load and run the model independently:

python test_pmml.py [--config config.json]

Output: Loads PMML, generates synthetic test data, runs inference, validates output fields.

Inspect a PMML File

Parse a .pmml / .xml model and list the declared input and output variables. Useful for:

  • Building the input_mapping / output_mapping blocks in config.json
  • Verifying that a model contains the fields your OPC UA pipeline provides
  • Quickly comparing two model versions without loading a PMML backend

It reads three sections of the PMML document and deduplicates variables by name (segmented / ensemble models often repeat the same MiningField across sub-models):

Section What it reports
DataDictionary / DataField Every declared field, with dataType and optype
MiningSchema / MiningField Active fields → Inputs, predicted/target fields → Outputs
Output / OutputField Explicit outputs (predicted value, probability, decision) with feature and targetField

Any field that appears as both an input and an output is treated as an output only.

Command:

python inspect_pmml.py <pmml_file> [--json]

Parameters:

Parameter Required Description
pmml_file yes Path to the PMML / XML model file
--json no Emit a machine-readable JSON document instead of the human-readable report (useful for scripting config.json generation)

Exit codes: 0 on success, 1 if the file does not exist, 2 if the file is not valid XML.

Example:

python inspect_pmml.py ccpp_model.xml
PMML file    : ccpp_model.xml
PMML version : 4.3

DataDictionary (5 fields)
  name=AmbientTemperature   dataType=double  optype=continuous
  name=ExhaustVacuum         dataType=double  optype=continuous
  name=AmbientPressure       dataType=double  optype=continuous
  name=RelativeHumidity      dataType=double  optype=continuous
  name=ElectricalOutput_mo   dataType=double  optype=continuous

Inputs (4 active MiningFields)
  name=AmbientTemperature   dataType=-  optype=-
  name=ExhaustVacuum         dataType=-  optype=-
  name=AmbientPressure       dataType=-  optype=-
  name=RelativeHumidity      dataType=-  optype=-

Outputs (1 fields)
  name=ElectricalOutput_mo   feature=predictedValue  dataType=double  targetField=-

For scripted use, --json returns the same information as a structured object (file, pmml_version, data_fields, inputs, outputs).


Performance & Timing

Inference Cycle Timing

Each inference cycle is logged with two metrics:

  • model_ms: Time spent in model.predict() (PMML backend execution)
  • total_ms: Total cycle time (OPC input fetch → model → ThingWorx write)

Example log entry:

[INFO ] edge.inference: Inference cycle completed  model=2.34ms  total=45.67ms

Typical timings (on modern edge hardware):

  • sklearn-pmml-model: 1–5 ms for inference
  • pypmml (with Java): 5–20 ms for inference
  • Total cycle (including ThingWorx I/O): 10–100 ms

Optimizations Applied

  1. No-op property callback — Properties without explicit handlers use a lightweight no-op callback instead of NULL, avoiding SDK validation errors.
  2. BatchedThingWorx reads — Input properties are read periodically (configurable read_interval_s), not on every inference.
  3. Asyncio event loop — OPC UA subscriptions and ThingWorx calls don't block the main loop.
  4. executor-based C SDK calls — Synchronous C SDK calls run in a thread pool, keeping the event loop responsive.

Configuration Details

ThingWorx Section

Key Type Default Notes
host str Hostname or IP of ThingWorx server
port int 443 8016 for HTTP, 443 for HTTPS
app_key str Application key (authentication)
thing_name str Thing name (must exist on platform or use RemoteThingWithAllFeatures)
resource str /Thingworx/WS WebSocket endpoint path on the platform
gateway_name str PythonEdgeGateway Registered gateway name reported to ThingWorx
gateway_type str | null null Gateway type string (optional)
encryption bool true Set false for port 8016
cert_validation bool true Validate platform server certificate
self_signed_ok bool false Accept self-signed certs (dev only)
auto_reconnect bool true C SDK auto-reconnect on WebSocket loss
connect_timeout_ms int 10000 Connection timeout
connect_retries int 3 Initial connection retries
reconnect_interval_s int 30 Python-side reconnect backoff after write failure
property_write_timeout_ms int 5000 Timeout per property write
message_chunk_size int 8192 SDK message chunk size
frame_size int 8192 SDK WebSocket frame size
offline_msg_store_dir str | null null Directory to buffer outgoing messages while offline
input_properties list [] Properties to read from platform and pass to model
output_properties list [] Properties to write model predictions to

input_properties[] entry:

Key Required Notes
name yes ThingWorx property name
type yes NUMBER | INTEGER | LONG | STRING | BOOLEAN | DATETIME | BLOB
model_input_field no PMML field name to map the value to (defaults to name)
read_interval_s no Per-property refresh interval; the smallest value across all input properties is used

output_properties[] entry:

Key Required Notes
name yes ThingWorx property name
type yes Same values as above
model_output_field no PMML output field this property mirrors (used when output_mapping is not set)
description no Property description sent to the platform
push_type no ALWAYS (default), VALUE, NEVER, ON
push_threshold no Threshold for VALUE push type

OPC UA Section

Key Type Default Notes
server_url str OPC UA server endpoint (e.g. opc.tcp://kepware:49320)
subscription_ms int 500 Publishing interval for subscription-mode nodes
polling_cache_ms int 500 Publishing interval for the hidden cache behind polling-mode nodes
default_polling_interval_ms int 1000 Default polling interval when a polling node omits polling_interval_ms
polling_startup_delay_sec float 2.0 Seconds to wait for polling cache to fill before starting poll loops
keepalive_interval_sec float 30 How often to read ServerState as a liveness check
session_timeout_ms int 3600000 OPC UA session timeout
secure_channel_timeout_ms int 3600000 Secure channel lifetime
request_timeout_ms int 10000 Per-request timeout
initial_reconnect_delay float 5.0 Starting backoff between reconnect attempts
max_reconnect_delay float 60.0 Maximum backoff cap
reconnect_backoff_multiplier float 1.5 Multiplier applied after each failure
security dict {"mode": "none"} See below
nodes list [] OPC UA nodes

security object:

Key Required when Notes
mode always "none" | "username" | "cert" | "username_cert"
username, password mode is username or username_cert
policy mode is cert or username_cert e.g. Basic256Sha256
security_mode mode is cert or username_cert e.g. SignAndEncrypt
client_cert, client_key, server_cert mode is cert or username_cert Paths to PEM/DER files

Node entry:

{
  "node_id": "ns=2;s=Channel1.Device1.Temperature",  // OPC UA address
  "name": "Temperature",                              // Local name (used in input_mapping)
  "type": "NUMBER",                                   // "NUMBER", "STRING", "BOOLEAN", etc.
  "mode": "subscription",                             // "subscription" (default) or "polling"
  "polling_interval_ms": 1000                         // Required only when mode=="polling"
}
  • subscription mode forwards every server-pushed data change straight into the inference pipeline.
  • polling mode keeps a background subscription cache fresh and emits the cached value at polling_interval_ms. Use this with Kepware, which can return 0 on repeated direct reads but still pushes correct values through subscriptions.

Model Section

Key Type Default Notes
pmml_file str Path to .pmml model file
backend str "auto" "auto" (try sklearn, fallback pypmml), "sklearn-pmml-model", "pypmml"
trigger_on str "any" "any" (infer on each OPC change) or "all" (wait for all nodes first)
input_mapping dict {} Maps OPC UA node name → PMML input field name
output_mapping dict {} Maps PMML output field → ThingWorx property name

Troubleshooting

"No module named 'pandas'"

Ensure you're using the .venv Python:

.venv\Scripts\Activate.ps1
pip install pandas numpy sklearn-pmml-model asyncua

"Failed to load ThingWorx C SDK library"

Set the library path:

$env:TWCSDK_LIB_PATH = "C:\path\to\twCSdk.dll"
python edge_inference.py

"twApi_RegisterProperty: Invalid params"

Ensure all output property names in config.json match those defined on the ThingWorx platform Thing. Run test_connection.py to validate.

"OPC UA connection failed"

  • Verify endpoint_url is correct
  • Check firewall allows port 49320 (or your configured port)
  • Use test_pmml.py with dummy data if OPC UA is unreachable

Model inference very slow (>50ms)

If using pypmml, Java startup adds overhead. Switch to sklearn-pmml-model if your model type is supported:

"backend": "sklearn-pmml-model"

Architecture

Threading Model

  • Main thread: Receives Ctrl+C, initiates shutdown
  • asyncio event loop (main): OPC UA subscriptions, inference engine, ThingWorx input refresh
  • Executor thread pool (2 workers): C SDK calls (twApi_* functions are synchronous)
  • tasker thread (ThingWorx client): Runs twApi_TaskerFunction() every 500ms to drive SDK state machine

Memory Management

  • C SDK structures are wrapped by Python classes and auto-cleaned via __del__
  • PMML model is loaded once and held in memory for the duration
  • OPC UA subscription and ThingWorx connection are kept alive for as long as the app runs

Extending the Application

Add a custom transformation step

Insert logic between OPC UA read and model input:

# In InferenceEngine._run_inference(), after step 1:
if model_inputs.get("raw_temperature"):
    model_inputs["temperature_celsius"] = (model_inputs.pop("raw_temperature") - 32) * 5/9

Change inference trigger condition

Modify trigger_on in config, or add custom logic in InferenceEngine.on_opc_data_change():

# Only infer if temperature changed by >1 degree
if abs(new_temp - cached_temp) > 1.0:
    await self._run_inference(...)

Post-process model output

Add a validation or smoothing step:

# In InferenceEngine._run_inference(), after step 4:
if tw_values.get("ElectricalOutput_mo") < 0:
    tw_values["ElectricalOutput_mo"] = 0  # Clamp negative predictions

Logging

Set logging.level in config.json:

  • DEBUG: Detailed diagnostics (inference inputs, OPC values, property reads)
  • INFO: Key lifecycle events (connect, bind, model load, inference completed)
  • WARNING: Recoverable errors (property read failure, type mismatch)
  • ERROR: Unrecoverable errors (model load failure, OPC connection lost)

Log to file (rotating):

"logging": {
  "level": "INFO",
  "file": "edge_inference.log",
  "file_max_bytes": 10485760,   // 10 MB, default
  "file_backup_count": 5        // default
}
Key Default Notes
level INFO DEBUG | INFO | WARNING | ERROR
file null Path to log file; omit / set null for stdout only
file_max_bytes 10485760 Rotate after this many bytes
file_backup_count 5 Rotated file copies to keep

License & Support

This application is built on the ThingWorx C SDK Python wrapper (thingworx-python). For SDK issues, refer to the ThingWorx SDK documentation.

For questions or feedback, refer to the project repository.

About

ThingWorx Python Edge Machine Learning

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

No releases published

Packages

 
 
 

Contributors