Feature/general threading improvements

.github/workflows/CI.yml

@@ -39,7 +39,7 @@ jobs:
       - uses: actions/checkout@v2
 
       - name: install dependencies
-        run: pip3 install .
+        run: pip3 install -e .[test]
 
       - name: Run tests
         run: pytest . --pyargs

CLAUDE.md

@@ -0,0 +1,78 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Project
+
+`can-j1939` is a Python implementation of the SAE J1939 protocol stack on top of
+[python-can](https://python-can.readthedocs.org/). It supports both J1939-21 and J1939-22 (J1939-FD)
+data link layers, including transport protocols (BAM, CMDT / RTS-CTS), address claiming, and a
+number of diagnostic messages (DM1, DM11, DM14, DM22).
+
+## Common commands
+
+```bash
+# Install the package (editable for development)
+pip install -e .
+
+# Run the full test suite (matches CI)
+pytest . --pyargs
+
+# Run a single test file / test
+pytest test/test_ecu.py
+pytest test/test_memory_access.py::TestMemoryAccess::test_some_name -v
+```
+
+CI runs `pytest . --pyargs` on Python 3.10 across Ubuntu/macOS/Windows
+(`.github/workflows/CI.yml`).
+
+## Architecture
+
+The stack is layered: an **ECU** owns a **data-link layer** object and one or more
+**ControllerApplications**. Background work runs on a dedicated job thread.
+
+- `j1939/electronic_control_unit.py` — `ElectronicControlUnit` is the entry point. It owns the
+  `can.Bus`, a `MessageListener`, a job thread (`_async_job_thread`) that drives timers and
+  transport-protocol timeouts, and a list of subscribers. The `data_link_layer` constructor arg
+  (`'j1939-21'` or `'j1939-22'`) selects which DLL is instantiated. The ECU passes the DLL a
+  small surface of callbacks: `send_message`, `_job_thread_wakeup`, `_notify_subscribers`,
+  `_is_message_acceptable`. For tests, `send_message=` can be injected to bypass real CAN I/O.
+- `j1939/j1939_21.py` and `j1939/j1939_22.py` — the two DLL implementations. They share the
+  callback signature above and implement the transport protocols (TP-BAM, TP-CMDT / RTS-CTS,
+  and for J1939-22 the FD multi-session variants and Multi-PG / FEFF). Changes that touch
+  protocol behaviour usually need parallel updates in both files.
+- `j1939/controller_application.py` — `ControllerApplication` (CA) implements J1939/81 address
+  claiming, state machine (`NONE` → `WAITING_VETO` → `NORMAL` / `CANNOT_CLAIM`), per-CA
+  subscriptions, and `send_pgn` (which dispatches to the ECU's DLL).
+- `j1939/name.py`, `j1939/parameter_group_number.py`, `j1939/message_id.py` — value objects for
+  the J1939 NAME, PGN encoding, and 29-bit CAN identifier framing.
+- `j1939/diagnostic_messages.py`, `j1939/memory_access.py`, `j1939/Dm14Query.py`,
+  `j1939/Dm14Server.py`, `j1939/error_info.py` — diagnostic-message support (DM1/DM11/DM14/DM22),
+  including the DM14 memory-access client (`Dm14Query`) and server (`Dm14Server`).
+- `j1939/__init__.py` is the public API surface — anything users are expected to import lives
+  here.
+
+### Threading model
+
+All I/O and protocol timing flows through the ECU's job thread. The DLL never blocks on I/O
+itself — it enqueues work and calls `_job_thread_wakeup` to nudge the thread. Callbacks
+registered via `ca.subscribe(...)` or `ca.add_timer(...)` run on that job thread, so they
+must not block.
+
+### Tests
+
+- `test/` holds unit tests. `test_helpers/feeder.py` provides the `Feeder` fixture (registered
+  in `test_helpers/conftest.py`) which is the standard way to drive the stack from tests: it
+  replaces `ElectronicControlUnit.send_message` with a simulated bus, lets the test queue
+  expected RX/TX messages and PDUs in order, and asserts that the stack produces the expected
+  TX sequence. New protocol-level tests should follow that pattern instead of mocking
+  `python-can` directly.
+- `test_helpers/feeder.AcceptAllCA` is a CA subclass with `message_acceptable` overridden to
+  accept everything — use it when a test needs to receive peer-to-peer messages without setting
+  up a real claim.
+
+### Examples
+
+`examples/` contains runnable scripts mirroring the README quick-start (simple receive, own CA
+producer, transport protocols, multi-PG, diagnostic messages). When adding a new public
+feature, prefer extending an existing example over inventing a new pattern in the docs.

README.rst

@@ -72,7 +72,7 @@ Features
 * correct timeout and deadline handling
 * (under construction) almost complete testcoverage
 * diagnostic messages (see https://github.com/juergenH87/python-can-j1939/tree/master/examples/diagnostic_message.py)
-  - support of DM1 Tool and ECU functionaliy
+  - support of DM1 Tool and ECU functionaliy (all four SAE J1939-73 SPN conversion methods: 1, 2, 3, 4)
   - support of DM11 Tool functionaliy
   - support of DM22 Tool functionaliy
 

examples/diagnostic_message.py

@@ -54,7 +54,7 @@ def dm1_before_send():
     :return:
         list of dictionaries of all DTCs included in DM1
 
-    :rtype: list of dic: 'spn', 'fmi', 'oc'
+    :rtype: list of dic: 'spn', 'fmi', 'oc', 'cm'
     """
     lamp_status = {}
     # get lamp status (optional, if status not enter, lamp is switched off)
@@ -65,9 +65,12 @@ def dm1_before_send():
 
     # add all active DTCs
     # if no DTC is active return empty list
+    # 'cm' is the SAE J1939-73 SPN conversion method (1, 2, 3, or 4).
+    # Defaults to 4 (current standard) when omitted.
     dtc_list = []
-    dtc_list.append({'spn': 123, 'fmi': 31})           # occurrence counter is set to 0
-    dtc_list.append({'spn': 456, 'fmi': 1, 'oc': 132}) # with optional occurrence counter
+    dtc_list.append({'spn': 123, 'fmi': 31})                     # CM defaults to 4
+    dtc_list.append({'spn': 456, 'fmi': 1, 'oc': 132})           # with occurrence counter
+    dtc_list.append({'spn': 789, 'fmi': 2, 'oc': 5, 'cm': 3})    # legacy CM 3 layout
 
     return lamp_status, dtc_list
 
@@ -81,12 +84,12 @@ def main():
     # Connect to the CAN bus
     # Arguments are passed to python-can's can.interface.Bus() constructor
     # (see https://python-can.readthedocs.io/en/stable/bus.html).
-    # ecu.connect(bustype='socketcan', channel='can0')
-    # ecu.connect(bustype='kvaser', channel=0, bitrate=250000)
-    ecu.connect(bustype='pcan', channel='PCAN_USBBUS1', bitrate=250000)
-    # ecu.connect(bustype='ixxat', channel=0, bitrate=250000)
-    # ecu.connect(bustype='vector', app_name='CANalyzer', channel=0, bitrate=250000)
-    # ecu.connect(bustype='nican', channel='CAN0', bitrate=250000)   
+    # ecu.connect(interface='socketcan', channel='can0')
+    # ecu.connect(interface='kvaser', channel=0, bitrate=250000)
+    ecu.connect(interface='pcan', channel='PCAN_USBBUS1', bitrate=250000)
+    # ecu.connect(interface='ixxat', channel=0, bitrate=250000)
+    # ecu.connect(interface='vector', app_name='CANalyzer', channel=0, bitrate=250000)
+    # ecu.connect(interface='nican', channel='CAN0', bitrate=250000)   
 
     # subscribe to all (global) messages on the bus
     ecu.subscribe(on_message)

examples/j1939_21_cmdt_send_receive/j1939_receive.py

@@ -50,12 +50,12 @@ def main():
     # Connect to the CAN bus
     # Arguments are passed to python-can's can.interface.Bus() constructor
     # (see https://python-can.readthedocs.io/en/stable/bus.html).
-    ecu.connect(bustype='socketcan', channel='can0')
-    # ecu.connect(bustype='kvaser', channel=0, bitrate=250000)
-    # ecu.connect(bustype='pcan', channel='PCAN_USBBUS1', bitrate=250000)
-    # ecu.connect(bustype='ixxat', channel=0, bitrate=250000)
-    # ecu.connect(bustype='vector', app_name='CANalyzer', channel=0, bitrate=250000)
-    # ecu.connect(bustype='nican', channel='CAN0', bitrate=250000)
+    ecu.connect(interface='socketcan', channel='can0')
+    # ecu.connect(interface='kvaser', channel=0, bitrate=250000)
+    # ecu.connect(interface='pcan', channel='PCAN_USBBUS1', bitrate=250000)
+    # ecu.connect(interface='ixxat', channel=0, bitrate=250000)
+    # ecu.connect(interface='vector', app_name='CANalyzer', channel=0, bitrate=250000)
+    # ecu.connect(interface='nican', channel='CAN0', bitrate=250000)
 
     # add CA to the ECU
     ecu.add_ca(controller_application=ca)

examples/j1939_21_cmdt_send_receive/j1939_send.py

@@ -104,12 +104,12 @@ def main():
     # Connect to the CAN bus
     # Arguments are passed to python-can's can.interface.Bus() constructor
     # (see https://python-can.readthedocs.io/en/stable/bus.html).
-    ecu.connect(bustype='socketcan', channel='can0')
-    # ecu.connect(bustype='kvaser', channel=0, bitrate=250000)
-    # ecu.connect(bustype='pcan', channel='PCAN_USBBUS1', bitrate=250000)
-    # ecu.connect(bustype='ixxat', channel=0, bitrate=250000)
-    # ecu.connect(bustype='vector', app_name='CANalyzer', channel=0, bitrate=250000)
-    # ecu.connect(bustype='nican', channel='CAN0', bitrate=250000)
+    ecu.connect(interface='socketcan', channel='can0')
+    # ecu.connect(interface='kvaser', channel=0, bitrate=250000)
+    # ecu.connect(interface='pcan', channel='PCAN_USBBUS1', bitrate=250000)
+    # ecu.connect(interface='ixxat', channel=0, bitrate=250000)
+    # ecu.connect(interface='vector', app_name='CANalyzer', channel=0, bitrate=250000)
+    # ecu.connect(interface='nican', channel='CAN0', bitrate=250000)
 
     # add CA to the ECU
     ecu.add_ca(controller_application=ca)

examples/j1939_22_multi_pg.py

@@ -66,7 +66,7 @@ def main():
     ecu = j1939.ElectronicControlUnit(data_link_layer='j1939-22', max_cmdt_packets=200)
 
     # can fd Baud: 500k/2M
-    ecu.connect(bustype='pcan', channel='PCAN_USBBUS3', fd=True,
+    ecu.connect(interface='pcan', channel='PCAN_USBBUS3', fd=True,
                         f_clock_mhz=80, nom_brp=10, nom_tseg1=12, nom_tseg2=3, nom_sjw=1, data_brp=4, data_tseg1=7, data_tseg2=2, data_sjw=1)
 
     # subscribe to all (global) messages on the bus

examples/j1939_22_transport_protocols.py

@@ -62,7 +62,7 @@ def main():
     ecu = j1939.ElectronicControlUnit(data_link_layer='j1939-22', max_cmdt_packets=200)
 
     # can fd Baud: 500k/2M
-    ecu.connect(bustype='pcan', channel='PCAN_USBBUS1', fd=True,
+    ecu.connect(interface='pcan', channel='PCAN_USBBUS1', fd=True,
                         f_clock_mhz=80, nom_brp=10, nom_tseg1=12, nom_tseg2=3, nom_sjw=1, data_brp=4, data_tseg1=7, data_tseg2=2, data_sjw=1)
 
     # subscribe to all (global) messages on the bus

examples/own_ca_producer.py

@@ -101,13 +101,13 @@ def main():
     # Connect to the CAN bus
     # Arguments are passed to python-can's can.interface.Bus() constructor
     # (see https://python-can.readthedocs.io/en/stable/bus.html).
-    # ecu.connect(bustype='socketcan', channel='can0')
-    # ecu.connect(bustype='kvaser', channel=0, bitrate=250000)
-    ecu.connect(bustype='pcan', channel='PCAN_USBBUS1', bitrate=250000)
-    # ecu.connect(bustype='ixxat', channel=0, bitrate=250000)
-    # ecu.connect(bustype='vector', app_name='CANalyzer', channel=0, bitrate=250000)
-    # ecu.connect(bustype='nican', channel='CAN0', bitrate=250000)    
-    # ecu.connect('testchannel_1', bustype='virtual')
+    # ecu.connect(interface='socketcan', channel='can0')
+    # ecu.connect(interface='kvaser', channel=0, bitrate=250000)
+    ecu.connect(interface='pcan', channel='PCAN_USBBUS1', bitrate=250000)
+    # ecu.connect(interface='ixxat', channel=0, bitrate=250000)
+    # ecu.connect(interface='vector', app_name='CANalyzer', channel=0, bitrate=250000)
+    # ecu.connect(interface='nican', channel='CAN0', bitrate=250000)    
+    # ecu.connect('testchannel_1', interface='virtual')
 
     # add CA to the ECU
     ecu.add_ca(controller_application=ca)

examples/simple_receive_global.py

@@ -31,12 +31,12 @@ def main():
     # Connect to the CAN bus
     # Arguments are passed to python-can's can.interface.Bus() constructor
     # (see https://python-can.readthedocs.io/en/stable/bus.html).
-    # ecu.connect(bustype='socketcan', channel='can0')
-    # ecu.connect(bustype='kvaser', channel=0, bitrate=250000)
-    ecu.connect(bustype='pcan', channel='PCAN_USBBUS1', bitrate=250000)
-    # ecu.connect(bustype='ixxat', channel=0, bitrate=250000)
-    # ecu.connect(bustype='vector', app_name='CANalyzer', channel=0, bitrate=250000)
-    # ecu.connect(bustype='nican', channel='CAN0', bitrate=250000)    
+    # ecu.connect(interface='socketcan', channel='can0')
+    # ecu.connect(interface='kvaser', channel=0, bitrate=250000)
+    ecu.connect(interface='pcan', channel='PCAN_USBBUS1', bitrate=250000)
+    # ecu.connect(interface='ixxat', channel=0, bitrate=250000)
+    # ecu.connect(interface='vector', app_name='CANalyzer', channel=0, bitrate=250000)
+    # ecu.connect(interface='nican', channel='CAN0', bitrate=250000)    
 
     # subscribe to all (global) messages on the bus
     ecu.subscribe(on_message)

examples/simple_receive_peer_to_peer.py

@@ -31,12 +31,12 @@ def main():
     # Connect to the CAN bus
     # Arguments are passed to python-can's can.interface.Bus() constructor
     # (see https://python-can.readthedocs.io/en/stable/bus.html).
-    # ecu.connect(bustype='socketcan', channel='can0')
-    # ecu.connect(bustype='kvaser', channel=0, bitrate=250000)
-    ecu.connect(bustype='pcan', channel='PCAN_USBBUS1', bitrate=500000)
-    # ecu.connect(bustype='ixxat', channel=0, bitrate=250000)
-    # ecu.connect(bustype='vector', app_name='CANalyzer', channel=0, bitrate=250000)
-    # ecu.connect(bustype='nican', channel='CAN0', bitrate=250000)
+    # ecu.connect(interface='socketcan', channel='can0')
+    # ecu.connect(interface='kvaser', channel=0, bitrate=250000)
+    ecu.connect(interface='pcan', channel='PCAN_USBBUS1', bitrate=500000)
+    # ecu.connect(interface='ixxat', channel=0, bitrate=250000)
+    # ecu.connect(interface='vector', app_name='CANalyzer', channel=0, bitrate=250000)
+    # ecu.connect(interface='nican', channel='CAN0', bitrate=250000)
 
     # subscribe to all global and peer-to-peer messages with destination 0xFA
     ecu.subscribe(on_message, 0xFA)

j1939/controller_application.py

@@ -123,6 +123,28 @@ def remove_timer(self, callback):
         """
         self._ecu.remove_timer(callback)
 
+    def register_dependent(self, dependent):
+        """Register a helper whose ``stop()`` should be called on ECU shutdown.
+
+        Convenience forwarder to :meth:`ElectronicControlUnit.register_dependent`
+        for helpers that only hold a reference to a CA.
+
+        :param dependent:
+            Any object exposing a no-arg ``stop()`` method.
+        """
+        self._ecu.register_dependent(dependent)
+
+    def unregister_dependent(self, dependent):
+        """Remove a previously-registered dependent.
+
+        Convenience forwarder to
+        :meth:`ElectronicControlUnit.unregister_dependent`.
+
+        :param dependent:
+            The object previously passed to :meth:`register_dependent`.
+        """
+        self._ecu.unregister_dependent(dependent)
+
     def start(self, claim_delay=0.5):
         """Starts the CA
         :param claim_delay: