Comprehensive RTT Improvements and Bug Fixes

[fxd0h]

Summary

This issue consolidates 9 open RTT-related issues into a single improvement plan. Based on feedback from @hkpeprah on PR #250, I'm proposing a refactored approach that addresses multiple RTT bugs and feature requests while maintaining backward compatibility.

Motivation

RTT (Real-Time Transfer) is a critical feature for embedded development, but current limitations prevent its reliable use in automated workflows. The main motivation for addressing these issues is to enable reliable, automated RTT usage in:

• CI/CD Pipelines: Automated testing and validation of firmware behavior without manual intervention
• Audit and Compliance: Automated logging and monitoring of device behavior for compliance requirements
• Debugging and Analysis: Programmatic access to device logs for debugging, performance analysis, and troubleshooting
• Production Monitoring: Automated monitoring of deployed devices without requiring GUI tools

Currently, many of these use cases are difficult or impossible because:

• RTT auto-detection fails on newer devices (nRF54L15, nRF5340, etc.)
• Manual configuration is required, making automation challenging
• Error messages are unclear, making debugging difficult
• SEGGER's GUI tools (RTT Viewer, J-Link Commander) are not well-suited for automation and CI/CD environments

By fixing these issues, pylink can become a reliable, scriptable interface for RTT that works seamlessly in automated environments where SEGGER's tools fall short.

Related Issues

Critical (Must Fix)

• rtt_start() fails to auto-detect RTT control block without explicit search ranges #249: rtt_start() fails to auto-detect RTT control block without explicit search ranges
• RTT does not connect #233: RTT does not connect (nRF54L15)
• Option to set RTT Search Range #209: Option to set RTT Search Range
• Initialize RTT with address of RTT control block #51: Initialize RTT with address of RTT control block (oldest RTT issue, from 2019)

High Priority (Should Fix)

• exec_command(f'SetRTTTelnetPort 19021') raises JLinkException when success #171: exec_command(f'SetRTTTelnetPort 19021') raises JLinkException when success
• RTT no writing #234: RTT no writing (rtt_write() returns 0) - likely firmware misconfiguration (no down buffers) meh.
• Invalid error code: -11 from rtt_read() #160: Invalid error code: -11 from rtt_read()

Medium Priority (Nice to Have)

• Specify RTT Telnet port #161: Specify RTT Telnet port (server-side limitation documented)
• RTT Echo #111: RTT Echo (local echo option)

Context

PR #250 attempted to fix RTT auto-detection for nRF54L15, but @hkpeprah provided feedback suggesting we:

1. Keep jlink.py as a low-level API (simplify rtt_start())
2. Move convenience features to a separate pylink/rtt.py module
3. Let callers implement their own polling (like script/rtt does)

This issue expands on that feedback to address all open RTT-related issues together.

Proposed Solutions

Issue #249 & #233: RTT Auto-detection Fails

Problem: rtt_start() fails to auto-detect RTT control block on nRF54L15 and similar devices.

Solution:

• Simplify rtt_start() per @hkpeprah's feedback: remove polling parameters and internal polling logic
• Keep search_ranges parameter, but only configure if explicitly provided (no auto-generation)
• Create pylink/rtt.py convenience module with start_rtt_with_polling() function that handles polling externally
• Move auto-generation of search ranges to convenience module

Issue #209: Option to Set RTT Search Range

Problem: Need to specify search ranges for RTT control block detection.

Solution:

• Keep search_ranges parameter in rtt_start() (already implemented in PR Improve RTT auto-detection for nRF54L15 and similar devices #250)
• Add rtt_get_block_address() method to search memory for RTT control block magic string
• Add auto_detect_rtt_ranges() helper function in pylink/rtt.py module

Issue #51: Initialize RTT with Address of RTT Control Block

Problem: Need to specify explicit control block address (from .map file).

Solution:

• Add block_address parameter to rtt_start() method (approved by @hkpeprah in 2019)
• Add find_rtt_address_in_map() helper function to parse .map files

Issue #171: exec_command() Raises Exception on Success

Problem: SetRTTTelnetPort command raises JLinkException even when successful (informational message treated as error).

Solution:

• Fix exec_command() to distinguish informational messages from errors
• Add pattern matching for known informational message patterns
• Log informational messages at DEBUG level instead of raising exception

Issue #234: RTT Write Returns 0

Problem: rtt_write() returns 0, likely due to missing down buffers in firmware configuration.

Solution:

• Improve rtt_write() error messages to check if down buffers exist
• Provide helpful error message if no down buffers configured
• Guide users to check firmware RTT configuration

Note: This is a UX improvement, not a fix of the root cause. The actual problem (missing down buffers) must be fixed in firmware configuration. We can only help users diagnose the issue better.

Issue #160: Invalid Error Code -11 from rtt_read()

Problem: Error code -11 is unclear and doesn't provide helpful diagnostics.

Solution:

• Add better error handling and diagnostics for error code -11
• Check device connection health when error occurs
• Provide helpful error messages with possible causes

Note: This is a UX improvement. The error code -11 comes from the J-Link SDK and cannot be changed. We can only improve how pylink handles and reports this error to help users diagnose the underlying issue (device disconnection, GDB conflict, etc.).

Issue #251: Specify JLink Home Path

Problem: Need to specify custom JLink installation directory.

Solution:

• Add jlink_path parameter to JLink.__init__()
• Extend Library class to search for DLL/SO in specified directory
• Fully backward compatible

Issue #161: Specify RTT Telnet Port

Problem: Cannot specify custom RTT Telnet port (server-side limitation).

Solution:

• Document limitation (J-Link SDK doesn't support server-side port configuration)
• Suggest workarounds (use different JLink instances, different ports per instance)
• Improve error messages when port conflicts occur

Note: This is a limitation of the J-Link SDK, not something pylink can fix. The SDK doesn't expose an API to change the RTT Telnet server port. We can only document this limitation and provide workarounds. A true solution would require changes to the J-Link SDK itself.

Issue #111: RTT Echo

Problem: RTT example script shows local echo, but JLinkRTTClientExe has option to disable it.

Solution:

• Add read_rtt_without_echo() helper function in pylink/rtt.py
• Option to ignore first N bytes of output (treating them as echo)

Additional: Create pylink/rtt.py Convenience Module

Per @hkpeprah's feedback, we'll keep jlink.py as a low-level API and move convenience features to a separate pylink/rtt.py module. This module will include:

• auto_detect_rtt_ranges(jlink) - Auto-generate search ranges from device RAM
• start_rtt_with_polling(jlink, ...) - Start RTT with automatic polling
• reconnect_rtt(jlink, ...) - Reconnect RTT after device reset (reconfigures parameters)
• rtt_context(jlink, ...) - Context manager for automatic cleanup
• monitor_rtt_with_reset_detection(jlink, ...) - Integration with Issue Feature Request: Reset Detection via SWD/JTAG Connection Health Monitoring #252
• find_rtt_address_in_map(map_file_path) - Helper to find address from .map file
• read_rtt_without_echo(jlink, ...) - Read RTT ignoring local echo

Implementation Approach

All changes will maintain backward compatibility - existing code should continue working without modifications. The low-level API (jlink.py) will stay simple per @hkpeprah's feedback, with convenience features moved to the pylink/rtt.py module.

A detailed implementation plan is available in the PR #250 documentation.

Benefits

• Backward Compatible: Existing code continues to work without changes
• Follows Maintainer Feedback: Low-level API stays simple, convenience features in separate module
• Addresses Multiple Issues: 9 issues resolved together
• Better Error Messages: Users get helpful diagnostics when things go wrong
• Extensible: Easy to add new features without touching the low-level API

Testing Strategy

• Unit tests for new methods and error handling
• Integration tests with nRF54L15 (primary target device)
• Backward compatibility tests to ensure existing code continues working
• Device tests across multiple ARM Cortex-M devices (STM32 H/F/G series, nRF series, LPC series)

Questions for Maintainer

1. Issue exec_command(f'SetRTTTelnetPort 19021') raises JLinkException when success #171 approach: For fixing exec_command(), should I fix it directly (Option A) or create exec_command_silent() (Option B)? You mentioned Option B earlier, but Option A seems cleaner to me. What do you think?

2. Scope approval: Is it okay to address all 9 issues in a single PR, or would you prefer I split them into multiple PRs?

References

• PR Improve RTT auto-detection for nRF54L15 and similar devices #250: Improve RTT auto-detection for nRF54L15 and similar devices #250
• Related issues: rtt_start() fails to auto-detect RTT control block without explicit search ranges #249, RTT does not connect #233, RTT no writing #234, Option to set RTT Search Range #209, exec_command(f'SetRTTTelnetPort 19021') raises JLinkException when success #171, Specify RTT Telnet port #161, Invalid error code: -11 from rtt_read() #160, RTT Echo #111, Initialize RTT with address of RTT control block #51, Question: how to speicfy the SEGGER JLink home path #251, Feature Request: Reset Detection via SWD/JTAG Connection Health Monitoring #252