RFC: RxInferBoard — A Debugging and Analysis UI for RxInfer Inference

[bvdmitri]

Author: @bvdmitri
Date: Nov 11 2025
Status: Draft
Version: v0.1
Target Audience: RxInfer users, contributors, and maintainers

RxInferBoard — A Debugging and Analysis UI for RxInfer Inference

1. Summary

This RFC proposes a new system for debugging inference inside RxInfer, designed to make identifying issues during inference more efficient and to enable better analysis of obtained results. The system introduces a dedicated UI (RxInferBoard) for exploring inference events, message passing, and posterior updates interactively.

2. Motivation

2.1 Why This Change Is Needed

Debugging inference in RxInfer currently requires significant time and expertise. Users often resort to guessing what is happening internally, making the debugging process unreliable and slow.

2.2 Problems This Solves

The proposed system aims to reduce the time and complexity of debugging inference and models by providing structured insight into message passing, variable updates, and the overall inference flow.

2.3 Who Benefits

• Primary: RxInfer users who need to debug or understand inference behavior.
• Secondary: Developers and maintainers who can more easily identify bugs and performance issues.

3. Background / Context

RxInfer currently performs message passing internally, but this mechanism is not exposed through the user API. Users typically interact with the infer function, which abstracts away the underlying message passing process.

While this abstraction simplifies usage, it limits users’ ability to inspect or understand failures or unintuitive inference results.

Existing debugging tools, such as DebugAddons, offer limited functionality and require substantial manual effort. Visualizing factor graphs helps identify structural issues (e.g., loops), but offers no visibility into what occurs during inference. Moreover, model visualization is currently implemented in GraphPPL, which itself does not perform inference.

In practice, this results in a “black box” inference process that is hard to reason about or explain—contrary to the transparency promised by factor graphs and message passing.

4. Proposal

Introduce a dedicated UI (RxInferBoard) that visualizes inference in RxInfer, including:

• Summary and structure of the model
• Order of message computations
• Content of each message for every iteration and variable
• Posterior updates in sequence
• Posterior distributions across iterations

4.1 Key UI Features

The UI will allow users to:

• Choose a variable in the model and observe all posterior updates across variational iterations
• Display the messages that led to each posterior update, including originating nodes
• Visualize the chronological order of all message computations
• Inspect the content of each message
• For variables representing state sequences (e.g., in state-space models), compare multiple states (e.g., first vs. last) side by side
• Compare different variables side by side
• Inspect all inference events (e.g., message fired → posterior updated → message fired → posterior updated, etc.) in order

5. Architecture Overview and API / Interface Changes

5.1 Overview

A new flag will be introduced to the infer function that, when enabled, logs inference activity (message order, content, posterior updates) in a language-agnostic format.

Each inference event—such as message updates, posterior updates, and free energy computations—will be recorded as a structured log entry.

This log can then be loaded into RxInferBoard, a standalone web application (proposed: Next.js) that visualizes and analyzes inference traces.

5.2 Logging Format

The log format should be language-independent (not Julia-specific).
Options include:

• CSV: simple and portable
• JSON: structured and flexible
• Custom format: domain-optimized (to be discussed)

Each log entry would represent an event with fields such as:

timestamp, event_type, variable, source_node, target_node, iteration, message_content, posterior_content, free_energy

5.3 Example Usage

Julia

result = infer(
    model = my_model(),
    data = (y = observations,),
    rxinfer_board_logfile = "Experiments/RxInfer_mymodel_experiment.log"
)

Command Line

./rxinfer-board

From within the UI

Load "Experiments/RxInfer_mymodel_experiment.log"

5.4 Implementation Stack

• Backend: No dedicated backend required initially — the UI can parse static log files
• Frontend: Web-based (Next.js, TypeScript)
• Future Extensions: Real-time mode (RxInferBoard connects to running inference session)

6. Impact

• Performance:
  Enabling logging will slow inference due to the additional data recording overhead.

• Backward Compatibility:
  Existing workflows remain unaffected. The logging feature is opt-in via the new flag.

• Codebase Changes:
  Certain internal structures will need adjustment to support event logging and serialization.

7. Alternatives Considered

7.1 Extending DebugAddon

Developing on top of the existing DebugAddon was considered but found limiting:

• It does not track Free Energy computations
• It does not expose message computation order
• It is Julia-specific, making it unsuitable for integration with a modern web UI
• Building a full UI in Julia (e.g., via Pluto) restricts interactivity and scalability

7.2 Pluto-based UI

A Pluto-based prototype (by @abpolym) demonstrates feasibility but remains constrained by Pluto’s reactive notebook paradigm.
The proposed web UI (Next.js) offers greater flexibility, modularity, and a more modern experience.

8. Open Questions

1. Log Format

   • Should we use CSV for simplicity or JSON for flexibility?
   • Is a custom lightweight text format preferable?

2. Free Energy Tracking

   • Should free energy computations be included by default, or made optional?

3. Live Connection Mode

   • Should RxInferBoard later support live monitoring (e.g., auto-detect and attach to ongoing inferences)?
   • For MVP, file-based loading is simpler and avoids complexity.

9. References

• TensorBoard — inspiration for visualization and debugging

[abpolym]

1. You could save to jld2 files or other compressed data formats to save some disk space and load with FileIO.
2. Live monitoring is really nice but for that the monitoring and plotting needs to be in a separate process or thread to not interfere with the actual inference

[bvdmitri]

Those are useful suggestions, but I don’t think we’re at that stage yet. Historically, one of the main issues with RxInfer has been our tendency to prioritize performance over usability. We’ve often focused too heavily on efficiency — the “save some disk space” mindset — and as a result, we ended up with extremely fast inference that’s difficult to debug, because the user experience was always secondary to performance goals.

I think it’s time to invert that process. First, let’s make things work — and work in a way that’s convenient and intuitive to use. Only then should we focus on making it efficient, such as optimizing disk space or performance. For example, if we decide that using an SQLite database (as @meditans suggested) is the right approach, any effort spent now on fine-tuning JLD2 space optimization would become irrelevant.

[meditans]

Thank you @bvdmitri for this proposal: exposing the users to the details of the inference process would be incredibly useful for understanding the system better.

I actually have a suggestion here: in my opinion this proposal conflates two aspects that are orthogonal, and I fear pursuing both at the same time would yield a suboptimal result. They are:

1. How to make available to the user enough data about the inference process
2. How the data (or part of it) can be displayed in a pretty web-ui

The first is essentially a data question, and I think it should be approached through the lens of what data you would put in a database. You hinted at this in your field description, which is a good starting point.
The second is a UI question about a necessarily specific way of consuming the information you have available.

The risk of conflating the two together is that:

1. You fixate on a particular pattern of querying the data, making different debugging processes more difficult.
   (example: say I have a model, and I want to do many different inference runs, each one with a different hyperparameter. It could be useful to me to compare step-by-step the evolution of the posteriors for a specific variable across runs. This would be unwieldy in the single inference per file scenario UI, but if I have the data available, I can very easily construct a bare bone visualization (maybe in pluto, or even in the repl) that targets my use case, just querying the data).
2. The debugging info would not be available to the program itself, which is where it would be more useful. You already have a beautiful reactive interface for inference, why not use it for reactively presenting this kind of debug messages? That way one could react when say a posterior for a specific variable is too far from the prior, for example. It seems to me that this would open more possibilities on which you can build later.

As a bonus, if you disentangle the concerns, you get the Real time mode you mentioned for free. For example, just have RxInfer log asynchronously the events in a SQLite db, and have the frontend load from that as it pleases.

So, tl;dr: I would suggest concentrating on making available the right data (and documenting the process properly for end users!), which is also where you have the excellent expertise of your group and can move fast. Once that's done, and there is some feedback on debugging with it, another proposal could think about how to construct a good UI informed by some usage.

I hope my suggestion can be useful, but either way I absolutely wish to have better introspection capabilities for RxInfer. Also, thanks for all the work done so far! This project is amazing!

[bvdmitri]

Thats a great comment, I didn't think it this way, but I think you're right. The proposal needs more thought in this direction, I will make some changes to it once I think a bit more about what you say

[bvdmitri]

Probably related, @abpolym did a great analysis of one of the examples here. We should keep this kind of workflow in mind as well in addition to what I proposed

[bartvanerp]

Definitely in favor. As a first MVP, can we leverage upon TensorBoardLogger.jl before starting with custom interfaces/UI? It supports automatic refreshing, so that could help with the live mode.

Another ref that could be interesting: https://wandb.ai/site/

[bvdmitri]

I somehow forgot about TensorBoardLogger.jl - thanks for reminding!! I’ll spend some time exploring it further — I see two possible outcomes:

• It’s great and already covers all our potential use cases, in which case we should simply adopt it.
• It turns out to be only partially suitable, and we include it in the “Alternatives Considered” section alongside Pluto.

To make a definitive decision, I need to better understand its full capabilities.

What I definitely don’t want to do is export static images to TensorBoard (for example, a distribution or a free energy graph), because:

• We really need to preserve the order of events — just displaying distributions isn’t sufficient.
• In many cases, distributions aren’t even displayable (e.g., a 3D Gaussian or a Wishart is already impossible to visualize as a simple picture, though its contents could be represented as text).

I really like @meditans’s suggestion that displaying is orthogonal to data acquisition. In fact, we could easily support multiple ways to visualize the acquired inference events — our own UI, TensorBoard’s UI, or even a terminal-based one.

Perhaps the first step is to decide what kinds of data we want to collect to support different use cases. For simpler workflows, we could export this data to TensorBoard (for users who prefer that interface). For more complex scenarios, we might need a dedicated UI. And for very advanced or unconventional use cases, users should be able to query the data directly — similar to what we can do now with DebugAddons, but in a much more convenient and user-friendly way.

[skoghoern]

Hi all, just wanted to checkin how things are progressing on the data accumulation vs. visualization part? would much appreciate simpler debugging:)

[bvdmitri]

Hey @skoghoern , there wasn't any progress on that recently due to lack of time on that unfortunately :(

[bvdmitri]

@skoghoern feel free to comment here #566 (comment)

[RetrospectiveRotations]

Great initiative. I feel that the primary objective is to debug code but I think the analysis of what is going on in the 'black box' is a much more interesting objective.

Log Format: I would pick a format that is easiest for live monitoring.
Free Energy Tracking: its already an option in the infer function so if that is enabled then yes.
Should RxInferBoard later support live monitoring (e.g., auto-detect and attach to ongoing inferences)?: Yes!

[bvdmitri]

Thanks for your thoughts @RetrospectiveRotations

What exactly do you mean here by Log Format? Do you mean the way it formatted? I agree with @meditans that the ultimate goal is to decide what data to collect and NOT how to display it. The display part is a separate process. Someone would prefer to log it in a textual log format in their terminal and some would prefer to use a fancy UI for the same purposes. So in the #566 (comment) I propose to adopt similar to OpenTelemetry data structure which can be formated any way you like, textual or fancy UI with external providers. Think of it as an intermediate representation that can be converted/formatted in either simple text or JSON for OpenTelemetry collectors or data structure supported by TensorBoard e.g. I like this ones from TensorBoard:

So basically we should not think about how to display/format logs just yet, but instead we should think what data we want to collect and at which points in order to support various display formats later on.

Free Energy Tracking - yes agree!

RxInferBoard live monitoring - definitely!

[RetrospectiveRotations]

It seems you know much more about data formatting than I do. I was simply saying to pick whatever is most suitable/versatile for the intended objectives (live tracking, debugging, etc.)

[bvdmitri]

#616