Port Event class to Rust by erikjohnston · Pull Request #19701 · element-hq/synapse

erikjohnston · 2026-04-16T11:55:13Z

Ports the event class to Rust.

The main difference here are:

There is now a single event class
We now validate a lot more at event construction time than we previously did (we basically checked nothing before). This required some changes to the tests, including Fix tests to work with Rust event matrix-org/sytest#1423

Reviewable commit-by-commit.

Overview of Event Rust structure

The format of the event struct in Rust is quite different than that in Python.

The top-level looks like:

pub struct Event {
    /// The parsed event JSON.
    fields: FormattedEvent,

    /// The event ID. For format v1 this is read directly from the JSON;
    /// for v2+ it is computed from the canonical-JSON hash at
    /// construction time and cached here.
    event_id: Arc<str>,

    /// Synapse-internal per-event state that lives outside the federated
    /// JSON (e.g. outlier flag, soft-failure, stream positions).
    #[pyo3(get)]
    internal_metadata: EventInternalMetadata,

    /// The room version this event was parsed for.
    #[pyo3(get)]
    room_version: &'static RoomVersion,

    /// `None` for accepted events; otherwise a short reason set by auth
    /// when the event was rejected.
    rejected_reason: Option<Box<str>>,
}

which includes the actual parsed event in FormattedEvent, plus the rest of the event metadata.

pub struct FormattedEvent<E = Arc<EventFormatEnum>> {
    #[serde(default)]
    pub signatures: Signatures,

    #[serde(default)]
    pub unsigned: Unsigned,

    #[serde(flatten)]
    pub specific_fields: E,

    #[serde(flatten)]
    pub common_fields: Arc<EventCommonFields>,
}

The struct is further split into the common fields, format specific fields, plus the signatures and unsigned. We split out the signature and unsigned fields as they are mutable, so when we clone the event we can still share the common and specific fields and only copy signature and unsigned.

The specific_fields are the fields that depend on the format version. They can either be a specific format (e.g. E = EventFormatV1) or a type-erased enum EventFormatEnum that is across all room versions:

pub enum EventFormatEnum {
    V1(EventFormatV1),
    V2V3(EventFormatV2V3),
    V4(EventFormatV4),
    VMSC4242(EventFormatVMSC4242),
}

For example:

/// Shared flat-list encoding of `auth_events` and `prev_events`, reused
/// by every format from v2/v3 onwards.
#[derive(Serialize, Deserialize)]
pub struct SimpleAuthPrevEvents {
    pub auth_events: Vec<String>,
    pub prev_events: Vec<String>,
}

/// Version-specific fields for room versions 3-10.
#[derive(Serialize, Deserialize)]
pub struct EventFormatV2V3 {
    pub room_id: Box<str>,
    #[serde(flatten)]
    pub auth_prev_events: SimpleAuthPrevEvents,
}

Dev notes

As discussed in #element-backend-internal:matrix.org

Small prerequisites for porting the Python EventBase hierarchy to Rust: - duration: make `from_milliseconds` const and add an `IntoPyObject` impl for owned `SynapseDuration`, so the new Rust `Event.sticky_duration()` can return one directly to Python. - internal_metadata: rename `copy()` to `deep_copy()` (matching the new naming used by the rest of the events module) and make `new()` callable from sibling modules. - json_object: expose `object` as a `pub` field and add a `get_field` helper so the new Event class can read from it without going through Python. - signatures, unsigned: add `deep_copy()` methods so the new Event class can implement its own deep-copy.

Adds a single `Event` Rust pyclass that replaces the Python EventBase / FrozenEventV{1,2,3,4,VMSC4242} hierarchy. The class is added but not yet wired into Python — callers continue to use the existing Python classes in this commit; the migration follows in the next commit. The internals use an `FormattedEvent` over `EventFormatV{1,2V3,4,VMSC4242}` structs sharing an `EventCommonFields`. Format-specific behaviour (prev_event_ids, auth_event_ids, room_id derivation for v12 create events, etc) is encapsulated per variant. Event IDs are computed in the constructor for v3+ formats; v1/v2 use the `event_id` field as-is. Two supporting Rust modules are added at the same time: - `events::constants` — string constants for event types, top-level fields, and per-event-type content fields, used to keep the redaction rules and field accessors readable. - `events::utils` — `redact()`, `compute_event_reference_hash()`, and `calculate_event_id()`, ported from `synapse.crypto.event_signing` / `synapse.events.utils`.

Replace the abstract `synapse.events.EventBase` and the concrete `FrozenEvent`, `FrozenEventV2`, `FrozenEventV3`, `FrozenEventV4`, and `FrozenEventVMSC4242` Python classes with a single Rust-backed `Event`, exposed via `synapse.synapse_rust.events.Event`. `EventBase` becomes a `TypeAlias` for `Event` so that the existing type annotations across the codebase keep working. Notable behavioural notes: - `make_event_from_dict()` now constructs the Rust class. Event IDs for v3+ formats are computed in the constructor (instead of lazily on first access). - `clone_event()` is now a single `event.deep_copy()` call. The old shallow copy of `unsigned` was effectively a deep copy in practice; `deep_copy()` matches that. - The third-party event-rules callback no longer needs to call `event.freeze()` — Events are immutable from Python by construction. - A small `assert_never` is added in `events_worker.py` to make the `redact_behaviour` switch exhaustive now that the type checker can see all branches. All test fixtures that constructed `FrozenEventV3` etc. directly are updated to construct `Event` instead.

Adapt tests that mutated Python event internals (`_event_id`, `_dict`, direct attribute assignment, `FrozenEventV3(...)` construction) to work with the new Rust-backed `Event` class: - Rebuild events via `make_event_from_dict` / `make_test_event` instead of patching attributes in place. - Plumb `rejected_reason` through `_join_rules_event` rather than assigning to `rejected_reason` after construction. - Replace the hand-built event in `test_msc4242_state_dag` with a `Mock(spec=EventBase)` since the test only needs a handful of attributes. - Add `# type: ignore` for the deprecated `event.user_id` / `event[key]` accessors and for assigning to `event.content`. - In `make_test_event`, drop the default `room_id` for v11+ create events so each gets a distinct hash-derived room ID.

We have to take a slightly different approach here as we can't subclass the native Event type.

Now that we do a bit more validadtion of events, it's possible that an event persisted in the database may now not pass validation. This shouldn't happen, but let's handle it correctly by logging and returning that we couldn't find the event. This is the same as what we do if we can't parse the JSON.

MadLittleMods · 2026-05-19T18:57:26Z

+    /// The event ID. For format v1 this is read directly from the JSON;
+    /// for v2+ it is computed from the canonical-JSON hash at
+    /// construction time and cached here.
+    event_id: Arc<str>,


What benefit are we getting from this being a Arc<str> vs an owned String?

Also a general question of the Arc/Box usage throughout these new types

Ah yeah, so Box<str> vs String is mostly about space, since String has to store a capacity it takes up more memory. It also has mild advantage that is really indicates its immutable.

In this particular case, we also use Arc<str> so that we can point to the same event_id which is defined in EventFormatV1. (Will add a comment to that effect).

MadLittleMods · 2026-05-19T19:09:48Z

+//! On-the-wire representations of Matrix events, parameterised by event format
+//! version.


We could explain this a little more plainly. Like "event JSON" and "parsed".

Reading far enough, I see a "Serialization and deserialization" but I think we could explain a bit here.

MadLittleMods · 2026-05-19T19:12:50Z

+//! don't need to be parsed up front. Generally, optional fields should be
+//! handled via `other_fields`, as this saves space when they are not present.


Generally, optional fields should be handled via other_fields, as this saves space when they are not present.

These details should be on described next to other_fields

MadLittleMods · 2026-05-19T22:00:14Z

-        with self.assertRaises(NotImplementedError):
-            isinstance(object(), EventProtocol)


We could still test this raises NotImplementedError right?

MadLittleMods · 2026-05-19T22:00:35Z

            ev: EventBase, state: StateMap[EventBase]
        ) -> tuple[bool, JsonDict | None]:
-            ev.content = {"x": "y"}
+            ev.content = {"x": "y"}  # type: ignore[misc]


Explain type ignore (comment)

MadLittleMods · 2026-05-19T22:01:41Z

-            room_version=RoomVersions.MSC4242v12,
-        )
-        ev._event_id = id  # type: ignore[attr-defined]
-        assert supports_msc4242_state_dag(ev)


Why no longer using real events?

MadLittleMods · 2026-05-19T22:05:17Z

    def __iter__(self) -> Iterator[str]: ...
    def __eq__(self, other: object) -> bool: ...
+
+class Event:


Need to manually transfer docstrings

MadLittleMods · 2026-05-19T22:06:03Z

+    }
+
+    #[getter(state_key)]
+    fn state_key_attr(&self) -> PyResult<&str> {


Why state_key_attr naming?

…y set

MadLittleMods · 2026-05-21T18:38:47Z

+/// flat object matching the Matrix spec.
+#[derive(Serialize, Deserialize)]
+pub struct FormattedEvent<E = Arc<EventFormatEnum>> {
+    /// The event's signatures. This is a mutable field.


Suggested change

/// The event's signatures. This is a mutable field.

/// The event's signatures. Kept separate from common/specific fields as this is a mutable field.

(also applies for unsigned)

Previous conversation

MadLittleMods · 2026-05-21T18:40:53Z

+/// The `signatures` and `unsigned` fields are kept as dedicated typed
+/// wrappers because they round-trip between Rust and Python repeatedly
+/// and benefit from caching. `common_fields` and `specific_fields` are


I'm not following exactly. I would think common_fields/specific_fields are the ones that are shared/cached because they don't change.

How are signatures/unsigned being cached?

This also goes against what's written above AFAICT:

//! The `signatures` and `unsigned` fields are kept distinct from the //! common/specific as they allow mutation. When copying an event they need to //! be deep-copied, but the common/specific fields (which are immutable) can be //! shared.

MadLittleMods · 2026-05-21T18:43:00Z

+            specific_fields: Arc::clone(&self.specific_fields),
+            common_fields: Arc::clone(&self.common_fields),


👌 Perfect. I feel like we should drop a note about the writable aspect protecting us here.

MadLittleMods · 2026-05-21T18:48:25Z


 def clone_event(event: EventBase) -> EventBase:
-    """Take a copy of the event.
+    """Take a copy of the event."""


With the newly learned details from #19701 (comment), we're luckily protected out of the box 👍

Perhaps more along these lines of a hint:

Suggested change

"""Take a copy of the event."""

"""

Take a copy of the event.

Only `unsigned`/`signatures` is mutable. Mutating other properties will result in `AttributeError` (not writable).

"""

MadLittleMods · 2026-05-21T18:56:26Z

+        if event.fields.common_fields.type_state_key_tuple() != Some((M_ROOM_CREATE, ""))
+            && auth_event_ids.is_empty()
+        {
+            return Err(PyRuntimeError::new_err(


The "assert" language signals an invariant check. It better describes "this should never happen if code is correct" vs. "something went wrong during execution"

MadLittleMods · 2026-05-21T19:04:00Z

+            return Err(PyRuntimeError::new_err(
+                "auth_event_ids is unexpectedly empty for a non-create event",
+            ));


More:

Suggested change

return Err(PyRuntimeError::new_err(

"auth_event_ids is unexpectedly empty for a non-create event",

));

"auth_event_ids has not been calculated for event_id={}. All events (aside from the `m.room.create` event should have some `auth_event_ids` set.",

Even more:

Suggested change

return Err(PyRuntimeError::new_err(

"auth_event_ids is unexpectedly empty for a non-create event",

));

"Expected auth_event_ids for event_id={}. All events (aside from the `m.room.create` event) should have some `auth_event_ids` set. Either the `auth_event_ids` have not been calculated yet or this is a Synapse programming error.",

erikjohnston force-pushed the erikj/events_rust branch from 41d9ebb to 1c4406b Compare May 8, 2026 11:59

erikjohnston changed the base branch from develop to erikj/port_event_content May 8, 2026 12:01

Base automatically changed from erikj/port_event_content to develop May 8, 2026 13:19

MadLittleMods reviewed May 8, 2026

View reviewed changes

Comment thread rust/src/events/mod.rs Outdated

erikjohnston force-pushed the erikj/events_rust branch from 001cddc to 1a6900f Compare May 8, 2026 14:39

erikjohnston changed the base branch from develop to erikj/room_versions_rust May 8, 2026 14:39

erikjohnston force-pushed the erikj/room_versions_rust branch from 48b0729 to f1705f2 Compare May 8, 2026 14:40

erikjohnston force-pushed the erikj/events_rust branch 3 times, most recently from 68b0ec2 to 7c5c0b7 Compare May 12, 2026 12:08

Base automatically changed from erikj/room_versions_rust to develop May 14, 2026 10:21

erikjohnston force-pushed the erikj/events_rust branch 2 times, most recently from db0b3f3 to 2b8272e Compare May 15, 2026 13:15

erikjohnston mentioned this pull request May 15, 2026

Fix tests to work with Rust event matrix-org/sytest#1423

Open

erikjohnston added 6 commits May 15, 2026 15:59

Fix EventProtocol to work with the Rust class

5542419

We have to take a slightly different approach here as we can't subclass the native Event type.

Newsfile

a5081d1

erikjohnston force-pushed the erikj/events_rust branch from 2b8272e to a5081d1 Compare May 15, 2026 15:40

erikjohnston marked this pull request as ready for review May 18, 2026 08:56

erikjohnston requested a review from a team as a code owner May 18, 2026 08:56

MadLittleMods added rust Z-Rust labels May 18, 2026

MadLittleMods reviewed May 19, 2026

View reviewed changes

erikjohnston added 3 commits May 21, 2026 13:38

s/on-the-wire/over-the-wire

cdc09df

Comment which fields are mutable and immutable

d7a8dc5

Docstring for type_state_key_tuple

d4801f7

erikjohnston added 2 commits May 21, 2026 13:56

Convert get_room_id_for_optional_room_id into match

f5f7e7c

Update error for auth_event_ids

7115164

erikjohnston force-pushed the erikj/events_rust branch from a87f1e0 to 7115164 Compare May 21, 2026 13:02

erikjohnston added 12 commits May 21, 2026 14:10

Ensure we don't have stray auth_events or event_id fields unexpectedl…

69c8d22

…y set

Small refactor to ensure we call 'validate' for all formats

e82ebe1

Fix MAX_PDU_SIZE_BYTES

8e6e29c

Make JsonObject.object private

27e312d

Add room version to error

fc65b1d

Rename to parsed_event

24e7f85

Update get_dict docstring

aae2de1

Mention where get_templated_pdu_json is used

894d223

Drop SimpleAuthPrevEvents

1bbb033

s/debug_assert/ensure

b99fd73

Make MAX_DURATION a SynapseDuration

c9e9d76

Fix tests

025f1a4

MadLittleMods reviewed May 21, 2026

View reviewed changes

		//! On-the-wire representations of Matrix events, parameterised by event format
		//! version.

		//! don't need to be parsed up front. Generally, optional fields should be
		//! handled via `other_fields`, as this saves space when they are not present.

		with self.assertRaises(NotImplementedError):
		isinstance(object(), EventProtocol)

	/// The event's signatures. This is a mutable field.
	/// The event's signatures. Kept separate from common/specific fields as this is a mutable field.

		specific_fields: Arc::clone(&self.specific_fields),
		common_fields: Arc::clone(&self.common_fields),

Conversation

erikjohnston commented Apr 16, 2026 • edited by MadLittleMods Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overview of Event Rust structure

Dev notes

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

erikjohnston commented Apr 16, 2026 •

edited by MadLittleMods

Loading