Phase 3 C6 / #200: Email connector — iMIP calendar invite extraction - #253
Conversation
Add deterministic structured extraction to the IMAP Email connector: `extract()` runs an extraction cascade over staged RFC 822 messages and, today, turns iMIP calendar invites (text/calendar; method=REQUEST|REPLY) into the same appointment fact cluster the Calendar connector emits (user has_event <event> typed Appointment, plus located_in and attending), reusing a new shared mimir-connectors::ical module (DRY). The email is treated as provenance (IMAP UID is each fact's raw_reference), not the fact: no per-email communication facts and no Person entities auto-created from From/To headers, so marketing/spam produces no junk. The connector now authors user-scoped facts against the injected ConnectorContext user_identity, matching the Calendar connector. mail-parser 0.11.5 added under the gmail feature; icalendar/chrono-tz now shared with calendar. Calendar connector refactored to reuse the shared ical module (RawCalDavEvent wraps RawVEvent; event_to_facts delegates); its tests updated and remain green. Email gains unit tests (REQUEST/REPLY/PUBLISH/CANCEL gating, no-identity, plain email), a KB integration test (F5 resolution, Appointment overlay, connector provenance), and a fake-IMAP → extract round-trip. Follow-ups: schema.org JSON-LD extraction (#249), LLM prose extraction (C7 / #201), Photos took_photo fallback (#250). Semver bumped to 0.85.0 (MINOR). Closes #200.
|
Warning Review limit reached
Next review available in: 22 minutes You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (8)
📝 WalkthroughWalkthroughThe release adds shared iCalendar VEVENT parsing and fact generation. CalDAV now uses the shared representation. The Email connector parses staged RFC 822 messages and extracts facts from iMIP Sequence Diagram(s)sequenceDiagram
participant IMAP
participant EmailConnector
participant mail_parser
participant ical
participant KnowledgeGraph
IMAP->>EmailConnector: fetch and stage RFC 822 message
EmailConnector->>mail_parser: parse MIME content
mail_parser-->>EmailConnector: text/calendar part
EmailConnector->>ical: parse REQUEST or REPLY VEVENT
ical-->>EmailConnector: appointment, location, and attendee facts
EmailConnector->>KnowledgeGraph: provide facts with UIDVALIDITY-qualified provenance
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
✨ Simplify code
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 11
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/email-connector.md`:
- Around line 86-90: Update the Dependencies paragraph to distinguish Gmail-only
extraction dependencies from shared calendar dependencies: identify mail-parser
as gated by the gmail feature, and state that icalendar, chrono-tz, and the
shared ical module are available under any(feature = "calendar", feature =
"gmail").
In `@docs/wiki/email-connector.md`:
- Around line 22-26: Update the “The email is the evidence, not the fact”
section to state that primary has_event extraction depends on
ConnectorContext::user_identity, that supported iMIP REQUEST and REPLY parts may
produce facts regardless of sender, and that provenance uses the IMAP UID rather
than the RFC Message-ID header.
In `@mimir-connectors/src/email/mod.rs`:
- Around line 993-1029: Update the invite_email fixture to construct the raw
message with CRLF separators throughout, matching the plain_email fixture and
real IMAP BODY.PEEK[] wire format; preserve the existing headers, MIME
boundaries, calendar content, and method interpolation.
- Around line 751-753: In the staging flow around the buffer lock and staged
messages, release the mutex guard immediately after draining the buffer, before
any MIME parsing or fact-building loop begins. Explicitly drop the guard after
std::mem::take, then continue parsing the owned staged messages without holding
the lock.
- Around line 754-761: Thread the mailbox UIDVALIDITY from run_sync through the
staging flow into each RawEmail, then update the raw_ref construction in the
invite-extraction loop to use the cursor-style "{uid_validity}:{uid}" format.
Preserve the existing parsing and fact extraction behavior while ensuring
references differ across UIDVALIDITY epochs.
- Around line 1338-1340: Replace the tautological dr_smith > 0 assertion in the
attendee-facts test with a check over the inserted attending facts, asserting
that at least one has subject_id equal to dr_smith. Preserve the existing
entity-resolution setup and verify the attending fact specifically represents Dr
Smith as a Person.
- Around line 606-615: Update the text/calendar handling around the method match
to inspect the iCalendar body’s METHOD property when the MIME method parameter
is absent. Prefer the normalized MIME parameter when present, otherwise parse
and use the body METHOD line, while continuing to accept only REQUEST or REPLY
and skip unsupported or missing methods.
- Around line 599-605: The attachment-only iteration in the iMIP calendar
extraction flow misses inline text/calendar parts within multipart/alternative
messages. Replace the attachments() traversal around the message attachment
handling with a full MIME or body/non-attachment calendar-part traversal, while
retaining multipart/mixed support, and add coverage for an inline
multipart/alternative calendar without a filename.
In `@mimir-connectors/src/ical.rs`:
- Around line 328-330: Update the attendee name normalization around the
`strip_prefix` call to remove the `mailto:` scheme regardless of letter casing,
while preserving the existing trimming of whitespace and surrounding quotes and
leaving other values unchanged.
- Around line 98-140: Update the VEVENT filter and all property lookup paths in
the calendar conversion to use case-insensitive iCalendar name matching,
including prop_str, tzid_of, DTSTART, DTEND, ORGANIZER, and attendee handling.
Replace case-sensitive find_prop usage or introduce a shared case-insensitive
lookup helper, while preserving the existing RawVEvent field mapping and
datetime parsing behavior.
In `@README.md`:
- Line 35: Update the README description for mimir-connectors so schema.org
JSON-LD extraction references `#249`, while free-text LLM extraction references C7
/ `#201`; keep the surrounding connector and roadmap text unchanged.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 559fce22-49db-4603-b560-659518af9366
📒 Files selected for processing (13)
CHANGELOG.mdCargo.tomlMimir-Implementation-Context.mdREADME.mddocs/email-connector.mddocs/wiki/email-connector.mddocs/wiki/what-works-now.mdmimir-connectors/Cargo.tomlmimir-connectors/src/calendar/caldav.rsmimir-connectors/src/calendar/mod.rsmimir-connectors/src/email/mod.rsmimir-connectors/src/ical.rsmimir-connectors/src/lib.rs
📜 Review details
🧰 Additional context used
🪛 LanguageTool
Mimir-Implementation-Context.md
[style] ~146-~146: Would you like to use the Oxford spelling “centralizes”? The spelling ‘centralises’ is also correct.
Context: ... connector whose status is Active and centralises spawn-on-startup, restart with exponent...
(OXFORD_SPELLING_Z_NOT_S)
[uncategorized] ~146-~146: Use a comma before ‘so’ if it connects two independent clauses (unless they are closely connected and short).
Context: ...lize_and_insertin an isolated sub-task so a connector panic is caught viaJoinEr...
(COMMA_COMPOUND_SENTENCE_2)
[uncategorized] ~146-~146: Use a comma before ‘so’ if it connects two independent clauses (unless they are closely connected and short).
Context: ...oto degrades to the C1 coords-only shape so no data is lost. Knowledge-graph growth...
(COMMA_COMPOUND_SENTENCE_2)
[style] ~146-~146: Would you like to use the Oxford spelling “serializes”? The spelling ‘serialises’ is also correct.
Context: ...al pass, since is a relative window), serialises concurrent triggers per connector via a...
(OXFORD_SPELLING_Z_NOT_S)
[locale-violation] ~146-~146: Lower-case ‘Ok’ is American English. For British English use “OK”.
Context: ...d returns the cycle's TriggerOutcome (Ok { fetched, new_cursor }, AuthExpired...
(OK)
[locale-violation] ~146-~146: Lower-case ‘Ok’ is American English. For British English use “OK”.
Context: ...e); run_cycle takes SyncOptions and CycleOutcome::Ok carries the SyncOutcome. - mimir —...
(OK)
README.md
[typographical] ~44-~44: Conjunctions like ‘and’ should not follow semicolons. Consider using a comma, or removing the conjunction.
Context: ...ents surfacing in the “Upcoming” section; and the only connector write-back, act() crea...
(CONJUNCTION_AFTER_SEMICOLON)
docs/email-connector.md
[uncategorized] ~52-~52: Possible missing comma found.
Context: ...s on every fact as the raw_reference. Today the cascade has one layer — **iMIP cale...
(AI_HYDRA_LEO_MISSING_COMMA)
[style] ~58-~58: The noun “invitation” is usually used instead of ‘invite’ in formal writing.
Context: ...config.toml [identity] name), so an invite surfaces in the user's "Upcoming" memor...
(AN_INVITE)
[typographical] ~58-~58: It appears that a comma is missing.
Context: ... treats as user_entity_id. Without an identity the primary fact is skipped; the event ...
(DURING_THAT_TIME_COMMA)
[style] ~86-~86: The noun “invitation” is usually used instead of ‘invite’ in formal writing.
Context: ...wledge-graph integration test stages an invite, runs extract() → `normalize_and_inse...
(AN_INVITE)
🔇 Additional comments (21)
Cargo.toml (1)
10-10: LGTM!CHANGELOG.md (1)
3-14: LGTM!Mimir-Implementation-Context.md (1)
4-4: LGTM!Also applies to: 146-146
README.md (1)
44-44: LGTM!docs/email-connector.md (1)
5-12: LGTM!Also applies to: 50-63
docs/wiki/email-connector.md (1)
4-8: LGTM!docs/wiki/what-works-now.md (1)
3-6: LGTM!mimir-connectors/src/ical.rs (5)
1-32: LGTM!
168-254: LGTM!
269-310: LGTM!
347-415: LGTM!
417-663: LGTM!mimir-connectors/src/lib.rs (1)
67-71: LGTM!mimir-connectors/src/calendar/caldav.rs (2)
468-483: LGTM!Also applies to: 742-763, 791-801
449-466: 📐 Maintainability & Code QualityNo change needed.
RawVEventis re-exported frommimir-connectors/src/ical.rs, so the bare intra-doc link is valid unless rustdoc broken intra-doc links are denied.mimir-connectors/src/calendar/mod.rs (1)
43-43: LGTM!Also applies to: 56-56, 414-426
mimir-connectors/Cargo.toml (2)
155-155: LGTM!
135-141: 📐 Maintainability & Code QualityNo action needed. The
mail-parserdependency is on the latest stable release, the exact version keeps the MIT/Apache-2.0 licence, and its default feature set includes the decode support shown in the crate documentation.mimir-connectors/src/email/mod.rs (3)
48-48: LGTM!Also applies to: 235-242, 253-253, 262-262, 295-297, 807-813
1031-1041: LGTM!Also applies to: 1043-1047, 1049-1184, 1186-1337, 1341-1360
1846-1898: LGTM!
Closes the 11 actionable CodeRabbit findings on the Email iMIP invite extraction PR (#200): - iMIP MIME walk: iterate message.parts (not attachments()) so a text/calendar part inside multipart/alternative with no Content-Disposition: attachment is no longer missed. - METHOD fallback: resolve METHOD from the MIME Content-Type method parameter, falling back to the iCalendar body METHOD property (RFC 6047 §2.4 makes the parameter optional). - Globally-unique provenance: raw_reference is now {uid_validity}:{uid} (matching the persisted cursor format), not a bare IMAP UID unique only within one mailbox + UIDVALIDITY epoch. imap::RawEmail gains a uid_validity field populated by fetch_since. - Buffer lock: drain the staged buffer and release the mutex guard before the CPU-bound MIME parse loop so a concurrent sync() cycle is not blocked from staging new mail. - iCalendar name matching: parse_ical_to_vevents looks up VEVENT properties case-insensitively (icalendar 0.17.x find_prop matches case-sensitively, but RFC 5545 names are case-insensitive); the VEVENT component match is case-insensitive; participant_display strips the mailto: scheme case-insensitively (RFC 3986). - Tests: invite fixture now uses CRLF (RFC 5322/MIME + IMAP BODY.PEEK[] wire format); the tautological assert!(dr_smith > 0) is replaced by a real check that the Dr Smith attending fact resolved to the right Person entity and points at the appointment Event; fake-IMAP provenance assertion updated for the UIDVALIDITY-qualified raw_reference. - Docs: docs/email-connector.md, docs/wiki/email-connector.md, and README.md corrected — mail-parser is Gmail-specific while icalendar/chrono-tz are shared with calendar; the wiki states the has_event primary fact depends on a configured user_identity, that only supported iMIP REQUEST/REPLY parts produce facts, and that provenance is the IMAP UID; the README splits future-work references (schema.org JSON-LD = #249, LLM free-text extraction = C7 / #201). Version bumped 0.85.0 -> 0.85.1 (PATCH: backwards-compatible fixes + doc corrections); CHANGELOG updated. All connectors tests (140 lib + integration suites), clippy, and fmt are green; workspace check is clean.
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@CHANGELOG.md`:
- Line 9: Update the provenance wording to use the UIDVALIDITY:UID form instead
of “IMAP UID” in README.md:44 and docs/wiki/email-connector.md:25;
CHANGELOG.md:9 and CHANGELOG.md:13 require no direct changes and provide context
for the qualified format.
In `@docs/email-connector.md`:
- Line 54: Update docs/email-connector.md at line 54 to describe all eligible
text/calendar MIME parts, including METHOD resolved from either the MIME
parameter or calendar body, and remove attachment-only and MIME-parameter-only
wording. Update README.md at line 44 to document the body METHOD fallback and
state that supported REQUEST/REPLY invitations can produce facts regardless of
sender.
In `@mimir-connectors/src/email/mod.rs`:
- Around line 617-633: Update the method extraction logic near the existing MIME
`method` lookup to independently normalize both the MIME parameter and calendar
`METHOD` property, then reject or skip the part whenever both values are present
and differ instead of preferring the MIME value. Preserve the existing handling
for matching values and single-source values, while ensuring `PUBLISH` and
`CANCEL` remain skipped. Add regression tests covering conflicting supported and
unsupported method pairs.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 48c57c35-30cc-4e3f-b78b-b942b6cb7c43
📒 Files selected for processing (8)
CHANGELOG.mdCargo.tomlREADME.mddocs/email-connector.mddocs/wiki/email-connector.mdmimir-connectors/src/email/imap.rsmimir-connectors/src/email/mod.rsmimir-connectors/src/ical.rs
📜 Review details
🧰 Additional context used
🪛 LanguageTool
docs/email-connector.md
[style] ~86-~86: The noun “invitation” is usually used instead of ‘invite’ in formal writing.
Context: ...wledge-graph integration test stages an invite, runs extract() → `normalize_and_inse...
(AN_INVITE)
🔇 Additional comments (8)
Cargo.toml (1)
10-10: LGTM!CHANGELOG.md (1)
3-8: LGTM!Also applies to: 10-12, 14-26
README.md (1)
35-35: LGTM!docs/email-connector.md (1)
5-12: LGTM!Also applies to: 86-90
docs/wiki/email-connector.md (1)
4-8: LGTM!mimir-connectors/src/ical.rs (1)
90-147: LGTM!Also applies to: 338-343
mimir-connectors/src/email/mod.rs (1)
526-526: LGTM!Also applies to: 584-616, 769-781, 1054-1058, 1192-1192, 1210-1210, 1266-1266, 1373-1393, 1402-1402, 1937-1944
mimir-connectors/src/email/imap.rs (1)
210-214: LGTM!Also applies to: 291-291, 323-323
|
|
||
| - **Functional correctness (iMIP MIME walk):** `EmailConnector::extract_invites` (`mimir-connectors/src/email/mod.rs`) now walks every MIME part (`message.parts`) instead of `attachments()`, so a `text/calendar` part nested in `multipart/alternative` with no `Content-Disposition: attachment` header (classified as a body part by `mail-parser`) is no longer missed. | ||
| - **Functional correctness (METHOD fallback):** the iMIP `METHOD` is now resolved from the MIME `Content-Type` `method` parameter when present, falling back to the iCalendar body `METHOD` property (RFC 6047 §2.4 makes the parameter optional). Only `REQUEST`/`REPLY` are extracted; `PUBLISH`/`CANCEL` are still skipped. | ||
| - **Data integrity (globally-unique provenance):** the email provenance `raw_reference` is now `{uid_validity}:{uid}` (matching the persisted cursor format), not a bare IMAP UID that is unique only within one mailbox + `UIDVALIDITY` epoch. `imap::RawEmail` gains a `uid_validity` field populated by `fetch_since`. |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== files =="
git ls-files | rg '(^|/)(CHANGELOG\.md|README\.md|docs/email-connector\.md|docs/wiki/email-connector\.md)$' || true
echo "== CHANGELOG relevant =="
if [ -f CHANGELOG.md ]; then
nl -ba CHANGELOG.md | sed -n '1,25p'
fi
echo "== README relevant =="
if [ -f README.md ]; then
nl -ba README.md | sed -n '38,50p'
fi
echo "== docs/wiki/email-connector relevant =="
if [ -f docs/wiki/email-connector.md ]; then
nl -ba docs/wiki/email-connector.md | sed -n '18,30p'
fi
echo "== docs/email-connector presence and relevant =="
if [ -f docs/email-connector.md ]; then
nl -ba docs/email-connector.md | sed -n '1,120p'
fi
echo "== search raw_reference / provenance / UIDVALIDITY / raw_reference =="
rg -n "raw_reference|provenance|global|unique|UIDVALIDITY|UID-validity|uid_validity|IMAP UID|IMAP uid|bare IMAP UID" README.md CHANGELOG.md docs docs/wiki || trueRepository: BhavsarDevansh/Mimir
Length of output: 311
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== tool availability =="
command -v sed awk grep find python || true
echo "== top-of-docs excerpts =="
for f in CHANGELOG.md README.md docs/email-connector.md docs/wiki/email-connector.md; do
if [ -f "$f" ]; then
echo "--- $f ---"
sed -n '1,60p' "$f"
fi
done
echo "== relevant text search =="
grep -RinE "raw_reference|provenance|global|unique|UIDVALIDITY|UID-validity|uid_validity|IMAP UID|IMAP uid|bare IMAP UID" -f <(grep -RlnE "raw_reference|provenance|global|unique|UIDVALIDITY|UID-validity|uid_validity|IMAP UID|IMAP uid|bare IMAP UID" README.md CHANGELOG.md docs | sort -u) README.md CHANGELOG.md docs 2>/dev/null || trueRepository: BhavsarDevansh/Mimir
Length of output: 35925
Use UIDVALIDITY-qualified provenance wording in the mail docs.
The implementation uses the UIDVALIDITY:UID form for cursor and raw_reference, so these references should not say the provenance is just the IMAP UID.
README.md#L44-L44: useUIDVALIDITY:UIDinstead of “IMAP UID”.docs/wiki/email-connector.md#L25-L25: use the same qualified provenance wording.
📍 Affects 3 files
CHANGELOG.md#L9-L9(this comment)CHANGELOG.md#L13-L13README.md#L44-L44docs/wiki/email-connector.md#L25-L25
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@CHANGELOG.md` at line 9, Update the provenance wording to use the
UIDVALIDITY:UID form instead of “IMAP UID” in README.md:44 and
docs/wiki/email-connector.md:25; CHANGELOG.md:9 and CHANGELOG.md:13 require no
direct changes and provide context for the qualified format.
|
|
||
| `extract()` drains the staged RFC 822 messages and runs a **deterministic (structured-parse) extraction cascade** over each. The email is treated as **provenance, not the fact**: the fact is about the real-world thing the email conveys (an appointment), and the email's `UIDVALIDITY`-qualified IMAP UID rides on every fact as the `raw_reference`. Today the cascade has one layer — **iMIP calendar invites**: | ||
|
|
||
| - A MIME attachment with `Content-Type: text/calendar; method=REQUEST | REPLY` is parsed with `mail-parser`, then the embedded `VEVENT` is parsed with the shared `mimir-connectors::ical` module (the same one the Calendar connector uses — DRY) and turned into the same appointment fact cluster the Calendar connector emits: a primary `user has_event <event>` (typed `EventType::Appointment`, recurrence from `RRULE` `FREQ`, temporal bounds from `DTSTART`/`DTEND`), `<event> located_in <place>`, and `<attendee> attending <event>`. Entities resolve via the full F5 chain in `normalize_and_insert`; facts carry `source_type = Connector`, `connector_type = Gmail`, `extraction_method = StructuredParse`. |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win
Keep the documented iMIP eligibility rules consistent.
The implementation accepts every text/calendar MIME part, resolves METHOD from the MIME parameter or body, and extracts only REQUEST/REPLY.
docs/email-connector.md#L54-L54: remove the attachment-only and MIME-parameter-only wording.README.md#L44-L44: document bodyMETHODfallback and state that supported invitations can produce facts regardless of sender.
📍 Affects 2 files
docs/email-connector.md#L54-L54(this comment)README.md#L44-L44
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/email-connector.md` at line 54, Update docs/email-connector.md at line
54 to describe all eligible text/calendar MIME parts, including METHOD resolved
from either the MIME parameter or calendar body, and remove attachment-only and
MIME-parameter-only wording. Update README.md at line 44 to document the body
METHOD fallback and state that supported REQUEST/REPLY invitations can produce
facts regardless of sender.
…THOD values Reject a text/calendar part when the MIME method parameter and the iCalendar body METHOD property are both present and disagree (RFC 6047 §2.4), instead of silently preferring the MIME value. Add regression tests for conflicting supported/unsupported pairs, the body-only fallback, and the no-source case. Update docs/email-connector.md, docs/wiki/email-connector.md, and README.md to match the extraction contract (any text/calendar part, METHOD from MIME param or body, has_event depends on user_identity, supported REQUEST/REPLY produce facts regardless of sender, UIDVALIDITY-qualified provenance). Bump 0.85.1 → 0.85.2.
1 participant
Closes #200.
Summary
Implements the Email connector's structured fact extraction (Phase 3 C6 / #200) with a deliberately narrower scope than the original issue wording — see the implementation note on #200 and the discussion below.
Email is provenance, not the fact.
EmailConnector::extract()now runs a deterministic extraction cascade over each staged RFC 822 message. Today the cascade has one layer — iMIP calendar invites: a MIME attachment withContent-Type: text/calendar; method=REQUEST|REPLYis parsed withmail-parserand the embeddedVEVENTis turned into the same appointment fact cluster the Calendar connector emits —user has_event <event>(typedEventType::Appointment, recurrence fromRRULEFREQ, temporal bounds fromDTSTART/DTEND), plus<event> located_in <place>and<attendee> attending <event>— resolved via the full F5 entity chain innormalize_and_insert.PUBLISH/CANCELare skipped (CANCEL lifecycle → #247).received_email_from/sent_email_to) and noPersonentities auto-created fromFrom/Toheaders, so marketing/spam produces no junk facts. A plain prose email with notext/calendarpart produces nothing in C6.mimir-connectors/src/ical.rs(parse_ical_to_vevents,vevent_to_facts,RawVEvent), gatedany(feature = "calendar", feature = "gmail"). The Calendar connector (RawCalDavEventnow wrapsRawVEvent;event_to_factsdelegates) reuses it, eliminating the duplicated parsing +rrule_to_recurrence+calendar_facthelpers.ConnectorContext::user_identity(theconfig.toml[identity] name), matching the Calendar connector;from_config_with_httpgains auser_identityparameter and the factory passesctx.user_identity. Without an identity the primaryhas_eventfact is skipped; location/attendee facts still emit.Why the narrow scope (vs. the issue's "headers/dates/contacts")
A dentist's free-text "see you Tuesday 3pm" with no
.ics, a flight boarding pass in prose, a bank statement, a job offer — none carry structuredtext/calendar, so C6 emits nothing for them. Those are the LLM layer (C7 / #201). Transactional emails embedding machine-readableschema.orgJSON-LD (Order,FlightReservation,Ticket, …) are a deterministic layer to be added in #249, between invites and the LLM. Auto-creating a contact from every sender would fill the KB withmarketing@…entities — exactly the "false facts" #200's discussion flagged — so contacts emerge only from invite attendees (structured) and C7's LLM extraction. The cascade is built so #249 and C7 slot in without restructuringextract().Key changes
mimir-connectors/src/email/mod.rs—extract()cascade +extract_invites();user_identitywiring.mimir-connectors/src/ical.rs(new) — sharedRawVEvent,parse_ical_to_vevents,vevent_to_facts(+ movedparse_ical_datetime,participant_display,rrule_to_recurrence).mimir-connectors/src/calendar/{caldav,mod}.rs— refactored to reuseical(DRY).mimir-connectors/Cargo.toml—mail-parser 0.11.5undergmail;icalendar/chrono-tzshared withcalendar.docs/email-connector.md,docs/wiki/email-connector.md,README.md,docs/wiki/what-works-now.md,Mimir-Implementation-Context.md,CHANGELOG.md.0.84.2→0.85.0(MINOR, new feature).Tests
REQUEST/REPLY/PUBLISH/CANCELgating, no-identity path, plain/marketing email → no facts.extract()→normalize_and_insertasserts F5 entity resolution (user / event / place / attendees), theAppointmentevents-subsystem overlay, secondary facts carrying no overlay, and connector provenance (Connector/Gmail/StructuredParse/raw_reference= IMAP UID).extract()round-trip proving the transport and extraction compose end-to-end.RawVEventand remain green (22 + 140 lib).cargo test -p mimir-connectors,cargo clippy -p mimir-connectors --all-targets, andcargo fmtare clean.Pre-existing failures (not introduced here; filed separately)
tests/e2e.rsdoes not isolate the knowledge DB to its tempdir, causing amigration 45 … modifiedchecksum failure on machines with a priorknowledge.db(reproduces onmain).mimir-connectors/src/oauth.rsdead-code warnings under--no-default-features.Related issues updated for context
#201 (C7 boundary sharpened), #234 (ExtractionMethod relevance), #249 (JSON-LD follow-up), #250 (Photos fallback).
Summary by CodeRabbit
New Features
REQUESTandREPLYcalendar invites from email, including appointments, locations and attendees.Bug Fixes
Documentation