Skip to content

[Bug]: Lark channel - Feishu retries every event (ack not accepted); retried messages re-executed as new chats #796

Description

@yjjoseph

[Bug] Lark channel: platform marks every long-connection event push as FAIL/timeout and retries it; redeliveries are re-executed as new chat messages

Platform

Windows 11

AionUi Version

2.1.49 (bundled AionCore ≈ v0.1.59). Checked the v2.1.50 changelog — no related fix.

Bug Description

Every message a user sends to the Lark (Feishu) bot over the WebSocket long connection is processed by AionUi 3–4 times over the following ~6 hours. From the user's side it looks like the bot "re-asks / re-answers questions that were already asked".

Root cause appears to be two compounding problems:

  1. The platform never accepts AionUi's event acknowledgment. In the Feishu Open Platform event log, every push (including the original, first delivery) is recorded with status FAIL after costMills ≈ 3752 ms (a timeout). Feishu therefore retries each event on its failure-retry schedule (~5 min / ~1 h / ~6 h, max 3 retries), reusing the same eventId.
  2. AionUi's dedup window is far shorter than the retry window. LARK_EVENT_DEDUP_TTL is only 5 minutes (crates/aionui-channel/src/constants.rs:55) and the cache is in-memory. The 1-hour and 6-hour redeliveries (and even the 5-minute one, which lands slightly past the TTL) all pass through and are dispatched to the agent again as brand-new user messages.

Each redelivery triggers a full agent run (token cost) and the bot sends the answer card to the user again.

Evidence

Feishu Open Platform event log (filtered 返回状态 = FAIL): the same eventId appears multiple times at retry-schedule intervals, all FAIL with costMills ≈ 3752 ms and a timeout field, e.g.:

  • eventId …52911e47310330… pushed at 2026-08-06 20:02:5x and again at 21:02:59.731 (same event, ~1 h apart)
  • eventId …205e3c97d977… pushed at 2026-08-06 20:01:15.722 and again at 2026-08-07 02:01:2x (~6 h apart, to the second)
  • eventId …4a545e5cd5db…4944 pushed at 2026-08-06 20:00:21.860 and again at 2026-08-07 02:00:26.774

AionUi local message store — each redelivery is stored and executed as a new user message (all times local, UTC+8):

User message (sent once) First processed Re-dispatched at
"Hi?" 08-06 19:13:39 20:19:02, 08-07 02:19:06
"Which model are you?" 19:45:14 19:51:05 (failed: conversation busy), 20:51:11, 02:51:15
"Translate: 2026年7月20日 …" (461 chars) 19:57:21 20:02:47, 21:02:58, 03:03:02
"Are you on?" 20:01:12 20:04:27, 20:17:02, 02:01:16, 02:04:31, 02:17:07

The 6-hour-later redeliveries land exactly +6h00m0Xs after the original, matching the platform retry schedule — these are unattended replays (03:03 AM local), not the user re-sending.

aioncore log excerpts:

[2026-08-06 19:13:21.469] [info]  aionui_channel::plugins::lark::plugin: Lark WebSocket connected
[2026-08-06 19:16:56.724] [error] aionui_channel::orchestrator: failed to send to agent error=Message send failed: Conversation is busy: conversation 54c4d720 is already running
[2026-08-06 20:02:47.002] [info]  aionui_channel::action: message dispatched to agent session_id=019fd6c7-… chat_id=oc_96c6c15b66c1cd5976bbae3312a6631d text_len=1218
[2026-08-06 21:02:56.017] [info]  aionui_channel::action: message dispatched to agent session_id=019fd6c7-… chat_id=oc_96c6c15b66c1cd5976bbae3312a6631d text_len=1218   # same message, 1 h later

Steps to Reproduce

  1. Configure the Lark channel (WebSocket long connection), pair a user, bind it to an agent conversation.
  2. Send any message to the bot; it is answered normally.
  3. In Feishu Open Platform → 事件与回调 → 日志检索, filter 返回状态 = FAIL: the push is logged as FAIL after ~3.7 s, and the same eventId reappears ~5 min / ~1 h / ~6 h later.
  4. Each redelivery is dispatched to the agent again and the bot replies again.

Expected Behavior

  1. The ack frame sent immediately on receipt (build_ack_frame, payload {"code":200}, sent in connect_and_listen before processing) should make the platform treat the push as successful — no FAIL entries, no retries. Please compare the response frame byte-for-byte against the official larksuite/oapi-sdk-go WS client (which headers are echoed — AionUi keeps only type/message_id/trace_id — plus payload_type/payload_encoding, biz_rt, and the exact response payload shape).
  2. Defense in depth: even when a redelivery does arrive, it should never be re-executed. The event-id dedup should cover the platform's full retry window — persist seen event_ids (or message_ids) for at least 24 h and across restarts, instead of a 5-minute in-memory cache.

Related

Image Image Image Image

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions