Cuga Integration Analysis.

[visahak]

@gaodan-fang @gjt-prog was doing an analysis on cuga and altk-evolve, and seems that we have missed some messages. Can we validate this issue?

Problem

The hand-rolled _convert_messages() in src/cuga/backend/evolve/integration.py (lines 100-119) only handles HumanMessage and AIMessage. It has two deficiencies:

1. ToolMessage skipped — the else: continue branch drops all tool results
2. AIMessage.tool_calls lost — only .content is extracted; for tool-calling turns, .content is often empty while the actual function name and arguments live in .tool_calls

The exported trajectory is incomplete: Evolve sees a conversation with gaps where tool interactions should be.

Solution

LangChain provides convert_to_openai_messages() which correctly handles all message types — including AIMessage with tool_calls and ToolMessage with tool_call_id. The entire hand-rolled method can be replaced.

File to Change

src/cuga/backend/evolve/integration.py — Replace _convert_messages()

Current implementation (lines 100-119):

@staticmethod
def _convert_messages(chat_messages: List[BaseMessage]) -> list:
    """Convert LangChain BaseMessage list to OpenAI conversation format."""
    result = []
    for i, msg in enumerate(chat_messages):
        if isinstance(msg, HumanMessage):
            role = "user"
        elif isinstance(msg, AIMessage):
            role = "assistant"
        else:
            logger.debug(f"Evolve: Skipping message {i} of type {type(msg).__name__}")
            continue
        content = msg.content if isinstance(msg.content, str) else str(msg.content)
        if content:
            result.append({"role": role, "content": content})
        else:
            logger.debug(f"Evolve: Skipping empty {role} message {i}")
    logger.debug(f"Evolve: Converted {len(result)}/{len(chat_messages)} messages")
    return result

Proposed implementation:

@staticmethod
def _convert_messages(chat_messages: List[BaseMessage]) -> list:
    """Convert LangChain BaseMessage list to OpenAI conversation format."""
    from langchain_core.messages import convert_to_openai_messages
    try:
        result = convert_to_openai_messages(chat_messages)
        logger.debug(f"Evolve: Converted {len(result)}/{len(chat_messages)} messages")
        return result
    except Exception as e:
        logger.warning(f"Evolve: Message conversion failed (non-fatal): {e}")
        return []

This produces correct OpenAI-format output for all message types:

LangChain Type	OpenAI Output	What's Preserved
HumanMessage	{"role": "user", "content": "..."}	Content
AIMessage (text only)	{"role": "assistant", "content": "..."}	Content
AIMessage (tool call)	{"role": "assistant", "tool_calls": [...], "content": ""}	Function name, arguments, call ID
ToolMessage	{"role": "tool", "tool_call_id": "...", "name": "...", "content": "..."}	Tool result, call ID, tool name

cc: @jayaramkr @vinodmut

[gaodan-fang]

Thanks for flagging this and for the analysis. I validated the behavior in cuga-agent, and your observation is correct.

What I confirmed:

1. state.chat_messages currently stores assistant turns as AIMessage(content=...) only in the CUGA Lite/Supervisor flow, so tool-call payloads are not preserved there.
2. I patched the Evolve export path in cuga-agent to use convert_to_openai_messages(), so ToolMessage and assistant tool_calls are now included when available.
3. However, in altk-evolve (tested with 1.0.10), save_trajectory stores raw message metadata, but tip generation (parse_openai_agents_trajectory) does not fully consume OpenAI-style top-level assistant.tool_calls and ignores role=tool observations in the guideline synthesis path.

So the issue is valid: we fixed the CUGA-side message conversion gap, but there is still a downstream parsing gap in altk-evolve if trajectories are provided in OpenAI tool_calls format. I can open a follow-up PR to make altk-evolve tips parsing handle:

• assistant tool_calls (top-level field), and
• tool observation messages (role: "tool").

[visahak]

Yes. Lets fix things on the ALTK-Evolve side as well cc: @jayaramkr

[gaodan-fang]

@visahak Upon further investigation:
cuga-agent: state.chat_messages doesn't come with tool_call information, while there's another variable in cuga's state, which is state.tool_calls, i thought it will have all the informations about tool_calls( which is a bit weird how does cuga handles tool_call information need to confirm with Sami) , but in real runs, i observed that it doesn't contain any tool_call information either.

Example of crm run in cuga-lite's state.chat_messages:
cuga_crm_state_chat_messages.txt

And upon the comparison of one of the cuga classic generated appworld's trajectory:
0d22252_1.json

As we can compare from above, neither of them have the tool call information, all the information are in messages' content which would mean we don't miss any message information and don't need to make the modification for cuga. But still, in order to corporate with other potential agents, in altk-evolve, we can still make the change to support tool calls and generate tips from there

[gaodan-fang]

Yes. Lets fix things on the ALTK-Evolve side as well cc: @jayaramkr

Issue opened for evolve's changes #223