[Feature Request] Support User-Controlled Tool Execution as Spring AI

[zhuyouxiang]

While PromptRunner is the standard interface for LLM calls, scenarios like fast research only require the LLM for query understanding and parameter decomposition.

In these cases, the final "Generation" step is redundant. Moreover, the "Long-In, Long-Out" nature of these tasks causes significant latency.

Could Embabel support User-Controlled Tool Execution, similar to Spring AI： User-Controlled Tool Execution?
This would allow PromptRunner to terminate immediately after tool execution, bypassing the final generation round to minimize latency and cost while preserving the Agent's intent-parsing capabilities.

[johnsonr]

Interesting use case.

Since we took full control of the agentic loop, an Embabel tool can trigger replanning. See Tool.replanAlways and related methods.

If this doesn't meet your need, we could also have a tool trigger loop exit. This would be easy to implement, but we'd have to decide how the user handles the return of the PromptRunner call. An exception?

[zhuyouxiang]

Thanks for the suggestion!

I've explored the Tool.replanAlways approach, but I found a limitation in the current implementation that prevents the application layer from retrieving the tool's execution result.

By the way, Is there a more concise way to retrieve the MCP Tool instance from OperationContext?

	@Action
	public String fastSearch(UserInput userInput, OperationContext context) {
		Tool webSearch = context
				.agentPlatform()
				.getToolGroupResolver()
				.resolveToolGroup(new ToolGroupRequirement(CoreToolGroups.WEB))
				.getResolvedToolGroup()
				.getTools()
				.getFirst();

		try {
			context.ai().withAutoLlm().withTool(Tool.replanAlways(webSearch)).generateText("");
		} catch (ReplanRequestedException e) {
			e.getBlackboardUpdater().accept(context.getAgentProcess().getBlackboard());
		}

		return "";
	}

When updating the blackboard, the tool's result isn't being properly mapped or preserved for the final output. Specifically:

fun replanAlways(tool: Tool): Tool {
    return ConditionalReplanningTool(tool) { context ->
        ReplanDecision("${tool.definition.name} replans") { bb ->
            context.artifact?.let { bb.addObject(it) }
        }
    }
}

Something else I noticed, but I'm not sure if it's a concern.
I initially used createObject instead of generateText，buildResult will exception when call outputParser

DefaultToolLoop
            for (toolCall in assistantMessage.toolCalls) {
                val shouldContinue = processToolCall(toolCall, state)
                if (!shouldContinue) {
                    // Tool requested replan - terminate loop gracefully
                    logger.info("Tool loop terminated for replan after {} iterations", state.iterations)
                    return buildResult("", outputParser, state)
                }
            }

[zhuyouxiang]

If this doesn't meet your need, we could also have a tool trigger loop exit. This would be easy to implement, but we'd have to decide how the user handles the return of the PromptRunner call. An exception?

Tool.replanAlways is not quite ideal, As it is per-tool, but since the LLM can trigger concurrent calls, we can't capture the complete results.
Return type handling is indeed a major challenge, perhaps we should expose Flux<ChatResponse> and append tool result to Message list?

[johnsonr]

There is now extensive support for taking artifacts from tools. See https://docs.embabel.com/embabel-agent/guide/0.3.4-SNAPSHOT/#artifactsinkingtool-capturing-tool-outputs

[johnsonr]

Re MCP tools, see https://github.com/embabel/impromptu/blob/main/src/main/java/com/embabel/impromptu/agents/ChatConfiguration.java for the best way to expose MCP tools in an isolated way.

[johnsonr]

Given this discussion, is there still a concrete enhancement that would meet your need? Thanks

[zhuyouxiang]

Given this discussion, is there still a concrete enhancement that would meet your need? Thanks

1. ReplanningTool meets my need, which handles Text (content) Result. This issue can be closed.

ReplanningTool searchOnly = new ReplanningTool(webSearch, "search only");

2. The handling of Result is inconsistent between ReplanningTool（content only) and Tool.replanAlways (using ConditionalReplanningTool? artifact only ), this needs to be reviewed.