feat: add Koog run-reporting integration - #153
Conversation
|
Verify that the entire evaluation flow functions correctly—including agent modifications—and ensure that new 65d8d6a Switched both Koog example apps to a real |
|
Koog allows you to create a wide variety of agents; how do you determine when a conversation has ended, or do you need the ability to append messages within the Aigentic platform? Koog has no live-append API; every Check if this is needed. |
|
Example runs by tag / submit a list of annotations with a run — investigate integration with pre-fetch and splice Aigentic example runs into Koog prompts. Fix that linked runs are properly reported and input_hash does not get corrupted. 
No annotations are submitted with the run (to the Koog agent). Results from the Koog agent are evaluated with the expected values from existing annotations. Check if this is needed. |
|
Add test / demo with Spring AI and Koog |
63264d5 to
3045288
Compare
|
We need to think about how to manage token costs, because Koog has many more providers. Koog's own Root cause: |
Adds src/integrations/koog, which lets agents built with JetBrains' Koog framework report their runs to the Aigentic platform by hooking into Koog's EventHandler feature and reconstructing an AgentRun from Koog's message/tool-call events, then posting it through the existing PlatformClient. Also adds src/examples/koog-example, a minimal OpenAI-backed Koog agent with one tool that exercises the integration end-to-end.
sendRun's result was previously discarded, making it impossible to submit annotations for a Koog-reported run. reportRunsToAigentic now takes an optional onRunReported callback invoked with the RunId on success.
Switches the Koog example to a @serializable result type instead of plain text, and makes reportRunsToAigentic generic over the Koog output type so the real result (not a toString() dump) reaches the platform's serializer. Plain-text Koog output couldn't produce a usable annotation - the platform's evaluation model requires a JSON object response, and toString() on a String never produced one. Uses StructuredRequest.Native rather than Manual, since Manual mode injects an extra instruction message that would otherwise become the run's RUN_CONTEXT message instead of the actual user question.
Adds fetchExampleRunPrompt, which fetches tagged example runs via platform.getRuns and builds a Koog Prompt seeded with them (system prompt + example turns wrapped in the same markers the native DSL's fetchExampleRunMessages uses), since AIAgentConfig.prompt is Koog's equivalent seed-ahead-of-run() hook. SimpleKoogExample and WeatherAgentService now build their agent via AIAgentConfig(prompt = ...) instead of the systemPrompt string convenience overload. Also fixes the koog integration module's Gradle config, which was missing useJUnitPlatform() and silently ran zero test assertions.
KoogModelIdentifier formatted as "$provider/$id" (e.g. "openai/gpt-4o"), which never matched aigentic-platform's cost pricing table (keyed on bare model names like "gpt-4o"), silently pricing every Koog-reported run at $0. Drop the unused provider param so it matches the convention every other ModelIdentifier already follows. Also fix AigenticPlatformClient.getRuns throwing on a 404 instead of returning an empty list - a "no runs match these tags" response is a normal, expected state (e.g. before any run has been tagged yet), not an error, and both fetchExampleRunMessages and fetchExampleRunPrompt already handle an empty list gracefully.
CI caught a formatting violation that would have been avoided by running spotlessApply locally first.
aigentic-platform dropped this field (legacy expected-tool-calls experiment, commit e5e5081d) from its own gateway.ws, domain model, and DB schema, but aigentic's copy of the contract was never updated to match. Since the field was nullable with no default, kotlinx.serialization still required the JSON key at decode time, so any GetRuns response containing a tool call failed to deserialize on the client - breaking example-run-by-tag fetching for both the native DSL and Koog integrations. Adds a standalone live-check example (ExamplePromptLiveCheck) used to reproduce this against a real backend and verify the fix.
reportRunsToAigentic derived RUN_CONTEXT from the first User-role message in the agent's live Prompt at the first LLM call. That breaks as soon as fetchExampleRunPrompt splices few-shot example messages ahead of the real question: the "first user message" becomes the example preamble instead of the real input, and since input_hash is a hash of RUN_CONTEXT content, every run sharing a tag with a matching example collapsed to the same input_hash regardless of the actual question. Capture RUN_CONTEXT from AIAgentContext.agentInput instead - the literal value passed to agent.run(input), available in onAgentStarting before the strategy graph appends anything (question or spliced examples) to the live prompt. Removes the now-unused Prompt.initialUserText().
fetchExampleRunPrompt fetched runs matching a tag and spliced their content into the Koog prompt, but discarded the RunIds it had just fetched - so reportRunsToAigentic had nothing to report and the platform's "Linked example runs" panel always showed empty, even for runs that genuinely used spliced-in examples. Change fetchExampleRunPrompt to return the matched RunIds alongside the Prompt, and add an exampleRunIds parameter to reportRunsToAigentic that flows into AgentRun.exampleRunIds - the same field the native DSL already populates via state.addExampleRunId, and what "Linked example runs" reads. Updates the two example call sites and the live-check tool accordingly.
Restored this file by hand after a spotlessApply run corrupted it (deleted it from disk due to a stale build/spotless-clean cache) before formatting could complete; the restored content was correct but not yet reformatted.
22c4689 to
8d15632
Compare
Explains the reporting-only nature of the Koog integration, what currently works (single-agent runs, tool/message mapping, structured output, few-shot examples), and what's not supported yet (Stuck outcome, Workflow DSL, cross-framework tools, attachments, JVM-only).
Reduces Koog wiring boilerplate to match the ergonomics of Koog's own
install(OpenTelemetry) { addLangfuseExporter() } pattern. Adds
defaultAigenticPlatform(), reading AIGENTIC_PLATFORM_NAME/SECRET/URL via
Koog's multiplatform env accessor, with explicit params overriding env
lookups. reportRunsToAigentic and fetchExampleRunPrompt now default their
platform param to it, and fetchExampleRunPrompt's tags now defaults to
emptyList(). A minimal app with no example-seeding is now a one-liner.
…ature
Replaces reportRunsToAigentic (an inline reified extension function) with
Aigentic : AIAgentGraphFeature<AigenticConfig, Aigentic>, matching Koog's
own install(OpenTelemetry) { } pattern exactly. The feature itself isn't
generic - AigenticConfig.outputSerializer is a plain KSerializer<Any>
field, set via the reified outputType<Output>() convenience helper at the
caller's own call site, the same way kotlinx.serialization's non-reified
encodeToString(serializer, value) works underneath its reified overload.
Every layer below the old reportRunsToAigentic already took a plain
KSerializer<O> parameter; reified was only ever needed in the thin
top-level Platform.sendRun wrapper.
For a minimal Koog app with no example-run seeding, wiring is now a true
one-liner: install(Aigentic) { task = Task(...) }. Structured output adds
one line: outputType<Output>(). Example-run seeding (fetchExampleRunPrompt)
is unaffected - still can't fold into install() for an unrelated reason
(no prompt-mutation hook in Koog) - but now sets tags/exampleRunIds inside
the same install(Aigentic) block used for reporting.
reportRunsToAigentic and manual AigenticPlatform construction were
removed from the codebase this session, replaced by the Aigentic Koog
feature. Updates the Basic Usage example to lead with the minimal
install(Aigentic) { task = ... } one-liner, adds an outputType<Output>()
example for structured output, and brings the weather example and the
supported/not-yet-supported lists in line with the current API.
2 participants
Adds src/integrations/koog, which lets agents built with JetBrains' Koog framework report their runs to the Aigentic platform by hooking into Koog's EventHandler feature and reconstructing an AgentRun from Koog's message/tool-call events, then posting it through the existing PlatformClient.
Also adds src/examples/koog-example, a minimal OpenAI-backed Koog agent with one tool that exercises the integration end-to-end.