Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 62 additions & 0 deletions .github/workflows/openwiki-update.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
name: OpenWiki Update

on:
workflow_dispatch:
schedule:
- cron: "0 8 * * *"

permissions:
contents: write
pull-requests: write

jobs:
update:
runs-on: ubuntu-latest
steps:
- name: Check out repository
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
with:
# Full history so `openwiki code --update` can diff HEAD against the
# commit it last documented; a shallow clone hides that commit and the
# update runs against an empty change summary.
fetch-depth: 0

- name: Set up Node.js
uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
with:
node-version: "22"

- name: Install OpenWiki
# mermaid + jsdom are optional; they add high-fidelity validation of Mermaid diagrams. Remove if your wiki has none.
run: npm install --global openwiki@0.3.3 mermaid@11.16.0 jsdom@29.1.1

- name: Run OpenWiki
run: openwiki code --update --print
env:
OPENWIKI_PROVIDER: baseten
BASETEN_API_KEY: ${{ secrets.BASETEN_API_KEY }}
OPENWIKI_MODEL_ID: "moonshotai/Kimi-K3"
# Required for the LangSmith connector's code-mode pull to authenticate.
# For extra workspaces, add OPENWIKI_LANGSMITH_API_KEY_2, _3, ... as repo
# secrets and env entries here.
OPENWIKI_LANGSMITH_API_KEY: ${{ secrets.OPENWIKI_LANGSMITH_API_KEY }}
# Optional: also trace this workflow's own OpenWiki run to LangSmith.
LANGSMITH_API_KEY: ${{ secrets.LANGSMITH_API_KEY }}
LANGCHAIN_PROJECT: openwiki
LANGCHAIN_TRACING_V2: "true"

- name: Create OpenWiki update pull request
uses: peter-evans/create-pull-request@22a9089034f40e5a961c8808d113e2c98fb63676 # v7
with:
add-paths: |
openwiki
AGENTS.md
CLAUDE.md
.github/workflows/openwiki-update.yml
branch: openwiki/update
commit-message: "docs: update OpenWiki"
title: "docs: update OpenWiki"
body: |
Automated OpenWiki documentation update.

This PR was generated by the scheduled OpenWiki workflow.
13 changes: 13 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,3 +103,16 @@ Before ending a work session:
5. Hand off remaining context clearly.

Do not commit or push unrelated uncommitted user changes.

<!-- OPENWIKI:START -->

## OpenWiki

This repository has a generated `openwiki/` evidence index. It is optional just-in-time context, not required startup reading.

- Treat source code and tests as authoritative. A brief's unknowns and review items are verification gaps, not automatic requirements.
- Prefer the narrowest quiet validation that proves the changed behavior. Preserve complete failure output.

The scheduled OpenWiki GitHub Actions workflow refreshes the repository wiki. Do not hand-edit generated OpenWiki pages unless explicitly asked; prefer updating source code/docs and letting OpenWiki regenerate.

<!-- OPENWIKI:END -->
7 changes: 7 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
<!-- OPENWIKI:START -->

## OpenWiki

See [AGENTS.md](AGENTS.md) for OpenWiki agent instructions.

<!-- OPENWIKI:END -->
24 changes: 24 additions & 0 deletions agentic_internet/tools/code_execution.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
"""Code execution tool for agents."""

import ast
import builtins
import io
import json
import logging
Expand All @@ -20,7 +21,30 @@

logger = logging.getLogger(__name__)

_BUILTIN_IMPORT = builtins.__import__


def _guarded_import(
name: str,
globals_: dict[str, Any] | None = None,
locals_: dict[str, Any] | None = None,
fromlist: tuple[str, ...] = (),
level: int = 0,
) -> Any:
"""Restricted ``__import__`` for executed code.

C extensions (e.g. numpy reductions) import submodules at runtime through
the frame's ``__builtins__``; without this entry they crash with
``KeyError: '__import__'``. The guard enforces the same module blocklist
as the AST validator, so aliased indirect imports stay blocked.
"""
if name.split(".")[0] in _ASTSafetyValidator.BLOCKED_MODULES:
raise ImportError(f"import of blocked module '{name}' is not allowed")
return _BUILTIN_IMPORT(name, globals_, locals_, fromlist, level)


ALLOWED_BUILTINS = {
"__import__": _guarded_import,
"print": print,
"len": len,
"range": range,
Expand Down
1 change: 1 addition & 0 deletions docs/exec-plans/active/.gitkeep
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
In-flight exec-plans live here (see docs/PLANS.md). Completed plans move to docs/exec-plans/completed/.
8 changes: 8 additions & 0 deletions openwiki/.last-update.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
{
"updatedAt": "2026-08-15T16:23:09.917Z",
"command": "init",
"gitHead": "bea3d8316ab249a8ae63910e4a1186ec147b51f4",
"model": "gpt-5.6-sol",
"status": "complete",
"language": "en"
}
1 change: 1 addition & 0 deletions openwiki/INSTRUCTIONS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
A code wiki for this repository.
4 changes: 4 additions & 0 deletions openwiki/agents/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# Files

- [Internet and Research Agents](internet-and-research.md) - Construction, tool registration, execution, chat, and research-history behavior for the core single-agent runtime.
- [Specialized Agents](specialized-agents.md) - Public domain-specific facades for browser automation, data analysis, content creation, market research, and technical support.
76 changes: 76 additions & 0 deletions openwiki/agents/internet-and-research.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
---
type: component guide
title: Internet and Research Agents
description: Construction, tool registration, execution, chat, and research-history behavior for the core single-agent runtime.
tags: [agents, runtime, research]
---

# Internet and Research Agents

`agentic_internet/agents/internet_agent.py` owns the default single-agent path. `InternetAgent` is the base public facade; `ResearchAgent` adds depth-specific prompts and in-memory history. CLI `chat`, `run`, `research`, and default `tools` all enter here.

## Construction

```mermaid
sequenceDiagram
participant Caller
participant IA as InternetAgent
participant MU as model_utils
participant CFG as settings
participant SA as smolagents
Caller->>IA: construct model tools and agent_type
IA->>MU: initialize_model
MU->>CFG: resolve alias provider and key
MU-->>IA: model
IA->>IA: assemble default tools when tools is falsey
IA->>SA: create ToolCallingAgent or CodeAgent
SA-->>IA: executable agent
```

*Construction resolves the model before default tools and the underlying smolagents agent.*

`__init__` accepts `model_id`, `tools`, `verbose`, `max_iterations`, `planning_enabled`, `agent_type`, and extra authorized imports. Important invariants:

- `tools or self._get_default_tools()` means `tools=[]` cannot express a tool-free agent; supply a nonempty custom list or change this behavior deliberately.
- `_initialize_model` delegates to `initialize_model`; a falsey result raises `ModelInitializationError` during construction.
- `agent_type == "code"` creates `CodeAgent(max_steps=max_iterations)` with fixed authorized imports (`pandas`, `numpy`, JSON/CSV/regex/date/time, `requests`, `urllib`, math/statistics/collections) plus caller additions.
- Every other value creates `ToolCallingAgent`; unknown values are not rejected. The tool-calling branch does not pass `max_iterations`.
- `planning_enabled` is stored but does not alter agent creation or execution.

The broader isolation implications are compared in [System Architecture](../architecture/overview.md).

## Default tools

`_get_default_tools` uses the import-time `settings` singleton:

| Gate | Registered tools |
|---|---|
| `settings.tools.web_search_enabled` | `WebSearchTool`, `WebScraperTool`, `NewsSearchTool` |
| Above plus `settings.exa_api_key` | `ExaSearchTool`, `ExaFindSimilarTool` |
| `settings.tools.browser_enabled` plus `settings.browser_use_api_key` | sync, async, and structured Browser Use tools |
| `settings.tools.code_execution_enabled` | `PythonExecutorTool`, `DataAnalysisTool` |
| Best-effort always | `smolagents.load_tool("calculator")`; failure is debug-logged and ignored |

MCP tools are never defaults; the [MCP CLI path](../tools/mcp-integration.md) injects them explicitly. Tool contracts live under [Web Search](../tools/web-search-and-scraping.md), [Exa](../tools/exa-search.md), [Browser Automation](../tools/browser-automation.md), and [Code Execution](../tools/code-execution-and-data.md).

## Run and chat lifecycle

`run(task, show_result=True, **kwargs)` optionally renders the task, calls `self.agent.run(task, **kwargs)`, optionally renders returned Markdown, and returns `str(result)`. It catches every exception, logs the traceback, and returns `"Error executing task: ..."`. Callers must inspect outcomes; exception-to-string conversion often leaves CLI exit status zero.

`chat()` is a terminal loop. `exit`, `quit`, and `bye` stop; `help` and `tools` display local help; all other input calls `run`. A local `{user, agent}` history is appended but never returned or persisted. `KeyboardInterrupt` exits, while other loop errors are printed and the loop continues. `_show_tools` expects each tool to expose a string `description`.

## ResearchAgent

`ResearchAgent` delegates construction and initializes `research_history`. `research(topic, depth)` selects fixed `quick`, `moderate`, or `deep` prompts, calls `run(show_result=False)`, then appends and returns:

```text
{topic, depth, findings, timestamp}
```

The timestamp is `pandas.Timestamp.now().isoformat()`. An unknown depth uses the moderate prompt but preserves the unknown value in returned metadata; the CLI prevents this through its own validation. `get_research_history()` returns the mutable internal list, not a defensive copy.

## Extension and validation

Override `_initialize_model`, `_get_default_tools`, or `_create_agent` for controlled variants; prefer injecting a nonempty tool list for consumers. When adding a default tool, implement and export it under `agentic_internet.tools`, add the correct setting/key gate here, update any K-LLM inventory separately, and test both enabled and disabled registration.

Focused coverage is currently narrow: `tests/test_exa_search.py` proves conditional Exa registration. There is no isolated test for model failure, agent construction, `run`, chat, `ResearchAgent`, planning, or tool-calling iteration limits. Use `uv run pytest tests/test_exa_search.py` plus the focused suite for the tool/model code changed; run `uv run pytest` for constructor-wide changes.
38 changes: 38 additions & 0 deletions openwiki/agents/specialized-agents.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
---
type: component guide
title: Specialized Agents
description: Public domain-specific facades for browser automation, data analysis, content creation, market research, and technical support.
tags: [agents, public-api]
---

# Specialized Agents

`agentic_internet/agents/specialized_agents.py` provides five root-exported subclasses of [InternetAgent](internet-and-research.md). They do not introduce new execution engines: each chooses constructor defaults, creates a detailed prompt, calls inherited `run`, and sometimes wraps the string in metadata. Prompt requests are intentions, not postcondition enforcement.

## Agent contracts

| Class and defaults | Methods and returned shape |
|---|---|
| `BrowserAutomationAgent`, `agent_type="code"`, `max_iterations=20` | `scrape_structured_data(url, data_schema)` requests JSON and returns a decoded dict only when the final text is a JSON object; arrays/scalars/invalid JSON become `{"raw_result": result}`. `fill_form` returns text. `monitor_website` performs one check and returns URL, target, findings, and a suggested interval—it does not schedule monitoring. |
| `DataAnalysisAgent`, code mode | `analyze_dataset(data, analysis_type)` returns `{analysis_type, insights}`. `compare_datasets` turns optional criteria into prompt bullets and returns text. Inputs are string-interpolated without size or schema controls. |
| `ContentCreationAgent`, tool-calling mode | `write_article(topic, style, word_count, sources_required)` returns topic/style/content/target count; it does not verify count or citations. `summarize_content(content, summary_type)` returns text. |
| `MarketResearchAgent`, tool-calling mode, `max_iterations=15` | `analyze_competitor` defaults to products, pricing, position, strengths, and weaknesses and returns `{company, aspects, analysis}`. `market_trends` returns `{industry, timeframe, trends}`. |
| `TechnicalSupportAgent`, code mode | `troubleshoot(problem_description, system_info)` JSON-renders optional system information and returns `{problem, system_info, solution}`. `code_review(code, language, focus_areas)` embeds a language fence and returns language, normalized focus list, and review. |

All inherited failures usually appear inside a result string because `InternetAgent.run` catches exceptions. A metadata wrapper can therefore look structurally successful while its content starts with `Error executing task:`. Code-mode classes inherit the local execution/network implications documented in [System Architecture](../architecture/overview.md).

## Ownership and dependencies

The module depends only on JSON handling and `InternetAgent`; actual models and tools come from the base class. `agentic_internet/__init__.py` and `agents/__init__.py` expose all five classes. `examples/advanced_usage.py` demonstrates selected methods but requires real model/provider configuration and is not a hermetic test.

## Extension recipe

For another domain facade:

1. Subclass `InternetAgent` and set defaults with `kwargs.setdefault` so callers can override them.
2. Keep side effects in registered tools rather than the prompt wrapper.
3. Define whether the method returns raw text or a stable dict. If parsing model output, preserve malformed output explicitly rather than silently discarding it.
4. Export from `agents/__init__.py` and root `agentic_internet/__init__.py` if it is public.
5. Add a focused test with construction bypassed or model/tool mocks, checking prompt-critical behavior and malformed/failure output—not only metadata.

`tests/test_specialized_agents.py` verifies object-versus-nonobject JSON handling for `scrape_structured_data` and article metadata. Constructor defaults and the other helper methods have no focused coverage. Run that file for facade changes and the base/tool suites when changing inherited behavior.
3 changes: 3 additions & 0 deletions openwiki/architecture/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Files

- [System Architecture](overview.md) - Runtime layers, dependency flow, orchestration paths, state ownership, and cross-cutting trust boundaries for Agentic Internet.
Loading
Loading