Skip to content

Feat/page index style local opt #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
humblemat810 wants to merge 8 commits into
base: main
Choose a base branch
from
Open

Feat/page index style local opt #2

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
7eb58ab
feat: smoke test for ollama
humblemat810 May 16, 2026
80d7b03
feat: page-index parser slices 1-6 refactor
humblemat810 May 17, 2026
46bd723
feat(page-index): add optional excerpt refinement and Qwen-first guid…
humblemat810 May 17, 2026
5b34b6c
feat: small medium tiny doc support with fuzzy repair
humblemat810 May 17, 2026
4cce5e6
test: test fuzzy match
humblemat810 May 17, 2026
3054417
refactor: fuzzy primitives from kogwistar
humblemat810 May 17, 2026
55b2135
feat: page index support retried parsing and structuring
humblemat810 May 18, 2026
5cdba22
fix: fuzzy offset compat, conftest, probe trace emit
humblemat810 May 19, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .vscode/settings.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,8 @@
"python-envs.defaultEnvManager": "ms-python.python:poetry",
"python-envs.defaultPackageManager": "ms-python.python:poetry",
"python.analysis.extraPaths": [
"./src"
"./src",
"./kogwistar"
],
"files.watcherExclude": {
"**/node_modules": true,
Expand Down
14 changes: 12 additions & 2 deletions conftest.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,17 @@
from pathlib import Path
import sys

_VENDORED_KOGWISTAR = Path(__file__).resolve().parent / "kogwistar"
if str(_VENDORED_KOGWISTAR) not in sys.path:
sys.path.insert(0, str(_VENDORED_KOGWISTAR))

import pytest


@pytest.fixture
def gemini_key():
import dotenv
dotenv.load_dotenv()
import os
return os.environ.get('GOOGLE_API_KEY')

dotenv.load_dotenv()
return os.environ.get("GOOGLE_API_KEY")
225 changes: 225 additions & 0 deletions kg_doc_parser/semantic_document_splitting_layerwise_edits.py
View file
Open in desktop

Large diffs are not rendered by default.

14 changes: 13 additions & 1 deletion kg_doc_parser/workflow_ingest/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,15 @@
run_page_index_source_workflow,
)
from .smoke_assets import generate_ocr_smoke_assets
from .page_index import PageIndexBlockSpec, PageIndexParseResult, build_page_index_workflow_input
from .page_index import (
BlockAssignment,
BlockAssignmentBatch,
CandidateBlock,
PageIndexBlockSpec,
PageIndexParseResult,
PageIndexValidationResult,
build_page_index_workflow_input,
)
from .models import (
BoundingBox,
CanonicalGraphWriteResult,
Expand Down Expand Up @@ -128,6 +136,7 @@
"PageIndexParseRequest",
"PageIndexBlockSpec",
"PageIndexParseResult",
"PageIndexValidationResult",
"ParseMode",
"TreeParseRequest",
"SemanticNode",
Expand All @@ -144,6 +153,9 @@
"OcrWorkflowCommandResult",
"PageIndexWorkflowCommandResult",
"LayerwiseWorkflowCommandResult",
"CandidateBlock",
"BlockAssignment",
"BlockAssignmentBatch",
"DEFAULT_WORKFLOW_ID",
"build_chat_model",
"build_chat_model_for_role",
Expand Down
Loading