Code cleanup: Ruff linting improvements

.claude/context/CONTEXT.md

@@ -23,6 +23,11 @@ The pattern itself.
 
 - [Branch Protection Workflow](decisions/decision-branch-protection-workflow.md) — Never push directly to master; all changes via feature branches and PRs
 
+## Implementation
+
+- [LanceDB Flat Schema Support](implementation/implementation-lancedb-flat-schema.md) — Handle both flat schemas (Contextus) and nested metadata with content column detection
+
 ## Reference
 
+- [Subtitle Test Data](reference/reference-subtitle-test-data.md) — Subtitle files as ideal test data for semantic search and embedding validation
 - [Version Bump Workflow](reference/reference-version-bump-workflow.md) — Files that need updating when bumping release version

.claude/context/implementation/implementation-lancedb-flat-schema.md

@@ -0,0 +1,121 @@
+---
+name: lancedb-flat-schema-support
+description: LanceDB databases can use flat schemas (columns) or nested metadata - handle both patterns with content column detection
+metadata:
+  type: implementation
+  tags: [lancedb, metadata, schema, contextus]
+---
+
+# LanceDB Flat Schema Support
+
+## Problem
+
+LanceDB supports two metadata storage patterns:
+
+1. **Nested metadata** (traditional): Single `metadata` column containing a dict/JSON
+2. **Flat schema** (Contextus-style): Individual columns like `project`, `filename`, `heading`, `type`, `chunk_index`
+
+Vector Inspector originally only supported pattern #1, causing Contextus databases to show empty metadata.
+
+## Solution Pattern
+
+### In `get_collection_info()`
+
+1. **Extract from PyArrow schema first** (most reliable):
+   ```python
+   schema = tbl.schema
+   reserved_columns = {"id", "vector", "embedding", "_distance"}
+
+   # Detect content column to exclude it
+   temp_schema = {field.name: str(field.type) for field in schema}
+   content_col = self._detect_content_column(name, temp_schema)
+   reserved_columns.add(content_col)
+
+   # All other columns are metadata
+   metadata_fields = [
+       field.name for field in schema
+       if field.name not in reserved_columns and not field.name.startswith("_")
+   ]
+   ```
+
+2. **Fallback to pandas** if schema extraction fails
+
+3. **CRITICAL**: Must also check `count is None` in fallback condition, not just metadata/vector detection
+
+### In `get_all_items()`
+
+1. **Detect content column FIRST** to build reserved set:
+   ```python
+   schema = {col: str(dtype) for col, dtype in df.dtypes.items()}
+   content_col = self._detect_content_column(collection_name, schema)
+   reserved_columns = {"id", "vector", "embedding", "_distance", content_col}
+   ```
+
+2. **Check for nested metadata column first**:
+   ```python
+   if "metadata" in df.columns:
+       # Parse nested format
+       raw_meta = df["metadata"].tolist()
+       metadatas = self._parse_metadata_list(raw_meta)
+   ```
+
+3. **Otherwise extract flat schema columns**:
+   ```python
+   else:
+       metadata_columns = [
+           col for col in df.columns
+           if col not in reserved_columns and not col.startswith("_")
+       ]
+
+       if metadata_columns:
+           records = df[metadata_columns].to_dict('records')
+           metadatas = [
+               {k: v for k, v in record.items()
+                if v is not None and (not isinstance(v, float) or not math.isnan(v))}
+               for record in records
+           ]
+   ```
+
+## Why Exclude Content Column
+
+The content column (e.g., "document", "text", "content") serves a special purpose:
+- It's displayed in the "Document" column of the data browser
+- If also included in metadata, it appears TWICE in the table (once as "Document", once as "document")
+- Users expect metadata to be *additional* fields, not duplicates of the main content
+
+## Edge Cases
+
+1. **Empty content column**: If content detection returns None or empty string, don't add it to reserved set
+2. **NaN values**: Filter out `math.isnan()` values when building metadata dicts from flat schemas
+3. **Count detection**: Even if schema extraction succeeds, must still fetch dataframe if `count is None`
+4. **Backward compatibility**: Nested metadata format must continue to work (used by most LanceDB databases)
+
+## Testing
+
+Always test both patterns:
+```python
+# Flat schema (Contextus-style)
+data = [
+    {
+        "id": "1",
+        "vector": [0.1, 0.2, 0.3],
+        "document": "content",
+        "project": "test",
+        "filename": "file.md",
+    }
+]
+
+# Nested metadata (traditional)
+conn.add_items(
+    collection,
+    documents=["doc1"],
+    metadatas=[{"key": "value"}],
+    ids=["id1"],
+    embeddings=[[0.1, 0.2]]
+)
+```
+
+## Files Modified
+
+- `src/vector_inspector/core/connections/lancedb_connection.py`: Both `get_collection_info()` and `get_all_items()`
+- Tests: `tests/providers/lancedb/test_flat_schema_metadata.py`

.claude/context/reference/reference-subtitle-test-data.md

@@ -0,0 +1,36 @@
+---
+name: subtitle-test-data
+description: Subtitle files as ideal test data for semantic search and embedding validation
+metadata:
+  type: reference
+  tags: [testing, embeddings, test-data, semantic-search]
+---
+
+## Subtitle Files for Semantic Search Testing
+
+Subtitle files (.srt, .vtt, .sub) make excellent test data for semantic search and embedding systems.
+
+### Why Subtitles Work Well
+
+1. **Natural Language**: Real conversational text with context and flow
+2. **Sequential Context**: Lines build on each other, testing context window handling
+3. **Varied Domains**: Movies, documentaries, lectures provide diverse vocabulary
+4. **Timestamp Metadata**: Built-in temporal structure for testing metadata handling
+5. **Manageable Size**: Individual subtitle entries are good chunk sizes (typically 1-3 sentences)
+6. **Widely Available**: Easy to obtain for testing without licensing concerns
+
+### Characteristics for Embedding Tests
+
+- **Semantic Relationships**: Related dialogue lines test similarity search accuracy
+- **Temporal Coherence**: Tests whether embeddings capture narrative flow
+- **Speaker Attribution**: Can test multi-speaker scenarios
+- **Domain Shifts**: Scene changes test how embeddings handle topic transitions
+
+### Usage in Vector Inspector
+
+Use subtitle files to validate:
+- Content column detection across providers
+- Embedding quality with different models (OpenAI, Ollama, HuggingFace)
+- Search result relevance and ranking
+- Metadata extraction and display
+- Performance with realistic document collections

.claude/context/reference/reference-version-bump-workflow.md

@@ -1,14 +1,17 @@
 ---
 name: version-bump-workflow
-description: How to bump release version in vector-inspector - files that need updating
+description: Version bump checklist - always update pyproject.toml, src/vector_inspector/__init__.py __version__, and docs/RELEASE_REASON.md together when changing release version numbers
 metadata:
   type: reference
-  tags: [release, versioning, workflow]
+  tags: [release, versioning, workflow, version-bump, release-prep, changelog, pyproject, __init__, RELEASE_REASON]
+  trigger_files: [pyproject.toml, src/vector_inspector/__init__.py, docs/RELEASE_REASON.md]
 ---
 
 # Version Bump Workflow
 
-When bumping a release version in vector-inspector, **three files** must be updated to stay in sync:
+When bumping a release version in vector-inspector (changing version numbers, updating pyproject.toml version field, preparing for release), **three files** must be updated to stay in sync:
+
+**Critical**: Never update just pyproject.toml or just __init__.py alone — all three files must change together or the release will be incomplete.
 
 ## Files to Update
 
@@ -76,8 +79,13 @@ def get_version():
 ❌ Only updating `pyproject.toml` → `__version__` out of sync for dev mode
 ❌ Only updating `__init__.py` → Release workflow won't trigger
 ❌ Forgetting `docs/RELEASE_REASON.md` → Release has no description
+❌ Updating pyproject.toml version without checking __init__.py
+❌ Changing version in code but not updating release notes
+❌ Editing CHANGELOG.md but forgetting to sync __version__
 
-✅ Update all three files together
+✅ Update all three files together — pyproject.toml, __init__.py, and RELEASE_REASON.md
+✅ Always check __init__.py when pyproject.toml version changes
+✅ Version numbers must match across all three files
 
 ## Related
 

.github/workflows/ci-tests.yml

@@ -12,12 +12,22 @@ on:
           - '["3.11", "3.12"]'
           - '["3.11"]'
           - '["3.12"]'
+  pull_request:
+    branches:
+      - master
+      - main
   push:
+    branches:
+      - master
+      - main
     paths:
       - 'pyproject.toml'
       - '.github/workflows/ci-tests.yml'
       - 'tests/**'
 
+permissions:
+  contents: read
+
 jobs:
   test:
     name: Test on Python ${{ matrix.python-version }}

.github/workflows/lint.yml

@@ -0,0 +1,38 @@
+name: Lint
+
+on:
+  pull_request:
+    branches:
+      - master
+      - main
+  push:
+    branches:
+      - master
+      - main
+
+permissions:
+  contents: read
+
+jobs:
+  lint:
+    name: Lint with Ruff
+    runs-on: ubuntu-latest
+    continue-on-error: true  # Advisory only - won't block PRs
+
+    steps:
+      - name: Check out repository
+        uses: actions/checkout@v4
+
+      - name: Set up Python
+        uses: actions/setup-python@v4
+        with:
+          python-version: "3.12"
+
+      - name: Install Ruff
+        run: pip install ruff
+
+      - name: Run Ruff linter
+        run: ruff check src/ tests/ --output-format=github
+
+      - name: Run Ruff formatter check
+        run: ruff format --check src/ tests/

CHANGELOG.md

@@ -5,6 +5,27 @@ All notable changes to Vector Viewer will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.8.4] - 2026-06-21
+
+### Fixed
+- **LanceDB flat schema metadata extraction**: Fixed metadata display for LanceDB databases with flat schemas (e.g., Contextus) where metadata is stored as individual columns rather than nested in a 'metadata' column
+- **Duplicate document column**: Prevented content column from appearing twice in data browser (once as content, once as metadata)
+- **Column width limits**: Added 600px max-width constraint to table columns to prevent excessive width
+
+### Changed
+- LanceDB connection now extracts metadata from PyArrow schema first for better performance
+- Content column (e.g., "document") is automatically detected and excluded from metadata fields
+
+### Added
+- Comprehensive tests for flat schema metadata extraction
+- Backward compatibility maintained with nested 'metadata' column format
+
+## [0.8.3] - 2026-06-21
+
+### Changed
+- Moved hashlib import to module level in base_connection.py
+- Improved platform-specific monospace font rendering in provider install dialog
+- Centralized content column detection logic with single return path
 
 ### Added - Phase 1 Implementation
 

CLAUDE.md

@@ -0,0 +1,12 @@
+<!-- contextus:start -->
+## Context (claude-contextus)
+
+This project uses [claude-contextus](https://github.com/anthonypdawson/claude-contextus):
+relevant patterns from `.claude/context/` are **retrieved and injected automatically**
+on each prompt — you don't need to search for them. Injected blocks are tagged
+`[contextus — …]`.
+
+Context capture behavior is controlled by the per-prompt injected footer — follow
+its instructions exactly (it may tell you to propose, write directly, or stay silent).
+New context files live under `.claude/context/<category>/` and are embedded automatically.
+<!-- contextus:end -->

docs/RELEASE_REASON.md

@@ -1,24 +1,22 @@
-# Release Notes (0.8.3) — June 21, 2026
+# Release Notes (0.8.4) — June 23, 2026
 
-Content column detection improvements and local embedding support.
+Critical LanceDB metadata extraction fixes and column rendering improvements.
 
-## Features
+## Bug Fixes
 
-- **Dynamic content column detection**: Automatically detects the best content column for each collection based on schema analysis (text/string fields, common naming patterns). Manual override support with persistence across sessions.
-- **Ollama embedding integration**: Local embedding generation via Ollama HTTP API for environments where HuggingFace is blocked or unavailable.
+- **LanceDB flat schema metadata extraction**: Fixed metadata display for LanceDB databases with flat schemas (e.g., Contextus) where metadata is stored as individual columns rather than nested in a 'metadata' column
+- **Duplicate document column**: Prevented content column from appearing twice in data browser (once as content, once as metadata)
+- **Column width limits**: Added 600px max-width constraint to table columns to prevent excessive width from breaking UI
 
 ## Improvements
 
-- Content column configuration UI shows both auto-detected recommendation and currently active column
-- Settings persistence for content column overrides per collection
-- Fixed LanceDB schema detection to use PyArrow schema instead of pandas DataFrame conversion (performance improvement)
-- Fixed threading deadlock in content column detection (switched from Lock to RLock)
-- Platform-specific monospace fonts in UI dialogs (Menlo on macOS, Consolas on Windows)
+- LanceDB connection now extracts metadata from PyArrow schema first for better performance
+- Content column (e.g., "document") is automatically detected and excluded from metadata fields
+- Backward compatibility maintained with nested 'metadata' column format
 
-## Bug Fixes
+## Testing
 
-- Fixed LanceDB metadata array length mismatch in search results
-- Fixed QThread leak in embedding configuration dialog
-- Fixed content column cache poisoning when schema unavailable
+- Added comprehensive tests for flat schema metadata extraction
+- CI workflow now runs unit tests on all pull requests
 
 ---

pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "vector-inspector"
-version = "0.8.3"
+version = "0.8.4"
 description = "A comprehensive desktop application for visualizing, querying, and managing vector database data"
 authors = [
     {name = "Anthony Dawson", email = "anthonypdawson+github@gmail.com"},
@@ -204,7 +204,26 @@ select = [
     "ARG",
     "RUF",
 ]
-ignore = ["E501", "I001", "UP045", "SIM105", "SIM108", "SIM117"]
+ignore = [
+    "E501",    # Line too long (black handles this)
+    "I001",    # Import block unsorted
+    "UP045",   # Use Union[X, Y] instead of X | Y for py3.9 compat
+    "SIM105",  # Use contextlib.suppress instead of try/except pass
+    "SIM108",  # Use ternary operator
+    "SIM117",  # Use single with statement
+    # Unused arguments - often required for interface compliance (Qt signals, abstract methods)
+    "ARG001",  # Unused function argument
+    "ARG002",  # Unused method argument
+    "ARG003",  # Unused class method argument
+    "ARG004",  # Unused static method argument
+    "ARG005",  # Unused lambda argument
+    # Test-specific tolerances
+    "E741",    # Ambiguous variable name (l, O, I) - common in math/test code
+    "RUF003",  # Ambiguous unicode in comments (×, −) - intentional in documentation
+    "RUF002",  # Ambiguous unicode in docstrings
+    "DTZ005",  # datetime.now() without tz - not critical for cache timestamps
+    "B904",    # raise without from - acceptable for wrapped import errors
+]
 
 [tool.ruff.lint.isort]
 combine-as-imports = true

src/vector_inspector/__init__.py

@@ -1,11 +1,11 @@
 """Vector Inspector - A comprehensive desktop application for vector database visualization."""
 
-__version__ = "0.8.3"  # Keep in sync with pyproject.toml for dev mode fallback
+__version__ = "0.8.4"  # Keep in sync with pyproject.toml for dev mode fallback
 
 
 def get_version():
     try:
-        from importlib.metadata import PackageNotFoundError, version
+        from importlib.metadata import version
 
         return version("vector-inspector")
     except Exception: