Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
85 commits
Select commit Hold shift + click to select a range
4bbdce3
feat(thoughtspot): package skeleton, CI workflow, and ASF header gate
djwaldo Sep 1, 2026
10a3458
fix(thoughtspot): add ASF header to pyproject.toml, pin CI actions to…
djwaldo Sep 1, 2026
8a4d93a
feat(thoughtspot): vocabulary constants with dialect registration flag
djwaldo Sep 1, 2026
b84ae93
feat(thoughtspot): YAML 1.2 codec so on/off/yes/no survive a round trip
djwaldo Sep 1, 2026
af4267f
test(thoughtspot): pin loader/dumper fixes independently with plain-P…
djwaldo Sep 1, 2026
c876c72
feat(thoughtspot): structured never-silent issue reporting
djwaldo Sep 1, 2026
bfd941a
feat(thoughtspot): custom_extensions stash with staleness-guarded res…
djwaldo Sep 1, 2026
cf61868
feat(thoughtspot): identifier normalisation with case-folded collisio…
djwaldo Sep 1, 2026
db7b364
fix(thoughtspot): reject ambiguous column refs; document normalise AS…
djwaldo Sep 1, 2026
e781a34
feat(thoughtspot): key derivation that refuses to manufacture false keys
djwaldo Sep 1, 2026
3c94e07
fix(thoughtspot): report empty-to_columns relationships, document KD3…
djwaldo Sep 1, 2026
43cfe07
docs(thoughtspot): README with direction statement and coverage matrix
djwaldo Sep 1, 2026
425c5ae
docs(thoughtspot): add identifier-derivation limitations, tighten L2 …
djwaldo Sep 1, 2026
53f5286
build(thoughtspot): match sibling converters' packaging and CI shape
djwaldo Sep 1, 2026
35aa59d
fix(thoughtspot): correct KD2's upstream-warning claim; harden _yaml …
djwaldo Sep 1, 2026
ad1a13e
fix(thoughtspot): fold diacritics in normalise(); close a column-ref …
djwaldo Sep 1, 2026
13dad2f
docs(thoughtspot): future-tense unimplemented claims; name the extern…
djwaldo Sep 1, 2026
cf102a0
docs(thoughtspot): fix README dev commands for the hatchling/uv packa…
djwaldo Sep 2, 2026
f17c2b9
fix(thoughtspot): flip DIALECT_IS_REGISTERED now that ossie#351 merged
djwaldo Sep 2, 2026
bf8b938
feat(thoughtspot): expression catalog vocabulary and spec-coverage gate
djwaldo Sep 2, 2026
010a4fb
fix(thoughtspot): reconcile 137 vs 146 with an explicit divergence set
djwaldo Sep 2, 2026
dc0485e
fix(thoughtspot): correct Spelling audit, drop dead code, gate empty …
djwaldo Sep 2, 2026
eeb2e62
feat(thoughtspot): expression emitters for direct, passthrough and un…
djwaldo Sep 2, 2026
5faa1fd
fix(thoughtspot): enforce the E8 wrapper instead of relying on caller…
djwaldo Sep 2, 2026
5f84354
fix(thoughtspot): make the E8 PARTITION BY guard whitespace-tolerant
djwaldo Sep 2, 2026
36714ee
feat(thoughtspot): catalog entries for aggregate functions and type c…
djwaldo Sep 2, 2026
048eb27
feat(thoughtspot): catalog entries for date/time functions
djwaldo Sep 2, 2026
ceb2196
feat(thoughtspot): catalog entries for string functions
djwaldo Sep 2, 2026
8334177
fix(thoughtspot): reject a passthrough template that wraps itself
djwaldo Sep 2, 2026
1a3ccc1
feat(thoughtspot): catalog entries for mathematical and conditional f…
djwaldo Sep 2, 2026
8365775
feat(thoughtspot): catalog entries for operators and constructs
djwaldo Sep 2, 2026
70341f2
fix(thoughtspot): escape IN/NOT IN braces; force external dispatch fo…
djwaldo Sep 2, 2026
a8716c8
feat(thoughtspot): catalog entries for window functions
djwaldo Sep 2, 2026
d4e1a7c
feat(thoughtspot): reverse-direction inventory (ThoughtSpot -> Ossie)
djwaldo Sep 2, 2026
6a0fd1e
fix(ci): trigger thoughtspot CI when the upstream spec file changes
djwaldo Sep 2, 2026
886e340
fix(thoughtspot): custom_extensions_fragment must be keyed by column,…
djwaldo Sep 2, 2026
aec1d45
fix(thoughtspot): correct the template contract and two catalog defec…
djwaldo Sep 2, 2026
4b1355a
docs(thoughtspot): remove citations to a private, deleted workspace f…
djwaldo Sep 2, 2026
43a7804
fix(thoughtspot): remove remaining internal-process leakage; add pass…
djwaldo Sep 2, 2026
df5537d
feat(thoughtspot): shallow formula tokenizer for reference rewriting …
djwaldo Sep 2, 2026
9858b4b
fix(thoughtspot): stop quote state leaking into bracket bodies, deleg…
djwaldo Sep 2, 2026
f9566c0
fix(thoughtspot): loosen column-ref table grammar to match its own am…
djwaldo Sep 2, 2026
e3e641a
docs(thoughtspot): drop review-process wording from a test comment
djwaldo Sep 2, 2026
536e6c7
feat(thoughtspot): bidirectional datatype map with declared losses named
djwaldo Sep 2, 2026
15e3030
docs(thoughtspot): remove citations to an internal revision id
djwaldo Sep 2, 2026
4e2de03
feat(thoughtspot): TML document set with the guid/brace/ordering/YAML…
djwaldo Sep 2, 2026
72c2cbc
fix(thoughtspot): sanitise dump_document_set filenames and strip nest…
djwaldo Sep 2, 2026
34e7919
fix(thoughtspot): reserve minted filenames by their final form, and c…
djwaldo Sep 2, 2026
9096c94
test(thoughtspot): add a self-policing guard against unresolvable ref…
djwaldo Sep 2, 2026
1f44b1e
feat(thoughtspot): convert TML columns to Ossie fields with verbatim …
djwaldo Sep 2, 2026
91e99b7
fix(thoughtspot): give convert_field a formulas map instead of an inl…
djwaldo Sep 3, 2026
729744f
fix(thoughtspot): log the genuine silent losses review round 2 found
djwaldo Sep 3, 2026
98a67e3
feat(thoughtspot): convert TML measures to Ossie metrics, composing c…
djwaldo Sep 3, 2026
863919c
fix(thoughtspot): stash the source metric shape for round-trip fidelity
djwaldo Sep 3, 2026
0083904
fix(thoughtspot): stop silent double aggregation on group_aggregate a…
djwaldo Sep 3, 2026
914aef6
fix(thoughtspot): only warn when an aggregate is nested, not when it …
djwaldo Sep 3, 2026
a3dbb6a
feat(thoughtspot): complete the TML -> Ossie direction with keys, sta…
djwaldo Sep 3, 2026
084dcb3
fix(thoughtspot): stash unconsumed column properties instead of dropp…
djwaldo Sep 3, 2026
8d3c9ac
fix(thoughtspot): validate relationship targets, centralize the ident…
djwaldo Sep 3, 2026
595ad6f
fix(thoughtspot): read SQL View columns from sql_view_columns[], not …
djwaldo Sep 3, 2026
4736a63
feat(thoughtspot): build Table TML documents from Ossie datasets
djwaldo Sep 3, 2026
22a13b8
fix(thoughtspot): a bare field's portable expression names the wareho…
djwaldo Sep 3, 2026
1f4307e
fix(thoughtspot): read the stashed warehouse column name back, and th…
djwaldo Sep 3, 2026
c63dc79
refactor(thoughtspot): centralize every custom_extensions[THOUGHTSPOT…
djwaldo Sep 3, 2026
411fcab
feat(thoughtspot): build the Model TML document under the R3-R9 impor…
djwaldo Sep 3, 2026
b28f205
fix(thoughtspot): rewrite formula cross-references to match normalise…
djwaldo Sep 3, 2026
f5df4d4
fix(thoughtspot): rename FULL_OUTER to OUTER in every emitted join type
djwaldo Sep 3, 2026
4b91893
fix(thoughtspot): drop is_hidden/was_auto_generated from emitted columns
djwaldo Sep 3, 2026
efcc0a4
feat(thoughtspot): complete the Ossie -> TML direction with joins and…
djwaldo Sep 3, 2026
6662e7b
fix(thoughtspot): a formula cross-reference is not a runtime parameter
djwaldo Sep 3, 2026
f3a93c4
fix(thoughtspot): audit and enforce X5's stash-currency rule across e…
djwaldo Sep 3, 2026
79b061a
fix(thoughtspot): unsurfaced-column membership is a second, orthogona…
djwaldo Sep 3, 2026
7ec0f97
test(thoughtspot): TPC-DS and minimal fixture pairs
djwaldo Sep 3, 2026
866730d
feat(thoughtspot): CLI entry point for both directions
djwaldo Sep 3, 2026
faa630c
test(thoughtspot): round-trip both directions, asserting preservation…
djwaldo Sep 3, 2026
519ea8e
fix(thoughtspot): restore a referencing join's own name and stop dupl…
djwaldo Sep 3, 2026
281e1f0
test(thoughtspot): property-based round-trip over adversarial identif…
djwaldo Sep 3, 2026
6976448
docs(thoughtspot): README, packaging and CI for the bidirectional con…
djwaldo Sep 3, 2026
730731d
docs(thoughtspot): generate expression, reverse, datatype and payload…
djwaldo Sep 7, 2026
5dde90a
fix(thoughtspot): state the substance of internal rule citations, dro…
djwaldo Sep 7, 2026
cd3a738
fix(thoughtspot): close the last mapping-doc rule-id family (I)
djwaldo Sep 7, 2026
2ff8c07
fix(thoughtspot): swap ONE_TO_MANY relationship endpoints per spec
djwaldo Sep 7, 2026
d10db31
fix(thoughtspot): degrade non-Latin field/metric names instead of dro…
djwaldo Sep 8, 2026
d84c505
test(thoughtspot): realign the TPC-DS fixture with the reference model
djwaldo Sep 8, 2026
0803a4d
feat(thoughtspot): one semantic model per document, no wrapper
djwaldo Sep 16, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 69 additions & 0 deletions .github/workflows/converter-thoughtspot-ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
#
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
#

name: Converters ThoughtSpot CI
on:
push:
branches: [ "main" ]
paths:
- 'converters/thoughtspot/**'
- '.github/workflows/converter-thoughtspot-ci.yml'
# catalog.py's spec_construct_names() parses this file directly (it is the
# upstream-spec coverage oracle, not vendored into converters/thoughtspot/),
# so a change here must trigger this workflow even though it touches no
# converter file. No sibling workflow parses the spec, hence this addition
# is not mirrored anywhere else.
- 'core-spec/expression_language.md'
pull_request:
branches: [ "main" ]
paths:
- 'converters/thoughtspot/**'
- '.github/workflows/converter-thoughtspot-ci.yml'
- 'core-spec/expression_language.md'

jobs:
build:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.10", "3.11", "3.12", "3.13", "3.14"]

steps:
- name: Checkout project
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: ${{ matrix.python-version }}

- name: Install uv
run: |
curl -LsSf https://astral.sh/uv/install.sh | sh
echo "${HOME}/.local/bin" >> "${GITHUB_PATH}"

- name: Sync dependencies
working-directory: converters/thoughtspot
run: |
uv sync

- name: Unit Tests
working-directory: converters/thoughtspot
run: |
uv run pytest
1 change: 1 addition & 0 deletions converters/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,7 @@ The Ossie specification currently defines extensions for the following vendors:
| `WISDOM` | WisdomAI domain |
| `NVIDIA_GSF` | NVIDIA Generative Semantic Fabric standalone YAML |
| `SIGMA` | Sigma Computing data model |
| `THOUGHTSPOT` | ThoughtSpot TML (Model + Table/SQL View) |

Each vendor may define custom extensions (via the `custom_extensions` field in the Ossie spec) to carry vendor-specific metadata that does not have an equivalent in the core specification.

Expand Down
25 changes: 25 additions & 0 deletions converters/thoughtspot/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.

__pycache__/
*.py[cod]
.pytest_cache/
*.egg-info/
dist/
build/
.venv/
venv/
311 changes: 311 additions & 0 deletions converters/thoughtspot/README.md

Large diffs are not rendered by default.

76 changes: 76 additions & 0 deletions converters/thoughtspot/docs/datatype-map.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->

<!-- GENERATED FILE -- do not edit by hand.
Produced by `tools/generate_reference_docs.py` from:
- `src/ossie_thoughtspot/datatypes.py`
Regenerate with:
uv run --python 3.13 python tools/generate_reference_docs.py
tests/test_reference_docs_current.py fails the suite if this file
drifts from what the generator currently produces. -->

# Ossie <-> ThoughtSpot Datatype Map

The bidirectional Ossie <-> ThoughtSpot TML datatype map. The map is **not injective** — several Ossie types collapse onto one TML spelling and cannot be told apart on the way back; see "Not injective" below.

## The closed Ossie datatype enum

`Boolean`, `Date`, `DateTime`, `DateTimeTz`, `Decimal`, `Float`, `Integer`, `Opaque`, `String`, `Time`

## Ossie -> TML

| Ossie datatype | TML `data_type` (default) | Notes |
|---|---|---|
| `Boolean` | `BOOLEAN` | connection-dependent spelling — `BOOLEAN` by default, `BOOL` when the connection's own TML spells it that way |
| `Date` | `DATE` | exact, single spelling |
| `DateTime` | `DATE_TIME` | exact, single spelling |
| `DateTimeTz` | `DATE_TIME` | ThoughtSpot has no offset-aware column type; the value becomes DATE_TIME and returns as DateTime. |
| `Decimal` | `DOUBLE` | exact, single spelling |
| `Float` | `DOUBLE` | connection-dependent spelling — `DOUBLE` by default, `FLOAT` when the connection's own TML spells it that way; ThoughtSpot has one approximate numeric type, so Float and Decimal both become DOUBLE and return as Decimal. |
| `Integer` | `INT64` | exact, single spelling |
| `Opaque` | `VARCHAR` | Opaque is Ossie's marker for a type outside the portable vocabulary; it becomes VARCHAR and returns as String. |
| `String` | `VARCHAR` | exact, single spelling |
| `Time` | `VARCHAR` | ThoughtSpot has no time-of-day column type; the value becomes VARCHAR. |

A column with no declared `datatype` at all infers `INT64` rather than raising — `datatype` is optional in Ossie, but TML rejects a column with no `db_column_properties` block at all.

## TML -> Ossie

| TML `data_type` | Ossie datatype |
|---|---|
| `BOOL` | `Boolean` |
| `BOOLEAN` | `Boolean` |
| `DATE` | `Date` |
| `DATE_TIME` | `DateTime` |
| `DOUBLE` | `Decimal` |
| `FLOAT` | `Float` |
| `INT64` | `Integer` |
| `VARCHAR` | `String` |

A TML `data_type` outside this map returns no Ossie datatype at all — `datatype` is optional in Ossie, so omitting it is preferred over inventing one.

## Not injective — declared losses

| Ossie datatype | Why the round trip is lossy |
|---|---|
| `DateTimeTz` | ThoughtSpot has no offset-aware column type; the value becomes DATE_TIME and returns as DateTime. |
| `Float` | ThoughtSpot has one approximate numeric type, so Float and Decimal both become DOUBLE and return as Decimal. |
| `Opaque` | Opaque is Ossie's marker for a type outside the portable vocabulary; it becomes VARCHAR and returns as String. |
| `Time` | ThoughtSpot has no time-of-day column type; the value becomes VARCHAR. |

Loading