Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
71 changes: 65 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ Yet another `gcloud spanner databases execute-sql` replacement for better compos
* Configurable gRPC logging (`off`, `metadata`, `payload` with payload caveat)
* (Experimental) CSV output
* Split query-plan and row output (`--plan-output`)
* In-process query plan rendering (`--plan-format=text|dot|mermaid|d2|svg|png`)
* (Experimental) Check whether the query can be executed as a partition query or not.

This tool is still pre-release quality and none of guarantees.
Expand Down Expand Up @@ -49,9 +50,10 @@ Flags:
--plan-output=STRING Write the query-plan artifact here and strip
stats.queryPlan from the primary document.
Enables split mode.
--plan-format=STRING Format of the plan artifact: json or yaml.
Defaults to --format when that is json or
yaml, otherwise json. Requires --plan-output.
--plan-format=STRING Format of the plan artifact: json, yaml,
text, dot, mermaid, d2, svg, or png. Defaults
to --format when that is json or yaml,
otherwise json. Requires --plan-output.
--discard-results Do not write the primary document
(plan-only). Requires --plan-output.
--redact-rows Redact result rows from output
Expand Down Expand Up @@ -91,6 +93,21 @@ Flags:
--try-partition-query (Experimental) Check whether the query can be
executed as partition query or not

Plan rendering
--plan-text-style=STRING Text plan style: current, traditional, or compact.
Defaults to current. Requires --plan-format=text.
--plan-wrap-width=INT Wrap width for text plans. 0 disables wrapping.
Requires --plan-format=text.
--plan-print=STRING Text plan sections: basic, enhanced, full, none,
or a comma-separated section list. Defaults to
basic. Requires --plan-format=text.
--plan-full Include full graph node detail. Requires a graph
--plan-format (dot, mermaid, d2, svg, png).
--plan-show-query Add a query-text node to graph output. Requires a
graph --plan-format.
--plan-show-query-stats Add query statistics to the query-text node.
Requires a graph --plan-format.

Timestamp Bound
--strong Perform a strong query.
--read-timestamp=STRING Perform a query at the given timestamp.
Expand Down Expand Up @@ -149,7 +166,7 @@ Setting `--plan-output` switches from the default combined document to split mod
|------|---------|---------|
| `--output PATH` (`-o`) | `-` | Destination of the primary document (metadata, rows, `stats` without `queryPlan`). |
| `--plan-output PATH` | unset | Enables split mode: write the plan artifact here and remove `stats.queryPlan` from the primary document. |
| `--plan-format json\|yaml` | follows `--format` when that is `json` or `yaml`, otherwise `json` | Format of the plan artifact. |
| `--plan-format json\|yaml\|text\|dot\|mermaid\|d2\|svg\|png` | follows `--format` when that is `json` or `yaml`, otherwise `json` | Format of the plan artifact. `json`/`yaml` write a `ResultSet` envelope; the others render in-process. |
| `--discard-results` | off | Do not write the primary document (plan-only). Requires `--plan-output`. |

`--redact-rows` is independent of `--discard-results`: redact still emits metadata and a CSV header; discard writes no primary bytes at all.
Expand All @@ -165,7 +182,7 @@ In split mode the two destinations must differ. Both on stdout (any spelling) is
Document contents:

- Primary document: the current `ResultSet` with `stats.queryPlan` removed. `stats.queryStats` and `stats.rowCount*` stay. CSV primary output is unchanged (rows only).
- Plan artifact: a `ResultSet` envelope without `rows` — `metadata` (for `rowType`) plus the full `stats` (`queryPlan`, `queryStats`, `rowCount*`). jq flags apply only to the primary document; the plan is never filtered.
- Plan artifact: `json`/`yaml` write a `ResultSet` envelope without `rows` — `metadata` (for `rowType`) plus the full `stats` (`queryPlan`, `queryStats`, `rowCount*`). Renderer formats write the rendered bytes instead. jq flags apply only to the primary document; the plan is never filtered.

Split mode disables jq early stop: remaining rows are drained so the final plan/stats can be captured, at the same server cost as reading everything. Rows drained only for the plan are not retained. `--jq-input-mode=lazy` still caches rows that jq actually consumed.

Expand All @@ -190,6 +207,37 @@ $ execspansql ${DATABASE_ID} --query-mode=PROFILE --discard-results \
--plan-output=plan.json --sql='SELECT * FROM Singers'
```

Renderer formats write through the same `--plan-output` sink. `--plan-format=text` is the built-in equivalent of piping `.stats.queryPlan` into `rendertree`. Graph formats (`dot`, `mermaid`, `d2`, `svg`, `png`) use embedded `spannerplanviz`; `svg`/`png` do not need an external Graphviz install.

| Flag | Applies to | Default |
|------|------------|---------|
| `--plan-text-style current\|traditional\|compact` | `text` | `current` |
| `--plan-wrap-width N` | `text` | `0` (off; never inferred from terminal width) |
| `--plan-print basic\|enhanced\|full\|none\|<sections>` | `text` | `basic` |
| `--plan-full` | graph formats | off |
| `--plan-show-query` | graph formats | off |
| `--plan-show-query-stats` | graph formats | off |

`--redact-rows` does not redact plans: predicates and metadata can still contain literals. `png` to a terminal (`--plan-output=-` or `/dev/stderr` when that fd is a TTY) is rejected; redirect or write a file instead. Renderer-only flags that do not apply to the chosen `--plan-format` are errors, not silent no-ops.

```
$ execspansql ${DATABASE_ID} --query-mode=PROFILE --format=experimental_csv \
--output=- --plan-output=/dev/stderr --plan-format=text \
--sql='SELECT * FROM Singers'
```

```
$ execspansql ${DATABASE_ID} --query-mode=PROFILE --redact-rows \
--discard-results --plan-output=- --plan-format=text \
--sql='SELECT * FROM Singers'
```

```
$ execspansql ${DATABASE_ID} --query-mode=PROFILE --discard-results \
--plan-output=plan.svg --plan-format=svg --plan-full \
--sql='SELECT * FROM Singers'
```

### Parameter support

Many Cloud Spanner clients don't support parameter.
Expand Down Expand Up @@ -265,7 +313,15 @@ Ctrl+C cancels both SQL execution and jq processing, including jq computations a

#### Example: Extract QueryPlan

[rendertree] command takes QueryPlan, and it can be extracted by jq filter.
`--plan-output` with `--plan-format=text` renders the plan without a second binary. The jq + [rendertree] pipeline remains available for the combined document.

```
$ execspansql ${DATABASE_ID} --query-mode=PROFILE \
--sql='SELECT * FROM Singers@{FORCE_INDEX=SingersByFirstLastName}' \
--discard-results --plan-output=- --plan-format=text
```

[rendertree] can still consume `.stats.queryPlan` from the combined JSON document:

```
$ execspansql ${DATABASE_ID} --query-mode=PROFILE --format=json \
Expand Down Expand Up @@ -457,3 +513,6 @@ exit status 1
* Non-`NORMAL` query modes (`PLAN`, `PROFILE`, `WITH_PLAN_AND_STATS`, and `WITH_STATS`) cannot be combined with `--enable-partitioned-dml`. The Partitioned DML client path ignores query mode and would execute writes.
* `--plan-output` requires a plan-producing query mode and cannot be combined with `--try-partition-query` or `--enable-partitioned-dml`.
* Split mode disables jq early stop so the plan artifact can be captured after the last `PartialResultSet`.
* `--plan-format=png` cannot write to a terminal; use a file or a redirected stdout/stderr.
* `--redact-rows` does not redact query plans.
* The Spanner emulator often omits `planNodes` from PLAN/PROFILE results; `--plan-output` then publishes the primary document and exits non-zero without a plan file.
3 changes: 2 additions & 1 deletion execution_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ type executionServer struct {
commits atomic.Int32
retry bool
failCommit bool
queryPlan *sppb.QueryPlan
}

func (s *executionServer) BeginTransaction(context.Context, *sppb.BeginTransactionRequest) (*sppb.Transaction, error) {
Expand All @@ -39,7 +40,7 @@ func (s *executionServer) ExecuteStreamingSql(_ *sppb.ExecuteSqlRequest, stream
RowType: &sppb.StructType{Fields: []*sppb.StructType_Field{{Name: "value", Type: &sppb.Type{Code: sppb.TypeCode_STRING}}}},
},
Values: []*structpb.Value{structpb.NewStringValue(fmt.Sprintf("attempt-%d", attempt))},
Stats: &sppb.ResultSetStats{RowCount: &sppb.ResultSetStats_RowCountExact{RowCountExact: 1}},
Stats: &sppb.ResultSetStats{QueryPlan: s.queryPlan, RowCount: &sppb.ResultSetStats_RowCountExact{RowCountExact: 1}},
})
}

Expand Down
54 changes: 52 additions & 2 deletions flags_validation_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -423,11 +423,61 @@ func TestValidateExecutionOptions(t *testing.T) {
mode: partitionedDML{},
err: "--plan-output cannot be combined with --enable-partitioned-dml",
},
{
name: "plan_format_allows_text",
o: opts{PlanOutput: "plan.txt", PlanFormat: "text", QueryMode: "PROFILE"},
mode: single{spanner.StrongRead()},
},
{
name: "plan_format_allows_svg",
o: opts{PlanOutput: "plan.svg", PlanFormat: "svg", QueryMode: "PROFILE"},
mode: single{spanner.StrongRead()},
},
{
name: "plan_format_rejects_unknown",
o: opts{PlanOutput: "plan.json", PlanFormat: "text", QueryMode: "PROFILE"},
o: opts{PlanOutput: "plan.json", PlanFormat: "html", QueryMode: "PROFILE"},
mode: single{spanner.StrongRead()},
err: "--plan-format must be json, yaml, text, dot, mermaid, d2, svg, or png",
},
{
name: "plan_text_style_requires_plan_output",
o: opts{PlanTextStyle: "compact", QueryMode: "PROFILE"},
mode: single{spanner.StrongRead()},
err: "--plan-text-style requires --plan-output",
},
{
name: "plan_text_style_rejects_json",
o: opts{PlanOutput: "plan.json", PlanFormat: "json", PlanTextStyle: "compact", QueryMode: "PROFILE"},
mode: single{spanner.StrongRead()},
err: "--plan-text-style cannot be used with --plan-format=json",
},
{
name: "plan_full_rejects_text",
o: opts{PlanOutput: "plan.txt", PlanFormat: "text", PlanFull: true, QueryMode: "PROFILE"},
mode: single{spanner.StrongRead()},
err: "--plan-full cannot be used with --plan-format=text",
},
{
name: "plan_show_query_rejects_yaml",
o: opts{PlanOutput: "plan.yaml", PlanFormat: "yaml", PlanShowQuery: true, QueryMode: "PROFILE"},
mode: single{spanner.StrongRead()},
err: "--plan-show-query cannot be used with --plan-format=yaml",
},
{
name: "plan_wrap_width_rejects_dot",
o: opts{PlanOutput: "plan.dot", PlanFormat: "dot", PlanWrapWidth: 80, QueryMode: "PROFILE"},
mode: single{spanner.StrongRead()},
err: "--plan-wrap-width cannot be used with --plan-format=dot",
},
{
name: "plan_text_style_allows_text",
o: opts{PlanOutput: "plan.txt", PlanFormat: "text", PlanTextStyle: "compact", PlanPrint: "enhanced", QueryMode: "PROFILE"},
mode: single{spanner.StrongRead()},
},
{
name: "plan_full_allows_mermaid",
o: opts{PlanOutput: "plan.mmd", PlanFormat: "mermaid", PlanFull: true, PlanShowQuery: true, QueryMode: "PROFILE", Sql: "SELECT 1"},
mode: single{spanner.StrongRead()},
err: "--plan-format must be json or yaml",
},
}

Expand Down
13 changes: 10 additions & 3 deletions format.go
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ func (c *preparedCommand) writeResult(ctx context.Context, result *queryResult,
if err != nil {
return err
}
return writePlan(sinks.plan, effectivePlanFormat(c.opts), result.Metadata, stats)
return c.writePlan(ctx, sinks, result.Metadata, stats)
}

enc, err := newEncoder(sinks.primary, c.Format, c.CompactOutput, c.JqRawOutput)
Expand Down Expand Up @@ -62,7 +62,7 @@ func (c *preparedCommand) writeResult(ctx context.Context, result *queryResult,
if err != nil {
return err
}
return writePlan(sinks.plan, effectivePlanFormat(c.opts), drained.Metadata, stats)
return c.writePlan(ctx, sinks, drained.Metadata, stats)
}

func (c *preparedCommand) writeResultSet(ctx context.Context, rs *sppb.ResultSet, sinks *outputSinks) error {
Expand Down Expand Up @@ -90,7 +90,7 @@ func (c *preparedCommand) writeResultSet(ctx context.Context, rs *sppb.ResultSet
}
}
sinks.MarkPrimaryComplete()
return writePlan(sinks.plan, effectivePlanFormat(c.opts), metadata, stats)
return c.writePlan(ctx, sinks, metadata, stats)
}

// printJQ owns encoder completion on both success and failure. Use the caller's
Expand All @@ -103,3 +103,10 @@ func printJQ(ctx context.Context, code *gojq.Code, input any, enc encoder) (err
}()
return jqresult.Print(enc, code.RunWithContext(ctx, input))
}

// Use the prepared statement for query text so rendering never reloads SQL files.
func (c *preparedCommand) writePlan(ctx context.Context, sinks *outputSinks, metadata *sppb.ResultSetMetadata, stats *sppb.ResultSetStats) error {
o := c.opts
o.Sql = c.statement.SQL
return writePlan(ctx, sinks.plan, effectivePlanFormat(o), metadata, stats, o)
}
5 changes: 5 additions & 0 deletions internal/planrender/render.go
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,11 @@ func Render(ctx context.Context, w io.Writer, format Format, rowType *sppb.Struc
return renderGraph(ctx, w, format, rowType, stats, opts)
}

// Validate reports whether opts apply to format.
func (o Options) Validate(format Format) error {
return o.validate(format)
}

func (o Options) validate(format Format) error {
if o.WrapWidth < 0 {
return fmt.Errorf("WrapWidth cannot be negative: %d", o.WrapWidth)
Expand Down
8 changes: 7 additions & 1 deletion main.go
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,13 @@ type opts struct {
Format string `name:"format" enum:"json,yaml,experimental_csv" default:"json" help:"Output format of the primary document."`
Output string `name:"output" short:"o" default:"-" help:"Destination of the primary document. Use - for stdout; /dev/stdout and /dev/stderr are mapped in-process."`
PlanOutput string `name:"plan-output" help:"Write the query-plan artifact here and strip stats.queryPlan from the primary document. Enables split mode."`
PlanFormat string `name:"plan-format" help:"Format of the plan artifact: json or yaml. Defaults to --format when that is json or yaml, otherwise json. Requires --plan-output."`
PlanFormat string `name:"plan-format" help:"Format of the plan artifact: json, yaml, text, dot, mermaid, d2, svg, or png. Defaults to --format when that is json or yaml, otherwise json. Requires --plan-output."`
PlanTextStyle string `name:"plan-text-style" help:"Text plan style: current, traditional, or compact. Defaults to current. Requires --plan-format=text." group:"Plan rendering"`
PlanWrapWidth int `name:"plan-wrap-width" help:"Wrap width for text plans. 0 disables wrapping. Requires --plan-format=text." group:"Plan rendering"`
PlanPrint string `name:"plan-print" help:"Text plan sections: basic, enhanced, full, none, or a comma-separated section list. Defaults to basic. Requires --plan-format=text." group:"Plan rendering"`
PlanFull bool `name:"plan-full" help:"Include full graph node detail. Requires a graph --plan-format (dot, mermaid, d2, svg, png)." group:"Plan rendering"`
PlanShowQuery bool `name:"plan-show-query" help:"Add a query-text node to graph output. Requires a graph --plan-format." group:"Plan rendering"`
PlanShowQueryStats bool `name:"plan-show-query-stats" help:"Add query statistics to the query-text node. Requires a graph --plan-format." group:"Plan rendering"`
DiscardResults bool `name:"discard-results" help:"Do not write the primary document (plan-only). Requires --plan-output."`
RedactRows bool `name:"redact-rows" help:"Redact result rows from output"`
CompactOutput bool `name:"compact-output" short:"c" help:"Compact JSON output (--compact-output of jq)"`
Expand Down
Loading
Loading