Hi,
I'm building an offline KQL validator using Kusto.Language and hit an issue with evaluate output schemas that use *.
The issue: * in the output schema should preserve source columns (per the docs), but the parser drops them.
Reproduction
var db = new DatabaseSymbol("testdb",
new TableSymbol("T", "(Id: string, Data: dynamic)"));
var cluster = new ClusterSymbol("test", db);
var globals = GlobalState.Default.AddOrReplaceCluster(cluster).WithCluster(cluster).WithDatabase(db);
var code = KustoCode.ParseAndAnalyze(
"T | evaluate bag_unpack(Data) : (*, x: long, y: long)", globals);
var result = code.ResultType as TableSymbol;
// Actual: [x, y] — Id is missing
// Expected: [Id, x, y] — Id preserved by *
Runtime comparison
datatable(Id:string, Data:dynamic) ["a", dynamic({"x": 1, "y": 2})]
| evaluate bag_unpack(Data) : (*, x: long, y: long)
| getschema
Runtime returns Id:string, x:long, y:long — all 3 columns present.
Without the schema hint, bag_unpack correctly returns [Id] [open]. But adding : (*, x, y) loses Id.
Root cause
VisitEvaluateOperator calls
CreateColumnsFromRowSchema on node.Schema.Schema.Columns, but never checks node.Schema.Schema.AsteriskToken — which the
grammar already parses and stores. Looks like the asterisk handling was missed when EvaluateRowSchema was introduced in
2f35577f.
Proposed fix
Insert before the existing CreateColumnsFromRowSchema call in VisitEvaluateOperator:
if (node.Schema.Schema.AsteriskToken != null && node.Schema.Schema.AsteriskToken.Width > 0)
{
var pluginResult = GetResultTypeOrError(node.FunctionCall) as TableSymbol;
if (pluginResult != null)
{
var explicitNames = new HashSet<string>();
for (int i = 0; i < node.Schema.Schema.Columns.Count; i++)
explicitNames.Add(node.Schema.Schema.Columns[i].Element.Name.SimpleName);
foreach (var col in pluginResult.Columns)
{
if (!explicitNames.Contains(col.Name))
columns.Add(col);
}
}
}
CreateColumnsFromRowSchema(node.Schema.Schema.Columns, columns);
Tested locally with 24 tests (bag_unpack, pivot, chained plugins, column conflicts, non-evaluate regressions).
Note: investigation and fix development were done with AI assistance (GitHub Copilot).
Hi,
I'm building an offline KQL validator using
Kusto.Languageand hit an issue withevaluateoutput schemas that use*.The issue:
*in the output schema should preserve source columns (per the docs), but the parser drops them.Reproduction
Runtime comparison
Runtime returns
Id:string, x:long, y:long— all 3 columns present.Without the schema hint,
bag_unpackcorrectly returns[Id] [open]. But adding: (*, x, y)losesId.Root cause
VisitEvaluateOperatorcallsCreateColumnsFromRowSchemaonnode.Schema.Schema.Columns, but never checksnode.Schema.Schema.AsteriskToken— which thegrammar already parses and stores. Looks like the asterisk handling was missed when
EvaluateRowSchemawas introduced in2f35577f.Proposed fix
Insert before the existing
CreateColumnsFromRowSchemacall inVisitEvaluateOperator:Tested locally with 24 tests (bag_unpack, pivot, chained plugins, column conflicts, non-evaluate regressions).
Note: investigation and fix development were done with AI assistance (GitHub Copilot).