[Bug]: Cloud_Spanner_to_GCS_Avro template ignores 'tableNames', always exports all tables

[Ziemowit-LH]

Related Template(s)

Cloud_Spanner_to_GCS_Avro

Template Version

latest

What happened?

Template ignores tableNames parameter.

gcloud dataflow jobs run spanner-export-job \
  --gcs-location=gs://dataflow-templates/latest/Cloud_Spanner_to_GCS_Avro \
  --service-account-email=tmp-df-spanner-import@${PROJECT}.iam.gserviceaccount.com \
  --project=${PROJECT} \
  --region=${REGION} \
  --network=${NETWORK} \
  --subnetwork=${SUBNETWORK} \
  --disable-public-ips \
  --parameters="^|^instanceId=${INSTANCE_ID}|databaseId=${DATABASE_ID}|tableNames=table1,table2|shouldExportRelatedTables=false|outputDir=${EXPORT_LOCATION}"

Export always contains all tables.
Even if tableNames contains a single, independent table.

Export fails (as expected) when nonexistent table is defined.

Resulting export manifest:

{
  "tables": [{
    "name": "mutex",
    "manifestFile": "mutex-manifest.json"
  }, {
    "name": "ObserverVersion",
    "manifestFile": "ObserverVersion-manifest.json"
  }, {
    "name": "MigrationsHistory",
    "manifestFile": "MigrationsHistory-manifest.json"
  }, {
    "name": "table1",
    "manifestFile": "table1-manifest.json"
  }, {
    "name": "table2",
    "manifestFile": "table2-manifest.json"
  }],
  "databaseOptions": [{
    "optionName": "version_retention_period",
    "optionType": "STRING",
    "optionValue": "1h"
  }]
}

Relevant log output

[dheerusri324]

Hi @Ziemowit-LH , I'd like to work on fixing this and submit a PR. Could you assign this issue to me?

[Ziemowit-LH]

Hello @dheerusri324,

I think you are asking the wrong person. I am not a member of this project. Just a user who reported the issue.

[dheerusri324]

Hello @dheerusri324,

I think you are asking the wrong person. I am not a member of this project. Just a user who reported the issue.

Oh ok, sorry 😅

So whom can I ask then?

[chamikaramj]

#4008 tries to fix this.