[Bug]: Beam roles infra - Unable to run the Dataflow TPU example

[yalah5084]

What happened?

I got beam_viewer and beam_writer permissions, but I am still unable to run the Dataflow TPU example. This is the error I get:

ERROR: (gcloud.services.enable) [[yalahuangfeng@gmail.com](mailto:yalahuangfeng@gmail.com)] does not have permission to access projects instance [apache-beam-testing] (or it may not exist): Permission denied to enable service [artifactregistry.googleapis.com]
Permission denied to enable service [cloudbuild.googleapis.com]
Permission denied to enable service [cloudresourcemanager.googleapis.com]
Permission denied to enable service [compute.googleapis.com]
Permission denied to enable service [dataflow.googleapis.com]
Permission denied to enable service [logging.googleapis.com]
Permission denied to enable service [storage.googleapis.com]
Help Token: AcxmRmJZDkJPrcUjccPtrihLnmxGTwrqWZE4AnvDBwMpnRisI1OE7NmQMKFehzED1RaCkUaMcq5HqVoqcv2H0B7KsWfI88_Gg-puAQY-mkgT3TDz. This command is authenticated as [yalahuangfeng@gmail.com](mailto:yalahuangfeng@gmail.com) which is the active account specified by the [core/account] property
- '@type': type.googleapis.com/google.rpc.PreconditionFailure
  violations:
  - subject: '110002'
    type: googleapis.com
- '@type': type.googleapis.com/google.rpc.ErrorInfo
  domain: serviceusage.googleapis.com
  reason: AUTH_PERMISSION_DENIED

Issue Priority

Priority: 2 (default / most bugs should be filed as P2)

Issue Components

• Component: Python SDK
• Component: Java SDK
• Component: Go SDK
• Component: Typescript SDK
• Component: IO connector
• Component: Beam YAML
• Component: Beam examples
• Component: Beam playground
• Component: Beam katas
• Component: Website
• Component: Infrastructure
• Component: Spark Runner
• Component: Flink Runner
• Component: Samza Runner
• Component: Twister2 Runner
• Component: Hazelcast Jet Runner
• Component: Google Cloud Dataflow Runner

[pabloem]

fyi @ksobrenat32

[pabloem]

working with @yalah5084 to figure this out.

The error above was from trying to enable services, which is not necessary - however we hit an error further down as well.

[ksobrenat32]

@yalah5084 Are you running the example over examples/notebooks/beam-ml/dataflow_tpu_examples.ipynb?

You can actually run something like the following to notice all of the services are already enabled fo apache-beam-testing :D

gcloud services list --enabled --project apache-beam-testing \
    --filter="name:(artifactregistry.googleapis.com OR cloudbuild.googleapis.com OR cloudresourcemanager.googleapis.com OR compute.googleapis.com OR dataflow.googleapis.com OR logging.googleapis.com OR storage.googleapis.com)"

So you can try to run the rest of the example, ignoring the enable part :)

[ksobrenat32]

working with @yalah5084 to figure this out.

The error above was from trying to enable services, which is not necessary - however we hit an error further down as well.

@pabloem If they need to have the permissions of enabling apis and services maybe it is a good idea to give them a higher role? like the beam_infra_manager role?

After a little bit of investigation I think the role for this is roles/serviceusage.serviceUsageAdmin, it can be added to the beam_infra_manager role if thats the case

Or what is the error further down? Maybe I could help with that?