[Feature Request]: Support non-default Google Cloud universe domains (GOOGLE_CLOUD_UNIVERSE_DOMAIN) #40569

What would you like to happen?

Beam should work in Google Cloud universes other than the default googleapis.com (e.g. Trusted Partner Cloud / sovereign clouds), where the client libraries pick the universe from GOOGLE_CLOUD_UNIVERSE_DOMAIN.

Today several places hard-code GDU URLs or pass them as explicit endpoint overrides. An explicit endpoint override takes precedence over the universe domain, so the configured universe is ignored.

Internal bug: 290704327

Issue Priority

Priority: 2 (default / most feature requests should be filed as P2)

Issue Components

• Component: Python SDK
• Component: Java SDK
• Component: Go SDK
• Component: Typescript SDK
• Component: IO connector
• Component: Beam YAML
• Component: Beam examples
• Component: Beam playground
• Component: Beam katas
• Component: Website
• Component: Infrastructure
• Component: Spark Runner
• Component: Flink Runner
• Component: Prism Runner
• Component: Twister2 Runner
• Component: Hazelcast Jet Runner
• Component: Google Cloud Dataflow Runner