Repository navigation
bigquery dataset should not always take project argument from client #2118
Description
The project in which a dataset resides is not necessarily the project in which jobs involving the dataset will be executed. For example, if querying the bigquery public data hacker news dataset the following will error
client = bigquery.Client(project='bigquery-public-data')
dataset = client.dataset('hacker_news')
client.extract_table_storage(..., dataset.table('comments'))
Because the user can't create a job in the project 'bigquery-public-data'. Instead the fairly convoluted:
client = bigquery.Client(project='my-project')
client.extract_table_storage(..., bigquery.Client(project='bigquery-public-data').dataset('hacker_news').table('comments'))
Despite the fact that the client involved will never be used.
The proposed fix is to add a kwarg project to the Dataset class which overrides the originating project.
Reacted by Tim Sweña (Swast)
Activity
tseaver commentedon Aug 18, 2016
@elibixby Thanks for the report!
tswast commentedon Aug 26, 2016
+1 to this. I filed a similar issue in gcloud-java googleapis/google-cloud-java#1123 which does provide a project as part of the dataset identifier (but it doesn't always use it, yikes!)
Allow overriding dataset's project during construction.
refactor: support agg_ops.RowNumberOp for sqlglot compiler (#2118)