Class Client (3.18.0)

Project ID for the project which the client acts on behalf of. Will be passed when creating a dataset / job. If not passed, falls back to the default inferred from the environment.

The OAuth2 Credentials to use for this client. If not passed (and if no _http object is passed), falls back to the default inferred from the environment.

HTTP object to make requests. Can be any object that defines request() with the same interface as requests.Session.request. If not passed, an _http object is created that is bound to the credentials for the current object. This parameter should be considered private, and could change in the future.

Default location for jobs / datasets / tables.

Default QueryJobConfig. Will be merged into job configs passed into the query method.

Default LoadJobConfig. Will be merged into job configs passed into the load_table_* methods.

The client info used to send a user-agent string along with API requests. If None, then default info will be used. Generally, you only need to set this if you're developing your own library or partner tool.

Client options used to set user options on the client. API Endpoint should be set through client_options.

Job identifier.

ID of the project which owns the job (defaults to the client's project).

Location where the job was run. Ignored if job_id is a job object.

How to retry the RPC.

The number of seconds to wait for the underlying HTTP transport before using retry.

Table or tables to be copied.

Table into which data is to be copied.

The ID of the job.

The user-provided prefix for a randomly generated job ID. This parameter will be ignored if a job_id is also given.

Location where to run the job. Must match the location of any source table as well as the destination table.

Project ID of the project of where to run the job. Defaults to the client's project.

Extra configuration options for the job.

How to retry the RPC.

The number of seconds to wait for the underlying HTTP transport before using retry.

A Dataset to create. If dataset is a reference, an empty dataset is created with the specified ID and client's default location.

Defaults to False. If True, ignore "already exists" errors when creating the dataset.

How to retry the RPC.

The number of seconds to wait for the underlying HTTP transport before using retry.

configuration job representation returned from the API.

How to retry the RPC.

The number of seconds to wait for the underlying HTTP transport before using retry.

A Routine to create. The dataset that the routine belongs to must already exist.

Defaults to False. If True, ignore "already exists" errors when creating the routine.

How to retry the RPC.

The number of seconds to wait for the underlying HTTP transport before using retry.

A Table to create. If table is a reference, an empty table is created with the specified ID. The dataset that the table belongs to must already exist.

Defaults to False. If True, ignore "already exists" errors when creating the table.

How to retry the RPC.

The number of seconds to wait for the underlying HTTP transport before using retry.

ID of the dataset.

Project ID for the dataset (defaults to the project of the client).

A reference to the dataset to delete. If a string is passed in, this method attempts to create a dataset reference from a string using from_string.

If True, delete all the tables in the dataset. If False and the dataset contains tables, the request will fail. Default is False.

How to retry the RPC.

The number of seconds to wait for the underlying HTTP transport before using retry.

Defaults to False. If True, ignore "not found" errors when deleting the dataset.

Job or job identifier.

ID of the project which owns the job (defaults to the client's project).

Location where the job was run. Ignored if job_id is a job object.

How to retry the RPC.

The number of seconds to wait for the underlying HTTP transport before using retry.

Defaults to False. If True, ignore "not found" errors when deleting the job.

A reference to the model to delete. If a string is passed in, this method attempts to create a model reference from a string using from_string.

How to retry the RPC.

The number of seconds to wait for the underlying HTTP transport before using retry.

Defaults to False. If True, ignore "not found" errors when deleting the model.

A reference to the routine to delete. If a string is passed in, this method attempts to create a routine reference from a string using from_string.

How to retry the RPC.

The number of seconds to wait for the underlying HTTP transport before using retry.

Defaults to False. If True, ignore "not found" errors when deleting the routine.

A reference to the table to delete. If a string is passed in, this method attempts to create a table reference from a string using from_string.

How to retry the RPC.

The number of seconds to wait for the underlying HTTP transport before using retry.

Defaults to False. If True, ignore "not found" errors when deleting the table.

Table or Model to be extracted.

URIs of Cloud Storage file(s) into which table data is to be extracted; in format gs://<bucket_name>/<object_name_or_glob>.

The ID of the job.

The user-provided prefix for a randomly generated job ID. This parameter will be ignored if a job_id is also given.

Location where to run the job. Must match the location of the source table.

Project ID of the project of where to run the job. Defaults to the client's project.

Extra configuration options for the job.

How to retry the RPC.

The number of seconds to wait for the underlying HTTP transport before using retry.

Type of source to be extracted.Table or Model. Defaults to Table.

Remaining positional arguments to pass to constructor.

The JSON object with a private key and other credentials information (downloaded from the Google APIs console).

Remaining positional arguments to pass to constructor.

The path to a private key file (this file was given to you when you created the service account). This file must contain a JSON object with a private key and other credentials information (downloaded from the Google APIs console).

A reference to the dataset to fetch from the BigQuery API. If a string is passed in, this method attempts to create a dataset reference from a string using from_string.

How to retry the RPC.

The number of seconds to wait for the underlying HTTP transport before using retry.

Job identifier.

ID of the project which owns the job (defaults to the client's project).

Location where the job was run. Ignored if job_id is a job object.

How to retry the RPC.

The number of seconds to wait for the underlying HTTP transport before using retry.

A reference to the model to fetch from the BigQuery API. If a string is passed in, this method attempts to create a model reference from a string using from_string.

How to retry the RPC.

The number of seconds to wait for the underlying HTTP transport before using retry.

A reference to the routine to fetch from the BigQuery API. If a string is passed in, this method attempts to create a reference from a string using from_string.

How to retry the API call.

The number of seconds to wait for the underlying HTTP transport before using retry.

Project ID to use for retreiving service account email. Defaults to the client's project.

How to retry the RPC.

The number of seconds to wait for the underlying HTTP transport before using retry.

A reference to the table to fetch from the BigQuery API. If a string is passed in, this method attempts to create a table reference from a string using from_string.

How to retry the RPC.

The number of seconds to wait for the underlying HTTP transport before using retry.

The destination table for the row data, or a reference to it.

Row data to be inserted. If a list of tuples is given, each tuple should contain data for each schema field on the current table and in the same order as the schema fields. If a list of dictionaries is given, the keys must include all required fields in the schema. Keys which do not correspond to a field in the schema are ignored.

The fields to return. Required if table is a TableReference.

Keyword arguments to insert_rows_json.

The destination table for the row data, or a reference to it.

The fields to return. Required if table is a TableReference.

The number of rows to stream in a single chunk. Must be positive.

Keyword arguments to insert_rows_json.

A pandas.DataFrame containing the data to load. Any NaN values present in the dataframe are omitted from the streaming API request(s).

The destination table for the row data, or a reference to it.

Row data to be inserted. Keys must match the table schema fields and values must be JSON-compatible representations.

Unique IDs, one per row being inserted. An ID can also be None, indicating that an explicit insert ID should not be used for that row. If the argument is omitted altogether, unique IDs are created automatically. .. versionchanged:: 2.21.0 Can also be an iterable, not just a sequence, or an AutoRowIDs enum member. .. deprecated:: 2.21.0 Passing None to explicitly request autogenerating insert IDs is deprecated, use AutoRowIDs.GENERATE_UUID instead.

Insert all valid rows of a request, even if invalid rows exist. The default value is False, which causes the entire request to fail if any invalid rows exist.

Accept rows that contain values that do not match the schema. The unknown values are ignored. Default is False, which treats unknown values as errors.

Treat name as a template table and provide a suffix. BigQuery will create the table based on the schema of the template table. See https://cloud.google.com/bigquery/streaming-data-into-bigquery#template-tables

How to retry the RPC.

The number of seconds to wait for the underlying HTTP transport before using retry.

one job resource from API response

Project ID to use for retreiving datasets. Defaults to the client's project.

True if results include hidden datasets. Defaults to False.

An expression for filtering the results by label. For syntax, see https://cloud.google.com/bigquery/docs/reference/rest/v2/datasets/list#body.QUERY_PARAMETERS.filter

Maximum number of datasets to return.

Token representing a cursor into the datasets. If not passed, the API will return the first page of datasets. The token marks the beginning of the iterator to be returned and the value of the page_token can be accessed at next_page_token of the google.api_core.page_iterator.HTTPIterator.

How to retry the RPC.

The number of seconds to wait for the underlying HTTP transport before using retry.

Maximum number of datasets to return per page.

Project ID to use for retreiving datasets. Defaults to the client's project.

If set, retrieve only child jobs of the specified parent.

Maximum number of jobs to return.

Opaque marker for the next "page" of jobs. If not passed, the API will return the first page of jobs. The token marks the beginning of the iterator to be returned and the value of the page_token can be accessed at next_page_token of google.api_core.page_iterator.HTTPIterator.

If true, include jobs owned by all users in the project. Defaults to :data:False.

If set, include only jobs matching the given state. One of: * "done" * "pending" * "running"

How to retry the RPC.

The number of seconds to wait for the underlying HTTP transport before using retry.

Min value for job creation time. If set, only jobs created after or at this timestamp are returned. If the datetime has no time zone assumes UTC time.

Max value for job creation time. If set, only jobs created before or at this timestamp are returned. If the datetime has no time zone assumes UTC time.

Maximum number of jobs to return per page.

A reference to the dataset whose models to list from the BigQuery API. If a string is passed in, this method attempts to create a dataset reference from a string using from_string.

Maximum number of models to return. Defaults to a value set by the API.

Token representing a cursor into the models. If not passed, the API will return the first page of models. The token marks the beginning of the iterator to be returned and the value of the page_token can be accessed at next_page_token of the google.api_core.page_iterator.HTTPIterator.

How to retry the RPC.

The number of seconds to wait for the underlying HTTP transport before using retry.

Maximum number of models to return per page. Defaults to a value set by the API.

The table or reference from which to get partition info

How to retry the RPC.

The number of seconds to wait for the underlying HTTP transport before using retry. If multiple requests are made under the hood, timeout applies to each individual request.

Maximum number of projects to return. Defaults to a value set by the API.

Token representing a cursor into the projects. If not passed, the API will return the first page of projects. The token marks the beginning of the iterator to be returned and the value of the page_token can be accessed at next_page_token of the google.api_core.page_iterator.HTTPIterator.

How to retry the RPC.

The number of seconds to wait for the underlying HTTP transport before using retry.

Maximum number of projects to return in each page. Defaults to a value set by the API.

A reference to the dataset whose routines to list from the BigQuery API. If a string is passed in, this method attempts to create a dataset reference from a string using from_string.

Maximum number of routines to return. Defaults to a value set by the API.

Token representing a cursor into the routines. If not passed, the API will return the first page of routines. The token marks the beginning of the iterator to be returned and the value of the page_token can be accessed at next_page_token of the google.api_core.page_iterator.HTTPIterator.

How to retry the RPC.

The number of seconds to wait for the underlying HTTP transport before using retry.

Maximum number of routines to return per page. Defaults to a value set by the API.

The table to list, or a reference to it. When the table object does not contain a schema and selected_fields is not supplied, this method calls get_table to fetch the table schema.

The fields to return. If not supplied, data for all columns are downloaded.

Maximum number of rows to return.

Token representing a cursor into the table's rows. If not passed, the API will return the first page of the rows. The token marks the beginning of the iterator to be returned and the value of the page_token can be accessed at next_page_token of the RowIterator.

The zero-based index of the starting row to read.

The maximum number of rows in each page of results from this request. Non-positive values are ignored. Defaults to a sensible value set by the API.

How to retry the RPC.

The number of seconds to wait for the underlying HTTP transport before using retry. If multiple requests are made under the hood, timeout applies to each individual request.

A reference to the dataset whose tables to list from the BigQuery API. If a string is passed in, this method attempts to create a dataset reference from a string using from_string.

Maximum number of tables to return. Defaults to a value set by the API.

Token representing a cursor into the tables. If not passed, the API will return the first page of tables. The token marks the beginning of the iterator to be returned and the value of the page_token can be accessed at next_page_token of the google.api_core.page_iterator.HTTPIterator.

How to retry the RPC.

The number of seconds to wait for the underlying HTTP transport before using retry.

Maximum number of tables to return per page. Defaults to a value set by the API.

The destination table to use for loading the data. If it is an existing table, the schema of the pandas.DataFrame must match the schema of the destination table. If the table does not yet exist, the schema is inferred from the pandas.DataFrame. If a string is passed in, this method attempts to create a table reference from a string using from_string.

Number of upload retries. Defaults to 6.

Name of the job.

The user-provided prefix for a randomly generated job ID. This parameter will be ignored if a job_id is also given.

Location where to run the job. Must match the location of the destination table.

Project ID of the project of where to run the job. Defaults to the client's project.

Extra configuration options for the job. To override the default pandas data type conversions, supply a value for schema with column names matching those of the dataframe. The BigQuery schema is used to determine the correct data type conversion. Indexes are not loaded. By default, this method uses the parquet source format. To override this, supply a value for source_format with the format name. Currently only CSV and PARQUET are supported.

[Beta] The compression method to use if intermittently serializing dataframe to a parquet file. Defaults to "snappy". The argument is directly passed as the compression argument to the underlying pyarrow.parquet.write_table() method (the default value "snappy" gets converted to uppercase). https://arrow.apache.org/docs/python/generated/pyarrow.parquet.write_table.html#pyarrow-parquet-write-table If the job config schema is missing, the argument is directly passed as the compression argument to the underlying DataFrame.to_parquet() method. https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.to_parquet.html#pandas.DataFrame.to_parquet

The number of seconds to wait for the underlying HTTP transport before using retry. Depending on the retry strategy, a request may be repeated several times using the same timeout each time. Defaults to None. Can also be passed as a tuple (connect_timeout, read_timeout). See requests.Session.request documentation for details.

A pandas.DataFrame containing the data to load.

A file handle opened in binary mode for reading.

Table into which data is to be loaded. If a string is passed in, this method attempts to create a table reference from a string using from_string.

If True, seek to the beginning of the file handle before reading the file. Defaults to False.

The number of bytes to read from the file handle. If size is None or large, resumable upload will be used. Otherwise, multipart upload will be used.

Number of upload retries. Defaults to 6.

Name of the job.

The user-provided prefix for a randomly generated job ID. This parameter will be ignored if a job_id is also given.

Location where to run the job. Must match the location of the destination table.

Project ID of the project of where to run the job. Defaults to the client's project.

Extra configuration options for the job.

The number of seconds to wait for the underlying HTTP transport before using retry. Depending on the retry strategy, a request may be repeated several times using the same timeout each time. Defaults to None. Can also be passed as a tuple (connect_timeout, read_timeout). See requests.Session.request documentation for details.

Row data to be inserted. Keys must match the table schema fields and values must be JSON-compatible representations. .. note:: If your data is already a newline-delimited JSON string, it is best to wrap it into a file-like object and pass it to load_table_from_file:: import io from google.cloud import bigquery data = u'{"foo": "bar"}' data_as_file = io.StringIO(data) client = bigquery.Client() client.load_table_from_file(data_as_file, ...)

Table into which data is to be loaded. If a string is passed in, this method attempts to create a table reference from a string using from_string.

Number of upload retries. Defaults to 6.

Name of the job.

The user-provided prefix for a randomly generated job ID. This parameter will be ignored if a job_id is also given.

Location where to run the job. Must match the location of the destination table.

Project ID of the project of where to run the job. Defaults to the client's project.

Extra configuration options for the job. The source_format setting is always set to NEWLINE_DELIMITED_JSON.

The number of seconds to wait for the underlying HTTP transport before using retry. Depending on the retry strategy, a request may be repeated several times using the same timeout each time. Defaults to None. Can also be passed as a tuple (connect_timeout, read_timeout). See requests.Session.request documentation for details.

URIs of data files to be loaded; in format gs://<bucket_name>/<object_name_or_glob>.

Table into which data is to be loaded. If a string is passed in, this method attempts to create a table reference from a string using from_string.

Name of the job.

The user-provided prefix for a randomly generated job ID. This parameter will be ignored if a job_id is also given.

Location where to run the job. Must match the location of the destination table.

Project ID of the project of where to run the job. Defaults to the client's project.

Extra configuration options for the job.

How to retry the RPC.

The number of seconds to wait for the underlying HTTP transport before using retry.

SQL query to be executed. Defaults to the standard SQL dialect. Use the job_config parameter to change dialects.

Extra configuration options for the job. To override any options that were previously set in the default_query_job_config given to the Client constructor, manually set those options to None, or whatever value is preferred.

ID to use for the query job.

The prefix to use for a randomly generated job ID. This parameter will be ignored if a job_id is also given.

Location where to run the job. Must match the location of the table used in the query as well as the destination table.

Project ID of the project of where to run the job. Defaults to the client's project.

How to retry the RPC. This only applies to making RPC calls. It isn't used to retry failed jobs. This has a reasonable default that should only be overridden with care.

The number of seconds to wait for the underlying HTTP transport before using retry.

How to retry failed jobs. The default retries rate-limit-exceeded errors. Passing None disables job retry. Not all jobs can be retried. If job_id is provided, then the job returned by the query will not be retryable, and an exception will be raised if a non-None (and non-default) value for job_retry is also provided. Note that errors aren't detected until result() is called on the job returned. The job_retry specified here becomes the default job_retry for result(), where it can also be specified.

Method with which to start the query job. See QueryApiMethod for details on the difference between the query start methods.

SQL query to be executed. Defaults to the standard SQL dialect. Use the job_config parameter to change dialects.

Extra configuration options for the job. To override any options that were previously set in the default_query_job_config given to the Client constructor, manually set those options to None, or whatever value is preferred.

Location where to run the job. Must match the location of the table used in the query as well as the destination table.

Project ID of the project of where to run the job. Defaults to the client's project.

The number of seconds to wait for the underlying HTTP transport before using retry.

The number of seconds to wait for the query to finish. If the query doesn't finish before this timeout, the client attempts to cancel the query.

How to retry the RPC. This only applies to making RPC calls. It isn't used to retry failed jobs. This has a reasonable default that should only be overridden with care.

How to retry failed jobs. The default retries rate-limit-exceeded errors. Passing None disables job retry. Not all jobs can be retried.

The maximum number of rows in each page of results from this request. Non-positive values are ignored.

The maximum total number of rows from this request.

The dataset to update.

The properties of dataset to change. These are strings corresponding to the properties of Dataset. For example, to update the default expiration times, specify both properties in the fields argument: .. code-block:: python bigquery_client.update_dataset( dataset, [ "default_partition_expiration_ms", "default_table_expiration_ms", ] )