refactor(bigquery): update code samples to use strings for table and dataset IDs #9136

emar-kar · 2019-08-28T15:30:55Z

Towards #8989
This PR contains five snippets:

client_list_jobs
client_query
copy_table
table_exists
table_insert_rows

sync forks

*.rst *.py test conf

Methods were divided into 3 files: - add label - get labels - delete labels *.rst - docs updated tests passed successfully

minor corrections, 'dataset_exists' moved to the 'Getting a Dataset' section

grammar fix

minor corrections, removed extra comments

deleted 'dataset_exists' and 'table_exists' methods

Added additional asserts into the test refactoring of the main file.

cosmetic chgs by 'black'

…_tearing

Chged an assertion parameter

plamut · 2019-09-19T16:54:21Z

@emar-kar Of course... I was apparently looking at snippets.py when writing the review comment, my bad. The number of commits is huge, and in retrospect I should have probably looked at the final set of changes right from the start.

emar-kar · 2019-09-19T16:58:14Z

@plamut yeah, the amount of commits is really huge. This is my bad. I’ll fix it with the next part.

tswast · 2019-09-26T21:37:42Z

bigquery/samples/copy_table.py

+    # table_id = "your-project.your_dataset.your_table_name"
+
+    orig_table = client.get_table(table_id)  # Make an API request.
+    dataset = client.get_dataset(dataset_id)  # Make an API request.


Calls to get_table and get_dataset are unnecessary. Let's use

# TODO(developer): Set source_table_id to the ID of the original table. # source_table_id = "your-project.source_dataset.source_table" # TODO(developer): Set destination_table_id to the ID of the destination table. destination_table_id = "your-project.destination_dataset.destination_table"

I also decided to move the num_rows assertion to the test file.

tswast · 2019-09-26T21:40:04Z

bigquery/samples/client_query.py

+    print("The query data:")
+    for row in query_job:
+        # Row values can be accessed by field name or index
+        print(row)


Let's demonstrate that fields can be accessed by name or index.

Suggested change

print(row)

print("name={}, count={}".format(row[0], row["count"]))

tswast · 2019-09-26T21:43:59Z

bigquery/samples/client_query.py

+    # TODO(developer): Construct a BigQuery client object.
+    # client = bigquery.Client()
+
+    query = (


Since we want to show more than one field in the print logic, let's select more than one column.

Suggested change

query = (

query = """

SELECT name, SUM(number) as total_people

FROM `bigquery-public-data.usa_names.usa_1910_2013`

WHERE state = 'TX'

GROUP BY name, state

ORDER BY total_people DESC

LIMIT 20

"""

tswast · 2019-09-26T21:46:31Z

bigquery/samples/tests/test_client_query.py

+    client_query.client_query(client)
+    out, err = capsys.readouterr()
+    assert "The query data:" in out
+    assert "Row(" in out


Since we're using the usa_1910_2013 table, the data won't change. We can use a specific value in our tests.

Suggested change

assert "Row(" in out

assert "name=James, count=272793" in out

- copy_table - client_query - test_copy_table - test_client_query

tswast · 2019-10-08T21:09:43Z

Thanks for your patience. I've been travelling a lot lately, but now I'm back.

Re: Make and API request vs Makes an API request, I'd interpret these in two ways:

(imperative) You, the developer, should Make and API request.
(descriptive) This line of code Makes an API request.

Since our code samples are included in how-to guides, our technical writing style guide requires (1) imperative.

emar-kar · 2019-10-08T21:13:11Z

Yeah, sorry. I pushed the commit with -s modification today. I'll revert it tomorrow. Thank you, appreciate your help.

emar-kar · 2019-10-09T10:59:30Z

@tswast One more thing with the comments. I also added Waits for the job to complete lines in #9212. Maybe it is required to be without -s too?

tswast · 2019-10-10T21:59:24Z

@tswast One more thing with the comments. I also added Waits for the job to complete lines in #9212. Maybe it is required to be without -s too?

Yes, without the -s would more closely match our tech writing style guide.

mf2199 and others added 30 commits August 8, 2019 23:11

Merge pull request #26 from googleapis/master

6b3a2a3

sync forks

Move every snippet to it's own file. Create test templates.

4c6d7b4

list_datasets_by_label

377310c

*.rst *.py test conf

Merge remote-tracking branch 'upstream/master'

5ce976f

manage_dataset_labels

a7b2376

Methods were divided into 3 files: - add label - get labels - delete labels *.rst - docs updated tests passed successfully

add_empty_column

0de5d8b

browse_table_data

5e05cd8

dataset_exists

f6b60a0

complete change of dataset_exists

a773251

Merge remote-tracking branch 'upstream/master'

8375877

five updated snippets

a0792bd

Update datasets.rst

95dc8f8

minor corrections, 'dataset_exists' moved to the 'Getting a Dataset' section

cosmetic chgs

cf7576e

client_list_jobs

0645e33

client_query

92efd59

copy_table

1814821

cosmetic chgs

a31bc1f

Update datasets.rst

725c515

minor corrections, 'dataset_exists' moved to the 'Getting a Dataset' section

cosmetic chgs

6ffc123

grammar fix

cosmetic chgs

5654dcd

grammar fix

table_exists

2ff376c

Update test_copy_table.py

948ddb5

minor corrections, removed extra comments

Update test_table_exists.py

e8d62c4

minor corrections, removed extra comments

Update snippets.py

28251ec

deleted 'dataset_exists' and 'table_exists' methods

update client_list_jobs

ad623ba

Added additional asserts into the test refactoring of the main file.

table_insert_rows

ec67aee

Update client_query.py

c8c0f70

cosmetic chgs by 'black'

Merge remote-tracking branch 'upstream/master'

42204ed

Merge commit '2d622fd2de8654c0e82a53f709bd377ab3e0a1ff' into snippets…

b4dade6

…_tearing

update client_query

febfa11

Chged an assertion parameter

emar-kar changed the title ~~BigQuery: Update code samples to use strings for table and dataset IDs~~ refactor(bigquery): Update code samples to use strings for table and dataset IDs Sep 25, 2019

emar-kar changed the title ~~refactor(bigquery): Update code samples to use strings for table and dataset IDs~~ refactor(bigquery): update code samples to use strings for table and dataset IDs Sep 25, 2019

comments rephrasing

ab504bf

plamut approved these changes Sep 25, 2019

View reviewed changes

plamut added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Sep 25, 2019

yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Sep 25, 2019

tswast requested changes Sep 26, 2019

View reviewed changes

emar-kar added 3 commits September 27, 2019 11:13

Merge branch 'master' into second-five-v2

176395e

updated as requested

e67d172

- copy_table - client_query - test_copy_table - test_client_query

comment lines update

da4a2bc

emar-kar added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Oct 7, 2019

yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Oct 7, 2019

lint fix

8852951

emar-kar added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Oct 8, 2019

comments update

6d533b3

emar-kar removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Oct 8, 2019

emar-kar added 2 commits October 8, 2019 12:42

Merge branch 'master' into second-five-v2

327373c

Update update_dataset_access.py

e55f698

emar-kar requested a review from tswast October 8, 2019 11:40

revert commit with -s

ef884b8

delete -s from comment lines

50394cd

tswast approved these changes Oct 14, 2019

View reviewed changes

emar-kar merged commit 01f6826 into googleapis:master Oct 15, 2019

emar-kar deleted the second-five-v2 branch October 16, 2019 15:03

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor(bigquery): update code samples to use strings for table and dataset IDs #9136

refactor(bigquery): update code samples to use strings for table and dataset IDs #9136

emar-kar commented Aug 28, 2019

plamut commented Sep 19, 2019

emar-kar commented Sep 19, 2019

tswast Sep 26, 2019

emar-kar Sep 27, 2019

tswast Sep 26, 2019

tswast Sep 26, 2019

tswast Sep 26, 2019

tswast commented Oct 8, 2019

emar-kar commented Oct 8, 2019

emar-kar commented Oct 9, 2019

tswast commented Oct 10, 2019

	print(row)
	print("name={}, count={}".format(row[0], row["count"]))

-    query = (
+    query = """
+    SELECT name, SUM(number) as total_people
+    FROM `bigquery-public-data.usa_names.usa_1910_2013`
+    WHERE state = 'TX'
+    GROUP BY name, state
+    ORDER BY total_people DESC
+    LIMIT 20
+    """

	assert "Row(" in out
	assert "name=James, count=272793" in out

refactor(bigquery): update code samples to use strings for table and dataset IDs #9136

refactor(bigquery): update code samples to use strings for table and dataset IDs #9136

Conversation

emar-kar commented Aug 28, 2019

plamut commented Sep 19, 2019

emar-kar commented Sep 19, 2019

tswast Sep 26, 2019

Choose a reason for hiding this comment

emar-kar Sep 27, 2019

Choose a reason for hiding this comment

tswast Sep 26, 2019

Choose a reason for hiding this comment

tswast Sep 26, 2019

Choose a reason for hiding this comment

tswast Sep 26, 2019

Choose a reason for hiding this comment

tswast commented Oct 8, 2019

emar-kar commented Oct 8, 2019

emar-kar commented Oct 9, 2019

tswast commented Oct 10, 2019