Use the Reddit API and BigQuery to build a Daily News Pipeline

Thanks to a well-documented API, we can use Python’s request library to derive the top news stories as well as associated user data…

How Data Engineers Can Use Python to Schedule BigQuery Queries

Learn how using Python to schedule BigQuery queries from a service account can save time and frustration for data engineers…

How Data Engineers Can Use SQL to Estimate BigQuery Storage Costs

For data engineers, SQL’s applications go beyond analysis; it can be a powerful tool for determining resource allocations.

No Whiteboard: Hiring Companies That Don’t Require a Coding Test

Test-averse data science candidates now have a resource to apply to companies that forgo the dreaded whiteboard test.

How To Create Nested Schemas in Python Using the Google BigQuery API

How data engineers can use Google’s BigQuery API in Python to specify nested schemas.

How to Restore Deleted BigQuery Tables

How to leverage BigQuery snapshots for data restoration.

BigQuery Hack: How to Quickly Populate a Table Without INSERT()

How to use BigQuery’s UI and a bit of SQL to populate a newly created table.

DeDuplicating: SQL vs. Python

Both SQL and Python offer powerful functions to help data engineers clean data and eliminate dreaded ‘dupes’ in datasets.

Cron-ological Order: How to Write Cron for Scheduling Pipelines

How to write cron expressions and why data engineers need to know them.

How to Use Python & SQL to Append New Rows to a BigQuery Table Without Overwriting Your Data

How to dynamically update rows in BigQuery using python and SQL without losing your historical data.

How Many Hours Per Week Should Data Professionals Really Dedicate to Learning?

Here’s how much time you should dedicate to learning new skills both at work and off-the-clock, according to experts.

SQL Quality Assurance Queries

How to construct queries in SQL to confirm data accuracy.