Write Your First ETL Pipeline (Part III)
Take a SQL script from a SQL environment to Google Cloud Platform by introducing a dynamic data check and upload step.
Zach Quinn
Everything Your Cloud Provider Won’t Tell You About Service Accounts
Learn service account logic, use cases and the unavoidable business problem they solve.
Stop The Bleeding: 4 Strategies To Troubleshoot, Triage Data Anomalies
Quickly identify, isolate and fix malfunctioning data pipelines for quality data, happier stakeholders and a stress-free workday.
Stop Writing Queries When You Need To Be Writing Views (With Examples)
Reduce the complexity and execution time of your queries with views for cleaner data and happier stakeholders.
Overcoming The Final Hurdle of Data Automation With Fewer Failures
Learn the components of data pipeline production to take your ETL build from code to cloud with automated, actionable results.
Track API Usage In Your Python Script — Not Your Credit Card Bill
In 5 lines of Python store information about requests, timestamps and status flags to avoid unexpected API charges.
Yes, You Can Edit Schemas In BigQuery–But Not How You Think
A hacky workaround for one of the biggest problems in SQL.
How You Can Use Python To Pull Stock Data For 3,000 Companies In Under 10 Minutes
How I used python to help a Wall Street banker pick stocks (part II).
How I’m Using Data Engineering To Help A Wall Street Banker Pick Better Stocks (Part I)
How a conversation between friends turned into a potentially lucrative data side project (with code walkthrough).
Stream Your Data Using Nothing But Python’s Requests Library
Understand when to stream data, how to configure a pipeline and learn from my mistakes building streaming pipelines.
Flattening Your SQL Tables — Not As Easy As It Sounds
How to expose deeply nested SQL fields — and why you need to be careful when handling complex raw data.
Write Your First SQL ETL Pipeline (Part II)
How to create and load an aggregate table for your GCP usage using ETL principles and SQL commands.