How I Reduced My Query’s Run Time From 30 Min. To 30 Sec. In 1 Hour
The query optimization steps a senior data engineer took to reduce the process time of a query processing 1 billion+ rows.
Latest
3 Python Web Scraping Projects You Can Do In 1 Weekend
Beginner-Friendly Python Web Scraping Projects.
The Ultimate Guide To GCP Observability
LQL, Python and Audit Strategies For Diagnosis & Triage. Part 1: Logging Query Language LQL can be used in the Logs Explorer to fetch real-time data on Google Cloud products like Cloud Functions and Virtual Machines as well as non-GCP resources like resources connected to Amazon Web Services’
A 5-Step Prompt To Land More Data Job Interviews
1 multistep job search prompt to objectively evaluate your candidacy before you even think about hitting “apply.”
Module 7: Cost & Performance Optimization
Module 6: Resilient Engineering
Module 5: Orchestration
Module 4: The Cloud Stack
Module 3: Automation
Module 2: Advanced SQL & Schema Design
Module 1: The Architect Mindset
Module 8
Tmux Is Tmagic
Leverage a subtle bash framework to execute data pipelines in the background, saving time for mission-critical tasks.
The Subtle Power Of SQL’s Overlooked DML Function
One function you gloss over has the power to save you hours of development time — and preserve data accuracy.
How Not To Annoy Senior Developers — Sincerely, A Senior Data Engineer
Proactive strategies covering how not to annoy your senior data scientists, engineers or developers from a senior’s perspective.
4 BigQuery Metadata SQL Queries To Save You Time, Money & Sanity
Never write another schema, save on storage costs and more.
The QA Pitfall That Tricks Most Data Engineers
Why matching row counts are the “silent killer” of data integrity.
This Dashboard Reveals My Crushing Debt & A Simpler Way To Visualize Data
Prioritizing clearer visualizations with a project that delivers a very personal, very real ROI.
Data-Driven Companies (Actually) Still Hiring Remotely
Highlighting a GitHub repo that might lead to your next remote data science opportunity — no matter where you live.
Why No One Can Find Your GitHub
Optimize your GitHub portfolio to increase discoverability and discover the new tool to push your repo to the masses.
Engineering Custom Email Alerts For Reddit In 3 Steps
Creating real-time email alerts in Python with the Reddit and Gmail APIs.
Why Your BigQuery SQL Query Will Or Won’t Run
Delve into the subtle technical and situational factors that determine whether your SQL query runs, stalls or fails.
Auto-Tag Sensitive BigQuery Data & Never Touch The UI Again
Eliminate a tedious data governance chore after understanding privacy taxonomies and a simple BigQuery API implementation.
Data Engineering’s 3rd “I”
Explaining state: How to maintain it, why pipeline runs need to be identical and breaking down an abstract term.