How You Can Use Python To Pull Stock Data For 3,000 Companies In Under 10 Minutes

How I used python to help a Wall Street banker pick stocks (part II).

Zach Quinn

— 8 min read
Share
Graph of time vs. wealth with money representing the bars.
In this case time is build and execution time. Photo by Morgan Housel on Unsplash.

This is the second part in a multi-part series. See part I below:

How I’m Using Data Engineering To Help A Wall Street Banker Pick Better Stocks (Part I)
How a conversation between friends turned into a potentially lucrative data side project (with code walkthrough).

Data Science Is Cool, But It’s Not Magic

Being immersed in the data science world (at work and in online communities), sometimes you forget how impressive and, frankly, cool, a well-conceived data product can be.

Those, like my investment banker friend, who work in data-adjacent industries (think non-tech analysts, marketers and others who touch data but don’t “know” the tech), treat data like a black box.

They expect an output to appear but typically don’t understand the input that generates their desired output.

Because these individuals don’t always understand the process of data retrieval and aggregation, they also mistake it for being something incredibly difficult–almost magical.

When I created the initial output I shared in part I, my friend told me that he didn’t know accurate data could be retrieved quickly (though he also had choice words for his organization’s data contractors, so it was a low bar).

Depending on the robustness and clarity of an API, sometimes it really is easy to create a pipeline–well, at least one without all the kinks worked out.

One of the easiest things to generate in programming is an output. Getting that output to be correct is the hard part.

So that’s what part II of my automate-an-investing-workflow series is about–the hard part.

Build Your Pipeline To A Data Engineering Career

You’ve reached the limit of the public preview. The full version of this post includes the implementation details: The code, the edge cases, and the "why" behind the architecture.

When you join PipelineToDE, you get:

  • The DA → DE Pathway Course: A structured roadmap to bridge the gap between analysis and engineering.
  • Weekly Senior Deep Dives: Fresh, tactical insights on Python, Cloud (GCP/AWS), and modern orchestration delivered every week.
  • Production-Ready Blueprints: Access to 80+ protected stories and code repos from my time in the trenches as a Senior DE
  • The DE Job Board (Coming Soon): Exclusive access to a curated board of high-agency Data Engineering roles.

Read more