portfolio / Stocktok - Internet Scale Systems _

For my final year, I opted into the module - Engineering Internet Scale Systems. My team of six built and deployed a full-stack production web application called StockTok. The brief asked us to build an application utilising a microservice architecture that could theoretically scale. Our idea was a social trading terminal that stood between Bloomberg and Reddit.

Researching a stock requires looking at stock charts, Reddit or Twitter for sentiment, and utilising other sites for news and fundamental information about the company. The aim of StockTok was to put these charts, financial metrics, social feed and related news all in one place. The goal was to emulate a simpler and more accessible version of a Bloomberg Terminal (upwards of $20,000/yr).

The Architecture

A requirement was to go with a microservices architecture rather than a monolithic one. The system was split into four independent backend services. Each would own an individual database and communicate through an API Gateway via HTTP.

• Market Service (Python / FastAPI) - real-time price data, OHLC charts, fundamental metrics
• User Service (C# / ASP.NET Core) - authentication, identity, and watchlists
• Social Service (C# / ASP.NET Core) - posts and comments for tickers
• News Service (C# / ASP.NET Core) - news articles from the Marketaux API

The API Gateway sits in front of all services and was built using YARP (Yet Another Reverse Proxy). Its job was to route requests from the Next.js frontend and validate JWT tokens so that individual services don’t have to concern themselves with authentication.

The languages for each service were chosen depending on how well they were suited to their task. Python has a great ecosystem for financial data (pandas, yfinance) so it made sense for the market service. C#’s defined types and ASP.NET’s built-in security made it fit for handling user data. Using multiple languages is something a microservice architecture makes possible that a monolith doesn’t.

Contributions

I came up with the initial concept and worked primarily on the market service, user watchlists and deployment.

Market Service: Built alongside a teammate using FastAPI and Pandas. The service utilised the Yahoo Finance API and handled current price, OHLC history at multiple timeframes, and fundamental metrics like EV/EBITDA. Some derived fundamental metrics weren’t returned raw by any free API since they had to be computed from separate data points, so a custom pipeline handled that before sending anything to the frontend. Pydantic models enforced the schema so API keys were made human-readable before the frontend had access to them.

I also ended up finding undocumented YFinance endpoint while browsing and inspecting the YFinance website that returned search results directly. This meant that a cached database wasn’t needed to maintain every valid ticker symbol. It was small but it kept the service clean.

Watchlists: The backend of the user watchlists had a database schema, services, and controller endpoints which were all implemented using C#/ASP.NET Core. These lived inside the User Service since watchlists are user-owned data.

Integration and Merging: With six people developing services in parallel, integrating them as they came together took some initial planning. We used a staging branch as a buffer so that services merged into release went through system testing, and only then hit master. Merging was often where configuration conflicts between services surfaced mostly with NPM.

Deployment: The application was deployed onto a production VM. This involved configuring Docker Compose for production and setting up nginx as the reverse proxy, and configuring the VM since it was a fresh Debian install. Environment variables took some time setting up as they behave differently enough between local development and a production environment that it caused a few issues. Other post-development bugs were - routing misconfigurations in the API gateway, Auth0 not cooperating in production, nginx config issues, some services not working on the network as intended. These kind of things only show up once you’re actually running on a server.

Docker

Every service ran in a Docker container. This was the first time I’d used Docker on a team project and it showed the value of containerisation. When a new team member needs to run the full stack locally it was a single command. The Dockerfiles define the build environment for each service so the same image runs locally and in production, which mostly eliminates environment-specific bugs.

The databases were also isolated per-service with each microservice owning its data and nothing having a shared database.

Working in a Team of Six

This was the largest team I’d worked in on a software project and here are a few things that helped collaboration:

We used a forking workflow where each person owned their own fork and opened pull requests into a shared release branch. Pull requests required review before merging, which caught a few security issues such as accidental .env commits to the public repository. I realised then that git history rewriting tools exist for exactly this situation.

We used Trello as a Kanban board and had weekly standups on Teams, with WhatsApp for smaller quick fixes. The workflow was Agile-adjacent without being particularly rigid about it.

Scope management was also something we had to actively work at since we had other modules to study for and exams to write. There were ambitious ideas floating around such as AI price predictions, real-time alerts, but we used the MoSCoW method (Must have, Should have, Could have, Won’t have) to keep the MVP grounded.

Main Takeaways

Building something with a microservice architecture taught me things that are hard to understand until you’re actually dealing with them. Service isolation is useful, such as when the social service had issues during testing, the market data kept working. But the isolation also means every inter-service interaction has to go through a network call, and you end up writing a lot of glue code (in API Gateway) that a monolithic architecture wouldn’t need.

Docker made the team’s development experience significantly more consistent than it would have been otherwise.

The deployment phase was very instructive to me. There’s a gap between something working on your machine and something running reliably on a server that only closes once you’ve been through it a few times.

The repository is at github.com/sequitria/StockTok if you want to look at the code.