When the Pseudo Markets project was first conceived in mid 2019, I had very little experience working with distributed systems. All of the previous backends I had worked with were huge monoliths that connected to a single relational data storage system. Since then, I have had the opportunity to work with systems that handle daily transactional volumes I wouldn’t even be able to fathom just a few years ago. One of the design decisions I made when coming up with the primary backend system for Pseudo Markets was driven by my lack of knowledge of making certain services independent, decoupled, and small. Despite not running into any immediate issues with the current design, I took it upon myself to apply what I have learned in the past year or so to re-architect the entire solution, starting with decoupling the large and tightly packed core functionality of the Unified API.
Several large pieces, such as the core equities trading platform, market data service, and portfolio performance reporting service have already been decoupled into their own services. This allows horizontal scaling as each service is distributed, whereas the previous architecture would only benefit from vertical scaling. This also allows the Unified API to just focus on one job, bringing together or “unifying” data from different services. Decoupling login and authentication from the Unified API brings the entire platform one step closer to becoming fully distributed. While most of the work is porting out and cleaning up the existing authentication code, I also have an excellent opportunity to start utilizing the Real Time Data Store (Aerospike) as a fully functional system of record. Currently, the RTDS is used as a high speed caching layer for the Market Data Service, but I have plans to move all data from the Relational Data Store (SQL Server) into RTDS.
The new Unified Auth Service will feature the capability to sync account, user, and token data across RTDS and RDS, provide data from either source, and function as both a standalone API as well as a DLL. There will be 3 modes of operation:
- RDS Mode: All logins will be processed using data from RDS
- RTDS Mode: All logins will be processed using data from RTDS
- Hybrid Mode: Logins will go through round robin to use RTDS or RDS
You can track progress here: https://github.com/pseudomarkets/PseudoMarkets.Unified.Auth. I’m hoping to get this working in our dev environment first, then promoting to our new DR server in Austin, before finally pushing it out to our two primary prod app servers in DFW.
Pseudo Markets Blog 