Tech IndustryFeb 21, 2019
AmazonBeffJezos!

How to test a distributed systems project?

I have made a toy distributed systems project which is a service that exposes multiple CRUD APIs. I am using some AWS component (Dynamo, SQS etc.) for every part of the service. Question 1: How do I test the scalability and availability of my system? I want to see how much load it can handle and how well does it scale from 10 users to a million users. Are there tools or test environments available? To simulate real world problems like servers go down or sudden increase in user traffic on a service? What I ideally want to learn is, if this API is used in a real world setting, and a startup starts with 10 users and grows to a million users, I need to learn how to scale the system (horizontal vs vertical) to keep up the availability of the system (reduce crashes or 500 errors). I have limited money to experiment on this and can spare $50 a month max. (Working within the free tier right now that AWS provides.) Question 2: Is this the right way to learn distributed systems architecture? I prefer a practical approach and not a theoretical one.

Add a comment
Amazon popozao Feb 21, 2019

In Amazon look for TPSGenerator in the wiki.

Amazon Sanchezos Feb 22, 2019

ChaosMonkey for failure scenarios

Bose justme2k19 Feb 22, 2019

Why is no one mentioning locust for stress testing

Indeed e5e3ych Feb 22, 2019

Good attempt. A+ for the effort. For testing, im sure you can find many load testing tools online depending on your app. If you have rest interface, maybe AB. Try to simulate failure scenarios by introducing delay in services (code a sleep), bring down instances, db master etc while load testing. Simulate replication delays, queue backups, imbalanced partitions, running CPU intensive stuff to increase load etc.

Amazon @&86/)/?3 Feb 23, 2019

I’m interested in doing something similar. Have limited knowledge in designing systems and would like to get practical experience in designing stuff. Does anyone know of a website where I can find projects/systems that I can build from scratch? Preferably something on data (pipelines or processing service etc).