A recent client of mine who has a rapidly growing userbase was afraid of the fact that after a while their infrastructure will not be enough. To minimize infrastructure costs, we have to make sure we know its current capability and we have to find bottlenecks to eliminate.
Serverless computing
AWS Lambda is an event-driven, serverless platform provided by Amazon and it lets you run code without dealing with scaling. It runs a pay only as much as you actually use pricing model with a quite good free limit so it is very cost-effective.
Its only drawback is a so called Cold Start. It happens during scaling when a new container with our code gets fired up and in some circumstances it may take a quite a while, depending on the programming language, memory size and dependencies. Even though AWS published an update that reduced this time by up to 10 times, my client has Cold Starts as long as 15 seconds with a lightweight Python application.
Stress test with JMeter
To find out the current capabilities of the infrastructure, first of all, we had to run some stress test. As I have already used JMeter a couple of times it took a few hours to get it done. It had to call only a few endpoint with randomly picked data combination.
I ran it with multiple user number. A user is a synchronous thread that runs the next request as soon as the previous finished so users run parallel.
Results
Since the database run with only one instance we knew it will be the bottleneck of the infrastructure at the moment but we wanted a picture of the current capabilities.
Conclusion
Since AWS Lambda scales automatically we measured the current capability of the ArangoDB running on an EC2 instance type of medium T2. With an ideal operating point of ~275 requests per second the average response times were under half a second where the database CPU usage got maxed out.
Switching the database to a scalable backend and optimizing the Python application itself will be another post.
