Assessing web application performance on single-core machine

Modern web applications developed on top of non-blocking I/O facilities can easily serve tens of thousands of concurrent requests per host (with moderate sized hardware). Together with horizontal scalability, scenarios where applications serve millions of concurrent requests are not new. Each web application instance being deployed is supposed to handle different amount of concurrent load depending on use cases and it needs to be deployed on machine with suitable hardware. Sometimes these applications are deployed on predetermined over-sized hardware which could result into unnecessary infrastructure usage cost. Assessing optimal hardware size for running an application could be important in such cases. I often wonder how much concurrent load a single-core machine can handle and hence writing this post based on my experiments

Test Scenario
For assessing baseline performance on single-core machine, I am taking example of basic demo web application having RESTful service endpoint. Usually these kind of applications don’t perform CPU-intensive calculations, rather they rely on external data sources and/or other applications to exchange data and perform some in-memory processing on data as per the required use cases. I have developed 2 applications (one with Java and another with Rust) having one RESTful service endpoint /api/demo. These applications are deployed on a host with single-core CPU. To serve a single request - implementation in this endpoint makes 8 different sequential REST API calls to another application running on remote host (to mimic requests to external systems). This another application is running on host with sufficient hardware to handle any number of concurrent requests which could potentially come from demo application running on single-core host. I have used Gatling application as load-testing tool which runs various load testing scenarios from yet another host. All 3 hosts are running as part of same virtual network in the cloud. Following diagram represents how applications are deployed across hosts -

Software Stack
Operating System : Ubuntu 22.04
Amazon Corretto JDK (17.0.4)
Rust 1.63.0

Java application is developed using Reactive spring boot framework and it is running on top of Netty. Rust application is developed using Axum (Web application framework) and Reqwest (HTTP client), both utilizes and runs on top of Tokio runtime. Source code for demo application and gatling load test scenarios is located at single-core-performance-test

Before capturing any stats from load testing scenarios, applications running on single-core machine are warmed up with 120 requests/sec for 10 minutes. Of course Rust application doesn’t need such extensive amount of warm up (indeed doesn’t need any warm up at all), but just for sake of maintaining consistency between executing load testing scenarios on Java and Rust demo applications, I execute warm up scenarios from gatling app in all cases. Actual load testing scenarios are executed for 5 minutes and statistics are captured. For different load testing scenarios, concurrent requests/sec is adjusted in such a way that CPU on single-core machine remains idle for around 10% - 15% throughout the run. Here are the stats of load testing scenarios when sending concurrent requests with establishing new connection for each request -

Load test stats - new connection for each request

When load testing scenario is sending concurrent HTTP requests with new connection for each request, resulting stats looks good considering single-core machine and as we can see Rust app is able to handle ~1.8 times more requests per second over same hardware resources. But if we look at load testing scenario stats for sending concurrent HTTPS requests with new connection for each request, Java app is able to handle constant load of just 195 requests/sec. Even if we see performance of natively generated code in Rust app, it is able to handle constant load of 540 requests/sec. We can see that establishing new SSL/TLS connection involves lot of overhead due to necessary initial handshaking between client and server. In this example - load test scenario is establishing new connection for each request, however in real-world if multiple requests are coming from same browser or from some sort of HTTP client instance (programmatically) - usually subsequent requests can reuse connection from pool of already established connections which are idle. Following load testing scenario stats are based on such use case where initially set of persistent connections are established and subsequent requests are sent to the server with 1 second delay from each established connection -

Load test stats - reuse persistent connections

As we can see when sending new requests by reusing already established connections - stats are comparatively better than establishing new connection for each request. Even if we consider scenarios with HTTPS requests - with Java app it is able to handle constant load of ~1100 requests/sec and with Rust app it is able to handle constant load of ~1950 requests/sec

In real-world - usually there is a mixed incoming traffic with some requests are establishing new connections and some requests are received on already established connections. We can expect this single-core machine can handle constant load of concurrent requests in range of the observed stats between above 2 scenarios. Remember these stats are specific to demo web application use case I have considered in this post. In my opinion - single-core machines are very much capable of serving few hundreds to thousands of requests per second and if that is not sufficient - horizontal and/or vertical scalability can always be implemented

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Parth Mistry

Enterprise Application and BigData engineering enthusiast. Interested in highly efficient low-cost app design and development with Java/Scala and Rust 🦀