Load testing the Cute server with 1 Million Connections

Introduction

In a previous post, we introduced the Cute server and explained how it enables developers to use Qt's signals and slots over a network. This post shows how the Cute server behaved under a load test. The benefits that the signals and slots mechanism brings when applied to perform network interactions are highlighted in another post.

The Cute server and SDKs enable objects to use signals and slots to communicate over a network.

QObject-derived classes containing signals and slots tagged as remote are mapped to endpoints on the server upon registration. Clients interact with remote objects through the RemoteObject class, which creates a remote object on the server and acts as a proxy to the created object. With the RemoteObject class, clients interact with remote objects as if they were local.

The Learning Hub has all the required information on how developers can use the Cute server and SDKs to streamline the development of modern connected systems using the signals and slots mechanism.

The load test was performed on AWS.

The Cute server is an innovative product that extends the signals and slots mechanism over a network. Thus, the software commonly used to load test a server is not applicable. Custom-made client software is required to load test the Cute server.

The developed client test app uses a manager-clients architecture. The client test app running in manager mode delegates the work to the instances running in client mode and consolidates their results. Then, the client test app running in manager mode informs how many remote objects were created and how many remote slot calls were made per second.

The Cute server loads all registered QObject-derived classes that should be made available for interaction with clients from a shared library.

The load test uses the following assets:

The Qt6 version of the Cute server Enterprise edition.
The Cute server configuration file.
The remote objects library.
The load test client app.

Because the client app has to handle many connections, Cute's epoll-based event handler is used on threads responsible for processing network traffic. Relying on Cute's epoll-based event handlers makes the client app Linux-only.

The client test app does not have its source available. However, the client test app does most of its work in a shared library. This shared library and the remote objects library used in the load test have their source code available on GitHub. Below are the results from running the load test with different configurations:

Each remote object used an exclusive WebSocket connection, and 10 remote slot calls were made on each remote object in the tests. The table informs how many remote objects were created and how many remote slot calls were done per second.

Cute can be configured to use the BPF to increase packet locality when processing network data. The BPF column informs if the Cute server was configured to use packet locality or not. The results shown in the table above were obtained with all instances running Debian Bullseye and with hyper-threading disabled.

The results show that packet locality is only relevant when the Cute server uses many workers.

If you know Qt, you can start coding connected apps right now. You are invited to run the load test by yourself.

The rest of this post provides step-by-step instructions on how to load test the Cute server with 1 million connections.

Required AWS infrastructure

The load test requires the following AWS/EC2 instances:

1 c5n.9xlarge instance to run the Cute server Enterprise.
10 c5n.2xlarge instances to run the client application in client mode.
1 c5n.2xlarge instance to run the client application in manager mode.

Create custom AMI

A custom AMI able to run both the server and the client test app is created from a base image running Debian Bullseye.

The instance requires the following assets:

The Qt6 version of the Cute server Enterprise edition.
The Cute server configuration file.
The remote objects library.
The load test client app.

After copying the assets to the instance, both the server and client app are set up with the following commands:

# update the instance
sudo apt-get update
sudo apt-get upgrade
sudo apt-get install tmux unzip

# Setting up the Cute server and the client test app
chmod a+x /home/admin/cute-server-enterprise.run
/home/admin/cute-server-enterprise.run --target /home/admin/cute-server
unzip libLoadTestLibrary.zip
unzip ClientTestApp.zip

The following modifications in the configuration files increase the network performance of the instance:

#Add the following to /etc/sysctl.conf
fs.file-max = 10000000
fs.nr_open = 10000000
net.ipv4.tcp_fin_timeout=30
net.core.netdev_max_backlog = 1310720
net.ipv4.tcp_max_syn_backlog = 1310720
net.core.somaxconn = 262144
net.ipv4.ip_local_port_range = 1024 65000
net.ipv4.tcp_mem = 15000000 18000000 20000000
net.core.rmem_default = 65536
net.core.rmem_max = 20485760
net.ipv4.tcp_rmem = 4096 16384 65536
net.core.wmem_default = 65536
net.core.wmem_max = 20485760
net.ipv4.tcp_wmem = 4096 16384 65536
net.core.optmem_max = 2048000
net.ipv4.fib_sync_mem = 4096000

#Add the following to /etc/security/limits.conf:
* soft nofile 10000000
* hard nofile 10000000

Now the instance is ready, and we can create an AMI from it. This AMI is used on all instances created in the load test.

Run client instances

The load test uses 10 c5n.2xlarge instances to run the client test app under client mode. These instances are set up with the following commands:

tmux new -s load-test
/home/admin/ClientTestApp/bin/DeploymentLoadTestClientApp

The commands above run the client app in client mode, making it wait for the client app running under manager mode to connect. The client test app uses port 2208 to communicate with its peers.

Run server instance

A c5n.9xlarge instance is used to run the Cute server. The command below runs the server as a service:

/home/admin/cute-server/bin/cute-server-enterprise -f /home/admin/cute-server.conf

And the commands shown below are useful while testing:

# The command below shows established connections
ss -s

# The command below shows memory usage in GB.
free -g

Run client instance as manager

Running the client test app in manager mode is the last step in the load test. The commands shown below should be run on a c5n.2xlarge instance:

tmux new -s load-test
/home/admin/ClientTestApp/bin/DeploymentLoadTestClientApp --clients='10.0.4.210;10.0.5.223;10.0.4.253;10.0.12.30;10.0.11.127;10.0.4.239;10.0.10.197;10.0.4.6;10.0.14.168;10.0.12.89' --server-address=10.0.13.26 --server-ports='2021;2022' --threads=4 --connections=100000 --iterationCount=10

The following command-line options configure the client test app to run in manager mode:

--clients: private IPs of the instances running the client app in client mode.
--server-address: private IP of the instance running the Cute server Enterprise.
--server-ports: ports that the Cute server Enterprise is listening to.
--threads: thread count used by each client test app running in client mode.
--connections: how many remote objects each client test app running in client mode will instantiate. Each remote object uses an exclusive network connection.
--iterationCount: how many times the remote slot will be called on each remote object instance.

The commands above run the client app in manager mode. In this mode, the client test app interacts with its peers running in client mode to load test the Cute server. After executing the test, the client test app running in manager mode informs how many remote objects were created per second and how many remote slots were called per second.

Tweak the test code

The steps shown above load tests the Cute server Enterprise with 1M connections and 10M requests. Besides using different instance types and quantities, the test can be modified as you wish by changing the server's configuration file and the client test app.

The client test app is closed source, but it uses a shared library to make most of its work.

The shared library used by the client test app and the remote objects library loaded by the Cute server are available on GitHub.

The code in the repository allows customization of both the client test app library and the remote objects library loaded by the Cute server. Before creating the AMI, custom-built client test app libraries and remote objects libraries should be copied respectively to the following folders: /home/admin/ClientTestApp/libs and /home/admin.

The load test shows that the Cute server is an enterprise-grade server that keeps its performance even under extreme loads.

The Cute server Blog