This tutorial will guide you step by step on how to do stress testing using apache benchmark (ab). But before we dive in, let us understand some basic terms mentioned below.
Software Testing
The act of checking that actual results match expected results and ensuring that the software system is free of defects. Performance testing can be further divided into 6 parts shown in the diagram below.
Performance testing is a software testing process used to test the speed, response time, stability, reliability, scalability, and resource utilization of a software application under a specific load. The main purpose of performance testing is to identify and remove performance bottlenecks in a software application. It is a subset of performance engineering and is known as “performance testing”.
Performance testing focuses on checking software programs:
• Speed – determines whether the application responds quickly
• Scalability — determines the maximum user load that a software application can handle.
• Stability — determines whether the application is stable under changing load.
Features and functions supported by the software system are not the only problems. What matters is the performance of the software application, such as its response time, reliability, resource utilization, and scalability. The goal of performance testing is not to find bugs, but to remove performance bottlenecks.
Performance testing is performed to provide stakeholders with information about their applications regarding speed, stability, and scalability. More importantly, performance testing reveals what needs to be improved before the product reaches the market. Without performance testing, the software is likely to suffer from problems such as: running slowly while being used by multiple users at the same time, inconsistency between different operating systems, and poor usability.
Types of performance tests:
• Load testing — checks the application’s ability to work under the expected user load. The goal is to identify performance bottlenecks before the software application is launched.
• Stress testing — involves testing the application under extreme load to see how it handles high traffic or data processing. The goal is to identify the breakpoint of the application.
• Endurance testing — done to ensure that the software can handle the expected load over a long period.
• Spike testing — tests the software’s response to sudden large spikes in user-generated load.
• Volume testing — a big number under volume testing. z. The data is stored in the database and the overall behavior of the software system is monitored. The objective is to check the performance of the software application at different database volumes.
• Scalability Testing — The goal of scalability testing is to determine the effectiveness of a software application when “scaling up” to support increasing user load. It helps you plan for expanding the capacity of your software system.
Basic parameters monitored during performance testing include:
• CPU utilization — the amount of time the processor spends executing idle threads.
• Memory usage — the amount of physical memory available for processes on the computer.
• Disk Time — The amount of time the disk is busy performing a read or write request.
• Bandwidth — shows the number of bits per second that the network interface is using.
• Private bytes — the number of bytes allocated to a process that cannot be shared with other processes. These are used to measure memory leaks and usage.
• Committed memory — the amount of used virtual memory.
• Memory pages/second — the number of pages written to or read from disk to resolve hard page faults. Fixed page faults are when code that is not from the current working set is called from elsewhere and read from disk.
• Fault pages/second — the overall rate at which fault pages are processed by the processor. This again occurs when a process requires code outside of its working set.
• CPU interrupts per second — is the avg. the number of hardware interrupts that the processor receives and processes every second.
• The length of the disk queue — is avg. No. of queued read and write requests for the selected disk during the sampling interval.
• Network output queue length — the length of the packet output queue in packets. Anything more than two indicates a delay and the taper should be stopped.
• Total Network Bytes Per Second — The rate at which bytes are sent and received on the interface, including frame characters.
• Response time — the time from when the user enters a request until the first character of the response is received.
• Throughput — the rate at which a computer or network receives data per second.
• Number of pooled connections — the number of user requests that are satisfied by pooled connections. The more requests fulfilled by the connections in the pool, the better the performance.
• the Maximum number of active sessions — the maximum number of sessions that can be active at one time.
• Access Ratios — This relates to the number of SQL statements that are processed by cached data instead of expensive I/O operations. This is a good place to start when troubleshooting bottlenecks.
• Number of hits per second — no. of visits to the web server during each second of the load test.
• Rollback segment — the amount of data that can be rolled back at any time.
• Database locks — the locking of tables and databases needs to be monitored and carefully tuned.
• Highest waits — monitored to see what wait times can be reduced by addressing how fast data is read from memory
• Number of threads — Application status can be measured by the number of threads that are running and currently active.
• Garbage collection — Refers to returning unused memory to the system. The wastecollection needs to be monitored for efficiency.
Which applications should we test?
Performance testing is only ever done for client-server-based systems. This means that any application that is not based on a client-server architecture must not require performance testing.
For example, Microsoft Calculator is not client-server based or multi-user; therefore, it is not a candidate for performance testing.
What is Apache Bench?
Apache Bench (ab) is a tool from the Apache organization for benchmarking the Hypertext Transfer Protocol (HTTP) web server. Although it is designed to measure the performance of the Apache web server, it can also be used to test any other web server that is equally good. With this tool, you can quickly find out how many requests per second your web server can handle.
Features of Apache Bench
Let’s take a look at the important features and limitations of Apache Bench.
Features and limitations are listed below −
• Since it is open-source software, it is freely available.
• It is a simple command-line computer program.
• It is a platform-independent tool. This means it can be invoked equally well on a Linux/Unix or Windows server.
• Can perform load and performance tests for web servers only — HTTP or HTTPS.
• Not extensible.
Note: If you are using windows, you need to install the ubuntu terminal from the Microsoft store. Ubuntu terminal
Once your Ubuntu is installed, you can create a username and password. You can follow the step-by-step process from this video. Youtube
Once everything is set up on your system, you can directly proceed to install Apache. Follow the steps below to install.
Apache Bench is a standalone application and does not depend on the Apache web server installation.
The following is a two-step process for installing Apache Bench.
1. Update package database.
apt-get update
Please note that symbol # before a terminal command means that the root user is issuing that command.
2. Install the apache2 utils package to get access to Apache Bench.
sudo apt-get install apache2-utils
Note that this doesn’t need to be installed on the same server where you want to run the test. You can install it even on your local desktop Ubuntu and run the test to a specific URL. This package provides additional tools that are useful for any web server. It includes:
3. Verifying Apache Bench Installation
Let us now see how to verify Apache Bench Installation. The following code will help verify the installation –
ab -V
4. When you see the above terminal output, it means you have successfully installed Apache Bench
5. Testing Apache.org Website
In this section, we will test the Apache.org Website. Let us first switch to the sudo user test –
su test → su anurag7270
6. To begin with, we will test the website of Apache organization, https://www.google.in/. We will first run the command, and then understand the output –
7. Running a stress test
To put your server under stress to check how it behaves with a considerable amount of requests, you can get started with the following command:
ab -c 100 -n 500 -r [URL]
Where:
-c: concurrency, the number of multiple requests to perform at a time. Default is one request at a time.
-n: requests, the number of requests to perform for the benchmarking session. The default is to just perform a single request which usually leads to non-representative benchmarking results.
-r: Don’t exit on socket receive errors.
8. And as a positional argument, the URL of your website with the specified module e.g / (for the homepage).
9. ab [options …..] URL
where,
· ab − Apache Bench command
· options − flags for the particular task we want to perform
· URL − path URL we want to test
10. Let’s try to test the website of Apache organization, https://www.apache.org/. We will first run the command, and then understand the output –
ab -n 100 -c 10 https://www.apache.org/
11. If you closely look at the output values, we have a different metric to understand the various output values returned by ab. Let us understand this one by one.
· Server Software − It is the name of the web server returned in the HTTP header of the first successful return.
· Server Hostname − It is the DNS or IP address given on the command line.
· Server Port − It is the port to which ab is connecting. If no port is given on the command line, this will default to 80 for HTTP and 443 for HTTPS.
· SSL/TLS Protocol − This is the protocol parameter negotiated between the client and server. This will only be printed if SSL is used.
· Document Path − This is the request URI parsed from the command line string.
· Document Length − It is the size in bytes of the first successfully returned document. If the document length changes during testing, the response is considered an error.
· Concurrency Level − This is the number of concurrent clients (equivalent to web browsers) used during the test.
· Time Taken for Tests − This is the time taken from the moment the first socket connection is created to the moment the last response is received.
· Complete Requests − The number of successful responses received.
· Failed Requests − The number of requests that were considered a failure. If the number is greater than zero, another line will be printed showing the number of requests that failed due to connecting, reading, incorrect content length, or exceptions.
· Total Transferred − The total number of bytes received from the server. This number is essentially the number of bytes sent over the wire.
· HTML Transferred − The total number of document bytes received from the server. This number excludes bytes received in HTTP headers
· Requests per second − This is the number of requests per second. This value is the result of dividing the number of requests by the total time taken.
· Time per request − The average time spent per request. The first value is calculated with the formula concurrency * time is taken* 1000 / done while the second value is calculated with the formula time taken * 1000 / done
· Transfer rate − The rate of transfer as calculated by the formula total read / 1024 / time taken.
12. Plotting the Output of Apache Bench
Here we will plot the relevant outcome to see how much time the server takes as the number of requests increases. For that, we will add the -g option in the previous command followed by the file name (here out.data) in which the ab output data will be saved –
ab -n 100 -c 10 -g out.data https://www.apache.org/
13. Let us now see the out.data before we create a plot –
less out.data
14. Let us now understand the column headers in the out.data file −
· starttime − This is the date and time at which the call started.
· seconds − Same as starttime but in the Unix timestamp format (date -d @1496160697 returns starttime output).
· ctime − This is the Connection Time.
· dtime − This is the Processing Time.
· ttime − This is the Total Time (it is the sum of ctime and dtime, mathematically ttime = ctime + dtime).
· wait − This is the Waiting Time.
(Tried this part…below, but got an error…will try later this part)
If we are working over a terminal or where graphics are not available, gnuplot is a great option. We will quickly understand it by going through the following steps.
Let us install and launch gnuplot −
sudo apt-get install gnuplot
It will take a few to install.
Once it’s installed on your system, you can use the following command on the ubuntu terminal
gnuplot
ab -c 20 -n 1000 [URL] > result.txt
ab -c 20 -n 1000 -e “result1.csv” [URL]
Conclusion
In Software Engineering, Performance testing is necessary before marketing any software product. It ensures customer satisfaction & protects an investor’s investment against product failure. We learned how Apache Bench can be used to load test any website or web application. Apache Bench can be a very valuable tool for determining how your web application server setup should be improved, to reduce bottlenecks and increase performance. Costs of performance testing are usually more than made up for with improved customer satisfaction, loyalty, and retention.
If you have any questions or suggestions regarding the blog, please comment down below. In case you have not followed me yet, please do. I hope to see you again for another blog.
Till then, take care!
References
For more information in detail. Read the article mentioned in the below link.
1. https://www.guru99.com/performance-testing.html
2.https://www.tutorialspoint.com/apache_bench/apache_bench_quick_guide.htm
3. https://www.ndchost.com/wiki/apache/stress-testing-with-apache-benchmark
