Systems that work well during development, deployed on a small scale, often fail to meet performance goals when the deployment is scaled up to support real levels of use.
We’ve all heard from ambulance dispatch systems to retail sites, or sites for the electronic submission of tax returns, systems fail as they scale and experience peak demands. All of these projects appear not to have identified and prioritised the major risks they faced. This is a fundamental stage of risk based testing, and applies equally to scalability testing or load testing as it does to functionality testing or business continuity testing. With no risk assessment they didn’t recognise that scaling was amongst the biggest risks, far more so than delivering all the functionality.
Trends towards Service Oriented Architecture (SOA) attempt to address the issue of scalability but also introduce new issues. Incorporating externally provided services into your overall solution means that your ability to scale now depends upon how these external systems operate under load. Assuring this is a demanding task and sadly the here is often overlooked.
Better practice is to start the development of a large scale software system with its performance clearly in mind, particularly with regard to scalability and load.To create this performance testing focus, first understand what type of performance testing is appropriate for your company or project. This may of course inevitably be timescale or budget driven.Two options to consider are Load Testing and Application Profiling:-
Load Testing takes representative business transactions ,(frequently called Use Cases or User Journeys) and multiplies them executing a volume based test against the application environment providing information as to how the application will sustain loads of say, hundreds or thousands of users. The reporting is generally based on application response times and times to complete transactions.
Load tests can be run from within your infrastructure, typically using hardware to act as load injectors. Equally, load testing can be executed from the Cloud, thus accessing the application or site over the Internet and from differing global locations if required.
Incorporating some network emulation (or “network virtualization”) into the load test will deliver even more tangible results in the context of your own or the proposed network conditions.
Excessive network latency or packet loss can make your application respond slower, behave erratically or even fail and since their behaviour and scalability change over different network conditions, most web and mobile applications are sensitive to the impact of the network. By not emulating real-world network conditions during your performance testing, you are very likely to pay the price through poor performance in production. As we all know, fixing the problem after the event is generally more costly than minimizing or even eliminating it before deployment.
Application Profiling is a technique which allows for the analysis of key transactions and their modeling under different network conditions. Not to be confused with Network Virtualization (above) which effectively implements real-world network conditions within the test environment, Application Profiling enables in-depth transaction analysis and modeling of performance under differing network constraints.
Transactions are examined in test environment, analyzed and any concerns in current performance are highlighted. This method involves detailed packet analyses and can offer some real insight into the how the application has been developed with the users in mind. In addition, transactions are modeled to confirm the anticipated behaviour on the proposed infrastructure and the response times predicted under differing network conditions (bandwidth, latency etc).
This is a lot faster than load testing and can be a real eye-opener acting as an early indicator of performance threats. As such this technique is often used as a precursor to a full load test, as the application in question can be optimized before being subjected to a volume based test.
Note that Application Profiling is also a valuable technique to use when troubleshooting live application performance issues. Lightweight agents can be distributed across the infrastructure to ascertain and prove the true cause of performance problems with the output meaningful to all personnel involved such as developers, network staff, DBAs and business management.
Ideally, one would employ both disciplines in pre-production (Load testing and Application Profiling); however this is not always possible.The two are very different in their set-up and output, so the selection of the method to meet the requirement is important. The check list below will help.
1. Research and quantify the data volumes and transaction volumes the target market implies.This alone can lead to reassessment of the priority of many features.
2. Determine the way features could be presented to users and the system structured in order to make scaling of the system easier; the functionality aligned to a single user desktop solution will differ when architecting a scalable alternative.
3. Recognise that an intrinsic part of the development process is load testing at representative scale on each incremental software release. This is continual testing, focusing on the biggest risk to the project; the ability to operate at full scale.
4. Ensure load testing is adequate both in scope and rigour. Load testing is not just about measuring response times with a performance test. The load testing program needs to include other types of performance testing, possibly network virtualisation, application profiling and stress testing.
5. Don’t forget that failures will occur. Large scale systems generally include server clusters with fail-over behaviour. Failure testing, fail-over testing and recovery testing carried out on representative scale systems operating under load should be considered and where appropriate included.
6. Don’t forget that load testing is generally completed in a dedicated test environment. You will need to take into account the network (and likely connectivity) of the real end users. Here, some key transaction profiling and modelling techniques and / or network virtualisation can be applied to extrapolate the expected response times to be achieved in the real world.
7. Recognise external services if you use them. Where you are adopting an SOA approach and are dependent on external services you need to be certain that the throughput and turnaround time on these services will remain acceptable as your system scales and its demands increase. A smart system architecture will include a graceful response and fall-back operation should the external service behaviour deteriorate or fail.
Verify Solutions provides software and services to help with:
- Live application performance monitoring and troubleshooting
- Application load testing and performance prediction and
- Functional, Security and Penetration Testing
Please do contact us if you feel we can help you.