| Systems that work well during development, | | | | 1. Research and quantify the data volumes and |
| deployed on a small scale, can fail to meet | | | | transaction volumes the target market implies. Some |
| performance goals when the deployment is scaled up | | | | of these figures can be eye openers and help the |
| to support real levels of use. | | | | business users realise the full scale of the system. |
| An apposite example of this comes from a major | | | | This alone can lead to reassessment of the priority |
| blue chip company that recently outsourced the | | | | of many features. |
| development of an innovative high technology | | | | 2. Determine the way features could be presented to |
| platform. Though development was behind schedule | | | | users and the system structured in order to make |
| this was deemed acceptable. The system gradually | | | | scaling of the system easier. Do not try and have |
| passed through functional elements of the user | | | | the same functionality you would have for a single |
| acceptance testing and eventually it looked like a | | | | user desktop solution provide an appropriate scalable |
| deployment date could be set. But then the supplier | | | | alternative. |
| started load testing and scalability testing. There | | | | 3. Recognise that an intrinsic part of the development |
| followed a prolonged and costly period of | | | | process is load testing at representative scale on |
| architectural changes and changes to the system | | | | each incremental software release. This is continual |
| requirements. The supplier battled heroically to | | | | testing, focusing on the biggest risk to the project: |
| provide an acceptable system, until finally the project | | | | the ability to operate at full scale. |
| was mothballed. | | | | 4. Ensure load testing is adequate both in scope and |
| This is not an isolated case. IT folklore abounds with | | | | rigour. Load testing is not just about measuring |
| similar tales. From ambulance dispatch systems to | | | | response times with a performance test. The load |
| web-sites for the electronic submission of tax | | | | testing programme needs to include other types of |
| returns, systems fail as they scale and experience | | | | load testing including stress testing, reliability testing, |
| peak demands. All of these projects appear not to | | | | and endurance testing. |
| have identified and ordered the major risks they | | | | 5. Don?t forget that failures will occur. Large scale |
| faced. This is a fundamental stage of risk based | | | | systems generally include server clusters with |
| testing, and applies equally to scalability testing or | | | | fail-over behaviour. Failure testing, fail-over testing |
| load testing as it does to functionality testing or | | | | and recovery testing carried out on representative |
| business continuity testing. With no risk assessment | | | | scale systems operating under load should be |
| they did not recognise that scaling was amongst the | | | | included. |
| biggest risks, far more so that delivering all the | | | | 6. Don?t forget catastrophic failure could occur. For |
| functionality | | | | large scale problems, disaster testing and disaster |
| Recent trends towards Service Oriented Architecture | | | | recovery testing should be carried out at |
| (SOA) attempt to address the issue of scalability but | | | | representative scale and loads. These activities can |
| also introduce new issues. Incorporating externally | | | | be considered the technical layers of business |
| provided services into your overall solution means | | | | continuity testing. |
| that your ability to scale now depends upon these | | | | 7. Recognise external services if you use them. |
| external system operate under load. Assuring this is a | | | | Where you are adopting an SOA approach and are |
| demanding task and sadly the load testing and stress | | | | dependent on external services you need to be |
| testing here is often overlooked. | | | | certain that the throughput and turnaround time on |
| Better practice is to start the development of a large | | | | these services will remain acceptable as your system |
| scale software system with its performance clearly in | | | | scales and its demands increase. A smart system |
| mind, particularly scalability testing, volume testing and | | | | architecture will include a graceful response and |
| load testing. To create this performance testing | | | | fall-back operation should the external service |
| focus: | | | | behaviour deteriorate or fail. |
| | | | |