Microsoft Word Hardware Reconfiguration Methodology V final2

Benchmarking

Before deploying any successful changes made to the C2 test systems onto the operational network, it is important to consider the performance-based issues that can inadvertently affect operating systems. One way to determine if a set of “successful” modifications will cause inadvertent changes on the operational network is to test them in some way. The standard method

of testing consists of examining application and service-based functionality as well as usability tests. Another method that is proposed herein is benchmarking.

Benchmarking is a performance-based test that measures the performance of the system, application, or service against some measurable unit. The most commonly used unit is time, although it any other useful performance-based unit is permissible. Benchmarking is useful because it can help to pinpoint slowdowns caused by new software or a set of modifications. It verifies if performance (could be the system as a whole or an application or service) is similar to before implemented changes. By benchmarking the system when important modifications are made to it various side effects, adverse or otherwise can be made known. Modifications can be tested individually or in groupings of likeminded changes. Furthermore, it possible to benchmark software according to changes in both the software itself and their configurations.

By benchmarking different modifications and attributing a performance-based score to pre- and post-modification systems it is possible to determine with good accuracy whether a given change or set of changes has been successful. A general rule of thumb is proposed: if the performance of a system, application, etc., is closely similar to the original then the change(s) can be considered successful. Conversely, noticeable slowdowns can be indicative of problems requiring resolution or simply of an inadequate or incompatible change or set of changes having been made.

The theory behind benchmarking is that non-effectual or counterproductive changes and modifications are more likely to cause system, application, and service slowdowns. Thus, if a set of modifications actually causes an unexpected and significant slowdown where none was previously seen then it is likely that those changes are either incorrectly set or are detrimental to the system and should be reversed.

However, in order for benchmarks to be useful, a base score is required and this can only be accomplished by benchmarking the original test systems C2 systems marked for deployment. The system, as well as key applications and services should be benchmarked and serve as a comparison for future benchmarks. In addition, benchmark results can vary widely due to extraneous factors; thus benchmarks should be repeated several times in order to average out the result. In so doing, benchmarking can help to objectively pinpoint potential performance gains and problems that are the result of one or more modifications.

5.