White Box Testing Defined
What IS White Box testing exactly? Well, white box testing – also known as glass box testing, code-based testing, transparent box testing, open box testing, or clear box testing – is a testing process which is so called because it involves the tester ‘seeing through’ the outer covering of the software/application, and working on the inner structure. Knowledge of the internal configuration of the programme, including aspects such as the source code, network protocols and IP addresses, are all required in order to carry out the tests. The overall purpose of a white box test is for the internal structure of the system in question to be evaluated in depth from the perspective of a developer / software programmer.
In general terms, white box testing can be broken down into three practical steps. These are; the preparation for the testing, the creation and execution of the actual tests and the production of a report setting out the results.
Preparation
Preparing to carry out white box testing involves becoming fully conversant with the internal operation of the application being tested. In-depth knowledge and understanding of the functionalities which drive the application is necessary if the tester is to be able to create tests which will genuinely be capable of discovering structural faults and loopholes within the software.
Part of the preparation process involves gaining an understanding of the source code of the application, including the programming language used. In addition, the tester must bring an awareness of secure coding practices to the creation of tests as testing for security loopholes is often one of the prime objectives of running white box testing.
Testing
Once the preparation phase has been completed and a full understanding of the source code developed, the tester will use a range of testing methods to check for anomalies, and establish a coherent flow and structure. Manual tests might involve writing more code to test the existing code, while testing tools can be used to automate the testing process. In general terms, all white box testing techniques involve providing test inputs and assessing the outputs generated.
Specific Tests
This part of the White Box testing process is the most in depth and lengthy. Testing the entirety of the source code of an application involves working through the various levels of said code, testing individual items and then testing the way in which these items interact with one another. The specific levels of testing include the following:
- Unit testing
Unit testing involves testing the individual components and units of the source code in order to ascertain whether these units act as intended. Running tests of this kind are extremely useful for identifying faults and anomalies which happened earlier in the development life cycle. Discovering a fault or defect at this stage, in the testing process, generally means it will be easier and less expensive to fix and prevent from occurring again in the future.
- Integration testing
Once testing the individual units of the source code has been done, integration testing involves combining those units and components and then testing how they operate as a whole. Testing of this kind identifies any faults or issues in the interactions between the units and interfaces over which these interactions take place.
- Regression testing
Regression testing takes place after the previous 2 stages in order to evaluate the changes made in the coding of the software following the unit and integration testing. The unit and integration testing carried out will often have resulted in the code having had bugs fixed, security features added, loopholes closed and any other changes identified as being required carried out. Regression testing involves rerunning the tests already carried out in order to confirm that the features already tested are still operating as desired in the aftermath of any changes.
- Code coverage analysis
No matter how thorough a tester feels they have been in carrying out unit, integration and regression testing, it is still likely that there are gaps in the code which haven’t been tested or uncovered. Code coverage analysis is intended to spot and eliminate those gaps.
Once gaps have been identified, the following testing techniques can be used to ensure that areas of code which were missed by the major testing cases – unit, integration and regression testing – have been identified and tested:
Statement Coverage
Using this technique means that every statement in the code has been tested at least once
Branch Coverage
This technique ensures that every possible path in the software has been tested
Function Coverage
This uses a formula which calculates the number of defined functions covered by the parameters of the main testing. The tester can also provide inputs designed to assess if the outputs, and therefore the operation of the functions, are as expected and desired.
Although the details of white box testing can seem extremely complex, and it’s for this reason why developers conducting the tests generally do not go into enough detail due to their commitments to getting the software to market. The technical complexity involved masks a relatively simple underlying framework. In basic terms, white box testing is designed to verify the work flow of the software by testing predetermined inputs against the desired outputs. In those cases where the input doesn’t produce the output which the developer expects, then a bug in the software has been uncovered.
White box testing should be an on-going process carried out as the software is being developed, after the code is written, and before being repeated following each subsequent modification.
Although the use of Statement, Branch and Function Coverage should ensure that 80% to 90% of the code in question has been tested, there are still other techniques which can be utilised to push further toward 100% coverage:
- Decision Coverage
- Condition Coverage
- Multiple Condition Coverage
- Finite State Machine Coverage
- Path Coverage
- Control flow testing
- Data flow testing
Various automated tools are available for carrying out white box testing, and some of the most popular are:
- Veracode – A suite of tools designed to identify and resolve software flaws. It supports a range of application languages such as .NET, C++ and JAVA, and can be used to test the security of mobile, desktop and web applications.
- Junit – A testing tool which works with the JAVA programming language
- HtmlUnit – A JAVA-based headless browser which is mainly used for running integration tests on web-based applications after JUnit has been used to run unit testing.
- PyUnit – A unit testing tool designed for use with software which uses the Python programming language.
- Cfix – Cfix is a unit testing framework created for C/C++ in order to make testing as simple and quick as possible.
Once all the testing has been completed, the tester should produce a report which sets out the results in an easily comprehensible format, detailing the specific tests carried out and listing the outputs derived from the various test inputs. The creation of a test report will enable the testing process itself to be analysed and maximised for efficiency, while providing a concrete document to be referred to in the future.
The pros and cons of white box testing
The pros
- White box testing is deep and thorough process which analyses the software being used at the fundamental level of coding
- It can identify unknown security issues
- The availability of automated white box testing tools speeds the process up and cuts costs
- The involvement of developers in white box testing means that issues can be quickly fixed without having to liaise with a quality assurance department
- Working through the code line by line enables developers – at the same time as spotting issues and problems – to optimise operations by removing or condensing superfluous code.
The cons
- The more complex the software being tested is, the longer and more expensive the process will be. Unit testing in depth can reduce the timescale somewhat, but is an expensive process on its own terms.
- White box testing requires the input of skilled technicians with in-depth knowledge of programming and languages used. Within a company, this might mean developers being pulled away from more direct revenue generating activity.
- Automated testing can save time but may have to be repeated on multiple occasions if the code in question is repeatedly updated and modified.
- The time and expertise required for comprehensive white box testing to take place renders it an expensive option.
- The tests are not carried out from the perspective of a user, which may limit the degree in which they can be applied in real world cases
Alternatives to white box testing
Black box testing and grey box testing are other alternatives. Black box testing is focussed solely on the functionality of an application, rather than its internal coding. In this way, black box testing more closely resembles the way in which the application or software will be approached by the end user, with no knowledge of or interest in the internal workings. The focus in black box testing is entirely on the inputs and outputs and how the software behaves. In between the two extremes of black box and white box testing sits grey box testing, which combines the two. In grey box testing there will be a limited understanding of the internal coding of the software being tested, and the tests carried out will focus on discovering faults in both the code and the functionality.
With thorough, proactive testing you can ensure that your software, systems and programmes are as secure and usable as possible, while uncovering bugs and gaps where external threats could infiltrate. This forms part of a solid Cyber Security strategy, ensuring your business is as secure as possible.
