26 Software Testing – Black Box Testing

R. Baskaran

 

BLACK BOX TESTING

 

Black box testing is a software testing technique where the functionalities of the system is tested without comprehending or looking into the internal structure.

 

LEARNING OBJECTIVES 

 

• To execute a program with the intent of finding an error.

• To check if the system meets the requirements and be executed successfully in the Intended environment.

• To check if the system is “Fit for purpose”.

• To study on the various types of Black box Testing and its mode of operation.

 

MOTIVATION 

 

The motivation for performing testing process include as follows:

  • People are not perfect as we make errors in design and code.
  • Goal of testing is to uncover as many errors as possible.
  • Important and expensive activity:
    • May spend 30-40% of total project effort on testing.
    • For safety critical system cost of testing is several times higher than all other activities combined.

 

Ways of Thinking 

 

Design and coding are creative activities. Testing is destructive and the primary goal is to “break” the code to uncover errors and faults. Often same person does both coding and testing. So the person must need “split personality” i.e. when they start testing, must become paranoid and malicious. This is surprisingly difficult as people don’t like to find out that they made mistakes.

 

Testing Objective 

 

·  Testing: a process of executing software with the intent of finding errors.

·  Good testing: a high probability of finding as-yet-undiscovered errors.

·  Successful testing: discovers unknown errors.

 

Test Data and Test Cases 

 

Test-to-pass assures that the software minimally works and does not push the capabilities of the software. It applies simple and straightforward test cases, without trying to “break” the program.Test data is the inputs which have been devised to test the system. Test cases provide inputs to test the system and the predicted outputs from these inputs if the system operates according to its specification. Test case specifies the inputs; the pre-test state of the software and the expected results (outputs of the state).

 

Test-to-fail is designing and running test cases with the sole purpose of breaking the software. It comprises of strategically chosen test cases to probe for common weaknesses in the software.

 

BLACK-BOX TESTING 

 

The characteristics of black-box testing comprises of the following:

 

◦  Program is treated as a black box.

◦  Implementation details do not matter.

◦  Requires an end-user perspective.

◦  Criteria are not precise.

◦  Test planning can begin early.

 

 

Black box testing is also known as specification-based testing. Black box testing refers to test activities using specification-based testing methods and criteria to discover program errors based on program requirements and product specifications. The major testing focuses on the following:

  • Specification-based function errors
  • Specification-based component/system behavior errors
  • Specification-based performance errors
  • User-oriented usage errors
  • Black box interface errors

 

Black box testing comprises of testing software against a specification of its external behavior without knowledge of internal implementation details It can be applied to software “units” (e.g., classes) or to entire programs. The external behavior is defined in API docs, functional specs, requirements specs, etc. Black box testing purposely disregards the program’s control structure and the attention is focused primarily on the information domain (i.e., data that goes in, data that comes out). The goal is to derive sets of input conditions (test cases) that fully exercise the external functionality.

 

Information Domain: inputs and outputs 

 

Inputs encompass of individual input values and hence many different values are tried for each individual input. The combinations of inputs include:

  • Individual inputs that are not independent from each other.
  • Programs process multiple input values together, not just one at a time.
  • Try many different combinations of inputs in order to achieve good coverage of the input domain.

 

In addition to the particular combination of input values chosen, the ordering and timing of the inputs can also make a difference.

 

Defining the input domain 

  • Boolean value
    • T or F
  • Numeric value in a particular range
    • 99 <= N <= 99
    • Integer, Floating point

 

Equivalence Partitioning 

 

Typically the universe of all possible test cases is so large that you cannot try them all. You have to select a relatively small number of test cases to actually run, but which test cases to choose. Equivalence partitioning helps answer this question.

 

Partition the test cases into “equivalence classes”. Each equivalence class contains a set of “equivalent” test cases. Two test cases are considered to be equivalent if we expect the program to process them both in the same way (i.e., follow the same path through the code). If you expect the program to process two test cases in the same way, only test one of them, thus reducing the number of test cases you have to run.

 

Create a test case for at least one value from each equivalence class.

 

When designing test cases, you may use different definitions of “equivalence”, each of which will partition the test case space differently. Test multiple values in each equivalence class as many times you’re not sure if you have defined the equivalence classes correctly or completely, and testing multiple values in each class is more thorough than relying on a single value.

 

Example 1: int Add (n1, n2, n3…)

  • Equivalence Definition 1: partition test cases by the number of inputs (1, 2, 3, etc.).
  • Equivalence Definition 2: partition test cases by the number signs they contain (positive, negative, both).
  • Equivalence Definition 3: partition test cases by the magnitude of operands (large numbers, small numbers, both).

 

Example 2: string Fetch (URL)

  • Equivalence Definition 1: partition test cases by URL protocol (“http”, “https”, “ftp”, “file”, etc.).
  • Equivalence Definition 2: partition test cases by type of file being retrieved (HTML, GIF, JPEG, Plain Text, etc.).
  • Equivalence Definition 3: partition test cases by length of URL (very short, short, medium, long, very long, etc.).

 

Equivalence Partitioning – example

Boundary Value Analysis 

 

When choosing values from an equivalence class to test the program; use the values that are most likely to cause the program to fail. Errors tend to occur at the boundaries of equivalence classes rather than at the “center”

 

◦  If (200 < areaCode && areaCode < 999) { // valid area code }- Wrong!

◦  If (200 <= areaCode && areaCode <= 999) { // valid area code }

◦  Testing area codes 200 and 999 would catch this error, but a center value like 770 would not.

 

In addition to testing center values, we should also test boundary values i.e. right on a boundary and very close to a boundary on either side. Create test cases to test boundaries of equivalence classes.

Boundary Value Analysis- example

Web Links

  • https://www.tutorialspoint.com/software_testing/software_testing_types.htm
  • https://softwaretestingfundamentals.com
  • www.softwaretestinghelp.com/types-of-software-testing/
  • https://en.wikipedia.org/wiki/Black-box_testing

 

Supporting & Reference Materials

  • Roger S. Pressman, “Software Engineering: A Practitioner’s Approach”, Fifth Edition, McGraw Hill, 2001.
  • Pankaj Jalote, “An Integrated Approach to Software Engineering”, Second Edition, Narosa Publications, 2005.
  • Ian Sommerville, “Software Engineering”, TEnth Edition,Pearson education, 2017.