23 Fundamentals of Software Testing

R. Baskaran

 

FUNDAMENTALS OF SOFTWARE TESTING 

 

Software testing is the evaluation of a system with the intention of finding an error or fault or a bug. It also checks for the functionalities of the system so that it meets the specified requirements.

 

LEARNING OBJECTIVES 

 

•  To execute a program with the intent of finding an error.

•  To check if the system meets the requirements and be executed successfully in the intended environment.

•  To check if the system is “Fit for purpose”.

•  To check if the system does what it is expected to do.

 

COMPUTER BUG 

 

In 1947, Harvard University was operating a room-sized computer called the Mark II. It encompassed mechanical relays and glowing vacuum tubes. The technicians program the computer by reconfiguring it and they had to change the occasional vacuum tube. A moth flew into the computer and was zapped by the high voltage when it landed on a relay. Bugs can also be known as:

 

·  Defect

·  Fault

·  Problem

·  Error

·  Incident

·  Anomaly

·  Variance

·  Failure

·  Inconsistency

·  Product Anomaly

·  Product Incidence

·  Feature

 

The programs that are developed contain defects which are hard to predict in the future, however it is highly likely, that the software we will develop in the future will not be significantly better.

 

Sources of Problems 

 

The problems can arise from various sources such as:

 

·  Requirements Definition: Erroneous, incomplete, inconsistent requirements.

·  Design: Fundamental design flaws in the software.

·  Implementation: Mistakes in chip fabrication, wiring, programming faults, malicious code.

·  Support  Systems: Poor programming languages, faulty compilers and debuggers, misleading development tools.

·  Inadequate Testing of Software: Incomplete testing, poor verification, mistakes in debugging.

·  Evolution: Sloppy redevelopment or maintenance, introduction of new flaws in attempts to fix old flaws, incremental escalation to inordinate complexity.

 

Adverse Effects of Faulty Software 

 

Faulty software causes many adverse effects such as:

 

·  Money Management: Fraud, violation of privacy, shutdown of stock exchanges and banks, negative interest rates.

·  Control of Elections: Wrong results (intentional or non-intentional).

·  Control of Jails: Technology-aided escape attempts and successes, accidental release of inmates, failures in software controlled locks.

·  Law Enforcement:  False arrests and imprisonments.

 

Project Mercury’s FORTRAN code had the following fault: DO I=1.10 instead of … DO I=1…10. The fault was discovered in an analysis of why the software did not seem to generate results that were sufficiently accurate. The erroneous 1.10 would cause the loop to be executed exactly once.

 

Military Aviation Problems 

 

An F-18 crashed because of a missing exception condition: if … then … without the else clause that was thought could not possibly arise. In simulation, an F-16 program bug caused the virtual plane to flip over whenever it crossed the equator, as a result of a missing minus sign to indicate south latitude.

 

Year Ambiguities 

 

In 1992, Mary Bandar received an invitation to attend a kindergarten in Winona, Minnesota, along with others born in ’88. Mary was 104 years old at the time. Mr. Blodgett’s auto insurance rate tripled when he turned 101. He was the computer program’s first driver over 100, and his age was interpreted as 1. This is a double blunder because the program’s definition of a teenager is someone under 20!

 

AT&T Bug 

 

In mid-December 1989, AT&T installed new software in 114 electronic switching systems. On January 15, 1990, 5 million calls were blocked during a 9 hour period nationwide. The bug was traced to a C program that contained a break statement within an switch clause nested within a loop. The switch clause was part of a loop. Initially, the loop contained only if clauses with break statements to exit the loop. When the control logic became complicated, a switch clause was added to improve the readability of the code.

 

A Norwegian bank ATM consistently dispersed 10 times the amount required. Many people joyously joined the queues as the word spread. A software flaw caused a UK bank to duplicate every transfer payment request for half an hour. The bank lost 2 billion British pounds. The bank eventually recovered the funds but lost half a million pounds in potential interest.

 

Making Rupee 

 

An Australian man purchased $104,500 worth of Sri Lankan Rupees. The next day he sold the Rupees to another bank for $440,258. The first bank’s software had displayed a bogus exchange rate in the Rupee position. A judge ruled that the man had acted without intended fraud and could keep the extra $335,758.

 

Bug in BoNY Software 

 

The Bank of New York (BoNY) had a $32 billion overdraft as the result of a 16-bit integer counter that went unchecked. BoNY was unable to process the incoming credits from security transfers, while the NY Federal Reserve automatically debited BoNY’s cash account.

 

SPECIFICATION 

 

You have to know what your product is before you can say if it has a bug -“if you can’t say it, you can’t do it”. A specification defines the product being created and includes the functional requirements that describe the features the product will support (E.g. on a word processor save, print, check spelling, change font etc) and non-functional requirements that focuses on the constraints of the product (E.g. Security, reliability, user friendliness, platform etc).

 

SOFTWARE BUGS 

 

A software bug occurs when at least one of these rules is true:

 

◦  The software does not do something that the specification says it should do.

◦  The software does something that the specification says it should not do.

◦  The software does something that the specification does not mention.

◦  The software does not do something that the product specification does not mention but should.

◦  The software is difficult to understand, hard to use or slow.

 

Most bugs are not because of mistakes in the code, the estimation of the bugs uncovered at various phases is as follows:

  • Specification (~= 55%)
  • Design (~= 25%)
  • Code (~= 15%)
  • Other (~= 5%)

 

Relative cost of bugs 

 

The bugs that are found at later phases cost more to fix. Cost to fix a bug increases exponentially (10x) i.e. it increases tenfold as time increases. E.g. a bug found during specification costs $1 to fix. If the bug is found in design cost is $10, if found in code cost is $100 and if found in released software cost is $1000. The cost keeps increasing tenfold as the time increases.

 

Bug Free Software 

 

Software is in the news for the wrong reason such as security breach, Mars Lander lost, hackers getting credit card information, etc. The development of a bug free software is difficult when software gets more features and supports more platforms, it becomes increasingly difficult to make it create bug-free. The common questions that arise with the development of a bug free software are:

 

◦  Why can’t software engineers develop software that just works?

◦  Do you think bug free software is unattainable?

◦  Are their technical barriers that make this impossible?

◦  Is it just a question of time before we can do this?

◦  Are we missing technology or processes?

 

GOAL OF A SOFTWARE TESTER

 

The goals of a software tester is to find bugs as early in the software development processes as possible and make sure they get fixed. The software tester must be careful not to get caught in the dangerous spiral of unattainable perfection. The expectation on a software tester involves checking the following:

  • Are they explorers?
  • Are they troubleshooters?
  • Are they relentless?
  • Are they creative?
  • Are they perfectionists (within reason)?
  • Do they exercise good judgment?
  • Are they tactful and diplomatic?
  • Are they persuasive?

 

TESTING

 

Testing is the confirming of correct behavior of a piece of software with respect to this context. Software testing is the evaluation of a system with the intention of finding an error or fault or a bug. It also checks for the functionalities of the system so that it meets the specified requirements. We know what we intended the code to do – we need to confirm that it does what we intended. Testing involves conducting test to ensure the production function correctly, but the test conducted must be looked into to check whether the tests are testing the right (relevant) “stuff”. The test is testing “most” of the code and not just repeatedly testing the same code.

 

Good Testing Practices

 

A good test case is one that has a high probability of detecting an undiscovered defect, not one that shows that the program works correctly. It is impossible to test your own program. A necessary part of every test case is a description of the expected result. Non-reproducible or on-the-fly testing is avoided. The test cases for valid as well as invalid input conditions are written and the results of each test are thoroughly inspected.

 

Web Links

  • https://www.tutorialspoint.com/software_testing/software_testing_types.htm
  • https://softwaretestingfundamentals.com
  • www.softwaretestinghelp.com/types-of-software-testing/

 

Supporting & Reference Materials

  • Roger S. Pressman, “Software Engineering: A Practitioner’s Approach”, Fifth Edition, McGraw Hill, 2017.
  • PankajJalote, “An Integrated Approach to Software Engineering”, Second Edition, Springer Verlag, 1997.
  • Ian Sommerville, “Software Engineering”, Sixth Edition, Addison Wesley, 2015.