Outline
- What is a Software Bug?
- What causes Software Bugs?
- How do we measure Software reliability?
- How much do Software Bugs cost?
- How do we manage Software Bugs?
Background
This image shows the first bug that was discover in a computer; which was a real, physical bug.
Bugs are everywhere:
Bugs are present everywhere in code and hardware. For instance, in Intel Pentium. It has a Floating-Point Division bug where if you enter the following equation into the PC calculator, it will not return the correct answer:
(4195835 / 3145727) * 3145727 – 4195835
It is easy for our brain to calculate that the answer of this equation is 0. However, Intel Pentium does not return 0. This bug was resolved many years ago and it should not be present in modern computer.
What is a Software Bug?
A common term used to describe a flaw, mistake, or failure in a computer system that produces an incorrect or unexpected result or causes it to behave in unintended ways.
Everything we don’t like is a bug – is that helpful? We can also think of bugs as symptoms or clues – they show us that something about what we have built isn’t working how we think it should.
With this perspective, bugs are quite important to us because they can clue us in to much larger problems.
A software bug occurs when one or more of the following five rules is true:
- The software doesn’t do something that the product specification says it should do.
- The software does something that the product specification says it shouldn’t do.
- The software does something that the product specification doesn’t mention.
- The software doesn’t do something that the product specification doesn’t mention but should.
- The software is difficult to understand, hard to use, slow, or—in the software tester’s eyes—will be viewed by the end user as just plain not right.
What causes Software Bugs?
Bad Specifications:
The number one cause of software bugs is the specification, why?
- Unwritten or incomplete
- Ambiguous
- Constantly changing
- Not communicated to the development team
Design:
The next large source of bugs is design:
- Inappropriate modelling
- Lack of modelling tools
- Time to market pressure
Developers frequently don’t consider design an important step – this leads to inappropriate consideration of design choices and the alternatives.
Coding errors:
- Software complexity
- Poor documentation
- Limited time
- Programmer skills
Examples of code bugs:
Typical coding mistakes:
- Improper use of class library
- Inheritance implementation error
- Wrong message
- Wrong object
- Override error
- Data flow error
- Misuse local variable
- Scope boundary error
- Domain knowledge
- Algorithm error
- Need refactoring
- Missing operation
- Unnecessary code
- Syntax error
How do we measure Software reliability?
What is Software reliability?
The probability of failure-free software operation for a specified period in a specified environment. It denotes a product’s reliability and impacts customer satisfaction or retention. For example in:
- Banking – unreliable behaviour in banks is not ideal!
- Video games – ever tried to play a multiplayer game and the servers were down?
- Healthcare – obviously, safety critical systems!
Software’s unreliability is not caused by aging but is due to bugs. The more bugs, the less reliable the software is likely to be!
However, failures seem random, hence Bayesian (probabilistic) theories over time are applied:
- R(t) – the probability of software still functioning at a given point in time (t)
- Availability/up-time – what percentage of the time will a service be available (particularly useful in online services)
- Mean time between failures (MTBF) – how long, on average, will you go before the system fails again
How much do bugs cost?
The cost of bugs grows logarithmically as they go uncaught. Most forms of testing only find about 1 bug out of every 3. And all tests together barely remove 85% of bugs during testing. Even the best companies and organizations have released products with expensive (but sometimes simple to fix) bugs.
How do we manage Software Bugs?
Bug tracking workflows:
- A tester finds a bug and reports it.
- The bug is assigned to a manager for initial triage.
- The manager resolves it or assigns it to a developer.
- Resolved bug is returned to a tester.
- The tester either closes the bug or reopens it with additional information or comments which start step 2 again.
Users and developers report bugs, to allow software developers to fix them. Bug reports are added to an issue tracking system or bug repository (Apache Bloodhound, Bugzilla, Jira, Trac, etc.)
What does Bug triaging mean?
Assigning a bug to the most appropriate/capable developer to address it.
- The triager analyzes, and assigns bug fixers for each new bug prioritising based on seriousness
Effective bug triage needs:
- Knowledge about the system/project.
- Descriptive bug reports.
- Good rates of reporting of bugs.
- Coordination with developers on different projects and with various specialties.
Software testers goals:
To find bugs as early in the software development processes as possible and make sure they get fixed.
Advice: Be careful not to get caught in the dangerous spiral of unattainable perfection:
You know you are finished when the only bugs left are the ones that you decide you can live with at least for now!
– Robert Sabourin, I am a Bug
Conclusion:
- What is a Bug? – Everything we don’t like is a bug
- What causes Bugs? – Code, design and spec issues.
- How do we measure Software reliability? – Mean Time Between Failure (MTBF)
- How much do bugs cost? – It depends when we find them.
- How do we manage bugs? – With bug tracking and a carefree outlook.
