The Art of Bug Hunting

I was reading an article in The Economists. It goes on the following note:

ON SEPTEMBER 14th2004, the radios in an air-traffic control centre in Palmdale, California shut down, grounding hundreds of flights in southern California and Nevada, and leading to five mid-air encounters between aircraft unable to talk to the ground controllers. Disaster was averted because aircraft managed to communicate with more distant back-up facilities. But why did Palmdale’s radios fail? A glitch in the software running the system meant the computers had to be re-booted every 30 days, and somebody forgot to do so. But software running a mission-critical system should not have to be restarted every month. The culprit: poor design and poorly tested software.

It indicates that current civilization banks on software, which needs to be reliable. However, humans are producing software. They can make mistakes. The software written by them is likely to crash, infested with bugs and with security holes.

What’s solution for this? Oddly enough, but to use even more software. Programmers are increasingly calling upon bug detecting tools for their help. These tools help them to detect-and-squash bugs. Programmers use these tools at various stages in the software development cycle.

Some of these tools help software developers/programmers to work along while developing, e.g., a handy tool like Debugger which is ubiquitously present in the IDE(Integrated Development Environment).   Other tools analyze the resulting software, or it’s under the hood code, for problems.  It makes the project manager aware by putting numbers on code quality, programmer productivity, and the cost-effectiveness of fixing particular bugs, so they can decide which bugs ought to be fixed.

Even though the top-down approach helps the project manager with indices of code quality, productivity, etc.  One can’t undermine the importance of low-level tools which allows programmers to comb through his code and detect bugs, conflicts, security holes, and other potential problems.

Such tools are now proliferating. These can categorize into two types: dynamic-analysis tools, which examine software as it runs to work out where breakdowns happen, and static-analysis tools, which look at code without actually running it to look for fundamental flaws.

To use an analogy, dynamic analysis is like watching a machine in operation, whereas static analysis is like poring over its blueprints. “Dynamic-analysis tools say, ‘Well, you’ve got a problem on something over here’, “Static-analysis tools say, ‘You’ve got a problem on line 123’.” The two types are complementary, most firms, and uses both.

Static analysis is a handy tool to the programmer. It helps him to identify a bug earlier. It is cheaper to fix. (A rule of thumb is that a bug which costs less to fix on the programmer’s desktop costs 100 times more  to fix once it gets into a build, and thousand times more if it is identified only after the software release.)

A static-analysis tool runs on a programmer’s desktop every time code is submitted for a build. The advantage of this approach is that programmers do not need to wait for a build to test their system. And when a whole team uses, it can spot potential conflicts between codes written by different programmers.

The dynamic analysis runs a code with a variety of test inputs for different test scenarios. It analyzes performance and validates whether it behaves as expected.  It ensures that it does not crash, goes into an endless loop or demands more than required main memory as it runs. The process can be automated by writing the test scripts.

Though static and dynamic analysis has been around for a while, it hardly encouraged more programmers to use them. It’s a challenge. The solution can be to use online service for code testing. Such service can widen the market for software testing by smaller companies as well.

 Integrated approach and Priority settings

Another approach to above challenges to integrate testing tools closely with existing programming tools.

One has to keep in his mind about the cost of searching and fixing all bugs. Not all bugs are worth fixing—but how can programmers decide which ones to concentrate on?  It helps a developer to get bugs and the estimated risk associated with each. It can help them to take a decision.

A risk-analysis approach is applied to software on a larger scale through efforts to develop metrics for code quality and programmer productivity. Atlassian, an Australian developer of software tools, Bamboo, which tracks trends in code over time, such as the number of bugs found.

Veracode’s, a firm based in Burlington, Massachusetts, provides analysis service which has a code-scoring tool that gives grades to code. It is taken over by Computer Associates (CA).

Keep an eye on…

It is likely that one can get overwhelmed by the information after searching on the net for the best tool available today for testing.  It is difficult to find a correct tool which can meet your different testing demands.

A few months back I have gone through the write-up on QASymphony’s site. It did cover the most of the contemporary tools.  Personally, I keep reading the newer development in this field by going through various articles, blogs on the net. I make it a point to have aperiodic visit to sites like IBM, Computer Associates (CA), etc.  Tools with (Acknowledgement: Figures, incidence reference, etc. used in this article is from The Economists and the net).

About the Author:

Atul Gunjal is Co-founder and CEO of IWORKTECH. It is Alpharetta, GA based company. It has development centre in Pune, India. IWORKTECH develops solutions and products in the areas like Utility, Healthcare and Education. It has wide experience in providing Enterprise Wide solution. Atul earned his Bachelor of Engineering in Electrical from the University of Pune, India. He directs IWORKTECH’s overall vision and business strategy, including expansion into new markets, high-level technology direction and operations. His previous experience helps him to convert ideas into tangible products. He got rich experience in building successful software and service organizations.