Analysis of FindBugs: 2013

We are back!!
Over the past few weeks, amidst our crazy schedule of exams and assignments, we were honing our skills and strengthening our fingers to give you a far more detailed analysis of our tool, FindBugs.
In this post, we intend to explain, in some detail, how FindBugs tool actually works and how effective it actually is in detecting possible errors in the code.
We have divided the post into 3 parts.The First Part explains how FindBugs actually works.The Second Part gives a brief introduction to the project on which we will be running FindBugs on to set the context and in the Third Part we will discuss the kind of bugs that FindBugs detected.
So, lets get started.
1.How FindBugs Tool detects errors
As discussed before, FindBugs actually analysis the generated Java bytecode and not the Java Source code.This is far more effective than running static analysis on the Source code because more often than not, source codes are subjected to compiler optimizations and this may lead to the creation of low level object files that are slightly different from the source files in executing paths.By analyzing the bytecode directly, FindBugs ensures that the analysis is happening on the actual mirror image of the to be generated executable file.
FindBugs uses two techniques to perform static analysis:
1.Syntactic Pattern matching technique: FindBugs linearly scans bytecode of java class files and searches for suspicious code patterns.By doing so, for every class file, FindBugs creates a state for each class.This in turn helps FindBugs to extend syntactic pattern matching to intraprocedural pattern matching to detect state changes in a class after a class has interacted with other classes.
2.Data Flow Analysis(DFA):Along with syntactic pattern matching, FindBugs uses programs Control Flow Graphs(CFG) to determine the possible execution paths of a program.The goal of this method is to
gather information about the possible set of values calculated at different points of program code. With the help of CFG, data flow analysis simulates application’s execution paths without actually executing the
code.With the help of DFA, FindBugs can ascertain the possible values a variable can have at different instances of program flow.For example, using DFA, FindBugs can ascertain that the variable 's' can be NULL and raises a red flag:
String s= getString();
if (s == null && s.isEmpty()) {
return;
}
So, now you know what techniques FindBugs uses to detect errors.Lets move on to the second part.
2.The sample project.
We want to give a very brief introduction about the sample project on which we want to run FndBugs on.
We think that this will help set the context for further references.We have chosen our sample project very carefully.Lets face it.To give you a demo of the findings of FindBugs, the code must have some bugs to begin with :)Therefore, if we go about choosing a very popular open source project, the chances of finding bugs in them will be very less.At the same time, if we choose a really simple project, we cannot analysis a lot(if not all)of the variety of errors FindBugs can identify.So, we choose the "Email Extractor" project.It is a decent, popular, mid sized Java project. It can be downloaded here.
Extractor works in Java and on any Operating System.Just type a valid http address and hit enter and the extractor will extract email addresses while visiting the link.To run this program, just download the file and execute it.It has a .jar extension but it will run if you have java installed.
We have another sample project on which we have run FindBugs. We will speak more about in in our screen cast and podcast.For the purpose of this blog post, we will be using only the "Email Extractor" project and its code.
Now that we have introduced you to the project and its source code, lets run FindBugs on it using the FindBugs Eclipse plugin and discuss what we found.
3.What we found
Our intent is to not just show the bugs that FindBugs tool detected.Since you already know where to find the source code and how to install FindBugs(which we had covered in our previous post), we suspect you don't need our help in just running to tool to see the results yourself. What we want to do, is to give you a brief analysis of FindBugs by testing its capabilities by twisting and tweaking the code and finding out the end results.
We will show you a few of our analysis here.
ANALYSIS 1:
We ran FindBugs on the class called Download.java.In the Bug Explorer, FindBugs said it detected a bug as follows:
public class Download {

public static String getPage(URL page) {
String output = "";
URLConnection openConnection = null;
try {
openConnection = page.openConnection();
} catch (IOException ex) {
System.err.println(ex);
}
BufferedReader in = null;
try {
*** in = new BufferedReader(new InputStreamReader(openConnection.getInputStream()));
} catch (IOException ex) {
System.err.println(ex);
}
String input = "";
if (in != null) {
try {
while ((input = in.readLine()) != null) {
output = output + input + "\r\n";
}
} catch (IOException ex) {
System.err.println(ex);
}
try {
in.close();
} catch (IOException ex) {
System.err.println(ex);
}
}
return output;
}
}

In the above code, FindBugs reported an error saysing that there is a possible NullPointer Dereferencing on line marked as "***".After performing Data Flow Analysis, FindBugs found that if the reference variable openConnection had a NULL value, then dereferencing it on line "***" could generate a NullPointerException.
This was a good catch!!.
But we wanted to stretch this test a little further.We know that all exceptions in Java inherit from the Exception Base class.So, instead of catching the IOException, we changed the code to catch a generic Exception as below:
try {
*** in = new BufferedReader(new InputStreamReader(openConnection.getInputStream()));
} catch (Exception ex) {
System.err.println(ex);
}
Now, that we are catching any possible Exception including the possible NullPointerException, upon running FindBugs again, ideally, FindBugs should not detect possible NullPointer dereferencing Error.
Viola!!
It didn't. This shows that by performing Data Flow Analysis,FindBugs determined that even though the variable openConenction could have a NULL value, it was surrounded by a relevant try/catch block so all was OK.
In Static Analysis parlance, the above error that FindBugs caught is called "True Positive".
If a static analysis tool produces a warning which does not indicate a real defect, this warning is considered to be a false positive (FP). Warnings revealing real errors are, on the contrary, true positives (TP).
Sometimes also terms false negative (FN) and true negative (TN) are used.The term false negative refers to unwanted cases where a tool does not report a warning for a real defect. True negative is a less commonly used term which means a desired case where a tool reports no warning because there is no defect.
ANALYSIS 2
This time, we wanted to compare the type of warnings given by Eclipse versus the warnings alerted by FindBugs. We used two metrics for comparisons.
1. Relevance
2. Readablilty
The file MT_Processor.java of our sample project uses Java Reflections technique to access a method of a class.
public class MT_Processor implements Runnable {

public static DefaultListModel threads = new DefaultListModel();
public static boolean processing = false;
public static boolean deactivate = false;
public static Thread th = null;

@Override
public void run() {
(****) th = new Thread(this);
if (threads != null) {
for (int i = 0; i < threads.size(); i++) {
processing = true;
String currentthread[] = threads.getElementAt(i).toString().trim().split(",");
try {
(**) Class c = Class.forName(currentthread[1]);
Method currentmethod = c.getMethod(currentthread[0], (Class[]) null);
currentmethod.invoke(null, (Object[]) null);
} catch (ClassNotFoundException ex) {
Logger.getLogger(MT_Processor.class.getName()).log(Level.SEVERE, null, ex);
}
}
}
}
Eclipse popped a warning on line (**) saying "Class is a raw type. References to generic type Class<T> should be parameterized".FindBugs did not.Upon further investigation we found out that even though "Class" is a raw type, since it was being used to retrieve a method from a particular class at run time, it was impossible to ascertain the class name at compile time.So this warning that was issued by Eclipse was not relevant.
FindBugs,by using Data Flow Analysis Technique, determined the "context" in which the raw type "class" was being used and therefore did not issue any warning.
This shows that the DFA technique used by FindBugs is far more sophisticated than that of the inbuilt static analysis tool present in Eclipse.
Furthermore, there was another warning detected by FindBugs that was skipped by Eclipse at line marked(****).Here, the variable "th" which was declared a static in the class definition was being assigned a value in a non-static instance method called "run()".Though this is not a bug in the strict sense, if there was a static method that tried to access the above variable before it was initialized by the non-static method, the results would be wrong.
FindBugs detected this flaw and flagged it as a "warning" and not a potential bug.This result strongly speaks for the FindBugs ability to detect non local dependencies in code and its ability to categorize it as a warning and not a possible Bug.

In this blog post we share a couple of our analysis with you.We did not explore the cases where FindBugs could generate False Positives(even though we told you what False Positives mean).In out Podcast and screen cast we will continue to explore these areas in detail on another project.So stay tuned :)

FindBugs is a static analysis tool for Java ( It can be downloaded here.)

Static analysis tools ascertains possible errors in the Source code of the program without actually executing the program.This helps developers to test parts of the software that they have coded without having to wait for all the modules of the software to be integrated together to run tests that expose faults.

There are several static analysis tools available for Java (see list here).So what makes FindBugs so special?Well, one of the factors that sets FindBugs apart from other static analysis tools is that the number of False Positives reported in the Source code by FindBugs is minimal.
In the paper, “Finding More Null Pointer Bugs, But Not Too Many” written by David Hovemeyer and Willian Pugh, the writers explain new techniques such as :

1. Suppressing warnings inside try/catch blocks- If a lines that could cause potential errors(such as NullPointerException in Java) is already handled by enclosing them in a try-catch block, then FindBugs does not report such lines as potential fault points.

2. Better handling of assertions- If assert statements are already included in the code and assertions are enabled then FindBugs does not generate warnings for the same.

3. Better handling of panic methods calls- If panic Methods are already coded to handle possible errors, FindBugs does not generate warnings to identify the same thus making its reports less verbose.

Use of above techniques in FindBugs makes it far more efficient than other static analysis tools( The above mentioned paper compares Null Pointer Analysis of FindBugs with another static analysis tool called Reasoning on Tomcat 4.1.24.It can be downloaded here).

Furthermore, comparative studies made by Peng Li and Baojiang Cui in their paper “A comparative study on software vulnerability static analysis techniques and tools”, at the Information Theory and Information Security (ICITIS), 2010 IEEE International Conference suggest that by using techniques such as Lexical Analysis –which divides the entire program into small fragments and verify the structure for each of these fragments against predetermined rules( such as Grammars as done by compilers) and Data Flow Analysis –a technique that collects semantic information from program code to determine whether a value in a variable is assigned to a possible vulnerability of the corresponding variable (such as a Null value in a variable that holds Object references in Java), FindBugs has the ability to report far more pitfalls and errors in the source code than many other Static analysis tools for Java.

In days to come, we will discuss how we tested FindBugs on a portion of an Open Source software and we will share our results and findings.

Analysis of FindBugs

Saturday, November 30, 2013

Final Screencast

Saturday, November 16, 2013

Final Podcast on FindBugs

Monday, November 4, 2013

FindBugs: This time a more detailed analysis!!

Monday, October 7, 2013

ScreenCast on FindBugs

Podcast

What is FindBugs?