Hi Everyone,
Happy thanksgiving you all, and here's the final episode in our multimedia series for the thorough introduction of FindBugs:
http://www.youtube.com/watch?v=pQwD_Mf7F6M
References:
http://findbugs.sourceforge.net/manual/
Email Extactor:
http://sourceforge.net/projects/simpleextractor/?source=navbar
Colony:
http://www.freecol.org/download.html
FindBugs Eclipse plugin:
http://findbugs.cs.umd.edu/eclipse
Saturday, November 30, 2013
Saturday, November 16, 2013
Final Podcast on FindBugs
Here is the link to our final Podcast on FindBugs tool.
Citations for Research papers referred to in the Podcast:
1. S. Wagner, F. Deissenboeck, M. Aichner, J. Wimmer, andM. Schwalb. An evaluation of two bug pattern tools for java. In Software Testing, Verification, and Validation, 2008 1st International Conference, pages 248–257, 2008
2. Kim and Ernst(2007)] Kim, S., Ernst, M. D., 2007.Prioritizing warning categories by analyzing software history. In: MSR ’07: Proceedings of the FourthInternational Workshop on Mining SoftwareRepositories. IEEE Computer Society, Washington,DC, USA, p. 27
3. Brittany Johnson, Yoonki Song, Emerson Murphy-Hill, and Robert Bowdidge, “Why Don't Software Developers Use Static Analysis Tools to Find Bugs?”, Accepted to the 2013 International Conference on Software Engineering.
Citations for Research papers referred to in the Podcast:
1. S. Wagner, F. Deissenboeck, M. Aichner, J. Wimmer, andM. Schwalb. An evaluation of two bug pattern tools for java. In Software Testing, Verification, and Validation, 2008 1st International Conference, pages 248–257, 2008
2. Kim and Ernst(2007)] Kim, S., Ernst, M. D., 2007.Prioritizing warning categories by analyzing software history. In: MSR ’07: Proceedings of the FourthInternational Workshop on Mining SoftwareRepositories. IEEE Computer Society, Washington,DC, USA, p. 27
3. Brittany Johnson, Yoonki Song, Emerson Murphy-Hill, and Robert Bowdidge, “Why Don't Software Developers Use Static Analysis Tools to Find Bugs?”, Accepted to the 2013 International Conference on Software Engineering.
Monday, November 4, 2013
FindBugs: This time a more detailed analysis!!
We are back!!
Over the past few weeks, amidst our crazy schedule of exams and assignments, we were honing our skills and strengthening our fingers to give you a far more detailed analysis of our tool, FindBugs.
In this post, we intend to explain, in some detail, how FindBugs tool actually works and how effective it actually is in detecting possible errors in the code.
We have divided the post into 3 parts.The First Part explains how FindBugs actually works.The Second Part gives a brief introduction to the project on which we will be running FindBugs on to set the context and in the Third Part we will discuss the kind of bugs that FindBugs detected.
So, lets get started.
1.How FindBugs Tool detects errors
As discussed before, FindBugs actually analysis the generated Java bytecode and not the Java Source code.This is far more effective than running static analysis on the Source code because more often than not, source codes are subjected to compiler optimizations and this may lead to the creation of low level object files that are slightly different from the source files in executing paths.By analyzing the bytecode directly, FindBugs ensures that the analysis is happening on the actual mirror image of the to be generated executable file.
FindBugs uses two techniques to perform static analysis:
1.Syntactic Pattern matching technique: FindBugs linearly scans bytecode of java class files and searches for suspicious code patterns.By doing so, for every class file, FindBugs creates a state for each class.This in turn helps FindBugs to extend syntactic pattern matching to intraprocedural pattern matching to detect state changes in a class after a class has interacted with other classes.
2.Data Flow Analysis(DFA):Along with syntactic pattern matching, FindBugs uses programs Control Flow Graphs(CFG) to determine the possible execution paths of a program.The goal of this method is to
gather information about the possible set of values calculated at different points of program code. With the help of CFG, data flow analysis simulates application’s execution paths without actually executing the
code.With the help of DFA, FindBugs can ascertain the possible values a variable can have at different instances of program flow.For example, using DFA, FindBugs can ascertain that the variable 's' can be NULL and raises a red flag:
String s= getString();
if (s == null && s.isEmpty()) {
return;
}
So, now you know what techniques FindBugs uses to detect errors.Lets move on to the second part.
2.The sample project.
We want to give a very brief introduction about the sample project on which we want to run FndBugs on.
We think that this will help set the context for further references.We have chosen our sample project very carefully.Lets face it.To give you a demo of the findings of FindBugs, the code must have some bugs to begin with :)Therefore, if we go about choosing a very popular open source project, the chances of finding bugs in them will be very less.At the same time, if we choose a really simple project, we cannot analysis a lot(if not all)of the variety of errors FindBugs can identify.So, we choose the "Email Extractor" project.It is a decent, popular, mid sized Java project. It can be downloaded here.
Extractor works in Java and on any Operating System.Just type a valid http address and hit enter and the extractor will extract email addresses while visiting the link.To run this program, just download the file and execute it.It has a .jar extension but it will run if you have java installed.
We have another sample project on which we have run FindBugs. We will speak more about in in our screen cast and podcast.For the purpose of this blog post, we will be using only the "Email Extractor" project and its code.
Now that we have introduced you to the project and its source code, lets run FindBugs on it using the FindBugs Eclipse plugin and discuss what we found.
3.What we found
Our intent is to not just show the bugs that FindBugs tool detected.Since you already know where to find the source code and how to install FindBugs(which we had covered in our previous post), we suspect you don't need our help in just running to tool to see the results yourself. What we want to do, is to give you a brief analysis of FindBugs by testing its capabilities by twisting and tweaking the code and finding out the end results.
We will show you a few of our analysis here.
ANALYSIS 1:
We ran FindBugs on the class called Download.java.In the Bug Explorer, FindBugs said it detected a bug as follows:
public class Download {
public static String getPage(URL page) {
String output = "";
URLConnection openConnection = null;
try {
openConnection = page.openConnection();
} catch (IOException ex) {
System.err.println(ex);
}
BufferedReader in = null;
try {
*** in = new BufferedReader(new InputStreamReader(openConnection.getInputStream()));
} catch (IOException ex) {
System.err.println(ex);
}
String input = "";
if (in != null) {
try {
while ((input = in.readLine()) != null) {
output = output + input + "\r\n";
}
} catch (IOException ex) {
System.err.println(ex);
}
try {
in.close();
} catch (IOException ex) {
System.err.println(ex);
}
}
return output;
}
}
In the above code, FindBugs reported an error saysing that there is a possible NullPointer Dereferencing on line marked as "***".After performing Data Flow Analysis, FindBugs found that if the reference variable openConnection had a NULL value, then dereferencing it on line "***" could generate a NullPointerException.
This was a good catch!!.
But we wanted to stretch this test a little further.We know that all exceptions in Java inherit from the Exception Base class.So, instead of catching the IOException, we changed the code to catch a generic Exception as below:
try {
*** in = new BufferedReader(new InputStreamReader(openConnection.getInputStream()));
} catch (Exception ex) {
System.err.println(ex);
}
Now, that we are catching any possible Exception including the possible NullPointerException, upon running FindBugs again, ideally, FindBugs should not detect possible NullPointer dereferencing Error.
Viola!!
It didn't. This shows that by performing Data Flow Analysis,FindBugs determined that even though the variable openConenction could have a NULL value, it was surrounded by a relevant try/catch block so all was OK.
In Static Analysis parlance, the above error that FindBugs caught is called "True Positive".
If a static analysis tool produces a warning which does not indicate a real defect, this warning is considered to be a false positive (FP). Warnings revealing real errors are, on the contrary, true positives (TP).
Sometimes also terms false negative (FN) and true negative (TN) are used.The term false negative refers to unwanted cases where a tool does not report a warning for a real defect. True negative is a less commonly used term which means a desired case where a tool reports no warning because there is no defect.
ANALYSIS 2
This time, we wanted to compare the type of warnings given by Eclipse versus the warnings alerted by FindBugs. We used two metrics for comparisons.
1. Relevance
2. Readablilty
The file MT_Processor.java of our sample project uses Java Reflections technique to access a method of a class.
public class MT_Processor implements Runnable {
public static DefaultListModel threads = new DefaultListModel();
public static boolean processing = false;
public static boolean deactivate = false;
public static Thread th = null;
@Override
public void run() {
(****) th = new Thread(this);
if (threads != null) {
for (int i = 0; i < threads.size(); i++) {
processing = true;
String currentthread[] = threads.getElementAt(i).toString().trim().split(",");
try {
(**) Class c = Class.forName(currentthread[1]);
Method currentmethod = c.getMethod(currentthread[0], (Class[]) null);
currentmethod.invoke(null, (Object[]) null);
} catch (ClassNotFoundException ex) {
Logger.getLogger(MT_Processor.class.getName()).log(Level.SEVERE, null, ex);
}
}
}
}
Eclipse popped a warning on line (**) saying "Class is a raw type. References to generic type Class<T> should be parameterized".FindBugs did not.Upon further investigation we found out that even though "Class" is a raw type, since it was being used to retrieve a method from a particular class at run time, it was impossible to ascertain the class name at compile time.So this warning that was issued by Eclipse was not relevant.
FindBugs,by using Data Flow Analysis Technique, determined the "context" in which the raw type "class" was being used and therefore did not issue any warning.
This shows that the DFA technique used by FindBugs is far more sophisticated than that of the inbuilt static analysis tool present in Eclipse.
Furthermore, there was another warning detected by FindBugs that was skipped by Eclipse at line marked(****).Here, the variable "th" which was declared a static in the class definition was being assigned a value in a non-static instance method called "run()".Though this is not a bug in the strict sense, if there was a static method that tried to access the above variable before it was initialized by the non-static method, the results would be wrong.
FindBugs detected this flaw and flagged it as a "warning" and not a potential bug.This result strongly speaks for the FindBugs ability to detect non local dependencies in code and its ability to categorize it as a warning and not a possible Bug.
In this blog post we share a couple of our analysis with you.We did not explore the cases where FindBugs could generate False Positives(even though we told you what False Positives mean).In out Podcast and screen cast we will continue to explore these areas in detail on another project.So stay tuned :)
Over the past few weeks, amidst our crazy schedule of exams and assignments, we were honing our skills and strengthening our fingers to give you a far more detailed analysis of our tool, FindBugs.
In this post, we intend to explain, in some detail, how FindBugs tool actually works and how effective it actually is in detecting possible errors in the code.
We have divided the post into 3 parts.The First Part explains how FindBugs actually works.The Second Part gives a brief introduction to the project on which we will be running FindBugs on to set the context and in the Third Part we will discuss the kind of bugs that FindBugs detected.
So, lets get started.
1.How FindBugs Tool detects errors
As discussed before, FindBugs actually analysis the generated Java bytecode and not the Java Source code.This is far more effective than running static analysis on the Source code because more often than not, source codes are subjected to compiler optimizations and this may lead to the creation of low level object files that are slightly different from the source files in executing paths.By analyzing the bytecode directly, FindBugs ensures that the analysis is happening on the actual mirror image of the to be generated executable file.
FindBugs uses two techniques to perform static analysis:
1.Syntactic Pattern matching technique: FindBugs linearly scans bytecode of java class files and searches for suspicious code patterns.By doing so, for every class file, FindBugs creates a state for each class.This in turn helps FindBugs to extend syntactic pattern matching to intraprocedural pattern matching to detect state changes in a class after a class has interacted with other classes.
2.Data Flow Analysis(DFA):Along with syntactic pattern matching, FindBugs uses programs Control Flow Graphs(CFG) to determine the possible execution paths of a program.The goal of this method is to
gather information about the possible set of values calculated at different points of program code. With the help of CFG, data flow analysis simulates application’s execution paths without actually executing the
code.With the help of DFA, FindBugs can ascertain the possible values a variable can have at different instances of program flow.For example, using DFA, FindBugs can ascertain that the variable 's' can be NULL and raises a red flag:
String s= getString();
if (s == null && s.isEmpty()) {
return;
}
So, now you know what techniques FindBugs uses to detect errors.Lets move on to the second part.
2.The sample project.
We want to give a very brief introduction about the sample project on which we want to run FndBugs on.
We think that this will help set the context for further references.We have chosen our sample project very carefully.Lets face it.To give you a demo of the findings of FindBugs, the code must have some bugs to begin with :)Therefore, if we go about choosing a very popular open source project, the chances of finding bugs in them will be very less.At the same time, if we choose a really simple project, we cannot analysis a lot(if not all)of the variety of errors FindBugs can identify.So, we choose the "Email Extractor" project.It is a decent, popular, mid sized Java project. It can be downloaded here.
Extractor works in Java and on any Operating System.Just type a valid http address and hit enter and the extractor will extract email addresses while visiting the link.To run this program, just download the file and execute it.It has a .jar extension but it will run if you have java installed.
We have another sample project on which we have run FindBugs. We will speak more about in in our screen cast and podcast.For the purpose of this blog post, we will be using only the "Email Extractor" project and its code.
Now that we have introduced you to the project and its source code, lets run FindBugs on it using the FindBugs Eclipse plugin and discuss what we found.
3.What we found
Our intent is to not just show the bugs that FindBugs tool detected.Since you already know where to find the source code and how to install FindBugs(which we had covered in our previous post), we suspect you don't need our help in just running to tool to see the results yourself. What we want to do, is to give you a brief analysis of FindBugs by testing its capabilities by twisting and tweaking the code and finding out the end results.
We will show you a few of our analysis here.
ANALYSIS 1:
We ran FindBugs on the class called Download.java.In the Bug Explorer, FindBugs said it detected a bug as follows:
public class Download {
public static String getPage(URL page) {
String output = "";
URLConnection openConnection = null;
try {
openConnection = page.openConnection();
} catch (IOException ex) {
System.err.println(ex);
}
BufferedReader in = null;
try {
*** in = new BufferedReader(new InputStreamReader(openConnection.getInputStream()));
} catch (IOException ex) {
System.err.println(ex);
}
String input = "";
if (in != null) {
try {
while ((input = in.readLine()) != null) {
output = output + input + "\r\n";
}
} catch (IOException ex) {
System.err.println(ex);
}
try {
in.close();
} catch (IOException ex) {
System.err.println(ex);
}
}
return output;
}
}
In the above code, FindBugs reported an error saysing that there is a possible NullPointer Dereferencing on line marked as "***".After performing Data Flow Analysis, FindBugs found that if the reference variable openConnection had a NULL value, then dereferencing it on line "***" could generate a NullPointerException.
This was a good catch!!.
But we wanted to stretch this test a little further.We know that all exceptions in Java inherit from the Exception Base class.So, instead of catching the IOException, we changed the code to catch a generic Exception as below:
try {
*** in = new BufferedReader(new InputStreamReader(openConnection.getInputStream()));
} catch (Exception ex) {
System.err.println(ex);
}
Now, that we are catching any possible Exception including the possible NullPointerException, upon running FindBugs again, ideally, FindBugs should not detect possible NullPointer dereferencing Error.
Viola!!
It didn't. This shows that by performing Data Flow Analysis,FindBugs determined that even though the variable openConenction could have a NULL value, it was surrounded by a relevant try/catch block so all was OK.
In Static Analysis parlance, the above error that FindBugs caught is called "True Positive".
If a static analysis tool produces a warning which does not indicate a real defect, this warning is considered to be a false positive (FP). Warnings revealing real errors are, on the contrary, true positives (TP).
Sometimes also terms false negative (FN) and true negative (TN) are used.The term false negative refers to unwanted cases where a tool does not report a warning for a real defect. True negative is a less commonly used term which means a desired case where a tool reports no warning because there is no defect.
ANALYSIS 2
This time, we wanted to compare the type of warnings given by Eclipse versus the warnings alerted by FindBugs. We used two metrics for comparisons.
1. Relevance
2. Readablilty
The file MT_Processor.java of our sample project uses Java Reflections technique to access a method of a class.
public class MT_Processor implements Runnable {
public static DefaultListModel threads = new DefaultListModel();
public static boolean processing = false;
public static boolean deactivate = false;
public static Thread th = null;
@Override
public void run() {
(****) th = new Thread(this);
if (threads != null) {
for (int i = 0; i < threads.size(); i++) {
processing = true;
String currentthread[] = threads.getElementAt(i).toString().trim().split(",");
try {
(**) Class c = Class.forName(currentthread[1]);
Method currentmethod = c.getMethod(currentthread[0], (Class[]) null);
currentmethod.invoke(null, (Object[]) null);
} catch (ClassNotFoundException ex) {
Logger.getLogger(MT_Processor.class.getName()).log(Level.SEVERE, null, ex);
}
}
}
}
Eclipse popped a warning on line (**) saying "Class is a raw type. References to generic type Class<T> should be parameterized".FindBugs did not.Upon further investigation we found out that even though "Class" is a raw type, since it was being used to retrieve a method from a particular class at run time, it was impossible to ascertain the class name at compile time.So this warning that was issued by Eclipse was not relevant.
FindBugs,by using Data Flow Analysis Technique, determined the "context" in which the raw type "class" was being used and therefore did not issue any warning.
This shows that the DFA technique used by FindBugs is far more sophisticated than that of the inbuilt static analysis tool present in Eclipse.
Furthermore, there was another warning detected by FindBugs that was skipped by Eclipse at line marked(****).Here, the variable "th" which was declared a static in the class definition was being assigned a value in a non-static instance method called "run()".Though this is not a bug in the strict sense, if there was a static method that tried to access the above variable before it was initialized by the non-static method, the results would be wrong.
FindBugs detected this flaw and flagged it as a "warning" and not a potential bug.This result strongly speaks for the FindBugs ability to detect non local dependencies in code and its ability to categorize it as a warning and not a possible Bug.
In this blog post we share a couple of our analysis with you.We did not explore the cases where FindBugs could generate False Positives(even though we told you what False Positives mean).In out Podcast and screen cast we will continue to explore these areas in detail on another project.So stay tuned :)
Subscribe to:
Posts (Atom)