developerWorks chat: Improve Your Java Code Quality Date: November 17, 2006 Time 12:30 EST Moderator: Andrew Glover, Stelligent Incorporated 12:30 AndyGlover: Hey everyone, I'm Andy Glover- let's start this thing off! 12:30 AndyGlover: I'm the author of devWorks "In pursuit of code quality"article series 12:31 AndyGlover: I'm the president of Stelligent-- we help companies address code quality early in the development lifecycle 12:32 AndyGlover: Also in this Chat is Paul Duvall and Levent Gurses 12:32 AndyGlover: they are Stelligent employees. Paul is the author of IBM's "Automation for the people" series 12:33 RyanSchutt: hello 12:33 dodiligence: Hi this is Tom McKittrick. 12:33 RyanSchutt: What's a good tool to use for java code metrics? 12:34 AndyGlover: There are a few-- check out PMD and JavaNCSS 12:34 AndyGlover: PMD -- http://pmd.sourceforge.net/ 12:34 AndyGlover: JavaNCSS-- http://www.kclee.de/clemens/java/javancss/ 12:35 AndyGlover: If you are an eclipse user, check out http://metrics.sourceforge.net/ 12:35 Paul_Duvall: CheckStyle is a good alternative to PMD 12:35 JosephLowe: what about jalopy? 12:36 Paul_Duvall: Jalopy is good if you don't mind it formatting the code for you -- and it doesn't really provide code metrics 12:36 AndyGlover: Another metrics tool for Java is JDepend 12:36 AndyGlover: You can find JDepend here: http://clarkware.com/software/JDepend.html 12:37 AndyGlover: Jalopy can be found here: http://jalopy.sourceforge.net/ 12:37 AndyGlover: CheckStyle-- http://checkstyle.sourceforge.net/ 12:37 RyanSchutt: What if my project is C#? 12:37 AndyGlover: The challenge with code metrics is understanding what they mean. 12:38 AndyGlover: [RyanSchutt] -- if you are doing C# check NDepend 12:38 AndyGlover: what all the tools mentioned before don't do is trend info 12:38 AndyGlover: code metrics make more sense over time rather than specific points in time 12:39 jaaloi: are there tools that do provide trend info? 12:39 AndyGlover: Check out http://qalab.sourceforge.net/ 12:39 AndyGlover: QALab aims to provide trending 12:40 AndyGlover: with trending, it becomse important to run metrics OFTEN-- hence, the importance of Continuous Integration 12:40 AndyGlover: if you can run tools like PMD, JavaNCSS, etc on a regular basis you can spot trends 12:40 James_Bond: Speaking of Continuous Integration which CI server should I use?? 12:41 NeilC: is there another way besides code metrics to measure code quality? 12:41 AndyGlover: [James_Bond] -- check out Paul's article on the subject: http://www-128.ibm.com/developerworks/java/library/j-ap09056/ 12:42 AndyGlover: [NeilC]-- code coverage can help along with test results! 12:42 AndyGlover: code quality is kind of a moving target when you attempt to define it 12:42 Paul_Duvall: RE: trending...apparently XRadar puts together results from a number of open source tools - not sure if it trends 12:42 billhiggins: anyone out there use CodeReview inside of RAD? I used to find that was good for smoking out non-intuitive bugs that only manifest themselves in production server environments with lots of concurrent users. 12:43 NeilC: can you explain the relationship between code quality and test coverage? 12:43 Paul_Duvall: http://xradar.sourceforge.net/ 12:44 AndyGlover: [NeilC] -- The relationship is loose. With code coverage you can easily spot areas WITHOUT corresponding tests. If you can correlate that data with other metrics, like complexity, you can spot areas of "risk". Areas with high complexity and no coverage could indicate risk-- the opposite is the that areas with high coverage and low complexity COULD be an indication of quality. 12:44 billhiggins: basically with test coverage and automated unit tests you're trying to get warm fuzzies that: 1) your code does what you expect it to do (esp. for external clients of your code), and 2) you can verify that some % of your code does what it says it's going to do 12:46 billhiggins: if you don't have tests for some class, it's impossible to consistently say that it behaves as specified 12:46 AndyGlover: [billhiggins] -- yes, good point. it also comes down to your definition of "quality" 12:47 billhiggins: tests can only say that the code behaves as it was specified to behave - it's not a guarantee of quality 12:47 AndyGlover: agreed 12:47 AndyGlover: [all]-- what do you think quality is when it comes to code? 12:47 Heman10: I am a rookie to java development so pardon my ignorance. Is it not enough to just run unit tests for the section you are modifying with each release? 12:47 billhiggins: if I have a simple add(int, int) that takes 10 minutes to execute but executes correctly, it's still bad (if correct), code 12:48 jayallen: [andy] -- to me it's totally what the user thinks of the software 12:48 AndyGlover: [Herman10]- thats a great start! 12:49 billhiggins: [Heman10] - why not run all of your unit tests? the hard work is writing the unit tests - running them is easy 12:49 Paul_Duvall: i see code quality as related to "ilities' like maintainability, extensibility, etc... 12:49 billhiggins: if some subset of your code hasn't changed then presumably the unit tests are still grean 12:49 AndyGlover: [jayallen]-- yeah, and you'll find that users don't care about complexity, etc! but what you'll find is that BOTH are related-- in terms of how users define quality and how we do 12:50 Heman10: I was reading your article " Use test categorization for agile builds" and I was considering the case when the build takes some 3-4 hours 12:50 Levent_Gurses: [andy] - it has to do with a functioning software that makes its users achieve their goals within a reasonable amount of time... 12:50 NeilC: code quality means that the code is easily to maintain and adapt, I think 12:50 AndyGlover: [NeilC]-- yeah, I agree w/you. 12:51 billhiggins: well users will indirectly care about complexity when the next release takes disproportionately long to the amount of new function 12:51 NeilC: so what are some strategies to make code more maintainable? 12:51 billhiggins: because the devs have to alter spaghetti 12:52 AndyGlover: [NeilC]-- #1 strategy-- Developer Testing 12:52 AndyGlover: [NeilC]-- without tests you can't change the code easily or with confidence 12:53 AndyGlover: [NeilC]-- metrics are secondary-- they are a passive measurement-- they tell you what's already happended and give you areas of risk 12:53 billhiggins: another thing is just programming fundamentals - be very clear on the contracts for published classes and methods 12:53 billhiggins: e.g. input/output/preconditions/postconditions/invarients/etc 12:53 AndyGlover: [billhiggins]]- yeah, good point! 12:53 NeilC: we seem to be confused about the distiction between quality and corectness. aren't tests written by developers focused on correctness? 12:54 billhiggins: in the Eclipse/Jazz org, we really try to differentiate between published interfaces/classes and internal classes. the requisite level of quality for published interfaces/classes is very high in terms of requring 100% unit test coverage 12:54 AndyGlover: [NeilC]-- I think they ensure both-- one is related to the other 12:54 billhiggins: but internal stuff may or may not have unit tests - it's all about whether it's callable by external users 12:55 AndyGlover: [billhiggins]-- how are you measuring coverage? 12:55 billhiggins: correctness is the basic pre-req for quality 12:55 billhiggins: um, unreleased tools 12:55 billhiggins: coming to a web site near you in 2007 12:56 billhiggins: someone said something earlier about the 'ilities - I think that's on the right track 12:56 NeilC: How could a team that has little experience with metric tools begin to incorporate quality metrics into our development process? 12:56 billhiggins: quality: 1) correctness, 2) performance & security & adaptability, etc. 12:56 AndyGlover: [NeilC]-- start to capture them first-- add one tool to your build (be it Ant or Maven) 12:57 Levent_Gurses: [billhiggings] what is the unreleased tool you are referring to? 12:57 AndyGlover: [NeilC]-- see if those metrics actually make sense-- remember-- they are OBJECTIVE measurements but they must be applied SUBJECTIVELY 12:58 billhiggins: [Levent_Gurses] - don't want to go off topic - feel free to email me (bhiggins@us.ibm.com) 12:58 AndyGlover: [NeilC]-- if you all have tests (i.e. in JUnit, for example) see if the two overlap. If there area areas of risk-- do you have tests that cover them? 12:58 billhiggins: Andy - can you talk a bit about how team culture and process affect code quality over time? 12:59 billhiggins: because a lot of the time is that this stuff is straightforward to understand, but hard to implement because of schedule pressures 12:59 billhiggins: i.e. even though it pays off in the long run, it steals cycles from new functional development in the short term 12:59 AndyGlover: [billhiggins] -- if teams turn over a lot -- code quality will be affected. regardless, you have to think in terms of People/Process/Technology 10:00 AndyGlover: [billhiggns]-- YES! Quality DOES cost up front. See http://www.testearly.com/2006/11/07/quality-isnt-free/ 10:00 billhiggins: re: turnover - one of the nice things about having a unit test suite is that it provides concrete examples of how the code should work, which is very helpful (esp. when code is poorly commented) 10:01 billhiggins: so how do you convince an org that they should make the upfront costs? 10:01 billhiggins: 'cause I can imagine it might be a tough conversation to say "reduce functionality by 20% to make these investments in testing infra which will have some unspecified payoff over the lifetime of the app" 10:01 AndyGlover: [billhiggins]-- YEP! getting people to focus on quality means looking long term from a management standpoint. You convince them will real ROI metrics-- how long is the application gonna be around? How much do you spend fixing defects in late cycles? 10:02 billhiggins: it may make a lot of sense to a lead arch/dev, but not so much a project sponsor who is a business guy/gal and wants his/her 20 shiney new features 10:02 billhiggins: do you have any case studies that you could reference if you were making the pitch to a sceptical org? 10:02 AndyGlover: [ billhiggins]-- yeah, you bring up a good point-- we often find that business people who have "done it before" are more apt to listen 10:03 JosephLowe: compared to junit4 why is testng more suitable for high-scale testing? 10:03 AndyGlover: [ JosephLowe]-- well, they are both fairly flexible with higher level testing 10:04 AndyGlover: [ JosephLowe]-- but right now, JUnit 4 isn't easily supported with build processes like Ant 10:04 AndyGlover: [ JosephLowe] and JUnit 4 is only Java 1.5 10:04 AndyGlover: [ JosephLowe] TestNG supports Java 1.4 and is easily run in Ant, Maven 1&2 and command line 10:06 James_Bond: What do you use for web based functional testing 10:06 AndyGlover: [James_Bond]-- there are a number of tools available-- you can check out the Rational Suite of tools 10:06 AndyGlover: like Rational Robot 10:07 AndyGlover: then there are some open source tools too-- check out Selenium (http://www.openqa.org/selenium/) 10:07 Paul_Duvall: Selenium and Watir are good for developer testing - FIT-style 10:07 Paul_Duvall: easy to run Selenium in a CI environment 10:08 billhiggins: do you mean FIT as in Ward Cunningham's FIT? 10:08 Levent_Gurses: You can even try out JMeter (has some load testing capability as well) 10:08 JosephLowe: also maxq 10:08 billhiggins: http://fit.c2.com/ 10:08 billhiggins: I've been meaning to check that out 10:08 AndyGlover: FIT style meaning tables 10:08 AndyGlover: that non-programmers can write 10:09 Paul_Duvall: yep, FIT (Framework for Integration Testing) tablular driven format - define in an HTML or other type of table 10:09 AndyGlover: Check out "In pursuit of code quality: Resolve to get FIT" at http://www.ibm.com/developerworks/java/library/j-cq02286/index.html 10:09 AndyGlover: Fitnesse is a Wiki front end to FIT too 10:09 AndyGlover: http://fitnesse.org/ 10:09 AndyGlover: For the C# person earlier-- FIT has a .NET extension 10:10 Paul_Duvall: Selenium movie on test early: http://www.testearly.com/2006/10/04/selenium-using-selenium-ide-selenium-remote-control-and-ant/ 10:10 AndyGlover: C# Fit-- check out http://www.testearly.com/2006/06/04/nets-fit-is-a-snap/ 10:11 scrly: any thoughts on best practicing for those learning/starting unit testing in an org? 10:11 AndyGlover: More on Fitnesse-- check out "Fitnesse Testing for Fast-Paced Agile Web Development" 10:11 AndyGlover: at http://today.java.net/pub/a/today/2005/11/22/fitnesse-testing-for-agile-web-development.html 10:12 Paul_Duvall: [scrly] on an existing code base or new development? 10:12 Levent_Gurses: Here is one more Fitnesse article: http://www.javaworld.com/javaworld/jw-02-2006/jw-0220-fitnesse.html 10:12 scrly: large existing 10:12 AndyGlover: [scrly]-- if you are working on legacy project which doesn't have tests-- make a point to write tests for all new modifications 10:12 AndyGlover: [scrly]-- it's hard to retrofit existing code with true UNIT tests 10:13 jayallen: i mainly do web application development, what do you recommend for system testing? 10:13 AndyGlover: with large existing code bases-- higher level tests can give you a big bang for the buck 10:14 Paul_Duvall: [scrly] yeah, what andy said and focus on the parts of the system that are of higher business value and higher complexity 10:14 AndyGlover: [jayallen]-- there are a number of tools ranging from the Rational suite to open source alternatives like Selenium 10:14 jayallen: ah, nice 10:15 jayallen: i've used an open source one that i'm blocking on right now 10:15 scrly: would you suggest junit or other tools for higher level testing? 10:15 AndyGlover: Rational Functional Tester @ http://www-306.ibm.com/software/awdtools/tester/functional/index.html 10:15 Paul_Duvall: [jayallen] jWebUnit is another one - runs it within the container 10:15 AndyGlover: selenium @ http://www.openqa.org/selenium/ 10:15 JasonVC: My shop has used Grinder to some good effect. 10:16 JasonVC: It's for load testing, but you can script it for many purposes. 10:16 AndyGlover: [scrly]-- higher level testing is easier with TestNG 10:16 jayallen: thanks, checking out selenium right now 10:17 AndyGlover: See "In pursuit of code quality: JUnit 4 vs. TestNG" @ http://www.ibm.com/developerworks/java/library/j-cq08296/index.html 10:17 Paul_Duvall: Yep, Grinder and JMeter both handle load testing 10:17 billhiggins: are there any articles that talk about someone who's successfully implemented automated functional testing? I've heard of the products but never seen a case study of a project that really applied them successfully (to be fair, I haven't been looking very hard for such a case study) 10:19 JosephLowe: [bill] check out this: http://www.rttsweb.com/news/docs/func_automation_study.jsp 10:19 AndyGlover: I'm sure if you check out Rational's site or Mercury's you'll find case studies there 10:20 JosephLowe: anyone have thoughts on EMMA over Cobertura? 10:20 AndyGlover: Cobertura's reports are slicker 10:21 AndyGlover: but Emma reports coverage more precisely 10:21 AndyGlover: Nothing says you can't run them both side by side 10:22 AndyGlover: Cobertura: http://cobertura.sourceforge.net/ 10:22 AndyGlover: emma: http://emma.sourceforge.net/ 10:22 AndyGlover: [all]--anyone using a different coverage tool? 10:23 AndyGlover: Also, don't forget that Coverage Reports can fool you. 10:23 AndyGlover: See "In pursuit of code quality: Don't be fooled by the coverage report" @ http://www.ibm.com/developerworks/java/library/j-cq01316/index.html 10:23 James_Bond: I am using Clover 10:24 AndyGlover: yeah, that's another great tool 10:24 AndyGlover: Clover-- http://www.cenqua.com/clover/ 10:24 JosephLowe: what about hansel and jester? 10:24 AndyGlover: 2.0 is coming out shortly and has a load of new features 10:24 AndyGlover: Hansel and Jester are more "proactive" coverage tools 10:25 AndyGlover: [JosephLowe]-- these tools proactively measure coverage by examining the source code 10:25 AndyGlover: [JosephLowe] Jester actually changes your source code 10:25 AndyGlover: [JosephLowe] the thought being that if changing the source DOESN'T cause a failure, then the tests aren't effective 10:26 AndyGlover: [JosephLowe] Hansel will examine the byte code of source code during testing and verify pathing-- if paths are NOT covered, Hansel will fail the test case 10:27 AndyGlover: [JosephLowe] in practice Jester is quite buggy 10:28 AndyGlover: Check out this article on Hansel here: http://www.ibm.com/developerworks/java/library/j-cwt02095/ 10:28 JosephLowe: [andy] Is Branch coverage important? 10:28 AndyGlover: Jester here: "Test your tests with Jester " @ http://www.ibm.com/developerworks/java/library/j-jester/ 10:28 AndyGlover: [JosephLowe] -- yeah, its a better indication of coverage than statement coverage 10:29 AndyGlover: you can correlate branch coverage with compelxity 10:29 billhiggins: gotta run - thanks Andy and others - it was informative! 10:30 AndyGlover: a ratio of compexity to tests can indicate coverage 10:30 AndyGlover: in a perfect world for every CCN there would be test case 10:30 AndyGlover: CCN is a measurement of branching/pathing 10:30 Paul_Duvall: line coverage is whether you've written a test for a line (regardless of the number of paths in a line). branch coverage is whether you've written a test for each side of an if statement, etc. 10:32 JosephLowe: thanks andy this has been very valuable 10:32 AndyGlover: thank you 10:32 AndyGlover: good resources for reading before we wrap up: 10:33 AndyGlover: Check out http://erik.thauvin.net/linkblog/, which is a blog aggregator 10:33 AndyGlover: Also, http://www.opensourcetesting.org/ is another good aggregator 10:34 AndyGlover: Books: Working Effectively with Legacy Code, Refactoring: Improving the Design of Existing Code, JUnit Recipes: Practical Methods for Programmer Testing, and JUnit in Action 10:34 AndyGlover: of course, don't forget Code complete 10:34 AndyGlover: Lastly, developerWork's Java zone is an excellent resource 10:35 AndyGlover: thank you everyone-- it's been a fun hour! if you have additional questions, feel free to post them on the associated forum at http://www-128.ibm.com/developerworks/forums/dw_forum.jsp?forum=812&cat=10 10:37 AndyGlover: Thanks to DevWorks for making this happen 10:38 jaaloi: You're welcome, Andy. Stay tuned to the Java zone for details on upcoming chats.