Blog JUnit vs TestNG a comparison
Since I launched this site, I have done a few tutorials on the two major unit testing frameworks in the Java space: JUnit and TestNG (you can view them in the Tutorials section of this site). Each of them are very competent frameworks that provide a robust set of features that allow developers to create complex test scenarios to adequately test just about any piece of code. But if you are starting a new project, which should you choose? If you already have a suite of test cases in one or the other, should you switch? Let’s find out.
| Feature | JUnit | TestNG |
| User Defined Life Cycle |  |
|
| Test Organization (groups, etc) | |
|
| Distributed Test Execution | |
|
| Parallel Test Execution | |
|
| Data Driven Tests | |
|
| Dependency Testing | |
|
| IDE Integration | |
|
| Ant Integration | |
|
| Maven Integration | |
|
| Domain Specific Extensions (Database, HTTP, etc) | |
|
| Active Community | |
|
The criteria I looked at, as you can see above was pretty diverse. User defined life cycle is the ability to tell the test runner what methods to execute in what order. Both JUnit and TestNG allow you to do this via annotations. The ability to organize my tests is another nice to have. TestNG allows you to group tests across test classes via annotations. This is a handy tool for crateing groups for different development tasks (checking in, continuous integration build, by functional area, etc) that JUnit doesn’t have. The most grouping you can do with JUnit is by grouping test methods together in a single class. On a large project the ability to run tests in either a distributed mode or in a parallel mode can definitely speed up execution. Yet another two points for TestNG. We have all come across very data intensive areas that require a large amount of data to truly test all scenarios. TestNG’s DataProvider method allows for the ability to build a data set for a single test scenario. The scenario is then executed over all of the data. Very powerful stuff. Dependency testing is also something that is important in a large system. I know these are both technically “unit test” frameworks, but they still should be able to handle all testing scenarios we throw at it. Given JUnit’s rigid enforcement of testing in isolation, it is difficult to test dependencies. Yet another point to TestNG.
Both frameworks integrate well into all major IDEs as well as the major build tools of the day. JUnit has a large collection of third party add ons that can help in domain specific areas like database testing and HTTP front end testing. However since TestNG can execute JUnit tests, you don’t loose the ability to use those tools by using TestNG. They both have an active community with a plethora of documentation available on the net (although at the time of this writing, JUnit.org has been down for a few days. Not sure what is going on there).
For me, the decision is clear. TestNG is a high caliber testing framework that can assist in just about any testing scenario you can think of. The guys over at TestNG did a good job. They originally intended to take JUnit and make it better. Given the easy learning curve and powerful features, it’s hard to argue against TestNG as the way to go in unit testing.
My understanding of TestNG is that it does not reload classes upon executing each test method. This was due to a lack of understanding of the Java language by the TestNG implementor. He even knew it was happening in Junit but apparently ignored that fact. And now so many people rely on the fact that classes won’t be reloaded, that he feels he can’t change testng…
Stuff such as this concerns me to an extent that I’ve never picked up the framework. With regards to grouping tests with annotations, putting tests in different test folders have actually always sufficed for the stuff that we’ve been doing. such strategy doesn’t support multiple groupings of the same tests… but we’ve never felt the need for that.
just my 2 cents
kasper, http://firstclassthoughts.co.uk/
I’m somewhat aghast that any Java developer would say that object should be reinstatiated on every method invocation, and that NOT having this behaviour is due to a misunderstanding of how Java works. Serious Kaspar, do you have a job doing Java stuff?
Exactly how many people do this? Create an object, invoke a method on it, throw it away, create another instance, invoke a 2nd method, and so on?
Hi Kasper,
I don’t even know where to start
The motivations behind TestNG are very clearly explained in the documentation, in the multiple articles that have covered TestNG since its inception and, of course, in our book (”Next Generation Testing in Java”, check it out, you might find a few things that will challenge your thinking).
And by the way, get your vocabulary straight: we are not talking about reloading classes but reinstantiating them.
Let me give you a short summary of these reasons since you obviously didn’t bother researching any of this: keeping the values of the test objects between method invocations is not only intuitive to any Java developer (as Hani pointed above) but it’s also the only sane way to maintain expensive state across your tests (I’m betting you use static fields to achieve this?).
Please check out some of the material I have written over the past years on the subject and I’ll be happy to discuss specific issues with you, as long as you refrain from using ad hominem attacks.
–
Cedric
I’m not sure that I understand the issue you guys are referring to.
Are you saying that TestNG instantiates the objects to test only once, and then uses the same objects for all of the tests?
If so, wouldn’t this cause unexpected dependencies between the current test and previous tests?
I’m imagining some complex object that holds a number internally as part of its function. It’s a very magic number that gets changed when certain methods are called. This means that in the course of testing, the number will get changed depending on what tests are run and in what order. In a later test, you call addToNumber(int value), and then getCurrentNumber() to make sure it added correctly. However, if previous tests called methods that muck about with this number, how can you write an effective test for addToNumber short of reinstantiating the object?
The www.michaelminella.com is good resource, respect, owner. But see this state select hot water heater