Question

Why using Integration tests instead of unit tests is a bad idea?

Let me start from definition:

Unit Test is a software verification and validation method in which a programmer tests if individual units of source code are fit for use

Integration testing is the activity of software testing in which individual software modules are combined and tested as a group.

Although they serve different purposes very often these terms are mixed up. Developers refer to automated integration tests as unit tests. Also some argue which one is better which seems to me as a wrong question at all.

I would like to ask development community to share their opinions on why automated integration tests cannot replace classic unit tests.

Here are my own observations:

  1. Integration tests can not be used with TDD approach
  2. Integration tests are slow and can not be executed very often
  3. In most cases integration tests do not indicate the source of the problem
  4. it's more difficult to create test environment with integration tests
  5. it's more difficult to ensure high coverage (e.g. simulating special cases, unexpected failures etc)
  6. Integration tests can not be used with Interaction based testing
  7. Integration tests move moment of discovering defect further (from paxdiablo)

EDIT: Just to clarify once again: the question is not about whether to use integration or unit testing and not about which one is more useful. Basically I want to collect arguments to the development teams which write ONLY integration tests and consider them as unit tests. Any test which involve components from different layers is considered as integration test. This is to compare to unit test where isolation is the main goal.

Thank you, Andrey

 45  17340  45
1 Jan 1970

Solution

 41

Integration tests tell you whether it's working. Unit tests tell you what isn't working. So long as everything is working, you "don't need" the unit tests - but once something is wrong, it's very nice to have the unit test point you directly to the problem. As you say, they serve different purposes; it's good to have both.

To directly address your subject: integration tests aren't a problem, aren't the problem. Using them instead of unit tests is.

2009-11-24

Solution

 26

I find integration tests markedly superior to unit tests. If I unit test my code, I'm only testing what it does versus my understanding of what it should do. That only catches implementation errors. But often a much bigger problem is errors of understanding. Integration tests catch both.

In addition, there is a dramatic cost difference; if you're making intensive use of unit tests, it's not uncommon for them to outweigh all the rest of your code put together. And they need to be maintained, just like the rest of the code does. Integration tests are vastly cheaper -- and in most cases, you already need them anyway.

There are rare cases where it might be necessary to use unit tests, e.g. for internal error handling paths that can't be triggered if the rest of the system is working correctly, but most of the time, integration tests alone give better results for far lower cost.

2010-10-10