What are the big improvements between guava and apache equivalent libraries?

First of, as javamonkey79 explained, while Google Guava and Apache Commons do share similar features, they also both have functionality that is absent from their counterpart. Thus, limiting yourself to only one library might be unwise.

That being said, if I had to choose, I'd opt to use Guava, keeping Apache Commons around for the (rare) cases where Guava does not have the needed functionality. Let me attempt to explain why.

Guava is more "modern"

Apache Commons is a really mature library, but it's also almost 10 years old, and targets Java 1.4. Guava was open sourced in 2007, targets Java 5, and thus Guava greatly benefits from the Java 5 features: generics, varargs, enums, and autoboxing.

According to the Guava developers, generics are one reason they chose to create a new library instead of improving Apache Commons (see the google-collections FAQ, under the title "Why did Google build all this, when it could have tried to improve the Apache Commons Collections instead?").

I agree with them: while often criticized (no reification, limited due to backward compatibility), Java generics are still very useful when used appropriately, like Guava does. I'd rather quit than work with non-generified collections!

(Note that Apache Commons 3.0, does target Java 1.5+)

Guava is very well designed / documented

The code is full of best practices and useful patterns to make the API more readable, discoverable, performant, secure, thread-safe...

Having read Effective Java (awesome book BTW), I see these patterns everywhere in the code:

• factory methods (such as ImmutableList.copyOf())
• builder pattern (ImmutableList.builder(), Joiner, CharMatcher, Splitter, Ordering, ...)
• immutability (immutable collections, CharMatcher, Joiner, Splitter,...)
• implementation hiding (Predicates.xXx, ...)
• favoring composition over inheritance(the ForwardXXX collections)
• null-checks
• enum-singleton pattern
• serialization proxies
• well thought-out naming conventions

I could go on for hours explaining the advantages brought by these design choices (tell me if you want me to). The thing is, these patterns are not only "for the show", they have a real value: the API is a pleasure to use, easier to learn (did I forget to say how well documented it is?), more efficient, and many classes are simpler / thread-safe due to their immutability.

As a bonus point, one learns a lot by looking at the code :)

Guava is consistent

Kevin Bourrillion (Guava's lead developer) does a great job maintaining a high level of quality / consistency across the library. He is of course not alone, and a lot of great developers have contributed to Guava (even Joshua Bloch, who now works at Google!).

The core philosophies and design choices behind Guava are consistent across the library, and the developers adhere to very good (IMO) API design principles, having learned from past mistakes of the JDK APIs (not their mistakes, though).

Guava has a high power-to-weight ratio

The Guava designers resist the temptation to add too many features, limiting the API to the most useful ones. They know it's very hard to remove a feature once added, and follow Joshua Bloch's motto on API design: "When in doubt, leave it out". Also, using the @Beta annotation allows them to test some design choices without committing to a specific API.

The design choices mentioned above allow for a very compact API. Simply look at the MapMaker to see the power packed inside a "simple" builder. Other good (albeit simpler?) examples are CharMatcher, Splitter, and Ordering.

It's also very easy to compose various parts of Guava. For example, say you want to cache the result of a complex function? Feed this function to your MapMaker and BINGO, you got a thread-safe computing map/cache. Need to constrain the map/function inputs to specific Strings? No problem, wrap it inside a ConstrainedMap, using a CharMatcher to reject inappropriate Strings...

Guava is in active development

While the development of Apache Commons seems to have accelerated with the work on Commons Lang 3.0, Guava seems to pick up more steam at the moment, while Google open sources more of their internal classes.

Since Google heavily relies on it internally, I don't think it's going to disappear any time soon. Plus, open sourcing its common libraries allows Google to more easily open source other libraries that depend on it (instead of repackaging them, like Guice currently does).

Conclusion

For all the above reasons, Guava is my go-to library when starting a new project. And I am very grateful to Google and to the awesome Guava developers, who created this fantastic library.

PS: you might also want to read this other SO question

PPS: I don't own any Google stock (yet)

I've been using guava since Aug 2010, starting with the r06 release. Basically, I had a greenfield java library to develop, so I looked around for the best adjunct library for the J2SE API. Traditionally, we'd used the Apache Commons libraries, but I wanted to see what was out there and began using Guava.

Pros

1. Java 5.0 language constructs. The library takes most of its design cues from Bloch's "Effective Java: 2nd Edition": Immutability, builder pattern, factories instead of constructors, Generics, etc. This makes your code tighter and more expressive.
2. Functional programming support, in particular with the top-level Function and Predicate interfaces.

Cons

1. It's not a sufficient replacement for Apache Commons, in particular commons-codec.
2. There's not a 'guava cookbook'. The library is both minimalistic and orthogonal. Thus, there's a definite learning curve to take full advantage of it. As mentioned, the Javadoc is excellent, but some longer source code case studies would be helpful.
3. If you're in an environment requiring Java 1.3 or 1.4, you're out of luck.

To me, Guava makes Java feel closer to a terse, expressive scripting language, and that's great.

In my experience I don't perceive that they contend with one another, or that guava improves on the apache libs. Rather, guava complements the apache libs. There are classes and utilities in guava that are not in apache and vice versa.

Therefore, I don't know that you need to switch per se - I would say "use the right tool for the right job".