Bad performance with Guava Cache on Android
CacheBuilder was designed for server-side caching, where concurrency was a primary concern. It therefore trades off single-threaded and memory overhead in exchange for better multi-threaded behavior. Android developers should use LruCache, LinkedHashMap, or similar where single-threaded performance and memory are the primary concerns. In the future there may be a concurrencyLevel=0 to indicate that a lightweight, non-concurrent cache is required.
Android code is not always optimised the same way as it is in a JVM. What can perform very well in Java might not perform as well in Android. I suggest you write a very simple cache of your own. e.g. using LinkedHashMap.removeEldestEntry() and see how that goes.