How to compress Bitmap as JPEG with least quality loss on Android?

After some investigation I found the culprit: Skia's YCbCr conversion. Repro, code for investigation and solutions can be found at TWiStErRob/AndroidJPEG.

Discovery

After not getting a positive response on this question (neither from http://b.android.com/206128) I started digging deeper. I found numerous half-informed SO answers which helped me tremendously in discovering bits and pieces. One such answer was https://stackoverflow.com/a/13055615/253468 which made me aware of YuvImage which converts an YUV NV21 byte array into a JPEG compressed byte array:

YuvImage yuv = new YuvImage(yuvData, ImageFormat.NV21, width, height, null);yuv.compressToJpeg(new Rect(0, 0, width, height), 100, jpeg);

There's a lot of freedom going into creating the YUV data, with varying constants and precision. From my question it's clear that Android uses an incorrect algorithm.While playing around with the algorithms and constants I found online I always got a bad image: either the brightness changed or had the same banding issues as in the question.

Digging deeper

YuvImage is actually not used when calling Bitmap.compress, here's the stack for Bitmap.compress:

• libjpeg/jpeg_write_scanlines(jcapistd.c:77)
• skia/rgb2yuv_32(SkImageDecoder_libjpeg.cpp:913)
• skia/writer(=Write_32_YUV).write(SkImageDecoder_libjpeg.cpp:961)
  [WE_CONVERT_TO_YUV is unconditionally defined]
• SkJPEGImageEncoder::onEncode(SkImageDecoder_libjpeg.cpp:1046)
• SkImageEncoder::encodeStream(SkImageEncoder.cpp:15)
• Bitmap_compress(Bitmap.cpp:383)
• Bitmap.nativeCompress(Bitmap.java:1573)
• Bitmap.compress(Bitmap.java:984)
• app.saveBitmapAsJPEG()

and the stack for using YuvImage

• libjpeg/jpeg_write_raw_data(jcapistd.c:120)
• YuvToJpegEncoder::compress(YuvToJpegEncoder.cpp:71)
• YuvToJpegEncoder::encode(YuvToJpegEncoder.cpp:24)
• YuvImage_compressToJpeg(YuvToJpegEncoder.cpp:219)
• YuvImage.nativeCompressToJpeg(YuvImage.java:141)
• YuvImage.compressToJpeg(YuvImage.java:123)
• app.saveNV21AsJPEG()

By using the constants in rgb2yuv_32 from the Bitmap.compress flow I was able to recreate the same banding effect using YuvImage, not an achievement, just a confirmation that it's indeed the YUV conversion that is messed up. I double-checked that the problem is not during YuvImage calling libjpeg: by converting the Bitmap's ARGB to YUV and back to RGB then dumping the resulting pixel blob as a raw image, the banding was already there.

While doing this I realized that the NV21/YUV420SP layout is lossy as it samples the color information every 4th pixel, but it keeps the value (brightness) of each pixel which means that some color info is lost, but most of the info for people's eyes are in the brightness anyway. Take a look at the example on wikipedia, the Cb and Cr channel makes barely recognisable images, so lossy sampling on it doesn't matter much.

Solution

So, at this point I knew that libjpeg does the right conversion when it is passed the right raw data. This is when I set up the NDK and integrated the latest LibJPEG from http://www.ijg.org. I was able to confirm that indeed passing the RGB data from the Bitmap's pixels array yields the expected result. I like to avoid using native components when not absolutely necessary, so aside of going for a native library that encodes a Bitmap I found a neat workaround. I've essentially taken the rgb_ycc_convert function from jcolor.c and rewrote it in Java using the skeleton from https://stackoverflow.com/a/13055615/253468. The below is not optimized for speed, but readability, some constants were removed for brevity, you can find them in libjpeg code or my example project.

private static final int JSAMPLE_SIZE = 255 + 1;private static final int CENTERJSAMPLE = 128;private static final int SCALEBITS = 16;private static final int CBCR_OFFSET = CENTERJSAMPLE << SCALEBITS;private static final int ONE_HALF = 1 << (SCALEBITS - 1);private static final int[] rgb_ycc_tab = new int[TABLE_SIZE];static { // rgb_ycc_start    for (int i = 0; i <= JSAMPLE_SIZE; i++) {        rgb_ycc_tab[R_Y_OFFSET + i] = FIX(0.299) * i;        rgb_ycc_tab[G_Y_OFFSET + i] = FIX(0.587) * i;        rgb_ycc_tab[B_Y_OFFSET + i] = FIX(0.114) * i + ONE_HALF;        rgb_ycc_tab[R_CB_OFFSET + i] = -FIX(0.168735892) * i;        rgb_ycc_tab[G_CB_OFFSET + i] = -FIX(0.331264108) * i;        rgb_ycc_tab[B_CB_OFFSET + i] = FIX(0.5) * i + CBCR_OFFSET + ONE_HALF - 1;        rgb_ycc_tab[R_CR_OFFSET + i] = FIX(0.5) * i + CBCR_OFFSET + ONE_HALF - 1;        rgb_ycc_tab[G_CR_OFFSET + i] = -FIX(0.418687589) * i;        rgb_ycc_tab[B_CR_OFFSET + i] = -FIX(0.081312411) * i;    }}static void rgb_ycc_convert(int[] argb, int width, int height, byte[] ycc) {    int[] tab = LibJPEG.rgb_ycc_tab;    final int frameSize = width * height;    int yIndex = 0;    int uvIndex = frameSize;    int index = 0;    for (int y = 0; y < height; y++) {        for (int x = 0; x < width; x++) {            int r = (argb[index] & 0x00ff0000) >> 16;            int g = (argb[index] & 0x0000ff00) >> 8;            int b = (argb[index] & 0x000000ff) >> 0;            byte Y = (byte)((tab[r + R_Y_OFFSET] + tab[g + G_Y_OFFSET] + tab[b + B_Y_OFFSET]) >> SCALEBITS);            byte Cb = (byte)((tab[r + R_CB_OFFSET] + tab[g + G_CB_OFFSET] + tab[b + B_CB_OFFSET]) >> SCALEBITS);            byte Cr = (byte)((tab[r + R_CR_OFFSET] + tab[g + G_CR_OFFSET] + tab[b + B_CR_OFFSET]) >> SCALEBITS);            ycc[yIndex++] = Y;            if (y % 2 == 0 && index % 2 == 0) {                ycc[uvIndex++] = Cr;                ycc[uvIndex++] = Cb;            }            index++;        }    }}static byte[] compress(Bitmap bitmap) {    int w = bitmap.getWidth();    int h = bitmap.getHeight();    int[] argb = new int[w * h];    bitmap.getPixels(argb, 0, w, 0, 0, w, h);    byte[] ycc = new byte[w * h * 3 / 2];    rgb_ycc_convert(argb, w, h, ycc);    argb = null; // let GC do its job    ByteArrayOutputStream jpeg = new ByteArrayOutputStream();    YuvImage yuvImage = new YuvImage(ycc, ImageFormat.NV21, w, h, null);    yuvImage.compressToJpeg(new Rect(0, 0, w, h), quality, jpeg);    return jpeg.toByteArray();}

The magic key seems to be ONE_HALF - 1 the rest looks an awful lot like the math in Skia. That's a good direction for future investigation, but for me the above is sufficiently simple to be a good solution for working around Android's builtin weirdness, albeit slower. Note that this solution uses the NV21 layout which loses 3/4 of the color info (from Cr/Cb), but this loss is much less than the errors created by Skia's math. Also note that YuvImage doesn't support odd-sized images, for more info see NV21 format and odd image dimensions.

Please use the following method:

public String convertBitmaptoSmallerSizetoString(String image){    File imageFile = new File(image);    Bitmap bitmap = BitmapFactory.decodeFile(imageFile.getAbsolutePath());    int nh = (int) (bitmap.getHeight() * (512.0 / bitmap.getWidth()));    Bitmap scaled = Bitmap.createScaledBitmap(bitmap, 512, nh, true);    ByteArrayOutputStream stream = new ByteArrayOutputStream();    scaled.compress(Bitmap.CompressFormat.PNG, 90, stream);    byte[] imageByte = stream.toByteArray();    String img_str = Base64.encodeToString(imageByte, Base64.NO_WRAP);    return img_str;}

Below is my Code:

public static String compressImage(Context context, String imagePath){    final float maxHeight = 1024.0f;    final float maxWidth = 1024.0f;    Bitmap scaledBitmap = null;    BitmapFactory.Options options = new BitmapFactory.Options();    options.inJustDecodeBounds = true;    Bitmap bmp = BitmapFactory.decodeFile(imagePath, options);    int actualHeight = options.outHeight;    int actualWidth = options.outWidth;    float imgRatio = (float) actualWidth / (float) actualHeight;    float maxRatio = maxWidth / maxHeight;    if (actualHeight > maxHeight || actualWidth > maxWidth) {        if (imgRatio < maxRatio) {            imgRatio = maxHeight / actualHeight;            actualWidth = (int) (imgRatio * actualWidth);            actualHeight = (int) maxHeight;        } else if (imgRatio > maxRatio) {            imgRatio = maxWidth / actualWidth;            actualHeight = (int) (imgRatio * actualHeight);            actualWidth = (int) maxWidth;        } else {            actualHeight = (int) maxHeight;            actualWidth = (int) maxWidth;        }    }    options.inSampleSize = calculateInSampleSize(options, actualWidth, actualHeight);    options.inJustDecodeBounds = false;    options.inDither = false;    options.inPurgeable = true;    options.inInputShareable = true;    options.inTempStorage = new byte[16 * 1024];    try {        bmp = BitmapFactory.decodeFile(imagePath, options);    } catch (OutOfMemoryError exception) {        exception.printStackTrace();    }    try {        scaledBitmap = Bitmap.createBitmap(actualWidth, actualHeight, Bitmap.Config.RGB_565);    } catch (OutOfMemoryError exception) {        exception.printStackTrace();    }    float ratioX = actualWidth / (float) options.outWidth;    float ratioY = actualHeight / (float) options.outHeight;    float middleX = actualWidth / 2.0f;    float middleY = actualHeight / 2.0f;    Matrix scaleMatrix = new Matrix();    scaleMatrix.setScale(ratioX, ratioY, middleX, middleY);    assert scaledBitmap != null;    Canvas canvas = new Canvas(scaledBitmap);    canvas.setMatrix(scaleMatrix);    canvas.drawBitmap(bmp, middleX - bmp.getWidth() / 2, middleY - bmp.getHeight() / 2, new Paint(Paint.FILTER_BITMAP_FLAG));    if (bmp != null) {        bmp.recycle();    }    ExifInterface exif;    try {        exif = new ExifInterface(imagePath);        int orientation = exif.getAttributeInt(ExifInterface.TAG_ORIENTATION, 0);        Matrix matrix = new Matrix();        if (orientation == 6) {            matrix.postRotate(90);        } else if (orientation == 3) {            matrix.postRotate(180);        } else if (orientation == 8) {            matrix.postRotate(270);        }        scaledBitmap = Bitmap.createBitmap(scaledBitmap, 0, 0, scaledBitmap.getWidth(), scaledBitmap.getHeight(), matrix, true);    } catch (IOException e) {        e.printStackTrace();    }    FileOutputStream out = null;    String filepath = getFilename(context);    try {        out = new FileOutputStream(filepath);        scaledBitmap.compress(Bitmap.CompressFormat.JPEG, 80, out);    } catch (FileNotFoundException e) {        e.printStackTrace();    }    return filepath;}public static int calculateInSampleSize(BitmapFactory.Options options, int reqWidth, int reqHeight) {    final int height = options.outHeight;    final int width = options.outWidth;    int inSampleSize = 1;    if (height > reqHeight || width > reqWidth) {        final int heightRatio = Math.round((float) height / (float) reqHeight);        final int widthRatio = Math.round((float) width / (float) reqWidth);        inSampleSize = heightRatio < widthRatio ? heightRatio : widthRatio;    }    final float totalPixels = width * height;    final float totalReqPixelsCap = reqWidth * reqHeight * 2;    while (totalPixels / (inSampleSize * inSampleSize) > totalReqPixelsCap) {        inSampleSize++;    }    return inSampleSize;}public static String getFilename(Context context) {    File mediaStorageDir = new File(Environment.getExternalStorageDirectory()            + "/Android/data/"            + context.getApplicationContext().getPackageName()            + "/Files/Compressed");    if (!mediaStorageDir.exists()) {        mediaStorageDir.mkdirs();    }    String mImageName = "IMG_" + String.valueOf(System.currentTimeMillis()) + ".jpg";    return (mediaStorageDir.getAbsolutePath() + "/" + mImageName);}