Images, Loading

This page discusses loading an image file in Java.

The Pumpernickel codebase includes classes that combine the improved performance of an ImageProducer/ImageConsumer model with the easy creation of a BufferedImage.

Context

As a basic refresher: an "image" is usually one of two things in Java:

1. A java.awt.image.BufferedImage is an Image subclass that keeps all of its pixels in memory. The pixel data is always immediately available (although grabbing the raw pixels can be a little hard to navigate).
2. If you don't have a BufferedImage, then you should think of your image as simply an abstract java.awt.Image. In reality this is (almost?) always going to be a sun.awt.image.ToolkitImage. This object may represent an image that is not loaded yet, so it may not immediately know its width or height yet. (And when fully loaded: a ToolkitImage is backed a BufferedImage (via a ImageRepresentation object).) Note: the call graphics2D.drawImage(image) can simply return false if you pass it an image that isn't fully loaded yet.

You should not create your own java.awt.Image subclass. Someone (the name is obscured) back in 2002 wrote:

Yes, we need to update the documentation in java.awt.Image to state that [subclassing Image] is not allowed.

I found this out the hard way when I tried creating my own Image subclass. If you call graphics2D.drawImage(image) that method simply always returns false for custom subclasses. (I tried raising this topic on the lib client mailing list, but it didn't spark any interest.)

Instead you can create your own ImageProducer, then call Image myImage = Toolkit.getDefaultToolkit().createImage(myImageProducer);. This creates a ToolkitImage for you.

Preexisting Support

ImageIO

Code Sample

public BufferedImage load(URL url) throws Exception {
    return ImageIO.read(url);
}

Discussion

The ImageIO class stands out as offering the simplest interface (at one line). Also returning a copy of the pixel data is helpful. But this is also the slowest model.

MediaTracker

Code Sample

public BufferedImage load(URL url) throws Exception {
    Image image = Toolkit.getDefaultToolkit()
            .createImage(url);

    MediaTracker mediaTracker = new MediaTracker(new Label());
    mediaTracker.addImage(image, id);
    mediaTracker.waitForAll();

    return null;
}

Discussion

This approach feels odd for multiple reasons:

1. It's called "Media". And the documentation states that it can support audio, but it doesn't yet. So it generally feels like this design was never realized to its original intention. (And I feel confident saying that the good folks who manage the java.desktop module are not eager to revise stable 20-year-old code.)
2. You have to construct a MediaTracker with a java.awt.Component. It doesn't have to be a displayable Component. This is because the Component's Toolkit is being used to make sure the image is ready to display on that GraphicsConfiguration. Which seems sort of helpful, but it also seems like there should be a headless way to track your images, too.

Internally the MediaTracker uses a sun.awt.image.ImageRepresentation to construct a BufferedImage. (If that (or something like it) were publicly available outside of the sun.* package I might not have any need to go down this rabbit hole at all!)

PixelGrabber

Code Sample

public BufferedImage load(URL url) throws Exception {
    Image image = Toolkit.getDefaultToolkit()
            .createImage(url);
    Dimension size = ImageSize.get(image);
    PixelGrabber grabber = new PixelGrabber(image, 0, 0, size.width, size.height, false);
    grabber.grabPixels();
    Object pixels = grabber.getPixels();
    if (pixels instanceof int[]) {
        int[] intPixels = (int[]) pixels;
        return new QBufferedImage(grabber.getColorModel(), size.width, size.height, intPixels);
    } else {
        byte[] bytePixels = (byte[]) pixels;
        return new QBufferedImage(grabber.getColorModel(), size.width, size.height, bytePixels);
    }
}

Discussion

This is slightly better because it actually produces pixels as either a byte or int array. But there are a couple of significant usage hurdles:

1. The code above uses ImageSize, which is my own creation. (It uses asynchronous ImageObserver feedback to block until the image dimensions are provided.) What if you don't know the image size? It seems weird to me that there isn't a constructor that seems to take this usage into account. I wonder if there's something (very?) important about the original authors' intent here that I'm missing.
2. There is no intuitive way to convert an array of ints into a BufferedImage. I added a special constructor to my own QBufferedImage class for this purpose. But again: it seems weird that the design encourages you to get "the pixels", but it doesn't wrap the pixels in a helpful container.

But this approach doesn't rely on a Component, so that's nice.

Also in 2003 the PixelGrabber author (Jim Graham) wrote a comment to clarify:

Side note for historical context - PixelGrabber should never have implemented the ImageConsumer interface in the first place and should have used a private helper class to do the pixel consumption, but it is too late to fix that now...

Pumpernickel

Code Sample

The ImagePixelIterator also supports a one-line invocation:

BufferedImage bi =  ImagePixelIterator.createBufferedImage(url);

Or the alternative code sample below converts all incoming pixels to the desired image type as they are delivered. (That is: this does NOT load the entire image in one image format and then convert it to an ARGB in a second pass. The conversion takes place as each set of pixels are delivered - and they're usually delivered one row at a time.)

public BufferedImage loadARGB(URL url) throws Exception {
    return ImagePixelIterator.createBufferedImage(url, ImageType.INT_ARGB);
}

Comparison

The ImagePixelIterator, MediaTracker and PixelGrabber all rely on the same basic mechanism: they attach an ImageConsumer to an ImageProducer.

The ImagePixelIterator simply goes one step farther and wraps the pixel data it receives in a QBufferedImage.

Performance

I tasked each model with loading a 1,200 x 800 JPG 50 times. The execution time resembles:

And remember this is for 50 consecutive operations. So really ImageIO (the slowest) loads this "medium" sized image in about 75 milliseconds. In my opinion that's not bad. If the user selected this file with a file dialog: the file dialog probably took longer to open/close.

So if you're evaluating performance of loading one image at a time: I'd recommend sticking with ImageIO for simplicity's sake. But if you're loading hundreds of images during startup, or processing batch image operations somehow: it may be worth exploring how to maximize performance with one of the other three models.

Future Research

I'm wrapping my research here for now. But depending on your needs this is obviously something you could spend days (or weeks) researching.

If I pick this back up someday, I want to be sure to look at discussions like this one. This has it all: custom ImageReaders, a reference to turbo-jpeg, and references to deprecated sun.* classes.