Rawsie - Compress to Impress

‘The first raw image compression algorithm designed for human eye and machine vision equally. Rawsie optimizes the noise encoding at the raw pixel level without compromising the signal’

Yay! ~ I finally have the Rawsie application! I have been waiting for this for two years and I can now sit down and compress all my Sony α7 III raw files. I will see how grateful my NAS and my secondary external backup units will be after I have finished the work. I have always been an advocate of compression to save space as long as there is no loss in quality. For a while now, I have been watching the space on my NAS slowly being eaten away by images from shows and studio work.

Rawsie can take a RAW file, from supported cameras, and compress them down to a Digital Negative (DNG) file without loss in quality or editing ability. You can simply drag and drop a folder of images to the application window or you can select a Lightroom catalog and choose which folders you want to compress. Rawsie will then relink the files in the catalog for you. I am using Capture One so I have just been dragging multiple folders and watching the magic work.

For comparison, the Sony α7 III .ARW files are circa 24 meg in size. When I come home from shooting a cabaret show, the SD Card can be holding 16 Gig of data or more. That is a lot to store when I have a regular show each week. Rawsie will take a look at these files and crunch them down to circa 8-10 meg per image depending on the content. This is a massive saving in file size! This also means the ingestion of the files to Capture One or Lightroom as well as backing up the files to my Nas is faster and doesn’t take up as much room either.

The downside, depending on your point of view, is the files are converted to Adobes DNG format. Some will care about that, I do not. The DNG format is open source and can be read by many applications.

File size comparisons ~ I dug out the test shots from when I first purchased the Sony 24-105 f4.0 lens and took a few quick shots along the North bank of the Thames. Simple shots, nothing fancy, a few buildings, and a couple of tourist boats on the Thames. You can immediately see from the graphs below, how much was saved by Rawsie.

The file sizes come down to below half in most cases… I could hear my NAS cheering, although that could be the fans speeding up!

Image Quality - The DNG files are fully editable and retain all the raw sensor data… although I have seen a slight colour shift when compared to the original in Capture One. This is nothing major and a quick edit sorts out the issue, but I feel that is nothing to worry about. I also noticed that Capture One does not recognize the camera type for the DNG file where the ARW file does. This again is not a great loss and is easily fixable in Capture One. I know which lens was used, so I select it for one image, then cut and past the changes to the others. The histograms do show a slight variance, any differences are barely perceptible in the main images. For the work I produce, it really doesn’t matter that much.

I threw in a few edits on the ARW file and then copied them to the DNG version. Again there was no perceptible change. I then zoomed in on both to see if there were any differences in detail, none of which I could see. So a few clicks later and everything is fine. You can click on the images below to take a closer look.

View fullsize

View fullsize

View fullsize

View fullsize

Issues ~ As I have not been using Lightroom exclusively for some time now, all my editing has been in Capture One. So there is no linkage available in the current version. So once the shots are converted and synced back into the catalog there are a number of issues:

• They are not the original files so any edits I have done are lost.

• Any colour tagging I have added, to show which of my studio shows not to use, is also lost.

• All preview files need to be rendered again.

All of the above are not too much of an issue, edits can be done again as I have a standard edit style and keep to a set way of cropping, etc… The colour tagging can be done again by checking messages from the models/performers from the day. The preview files can be rendered again without issue.

In Conclusion ~ This has already been inserted into my workflow and I have been busy converting all my ARW files to DNG format over the past few days. So how much space did I save? In total, Rawsie saved me a colossal 480 gigs of data…. nearly half a Terrabyte! It did take me a while to delete the old files and copy and back up the new DNG files to the NAS and secondary external devices. Capture One needed to delete the ARW files from the catalog and read the new DNG files, this took some time too.

There is definitely a speed increase when using the files in Capture One and Lightroom, previews are generated quickly, and cutting and pasting settings between large ranges of images is much faster too. Exporting speeds are still the same though.

This is a fantastic piece of software and has massively reduced the storage needs of my work. It does not support all cameras though, you can find a list here as well as pricing. Rawsie Supported Cameras. If you are a wedding photographer who takes thousands of photos a year or just a casual shooter, then this will save you a lot of time and money. Although it is quite expensive for a perpetual license it will start paying for itself immediately.

SpaceSaver - Move over JPEGmini!

While scanning some Black Friday deals before going to bed I stumbled across an application that could compress JPEG files in a similar fashion to JPEGmini. ‘aha! But I bet not as good as JPEGmini!’ I thought to myself… A few minutes later... Oh my god!!!!

So I was not going to bed then... What started was an hour's work experimenting with this new application called SpaceSaver which allowed a 500 image free trial. This application claims to be fast due to its modern architecture and reduces the file size of your JPEG images while protecting the image quality.

The first thing I noticed was the very clean interface, it cannot be resized but it is thinner than JPEGMini’s bulky interface. There are options you would expect from an application like this, to have multiple image sizes exported and where to have them sent, either to a new folder or to overwrite the originals. You can drag and drop a folder of images to the GUI or choose a folder by clicking on the plus button.

I like the addition of the Worldwide count of data saved where JPEGmini will only tell you your own saving. I wonder if it can tell where in the world the most space has been saved?

So I decided to have a head-to-head fight with JPEGmini and see who would come out on top. I picked out a recent shoot at the Phoenix Arts Club as the test. This has 311 JPEG images saved out at 2000 pixels on the long side, exported from Capture One. I then made a copy of the exported folder, one for JPEGmini to get its teeth into and one for SpaceSaver to chew on.

Time to queue the bar charts!

Compression Comparison / 2000px / 746 Meg Original Folder Size

OK, so the times were comparable. JPEGmini would start to work on the images as soon as you drop the folder into the GUI whereas SpaceSaver would analyze the images and then get to work so it lost a couple of seconds, but the drop of 61 Meg as a result for an extra 1 second is amazing.

Compression comparison / 5000px / 5.2 Gig Original Folder Size

So let’s hit them both with something bigger. I exported the same set of images but at full resolution, circa 5000 pixels on the long side. Yeah ok, then that’s a big difference in speed and compression!

Large resolution compression comparison

So how does Spacesaver compare with much larger resolution images? I exported three larger images taken from my portrait collection. Resolutions were circa 5000-8000 on the long side. Again the folder was copied twice for JPEGmini and SpaceSaver to work on.

@aya.lstorm_burlesque

That is incredible! That was then I recalled another thing about JPEGmini. It has a limitation of 128 megapixels. whereas SpaceSaver has no restrictions. I have two panoramic images that are way over 128 megapixels. The dimensions are circa 27000 by 5000 pixels and I have no idea why I took the images in the first place… however, I grabbed them kicking and screaming into SpaceSaver. The first was 63 meg, the second was 87 meg. SpaceSaver thought about it and compressed them down to 19 and 27 meg respectively.

In conclusion, will SpaceSaver become part of my workflow? Hell yes! It is quicker and better than JPEGmini by a long shot and also ( and for me, this is a big thing ) Spacesaver can be used in a Window command batch file so I am one step closer to automating the export, compress, pack, and delivery package system I have been working on. I only need the app to quit after the list of images has been completed, then the batch file can continue its job. Maybe in an update!

You can get a 500 image trial at https://www.getspacesaver.com/ If you use cloud-based storage or home NAS solutions then this will drastically increase its storage potential.

JPEGMINI - ENTROPY INCREASES

The JPEGmini application can take a jpeg file and compress it to around a third of its original size without any quality loss. The pro version can work with file up to 128 megapixels. Some of my larger panoramic images have been around 80 meg in size when thrown into JPEGmini, the result is a staggering 42 meg reduction in file size!

So how far can it go? As an experiment, I tried to see the benefits of continuing to run a jpeg through JPEGmini until the result reports no further compression available. The window will show how much has been removed. Once that reads 0.00 then I have my final image. So I took a portrait image from a recent shoot to test out.

• The original Raw file is circa 24 meg in size.

• I exported the Raw to a Jpeg at a full 100% quality setting. This produced a Jpeg with a file size of 17.429 KB. Just short of 17.5 meg.

• I then ran it through Jpg Cleaner to remove the metadata. Not surprisingly this removed very little, the size was now 17.417 KB.

• I then ran the Jpeg through the first pass of JPEGmini, this reduced the file down to 3.470 KB.

• I then ran the file through Jpg Cleaner each time, as I have found this removes the flag that tells JPEGmini that his already been JPEGmini’d. You can then continue to JPEGmini the file until the result reports 0.00 removed.

• So finally the resulting file was crushed to 1.838 KB, just short of a 2 meg file. If this was done to a file exported at 2000 pixels on the long side the file size would be around 500k, less than half a Meg.

• There was some quality degradation if you do this. Mostly in very fine detail, however, you can see this only when you zoom in to 100%.

So how does this equate to a full shoot package? Is it worth the extra time to keep passing the files though JPEGmini?

• So I timed the process. I exported a shoot of 244 images all at 2000px on the long side. The folder size was 717 Meg. The export took around 5 minutes.

• Throwing the folder into Jpg Cleaner took 4 seconds and removed 2.7meg. We were down to 714 Meg. The Jpg Cleaner app was left open to quickly run the sequence again once the first stage of JPEGmini was done.

• The first stage of JPEGmini took just 47 seconds to complete. Jpg Cleaner took 2 seconds to clear the JPEGmini flag from the files. The folder size was down to 135meg in size.

• The next few passes through JPEGmini will get faster. There will be an increase in files that JPEGmini cannot compress any further. These will be skipped.

• JPEGmini = 34 seconds | JpegClean = 2 seconds

• JPEGmini = 33 seconds | JpegClean = 2 seconds

• JPEGmini = 17 seconds | JpegClean = 2 seconds

• In total, the process took under 3 minutes to compress to a point of diminishing returns.

• The folder size was now down to 76 Meg in size. Just over 10% of its original size and about 65% of the JpegMini’d folder size.

There is some degradation but again you can see this when you zoom in to 100%. As these are for use on web pages and social media this is not too much of an issue. If the work was for printing then I would not attempt this secondary process.