Olympus-OM
[Top] [All Lists]

[OM] Shadow color shifts [was5D II pattern noise problem]

Subject: [OM] Shadow color shifts [was5D II pattern noise problem]
From: Moose <olymoose@xxxxxxxxx>
Date: Thu, 03 Dec 2009 03:24:04 -0800
Chuck Norcutt wrote:
> I could see that it was wrong without pushing since what should have been 
> varying dark gray values were almost all zero immediately after opening the 
> file in ACR.  There is clearly an ACR bug and it occurs immediately on 
> opening the file... no alterations required.  The question is whether this is 
> limited to the 5DMkII.
>   

I've been reading, testing and thinking. I think I understand what's 
happening. whether I can explain it clearly or not ...

First, take a look at the two histograms under the heading "Linear 
Histogram", about 30% down this page. 
<http://www.guillermoluijk.com/tutorial/dcraw/index_en.htm>

Fig. 5 shows the linear data from  a sensor. F-stops involve 
doubling/halving the amount of light for each full stop, the legend on 
the bottom shows how that plays out. Let me put it in numbers, adjusted 
from the 12-stop range from a 12 bit sensor to the 14 bits of the 5DII. 
The top stop has a range of 8,182 values in which to store levels of 
brightness and:

14    8192
13    4096
12    2048
11    1024
10     512
 9     256
 8     128
 7      64
 6      32
 5      16
 4       8
 3       4
 2       2
 1       2

So - When the subject is underexposed by four stops, the number of 
possible values goes from 16,384 down to 2,048. Not so bad, 8 bit is 
only 256, right?

Ah, but look at the shadows. For a correctly exposed shot, there are 32 
values to hold all the data for the bottom 5 stops. No matter that it 
looks like more after gamma adjustment, it's still those few values 
up-sampled.

In the 4 stop underexposed version, data from some of the higher stops 
have to be squeezed down into those 32 values. So, potential data detail 
is lost, compared to proper exposure. This is true even if the sensor 
can perfectly discriminate the actual values way down at such low levels 
- the data is lost to the digitization process.

To the extent that the sensor loses effective precision in the dark, 
things get worse. Think of a digital meter. It's not uncommon to have 
finer resolution than precision. The meter reads to 0.01, but the 
accuracy is ±0.03. so a reading of 1.35. although it looks accurate on 
the LCD, is really somewhere between 1.32 and 1.38. When you get down to 
very low readings, the precision means values may be 100% or more off 
the indicated value.

I assume the same holds true for values out of the A/D converters in 
cameras.

A shot that is intentionally heavily underexposed has its true shadows 
recorded in only a handful of values and its lower mid-tones compressed 
into maybe 100 values. Then in the RAW converter, we try to pull those 
values back up the where they would be in a proper exposure. That means 
lots of big up interpolation.

To the extent that the sensor system is differentially non-linear 
between channels at very low light levels, the reconstituted image will 
have inaccurate color. To the extent that quantization, what might also 
be called pixelization or rounding effects as analog values were 
digitized down at the bottom, there will be color/luminance irregularities.

Why then, does Canon's DPP do a better job in this test than ACR? I 
think the answer is simple. First, you should read on down in the page 
linked above, on through the "Sensor Saturation Level" section. It may 
make your head hurt, but will give an idea why the maker of a sensor 
system, with full in-depth knowledge of its characteristics, can 
properly fine tune conversion settings for it.

Adobe, on the other hand has to set it's default conversion behavior to 
accommodate a vast array of sensor systems. Sure, they can tune settings 
defaults for each camera based on some test shots, but they can't really 
know what's going on down in the depths of each system.

I tried converting CH's sample color chart image in DPP and ACR. I then 
tried the white balance dropper on various gray steps. In both cases, 
there was a difference between the bright and darker patches. the 
lighter the patch, the more accurate looking the color.

Overall, as expected from the above, the DPP conversion was better than 
the ACR, and the difference became greater the darker you went. The 
darker ACR patches had a distinct magenta cast, Which makes me think it 
may have something to do with the effect on relative sensitivity at very 
low light levels of the use of 2 green sensors for each single red and 
blue sensor - or not.

My conclusions?

1. It's foolish to expect recovered shadows to be as color and luminance 
accurate as if they were mid-tones. The technology just doesn't support 
it. HDR image file formats use floating point decimal numbers. If future 
cameras adopt a floating point RAW format, this would be different. For 
now, we have to develop shooting techniques to accommodate these 
limitations of our cameras to our photographic purposes.

2. There probably isn't a programming error in ACR. It just doesn't know 
as much about particular sensor systems as do the manufacturers.

3. I may try using DPP for images where I care about color accuracy and 
must pull shadows up a lot. On the other hand, the default color results 
I get from ACR on normal 5D RAW files look more color accurate to me 
than the defaults from DPP and ACR gives me more and more useful 
controls. So I'll stick with ACR for most work.

Remember, most of my shooting is of natural subjects, where there is no 
absolute measure of color accuracy. In many cases, the shadows in the 
subject are dark enough that my own eyes are running mostly on B&W 
cones, so even if I had perfect color memory, I might not know what 
color those shadows 'really' were.

4. While all this is interesting to me in a theoretical way and I've 
learned a lot that may help inform my image creation and processing, I 
don't see how it has had a practical effect on the vast majority of the 
images I've processed. Even where it may have, I'm not sure how I could 
tell.

So I take a sunset shot with underexposed foreground, pull it up and get 
some magenta shift. There's a lot of magenta in the shadow areas of 
sunset scenes as I see them with my eyes. How can I tell the difference? 
Why should I care? If it looks right to eyes that have seen many 
thousands of sunsets, why worry? It it looks too magenta, I can just 
pull magenta saturation down a bit. There really is no absolute in most 
photographs.

Moose
-- 
_________________________________________________________________
Options: http://lists.thomasclausen.net/mailman/listinfo/olympus
Archives: http://lists.thomasclausen.net/mailman/private/olympus/
Themed Olympus Photo Exhibition: http://www.tope.nl/

<Prev in Thread] Current Thread [Next in Thread>
Sponsored by Tako
Impressum | Datenschutz