Color

C

o

l

o

r

Dexter Studios R&D

Wanho Choi

Light Spectrum

• Light is composed of electromagnetic waves.

Visible Spectrum

• Not all these waves are visible to the naked human eye. (visible range: 380 nm ~ 780 nm)

Color from Light

• Isaac Newton’s experiment (1966)

• Pure colors (or spectral colors) cannot split into more colors.

Why does an apple appear red?

• An apple absorbs the visual wavelengths and only reflects the red ones.

• When this light enters the eye, the brain then perceives this stimulus as a color.

Three Requirements for Color

light source

reflector (object)

sensor (eye)

Color Perception

• Light(빛) ➠ cornea(각막) ➠ retina(망막) ➠ light receptor(광수용체) ➠ optic nerve(시신경) ➠ brain(뇌) • Color is the result of visual perception of light between 380 ~ 780 nm.

http://www.nanosysinc.com/dot-color-archive/2013/07/16/how-much-color-gamut-do-displays-really-need-part-2-how-we-perceive-color

Color Perception

• Human color perception is affected by various factors - Nearby colors (주변색)

- Adaptation to previous views (이전색에 대한 적응) - State of mind (인지시의 정신 상태)

http://www.psy.ritsumei.ac.jp/~akitaoka/shikisai2005.html

Structure of Human Eye

Photoreceptor Cells

Photoreceptor Cells

• 간상 세포 (rod cell) • 원기둥 모양 (cylindrical shape) • 평균 9천만개 (avg. 90,000,000 cells) • 명암을 구별 (monochromatic vision) • 약한 빛을 감지 (night-vision) • 원추 세포 (cone cell) • 원뿔 모양 (conical shape) • 평균 6백만개 (avg. 6,000,000 cells) • 색상을 구별 (color vision) • 강한 빛을 감지 (day-vision)

Photoreceptor Cells

Photoreceptor Cells

rods

Photoreceptor Cells

rods

cones

Three Types of Cone Cells

• Each kind has a different spectral response.

http://en.wikipedia.org/wiki/Photoreceptor_cell http://www.achromatopsia.info/color-blindness/

Three Types of Cone Cells

• That is why we need three parameters (trichromatic values) to describe a color.

Tristimulus Human Vision

Tristimulus Human Vision

• Each kind has a different spectral response.

- S-cone (β-cone) : short wavelength

- M-cone (γ-cone) : medium wavelength - L-cone (ρ-cone) : long wavelength

• (# of S-cones) < (# of M-cones), (# of L-cones)

• Luminance:

luminance (Y)

Y (

λ

)

= L(

λ

)

+ M (

λ

)

+ S(

λ

)

http://www.cyberphysics.co.uk/Q&A/KS5/medical/eye/Q7.html

Tristimulus Human Vision

• Sensitive to wavelengths from 380-780 nm

• Yellow-green is perceived as being most luminous.

luminance (Y)

Tristimulus Human Vision

• Green wavelength of light appear to be lighter than similar amounts (photons) of blue or red.

luminance (Y)

How do we see color?

Colorimetry

• The science of measuring colors

• Measuring colors is difficult since every person sees color a little differently. • So, colorimetry measures how an average person will see the color instead.

Color Matching Experiment

• A tedius series of experiments with actual human observers by Wright & Guild (respectively) • Tristimulus concept based experiments

• Three primaries: red (700 nm), green (546.1 nm), blue (435.8 nm)

Color Matching Experiment

• A tedius series of experiments with actual human observers by Wright & Guild (respectively) • Tristimulus concept based experiments

• Three primaries: red (700 nm), green (546.1 nm), blue (435.8 nm)

Color Matching Experiment

• Standard observer: average healthy person's sensitivity to lights and colors with 2° FOV • Due to the non-even distribution of cones, the results depend on the observer's FOV.

• Re-experiment (1964) with 10° FOV (most widely used today)

Color Matching Experiment

• Almost all colors can be visually matched by the combinations of three different lights.

Color Matching Functions

• At each wavelength, the ordinates show the amounts of the three primaries required by the standard observer to match a light of that wavelength.

• The first defined quantitative link between physical pure colors (i.e. wavelengths) in the

electromagnetic visible spectrum, and physiological perceived colors in human color vision

Color Matching Functions

• In some cases, reddish tone is not vanished even thought red light is not used.

• A negative amount of a certain primary is needed for a match. (just mathematically)

If C

≠ aR + bG + cB for all a,b,c ≥ 0

But, C

+ aR = bB + cC for any a,b,c ≥ 0

∴C = −aR + bG + cB for any a,b,c ≥ 0

http://en.wikipedia.org/wiki/CIE_1931_color_space

CIE RGB vs CIE XYZ

• Negative values are really hard to deal with! (It is impossible to set negative value for devices.) • Another analytical matching functions needs to be developed with nowhere negative.

CIE 1931 XYZ CIE 1931 RGB

CIE 1931 XYZ Color Space

• The primary colors are not real colors (They cannot be generated with any light spectrum.) • It encompasses all color sensations that an average person can experience.

• It serves as a standard reference against which many other color spaces are defined.

Meaning of X, Y, and Z

X

=

[

x (

λ

)

⋅ R(

λ

)

⋅ L(

λ

)

]

d

λ

380 780

∫

Y

=

[

y(

λ

)

⋅ R(

λ

)

⋅ L(

λ

)

]

d

λ

380 780

∫

Z

=

[

z (

λ

)

⋅ R(

λ

)

⋅ L(

λ

)

]

d

λ

380 780

∫

R(λ): light energy from reflector L(λ): light energy from light source CIE XYZ spectral sensitivity

SIGGRAPH 2009 Courses: Color Imaging

Meaning of X, Y, and Z

• The mathematically reformulated tristimulus values with non-negative values

• These values do not directly correspond to red, green, and blue, but are approximately so. • Y: luminance factor (The Y-curve is similar to the luminous efficiency curve.)

CIE xyY Color Space

• 3D XYZ color space → 2D xy color space with normalization and fixed Y coordinate

x

= X / (X +Y + Z)

y

= Y / (X +Y + Z)

z

= Z / (X +Y + Z)

∴ x + y + z = 1

xyY

chromaticity luminance (fixed as 100)

CIE xyY Color Space

plane : x

+ y + z = 1

CIE 1931 Chromaticity Diagram

• Tongue-shaped or horseshoe-shaped figure

• All of the visible spectrum to the average person • Non-negative values of x and y

• Spectral locus : curved edge (single wavelength colors) • Line of purples: the straight edge on the lower part

• White point: (x,y) = (1/3, 1/3)

CIE 1931 Chromaticity Diagram

• Line: all possible colors by mixing two colors

• Triangle: all possible colors by mixing three colors

• But, 0.5A+0.5B is not the midpoint of the line segment AB.

MacAdam Ellipses

• Indistinguishable color regions

• A distance on the xy chromaticity diagram does not

correspond to the degree of difference between two colors.

• Euclidean distance in x,y not a good metric for perceptual similarity.

Metamerism

• The same perceived color can be produced by different combinations of various colors.

• Even though the spectral components of two lights are obviously different, to our eyes they may appear as the same color.

Metamerism

• Metamerism occurs when two samples appear identical under one set of viewing conditions, but not under another set of conditions. When this occurs, we call the two samples a

metameric pair.

Cons of XYZ Color Space

• Primaries are imaginary (non-intuitive)

• One cannot build a display with such primaries. (almost possible, but expensive): impractical • A lot of XYZ values which cannot correspond to physical colors: wasteful

• Hence we cannot work with just CIE XYZ.

Color Space

• An abstract model which describes the range of colors • A n-dimensional space (typically, n=3)

• A useful method to understand the color capabilities of a particular digital device or file

- It represents what a camera can see, a monitor can display or a printer can print, and etc.

Some Confusing Terms

• Color model: a mathematical model describing the way colors can be represented as n-tuples

- RGB, CMYK, HSV, HSL, etc.

• Color space: specific values for the primaries

- CIE XYZ, CIE LUV, sRGB, Adobe RGB, Apple RGB, REC. 709 RGB, ACES RGB, etc.

RGB Color Model

• R: red, G: green, B: blue • An additive color model

• It is easy to understand and use. • A device-dependent color model

- Different devices detect or reproduce a given RGB value differently.

RGB Cube

• A RGB color model based 3D color space

CIE Color Spaces

• CIE 1931 XYZ (aka "CIE 1931”)

- The 1st color space based on measurements of human color perception, and the basis for almost all others

• CIELUV

- A modification of "CIE 1931 XYZ" to display color differences more conveniently

• CIELAB (or L*a*b* or Lab)

- The more linear color space than others. Perceptually linear means that a change of the same amount in a color value

should produce a change of about the same visual importance.

• CIEUVW

sRGB Color Space

• A color space developed by both HP and Microsoft in 1996 • s: standard

• Standard color space for monitors, printers, and the internet

Adobe RGB Color Space

• An RGB color space developed by Adobe Systems, Inc. in 1998 • To supplement for cyan-green loss in sRGB

• It was designed to encompass most of the colors achievable on CMYK color printers. • Adequate for representing green forest, blue sky, etc.

sRGB vs Adobe RGB

Color Gamut

• A subset of colors which can be represented within a given color space or by a device

Color Gamut Comparison

Raw File

• A original file produced by a digital sensor (camera, scanner, etc.) • Untouched raw information

• No compressed, no encoding, so no data loss

Color Depth

http://photography.tutsplus.com/articles/bit-depth-explained-in-depth--photo-8514 http://www.theasc.com/magazine/april05/conundrum2/image11.html

Color Depth

https://documentation.apple.com/en/finalcutpro/usermanual/index.html#chapter=52%26section=7%26tasks=true https://jackhaywood.wordpress.com/y12/new-media/unit-19/raster-vs-vector/

Gradation vs Dynamic Range

• Gradation: the gradual change between the largest and smallest values • Dynamic range: the ratio between the largest and smallest values

poor gradation rich gradation high dynamic range low dynamic range

Scene-Referred vs Output-Referred

Scene-Referred vs Output-Referred

• A transformation from scene-referred to output-referred

Tone Mapping

HDRI

LDRI

LDR

Linearity

• Additivity

• Homogeneity

f(x

+ y) = f(x) + f(y)

f(ax)

= af(x) for all a

It’s predictable!

Gamma

• Non-linear relationship between input and output

V

_out

= V

_in

γ

http://www.dfstudios.co.uk/articles/programming/image-programming-algorithms/image-processing-algorithms-part-6-gamma-correction/

γ

< 1

γ

= 1

γ

> 1

1 1 0 0

Non-linear Human Vision

• Human eyes do not respond linearly to light.

- The human eye is more sensitive to small variation of light in a dark environement than in a bright one (Weber’s law)

• Digital sensors (digital cameras) respond linearly to light.

- When twice the number of photons hit the sensor, it receives twice the signal.

http://www.cambridgeincolour.com/tutorials/gamma-correction.htm http://www.peachpit.com/articles/article.aspx?p=1709190&seqNum=3

Gamma Encoding

• A technique to optimize the usage of bits when encoding an image using logarithm

• Use more bits for dark range!

• Gamma encoded image: .jpg, .tiff, etc.

• Gamma compression

http://www.cambridgeincolour.com/tutorials/gamma-correction.htm http://www.wildlifeinpixels.net/blog/gamma-encoding/

Non-linear Output Device

60

128

100

160

200

Non-linear Output Device

• Output devices (CRT/LCD monitors, prints) also have non-linearity in their color response. • Every monitor has different response curve.

• RGB(128,128,128) is not half as bright as RGB(255,255,255).

http://filmicgames.com/archives/299 http://filmicgames.com/archives/299

Gamma Correction

• Gamma encoding was developed to compensate for the

input–output characteristic of monitors.

• Altering the input signal by gamma compression can

cancel this nonlinearity (monitor transfer function).

Gamma Correction

Linear Workflow

• Most of renderer works in linear space internally.

Linear Workflow

• Most of renderer works in linear space internally.

• Most of texture images are stored using non-linear encoding (gamma applied).

wrong

Linear Workflow

• Most of renderer works in linear space internally.

• Most of texture images are stored using non-linear encoding (gamma applied).

right

Color Management

food industry

retail

media

Color Management

• Color pipeline: a set of color transformations

• Well-defined color pipeline = rigorous transformation at each stage • Goal

- The seamless intercutting of VFX and non-VFX shots

- The consistent color across a range of devices within a pipeline

• During the visual effects process, the display transform is never baked

into the imagery except for “throw-away” deliveries such as editorial output or rough preview screenings.

Color Management

R 205 G 012 B 005 R 205 G 012 B 005 R 205 G 012 B 005 device 1 device 2 source device R ? G ? B ? R 205 G 012 B 005 device 2 source device

Introduction to Color Management for Film and TV

R 205 G 012 B 005

device 1 device 2

Color Space Transform

• A transformation between two different color spaces (color space conversion)

′

R

= aR + bG + cB

′

G

= dR + eG + fB

′

B

= gR + hG + iB

′

R

′

G

′

B

⎡

⎣

⎢

⎢

⎢

⎤

⎦

⎥

⎥

⎥

=

a b

c

d

e

f

g h

i

⎡

⎣

⎢

⎢

⎢

⎤

⎦

⎥

⎥

⎥

R

G

B

⎡

⎣

⎢

⎢

⎢

⎤

⎦

⎥

⎥

⎥

linear color space linear color space

Color Space Transform

• A transformation between two different color spaces (color space conversion)

• Since a color space conversion may cause some errors such as distortion, interpolation,

clipping, round-off, etc, you must minimize the use of color space.

′

R

= aR + bG + cB

′

G

= dR + eG + fB

′

B

= gR + hG + iB

′

R

′

G

′

B

⎡

⎣

⎢

⎢

⎢

⎤

⎦

⎥

⎥

⎥

=

a b

c

d

e

f

g h

i

⎡

⎣

⎢

⎢

⎢

⎤

⎦

⎥

⎥

⎥

R

G

B

⎡

⎣

⎢

⎢

⎢

⎤

⎦

⎥

⎥

⎥

Some Transformation Matrices

• The RGB values must be linear and in the nominal range [0.0, 1.0].

CIE Standard Illuminant D65

• One of the commonly used standard illuminants defined by CIE (white point: 6500 K) • D series: standard illumination conditions at open-air in different parts of the world • Midday sun in Western Europe / Northern Europe (daylight illuminant)

• There are no actual D65 light sources, only simulators.

• D65 should be used in all colorimetric calculations requiring representative daylight unless there are specific reasons for using a different illuminant.

CIE Standard Illuminant D65

CIE Standard Illuminant D65

Color Temperature

LUT

• Look-Up Table (pre-calculated sets of data)

• An excellent technique for optimizing the evaluation of functions that are expensive to compute and inexpensive to cache

• For data requests that fall between the table's samples, an interpolation algorithm can generate reasonable approximations by averaging nearby samples.

y

= x

3

LUT

• Directly applying multiple sophisticated color transforms to high-resolution imagery is heavy.

• Consider an analytical color operator, f(x), applied to an 8-bit grayscale image. • The naive implementation would be to evaluate f(x) for each pixel.

• However, one may observe that no matter how complex the function, it can evaluate to only one of 255 output values (corresponding to each unique input).

• Thus, an alternate implementation would be to tabulate the function's result for each possible input value, then to transform each pixel at runtime by looking up the stored solution.

LUT in Color Correction Pipeline

LUT in Color Correction Pipeline

LUT in Color Correction Pipeline

• LUTs are also useful when wanting to separate the calculation of a transform from its application. For example, in color pipelines it is often useful to bake a series of color

transforms into a single lookup table, which is then suitable for distribution and re-use, even in situations where the original data sets are not appropriate for distribution.

•

ACES

• Academy Color Encoding System (or Space, or Specification)

• A color image encoding system proposed by the Academy of Motion Picture Arts & Sciences

Why ACES?

• Until recently moving a project between facilities often resulted in color changes caused by

different display devices or even a number of the same devices calibrated differently. Also a

look developed result would be viewed differently when played on new display devices. This is a fact of life in many workflows today and results in much time and effort spent to cope with

these color issues.

• Additionally there are new display devices appearing all the time. Flat panels, projectors and laser imaging devices all have different colorimetric properties and the list keeps growing.

Program producers quite correctly want to future-proof their projects and retain their artistic vision on any future display device.

• ACES is designed to eliminate these issues making high-end post more efficient today and allowing easier use of content in the future on new display devices.

Academy Color Encoding Specification

• A scene-referred image format with standard, future-proof color space

• A very wide color gamut (R,G,B values exceed the human visible range.) • Fixed middle gray point: (0.18, 0.18, 0.18)

• When stored on disk, no gamma (or log) is pre-applied

• A unification of scene-linear floating-point HDR workflows (16-bit half-precision) • A constrained, or limited, version of the OpenEXR file format with extra metadata • The only allowable channel layouts are stereo and non-stereo RGB(A).

Academy Color Encoding Space

ACES Workflow

• ACES enables consistent color rendition in pipelines including any combination of ACES

compliant cameras, processing and display devices both now and in the future.

ACES

space

various

input

various

output

ACES Workflow

• ACES enables consistent color rendition in pipelines including any combination of ACES

compliant cameras, processing and display devices both now and in the future.

ACES

space

various

input

various

output

IDT _RRT/ODT CGI

ACES Workflow

ACES Workflow

ACES Workflow: IDT

• Input Device Transform

• Conversion from cameras, scanners, or other image sources to ACES

• Academy Color Encoding Specification: linear RGB 16-bit floating-point format

- It remove any capture characteristics that relate to the camera, lens, sensor, or recording method and as accurately as

possible reproduce the physical light of the scene.

ACES Workflow

ACES Workflow: ACES

• With conceptually unlimited gamut and dynamic range

• ACES color space is great for doing mathematical operations in. • It is almost impossible to hit the walls and clip or limit color.

ACES Workflow: LMT

• Look Modification Transform • Optional

• Once images are in ACES color space, the LMT provides a way to customize a starting point for color correction. The LMT doesn't change any image data whereas an actual grade works directly on the ACES pixels. An example of a LMT would be a “day for night” or “bleach

ACES Workflow

ACES Workflow: RRT

• Reference Rendering Transform

• Conversion from ACES (scene-referred) to OCES (output-referred)

- OCES: The output referred color space created by applying the RRT to ACES. It is still an idealized color space, and needs

an ODT to look correct on any real world display.

• It accommodates for the difference between outdoor and cinema viewing environments. • It applies color adjustments associated with pleasing image reproduction.

ACES Workflow: ODT

• Output Device Transform

• ODTs maps the image from the high dynamic range of OCES to an ideal display format such as Rec. 709.

• ODTs are applied to prepare the images for display on other devices. • Each display standard will need its own ODT.

• As with IDTs, the ODTs may be supplied by device vendors, software vendors, and AMPAS itself.

OCIO

• OpenColorIO

• An open-source color pipeline created sponsored by Sony Picture Imageworks (SPI)

• It enables color transforms and image display to be handled in a consistent manner across

multiple graphics applications in cinematic color pipeline.

• New profiles are easily created from the atomic operators common to post-production color

processing (1D LUTs, 3D LUTs, HDR processing conversions, etc), and can then be used (and shared) just as easily as the included defaults.