Two lectures of material
Probably the single greatest human invention is television. Although we may have some concerns about the content of what we see on TV -- the technology of TV has completely and irrevocably changed the human race.
As with audio recording, television is designed to be perceived by biological components which may differ from person to person. However, the biological peculiarities of the human eye are much more exploited in television than in audio.
I. The physiology of vision
A. Trireceptor theory of vision -- or why we use RGB monitors
If you ask someone why red, green and blue are used in computer monitors -- the immediate answer is "Because these are the primary colors". If you then ask, "But why are these the primary colors?" -- the answer you get is that "If you mix light of these colors together you can make any color". However, if you then ask "Why is this so?" -- usually the answer is dead silence. (No doubt the person being asks the question thinks you are too weird for words ...)
However, there are some very fascinating reasons why RGB are the primary colors -- and they have nothing to do with either light ... or Fourier series. They lie in the nature of the human eye.
The major light sensing element in the human eye is the retina[1].
The retina consists of a number of different and important layers of cells. Notice however, that from an engineering viewpoint, the retina is constructed upside down. The major light receptors (the rods and cones) are at the bottom of the retina -- not at the top. Thus, light must pass through nerves, blood, and tissue before reaching the main photoreceptors. This means that some serious signal processing is required by the retina and the brain to edit out these structures in your visual perception!
There are two types of photoreceptors, rods and cones. Rods are responsible for low level light detection and are most sensitive in the blue/green. They are very light sensitive and motion sensitive, but at the cost of resolution. Rods are virtually missing in the fovea (the center of the visual field) but are scattered elsewhere throughout the eye -- thus governing peripheral vision as well as nighttime vision. At night, the fovea is very insensitive and most of the visual information is being carried by rods in the periphery of your eye. Cones carry the color information and provide higher resolution -- but at the cost of sensitivity. Cones are concentrated in the fovea, providing high resolution central daytime vision.
Rods contain a blue/green pigment called rhodopsin. Cones contain three pigments, a blue-sensitive pigment[2] (447 nm), a green sensitive pigment called chlorolabe (540 nm), and a red sensitive pigment called erythrolabe (577 nm) . These three pigments are the pigments responsible for "primary colors". Individuals missing one or two of these pigments will have various forms of color blindness. (Notice that three pigments is by no means standard. There is a species of shrimp that has 20 different photoreceptor pigments!)
The graph below indicates the wavelength dependence of the three cone pigments in the human eye[3].
The idea of trireceptor vision was worked out far before the physical mechanism of retinal pigments was understood. A common diagram for describing human color perception was developed by the International Commission on Illumination (CIE). The CIE diagram is an attempt to precisely quantify the trireceptor nature of human vision.
Color perceptions were measured by giving subjects various combinations of the three standard CIE primary colors (435.8 nm, 546.1 nm and 700 nm) and measuring their perceptions. These perceptions are plotted on an x-y diagram called the CIE Chromaticity diagram[4].
The "pure" colors lie along the outer locus of the diagram, and the center of the diagram (CIE Illuminant C) is the CIE defined "white". Thus, any point on this diagram can be uniquely indicated by an x,y value. Therefore, in some fundamental way, only two basis functions are really require to describe any particular color or hue. This fact will become very important to us in color TV.
Jumping ahead just a bit, it is interesting to compare the CIE Chromaticity diagram against commonly observed colors. In the follow graph, the CIE Chromaticity diagram is overlaid against surface colors of common paints and dyes (the gray blob) as well as the primary color triangle of the American NTSC (National Television Systems Committee) color television system and the European PAL (Phase Alternation Line rate) and SECAM (Sequential Couleur avec Memoire) systems[5].
B. Visual persistence
The human eye retains an image for a fraction of a second after it views the image. This property (called persistence of vision) is essential to all visual display technologies. The basic idea is quite simple, single still frames are presented at a high enough rate so that persistence of vision integrates these still frames into motion.
Motion pictures originally set the frame rate at 16 frames per second. This was rapidly found to be unacceptable and the frame rate was increased to 24 frames per second. In Europe, this was changed to 25 frames per second, as the European power line frequency is 50 Hz. (Just as an aside, 24 frame/second American movies are routinely broadcast at 25 Hz in Europe ... the 4% difference does not seem to bother anyone!)
When NTSC television standards were introduced, the frame rate was set at 30 Hz (1/2 the 60 Hz line frequency). Then, the rate was moved to 29.97 Hz to maintain 4.5 MHz between the visual and audio carriers. (As we will see -- this decision has lead to some problems in developing an HDTV standard.) Movies filmed at 24 frames per second are simply converted to 29.97 frames per second on television broadcasting.
Now, there is a glitch. For some reason, the brighter the still image presented to the viewer ... the shorter the persistence of vision. So, bright pictures require more frequent repetition. If the space between pictures is longer than the period of persistence of vision -- then the image flickers. Large bright theater projectors avoid this problem by placing rotating shutters in front of the image in order to increase the repetition rate by a factor of 2 (to 48) or three (to 72) without changing the actual images.
Unfortunately, there is no easy way to "put a shutter" in front of a television broadcast! Therefore, to arrange for two "flashes" per frame, the flashes are created by interlacing.
The basic idea here is that a single frame is scanned twice. The first scan includes only the odd lines, the next scan includes only the even lines. With this method, the number of "flashes" per frame is two, and the field rate is double the frame rate. Thus, NTSC systems have a field rate of 59.94 Hz and PAL/SECAM systems a field rate of 50 Hz.
Although interlacing sounds like a great idea -- a number of aberrations appear due to the fact that you really do not have a frame rate of 50/60 Hz. For example, vertically adjacent picture elements do not appear at the same time. If the scene is moving, then this creates a series of serrations on the edge of moving objects. Other aberrations include such things as misalignment (where the horizontal edges of one scan do not match with the next), and interline flicker (where slight mismatches between subsequent lines cause a shimmering effect).
The other situation that must be considered is rapid motion. If the still frame images are presented at too low a rate, rapid motion becomes jerky and odd looking. This is especially a problem in action movies -- where high speed chase scenes are common. However, as of yet, there has been little interest in converting movie projectors to either 29.97 or 30 Hz due to the large investments in such equipment.
II. Basic black and white television
In a basic black and white TV, a single electron beam is used to scan a phosphor screen. The scan is interlaced, that is -- it scans twice per photographed frame.
The information is always displayed from left to right. After each line is written, when the beam returns back to the left, the signal is blanked (remember, the standard was invented before digital video!). When the signal reached the bottom it is blanked until it returns to the top to write the next line[6,].
Conventional NTSC has 525 vertical lines. However lines number 248 to 263 and 511 to 525 are typically blanked to provide time for the beam to return to the upper left hand corner for the next scan. Notice that the beam does not return directly to the top, but zig-zags a bit[7]
The vertical scanning signal for conventional black and white NTSC is quite straightforward. It is simply a positive ramp until it is time for the beam to return to the upper left hand corner. Then it is a negative ramp during the blanked scan lines.
The horizontal scan signal is very much the same. The horizontal scan rate is 525*29.97 or 15,734 Hz. Therefore, 63.6 uS are allocated per line. Typically about 10 uS of this is devoted to the blanking line on the horizontal scan. There are 427 pixels per horizontal scan line[8] and so each pixel is scanned for approximately 125 nS. (The 427 pixel number comes from taking 1/4MHz, assuming 53.3 uS per active scan, and noting that two pixels encompass each wavelength.)
The electron beam is analog modulated across the horizontal line. The modulation then translates into intensity changes in electron beam and thus gray scale levels on the picture screen.
The combination horizontal blanking signal and synchronization pulse is quite well defined. For black and white TV, the "front porch" is 0.02 times the distance between pulses, and the "back porch" is 0.06 times the distance between pulses[9].
For color signals, a color synchronization signal called the color burst is inserted into the "back porch" of the horizontal blanking signal. Not only does this set the phase of the color signal (more on this later), it also provides a really easy way to tell if you have a black-and-white signal (no color burst) or a color signal (with a 8-10 cycle color burst)[10].
The vertical blanking signal also has a number of synchronization pulses included in it. These are illustrated below[11].
The television bandwidth is 6 MHz. The sub-carrier for the color is 3.58 MHz off the carrier for the monochrome information. The sound carrier is 4.5 MHz off the carrier for the monochrome information. There is a gap of 1.25 MHz on the low end, and 0.25 MHz on the high end to avoid cross-talk with other channels[12].
Television has a maximum frequency bandwidth of 6 MHz. This says that the highest resolution signal is something like 1/6MHz or 166.7 nS. This is consistent with a 330 element scan line with a 8.7 uS blanking time.
III. Color Television
One of the great electrical engineering triumphs was the development of color television in such a way that it remained compatible with black and white television. A major driving force behind the majority of current color TV standards was to allow black-and-white TVs to continue to be able to receive a valid TV signal after color service was in place.
The selection of the display primary colors occurred first -- due to the relatively small range of available phosphors. The camera filter characteristics were then established next in order to provide a matched system for accurate color representation. However, over the years, the camera and receiver filters have been deliberately mismatched in order to obtain higher brightness values. Typically, critical colors (such as flesh tones) are set to be accurately reproduced and non-critical colors are allowed to drift.
In the most basic form, color television could simply be implemented by having cameras with three filters (red, green and blue) and then transmitting the three color signals over wires to a receiver with three electron guns and three drive circuits.
Unfortunately, this idealized view is not compatible with the previously allocated 6 MHz bandwidth of a TV channel. It is also not compatible with previously existing monochrome receivers.
Therefore, modern color TV is carefully structured to preserve all the original monochrome information -- and just add on the color information on top. To do this, one signal, called luminance (Y) has been chosen to occupy the major portion (0-4 MHz) of the channel. Y contains the brightness information and the detail. Y is the monochrome TV signal.
Consider the model of a scene being filmed with three cameras. One camera has a red filter, one camera a green filter and one camera a blue filter. Assume that the cameras all adjusted so that when pointed at "white" they each give equal voltages. To create the Y signal, the red, green and blue inputs to the Y signal must be balanced to compensate for the color perception misbalance of the eye. The governing equation is:
This is the "monochrome" part of the TV signal. It officially takes up the first 4 MHz of the 6 MHz bandwidth of the TV signal. However, in practice, the signal is usually band-limited to 3.2 MHz in order to avoid interfering with the chrominance signal.
Two signals are then created to carry the chrominance information. One of these signals is called "Q" and the other is called "I". They are related to the R, G and B signals by:
The positive polarity of Q is purple, the negative is green. The positive polarity of I is orange, the negative is cyan. Thus, Q is often called the "green-purple" or "purple-green" axis information and I is often called the "orange-cyan" or "cyan-orange" axis information.
It turns out that the human eye is more sensitive to spatial variations in the "orange-cyan" than it is for the "green purple". Thus, the "orange-cyan" or I signal has a maximum bandwidth of 1.5 MHz and the "green purple" only has a maximum bandwidth of 0.5 MHz.
Now, the Q and I signals are both modulated by a 3.58 MHz carrier wave. However, they are modulated out of 90 degrees out of phase. These two signals are then summed together to make the C or chrominance signal.
The nomenclature of the two signals aids in remembering what is going on. The I signal is In-phase with the 3.58 MHz carrier wave. The Q signal is in Quadrature (i.e. 1/4 of the way around the circle or 90 degrees out of phase, or orthogonal) with the 3.58 MHz carrier wave.
Now, this new chrominance signal (formed by I and Q) has the interesting property that the magnitude of the signal represents the color saturation, and the phase of the signal represents the hue[13].
Now, since the I and Q signals are clearly phase sensitive -- some sort of phase reference must be supplied. This reference is supplied after each horizontal scan and is included on the "back porch" of the horizontal sync pulse.
The phase reference consists of 8-10 cycles of the 3.58 MHz signal. It is called the "color burst" and looks something like this[14].
Thus, the chrominance signal begins to take on a life of its own -- without consideration of the formation I and Q signals.
The following is a sketch of how the video signal would look for a color bar test signal[15].
The following is an alternative way of sketching the same idea[16].
In some fundamental way, I and Q signals can be thought of as basis functions for the C signal. However, they are not the only possible set of basis functions. To understand the other options, it helps to sketch the phase relations of the chrominance signals in reference to the color burst[17].
The color burst sets the entire phase of the system. The I axis is defined to be 57 degrees from the color burst. The Q axis is defined to be 90 degrees from that. If the chrominance signal is completely I (orange-cyan), then it will lie on the I axis -- if it is completely Q (green-purple) it will lie along the Q axis. If the signal is completely yellow-green, then it will lie directly on the burst axis.
Now, there are two important places to look. 90 degrees from the burst is a signal called R-Y which is the difference between the red signal and the luminance. (Originally, the R-Y signal was set at 90 degrees and then the 33 degree shift to the I-Q axes determined experimentally).
180 degrees from the burst is B-Y which is which is the difference between the blue signal and the luminance. Finally, 246 degrees from the burst is G-Y which is the difference between the green signal and the luminance.
Now, note particularly the R-Y and B-Y signals. Since the R-Y signal lies on the y axis and the B-Y signal on the x axis (with reference to the color burst), then re-extracting the vector components of these signals simply requires multiplying the chrominance by a cosine or sine wave.
For example, to extract the B-Y signal, the chrominance is multiplied by cos(wt)
If a low-pass filter is used, then the second harmonic signal can be removed, leaving only the term:
If the reference signal voltage is set at 2, then the final result is C.cos() -- which is the x-projection of C or B-Y. A similar argument goes for R-Y, except the signal is demodulated using the sine rather than the cosine. Two synchronous demodulators (and the associated low-pass filters) are needed to extract these signals.
Block diagrams of alternative ways to do this are shown below[18].
An example of a circuit to do the demodulation is given below[19].
Notice that once you have R-Y and B-Y, then G-Y can be determined by combining these two signals as:
Also notice that there is the option of using R-Y, G-Y and B-Y can be used as inputs to the color guns on the TV. Then -Y can be used as the low voltage reference -- and "BINGO" you have R, G and B happily out of the three guns. We can diagram this method (often called picture tube matrixing) by the flow chart below[20].
Alternatively, the R, G, and B signals can be derived from the difference signals (R-Y, G-Y, B-Y). This is called pre-picture tube matrixing and is shown below[21].:
Now, for some subtleties:
It turns out that the human eye is more sensitive to spatial variations in the "orange-cyan" than it is for the "green purple". Thus, the "orange-cyan" or I signal has a maximum bandwidth of 1.5 MHz and the "green purple" only has a maximum bandwidth of 0.5 MHz.
So, when the signal is transmitted, the 0-4 MHz part of the signal is the Y component, and then the C component (formed of Q and I) is shifted up to a center of 3.58 Mhz. The audio signal is shifted further up to a center at 4.5 MHz.
Now, there are some minor difficulties. If the full 1.5 MHz of the I signal is transmitted on both sidebands, then it will run into the audio signal at 4.5 MHz. Thus, only the lower sideband is transmitted between 0.5 and 1.5 MHz. The upper sideband for both the Q and I signals is limited at 0.5 MHz. Now, notice that the lower sideband of the I signal runs into the upper part of the Y signal. For this reason, sometimes the I signal is bandlimited to 0.5 MHz on both sidebands (thus reducing the spatial resolution of the "orange-cyan" signal[22].
IV. A quick overview of PAL and SECAM in relation to NTSC
There are three major TV standards used in the world today. These are the American NTSC (National Television Systems Committee) color television system, the European PAL (Phase Alternation Line rate) and the French-Former Soviet Union SECAM (Sequential Couleur avec Memoire)[23].
The largest difference between the three systems is the vertical lines. NTSC uses 525 lines (interlaced) while both PAL and SECAM use 625 lines. NTSC frame rates are slightly less than 1/2 the 60 Hz power line frequency, while PAL and SECAM frame rates are exactly 1/2 the 50 Hz power line frequency.
lines active vertical aspect horizontal frame rate
lines resolution ratio resolution
NTSC 525 484 242 4/3 427 29.94
PAL 625 575 290 4/3 425 25
SECAM 625 575 290 4/3 465 25
Summary table for existing interlaced color TV standards[24].
All three systems use the same definition for luminance:
However, the three systems do not use the same definitions for primary R, G and B colors from the CIE diagram. Summarizing the differences:
NTSC x y
R 0.67 0.33
G 0.21 0.71
B 0.14 0.08
White 0.310 0.316
PAL/SECAM x y
R 0.64 0.33
G 0.29 0.60
B 0.15 0.06
White 0.313 0.329
As you can see from the CIE drawing below, these numbers essentially translate into a slightly small color triangle -- with the largest differences in the green[25].
The color encoding principles for the PAL system are the same as those of the NTSC system -- with one minor difference. In the PAL system, the phase of the R-Y signal is reversed by 180 degrees from line to line. This is to reduce color errors that occur from amplitude and phase distortion of the color modulation sidebands during transmission.
Saying this more mathematically, the chrominance signal for NTSC transmission can be represented in terms of the R-Y and B-Y components as[26].
The PAL signal terms its B-Y component U and its R-Y component V and phase-flips the V component (line by line) as:
The U and V components in PAL have roughly the same bandwidth as the I and Q components in NTSC. Both U and V are approximately 1.3 MHz. (Recall that I is 1.5 MHz and Q is 0.5 MHz.)
In order to synchronize this phase flipping, the color burst swings + and - 45 degrees about the axis. Since the sign of the V component tracks with the phase of the color burse, this then produces the required 180 degree line by line phase flip. The U signal remains untouched[27].
SECAM is quite different from both PAL and NTSC. Although all three systems share a common luminance signal Y and difference signals R-Y, and B-Y -- the SECAM system differs very strongly from PAL and NTSC in the manner in which the signals are modulated.
In SECAM the R-Y and B-Y signals are transmitted alternately every line. (The Y signal remains on for each line). Since there is an odd number of lines on any given scan, any line will have R-Y information on the first frame and B-Y on the second.
Furthermore, the R-Y and B-Y information is transmitted on different subcarriers. The B-Y sub-carrier runs at 4.25 MHz and the R-Y subcarrier runs at 4.4 MHz. The maximum possible frequency deviations for the R-Y signal is - 506 kHz and + 350 kHz and for the B-Y signal is -350 kHz and + 506 kHz[28].
In order to synchronize the line switching, alternate R-Y and B-Y sync signals are provided for nine lines during he vertical blanking interval following the equalizing pulses after the vertical sync.
Next: HDTV