RGB formats are usually straightforward: red, green, and blue with a given pixel size. RGB24 is the most common, allowing 8 bits and a value of 0-255 per color component. RGB frame grabbing is best if your image processing and/or storage requirements call for it. RGB capture can be saved directly as a bitmap by adding a simple header. Windows XP GDI (Graphic Display Interface) and Linux GUIs work with and display RGB bitmaps.

The problem with RGB, however, is that it is not the best mapping for representing visual perception. YUV color-spaces are a more efficient coding and reduce the bandwidth more than RGB capture can. Most video cards, therefore, render directly using YUV or luminance/chrominance images. The most important component for YUV capture is always the luminance, or Y component. For this reason, Y should have the highest sampling rate, or the same rate as the other components. Generally, 4:4:4 YUV format (equal sample per component) is a waste of bandwidth because the chrominance can be adequately sampled at half the sample rate of the luminance without the eye being able to notice the difference. YUV 4:2:2 and 4:2:0 formats are generally used with the preference on 4:2:2. 4:2:2 means the chrominance components are sampled horizontally at half the rate of the luminance. 4:2:0 means the Cb (blue chrominance) and Cr (red chrominance) components are sub-sampled at a factor of 2 in the vertical horizontal directions.

Capturing to YUV (or YCrCb), besides being more efficient, is best when the images are to be rendered by a standard video display directly. Additionally, some image compression algorithms, such as JPEG, directly support YUV, so there is no need for RGB conversion. In the YCrCb model, 0x80 is the black level for Cr and Cb, and 0x10 is the black level for Y.

Most DirectX 8, 9, and 10 hardware supports direct display of YUV images. With the correct format, you can render directly to the video card without transforming the image with MMX instructions or other (preferably optimized) code.

The most common video cards support packed YUV mode, YUY2 (4:2:2) and/or NV12 (4:2:0) natively. The byte ordering for YUY2 is as follows:

YUY2 format (4:2:2 = width * height Y values, width * height / 2 Cr,Cb values = 2 * W * H total bytes)

NV12 is a quasi-planar format and has all the Y components first in the memory, followed by an array of UV packed components. For a 640x480 NV12 image, the layout is as follows:

NV12 format( 640 * 480 Y values, 640 * 480 / 4 Cr and Cb values = 3/2 H * W total )

Many hardware capture boards are most efficient using planar YUV mode. They capture YUV on a plane-by-plane basis. Sensoray's 2255, for example, can capture directly into 4:2:2 YUV422P format. (It also supports YUY2 and RGB formats in the SDK.) The planar format is advantageous when the final YUV format is not known, because each component can be directly referenced as an array. A separate pointer can simply point to each of the component arrays. The packed formats do not allow this. Additionally, from the planar format, it is trivial to create the packed YUV formats such as NV12 or YUY2 with MMX optimized instructions for DirectX VA (DirectX Video Accelerated) or SDL display in Windows or Linux respectively.

YUV planar (4:2:2), 640 x 480 image