Graphics Software Development for DSP

Computer graphics can be as simple as a library of functions that draw geometric shapes, such as lines, rectangles or polygons on a 2-dimensional plane, or copy pixels from one plane to another. Such planes are called bitmaps, drawing surfaces or canvases. They may represent the pixels on a visible display device (such as LCD), or they may be stored somewhere off screen, in an unseen memory area. Bitmaps have a bit depth that determines how many colors they can represent and their pixels may be defined in either the RGB or YUV color space.

The process of drawing shapes on a bitmap is called rasterization or rendering. This can be done directly by software running on the host processor (ARM®), digital signal processor (DSP) or by graphics acceleration hardware. Generally, it’s best to use an accelerator whenever possible because it will require fewer clock cycles and less power to achieve the same rendering.

In 2D graphics, copying pixels from one bitmap to another is a common operation called a BitBlt (Bit Block Transfer), which can be implemented in hardware or software. A typical 2D graphics accelerator is really a BitBlt accelerator, implemented in hardware. It is similar to DMA, but specialized to transferring data on pixel (rather than word) boundaries. BitBlt operations are the foundation of 2D graphics and this is how most graphical user interfaces (GUIs) have traditionally been built.

Simple 2D Graphics

Many applications are built entirely on relatively simple BitBlt operations. Some examples are the graphical user interfaces of Microsoft® Windows®, Linux, MacOS®, set-top boxes and mobile device platforms like Symbian OS™. Each time a user opens or drags a window in a GUI, hundreds of BitBlt operations are instantiated, so the performance of the BitBlts is important to achieve a responsive user interface. The math that is used to calculate BitBlt operations is typically all integer (no floating point), so this helps to achieve good performance.

The main limitation of BitBlt-based graphics is that they do not scale well. User interfaces are typically built for a specific display size (i.e., VGA; 640 × 480 pixels) and it’s difficult to make the graphics adaptable to different screen sizes. A good example of this is what happens to the Windows desktop when it’s changed from a high-resolution mode to VGA mode. The icons become huge and pixilated and most of them no longer fit on the display.

OMAP 2 devices feature a dedicated BitBlt accelerator which applications can access through the PVR2D API (Application Programming Interface). Under Windows Embedded® CE and Windows Mobile®, their DirectX® drivers will utilize the 2D accelerator directly.

OMAP 3 devices approach 2D acceleration differently. Instead of using BitBlt operations, OMAP 3 devices avoid using BitBlts entirely in favor of scalable 2D graphics programmed through OpenVG™ or a 3D interface, such as OpenGL® ES or DirectX Mobile (for Windows).

2D, 2.5D or 3D?

For those uninitiated to the graphics world, it can be difficult to judge whether a particular application is really implemented with 3D graphics, or if it is just making clever use of 2D, to look like 3D. For example, icons which appear to spin in 3D are often achieved on a 2D GUI by repeatedly copying a sequence of still images (BitBlts). Many games feature 3Dlooking characters and objects that really only move in two dimensions (around the screen). This is easy to achieve by applying shading effects to the objects once, then moving them about with BitBlts operations. This typically requires that the “light source” in the scene is at a fixed position; usually somewhere off of the screen. This works well for many applications.

The main criteria that necessitates 3D graphics capabilities is if the user will navigate into the scene at substantial depth, and/or the user’s view will rotate into the depth dimension (for example, the user can “turn around” and look at what’s behind him). Further, if these depth movements must be performed interactively and in real-time in order to achieve an immersive feel of “being in the scene”, this requires the shading of all the objects in the scene be rendered repeatedly to account for each change in the relative position of the user’s view and light source(s). These requirements, taken together, demand 3D graphics capabilities.

Another criterion is the need for a transformation matrix to convert the coordinate system in which the scene is built into the coordinate system of the user’s graphics display. For example, a VGA display is 640 × 480 pixels, but immersive 3D environments require a much larger range of coordinates than VGA allows. Typically, this requires coordinates with such a large dynamic range that they are best specified with either single- or doubleprecision floating-point. However, there are a few notable examples of this being done entirely with fixed-point precision (such as the first Sony PlayStation®).

Finally, 3D graphics accelerators are designed to copy bitmaps onto surfaces with any arbitrary orientation (in three dimensions) with respect to the camera (the display) and to transform the pixels so that their perspective is correct on the new shape. This is called texture mapping and it requires interpolation and filtering that is well beyond the BitBlt capabilities of 2D. The example below contrasts a typical BitBlt operation, which simply copies some portion of the source bitmap, with a texture mapping operation, which maps the bitmap to a different plane in 3D space which requires the pixels to be interpolated to achieve the correct perspective in the scene.

Of course, 3D accelerators add many more capabilities as well, like the ability to handle changes in light sources and removing surfaces which are obscured by other surfaces in the scene (culling).

Graphics Interface Standards

Read the above blog