Creating an Overlay

This chapter shows you how to write text and draw graphics on the live video images.

The source code for this sample program can be found in the samples\VB6\Creating an Overlay directory.

Setting up the Project

First of all, create a new project and add IC Imaging Control to the form as shown in the FirstSteps chapter.

Then, insert three new buttons to your form and name them cmdSelectDevice, cmdLiveVideo and cmdImageSetting. Label the buttons Device, Start and Settings respectively. These buttons will be used to select the video capture device, start and stop the live video and open the image settings dialog.

Now, implement the handler subs for the three buttons:

Private Sub cmdSelectDevice_Click()
    cmdLiveVideo.enabled = False
    cmdLiveVideo.Caption = "Start"
    cmdImageSettings.enabled = False
 
    ' Stop the live video, if it runs.
    If ICImagingControl1.DeviceValid = True Then
        If ICImagingControl1.LiveVideoRunning Then
            'Stop the timer.
            Timer1.enabled = False
            ICImagingControl1.LiveStop
            cmdLiveVideo.Caption = "Start"
        End If
    End If
 
    ' Now, call the device settings dialog to select the device.
    ICImagingControl1.ShowDeviceSettingsDialog
    
    If ICImagingControl1.DeviceValid = True Then
        cmdLiveVideo.enabled = True
        cmdImageSettings.enabled = True
    End If
End Sub

Private Sub cmdLiveVideo_Click()
    If ICImagingControl1.DeviceValid Then
        MousePointer = vbHourglass
        If ICImagingControl1.LiveVideoRunning Then
            Timer1.enabled = False
            ICImagingControl1.LiveStop
            cmdLiveVideo.Caption = "Start"
        Else
            StartLiveVideo
            cmdLiveVideo.Caption = "Stop"
        End If
        MousePointer = vbNormal
    End If
End Sub

Private Sub cmdImageSettings_Click()
    If ICImagingControl1.DeviceValid Then
        ICImagingControl1.ShowPropertyDialog
    End If
End Sub

Also, implement a sub named StartLiveVideo. It is called by cmdLiveVideo_Click and will be used later to do the initial drawings in the overlay.

Run the program: You should be able to select a video capture device and see the live video.

Using the OverlayBitmap Object

IC Imaging Control provides a special object for drawing and writing on the live video. The object is called OverlayBitmap.

The OverlayBitmap object contains a bitmap, which has the same size as the video format, or the size of a frame filter that transforms the image data before the overlay bitmap is applied. An application can draw text and graphics on this overlay bitmap. The overlay bitmap is copied to each sample of the video. A dropout color can be specified to determine, which pixels of the overlay bitmap are transparent and therefore not copied to the live video.

The overlay bitmap can only be used, if ICImagingControl.LiveStart has been called at least once after the video format has been changed. It is only visible, if the live video is running.

Enabling and disabling overlay for device, sink and display paths

Since version 3.0, IC Imaging Control allows you to select whether the overlay is drawn on live video, on grabbed and recorded images, or both.

You can use ICImagingControl.OverlayBitmapPosition to set an ORed combination of PathPositions values that specify the path positions at which overlay bitmap objects are inserted.

Enabling the overlay bitmap is a two-step process: The OverlayBitmapPosition has to be set before starting live mode and determines, at which path positions overlay operations are possible. OverlayBitmap.Enable enables or disables the application of a specific overlay bitmap while live mode is running.

Accessing the OverlayBitmap Objects

To access one of the 3 OverlayBitmap objects, use ICImagingControl.OverlayBitmapAtPath. The parameter of that property selects the position of the OverlayBitmap that is accessed.

The ICImagingControl.OverlayBitmap property returns the OverlayBitmap for the device path.

Setting up the Overlay

A program can draw on any of the overlay bitmaps once live mode has been started. Live mode has to be started first, because the actual size and color format of the overlay bitmaps can depend on frame filters, thus it is impossible to determine the formats before.

The StartLiveVideo starts live video and then draws several graphics on the 3 overlay bitmaps:

Private Sub StartLiveVideo()
    Dim ob As OverlayBitmap
    FrameCount = 0
 
    If ICImagingControl1.DeviceValid = True Then
    
        ICImagingControl1.LiveStart
        
        ' Initialize overlay bitmaps
        SetupOverlay ICImagingControl1.OverlayBitmapAtPath(PATHPOSITION_DEVICE)
        SetupOverlay ICImagingControl1.OverlayBitmapAtPath(PATHPOSITION_DISPLAY)
        SetupOverlay ICImagingControl1.OverlayBitmapAtPath(PATHPOSITION_SINK)
        
        ' Display a coordinate system on the device overlay
        DrawCoordinatesystem ICImagingControl1.OverlayBitmapAtPath(PATHPOSITION_DEVICE)
 
        ' Load a  bitmap file and display it on the display overlay
        ShowBitmap ICImagingControl1.OverlayBitmapAtPath(PATHPOSITION_DISPLAY)
        
        ' Draw overlay info boxes
        DrawOverlayInfo ICImagingControl1.OverlayBitmapAtPath(PATHPOSITION_DEVICE)
        DrawOverlayInfo ICImagingControl1.OverlayBitmapAtPath(PATHPOSITION_DISPLAY)
        DrawOverlayInfo ICImagingControl1.OverlayBitmapAtPath(PATHPOSITION_SINK)
 
        ' Enable the timer. In the timer event handler sub, the current
        ' time is displayed.
        Timer1.enabled = True
        Timer1_Timer
    End If
End Sub

The SetupOverlay sub is used to initialize each of the 3 overlays. It is called after live mode has been started.

Private Sub SetupOverlay(ob As OverlayBitmap)
 
    ' Enable the overlay bitmap for drawing.
    ob.Enable = True
 
    ' Set magenta as dropout color.
    ob.DropOutColor = RGB(255, 0, 255)
 
    ' Fill the overlay bitmap with the dropout color.
    ob.Fill ob.DropOutColor
    
    ' Print text in red.
    ob.FontTransparent = True
    ob.DrawText RGB(255, 0, 0), 10, 10, "IC Imaging Control 3.0"
    
End Sub

The overlay is enabled by calling OverlayBitmap.Enable, and a text is displayed in the upper left hand corner of the bitmap. The sub also sets a dropout color.

Dropout Color

The OverlayBitmap.DropOutColor property of the OverlayBitmap object determines, which pixel color is transparent on the live video. Each pixel that has the dropout color is not copied to the live video. The dropout color can be set with Visual Basic's RGB function. It defaults to black. For this reason, the background of the text written above is transparent. The FontBackColor property of the OverlayBitmap object is set to black as default too.

In order to be able to use black color in the overlay, we change OverlayBitmap.DropOutColor to another color, for example magenta (RGB(255,0,255)). If the application is started, only a black image with our text can be seen. The image is black, because the overlay bitmap is black and the dropout color has been changed to magenta. Therefore, the overlay bitmap must be filled with magenta. The method OverlayBitmap.Fill does this for us. When the dropout color is changed, the text background is visible as a solid black rectangle. To hide this, the property OverlayBitmap.FontTransparent can be set to True.

Drawing a Coordinate Plane

In the next step, we are going to draw a coordinate plane, consisting of lines and labels. To do this, we are going to use the DrawLine and DrawText methods of the OverlayBitmap object. Additionally, we are going to change the font used to write the labels.

We are going to implement a new sub called DrawCoordinatesystem. We want to use an 8 dpi Arial font for the labels. To achieve this, a new StdFont object Font must be created. This object has font properties like font name, font size and other font attributes. Before changing the font, we save the current font in order to be able to restore it after we have completed the coordinate plane.

The Font property of the OverlayBitmap object receives the StdFont object. Now, the new font is used for all following OverlayBitmap.DrawText calls.

The complete sub that draws the coordinate plane is implemented as follows:

Private Sub DrawCoordinatesystem(ob As OverlayBitmap)
    Dim Col As Integer
    Dim Row As Integer
    Dim i As Integer
    Dim Font As New StdFont
    Dim OldFont As StdFont
 
    Font.Name = "Arial"
    Font.Size = 8
 
    With ob
 
        ' Calculate the center of the video image.
        Col = ICImagingControl1.Width / 2
        Row = ICImagingControl1.Height / 2
        Set OldFont = .Font
        .Font = Font
 
        .DrawLine RGB(255, 0, 0), Col, 0, Col, ICImagingControl1.Height
        .DrawLine RGB(255, 0, 0), 0, Row, ICImagingControl1.Width, Row
 
        For i = 0 To Row Step 20
            .DrawLine RGB(255, 0, 0), Col - 5, Row - i, Col + 5, Row - i
            .DrawLine RGB(255, 0, 0), Col - 5, Row + i, Col + 5, Row + i
            If i > 0 Then
                .DrawText RGB(255, 0, 0), Col + 10, Row - i - 7, i / 10
                .DrawText RGB(255, 0, 0), Col + 10, Row + i - 7, -i / 10
            End If
        Next
 
        For i = 0 To Col Step 20
            .DrawLine RGB(255, 0, 0), Col - i, Row - 5, Col - i, Row + 5
            .DrawLine RGB(255, 0, 0), Col + i, Row - 5, Col + i, Row + 5
            If i > 0 Then
                .DrawText RGB(255, 0, 0), Col + i - 5, Row - 17, i / 10
                .DrawText RGB(255, 0, 0), Col - i - 10, Row - 17, -i / 10
            End If
        Next
        .Font = OldFont
 
    End With
 
End Sub

Displaying the Current Time

To display the current time on the live video, a timer is inserted in the form. It is called timer1 by default and its timer interval is set to 1000 milliseconds in the property window. The timer event handler sub is called Timer1_timer. The current time is displayed in the lower right hand corner. It has a white font with a black background. Furthermore, it has its own font. Therefore, a StdFont object must be created and used, as described above. The property OverlayBitmap.FontTransparent is set to False and OverlayBitmap.FontBackColor is set to black (RGB 0 0 0).

Private Sub Timer1_Timer()
 
    If ICImagingControl1.LiveVideoRunning = True Then
        Set ob = ICImagingControl1.OverlayBitmapAtPath(PATHPOSITION_DEVICE)
        
        Dim OldFont As StdFont
        Set OldFont = ob.Font
        
        Dim Font As New StdFont
        Font.Name = "Arial"
        Font.Bold = True
        Font.Size = 14
        ob.Font = Font
 
        ' Draw the time in the lower left corner of the video window.
        Dim Col As Integer
        Dim Row As Integer
        Col = ICImagingControl1.Width - 81
        Row = ICImagingControl1.Height - 20
 
        ' Setup the background color and drawing mode.
        ob.FontTransparent = False
        ob.FontBackColor = RGB(0, 0, 0)
 
        ' Draw the text with white color.
        ob.DrawText RGB(255, 250, 250), Col, Row, Time
        
        ' Reset the font and drawing mode.
        ob.FontTransparent = True
        ob.Font = OldFont
    End If
End Sub

OverlayUpdate Event

IC Imaging Control provides an OverlayUpdate event. This event is called after a new frame has been delivered from the video capture device. Therefore, it is possible to insert individual information on to each frame (i.e. frame counter).

The OverlayUpdate sub in the example counts the frames in the variable FrameCount. The frame counter is used to draw a growing triangle. If the frame counter is greater than 25, it is reset to 0 and the triangle is deleted. Therefore, if a solid rectangle of the same size is drawn over the triangle. The color of the rectangle must be set to the DropOutColor to make it transparent. The method DrawSolidRect provides this functionality.

The OverlayUpdate event handler is implemented as follows:

Private Sub ICImagingControl1_OverlayUpdate(ByVal Overlay As ICImagingControl3Ctl.OverlayBitmap, ByVal SampleStartTime As Double, ByVal SampleEndTime As Double)
    Dim Font As New StdFont
    Dim OldFont As StdFont
    Font.Name = "Arial"
    Font.Size = 8
 
    With Overlay
        If Overlay.PathPosition = PATHPOSITION_DISPLAY Then
        
            FrameCount = FrameCount + 1
    
            If FrameCount > 25 Then
                 FrameCount = 0
    
                ' Delete the triangle.
                .DrawSolidRect .DropOutColor, 10, _
                                              ICImagingControl1.Height - 70, _
                                              62, _
                                              ICImagingControl1.Height - 9
            End If
    
            ' Draw the new triangle line.
            .DrawLine RGB(200, 255, 0), FrameCount * 2 + 10, _
                                        ICImagingControl1.Height - 10, _
                                        FrameCount * 2 + 10, _
                                        ICImagingControl1.Height - 10 - FrameCount
    
             ' Print the current frame number:
    
             ' Set the background color to the current dropout color
             ' and make the font opaque.
             .FontBackColor = .DropOutColor
             .FontTransparent = False
    
             ' Save the current font.
             Set OldFont = .Font
    
             ' Set the new created font.
             .Font = Font
    
             ' Draw the text.
             .DrawText RGB(200, 255, 0), 70, ICImagingControl1.Height - 19, Str(FrameCount) + "  "
    
             ' Restore the previously used font.
             .Font = OldFont
             
        End If
    End With
 
End Sub

GDI Functions in Visual Basic 6

The OverlayBitmap object does not provide all functionality of the graphic device interface (GDI). Therefore, the OverlayBitmap provides a method that returns a device context. The method is called GetDC. The bitmap of the OverlayBitmap object is already selected in the returned device context. It can be used for all GDI functions like BitBlt, Ellipse etc.

To use the GDI functions, they must be declared on top of the form's source code:

' The GDI (graphics device interface) functions
' must be declared before they can be used.
Private Declare Function CreateCompatibleDC Lib "gdi32.dll" _
                          (ByVal handle As Long) As Long
 
Private Declare Function DeleteDC Lib "gdi32.dll" _
                          (ByVal handle As Long) As Long
 
Private Declare Function SelectObject Lib "gdi32.dll" _
                          (ByVal DC As Long, ByVal HGDIObj As Long) As Long
 
Private Declare Function BitBlt Lib "gdi32.dll" ( _
                              ByVal hdcDest As Long, _
                              ByVal nXDest As Long, _
                              ByVal nYDest As Long, _
                              ByVal nWidth As Long, _
                              ByVal nHeight As Long, _
                              ByVal hdcSrc As Long, _
                              ByVal nXSrc As Long, _
                              ByVal nYSrc As Long, _
                              ByVal dwRop As Long _
                              ) As Boolean

A new sub ShowBitmap is implemented to demonstrate how to render a bitmap onto the overlay bitmap. It uses the GDI function BitBlt to copy one bitmap onto another. The sub contains a variable of Visual Basic's StdPicture type. It is used to load a bitmap from file. Also two long variables are declared, that receive the device contexts. BitBlt copies the bitmap data from one device context to another device context. Very important: The bitmap sizes of the StdPicture object must be converted in to pixels, because the overlay bitmap uses pixels as its units.

After the bitmap has been loaded and its size has been converted to pixels, the device context of the OverlayBitmap object is retrieved:

Set ob = ICImagingControl1.OverlayBitmap
obDC = ob.GetDC

If an error occurs or GetDC is called before LiveStart has been called, 0 is returned. This means, the device context does not exist at this time. This has to be checked by an if statement.

Once a valid device context has been retrieved, a second, compatible device context must be created. This is done by a call to the GDI function CreateCompatibleDC. With the GDI function SelectObject, the bitmap in the StdPicture object is selected in the new device context. As soon as these steps have been completed, the BitBlt function is called. The target coordinates are so calculated that the loaded bitmap appears in the upper right hand corner of the video image.

'
' ShowBitmap
'
' This sub demonstrates how to use OverlayBitmap.GetDC to blit a bitmap
' from a file on the live video.
' The bitmap will be blitted with transparency on the live video because
' it's background color is magenta (load the image "Hardware.BMP"
' with "Paint.exe" to verify this). Magenta is the currently set
' dropout color. The used GDI graphic functions are based on pixel units.
' Therefore, the scaling functions of Visual Basic are called to get the
' size of the loaded bitmap in pixels.
'
Private Sub ShowBitmap(ob As OverlayBitmap)
    Dim Pic As New StdPicture
    Dim PicWidth As Integer
    Dim PicHeight As Integer
    Dim obDC As Long
    Dim SourceDC As Long
    Dim Col As Integer
 
    ' Load a BMP file.
    Set Pic = LoadPicture(App.Path & "\hardware.bmp")
 
    ' Transform the size of the picture from himetric to pixel units.
    PicWidth = ScaleX(Pic.Width, vbHimetric, vbPixels)
    PicHeight = ScaleX(Pic.Height, vbHimetric, vbPixels)
 
    ' Calculate a column to display the bitmap in the
    ' upper right corner of Imaging Control.
    Col = ICImagingControl1.Width - 5 - PicWidth
 
    ' Get the DC of the OverlayBitmap object.
    obDC = ob.GetDC
    If obDC <> 0 Then
        ' Create a compatible DC that is used to hold the loaded bitmap.
        SourceDC = CreateCompatibleDC(obDC)
        If SourceDC <> 0 Then
            ' Select the loaded bitmap in the source DC.
            SelectObject SourceDC, Pic.handle
 
            ' Now blit the source DC in the overlay bitmap DC.
            ' This copies the loaded bitmap to the overlay bitmap.
            BitBlt obDC, Col, 5, PicWidth, PicHeight, _
                    SourceDC, 0, 0, 13369376  ' 13369376 is SRCCOPY
 
            ' Delete the DC source that is no longer needed to avoid handle leaks.
            DeleteDC SourceDC
        End If
 
        ' Release the DC.
        ob.ReleaseDC obDC
    End If
End Sub

Do not forget to delete the newly created device context by a call to the GDI function DeleteDC and to release the OverlayBitmap device context by a call to the method ReleaseDC of the OverlayBitmap object.

Setting the Overlay Path position

The end user of our example program can control which of the 3 overlay bitmaps are drawn on the live video. To change the PathPositions, the following helper sub is used:

Private Sub SetOverlayPathPosition(pos As PathPositions, enabled As Boolean)
 
    ' Remember whether live video has to be restarted
    Dim MustRestart As Boolean
    MustRestart = ICImagingControl1.LiveVideoRunning
    If MustRestart Then ICImagingControl1.LiveStop
    
    ' Adjust the overlay bitmap path position
    Dim newPP As PathPositions
    If enabled Then
        newPP = ICImagingControl1.OverlayBitmapPosition Or pos
    Else
        newPP = ICImagingControl1.OverlayBitmapPosition And Not pos
    End If
    ICImagingControl1.OverlayBitmapPosition = newPP
    
    ' Restart live video
    If MustRestart Then StartLiveVideo
End Sub

This sub enables or disables the overlay at a specified path position.

Setting the Overlay's Color Mode

An application can decide whether it wants a color or grayscale overlay. The OverlayBitmap.ColorMode property can be used to select one of the available color modes:

OVERLAY_COLORMODE_BESTFIT

The overlay bitmap format is chosen to match the format of the image stream at the position of the overlay bitmap.

OVERLAY_COLORMODE_COLOR

The overlay bitmap format is color. You can draw colored lines and text on the overlay.

OVERLAY_COLORMODE_GRAYSCALE

The overlay bitmap format is grayscale. You can only draw grayscale lines and text.

When the format of the image data stream does not match the overlay bitmap color mode, the image data is automatically converted.

<< Programmer's Guide