HomeSupportDocumentationIC Imaging Control .NETPerforming Advanced Image Processing

Performing Advanced Image Processing

This chapter illustrates how to perform sophisticated image processing.

The source code for the VB.NET and C# versions of this sample program can be found in the directories samples\VB71\Advanced Image Processing, samples\C#\Advanced Image Processing. Before reading this chapter, we suggest that you first take a look at Displaying Buffers on the ImageAvailable Event.

Overview

This chapter illustrates how to:

  • display image buffers
  • render an overlay on the live image
  • process mouse events
  • process data contained in image buffers

The sample described in this chapter lets you draw a rectangle on the live image using the mouse. The region specified by the rectangle is used to check whether the live image has changed. If so, the changed image is displayed. The threshold for the difference between the images can be set by the user.

Setting up the Project

Create a new project and add IC Imaging Control to the form. Before you run the program, select the video device, input and video format as shown in the First Steps Visual Studio .NET 7.1 chapter. Alternatively, run the program without selecting a device. In this case, the program shows the device selection dialog provided by IC Imaging Control. If you close this dialog without making a selection, the displays error message and terminates.

Add two buttons to the form and label them Device and Settings. Name them cmdDevice and cmdSettings respectively. If you click the "Device" button, a device selection dialog is displayed. After you have selected a valid device, setupDevice is called. This is a helper procedure, which sets up a FrameHandlerSink that is used to grab images. The sink is initialized with a ring buffer size of 5 and Y800 as color format. ICImagingControl.LiveDisplay is disabled, because the buffers are drawn by hand in this sample. The Settings button shows a dialog that allows you to adjust the VCDProperties of the currently selected device. The code for both buttons looks as follows: 

[VB.NET]
Private Sub cmdDevice_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles cmdDevice.Click
    With IcImagingControl1
        If .LiveVideoRunning Then
            .LiveStop()
        End If
 
        .ShowDeviceSettingsDialog()
 
        If .DeviceValid Then
            cmdStart.Enabled = True
            cmdSettings.Enabled = True
            MakeDeviceSettings()
        End If
    End With
End Sub

[C#]
private void cmdDevice_Click(object sender, System.EventArgs e)
{
    if( icImagingControl1.LiveVideoRunning )
    {
        icImagingControl1.LiveStop();
    }
 
    icImagingControl1.ShowDeviceSettingsDialog();
 
    if( icImagingControl1.DeviceValid )
    {
        cmdStart.Enabled = true;
        cmdSettings.Enabled = true;
        MakeDeviceSettings();
    }
}

[VB.NET]
Private Sub cmdSettings_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles cmdSettings.Click
    IcImagingControl1.ShowPropertyDialog()
End Sub

[C#]
private void cmdSettings_Click(object sender, System.EventArgs e)
{
    icImagingControl1.ShowPropertyDialog();
}

Declaring Global Variables

Create a user data type named RECT to hold the coordinates for a rectangle by inserting the following code at the beginning of your Form class. 

[VB.NET]
Private Structure RECT
    Dim Left As Integer
    Dim Top As Integer
    Dim Right As Integer
    Dim Bottom As Integer
End Structure

[C#]
private struct RECT
{
    public int Left;
    public int Top;
    public int Right;
    public int Bottom;
}

Now, insert three global variables. 

[VB.NET]
Private DisplayBuffer As TIS.Imaging.ImageBuffer
Private UserROI As RECT
Private UserROICommited As Boolean

[C#]
private TIS.Imaging.ImageBuffer DisplayBuffer;
private RECT UserROI;
private System.Windows.Forms.TrackBar sldThresholdSlider;
private TIS.Imaging.ICImagingControl icImagingControl1;
private bool UserROICommited;
  • DisplayBuffer will contain a reference to the buffer that will be displayed on the ImageAvailable event. When deciding whether displaying a new buffer is necessary, the new buffer is compared to DisplayBuffer.
  • UserROI will contain the region of interest the user has set with the mouse.
  • UserROICommited will be used to toggle between drawing and testing.

Adding the Mouse Event Procedures

Add event procedures for ICImagingControl's MouseDown, MouseUp, and MouseMove events. Insert the following code into the event procedures:

MouseDown: 

[VB.NET]
If Not UserROICommited And (e.Button = MouseButtons.Left) Then
    UserROI.Left = e.x
    UserROI.Top = IcImagingControl1.Height - e.y
End If

[C#]
if( !UserROICommited && (e.Button == MouseButtons.Left) )
{
    UserROI.Left = e.x;
    UserROI.Top = icImagingControl1.Height - e.y;
}

MouseUp: 

[VB.NET]
If Not UserROICommited And Not (e.Button = MouseButtons.Left) Then
    UserROI.Right = e.x
    UserROI.Bottom = IcImagingControl1.Height - e.y
End If

[C#]
if( !UserROICommited && !(e.Button == MouseButtons.Left) )
{
    UserROI.Right = e.x;
    UserROI.Bottom = icImagingControl1.Height - e.y;
}

MouseMove: 

[VB.NET]
If Not UserROICommited And (e.Button = MouseButtons.Left) Then
    UserROI.Right = e.x
    UserROI.Bottom = IcImagingControl1.Height - e.y
End If

[C#]
if( !UserROICommited && (e.Button == MouseButtons.Left) )
{
    UserROI.Right = e.x;
    UserROI.Bottom = icImagingControl1.Height - e.y;
}

Because we use Y800 as sink color format, the mouse positions are identical to the pixel positions in the image buffers. Thus, we can set the UserROI.bottom and .top members to YPos.

When using a bottom-up sink color format such as RGB8, you had to transform the mouse positions to ICImagingControl1.Height - YPos.

Adding Functionality to the Application

As described in Displaying Buffers on the ImageAvailable Event, create two buttons ( Start / Stop ), the Form_Load event procedure and an ImageAvailable event handler. Add the line DisplayBuffer.ForceUnlock to the Stop_Click event handler. This is necessary as the DisplayBuffer will be locked in the ImageAvailable event procedure. When the control is stopped, the DisplayBuffer needs to be unlocked. Insert the following code into the ImageAvailable handler: 

[VB.NET]
Private Sub IcImagingControl1_ImageAvailable(ByVal sender As Object, ByVal e As TIS.Imaging.ICImagingControl.ImageAvailableEventArgs) Handles IcImagingControl1.ImageAvailable
    Try
        Dim Region As RECT
        Region = NormalizeRect(UserROI)
 
        ContinousMode(e.bufferIndex, Region)
    Catch ex As Exception
        System.Diagnostics.Trace.WriteLine(ex.Message)
    End Try
End Sub

[C#]
private void icImagingControl1_ImageAvailable(object sender, TIS.Imaging.ICImagingControl.ImageAvailableEventArgs e)
{
    try
    {
        RECT Region;
        Region = NormalizeRect(UserROI)
 
        ContinousMode(e.bufferIndex, Region)
    }
    catch(Exception ex)
    {
        System.Diagnostics.Trace.WriteLine(ex.Message);
    }
}

This event handler uses the function NormalizeRect to make sure the rectangle's coordinates are not exchanged, e.g. left greater than right. It also calls the sub ContinousMode to paint that rectangle onto the image buffer. NormalizeRect is implemented as follows: 

[VB.NET]
Private Function NormalizeRect(ByRef val As RECT) As RECT
    Dim Tmp As Integer
    Dim r As RECT
    r = val
    If r.Top > r.Bottom Then
        Tmp = r.Top
        r.Top = r.Bottom
        r.Bottom = Tmp
    End If
    If r.Left > r.Right Then
        Tmp = r.Left
        r.Left = r.Right
        r.Right = Tmp
    End If
    If r.Top < 0 Then
        r.Top = 0
    End If
    If r.Left < 0 Then
        r.Left = 0
    End If
    If r.Bottom >= IcImagingControl1.ImageHeight Then
        r.Bottom = IcImagingControl1.ImageHeight - 1
    End If
    If r.Right >= IcImagingControl1.ImageWidth Then
        r.Right = IcImagingControl1.ImageWidth - 1
    End If
    NormalizeRect = r
End Function

[C#]
private RECT NormalizeRect(RECT val)
{
    int Tmp;
    RECT r;
 
    r = val;
 
    if( r.Top > r.Bottom )
    {
        Tmp = r.Top;
        r.Top = r.Bottom;
        r.Bottom = Tmp;
    }
 
    if( r.Left > r.Right )
    {
        Tmp = r.Left;
        r.Left = r.Right;
        r.Right = Tmp;
    }
 
    if( r.Top < 0 )
    {
        r.Top = 0;
    }
 
    if( r.Left < 0 )
    {
        r.Left = 0;
    }
 
    if( r.Bottom >= icImagingControl1.ImageHeight )
    {
        r.Bottom = icImagingControl1.ImageHeight - 1;
    }
 
    if( r.Right >= icImagingControl1.ImageWidth )
    {
        r.Right = icImagingControl1.ImageWidth - 1;
    }
 
    return r;
}

The sub ContinousMode is implemented in the following way: 

[VB.NET]
Private Sub ContinousMode(ByVal BufferIndex As Integer, ByVal Region As RECT)
    DisplayBuffer.Unlock()
    DisplayBuffer = IcImagingControl1.ImageBuffers(BufferIndex)
    DisplayBuffer.Lock()
    DrawRectangleY8(DisplayBuffer, Region)
    IcImagingControl1.DisplayImageBuffer(DisplayBuffer)
End Sub

[C#]
private void ContinousMode(int BufferIndex, RECT Region)
{
    //DisplayBuffer.Unlock();
    DisplayBuffer = icImagingControl1.ImageBuffers[BufferIndex];
    //DisplayBuffer.Lock();
 
    DrawRectangleY8(DisplayBuffer, Region);
 
    icImagingControl1.DisplayImageBuffer(DisplayBuffer);
}

First the DisplayBuffer is unlocked. It is generally good practice to release locks that are no longer required, otherwise the control can not write to that buffer. Then the image buffer specified by BufferIndex of the ImageBuffers collection is assigned to DisplayBuffer. This buffer is locked with DisplayBuffer.Lock to prevent DisplayBuffer from being overwritten by IC Imaging Control. Then the DrawRectangleY8 sub procedure draws a rectangle specified by Region into the image buffer. To display the buffer, ICImagingControl.DisplayImageBuffer is called with DisplayBuffer as its parameter. The buffer now contains a white rectangle that represents the current region of interest.

Implemented this way, IC Imaging Control enables the user to select a region by painting a rectangle on the control while a live video is running.

Inserting the "CompareMode"

Now, we add the functionality to only update the display, if something changes in the region of interest. Create a button, named cmdROICommit and add the following code to it's click event procedure: 

[VB.NET]
Private Sub cmdROICommit_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles cmdROICommit.Click
    If Not UserROICommited Then
        UserROICommited = True
        cmdROICommit.Text = "Reset ROI"
    Else
        UserROICommited = False
        cmdROICommit.Text = "Set current ROI"
    End If
End Sub

[C#]
private void cmdROICommit_Click(object sender, System.EventArgs e)
{
    if( !UserROICommited )
    {
        UserROICommited = true;
        cmdROICommit.Text = "Reset ROI";
    }
    else
    {
        UserROICommited = false;
        cmdROICommit.Text = "Set current ROI";
    }
}

When this button is clicked, the region of interest is committed. The application then switches to the "CompareMode". In this mode, the display is only updated, if something within the region of interest has changed. When the "CompareMode" is already running, the button toggles to "ContinousMode".

Updating the Code in the Event Procedures

Add the the following code to the cmdStart_Click event procedure: 

[VB.NET]
cmdROICommit.Enabled = True

[C#]
cmdROICommit.Enabled = true;

This enables the painting of the ROI every time you start the live video. This is named "ContinousMode" in this example.

Add the following code to the cmdStop_Click event procedure:. 

[VB.NET]
If UserROICommited Then
    cmdROICommit_Click(sender, e)  ' Change CompareMode to ContinousMode.
End If
cmdROICommit.Enabled = False

[C#]
if( UserROICommited )
{
    cmdROICommit_Click(sender, e);  // Change CompareMode to ContinousMode.
}
cmdROICommit.Enabled = false;

This will reset the mode from "CompareMode" to "ContinousMode" when the "Stop" button is clicked and the "CompareMode" is active. Further, add the line cmdROICommit.Enabled = False to the Form_Load event procedure.

Then alter the code in the ImageAvailable event procedure: 

[VB.NET]
Dim Region As RECT
Region = NormalizeRect(UserROI)
 
If Not UserROICommited Then
    ContinousMode(e.bufferIndex, Region)
Else
    CompareMode(e.bufferIndex, Region)
End If

[C#]
RECT Region;
Region = NormalizeRect(UserROI);
 
if( !UserROICommited )
{
    ContinousMode(e.bufferIndex, Region);
}
else
{
    CompareMode(e.bufferIndex, Region);
}

This calls the appropriate sub procedures, depending on UserROICommited.

The "CompareMode" Procedure

The new ImageAvailable event procedure calls the sub procedure CompareMode, which is implemented as follows: 

[VB.NET]
Private Sub CompareMode(ByVal BufferIndex As Integer, ByVal Region As RECT)
    Dim IBOld, IBNew As TIS.Imaging.ImageBuffer
    IBOld = DisplayBuffer
    IBNew = IcImagingControl1.ImageBuffers(BufferIndex)
 
    If CompareRegion(IBOld, IBNew, Region, sldThresholdSlider.Value) Then
        DisplayBuffer.Unlock()
        DisplayBuffer = IBNew
        DisplayBuffer.Lock()
 
        DrawRectangleY8(IBNew, Region)
 
        IcImagingControl1.DisplayImageBuffer(DisplayBuffer)
    End If
End Sub

[C#]
private void CompareMode(int BufferIndex, RECT Region)
{
    TIS.Imaging.ImageBuffer IBOld, IBNew;
 
    IBOld = DisplayBuffer;
    IBNew = icImagingControl1.ImageBuffers[BufferIndex];
 
    if( CompareRegion(IBOld, IBNew, Region, sldThresholdSlider.Value) )
    {
        //DisplayBuffer.Unlock()
        DisplayBuffer = IBNew;
        //DisplayBuffer.Lock()
 
        DrawRectangleY8(IBNew, Region);
 
        icImagingControl1.DisplayImageBuffer(DisplayBuffer);
    }
}

This sub calls the CompareRegion sub function, which compares the region of interest of the image buffers. If they differ, the old DisplayBuffer is unlocked and the new image buffer IBNew is assigned to DisplayBuffer. DisplayBuffer then contains the new image. The new DisplayBuffer is locked to prevent it from being overwritten by IC Imaging Control. The region of interest is drawn as a rectangle into the buffer, and the buffer is displayed, using ICImagingControl.DisplayImageBuffer.

The CompareRegion Function

The CompareRegion function compares the region of interest of the two image buffers Buf and Buf2. The differences of the pixels in the specified Region are added. The result is saved in the variable GreyscaleDifference. After adding, GreyscaleDifference is divided by the count of pixels in the region. GreyscaleDifference is compared to the threshold value passed to the function. If the threshold is lower or equal to GreyscaleDifference, the regions differ and the function returns True, False otherwise. The CompareRegion function is implemented as follows: 

[VB.NET]
Private Function CompareRegion(ByVal Buf As TIS.Imaging.ImageBuffer, ByVal Buf2 As TIS.Imaging.ImageBuffer, ByVal Region As RECT, ByVal Threshold As Integer) As Boolean
    Dim x, y As Integer
    Dim GreyscaleDifference As Integer
    Dim PixelCount As Integer
 
    PixelCount = (Region.Bottom - Region.Top) * (Region.Right - Region.Left)
    If PixelCount > 0 Then
        GreyscaleDifference = 0
        For y = Region.Top To Region.Bottom
            For x = Region.Left To Region.Right
                GreyscaleDifference = GreyscaleDifference + Math.Abs(CInt(Buf(x, y)) - CInt(Buf2(x, y)))
            Next x
        Next y
        GreyscaleDifference = GreyscaleDifference / PixelCount
        If GreyscaleDifference > Threshold Then
            CompareRegion = True
        Else
            CompareRegion = False
        End If
    Else
        CompareRegion = False
    End If
End Function

[C#]
private bool CompareRegion(TIS.Imaging.ImageBuffer Buf, TIS.Imaging.ImageBuffer Buf2, RECT Region, int Threshold)
{
    int x, y;
    int GreyscaleDifference;
    int PixelCount;
 
    PixelCount = (Region.Bottom - Region.Top) * (Region.Right - Region.Left);
    if( PixelCount > 0 )
    {
        GreyscaleDifference = 0;
        for( y = Region.Top; y <= Region.Bottom; y++ )
        {
            for( x = Region.Left; x <= Region.Right; x++ )
            {
                GreyscaleDifference = GreyscaleDifference + Math.Abs( (int)(Buf[x, y]) - (int)(Buf2[x, y]) );
            }
        }
        GreyscaleDifference = GreyscaleDifference / PixelCount;
        if( GreyscaleDifference > Threshold )
        {
            return true;
        }
        else
        {
            return false;
        }
    }
    else
    {
        return false;
    }
}

<< Programmer's Guide

This site is part of The Imaging Source Network. Other sites include Company Portal, Image Processing, Astronomy Cameras, Astronomy Cameras Blog, Blog caméras d'astronomie, Astronomy Cameras Competition, TX Text Control, TX Text Control Blog and Forums.
ISO 9001:2000 certifiedISO 9001:2000 certified.
© 1991-2008 The Imaging Source Europe GmbH. All rights reserved.