Home / WPF / Getting started with real-time image processing with C#

Getting started with real-time image processing with C#

One of the interesting fields in industrial automation is real time image processing and computer vision. It’s quite simple to write applications that access to a camera in C#, both with Windows Forms and WPF, and in this article we will see how to connect to a USB camera (or your webcam) or an IP camera using AForge framework, to do a basic image processing. AForge is an open-source framework that is very well documented and easy to get started with. You can find the website at this address: http://www.aforgenet.com/.

Sample application

You can download the sample application on Github at this address: https://github.com/mesta1/AForge-examples

Watch on youtube

Installing AForge

AForge can be downloaded from NuGet, just browse for AForge, and download AForge.Video.DirectShow to connect to camera, and AForge.ImageProcessing to access to image processing libraries.

AForge nuget

Then reference System.Drawing. If you work with Windows Forms, System.Drawing is referenced by default, but to work with WPF, System.Drawing must be referenced manually because AForge works with Bitmaps.

System_Drawing

Connecting to a USB camera (or a webcam)

To connect to the camera you first need to get a the list of devices and connect to the right one.
This can be done with this code:

var videoDevicesList = new FilterInfoCollection(FilterCategory.VideoInputDevice);
foreach (FilterInfo videoDevice in videoDevicesList)
{
    cmbVideoSource.Items.Add(videoDevice.Name);
}

Then when you have a video device, to connect to the camera you have to create a VideoCaptureDevice object by passing a MonikerString, that you can get from the video device:

videoSource = new VideoCaptureDevice(videoDevicesList[cmbVideoSource.SelectedIndex].MonikerString);
videoSource.NewFrame += new NewFrameEventHandler(video_NewFrame);
videoSource.Start();

A video is composed of different images that are displayed at a very fast rate, so to display a video you need an Image control and you have to swap the images. the images correctly is crucial, otherwise the application will consume all the system memory in a few minutes.
To display a video in Windows Forms we will use a PictureBox control; to display it in WPF we will use an Image control. The camera is an asynchronous I/O, so you need to subscribe to the event NewFrame to update the image.
Here is the Windows Form code, assuming that you created a pictureBox1 in the main form:

private void video_NewFrame(object sender,NewFrameEventArgs eventArgs)
{   
    // get new frame
    Bitmap bitmap = (Bitmap)eventArgs.Frame.Clone();    
    // here you can process the frame
    pictureBox1.Image = bitmap;    
}

And here is the WPF code to update the Image control:

private void video_NewFrame(object sender,NewFrameEventArgs eventArgs)
{
    try
    {
        BitmapImage bi;
        using (var bitmap = (Bitmap)eventArgs.Frame.Clone())
        {
            bi = bitmap.ToBitmapImage();
        }
        bi.Freeze(); // avoid cross thread operations and prevents leaks
        Dispatcher.BeginInvoke(new ThreadStart(delegate { videoPlayer.Source = bi; }));
    }
    catch (Exception exc)
    {
        MessageBox.Show("Error on _videoSource_NewFrame:\n" + exc.Message, "Error);
        StopCamera();
    }
}

Connecting to an IP camera

The connection to an IP camera is similar to the connection to a USB camera, but instead of VideoInputDevices you must create a JPEGStream or a MJPEGStream, depending on the specifications of your webcam.

if (UseJpegStream)
{
    _videoSource = new JPEGStream(ConnectionString);
}
else // UseMJpegStream
{
    _videoSource = new MJPEGStream(ConnectionString);
}
_videoSource.NewFrame += new NewFrameEventHandler(video_NewFrame);
_videoSource.Start();

Also you need to know the url of the ip camera to get connected.

Real time image processing

Once you have access to the pictures, Bitmaps in this case, you can apply some filters to start recognizing objects from the camera.
You can find some examples on image processing for recognizing objects, shapes and colors on the AForge image processing sample page.
There are also some examples on surveillance and movement detection on the AForge computer vision samples page.
In the sample application you can find an example of thresholding an image and display it in WPF:

BitmapImage bi;
using (var bitmap = (Bitmap)eventArgs.Frame.Clone())
{
    if (Grayscaled)
    {
        using (var grayscaledBitmap = Grayscale.CommonAlgorithms.BT709.Apply(bitmap))
        {
            bi = grayscaledBitmap.ToBitmapImage();
        }
    }
    else if (Thresholded)
    {
        using (var grayscaledBitmap = Grayscale.CommonAlgorithms.BT709.Apply(bitmap))
        using (var thresholdedBitmap = new Threshold(Threshold).Apply(grayscaledBitmap))
        {
            bi = thresholdedBitmap.ToBitmapImage();
        }
    }
    else // original
    {
        bi = bitmap.ToBitmapImage();
    }
}
bi.Freeze(); // avoid cross thread operations and prevents leaks
Dispatcher.BeginInvoke(new ThreadStart(delegate { videoPlayer.Source = bi; }));

Drawing on an image

find books computer vision
Once you processed an image and extracted the information you need, like position of objects (blobs) in pixels, painting on an image is trivial.
Pick the original Bitmap, create a Pen object and draw geometric figures, like lines, circles and rectangles that connect the pixels.
For example this code draws a red rectangle with a thickness of 1px:

Graphics g = Graphics.FromImage(myBitmap);
var pen = new System.Drawing.Pen(System.Drawing.Color.Red, /*thickness = */ 1);
g.DrawRectangle(pen, x1, y1, x2 - x1, y2 - y1);
pen.Dispose();
g.Dispose();

WPF Post-It detector

post-it detector
This is a complete WPF application that connects to a USB camera to track the Post-it position on a picture.
This makes use of Color Filtering, Thresholding and Blob processing algorithms to find the borders of the post it.
You can select a color from the image by clicking the Color picker button.
Then you can adjust the filtering and thresholding of the processed image to recognize the borders.
The image process is in this method:

private List<IntPoint> FindCorners(Bitmap bitmap)
{
    List<IntPoint> corners = new List<IntPoint>();
    using (var clone = bitmap.Clone() as Bitmap)
    {
        new EuclideanColorFiltering(new AForge.Imaging.RGB((byte)Red, (byte)Green, (byte)Blue), Radius).ApplyInPlace(clone);
        using (var grayscaledBitmap = Grayscale.CommonAlgorithms.BT709.Apply(clone))
        {
            new Threshold(Threshold).ApplyInPlace(grayscaledBitmap);
            if (Inverted)
            {
                new Invert().ApplyInPlace(grayscaledBitmap);
            }
            BlobCounter blobCounter = new BlobCounter();
            blobCounter.FilterBlobs = true;
            blobCounter.MinWidth = 50;
            blobCounter.MinHeight = 50;
            blobCounter.ObjectsOrder = ObjectsOrder.Size;
            blobCounter.ProcessImage(grayscaledBitmap);
            Blob[] blobs = blobCounter.GetObjectsInformation();
            // create convex hull searching algorithm
            GrahamConvexHull hullFinder = new GrahamConvexHull();
            for (int i = 0, n = blobs.Length; i < n; i++)
            {
                List<IntPoint> leftPoints, rightPoints;
                List<IntPoint> edgePoints = new List<IntPoint>();
                // get blob's edge points
                blobCounter.GetBlobsLeftAndRightEdges(blobs[i], out leftPoints, out rightPoints);
                edgePoints.AddRange(leftPoints);
                edgePoints.AddRange(rightPoints);
                // blob's convex hull
                corners = hullFinder.FindHull(edgePoints);
            }
        }
    }
    return corners;
}

More image processing libraries

If AForge is not enough, there are more famous open-source frameworks, like OpenCV in C++, or its C# port Emgu-CV.

Sample application

You can find a sample application on GitHub: https://github.com/mesta1/AForge-examples
There you will find 4 projects:
1) one that connects a USB camera to a Windows Form application,
2) one to connect a USB camera to a WPF application,
3) one to connect an IP camera to a WPF application,
4) and the last one shows how to apply some filter on the displayed image with WPF.

If you are not familiar with launching different projects from Visual Studio, just right-click on a project in the solution explorer and select “Set as Startup project”.

More examples can be found on Codeproject , just by searching for AForge.

20 comments

  1. Hi
    Thanks for the Article, it’s great, I am working on a small project (hobby of mine) but am not great at proggramming or codind. Your article helped me a lot. I have a problem with connecting to an IP Camera. I am using a HIK Vision Camera. I have the IP address, Username and password. I stream videos directly from my browser. When I run the program (after putting my ip address), It doesn’t show my camera as a video source. I tried putting the adress this way “http://:@:/” but it stil doesn’t stream. neither through rtsp port.
    Thanks in advance.

  2. Hi, Thank you for the Article, I have a dahua IPC-HF5231E ip camera but i cannot connect to cam mjpeg or jpeg stream. Can i use rtsp or onvif with AForge library.

  3. Main stream does not support mjpeg only h264 and h265. I had look before that article but ozeki cant connect to camera and always give null pointer exception.

  4. Hi, congratulations for your excelent article. I have a question: how can I introduce a ‘IF’ statement to trigger an action if the result is ‘true’ (in case the program detects a previous image). Thank you.

  5. Hi, it was a very good article, I have a camera connected in a car and it send information via GPRS to a server, is there a possibility to have a live streaming video in a windows form ?

  6. Hi, great article. I was trying to implement your demo code with a Foscam, R2 model, IP camera. It seems the problem I am having is that the camera gives back html instead of a JPEG to the JPEGStream object. Do you have any suggestions? (I can connect to this camera with VLC using a rstp connection string.

  7. I’m starting new project engaging face recognition. Would you, friends, recommand AForge or EmguCV? I’m new with this. Difficulty os not a problem, I have few years of experience with C#. Please, help 🙂

  8. Thanks Mate. Works like charm in wpf. You save me 🙂

  9. hi, i have a problem here. in the video. i stuck at 1.05 min. i cant generate the code .can you help me.

  10. I just started an IP camera project with C #, can you specify more about what it looks like in winform?

    • I made the usb camera connection in winform but it didn’t work with the ip camera, I tried the following command
      MJPEGStream mpeg;
      mpeg = new MJPEGStream ();
      mpeg.Login = “admin”;
      mpeg.Password = “”;
      mpeg.Source = “http://192.168.100.101:8899/img/mjpeg.cgi”;
      mpeg.NewFrame + = new NewFrameEventHandler (video_newFrame);
      mpeg.Start ();

  11. Great tutorial. Have you have any success with zooming in and out using pinching on an WPF app?

  12. Greate article.
    But My list is empty.
    I use Dahua Ez-IP B2B40-zs?
    http://192.168.1.108/cgi-bin/snapshot.cgi
    ?

Leave a Reply

Your email address will not be published.