Detect Flappy Bird in Video Using OpenCV

Introduction

This project is designed to teach you how to detect the Flappy Bird character in a video using the cv2.matchTemplate method provided by OpenCV, a powerful library for image processing and computer vision. The cv2.matchTemplate function is a useful tool for template matching, which is a technique in digital image processing for finding small parts of an image that match a template image. In this project, we'll create a Python project that applies this method to detect the Flappy Bird character in a video, making it a practical example for beginners and intermediates in Python and OpenCV.

👀 Preview

🎯 Tasks

In this project, you will learn:

• How to import and use the OpenCV library for image processing.
• How to write a function to process video and image data.
• How to implement template matching for object detection in video frames.
• How to annotate and display results in a video.

🏆 Achievements

After completing this project, you will be able to:

• Use OpenCV for basic image and video processing tasks.
• Apply template matching techniques to detect objects in video data.
• Annotate and visualize the results of object detection in video.

Create Project Files

Begin by preparing your project workspace. Create a single Python script file named bird_detection.py.

touch bird_detection.py

Additionally, ensure you have a video file (e.g., flappybird.mp4) and a bird image file (e.g., bird.png) in your project directory. These files will be used to detect the bird in the video.

Project Directory Structure

project/
│
├── bird_detection.py ## Your Python script
├── flappybird.mp4 ## Video file for processing
└── bird.png ## Bird image to detect in the video

Import OpenCV

In bird_detection.py, start by importing the OpenCV library. This library provides functions for image processing and computer vision.

import cv2

By importing cv2, we gain access to all the OpenCV functionalities.

Define Bird Detection Function

Define a function called find_and_display_bird. This function will take the paths to a video file and a bird image, and it will display the bird in each frame of the video.

def find_and_display_bird(video_path, bird_image_path):
    ## Function details will be added in the next steps

This function is the core of our project, where most of our detection logic will reside.

Load Video and Bird Image

In this step, the script loads the video and bird image files using OpenCV, which is essential for the detection process.

    ## Load video and bird image
    cap = cv2.VideoCapture(video_path)
    bird = cv2.imread(bird_image_path)

    ## Check if files were loaded successfully
    if not cap.isOpened():
        print(f"Error: Could not open video {video_path}")
        return
    if bird is None:
        print(f"Error: Could not load bird image {bird_image_path}")
        return

Explanation of Loading Process:

• cv2.VideoCapture(video_path): This function is used to capture a video.
  • video_path is the path to the video file.
  • It returns a video capture object (cap), which allows you to read frames from the video.
• cv2.imread(bird_image_path): This function reads an image from the specified file.
  • bird_image_path is the path to the bird image.
  • It returns the image (bird) in the form of a multi-dimensional array, which represents the pixels of the image.

Checking File Load Success:

• It's crucial to check whether the video and image files have been loaded successfully to prevent errors during processing.
• cap.isOpened(): This method checks if the video capture has been initialized correctly. If it returns False, it means the video file couldn't be opened.
• if bird is None: This condition checks if the bird image has been loaded correctly. If bird is None, it means the image file couldn't be read.

Loading the video and bird image correctly is a fundamental step in the bird detection process, as it ensures that the data required for processing is accessible and in the right format.

Process Video Frame by Frame

In this step, the script processes each frame of the video to detect the bird using two key functions from OpenCV: cv2.matchTemplate and cv2.minMaxLoc.

    print("Processing video...")

    ## Process video frame by frame
    while True:
        success, im = cap.read()
        if not success:
            break

        res = cv2.matchTemplate(im, bird, cv2.TM_CCOEFF)
        _, max_val, _, max_loc = cv2.minMaxLoc(res)

        ## Drawing and text annotation code will be added in the next step

Explanation of cv2.matchTemplate:

• cv2.matchTemplate is used for template matching, a method in computer vision for finding a sub-image in a larger image.
• In this function:
  • im is the current frame from the video.
  • bird is the template image (in this case, the bird image we're looking for).
  • cv2.TM_CCOEFF is the method used for template matching. There are several methods available in OpenCV, but TM_CCOEFF is effective for this purpose as it uses the correlation coefficient for matching.
• The function returns a grayscale image res, where each pixel denotes how much does the neighbourhood of that pixel match with the template.

Explanation of cv2.minMaxLoc:

• cv2.minMaxLoc is used to find the location of the minimum and maximum values in a grayscale image.
• In this context, it's used to find the highest matching area in the res image.
  • The function returns four values: the minimum value, the maximum value, the location of the minimum value, and the location of the maximum value.
  • For TM_CCOEFF, the maximum value (max_val) and its location (max_loc) are of interest, as they represent the point where the template (bird) has the highest match.
• max_loc gives us the top-left corner of the rectangle where the bird is found in the frame.

By repeatedly applying these two functions to each frame of the video, the script can locate the bird in each frame, allowing for real-time detection as the video plays. This is a fundamental technique in video analysis and computer vision, enabling the identification and tracking of objects across frames.

Annotate and Display Results

In this step, the script annotates and displays each frame with the detected bird area by drawing a rectangle around it and displaying the position.

        ## Draw rectangle around the bird
        left, top = max_loc
        right, bottom = left + bird.shape[1], top + bird.shape[0]
        cv2.rectangle(im, (left, top), (right, bottom), 255, 2)

        ## Add text annotation for position
        position_text = f"Position: ({left}, {top})"
        cv2.putText(im, position_text, (left, top), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (255, 255, 255), 1)

        ## Display the image
        cv2.imshow('Bird Detection', im)

        ## Break loop on 'q' key press
        if cv2.waitKey(1) & 0xFF == ord('q'):
            break

Drawing a Rectangle Around the Bird:

• cv2.rectangle(im, (left, top), (right, bottom), 255, 2): This function draws a rectangle on the frame.
  • The parameters (left, top) and (right, bottom) define the top-left and bottom-right corners of the rectangle.
  • 255 sets the color of the rectangle (white in this case).
  • 2 is the thickness of the rectangle lines.

Adding Text Annotation:

• cv2.putText: This function puts text on the frame.
  • It displays the position of the bird in the frame.
  • position_text is the text to be displayed, showing the top-left corner of the detected bird area.
  • The position where the text is displayed is set to (left, top) which is the top-left corner of the rectangle.
  • cv2.FONT_HERSHEY_SIMPLEX, 0.5, (255, 255, 255), 1 set the font style, size, color (white), and thickness of the text.

Displaying the Image:

• cv2.imshow('Bird Detection', im): This function displays the current frame with the rectangle and text annotations.
• cv2.waitKey(1) & 0xFF == ord('q'): This line allows the script to display the frame for a short time and waits for the 'q' key to be pressed to exit the loop. This enables real-time video playback with annotations.

Annotating and displaying the results in real-time are crucial for visualizing the effectiveness of the bird detection process. It allows users to see the detection in action and verify its accuracy.

Release Resources and Close Windows

After processing the video, release all resources and close any open windows.

    ## Release resources and close windows
    cap.release()
    cv2.destroyAllWindows()
    print("Video processing complete.")

This cleanup step is crucial for efficient resource management in any application that handles files and windows.

Example Usage

Finally, provide an example of how to use the find_and_display_bird function.

## Example usage
if __name__ == "__main__":
    find_and_display_bird('flappybird.mp4', 'bird.png')

Now we've completed all the steps, we can run the code in the desktop environment by using the following command:

cd ~/project
python bird_detection.py

Summary

In this project, you've learned how to set up a Python project to detect a bird in a video using OpenCV. Starting from creating project files, importing OpenCV, defining the bird detection function, loading and processing video frames, annotating and displaying results, to cleaning up resources, you've gone through all the essential steps to build a basic computer vision application. This project serves as a foundational skill in image processing and computer vision using Python and OpenCV.