Video Analysis and Motion Tracking – in Computer Vision

What You’ll Learn 📚

• Basic concepts of video analysis and motion tracking
• Key terminology and definitions
• Simple to complex examples with code
• Common questions and troubleshooting tips

Introduction to Video Analysis and Motion Tracking

Video analysis is the process of examining video frames to extract meaningful information. Motion tracking, a subset of video analysis, involves detecting and following the movement of objects across frames. These techniques are widely used in applications like surveillance, sports analytics, and autonomous vehicles.

Key Terminology

• Frame: A single image in a sequence of images that make up a video.
• Object Detection: Identifying objects within an image or frame.
• Tracking: Following the detected objects over time across multiple frames.

Getting Started: The Simplest Example

Let’s start with a simple example using Python and OpenCV, a popular computer vision library. We’ll track a colored object in a video.

import cv2

# Open the video file
cap = cv2.VideoCapture('video.mp4')

while cap.isOpened():
    ret, frame = cap.read()
    if not ret:
        break
    
    # Convert the frame to HSV color space
    hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)
    
    # Define range of color to track
    lower_color = (30, 150, 50)
    upper_color = (85, 255, 255)
    
    # Create a mask for the color
    mask = cv2.inRange(hsv, lower_color, upper_color)
    
    # Find contours
    contours, _ = cv2.findContours(mask, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
    
    # Draw the contours on the original frame
    for contour in contours:
        if cv2.contourArea(contour) > 500:
            x, y, w, h = cv2.boundingRect(contour)
            cv2.rectangle(frame, (x, y), (x + w, y + h), (0, 255, 0), 2)
    
    # Display the frame
    cv2.imshow('Frame', frame)
    
    # Break the loop on 'q' key press
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

cap.release()
cv2.destroyAllWindows()

💡 Lightbulb Moment: HSV color space is often used in color tracking because it separates color information from intensity, making it easier to define color ranges.

Progressively Complex Examples

Example 2: Multi-Object Tracking

Now, let’s track multiple objects using different colors.

# Additional code for tracking multiple colors
colors = {
    'red': ((0, 120, 70), (10, 255, 255)),
    'green': ((36, 25, 25), (86, 255, 255)),
    'blue': ((94, 80, 2), (126, 255, 255))
}

for color, (lower, upper) in colors.items():
    mask = cv2.inRange(hsv, lower, upper)
    contours, _ = cv2.findContours(mask, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
    for contour in contours:
        if cv2.contourArea(contour) > 500:
            x, y, w, h = cv2.boundingRect(contour)
            cv2.rectangle(frame, (x, y), (x + w, y + h), (0, 255, 0), 2)
            cv2.putText(frame, color, (x, y - 10), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 255, 0), 2)

Example 3: Optical Flow for Motion Tracking

Optical flow is a method to estimate motion between two frames. We’ll use the Lucas-Kanade method for dense optical flow.

import numpy as np

cap = cv2.VideoCapture('video.mp4')
ret, first_frame = cap.read()
prev_gray = cv2.cvtColor(first_frame, cv2.COLOR_BGR2GRAY)

mask = np.zeros_like(first_frame)
mask[..., 1] = 255

while cap.isOpened():
    ret, frame = cap.read()
    if not ret:
        break
    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    
    # Calculate dense optical flow
    flow = cv2.calcOpticalFlowFarneback(prev_gray, gray, None, 0.5, 3, 15, 3, 5, 1.2, 0)
    
    # Compute magnitude and angle of the flow
    magnitude, angle = cv2.cartToPolar(flow[..., 0], flow[..., 1])
    
    # Set image hue according to the flow direction
    mask[..., 0] = angle * 180 / np.pi / 2
    
    # Set image value according to the flow magnitude
    mask[..., 2] = cv2.normalize(magnitude, None, 0, 255, cv2.NORM_MINMAX)
    
    # Convert HSV to RGB (BGR) color representation
    rgb = cv2.cvtColor(mask, cv2.COLOR_HSV2BGR)
    
    # Display the frame
    cv2.imshow('Dense Optical Flow', rgb)
    
    # Update previous frame
    prev_gray = gray
    
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

cap.release()
cv2.destroyAllWindows()

Note: Optical flow is computationally intensive and may require a powerful machine for real-time processing.

Common Questions and Troubleshooting

1. Why is my video not playing? Ensure the video path is correct and the file is accessible.
2. Why are no objects being detected? Check the color range values and ensure they match the object’s color in the video.
3. Why is the tracking inaccurate? Adjust the contour area threshold and ensure the video resolution is sufficient.
4. Why is the optical flow slow? Reduce the frame size or use a more powerful machine.
5. How do I track objects in real-time? Use a webcam as the video source with cv2.VideoCapture(0).

Practice Exercises

• Modify the color tracking example to track a different color.
• Implement a feature to save the tracked video to a file.
• Experiment with different optical flow parameters to see their effects.

Remember, practice makes perfect! Keep experimenting and exploring the vast world of computer vision. You’ve got this! 🚀