I work as a software developer, research assistant at Wilson College of Textiles at NC State, & this project is in collaboration with Dr. Warren Jasper. I would like to thank him for this wonderful opportunity. In this project I actually learned how my software skills can create impactful results in real-world conditons by automating manual tasks.

1. Understanding the project:-

Calculating the distance at which the ball is going to have its first point of contact with the ground after being launched using a catapult.

Image explaining our project

2. Motivation:-

The reason behind doing this project is for data modeling that is to collect training data so it could later be used by machine learning regression models.

Data modeling table example

3. Traditional way:-

Lay aluminium foil in the expected area where the ball is going to land for a particular configuration, launch the ball, identify the dent on the foil, and calculate the distance using the measuring tape placed parallel with the foil.

Traditional way to calculate distance, using aluminum foil

This is a tedious process that takes a lot of time. It is a two-man job, and it is not feasible when sampling 10,000 data points.

4. Our method:-

Using a Raspberry Pi with an HD camera mounted on it, powered by a power bank, can be operated using a remote desktop making the system very mobile. Place Raspberry Pi camera as shown in the picture below.

Our system, Pi with HD camera powered by power bank

Remote desktop into Pi, and running the python script, depicting how the pi should be placed.

4.1. Calibration:-

Place a ruler scale, align the scale with green and red line, and get the value on the tape in inches respectively.

Here the value are, green value (mid_value) = 112 inches & Red value (minimum_value) = 84 inches. These values are calculated using the measuring tape in the background.

4.2. Ball tracking and distance calculation:-

So after weeks of research, trials and errors, after losing all hope and were almost about to scrap the project, my professor came up with a brilliant idea. Background separations:- So what we do is take the the first frame, convert it to b&w, and for each frame, we take the absolute the difference with this bg_image.

Raw footage

Background seperation

Ball tracked

Estimated distance

4.3 Challenges we faced:-

4.3.1 OpenCV not compatible with Raspberry Pi camera

First challenge, we faced was opencv library is not compatible with the Raspberry Pi camera, and thus have to use Picamera library to get camera object. Also we were getting very low fps, and had to increase it by trying out various Picamera techniques.

4.3.2 Hardware limitation for processing in real-time

We initially wanted to do this in real-time, but because of hardware limitations, the Pi was not able to process the frames as soon as it received them.

So we decided to use post-processing, that is first record the video, store the frames in a buffer, and process each frame one by one.

4.3.3 Speed and shape of ball made it impossible to track ball

The second problem we faced was that the ball was moving at a very high pace, which was visible maybe for 1-3 frames, and the program was not able to do any object detection.

So we even tried using a plain dark color background, covered using a cloth, but nonetheless because of how small and fast the ball was, the program was not able to track it.

So after weeks of research, trials and errors, after losing all hope and almost about to scrap the project, my professor came up with a brilliant idea.

4.3.4 Ball not being ball shaped

When the ball travels at a faster pace, in a particular frame, it appears to have a deformed shape, more like a comet.

Snapshot of ball shaped like comet, and not circular

So no existing articles, blogs or openCV functions which detect objects helped us.

This article gave us a good starting point, but was not able to track the ball because of how fast the ball was moving and the ball losing its shape, and the ball was only available for 1-3 frames.

So that is where we decided to actually just detect the contours in the image subtracted, get the area of the contour and if it is 100px, and the aspect ratio 3, that is it is more of an oval shape than a boxier shape, then draw a circle around it, and start tracking its coordinates of the centre.

As you can see in this particular frame, it is not a circle, but a deformed shape, which was quite challenging to track, but because of not tracking a circle, but tracking contours and having a threshold of area and aspect ratio, we were able to draw around the contour and mark a circle around it and get the centre coordinates.

Ball shaped like an amoeba

Deformed shape of ball

Ball correctly tracked

4.4. Coordinates to actual distance

Now that you have understood how we have the centre coordinates of the ball, but how to get the distance from it.

Now here is where the calibration part comes into the picture.

We have x, y coordinates of the circle with respect to the OpenCV window.

These values are calculated using measuring tape in the background, & ruler scale to align with the lines in OpenCV window.

Coordinates of the center of the ball after first point of contact with the ground.

Using above two pictures, 640 pixels == 56 inches

Therefore, each pixel has = 0.0875 inches

Now the coordinates are: 223 px, 263 px

Therefore, the ball is 223 * 0.0875 = 19.5125 inches

We have x, y coordinates of the circle with respect to the open cv window.

Relative distance of first point of contact of ball.

So the distance from the origin is 84 + 19.512 = 103.5125 inches, so you can see in the screenshot above.

If you find any bugs, or have suggestions, please write an email ([email protected]), or create a pull request on GitHub.

Currently doing masters in computer science at NC State, looking for full-time SWE roles starting May 2025, please write an email ([email protected]), if you have any leads.