A Python/Pygame Robot Coloured Ball Sorter

2015-09-10 | Robotics , python , arduino , pygame ,

Over term 2-3 of 2015, I worked on a ball sorter that allowed me to quickly send multi-coloured ping-pong balls to different outputs depending on their colour. This took me and my partner roughly 40+ hours of development and I submitted it for a few NCEA standards as part of my school technology and robotics course. The project is mostly finished - I planned on doing a few improvement to the colour recognition algorithms and remaking parts of the structure, and then exams hit, so I couldn't complete it. We submitted this project towards Catalysts Arduino Academy competition, winning us first place and a $50 gift card.

The original plan was to use an analog colour sensor paired with an arduino, but it turned out to be nearly impossible to program (and also incredibly boring). After much consideration and looking into other methods I settled on using a dedicated raspberry pi with a webcam as well as the arduino, communicating through serial over USB.

How it works now:

The Raspberry Pi (2, Model B) is connected to a Playstation Eye webcam. I use the python library PyGame to analyze the colour of the middle 50x50 pixels of the camera and take an average of the colour (as shown in the picture to the right, the average colour is displayed in the top corner). From there, I can retrieve the Hue, Saturation and Value (HSV) of the colour - if the saturation is over a certain amount, we know that there is a ball in front of the camera. The script checks to see if the hue is within the range of values for the each of the four colours. From there it takes a modal average of the colours after 50 ‘readings’ and sends the most common colour to the arduino through USB serial.

The arduino waits for a serial input and then moves the funnel servo to a certain position depending on which colour is received. It then moves the gate servo to an open position and then back to let a single ball through. Whilst doing so, the arduino updates the count of balls that has passed through the system and updates the LCD screen. The arduino also makes use of the arduino-academy provided piezo buzzer for playing a startup sound. It was supposed to also play a sound after each colour was sensed (with varying pitched based off hue) but I ran out of time due to mock exams.

I tried to keep everything modular - the RPi controls the colour sensing, whilst the arduino does everything else. The arduino will work without being attached to the raspberry pi (for instance connecting it to my laptop for debugging), although the python code on the RPi will throw an error if there is no arduino connected. All the functions in all the code is also modular and should work by themselves although I haven’t tested this in depth.

I only have some pictures from an earlier point in development due to forgetting to take photos. In the first picture, I have the TFTLCD screen from the Arduino Academy attached the UNO. The screen is displaying the amount of balls going through the system (I had not yet hooked up the servos). The other two pictures are of an earlier prototype. I have since remade most of the structure using our school laser cutter, and changed from the original Arduino Mega to the Uno.

I have since improved the code so that the machine sorts balls faster.