Robotic Candy Sorter

All of life's problems finally solved.

Mark (Moonyoung) Lee(

Peter A. Slater (

October 2016 to December 2016.

Introduction    top

Robotic Candy Sorter

The purpose of the Robotic Candy Sorter project was to implement a 3 degree of freedom robotic arm and vision system that can detect and sort candy by color. This was accomplished by building an integrated system that leverages high-level (Raspberry Pi) and low-level (PIC32) processing to accomplish an ambitious task. The Raspberry Pi (RPi) handled the image processing and sorting algorithms, while the PIC32 microcontroller (uC) maintained control of the motors by solving the inverse kinematics (IK).

Figure 1: The finished project.

Many embedded systems leverage a hierarchical structure in their product architecture for a more robust and efficient product. The more complex the product becomes, it is more preferable to segment tasks according to the processing unit’s role to observe optimal tradeoffs in the system. For instance, it would be impractical for a single processing unit to handle both user interface tasks while also handling real-time control tasks. User interface tasks are very tolerating to missed deadlines while the latter is not. Rather than having a single processing unit that can comprehensively handle both ends of the spectrum, such as real-time operating systems (RTOS), this project explores a more modular build by segmenting high-level and low-level tasks to separate processing units in order to implement a complex system like a vision-driven robotic arm.

The Robotic Candy Sorter is an interdisciplinary project that integrates mechanical, electrical, and software aspects. The report discusses the inverse kinematics involved in mapping servo angles to orthogonal spatial coordinates, the individual electrical systems, and the software that handles object recognition to drive the FSM. The segmentation of the processors’ roles in this project aligns well with our academic purposes as well. The RPi development of the project was done for Cornell University’s ECE 5725 Embedded OS course and the PIC32 development of the project was done for ECE 4760 Microcontrollers courses.