This research focuses on developing a cayenne pepper quality detection system using the YOLO (You Only Look Once) V5 algorithm. The system is designed to address the limitations of manual post-harvest sorting by classifying cayenne peppers into three categories: “good” (ripe), “bad” (rotten or dry), and “raw” (green), based on their visual characteristics. A dataset consisting of 565 images was manually collected, labeled using Roboflow, and pre-processed through resizing and orientation standardization. Model training was conducted over 150 epochs, achieving high detection performance with a mean average precision (mAP) of 99.5%, precision of 99.6%, and recall of 99.9%. Real-time testing demonstrated the system’s capability to detect and classify cayenne peppers with exceptional accuracy. This research is expected to enhance the efficiency and accuracy of the cayenne pepper sorting process, while paving the way for the application of YOLO-based systems to other agricultural commodities. Further research is recommended to expand the dataset and optimize model parameters for improved system performance.