The process of identifying the ripeness level of cayenne peppers is an important step in cultivation and post-harvest handling. Dependence on the quality factors of farmers, such as visual diversity and differences in ripeness perception, results in subjective harvest outcomes. This manual process is also prone to inconsistent results, as humans have time limitations, fatigue, and sometimes lack concentration when sorting for long periods. To minimize these issues, technological intervention is needed to mechanically classify the ripeness level of cayenne peppers. This research aims to develop a classification model for the maturity level of cayenne pepper plants. This research proposes the use of the CNN method for feature extraction and KNN for data classification based on the features extracted by CNN. From the test scenarios carried out, the classification carried out by KNN based on CNN feature extraction got the best accuracy of 99.33%, while the CNN classification model got the best accuracy of 87.33%.