With a qualitative approach, this study intends to describe in depth the socio-cultural background and challenges of agriculture in Southwest Sumba Regency (SBD) which cause the problem of high levels of poverty in the area despite having a wealth of great agricultural potential. This research found that the world of SBD agriculture faces challenges in the form of a high level of agricultural vulnerability and not optimal management of agricultural potential. These two challenges are difficult to overcome due to the lack of farming capital caused by debt bondage due to high-cost customary practices and the characteristics of subsistence farmers. On the other hand, the trade-oriented model of commercial farming of SBD while not leaving cultural values and local knowledge has been proven to be able to overcome the challenges of agricultural vulnerability and not optimal management of SBD agricultural potential.