The point is, without the exploitation, they would care for themselves. The original argument suggested that modern slaves would not be able to feed themselves without their masters providing the food. That's wrong.
The problem is you can't do a full social change. If all you do is stop the undesirable labor, you can end up with people unable to feed themselves. This has happened before with ending child labor.
Older forms of oppression have always existed. Look at other species to see that even without a culture or society, the weak are exploited.