It goes back to 19th-century distinctions between the "deserving" and "Undeserving" poor--and it was the "Overseers of the Poor" (an actual office in many New England towns) who got to decide who belonged in which category. As in my most recent journal entry, it's not a long step from accusations that the poor don't "deserve" anything to the workhouse, poor farm, etc.