how did slavery affect whites those who owned slaves and those who did not; those in the north and those in the south