Brown was born into poverty in Barnwell, S.C. He was raised by an aunt in Augusta, Ga. Abandoned by his divorced parents, he spent much of his childhood hustling to get by, shining shoes and dancing for soldiers visiting his aunt’s brothel. In school, he was often teased for wearing raggedy clothes, and he dropped out after sixth grade.