Moss was born in and raised in LaGrange, Georgia, the fourth of five children. After being orphaned at 16 he earned his B.A. at Morehouse College in 1956, before earning his Master of Divinity from the Morehouse School of Religion in 1959. At Morehouse, Moss was taught and mentored by Benjamin Mays, who was also a mentor to Martin Luther King Jr. While at Morehouse, Moss helped lead sit-ins and other activities to protest segregation.
He completed further graduate coursework at the Interdenominational Theological Center from 1960 to 1961. He also earned a Doctor of Ministry from United Theological Seminary in 1990, where he was taught by Samuel DeWitt Proctor and became friends with Jeremiah Wright, pastor of the famous Trinity United Church of Christ in Chicago and pastor of Barack Obama.