Leon Thomas was born Amos Thomas, Jr. on October 4, 1937, in East St. Louis, Illinois. He studied music at Tennessee State University. At the time of his studies, he had begun a singing career as a guest vocalist for the jazz bands of percussionist Armando Peraza, saxophonist Jimmy Forest, and guitarist Grant Green. His musical development at this time was shaped in part by seeing saxophonist John Coltrane perform in trumpeter Miles Davis's sextet during the late 1950s. Thomas moved to New York City in 1959, singing at the Apollo Theater as a vocalist for acts such as jazz ensemble The Jazz Messengers and singer Dakota Staton. In 1961, he joined the Count Basie Orchestra but soon left after being conscripted into the army.