Pitts was the second son born to Eugene Arratious and Katherine Daniels Pitts, sharecroppers, in Jones County, Georgia. His father named him after Bishop L. H. Holsey (1842-1920), who founded Paine College in 1883.
Pitts was licensed and ordained as a minister in the CME Church in in 1931. He completed his undergraduate degree from Paine College in 1941 and graduate degrees from Fisk Universita, Atlanta University, Peabody College and Western Reserve University.
After some time as head of a private high school in Cordele, Georgia, Pitts became the executive secretary of the Georgia Teachers and Education Association, representing 11,000 black teachers.
In 1961 Pitts was appointed president of Miles College. He led the college to a doubling of enrollment and a tenfold increase in its annual operating budget. He recruited alumnus Richard Arrington Jr to return to Miles acting dean and director of the college's summer school. Then quickly promoted him to chair the Natural Sciences Department.
Pitts secured Miles College's accreditation in 1969. A year later he brought national attention to the school when he convinced Harvard dean John Munro to leave Cambridge for Birmingham to serve as Miles' director of freshman studies.
He left to accept the presidency of Paine College in 1971 and died in his office at age 59. He was buried next to the Gilbert-Lambuth Memorial Chapel on campus.
Pitts served on the board of the Southern Regional Council in Atlanta, and on a Commission on Cooperation and Council between The United Methodist and The Christian Methodist Episcopal Churches. Pitts was an advocate for Civil Rights. He advised Frank Dukes, Miles student body president, who went on to form an Anti-Injustice Committee, which initiated a Selective Buying Campaign against downtown retailers.
During the 1963 Birmingham Campaign he was a member of the Central Committee, and helped organize the Salute to Freedom '63 concert at Miles to raise funds for the planned "March on Washington for Jobs and Freedom." He was later elected co-chair of the Community Affairs Committee for Operation New Birmingham, tasked with improving race relations following the Birmingham Truce. He was named to the bi-racial board of the American National Bank in 1964.
- "Lucius Pitts, Educator, Is Dead; President of Paine College, 59" (February 27, 1974) The New York Times