Grafman spent his boyhood in Pittsburgh, Pennsylvania, where he studied at the public schools and at the University of Pittsburgh. He entered the University of Cincinnati in 1926 and earned his Bachelor of Arts. From there he went on to Cincinnati's Hebrew Union College. He was ordained in 1931 and began his career as rabbi at Temple Adath Israel in Lexington, Kentucky.
Grafman arrived to take up the pulpit at Temple Emanu-El in Birmingham on December 7, 1941. During his tenure he joined the group of clergymen that attempted to defuse racial tensions during the Civil Rights movement. They criticized the city's segregation laws and George Wallace's 1963 inaugural speech calling for "segregation now, segregation forever."
But the clergymen also irritated civil rights leaders later that year by asking Martin Luther King, Jr to delay demonstrations in Birmingham and wait for the courts to act against racial discrimination. He co-signed the open letter, "A Call for Unity" to which Dr King responded with his "Letter from Birmingham Jail".
Grafman was one of the founders of Spastic Aid of Alabama (now United Ability) and served as its first president. He also helped establish the Institute for Christian Clergy to promote understanding and cooperation between Jewish and Christian leaders.
Rabbi Grafman continued to lead Temple Emanu-El until his retirement in 1975. He died at Baptist Montclair Hospital twenty years later. He was survived by his wife of 64 years, Ida Weinstein Grafman, two children, Ruth and Stephen, four grandchildren and two great-grandchildren.
Grafman's personal library was donated to Birmingham-Southern College.
- Saxon, Wolfgang (May 31, 1995) "Milton Grafman, 88, a Rabbi Active in Civil Rights Struggle". The New York Times
- Bass, S. Jonathan (2001) Blessed Are The Peacemakers: Martin Luther King, Jr., Eight White Religious Leaders, and the "Letter from Birmingham Jail". Baton Rouge: LSU Press. ISBN 0807126551