Jemison was the grandson of Robert Jemison Sr and the nephew of Robert Jemison Jr, noted Birmingham real-estate developers. He first took up his grandfather's golf clubs to hit hickory nuts in their yard and won junior tournaments while a student at Ramsay High School and at the Sewanee Military Academy. In his final year there he was promoted to cadet colonel and battalion commander for the corps of cadets. He enrolled at the University of Virginia in 1940.
Jemison enlisted in the U. S. Army at the outbreak of World War II and was commissioned an infantry officer. He was commissioned a Captain and assigned as personal aide to General Edmund Sebree, commander of the 35th Infantry Division. He was present for the liberation of Le Mans, Orleans and Nancy and earned a Combat Infantryman Badge and two Bronze Stars during his service. He also brought his eight iron with him to Europe, earning notice from General George Patton, who told him, "‘By God Jemison, if you can kill [Germans] with that [...] thing, do it!"
After returning to Alabama Jemison graduated from the University of Alabama and took a job as an agent for Mass Mutual.
Tutored by former baseball star Sam Byrd, Jemison continued to compete in golf tournaments, winning back-to-back state amateur titles (1956 and 1957) and later qualifying for the U. S. Amateurs four times. He returned to competitive golf in the 1970s as a senior golfer, with two Alabama State Senior Amateur titles (1976 and 1977), and qualifying for the U. S. Senior Open and the USGA Senior Amateur. Jemison served as president of the Alabama Golf Association, the Southern Golf Association, and later served as treasurer of the USGA's executive committee. He organized the USGA's Members Program and personally enrolled its first member, President Gerald Ford, in a White House ceremony on December 18, 1975.
Jemison was inducted into the Alabama Sports Hall of Fame in 1982. He was twice awarded the USGA’s Ike Grainger Award for his service to the organization. He and Wendell Givens published two books, one a memoir of his experiences as a soldier and golfer, and one a biography of his uncle, Robert Jemison Jr.
Jemison married the former Jess Ann Yarbrough of Huntsville and had three children, Anne (Heppenstall), Elbert III ("Bo"), and Richard.
- Jemison, Elbert S. and Wendell O. Givens (1997) Playback: From Hickory Nuts to Hall of Fame. Hole-High Publishers. ISBN 1885860102
- Thornton, William (November 23, 2010) "Birmingham businessman, golfer Elbert S. Jemison Jr, remembered as 'great servant'." The Birmingham News
- Shefter, David (November 24, 2010) "USGA Members Program Founder Jemison Dies." USGA.org
- Thompson, Ian (November 25, 2010) "Thompson: Elbert Jemison, the oracle of Birmingham golf, will be missed." The Birmingham News