Pittman earned his undergraduate degree at Davidson College in North Carolina and graduated magna cum laude from Harvard Medical School in 1952. He completed internships at Massachusetts General Hospital and the National Institutes of Health before joining the faculty of the George Washington University School of Medicine as an instructor. He moved to Birmingham in 1956 for a residency under internist Tinsley Harrison.
Pittman was tapped as Chief Resident for the Department of Medicine. After joining the faculty, he organized the first Medical Student Research Day in 1964. His own research was conducted into thyroid physiology and disease and authored a textbook on the subject. He later served as director of the Division of Endocrinology and Metabolism and co-chairman of the Department of Medicine. He was named Dean of the School of Medicine in 1973. Among his first actions was to extend the curriculum from 35 months to 4 years. Over the next 19 years he became well respected for his ability to recruit and retain high-caliber physicians to the UAB faculty.
Pittman commissioned a painting, Medical Giants of Alabama, from Mississippi portraitist Marshall Bouldin to hang in a lounge at the UAB Pittman Center for Advanced Medical Studies (PCAMS). Controversy over the painting's depiction of J. Marion Sims, who performed medical experiments on slaves, resulted in the painting's removal.
Pittman was married to endocrinologist Constance Ming-Chung Shen Pittman and had two sons, James and John. He died in January 2014, shortly after completing a biography of his mentor, Tinsley Harrison.
- Pittman, James A. (1964) Diagnosis and Treatment of Thyroid Diseases. F. A. Davis ISBN 080366950X
- Pittman, James A. (2014) Tinsley Harrison, M.D.: Teacher of Medicine. Montgomery: NewSouth Books ISBN 9781588382269