Starbuck was the son of Jesse Gayer and Amy Cox Starbuck, devout Ohio Quakers. He was hired by Robert Jemison Sr's Southern Bridge Company as a civil engineer in 1886 and moved to Avondale. Starbuck was appointed the company's secretary-treasurer in 1896.
Starbuck married the former Belle Cooper on December 27, 1888. He and his wife were active members of Avondale Methodist Church, and he served 20 years as Superintendent of its Sunday School and also served on the committee that constructed the second Avondale School building on Avenue C in 1890.
From 1887 to 1893 Starbuck was a member of the Avondale Board of Aldermen. He left to take on the position of tax collector for the city of Birmingham, and was later appointed as city engineer and elected Mayor of Avondale. In the 1909 Birmingham municipal election he was elected to represent the 10th Ward on the Birmingham Board of Aldermen.
Starbuck was also secretary-treasurer of the Avondale Building & Loan Association and president of the Avondale Library Board. He died in 1921 and was buried at Forest Hill Cemetery alongside his wife.
- Browne, Catherine Greene (2007) History of Avondale. Birmingham: A. H. Cather Publishing Co.