Gilmer was the son of Francis Meriwether and Sarah Eleanor Taylor Gilmer. He was educated in Montgomery County and graduated from the Georgia Military Institute in Marietta in 1858. He helped organize the military company known as the Metropolitan Guards which participated in taking the Pensacola Navy Yard and the barracks and fort at Barancas on January 12, 1861, one day after the secession of the states of Alabama and Florida.
During the Civil War he served with several different units as quartermaster, adjutant, and inspector general. After the war, Gilmer returned to Montgomery and took over his father's cotton brokerage. He subsequently became involved in the Elyton Land Company and the creation of the city of Birmingham.
In 1884 Gilmer was appointed by Governor Edward O'Neal as Adjutant General of the state and chief of the governor's staff. After two years in that office, he moved to Memphis, Tennessee, for a short while, returned to Montgomery briefly, and in 1889 moved his family to Seattle, Washington, where he lived for more than twenty years operating a general collection business.
Gilmer was married in 1864 to Lizzie B. Dixon of Memphis, Tennessee, and they had nine children.