Handley is the son of Loyd Handley and the former Elda Mae Miller. He operates Columbiana Eye Care at 112 South Main Street. He married the former Shelia Diannie Lindley in 1980 and has three children.
Handley has served as president of the Shelby County Cattleman Association and vice-president of the Alabama Cattleman Association. In 2008 he proposed a "Cowboy Parade" to be held the day before the annual Shelby County Cattleman's Rodeo. That event has since expanded into Cowboy Day in Columbiana.