Boswell graduated from Ensley High School and enrolled at the University of Alabama. He played football and baseball for the Alabama Crimson Tide. In his three seasons at left halfback under Coach Frank W. Thomas he also set punting record which stood for over 30 years. He graduated in the class of 1940 with plans to pursue a baseball career.
The year after graduation, Boswell entered the U. S. Army and was deployed to Europe with the 84th Infantry Division during World War II. He married Kathryn "Kitty" Lacy on April 5, 1942. On November 30, 1944 he was blinded by a German artillery shell that exploded while he was pulling an injured fellow soldier from a tank. He retired from the army with the rank of Major in 1946.
Boswell began playing golf with a sighted caddy to help guide his shots while at the Valley Forge General Hospital in Philadelphia, Pennsylvania. He helped to establish golf as a competitive activity for the blind, winning 28 United States and International Blind Golf Championships. He once shot a round of 81, and recorded a hole-in-one in the mid-1970s. Through golf, Boswell became friends with comedian Bob Hope, astronaut Alan Shephard, and numerous professional golfers. He would throw sighted golfing partners off guard with comments like "That's the worst swing I ever heard" or "I'm glad you see the hazards on this course -- they'd scare the hell out of me!"
In 1954 he founded the Boswell Insurance Agency, and later worked as an agent of Employers Insurance with offices in the Brown-Marx Building. From 1971 to 1979 he served as Commissioner of Revenue for the State of Alabama. He served on the board of the Callahan Eye Foundation Hospital and, from 1990 until his death, as Chairman of the Helen Keller Eye Research Foundation.
His Charley Boswell Celebrity Golf Classic, first hosted at Port Aquarius Golf Course in Talladega County in 1974. The annual event, which made its way to the Charley Boswell Golf Course at Highland Park, raised over $1.7 million for the Eye Foundation Hospital. The Birmingham VA Medical Center's Southeastern Blind Rehabilitation Center, dedicated in honor of Boswell, was opened in 1982.
Boswell's health declined after he suffered a fall in March 1995. A blood clot developed in his brain and he succumbed while at HealthSouth Medical Center. He was survived by his wife, two sons Steve and Charles, Jr, and a daughter, Kay McCarty. Boswell is interred at Elmwood Cemetery.
- Boswell, Charley (1969) Now I See. Birmingham: Highland Press. ISBN 0963027301
- Philadelphia Sports Writers Association Award, (1957)
- Ben Hogan Trophy of the Golf Writers Association of America, (1958)
- "This is Your Life" on the Ralph Edwards Television Show, 1960
- Football Hall of Fame Foundation Distinguished American Award, (1965)
- President's Distinguished Service Award from the Committee on Employment of the Handicapped, (1972)
- Alabama Sports Hall of Fame, (1972)
- Alabama Academy of Honor, (1983)
- United States Blind Golf Association Hall of Fame, (inaugural class, 2007)
- "Charley Boswell, Blind Golfer, 78." (October 25, 1995) New York Times