Taylor was a dance student at the Alabama School of Fine Arts where she met theater student Orenda Fink and put together Little Red Rocket. After she graduated in 1994, the band enjoyed a rapid climb into notability, signing with Geffen Records in 1997. The deal failed to produce a release and the band regrouped in Athens for a year before splitting.
Taylor and Fink then got involved with Omaha, Nebraska's Saddle Creek Records, working with Now It's Overhead and forming the duo Azure Ray which has released four albums. Taylor herself became a regular in Now It's Overhead and has appeared as a guest on recordings by Crooked Fingers, Bright Eyes, The Bruces, The Faint, and Moby. Her first full-length solo album, 11:11 was released on Saddle Creek Records on May 24, 2005. She has been romantically linked with Saddle Creek founder and Bright Eyes frontman Conor Oberst.