Chris Brown (vocals, keyboards) and Kate Fenner (vocals) were both founding members of the eight-member Bourbon Tabernacle Choir, who made a battle-of-the-bands appearance at the Masonic Temple in Toronto in 1985. The band was named after a dream that Brown had. The Bourbon Tabernacle Choir played the club circuit in Toronto, and continued to release cassettes independently. In total, they released three recordings between 1987 and 1990; the last, Sister Anthony, was produced by the legendary Bob Wiseman, and was the one that got the band noticed. Because of the Choir's independent success, the independent label Yonder picked them up and they released Superior Cackling Hen. They later released Shyfolk, which would prove to be their last album as an entity. The band decided to move to New York and try to make it there; this would prove to be its undoing.
Brown and Fenner were the only bandmembers to stick it out in the Big Apple, and both worked on side projects, Brown collaborating with Tyler Stewart of Barenaked Ladies and Gordie Johnson from Big Sugar to form Don't Talk...Dance! and Fenner appearing on Barenaked Ladies' Gordon, as well as a solo album of former Bourbon Tabernacle Choir member Dave Wall. The two of them released their first album as a duo, Other People's Heavens, in 1997. They toured extensively in support of the album. Brown was called upon by Barenaked Ladies to play keyboards for them when permanent bandmember Kevin Hearn fell ill. This was a great opportunity for Brown and Fenner, who, as a result, got to open up for Barenaked Ladies numerous times on tour. Geronimo, their second album as a duo, was recorded in the fall of 1998 by Scott Harding at Orange Music in Orange, NJ. Tony Scherr (bass), Mike Mazor (drums), and Michael Blake (saxophones) supported Brown and Fenner. Most of 1999 was spent touring to support Geronimo, which was released on their own label, b-music, and distributed in Canada by D.R.O.G. The Go On EP followed in 2004.