Forming in Melbourne, Australia, in 1995, and soon building a strong live following, Violetine signed a deal with Mushroom Records' Bark label. Their first EP, Violetine, was released that same year and was followed by the single "Gutless Feeling" in June 1996. By the time their next single, "Red," was released in March 1997, drummer Steve Leicester had been replaced Glenn Maynard, who was in turn replaced by Damian Pratt.
The band supported the touring Everclear in April 1998 and issued a single, "You Know" (May 1998), and the EP Birmingham (August 1998), both from their upcoming debut album. Small Speaker Joyland was released in September and, at the end of the year, Violetine left Bark for Epic/Sony. Their first release on their new label was the single "Any Day" (April 1999), followed by a reissue of Small Speaker Joyland (remixed), which again failed to find mainstream commercial success. Their next single, "Crush," was released in October 1999.