As a music fan I personally would not advise dropping bit rate, especially as it appears that your station's selling point appears to be playing specialist music for the Hispanic community of the USA although (I think) a narrower range than what would be more loosely called "Latino" music (that also influences a lot of pop dance from the USA and EDM genres in Europe via links with Spain / Balearic islands)
100 listeners is a lot for an online station and I work closely with one of the most popular online EDM stations which is regularly on the "top 10" list. (our streams are separated by music genre). Online radio is in its infancy compared to terrestrial radio - a decade rather than a century old.
Also it isn't that easy outside some European countries to listen to an online stream on the move - from what I've gathered smartphone data plan's aren't as generous in every part of the USA
.
When monitoring who is listening to Party Vibe Radio's live stream I've noticed the bulk of our listeners are in Northern Europe, Asia and particularly countries where there has been a long standing culture of "listening to radio via a cable" which dates back from the mid 20th century in many nations.
A lot of them are also young middle class students who are in halls of residence at a campus university or their student accomodation where they get a cheap/fast broadband conection, or working in "dot.com" type offices/businesses where not just a work PC but tolerance of the use of work resources to listen to music is available.This of course isn't the situation for 90% of people all over the world.
I've put a fair bit of effort into this demographics monitoring, ranging from directly engaging with the listeners who use our popular forum through to geolocating IP addreses and being mindful of different global timezones.
If you want more listeners and also advertisers/sponsors and can cope with the extra demands/bureaucracies I would also suggest investigating the new community radio licenses the FCC appear to offering (which would not prevent you also having your online stream). The effort involved may not be as prohibitive as it seems and most areas seem to encourage stations catering for ethnic minorities and their music, along similar lines to the European practice.