If you're sending your query out pretty widely (20+ agents) and getting zero bites, not even for a partial, the issue may be with your query, not your genre. Generally speaking, if the query is good, about one in ten will bite, even if they eventually pass on the work itself. No bites means it's time to reevaluate. I'll echo dios4vida and recommend posting your query for critique in the appropriate section -- there are some very helpful members in these parts that can help you find your weak spots :)
I agree that if an agent says they represent SF, then they represent SF. Straddling the genre line usually hurts you more than helps you. It's a common pitfall of querying authors to say, "Well, it's sort of science fiction, but it also has elements of humor and fantasy and literary." Try to avoid doing that at all costs. Pick your one genre and stick to your guns. Also avoid comparing your work to literary classics of the genre and/or someone hugely popular, like Douglas Adams. If you're going to compare, go with more contemporary works that sold well but weren't MEGA BOMBSHELL BESTSELLERS. Doing so shows that you're up to speed on the current (not past) climate of your genre and that you're not the sort of writer who thinks their book is the Next Big Thing.
If you're really feeling awkward about using the term "science fiction," you CAN go with the less-stigmatized "speculative fiction." However, it's a very broad term, and if your work is more space travel/aliens/other worlds-based, it'd be better just to stick with SF. Science fiction *is* still selling and in some cases doing well, so it's not a dead genre.