No, you do -not- have to kill all of the zoras in that room. Either there are other enemies in that room, it's a boss key door, or it's a one-way door. flitchard would be able to tell you better than I, as I do not remember it that well. I'm currently trying to find my way to the fifth dungeon, and through the Swamp Cave...hehe.
Edit: And now I found what I was missing before. I feel stupid...could've sworn I tested that wall ten times before, but now I find it bombable. *facepalms* Well, sometimes, that's what it takes, a fresh run.
Edited by Ryunaker, 24 January 2013 - 12:54 AM.