Head back to Hyrule. Find this screen and super-bomb the blocked cave entrance.
That screen is on the top-most row of screens, five or so screens east from the upper left corner.
From in that cave, go one screen north, one screen east. Bomb the rock in the middle of the + shaped formation. Drop down.
Go east. Use your Ice Arrows on the fire. Explore these two rooms. One of them will have a Subrosian.
Be sure to walk around a bit in the middle of the room. That triggers something important.
Next, go to the Swamps. Go west of the entrance to the Excavation two screens, then two screens south. Move the heavy boulder and go in the stairs entrance.
Make your way two screens east from where you arrive. You'll see a chest on the sand. Move into the northwest corner of that room, hugging the corner. Go a single tile diagonally southeast of where you're standing now. Use a Super-Bomb on that sand where you're standing. That will open the entrance to Subrosia.
From there, go south and west, and down the stairs. From there, it's just straight north to the L4 Sword, fighting enemies on the way.