Itemized:
The subscreen: The top-left is done, for a good karma playthrough. You can obtain the ??? item as well. The grey area is only possible on a bad karma playthrough. You can fill all 15 slots of the main "Items" section. You can fill the "Artifacts" section as well. An easy "Artifact" to miss is actually at the very beginning area of the game; it may look suspicious to pick up but doesn't actually cause any major consequence.
-Is it possible to get endings 3 and 4 now?
Ending 3 is a bad version of Ending 1 so you don't need it. You pretty much set Ending 1 in motion by doing something special with the Mask of Truth and it can't be reversed currently. Ending 4 is also a different variant of Ending 1 (they're also both labeled as Ending 1 because I'm bad; the credits told you which variant you got though) so you pretty much found whichever one your playthrough is giving you.
So all your plot-relevant endings have been obtained for the playthrough.
-What about that big key in the beach?
You can actually obtain it if you discover one of the mega-tier secrets. It's possible to discover it by accident, but it's one of the most mysterious ones. An NPC surprisingly alludes to it but it's really hard to pinpoint with the dialogue hint. This mega-tier secret in particular is really meant to be kind of out there as I really wanted to reward the most crazy of explorers later down the road.
-How many hearts are in the quest?
You can max out at 14 on a good karma run and 15 in a bad karma run. (It was supposed to be 14 and 16 but I goofed up something with the Hammer.)
-Are there really 5 keys in the Truth? I have only 3
Some of the keys in Truth are extremely difficult to find and it's actually possible to completely lose out on them in the quest. However, there are extra keys for it in a bonus area somewhere so if you find a mega-tier secret or two, you're able to get all the Truth bonuses anyway.
-There are five characters in Quintet, but I have four characters
The final character is obtained in one of the more plot-relevant mega-tier secrets. With your other questions you're already on your way to running into it.
-No final boss?? o.O
Believe it or not, the battle where you get the Mask of Truth is meant to be the final boss of the main story. Kaoru is the bonus boss. The game is just structured oddly in that there's a main story dungeon after the final boss. Ending 1 and the area it's in reveal some big plot elements but you can't actually get at the source of them in this quest.