I have the same issues, finding these two intervals most tricky to separate from each other.
For me it helped to have in mind that the perfect fourth is actually regarded a dissonant interval, while the perfect fifth is highly consonant (this due to the placement in the overtone series where the fourth shows up quite late).
If you listen, the fourth always has a bit of a "buzzing" sound, from the interferences of the waves, while the fifth lacks this and sound almost like the same tone (when played simultaneously).
Another way is to rapidly try to sing a triad within the interval. If it works you're surely dealing with a fifth.

I don't know if this is going to help you, but it helped me and is helping other guitarists. As a reference point for the 4th you can take the verse riff for Smells like Teen Spirit (C and F) or double stops commonly used in rock and blues (think Johnny B. Goode). Fifths are powerchords on guitar and most rock and metal riffs are based on them (I mostly use the riff for Ironman by Black Sabbath as a reference point).

Good tips there, I don't have a tip to share but I can tell you that while you struggle on these your brain is doing work behind the scenes and if you keep at it eventually it'll get easier to distinguish them, keep working on it daily and you will win

Hey, everybody! Thanks for the thoughts. Agree with Adam that there is a distinct buzziness to fourths, but that becomes less distinct as you go higher in register. Thinking in terms of basic RnR riffing is definitely helpful as well!