As long as you're playing tonal music, chord types and intervals will be strongly associated with the 7 note names, not just the 12 semitones. A perfect fourth will always be the same size on the staff, for example.
If it's "spelled" differently -- if an interval of that size is actually written as C-E#, for example -- then that's a sign that something weird is going on and you're not supposed to think of this interval as a fourth.
The fact that we write music using the 7-note diatonic scale is kind of like compression: it optimizes notation for the notes and intervals you're most likely to play. It's just not suited for atonal music, much like compression is not suited for random-looking data.
I know little music theory, but I've been playing instruments "by ear" for a long time and it made many things easier to me when I learned that if you're using diatonic scale, it's very unlikely that you'll use notes outside of the scale; e.g. if you're playing a song in C Major, most of the time you'll play C and D, not C# or Db, so if you need that note, it's easier to switch than having an almost unused line in the sheet.
If it's "spelled" differently -- if an interval of that size is actually written as C-E#, for example -- then that's a sign that something weird is going on and you're not supposed to think of this interval as a fourth.
The fact that we write music using the 7-note diatonic scale is kind of like compression: it optimizes notation for the notes and intervals you're most likely to play. It's just not suited for atonal music, much like compression is not suited for random-looking data.