Rule #1: "If it sounds good, it IS good."
There is no rule #2.
The rules of key are only simplified guidelines, to get familiar sounds where everything fits. If we obey those rules to the letter, music gets very boring.
All that matters about "key" is that we can tell which chord the tonic (key chord) is. Ie, it's a matter of sound, not of text book definitions.
In this case, C#m sounds like the key chord of the verse, and E sounds like the key chord of the chorus. Pretty normal relative-major-minor scenario. (Verse and choruses can often be in different keys, and the bridge of song will normally be in a different key.)
Still, there are theoretical explanations for all good sounds (as jseth says), if you're interested.
Normally, the F# and G# major chords (in key of E) would be called "secondary dominants". The job of F# would be V of B, and G# would be V of C#m.
One of the normal rules of key is that every chord in the key can have its own dominant chord; the V chord (B in E) is just the "primary dominant". The function of secondary dominants is to drive the progression forward with more energy, to "resolve" to their "target" chords.
Compare E-A-F#m-B with E-A-F#-B to hear the difference.
Also compare E-A-G#m-C#m with E-A-G#-C#m.
NB: this doesn't change the key of the song. We haven't thereby moved to the keys of B or C# minor. We could have - if the new key persists - but more likely we'd go straight back to the home key. Eg, the B following the F# major will often be a B7, clearly indicating a return to E major.
Of course, in this case, those chords are not resolving as they "should". The F# is going to either A or G#; and the G# is going to A. These would be called "deceptive cadences" (the term for a V chord going anywhere other than its I). G#-A (in this key) is a particularly common one, because A shares 2 notes with C#m, so makes a good substitute.
NB2: secondary dominants don't need a b7. They don't need to be dom7-type chords. The only secondary dominant that needs a b7 is the I7. Eg, in key of E, when it goes to an A chord, you can make the E into a secondary dominant by adding a D (b7) - check how it prepares your ear for the move to A. Otherwise, you just need the major 3rds on the other chords to get them working as secondary dominants.
Secondary dominants in key of E major:
E7 = V of A
F# = V of B
G# = V of C#m
C# = V of F#m
D# = V of G#m
You can of course add b7s to the other secondary dominants if you want.
(I lied about "every chord"
. The vii chord, D#dim can't have its own V chord, because it can't be a I, a tonic.)