Compression doesn’t just clamp down on signal whose level is over a certain threshold; using a compressor effectively brings up signal under a threshold (since with the dynamic range reduced, you can increase the level of the signal). A side effect of this is that in addition to raising the level of quiet passages, it also brings up any noise on the track.
If this becomes a problem with your tracks, you’ll be happy to hear that nearly all DAWs include a “strip silence” function that will analyze your track and slice out all the audio below a certain threshold, leaving you with multiple regions consisting of your desired audio signal, and removing the noise. If you use this function before compressing your track, you’ll be able to compress your signal to your heart’s content without worrying about raising the track’s noise floor.