Anyone who mixes audio needs a good working knowledge of which instruments “live” in which frequencies. But this is especially true of live mixing, where the volumes tend to blend everything into mush if you’re not careful. Use a high-pass filter to carve away the low end of instruments that don’t sit at the bottom of a mix, like horns or vocals. Use a low-pass filter to tame percussive high-frequency blasts from instruments such as snares.
If things do get muddy, try boosting the frequencies that are above the mud. For example, if your toms are a problem, try boosting their high end, and cutting the low end and/or low mids. This can reduce the low-end wash while still giving your toms presence in the song. The more you know about the frequency range covered by each instrument, the better you’ll be able to mix for clarity and separation, even in a dense, loud live mix.