midi keyboard interfacing with arduino pressing keys/turning knobs causes pops and clicks

Ok, I have a midi keyboard connected to my arduino, and I'm using the arduino midi library to read the keyboard and generate a certain frequency square wave signal based on the key being pressed. The signal goes into a speaker. The problem is that whenever I press a key or turn any knobs/wheels on the keyboard, I can hear a clicking, popping sound. Pressing a key makes one little pop, turning the knobs makes a bunch of pops as I turn them, it sounds kinda like zipping a zipper.

What's the...

midi keyboard interfacing with arduino pressing keys/turning knobs causes pops and clicks