I had posted a question about this in the Arduino.cc forum, where forum user dlloyd had helpfully pointed me to an excellent explanation of using MOSFETs as power switches. I found a very useful circuit diagram:
It uses two MOSFETs: a P-channel MOSFET up top, and then an N-channel MOSFET at the bottom. The EN is the pin by which you control the bottom MOSFET: when it is high, you allow the N-channel MOSFET to open and pull the gate pin on the P-channel MOSFET to ground (it is normally pulled high to the VIN+ through resister R1), opening the P-channel MOSFET to connect the Load to the VIN+.
I set this up on another small breadboard, without a button, to test it out. It worked, and was super stable (unlike before, where I had to adjust the pot periodically). I used a 10k resistor for R1.
The new power circuit in all its glory: