The H2 will do what you describe as well as, and possibly a bit better than, the H4. You didn't mention overdubbing, which is possilble with the H4 and H4n, not natively on the H2.
All these units can be connected to your computer and used as an audio interface. In this mode you use the built-in mics (or attach mics to the H4/H4n), preamp, and a/d but record directly to your computer hard disk.
I would choose the H4n over the H4 because it feels more rugged and you can record four tracks simultaneously using the internal and external mics. The H4n also has an easier to read screen, a slightly simpler user interface, and a monitor speaker.
But I would select the H2 for the use you describe. It's more compact, easier to stand on a convenient flat surface without any mounting device, provides extremely decent recording quality, and it's bucks cheaper.
The H4/H4n come with Cubase LE audio software, but you can download Audacity and use it without any payment, or download Reaper and use it without restrictions until you're ready to register for a very reasonable cost.
I've done a comparison of the H4n, the H2, and two other recording systems for my Home Brewed Music blog:
http://www.homebrewedmusic.com/2009/...-the-zoom-h4n/
Fran