Podcast are kinda like a radio show, but done in a computer format (usually MP3). you can listen whenever you like.
You can subscribe and download the file to your computer, or, often, you can just listen on the web page of the show.
Best podcast ever and a great place to start? Coverville. Great music show of cover music. Brian Ibbott has an seemingly endless collection of tunes. As of this writing there are about 560 Coverville shows.