You will also need some power source for the device.
You need a thing called a FTDI, that plugs into the USB port on your computer and it spits makes a virtual serial COM port. The output from that board needs to be 3.3V logic levels. Most of the so called FTDI boards can be switched between 3.3V and 5V. Make sure the one you get is switchable and set to 3.3V. Under $2 NOTE, you will need to find the proper driver for this board and they do not come with them. This can be challenging. Look at the numbers on the chip on the FTDI board and search.
You will need a way to power the esp8266 board itself. The board wants to run off 3.3V. There are a ton of ways you can go here. I have seen everything from regulating down the 5V on a computers USB port to small regulator boards that run off a wall wart, to using discrete regulators and batteries. I prefer one of the boards that plugs in the end of a solderless breadboard and can run off your USB port or a wall wart and supply 3.3 or 5V. I run mine off of a 9V wall wart. Make sure it is set to 3.3V. Under $1 for the board and under $4 for the wall wart.
You will need either some wires with female IDC connectors on one end and male's on the other or a "carrier board" for the module to allow you to plug it into the wireless breadboard. 40 F to M wires will run you under $4, as will a carrier board. I prefer the carrier board, but having the wires on hand is handy for other things.
Having a solderless breadboard is almost necessary if you want to play with more than LED's hanging off the I/O pins. Under $5 for that
Male to Male IDC wires are handy. You can use hook up wire but once you have used the ones with the ends it is hard to go back. Figure another $4 there.
And a few bucks in periphery. Some dropping resistors for LED's, some LED's, perhaps a few transistors for bigger loads...
I figure for around $20 you can have a pretty cushy set up