What do you mean by cheap?
What functions do you want? For example, are debugging or ICSP required?
Do you want to program DIP chips out of circuit without using special adapters?
What chips do you want it to work with?
Are you wiling to build it?
I use PicStart Plus (Microchip) and Inchworm ICD2 (Blueroomelectronicsr). Both are available only for serial port, but will work with some serial to USB adapters. Inchworm comes in a kit and is considerably cheaper, but does not support debugging of all chips.
You can learn to program PICs for free. Go to Microchip, download MPLAB current version (8.8). Find a tutorial for programming on the Internet. You can write the program, compile it ("build"), and see how it would run all within MPLab. MPLab allows you to set breakpoints, introduce a stimulus (for example, switch changes to an input), watch registers, measure timing, etc.
Of course, you cannot actually light an LED with MPLab, but you can observe the status of the pin to which the LED would be connected, if it were a real PIC.
Not directly. In theory, you could compile the pic basic and
load the hex file. Most of your questions can be answered at Microchip.com. BTW, I just noticed the current version of MPLab is 8.3. Sorry for the error.