If I wanted to make a basic doorbell, all I would need is a capacitor, a relay, a push button, and a speaker or a bell for sound?

 

It is simpler than that. A pushbutton, a buzzer or bell, and a battery or transformer for power. It is a simple series arrangement.

 

A speaker cannot produce a sound it self, you have to apply the proper signals.

Use a buzzer or a bell which has already a built in controller to generate sounds, as Bill suggested too.