There is a lot of confusion going on here. I have put a statement you have made in bold, because I think it is important to get this straight. The fact that a capacitor blocks DC does not in the least alter the requirement to have a loop. AC circuits require to be complete just as much as DC ones do.Thanks all for the great info. BUT this brings up another question. Some of you have said that there must be a closed loop.... this is not the case with a capacitor! There is no loop through it, but a difference of potential on the plates which allows an electric field to exist inside the cap. So... difference in potential, not the path, is key here right? So I could keep the + lead in the circuit, put the - lead to any ground, even earth ground, and still get the field in the cap. Right? Wrong?
Actually, the function of a bypass capacitor frequently requires it to be placed as physically close as possible to the point in the circuit where it is required, with both the "hot" and ground leads kept as short as possible to keep down parasitic inductance and resistance.
Edit: Connecting a bypass capacitor to a physically remote ground might get the same DC field in the capacitor, but generally it might not fulfil the purpose of providing a low-impedance local path for supply current fluctuations. The result might for instance be the creation of ground loop problems, possibly resulting in interference pickup or instability.