I thought the problem might be decoupling, but couldn't get to my computer to post.
Without knowing the parts or the circuit, here is some general guidance on power supply decoupling. The standard rule of thumb is a ceramic capacitor directly across the chip's power supply pins, as close as possible with the shortest possible leads. If the chip has two power pins, such as an opamp, then one cap from each pin to GND. 0.1 uF to 1.0 uF are common values. In many Analog Devices app notes, they show an additional 10 uF electrolytic capacitor in parallel with the ceramic ones. For something like an audio power amplifier chip, increase that to 100 uF.