ICs need to have power and an input signal as a minimum. Even then, you will only be able to test basic functions. Beware of switch-bounce if you are manually checking counters/flip-flops. You cannot expect a manual switch to issue a single pulse.
Also, any frequency sensitive issues will not be tested with manual switching for a minimal test.
Any microcontroller/ROM/RAM/or other programmable chip will be untestable without a program or controller.