The circuit works just as intended now, but I have a question about the logic.It's true that, with the switch not pressed, the LED on the output of the 74LS04 will be lit. That's probably not what you intended.
When the signal from the OR gates is high or low, how come only 1 of the LEDs light up when the button is pressed? How come both LEDs don't turn on, when in the schematic and breadboard, both LEDs are comng from the same "root".