The motor is a motor-generator combo, there is no digital speed sensor that would produce a square wave. The generator is an AC generator - evidenced by the sine wave output - but the frequency is not what the speed control circuit cares about. The schematic clearly shows an amplitude (diode) detector on the generator output, so it is looking at generator voltage only. C144 and high values on adjacent resistors makes it a peak detector. Difference amplifier A107 compares the divided down generator output (peak) voltage to 5V, and drives the motor via Q112 booster. First thing is to see if the 5V is clean, stable, and within tolerance. Next check +12V into Q112 to be clean, stable, and within tolerance. Since C144 stabilizes the generator voltage, if it went bad then the generator signal to A107C would pulse and muck with the speed regulation. On an old unit electrolytic caps go bad first, I would almost bet it's a problem with the +12V or +5V supply because of bad power supply filtering.

 

Yes that is true, but if it is a sensor on the motor so why wouldn't I see more of a square wave?

To regulate speed we need speed feedback. The word tacho normally implies such feedback as a DC voltage proportional to speed. Accordingly the servo system tries to match it with a reference voltage. There is another possibility - comparing frequencies, not DC voltages.
In this case a reference frequency is provided (instead of reference voltage) and a frequency feedback is provided by inductive sensor -typically tooth-wheel and a pick up coil with permanent magnet. This type of sensors produce sine wave. The classical tool to regulate speed in this way is called Phase Locked Loop (PLL). It comes in dedicated IC, but sometimes could be made by general components. The block diagram does say TACHO - so we can assume DC voltage proportional to speed. This is typically made from multiple phase AC (to reduce ripple) through rectification. Conceivably the rectifier could fail in way to produce sine. It may be incorporated in the motor and is just a diode rectifier (with pairs of diodes in half bridge arrangement per phase).