I've searched on this topic and got the answer that the input capacitor is there to remove any high frequency components. But the input to the regulator is mostly a DC input where does this frequency component comes from? One more reason that was given was stability, what stability are we talking about here?
Are ceramic capacitors any better than electrolytic capacitors in this respect? If yes, the why?

 

That information is on page 1 of the TI datasheet:

One more reason that was given was stability, what stability are we talking about here?

Transient response.

Are ceramic capacitors any better than electrolytic capacitors in this respect? If yes, the why?

Ceramic caps are better at higher frequencies. Electrolytic caps offer more capacitance per area.

 

A linear voltage regulator is a feedback control system. It is capable of controlling the output voltage with extremely fast response times.
A critical criteria in the design of a control system is the feedback delay time-constant. RC circuits contribute to this time constant. External capacitors are required to lengthen the time-constant and hence reduce the likelihood of high frequency oscillations in the feedback mechanism.

DC is a misunderstood label. In frequency terms, DC means 0Hz.
All signals have a frequency content from 0Hz to infinity. What you think of as being DC always has some higher frequency content.

All capacitors have internal resistance and inductance. The effective series resistance (ESR) is a quantitative model of this internal resistance and inductance, and is a function of frequency. The ESR of a ceramic capacitor is different from that of an electrolytic aluminum capacitor or an electrolytic tantalum capacitor. What works in one scenario could be detrimental in another. Knowledge and experience will help you decide what is the right one for the given application.

 

Relative capacitor ESR technology performance -

http://www.ti.com/lit/an/slyt194/slyt194.pdf

https://www.edn.com/design/test-and-measurement/4401984/Understand-linear-regulator-stability


Regards, Dana.