It doesn't matter if it kinda-sorta-works at 100ma, because it WON'T work at 1.5-Amps.
An LM317 Input-Voltage needs to be AT LEAST ~3-Volts above the expected Output-Voltage.
Some models will be slightly more, and others will be slightly less.

If my very advanced math is correct:

5 - 1.5 = 3.5 > 3

 

Below is the LTspice sim of a simple hysteretic type, buck switching regulator using common parts that you can play with:
This type of regulator does have a relatively high ripple voltage (likely not a factor for powering digital circuits), but it is unconditionally stable with no feedback-loop compensation.
The sim shows an output ripple voltage of about 80mVpp with a 1.5V output and 1.5A load.
The simulated efficiency is about 80%.

 

5V - 1.5V = 3.5V, LM317 can do that at ~1.5A with sufficient heat sinking.