The one thing that can cause a disaster is overheating the cells and making them explode. I have seen that happen a few times, never done it myself.Add a couple of diodes in series with the 5V out.
2 x 1N5004 diodes will work ok. Each diode will drop around 0.7V so the result will be pretty close.
One amp seems like a very high charging current, that would deliver the 1200 Ma Hours in just 1.2 hours, not 16. Could the desired current actually be 0.10 amps? THAT would seem a lot more reasonable. But feel the batteries every few minutes to see if they are getting hot.