Anyone have any alternatives to replace the battery in the simple circuit we are all here talking about?

If there was anything better than batteries, no one would use batteries.

 

Li-ion for the most part is the choice right now,. UPS's are starting to Li-ion batteries.
It's not the choice for a car. AGM is a new technology.
We have our coin cells.
We have super capacitors.
There are batteries with 10 year shelf life.
We might have radiation batteries before too long. Extremely low current.
Then there a fuel cells.
Then for space probes, you have a thermal nuclear source combined with a thermopile.

Generators
UPS's

and probably some I missed.

 

That's a good summary, @Tonyr1084 . I had a friend who overcharged a lithium battery, and he burned down his uPVC conservatory. It was only a small conservatory but the smoke damage to the rest of the house was awful.
I'm not sure all the old lead-acid chargers circuits were so dumb. I think in some cases the transformers were clever even if the electronics weren't. I think there were leakage reactance transformers to limit the current, and ferroresonant transformers to regulate the voltage (we had a conversation about them a couple of months back). I can't help thinking that these days we'd call them "smart transformers"! After that came thyristor-based controlled rectifiers.

Working for an emergency lighting company, we got a batch of32V transformers to replace older ones. Some, within minutes would burn up, others might take a day. It had something to do with improper shimming of shunt plates I think. I'm not the transformer guy. I just know that transformers can have shunts built into their iron structure to limit current somehow. I don't understand how, but that's what I think I know.

Ferroresonant transformers are something new to me. Few years back a school cleared out their older inventory of science lab power supplies consisting of a breaker, FR transformer, Autotransformer, variable AC out, bridge rectifier and filter for variable DC out. The AC was amazingly steady - which is where I learned that FR Transformers are pretty neat. Subsequently I've heard they are somewhat inefficient. The DC could pump out around 135VDC; so care is needed. But transformers on the whole are pretty basic things. Smart? I have yet to meet a smart transformer that consists of two coils and some kind of core.

Tinnitus at 2600Hz doesn't help.

Don't know my frequency but it's probably up there too. Worked for an electronics wholesale store. In the morning the boss always forgot to turn off the ultrasonic motion detectors. I could hear them. But then I became a rock-n-roll drummer. Played them for 8 years; loud and hard. Now it's not easy to hear the TV and when my grandson talks it's almost a whisper. Wish I'd have protected my hearing when I was young.

After this experience, batteries seem like a bad choice to power life.

Batteries are like cars. If neglected or abused they can be dangerous. But with proper care, maintenance and use they can be quite safe.

Alternatives to batteries? Well, someone already went through a pretty good list. But overall, the best sources of energy are nuclear, hydrocarbons, then batteries. There are other sources of storing energy, such as a flywheel, but that's huge, heavy and capitol expensive. Capacitors are gaining in the field. Super caps have greater energy density, but their voltages remain low. A good super cap can be a couple farads and only 2 volts maybe. Again, there are a lot of things I don't know. But as yet super caps are not a good choice for emergency lighting. Batteries remain the best option for that. After that there's generators. The benefit of a generator is that it can power a whole house if you get a big enough one. But so can batteries. The "Tesla" battery is meant to power the whole house. But their energy is limited to their charge. Once depleted they're dead. Gasoline can be poured into a fuel tank. Natural gas or propane powered generators can also run for a much longer time than batteries. That's why battery powered cars are still lagging behind gasoline powered cars. But the technology is improving.

For what you want, a simple power supply capable of maintaining proper voltage on a battery is the way to power an emergency light. But by nature, emergency lights are just for that purpose - an emergency. Still, one can be made out of simple electronics. If you opt for an AGM battery (deep cycle car type battery) then maintaining it at 13.8 volts is recommended. A relay can switch the light on in the advent of a power failure. But if you just want to light a light then a transformer and light bulb is the second best way to go. No extra gadgets, extra loads, extra considerations, just a source of electrical energy to power a light bulb. Whether LED or incandescent. The BEST way to go is to get a lamp that is designed to work on your mains voltage. Least amount of wasted energy.