I think they've made like 20 wall warts for every man, woman and child on the planet. I'm not kidding.
Basically, a wall wart is a plug-in transformer that has an AC rectifier and voltage regulator built in, with a cord.
Consider using NiMH rechargeable batteries though, particularly if you have young kiddies. Not a good idea to make them curious about things that are plugged into the wall, as they are likely to start plugging things in themselves. (yes, I did that when I was a kid. I still remember the blinding blue arc it made, too. )
Depending on the age of your kids, keep small batteries out of reach or secured in the toys, as they can be a choke hazard. Plug unused outlets. You can get safety plugs inexpensively at your local hardware stores.