A couple of things that haven't been mentioned that I really enjoyed when I visited my father in Atlanta a few years ago are the Coca Cola headquarters and the Underground shopping area.
The coke place was basically just cool because I was a kid and I liked learning more about my favorite drink and seeing all the strange products with the brand on them.
And the Underground was just really nice in general. There were a lot of craft shops and a kind of carnival atmosphere. It would probably be better for older kids, but if they are starting to get to the point where they enjoy shopping more than rides, then it's a good place to go.