A lot of the horror stories out there are from people who tried it a few years back, when things were much less well sorted. When your ISP supports it, it's not too bad. At my current house my cable provider turned it on and it took me about a week to notice something had changed.
Supporting dual-stack servers is slightly trickier. A lot depends on how old your software is. Recently maintained software like Apache will happily dual-stack right out of the box. I run a couple MUSHes that have 1990s-era code bases and they were a little trickier. One of them I was able to coax into accepting IPv6 connections over a shared 4/6 socket. The other one I ended up using socat to accept IPv6 connections and relay them via IPv4, which is ugly, but effective.