I talked to the diamond machinists at work today about this. Recommendations were that it was the tool and the ways. Oh, and the operator. Something about the nut at the wheels being off...

I was told that the finish cut should be at a faster machine speed, and that the tool should have a large radius. We manufacture tooling for lenses, and some of the ones we do are plain mirrors - called "planos" in the biz. The tools they use are large radius diamonds (large for them being in the .25"-.31" range) on hard copper. When the part is done, you can't tell it's been turned, but is a circular mirror of up to 24" diameter. Plus or minus .0005 is wide open tolerances for these guys.
As to the ways, they said they are too lose, and when I'm taking larger cuts to get to the finished diameter, the wavyness is the result of uneven pressure on the tool against the work. It sort of work hardens the surface, so that the finihed cut can't remove all of it evenly. So lighter cuts on the turning if I can't tighten up the ways more. Lapping the gibs should help, but I've been avoiding it. It was also recommended that I back off the leadscrew nuts before tightening the ways, as that may be influencing the way the slide fits on the bed.
So, I'll try tearing down the machine and lapping the gibs. Then reassemble, paying particular attention to get the best adjustment at the point where I do the most turning (near the headstock), and then worry about backlash. I'll post results when I get it dialed in.