Hi Picclock,
what Jari said about workpiece temperatures. Either work using coolants, or allow some time (it can take a lot!) for the workpiece to cool down before the final cuts. I've had some luck with bores made slightly oversize that shrunk to dimension, but I would have not tried that on workpieces that required more than 5 minutes to make again.
What works for me, especially when working with inserts, was making the final cut almost the the same depth than the roughing cuts. I/e: if I had a consistent diameter reduction when removing 1mm/dia during the roughing cuts, that will be the diameter reduction (now I think of it, that would be increase, in bores) I will use on the finishing cut. I may go 0.9 or 1.1, but I've quit shaving the wpcs. 0.1mm each passage. Be confident, take courage (and practice on scraps, first).
Another point I take special care into, is never making cuts less deep than the insert point radius. You advance the tool, say 0.1mm and the tool does not remove material. Another 0.1 and nothing again. On the third 0.1 it removes 0.4 and You're undersize.
Marcello