Can you clamp/bolt the pulley flat to the face plate, and rebore the bore? Then shrink or press in an insert of some kind.
In my experience with big pulleys clamping to a faceplate is by far the most stable way of machining them. Once you've done the bore you can then clamp it from within the bore to the faceplate and turn the outside too, if you see what I mean, meaning the whole lot will definitely be in line.