I came to a decision about the handle, (partly as I wanted to just get it finished, and partly as I wanted to use the machine from that end of the table!).
So, I measured up, removed all the feed unit etc, drilled the shaft (taper reamed), re fitted etc...

Made up a quick little collar...

..and then refitted the handle, (and taper reamed the handle+shaft as one), fitted the taper dowel, job done...


Another job ticked off the list!
I can always make up a new shaft, and fit a clutched handle if I change my mind in the future! (I don't think I will be though, as we have clutched handles on the bridgeports at work, and they drive us mad - faffy things which only really work if you stand at the end of the table facing the machine so you can push on them - no good for standing in front of the machine)