I probably don't get original question completely, because I have no radius turning equipment, but I would approach the problem like this.
1) Do you need long continuos movement under load or do can you set this "slide" to correct position and then clamp?
If you clamp it, you need only a small forces then, right? If extreme locational accuracy is not needed I would use consider pin and groove:
* Standard size pin, roller bearing pin etc. Fixed square to the movement. I.e. trough a plate and glued to it. It protrudes a litle.
* other part has a square groove
* This would only locate one axis and prevent parts from twisting out of plane. But would not retain it.
2) If you need precision, or forces are greater, then it's easier to do with line contact. E.g. V-groove vs. rod, but it all depends on forces and which directions they are acting.
I never managed to perfectly square hole between two parts with a twist drill. Got close when I milled a small grove on both parts to clear most of the drill web but only once. I would first drill/ream the hole and saw part in two with a thin saw, but that's not an option here.
Pekka