Basically, I think that you have got the rudiments of a very nice machine. My drawings( such as they are) include holders for cutters. Of course, you have a set of collets instead. Whether you want to do lathe tools, I can see no item but it is not a problem to make a square holder. In my drawings, there are two sizes of holder for taper and parallel holes/holders. Again, no great problem as these sort of follow what I have.
All that I would add- but merely for the sake of completeness, you will require a lot of dummy practice.
Altogether? NICE
Regards
Norman