Hmmmmm.... the more times I read this the more I think it is flexin in the tool and head assembly. If putting down force on the tool holder makes it cut properly I think that just means it is not able to spring out of the work. Suggestions, reduce tool holder overhang, reduce cut depth, try a different speed.
I would never be able to get Sally Shaper to cut straight down with a wide tool, I always have to use a pointed tool and use the square end or otherwise shaped tools for only very light finishing cuts.