Darren, single phase capacitor start motors will run in either direction and in fact you can usually make a uni-directional one reversable if you know what you are doing.
The way they work is that there are two wirings, the 'run' winding which gets full mains voltage through it and the 'start' winding which is in series with a capacitor.
In other words, thinking of the start winding alone, you have one end of the winding connected to the mains and the other end connects to the capacitor and the other end of the capacitor connects to the other mains lead. Either the coil end or the capacitor is connected via the 'start' centrifugal switch which opens when the motor is spun up a little.
To reverse such a motor the combination of start winding and capacitor is reversed in the circuit.
Preci: You just connected something wrong.
P.S. Are you sure it is not a one of Bogstandard's left handed or American capacitors?
