Having endured 16 months of hell from a noisy neighbour (had to threaten to sue him to fix the problem!), my sympathy lies with your neighbours ;)
First - engage your neighbours, talk to them ask them to help you solve the problem.
Second - it is not the level of noise that will be the problem, it will be the duration and repetition that causes them grief. If you can limit your noisiest periods to day-times and weekends it will keep them happier.
Third - it's much nicer to work in a quiet environment so, making sure your machines are not too noisy is always best .
It is possible to keep anything quiet; I've been on a film-set standing next to a truck mounted multi-kW generator and not been aware that it was even running. I've been in sound studios watching a drummer through a window beat the hell out of his kit in perfect silence.
There's only one way to stop noise - Mass and Isolation (I know that's two things, but they are intrinsically linked).
There will be both vibration and air-borne noise from your machining.
Fixing the airborne noise is about stopping it escaping and absorbing it, so double/triple glazed windows (that are shut while you're working ) and doors that are heavy and well sealed (look for 1/2 hour fire-check doors).
Carpeting, soft furniture curtains all help to absorb the noise (not all are conducive to machining so do what you can).
To stop vibration, mount your light-weight gear on heavy plinths (something like a couple of layers of granite work-top) float the plinth on rubber matting (e.g. carpet underlay - with more between layers. 200kg of granite takes a lot of energy to vibrate so, your small lathe/mill will not move it much and the rubber will stop that small movement being transmitted to the neighbours

As Stuart says, cabinets can act like resonators and make the sound louder - and open frame with shelves may be a better option.
I happy to suggest more if you need it

[edit]
I have the thing all put together very securely, and I get almost no vibration in my coffee cup on the counter top. Before I start ripping the whole thing apart, I wanted an idea of how isolated it needs to be.
That suggests to me that the floor and wall is adding mass to your workbench - not what you want! Pull the cabinet away from the wall and separate the bench from the floor with rubber mat. If possible add mass to the top of the bench (assuming it is strong enough) . How about a couple of cheap paving slabs under the work top ?