Hi there, Pekka,
First of all, it's important to remember that 1 inch of steel makes 13 inches of rust so don't panic! It looks worse than it is!
There have been a couple of threads on here describing removal by chemical methods and by electrolysis, I haven't tried those myself but no doubt the folks who have will contribute here.
If you're forced to remove rust by hand, assuming that the rust attack is spotty or patchy, I think it's important to confine your work to the actual area of the rust and leave the rust-free parts alone. My own preference is to use a scraper followed by a glass fibre brush. Several suppliers in the UK here stock a glass fibre brush whose bristles can be retracted into the handle, rather like the lead extension/retraction mechanism of a propelling pencil. It can be quite labour intensive and time consuming but good results can be achieved.