Sounds like a perfect job for a slitting saw - a really thin one ideally. Yes, you'd need to make two passes, and yes, you'd lose the thickness of the saw out of the offcut (hell, I have slitting saws thicker than 1/4"!!), but done carefully & if your mill is well trammed, you should get a decent piece of..... square-ish tiny brass?
Like you, I inwardly rail at the waste of metal cutting a big piece down to get a small part out of (which would have cheerfully fitted into the next size down); usually I end up grimacing and cutting it away TBH. Then again, i'm usually working in steel or aluminium, which are both cheaper than brass!