You know, G, I thought the same thing about concrete and pouring metal, but I've seen mixed reports and opinions about that lately, and from people I respect who have poured for a long time.
I think that for me, if and when I can build a melting shed, I will go with concrete floors, but just spread some sand on top of it around where spills are most likely -- by the furnace and the pouring area for the flasks. It can be swept up when not needed. Seems like you'd have the advantages of both.
Clay might be as much of a problem if it gets wet -- as it will if you get both rain and wind. So I'd still use sand over it in a pouring area. But lucky you for the clay -- maybe it is refractory enough to be useful in casting operations!