Unfortunately that's how my budget goes too...
a 5 pound chicken at the farmer's market is about $25-30... at the grocery store a whole chicken is about $10... when my weekly food budget is $80, I can't afford to buy the meat from the market.
Goat's milk I do buy at the market because I can buy a pint for Aeryn ($2.00) vs. the grocery store that only sells it by the gallon for $18.
My neighbour sells eggs for $2.50/dozen, that's $1.50 less than the grocery store sells regular white eggs for so of course I'm going to buy her free-range eggs from the chickens that wander over to visit sometimes
I do find now that I'm more concious of what I'm eating, it is costing us more to eat weekly... but we're feeling a lot better about it. I don't mind paying for good food and I would love to get to the point where I can eat whatever I wanted without worrying about how much it costs. Really I want to live somewhere that I can produce my own food so our grocery bill is next to nothing from the store.