Hubby and I didn't have a lot of money to buy a house either. We bought a trailer in a park first, it needed updates and we paid it off in three years and sold it for more than we bought it for. We used that money as a down payment on a house that we found someone willing to 'rent-to-own' to us. There were tons of conditions, and we had five years to pay the house off completely. Where I live we bought a 3-bed, 2-bath house and 6 acres of land for $25,000. There was no running water or electricity when we moved in so it's not like it was a nice house. AND it took us almost a year and asking almost 100 different home owners if they were willing to rent-to-own before we found this place.
That was 4 and a half years ago, and we paid the house off this month... it's OURS. But it's tiny and there's not enough land here to really do any farming. With the fix-ups we've done we can now sell this place for around $100,000 and that will buy about a 50-100 acre farm here (that again needs fixing). It took us a total of 10 years from when we started out on this 'getting a farm' idea... but we're 34 now and can buy our farm cash without having to worry about a mortgage. We're still planning on staying in the paid off house for a few years so we can save up some extra wiggle room (like a 6 month emergency fund) and have ALL of our debts paid off. I want to be in our dream farm before my kids start school (our daughter is 13 months old and #2 is on the way). I agree with Moolie, not adding to debt feels GREAT... and the more you pay off, the better you are going to feel. It really is a snowball once you start down the path of getting everything paid off.
I wouldn't think about getting your dream place on the first try, but maybe steps you can take to work your way toward that dream without getting in over your head. Can you rent a cheaper place, or buy a tiny place, that needs a little work so you could probably sell for a profit, to live for a few years while saving like crazy and paying off debts?
We have a program for buying farms here too, Farm Credit Canada... they require 25% down (which is pretty standard for a farm property here) but they have perks like deferring mortage payments for years so you can build up stock/income, rolling equipment and livestock purchases into the mortgage, etc.