Depending on your location $350k total will not get you a very large 'custom' house. $350k is good for a modular home but then you didn't mention the land. Then there's the price for contractors and never in my lifetime have I ever seen a contractor finish on time and within budget. This isn't to say they are shady but things happen. You just learn to have extra money for when this does happen (and it does happen).
As stated by others there's still the cost of land, laying foundation, piping, electrical, hvac, and codes codes permits and zoning and codes. Custom houses require someone to double-triple check housing codes. This costs money and a lot of time. You could hire a contractor to do this but its easy money for them.
"oh that room doesn't fit the zoning or code laws, it'll have to go down so you have to pay for us to remove it, re-supply and re-pay for us to put it up right."
Not to persuade you from looking into it but first find your ideal location and get the zoning, housing rules. See if your layout is acceptable (room sizes, ceiling clearances, stairs, exits, doors, windows, hvac, piping, electrical) and then ask the city about putting up your house there. Not every city is keen to just add a new road, piping or electrical infrastructure s to areas that have little return to them. I'm sure money fixes this though (every county is different).
Good luck!