New or Used?
First, do you want a new or a used machine? Many sewing machine shops will carry gently used machines that customers trade in. These machines have been recently serviced and may come with a 30-day warranty. In many cases, older machines that were well-built in the first place will be more reliable and more affordable in the long run than an entry-level machine from your local big-box store.
You can also watch Craigslist, yard sales, and thrift stores for used machines, though you should be aware that some may require a little professional help if they're not performing at their best. A tune-up will typically set you back about $50-$100. I have several used machines that I let students use in class -- a modern entry-level Janome, a Brother from the 1980s, and Brother, Singer and Elna machines from the 1950s. I picked up all but one of them at yard sales and thrift stores. And yes, all of them still sew like a dream.
As for new machines, I'm not a big fan of the ones that cost around $100 at Wal-Mart, Jo-Ann, or other big box stores. Most of the time, they're pretty clunky in how they operate, which makes for a less-than-enjoyable sewing experience. And a huge part of sewing is that we don't want to be stressed out while we're doing it, right? We sew because we want a fun, creative break from all of the other obligations in our lives. These machines are usually made with plastic parts inside, meaning that they generally will not hold up well over time. Of course if you're on a tight budget, they're definitely better than nothing.
But if you're able to spend a little more now, chances are that you'll end up with a machine that will last for years and years with minimal upkeep. For example, I'm still using the sewing machine I bought in 1994 for $450 and it still runs great. It's a Husqvarna Viking and I have absolutely loved it. Other people I know swear by Bernina machines. And Janome definitely seems to be an up and coming brand that has some good quality, affordable machines.
Features
It can be tempting to buy fancy machines with lots of stitches. However, I have to tell you that as much as I sew, I primarily use only two stitches -- the straight stitch and the zig-zag stitch. It's kind of like how car dealers always brag about how many cupholders a car has, but after you buy the car you realize that you only use one of them. If you think that you'll want to use your machine to hem pants or skirts, then a blind hemming stitch would also be helpful for you. And many machines offer special stretch stitches that work well with knit fabrics like lycra and cotton jersey.
Need more info?
I hope that this post has answered at least some of your questions about buying your first sewing machine. If you're looking for other opinions, plenty of other sites have covered this topic, including Sew, Mama, Sew, Prudent Baby, and CraftStylish, among others.
And if you're an experienced sewist, feel free to leave a comment if you feel I left anything out or if you have any other advice for budding sewers.
Happy sewing!
--------------------
Addendum: I didn't mention it in the original post, but visiting a local sewing machine dealer can be incredibly helpful. The sales staff can help guide you to a machine that will suit both your needs and your budget, and you can test-drive any machine to see how it feels for you personally. If you buy a machine from a dealer, the machine will come with a warranty and you also typically get a free lesson on how to use it. And your dealer will be able to help you down the road if you ever decide to buy additional accessories like a ruffler or rolled hem foot (both of which can be helpful for some projects).