WATCH: More personal finance videos on TheStreet TV | More videos from Brittany Umar

NEW YORK (MainStreet) — Vacation homes sound like a luxury for only the wealthiest among us. And it's true—most vacation homebuyers earn significantly more than the national median income, and almost half make an all-cash purchase, based on a recent study from the National Association of Realtors. Interestingly, 80% of vacation homeowners bought the property solely for personal use. But with sites like Airbnb and VRBO on the rise, the idea of owning a second home and renting it out for profit might seem appealing, even if you aren't part of the one percent. But before you start shopping for the perfect beach house, here's what you need to consider.

Mortgage Restrictions

Securing financing for a second home is not as easy as it is for a primary residence. Lenders view these loans as more risky, which means a mortgage for a vacation home is likely to have a higher interest rate and require a down payment of at least 30%. So make sure you can afford these increased out-of-pocket costs.

Tax Implications

The tax implications of a vacation home can be complicated, so it's worth enlisting an accountant to make sure you're reporting income and expenses correctly. You can't just list your property on a vacation site and wait for the checks to roll in—the IRS expects to collect on what you earn.

Maintenance and Repairs

Once you become a landlord, you can expect a certain degree of wear and tear on your vacation rental, as well as the responsibility that comes with it. You may find you spend more time on maintenance chores than enjoying your own two weeks of vacation. You'll also need to make sure the distance is manageable, so you can easily get to your property for repairs if need be.

Cash Flow

Perhaps the most important consideration is the cash flow you can count on from your investment. The average vacation home is only rented for 17 weeks a year, according to HomeAway.com. Spend time researching comparable properties and try to calculate how much annual income you can expect, and how it'll stack up against your carrying costs.

If all these considerations add up for you, then buying a vacation home may very well be in your future. Just make sure it's a well-informed and carefully thought out decision.

—Written by Lauren Lyons Cole for MainStreet