"How early should I get there?"
Give yourself plenty of time to set up your equipment (see below), use the bathroom several times, acclimate yourself with the race site, and (if you choose to do so) do a warm-up. I like to get to races 90 minutes early, which gives me about 20-30 minutes of warm-up, 20-30 minutes of setup, and 20-30 minutes of port-a-john excitement.
"Where do I keep all my stuff, like my bike and shoes?"
In a triathlon, all of your equipment is kept in a "transition area." You get a space on a "transition rack" which is usually a long metal beam that runs parallel to the ground--you either hook your handlebars over it or you hook your seat over it. The area near the wheel that touches the ground is where you put all the gear you'll need easy access to during the race--your helmet, shoes, etc. As a courtesy to other racers, push all gear you won't need during the race to the edge of the transition area so more participants have space near the racks (as space is usually pretty limited).
"What about security? Will people steal my stuff?"
You don't lock your bike in transition, but at many races, to remove your bike from transition after the race you need to show that your bike (which is affixed with your race number) matches the race number assigned to you. There have been only a few incidents of theft at triathlons, but there have been enough that people are aware of the issue, and any decent race will at least have people watching the transition area for shady goings-on.
"How does the competition itself work?"
Unless you're an "elite" or a "professional" you'll probably race primarily against people of the same gender and age as you. These "age groups" are generally binned every five years, though the lower and upper groups are sometimes a catchall (e.g. 18 and under, or 75 and older).
"How does the swim work?"
Usually, you'll start in a "wave" of between 30-150 athletes, though some races do a "mass start" or a "time-trial" (one-at-a-time) start. You can usually choose to wear a wetsuit (unless the water is too warm), and if the rules allow it I highly recommend wearing it. Many people find that their swim times are improved by about 10% when they wear a wetsuit that fits them properly.
After the swim, you'll remove your cap, goggles and top half of the wetsuit (on the way to the transition area) and enter the transition area. Some people like to remove the entire wetsuit in or near the swim exit, that's cool too. After removing your wetsuit, if you're wearing one, PUT ON YOUR HELMET. Grab all the other stuff you'll need on the bike (shoes, sunglasses, bottles, etc), put on what you need to, and get outa there!
"What rules are there on the bike?"
The most important things to remember are the rules concerning DRAFTING and BLOCKING. Don't do either of these, unless you happen to be racing a draft-legal race (which you probably aren't).
DRAFTING: Stay 3 bike lengths behind the person in front of you, except to pass. If you want to pass, you have 15 seconds to pass once you move inside of 3 bike lengths. If you get passed, you have 15 seconds to drop back outside of 3 bike lengths. Violation of this rule will get you a DRAFTING PENALTY.
BLOCKING: Stay to the right unless passing, just like the interstate. Don't cross the yellow line and don't pull out in front of other riders. Don't get a BLOCKING PENALTY!