Is it okay to give my reptile a bowl of tap water to drink from and bathe/swim in? Or is there something wrong with tap water that I should avoid?
The subject of tap water and it's safety for reptiles is still pretty much all based on opinions. I don't think that there have been any effective studies done on the subject that show either way.
I think it really depends on your town's water treatment plant, and how much it cleans the water. If you're happy with how your water tastes from the tap, I'd consider that okay to give your reptile, otherwise I'd consider treating/filtering the water.
The concern is that so many chemicals are added to the water. Fluoride, chlorine, chloramines, along with the metals that leech into it from the holding tanks and pipes. We've grown used to it, and studies and tests are done constantly to make sure that it's safe for our consumption, but the safety of animals isn't a part of the consideration during those tests.
My biggest concerns are not what's put into the water, so much as what isn't taken out of it. There are legal limits as to what can be found in tap water, based on it's harm to humans. While these contaminants aren't going to have any effects on humans at these limit, reptiles having a much smaller body mass, could be affected. See the EPA's list of legal limits on water contaminants here.
The reason it might not be safe for reptiles, where it is for dogs and cats is that dogs and cats have been exposed to the water practically as long as humans have, it is their water source just as much as it is ours. Well, except for cats so much as they also get water from their food.
Reptiles on the other hand, have a different physiological makeup than mammals. While they don't "drink" water through their skin like amphibians do, a lot of reptiles will absorb some water through their skin to stay hydrated.
Being relatively new and uncommon in the pet industry, reptiles have not been exposed to tap water for as long as us and the more common animals. Even more, if you have a more uncommon reptile and you didn't buy it from a breeder then it most likely came from the wild and hasn't been exposed to tap water at all.
I use water conditioner for the water I give my reptiles because I don't like the taste of the tap water myself, and treating the water isn't that much of a hassle in my opinion. I use reptisafe which I can can find for about $5, and it uses 2 drops per cup of water. You can also use the same water conditioner you would use for a fish tank, as long as it doesn't have any additives (like for their slime coating).
The other option I know people use, are brita water filters. They work by running the water through a carbon filter, which takes out most of the chemicals and harsh metals that can be found in the water. They might not clean it out entirely, as they're only concerned with making the water taste better really, but that should be enough to make the water safe for anyone.
The only thing you would avoid using is distilled water, because reptiles rely on some of the minerals found in water, and distilled water removes those minerals.