The info you've been given in the previous answers is good.
Water that has a decent buffering capacity (generally lots of carbonates) will be more resistant to pH fluctuations than water that does not. (If your water is pH 7 straight out of the tap then it likely has little to no buffering capacity which could definitely account for your dramatic pH swings.)
High levels of carbon dioxide in the water will tend to make it more acidic, so forcing the CO2 out should help to bring the pH back up again.
Partial water changes are also important since your buffer/carbonates can get used up over time, which can result in the water suddenly becoming more acidic. These partial changes are also necessary to remove nitrates from the water which will build up over time (as a result of fish waste) and can be harmful to your fish if they get to very high levels.
There are a few more things that might be worth considering when trying to account for fish loses and pH swings.
If the filter media and/or substrate dried out when the tank was being moved then it's likely that most of the beneficial bacteria that live there and convert fish waste (ammonia) into less toxic forms (nitrites and, eventually, nitrates) will have died off. This could result in a fatal ammonia spike if fish waste builds up faster than the bacteria can re-establish themselves.
You said that you have been using some sort of pH "balancer". From what I've seen, these often cause more problems than they solve. Check to see exactly what it is and how it works. Products meant to adjust pH can often result in less stable water conditions. And if you're looking for something to stabilize pH rather than adjust it, plain old baking soda (as others have suggested) is probably going to do just as good a job -- if not better -- and be much, much cheaper.
As far as using decor items to adjust your pH goes... this can work, just make sure you add things one at a time and carefully monitor your pH after adding each item so that you're aware of any rapid changes. As John Cavan said, rapid shifts in water chemistry should be avoided if at all possible.
Adding things like crushed coral or limestone will add buffering capacity and tend to raise pH.
Adding driftwood will actually lower the pH so I would advise against adding any wood to your tank if you're having trouble with it going low already.
In general, for a tank that is not overstocked, I would find it very unusual to have to do more than a 25% water change once per week. If you're having to change more water than this in order to keep the chemistry stable, then it's worth examining your stock levels, decor/hardscaping, water source, buffering capacity, water additives, etc. to see if you can identify the source of the instability. Hopefully the info you've been given in the answers here will be enough to pin it down and address it.