The other answers explain properly why the bowl is not suitable for the goldfish.
If you want to go on with this setup, then there is a small hack you can try. It is kind of expensive, and I do not know if it will actually solve the problems (I never had goldfish).
Use an external filter, and "connect" it to the bowl with the 2 mandatory tubes. This solution has a few advantages:
- mechanical filtration;
- additional (optional) filtering, depending on the model (from active carbon to UV light);
- creating colonies of beneficial micro-organisms inside the filtering medium in the filter;
- no space is used inside the bowl by the filter (except the 2 tubes).
It does seem a bit dirty after a day
That is a sure sign that mechanical filtration is needed. You can achieve that with a filter, or with water changes, as you already know.