I'll preface this by saying that this answer is from a bit of research.
According to the website PureGoldFish.com: Aggression, the bumping or nudging in a limited environment, particularly when at feeding time, is the larger fish asserting dominance, and according to the website:
some fish keepers may experience bullying if they have a goldfish that is much larger than the others. These may try to assert their dominance and play “king of the tank.”
and they advise that if you have small containers, to keep fish of the relative same size together to avoid this behaviour. However, having said that, according to the RSPCA's webpage Should Goldfish be kept with other fish?, goldfish are social animals and in normal circumstances are not aggressive, and pertinent to your question,
Solitary fish can exhibit depression and lethargy.
So, putting it all together, they should be able to coexist, but in a larger enclosure.