I recommend you separate your buck and doe immediately.
A quick glance at Wikipedia teaches us that rabbits are induced ovulators That don't have regular cycles like many mammals do.
Rabbits are fertile with in hours after giving birth.
If males are not removed at or prior to birth of the young, postpartum breeding is possible. Does that become pregnant on a postpartum breeding will not be able to nurse their previous litter until normal weaning age, and this can lead to decreased survival rates in both litters of Kids Management of Pregnant and Neonatal Dogs, Cats, and Exotic Pets; Cheryl Lopate; John Wiley & Sons, May 22, 2012
If you leave the babies in with the parents, you could in theory have 1300 rabbits in one year, of course your initial mother will be be physically wasted from spending 12 months pregnant and nursing at the same time, if she survives 12 months.
Agressive breeders will breed as often as every 60 days, but a healthier recommendation is not more more than every 90 days. Once mature your rabbits can breed with only a few seconds of contact. The buck should only be placed with the doe when sufficient time has passed for the mother to recover, and you are attempting to breed her.
The other point to consider is the rabbit bond. Rabbits form very strong bonds, that are difficult to create and to break. When rabbits break their existing bond due to death or forced separation care must be take to keep the "rabbit from sinking into deep depression which can threaten his or her life as well, especially if they were very devoted".
Additionally once the rabbits are separated, putting them together for short times are problematic. It can take days to months to get rabbits to live together happily without fighting.
If you have left young rabbits together until they are old enough to result in pregnancy that you have probably allowed the mother to get pregnant before she is physically mature enough for a healthy pregnancy. If you have placed intact (not spayed or neutered) rabbits together who are mature enough have a healthy pregnancy and allowed them to bond you have set them up for forced depression when the bond is broken.
Double Pregnancy (Superfetation) has been reported in rabbits but is not well documented in domestic rabbits. Additionally partial re-absorption and delayed delivery have been reported on some blogs. I did not find strong scientific studies on Superfetation for domestic rabbits, the possibility can not be ruled out, providing yet another reason for buck and doe separation.