If there are two rabbits living in the backyard (whole garden at day, predator-safe enclosure at night), should I worry about free-roaming cats?
Even though @Elmy has a point I am going to assume that you are worried stray cats might hurt your rabbits.
This is definitely something you should mind. Cats can prey on pet rabbits.
I have seen cats hunt rabbits down and kill them and my own cats have done the same.
As you can probably not keep all animals from entering your garden you should protect your rabbits.
Maybe also read this Guide on how to care for outdoor rabbits, it in addition points out that rabbits can die of simply having to be too close to a predator.
(This experience report applies for Western Europe)
I have had two dwarf rabbits (one male, neutered, 4 years old and one female, 2 years old) for two years. They spend their daytime in the backyard and their nighttime in a predator-safe enclosure (no entry for predator, more than one opportunity to hide for the rabbits). We have at least one cat in the neighborhood, which is now almost 3 years old. In the backyard, there are a lot of opportunities (which we had built) to hide for my rabbits, where the cat can't come in.
At first the rabbits were only supervised in the backyard. We watch over all activity through the windows, so we do not influence the behavior of the rabbits and other (wild) animals. We discovered that the rabbits have enough instinct and reflex to overlook and warn each other in case of the cat enters the garden. Since there are hiding places every meter, they are safe before the cats' instinct is triggered. Only one time we have seen the cat have physical contact with the rabbits: our female was sitting with her back in the hiding tunnel (more than one entrance) and boxed the curious cat with her front paws in the face. The cat ran away!
After this situation, and considering our acquired knowledge that they could and will hide, we left them in the backyard unsupervised for almost 5 hours per day. The rest of the daytime they stay in the backyard too, but with us around. At night they live in almost 6 square meters enclosure.
We made this decision because of the following thinking and reasons:
Rabbits love their freedom. They need to explore, run and rove around to visit their favorite places. So they can bear for example temperature differences until daytime. (Search for sun in the morning, lie in the shadows while it's the middle of the day).
Rabbits can speed up to 60 km/h (little more than 37 mph) so they need all space they could acquire.
Humans take risks as "okay" in exchange for improved life quality. For example, they like to drive cars for comfort despite the fact that more people die driving than walking.
Our rabbits wait impatiently every morning until we open the enclosure door to the backyard.
We live in an environment with a lot of residental buildings, so the risk of predators at daytime is very little.
We feed our rabbits "ad libidum" which means, in short, they always have enough "no risk" food to eat and can choose time, amount and additions (like garden plants) of their food themselves. Because of this, we know, they will not eat toxic plants because of hunger. But they explore and test our garden plants and eat the ones they need.
For more information you can visit this site: kaninchenwiese/aussenhaltung (please excuse me, this site is in German, but I have not found an equivalent site in English yet).
(Once again: This experience is located in Western Europe. Other places can require other behavior).