I have 4 cats and 4 boxes. They're all good animals but one has anxiety and temperment. 3 cats will use any box, don't care , she will only use her box and once another cat uses it she will no longer use her box; she will do her business wherevever . I was really sad for a moment thinking that she would never use a box again from how she was acting for a long time but I fixed that by putting her in a room by herself with a new/cleaned box. This works well because she started using it again and No one else has access to the box. Problem is... I don't want to leave her in a room all day by herself
To be honest, your one cat, being anxious, sounds like it'd be happier as an only cat, but hopefully you can make it as happy as possible in the situation by continuing to give it privacy with its box.
So continue to keep your cat's box in that room, and the other cats out of that room. But give your one cat outings from the room. If you're concerned the cat might try to eliminate during one of its outings, try to only let the cat out right after it has used its box, and keep an eye on it. If you catch it about to eliminate, try to distract it by either making a loud noise (clapping is good) or throwing a toy in its direction, whatever gets its attention, or whatever you find that works, as long as you don't do something that is rewarding it. Then quickly put the cat back into its room until it has used the box. Obviously don't shut the cat out of its room for very long periods of time, so it has plenty of opportunity to relieve itself.