You have different options here.
install some stairs
This you need to agree about with the other persons affected and the landlord. There are different option, as a ladder like thing to a window, a bridge from a tree nearby, or some steps at the rain-pipe. Some example pictures are here,here, here and here. The aim would be, to let an open window for the cat all times.
work with a schedule
Also you can train a schedule with your cat. I assume you has noticed some pattern already how your cat spends a day. You can train times with it, when you will open the door for it. Or have some special call/noise to let it know "door is open now for 10 minutes" (like a spoon at a plate or similar, our neighbour did it with his 3 cats at feeding time and to close the door for the night)
add a cat house
This can give your cat a shelter until you come home or until you open the door. It is a insulated weather protected house near the door. You could even pretent it is a decorative flower stand for the unknown. And you can add a cat flap door with chip recognition, if you are worried other animals or cats would take it over. An example you can see here with lots of pictures how to make it and an instruction in german language.