I agree with Yvette, mostly. She's right in saying you should never tell a dog off if he doesn't know any better, it will only make it worse.
However, I think it's important for a routine to be established. Dogs love routines and set days. They thrive on it, it makes them feel secure.
I used to have a rescue dog who was ill treated in the same manner and the routine sorted that out:
Walk in the morning before you feed your dog. This is the most important walk. Historically, the dogs ancestors got out of the den and roamed for food so they could rest all day if successful. Try to replicate that. It keeps their brain active and mimics their "normal" way of life.
Pee in the garden when you get home.
The second most important is the evening walk and it is exactly the same.
Before bed, another pee in the garden.
As per training, I tend to think it's very important for a dog's rehabilitation, especially with a German shepherd, they are highly intelligent and need to be challenged.
But as Yvette said, keep it to a minimal for now. Training starts a real relationship with a dog, this is where communication and trust come into place. I suggest food for training or a clicker would do well here.
It's important to know that dogs want that from humans, that's how they see their purpose.
An idea for training your dog,
Proactive passive training:
As I said, German shepherd are highly clever and tapping into his brain is possibly. An example:
Say you want to teach your dog to sit:
Find your dog's favourite treats. Keep it in your hand and make your dog smell it real well, to a point where he so badly want it, he'll do anything for it. The dog is likely to go through all the behaviour he knows: maybe bark or scratch or whine...including possible bad behaviours. Stay calm and keep making eye contact. Eventually, your dog will sit. As soon as his bum touches the ground, you must give a treat and praise. But no word command!!
Start in the house, in one room where there are few distractions. Repeat a lot...a lot!!! Once your dog gets it, you can start adding a hand command, still no word command. Increase the distance between you and the dog gradually, one step at the time. If you've gone too far, come back a step. Eventually try with
Just a hand command with distance, eventually add the"sit".
This takes patience, calmness and leadership. Untrained dogs are a bit like children sometimes, they'll throw tantrums sometimes. It's important to be patient here but the reward for both of you If you succeed is indescribable.
Good luck!
