It sounds like your dog views your one cat as a toy. If he stares long enough at her, she'll run away, which triggers his chasing instinct and is fun for him.
He learned that the first cat will hurt him if he tries to chase her, so she isn't a very fun toy. The second one doesn't hurt him, so she's more fun to play with. The staring is an intense fixation and waiting for the fun to begin, like a dog staring at his owner, waiting for the ball to be thrown. This tells you a lot about his personality.
- Border Collies are a working breed and absolutely require a lot of training, play and workout time, otherwise they develop unwanted behaviors out of boredom.
- Your dog is very toy fixated. His favorite games would probably be something like Frisbee, catching prey dummies or balls and recognizing and aborting toys by their names. He's probably less interested in agility courses or scent and tracking games, but that doesn't mean that these are boring or no fun for him.
- If you don't engage your dog in physically and mentally fulfilling games and trainings, he's at a high risk of developing Obsessive Compulsive Disorder (OCD). Here's a related question that demonstrates how this kind of behavior can gradually become worse and worse, but there are countless other examples where owners don't recognize the signs before it's too late.
At the current stage it doesn't sound like OCD yet and you should be able to tell him that his behavior is not acceptable. If you wait much longer, this behavior gets more and more ingrained and it will become harder to stop it.
Jumping at the door
There are different solutions for this problem, but most of them require you to be outside with your dog.
You should never open the glass door while or right after your dog jumped at it. That way you reward his jumping by letting him reach his goal.
One very simple solution might be hanging a curtain on the door, so your dog cannot see the cat anymore. The curtain must cover the whole area that your dog can reach and he mustn't peek over or under it. This removes just the symptom, though, not the cause.
If you're outside and see your dog jump at the door, you could walk up the door very purposefully, stand right in front of it, facing your dog and blocking his way. In dog language, you claim this spot for yourself. He obviously cannot continue jumping at the door if this spot is claimed. The disadvantage is that he might still jump up whenever you're not around.
Another solution is to connect the act of jumping at the door with negative experiences.
- Reprimand him every time he jumps at the door. Stop your reprimand when he stands with all 4 legs on the ground, then turn around and ignore him for several seconds (unless he continues jumping). If you open the door too soon after a reprimand, he might learn to ignore the reprimand because in the end he got what he wanted after jumping at the door. It just took him a few seconds longer.
- Have a bottle of tap water (about half a liter) with a sports cap ready at hand. As soon as your dog starts jumping at the door, you squirt the water at him by squeezing the bottle. Don't be afraid to empty the whole bottle in one go. The first negative impression is the most effective one.
- If your dog actually likes the water, fill an empty plastic bottle (again, half a liter) with a handful of old screws, nuts or washers. Shaking the bottle should produce a rumbling clatter. When your dog jumps at the door, throw the bottle at the ground near him. Never hit your dog with the bottle! Aim at the ground. The bottle should roll up towards your dog rumbling and clattering and scare him away. This might work from inside the house as well as outside.
- If you have a dog whistle in a pitch that's unpleasant for dogs, you can whistle whenever he jumps at the door.
Some people use remote controlled shock collars in such situations. I strongly advice against using these collars. Administering electric shocks to the throat is completely over the top and resembles torture rather than training.
Fixating your cat
What many people do, but doesn't work, is calling your dog to you. The dog stares at the cat, waiting for the fun to begin. You call him to you with the command "come here", but what you actually mean is "stop staring at the cat". If you dog comes to you, but there's no reward in it for him, he might stop listening to your commands altogether.
A better solution is engaging the dog in some game or training. It doesn't even matter what exactly you do, just offer some fun "together time". This should teach the dog "Staring at the cat is not fun, but not staring at the cat is fun". You can find some suggestions and inspiration here (and you are welcome to add your own suggestion).
You can reinforce this lesson even more by applying the "spot claiming" as described above. When your dog stares at the cat, walk up to him and stand between him and the cat. You can build up more pressure by gently shoving him away from the cat. As soon as he focuses on you instead of the cat, praise him and immediately start playing with him.
