I have a small/medium mixed breed dog who never ever gets along with bigger dogs.

This seems to indicate that your dog has poor doggie manners. The most obvious and ultimate long term solution is to get some help/behavior training with your dog so that he can get along with others.

All big dogs don't just suddenly decide to bark viciously and try to eat all the little dogs. There's something up, here.

I usually find myself having to kneel down next to my dog, put one arm around him, and use the other arm to keep the loose dog away from him.

It sounds like you do this a lot and that isn't good, either. When there is a loose dog, what you should be doing is keeping your dog close and change course as much as possible, even if it means venturing off the trail or sidewalk. You should retain a calm, relaxed, and casual manner. Your writing here reads to me that this becomes a very anxious, very stressful event where perhaps you instantly stop in place to protect your dog until the big dog is gone - if it's anything like that, then your behavior is perpetuating the bad air. 95% of the time, merely walking away will solve the issue - people who have their dogs loose trust their dogs enough to control them off leash - so you control your dog by walking away, and let them control their dog. Again, when you kneel down to protect your dog, you heighten the situation and make everything a lot worse - it also doesn't help that this is the last thing that another dog owner expects you to do, so you're surprising the other dog owner, which takes some of their control of the situation out of the picture. (Just to be clear, remaining on the sidewalk/trail is an unspoken word "Me and my dog are ok and safe, no issues here.")

The amount of anxiety and emotion that's going on here is really out of control and I think both you and your dog need some intervention from a trainer because it's going to be that - much more than the bad doggie manners - that causes a dog fight. Having some positive experiences instead of so many negative ones will also help you immensely.

I don't think carrying some kind of spray would be a good idea.

You can carry gel pepper mace but you must spray BOTH dogs. Otherwise the sprayed attacker will become the victim by your dog because dogs are opportunistic. Spraying both dogs also helps it look like you're NOT randomly attacking any dog you come across with pepper mace.

There was a mention of Leash Laws. It's important to understand what your leash laws are, exactly. There's many people who infer that "leash law = illegal to have dog off leash" which is probably very rarely the case.

I have a small/medium mixed breed dog who never ever gets along with bigger dogs.

This seems to indicate that your dog has poor doggie manners. The most obvious and ultimate long term solution is to get some help/behavior training with your dog so that he can get along with others.

All big dogs don't just suddenly decide to bark viciously and try to eat all the little dogs. There's something up, here.

I usually find myself having to kneel down next to my dog, put one arm around him, and use the other arm to keep the loose dog away from him.

It sounds like you do this a lot and that isn't good, either. When there is a loose dog, what you should be doing is keeping your dog close and change course as much as possible, even if it means venturing off the trail or sidewalk. You should retain a calm, relaxed, and casual manner. Your writing here reads to me that this becomes a very anxious, very stressful event where perhaps you instantly stop in place to protect your dog until the big dog is gone - if it's anything like that, then your behavior is perpetuating the bad air. 95% of the time, merely walking away will solve the issue - people who have their dogs loose trust their dogs enough to control them off leash - so you control your dog by walking away, and let them control their dog. Again, when you kneel down to protect your dog, you heighten the situation and make everything a lot worse - it also doesn't help that this is the last thing that another dog owner expects you to do, so you're surprising the other dog owner, which takes some of their control of the situation out of the picture. (Just to be clear, remaining on the sidewalk/trail is an unspoken word "Me and my dog are ok and safe, no issues here.")

The amount of anxiety and emotion that's going on here is really out of control and I think both you and your dog need some intervention from a trainer because it's going to be that - much more than the bad doggie manners - that causes a dog fight. Having some positive experiences instead of so many negative ones will also help you immensely.

I don't think carrying some kind of spray would be a good idea.

You can carry gel pepper mace but you must spray BOTH dogs. Otherwise the sprayed attacker will become the victim by your dog because dogs are opportunistic. Spraying both dogs also helps it look like you're NOT randomly attacking any dog you come across with pepper mace.

You can also carry a walking stick. Use your best judgement; I'm not advocating beating every dog you come across with a heavy club. : )

There was a mention of Leash Laws. It's important to understand what your leash laws are, exactly. There's many people who infer that "leash law = illegal to have dog off leash" which is probably very rarely the case.