What you need to look at is how many breaths your cat takes when resting or sleeping. It should be at about 20-30 a minute, but kittens and young cats will often breath a bit faster than this.
During the day when the cat is active, the breathing will be faster, often by a lot; what you need to look at is how fast the breathing slows down when your cat lays down to rest.
If the breathing doesn't slow down in 2 to 3 minutes when the cat is at rest, you might want your vet to take a look at him.
Rapid breath might be a sign of infection or heart anomalies, but your vet will look for this during the regular checkup of your cat.
If your cat takes more than 40 breaths a minute over a long period of time when it is resting, you need to take him to the vet; heart anomalies are not uncommon in cats.