I got my rabbit 4 years ago over the summer, and when we just got her, I was always the one cleaning her litterbox. After school started however, my mom would sometimes clean it. We still now take turns cleaning it, and I noticed that whenever I clean it, she never kicks out any of the litter (recycled paper pellets), but when my mom does it, she kicks out a lot of it. This can't be just by chance because its been happening for four years. I am guessing she is able to smell who cleaned the litterbox, but why does she kick out the litter whenever my mom cleans it?
The short answer is:
Most likely your rabbit is seeing your mom as a potential revival for dominance of the space that your rabbit considers her's. Presumably your rabbit has trained you to groom her on demand, making you a non-rival, but if your mom is not spending as much time with the bunny, she may be considered a rival and your rabbit is attempting to make it clear that THIS LITTER BOX IS MY TERRITORY!
Rabbits digging out their litter box is not an unusual occurrence, we see it most with female rabbits. We often see it when the rabbit has decided that the litter needs to be cleaned before the person does. It is also seems more prevalent in the spring.
Domestic rabbits come from a heritage of living in underground warrens, most of the competition is between females to have a space of their own to have babies. They are naturally clean animals that use one area as a litter box and clean it out as needed. They have strong territory ownership and dominance traits. The dominate bunny is groomed on demand and visitors are allowed in Her territory only by permission.
- In some case (but not as described in this question) the rabbit just decides to clean the litter box before you do. Ensuring that litter box is changed regularly with rabbit appropriate litter can help.
- Spaying your rabbit will have some impact on lowering hormones, but will not completely eliminate digging behavior.
- The more time your mom spends petting and grooming the bunny the less the she will be seen as a rival. Focus on nose, ears, face and cheeks; stay away from hips and rear legs for maximum "you are the boss bunny" impact.
- When emptying the litter box, grab a few poops and put on top of the new litter. Rabbits use poops for territory marking If your rabbit finds her own poops in the litter box after you mom cleans it, she will be less likely to feel like there is a territory dispute.
- In the most difficult cases even with the above if the behavior continues, a high sided digging box can be used as litter box as seen in the last pictures in this answer. Your rabbit can move litter from one side to other to her hearts content, and it all stays in the box.
Our dominate bunny Harmony (spayed female) is the uncontested boss of her people, the two boy bunnies (neutered) she is bonded to, and her space. Currently she is using a high side digging box as her only litter box. Sometimes she just has to dig and move litter about.
In the situation you describe it would seem to be a struggle for dominance between your mom and your bunny, but that is not the only cause of litter digging behavior.