My cat attacks whoever cleans her kittens pee or pooh. Except if it was me cleaning it. Why? What does she think of this? If I cleaned it, she'd just check the clothe and it's over. It seems shes satisfied. If anyone else comes near the pee of her kittens it's war. She allows her kittens to be handled and played with though.
female cats are very protective of their kittens and the waste the kittens make do have the smell of them so this might trigger the mothers protective instinct.
the mother trust you more than the other people around for now,but this is not positive for the kittens they need human interaction to be good house cats.
you do not say if the mother cat allows the kittens to be lifted and played with by other people,it is important for the kittens to have human interaction from the time their eyes open for them to be used to people.