Generally speaking, dogs don't normally dwell on any one thing for very long. After they use the bathroom, they move onto something else. That is why, a lot of times, when people are referencing Dog training they say they have to be "caught in the act" otherwise it isn't effective. Once they have finished, they are finished and their mind goes elsewhere.
If they do happen to look at you cleaning it, it's hard to say whether they would really have any mental association with what your actually doing. Dogs are very smart, but there are a lot of conflicting opinions, and thoughts on dogs, their thought patterns, and their level of intelligence.
A few things can be found online, but again the majority of these are opinions with no real scientific backings.
My guess, is that if the dog see's you, they are intelligent, so he associated the fact that your cleaning up after him. It also depends on his level of understanding. If he goes in the house, and is already trained enough to realize it's wrong they probably feel a hint of guilt that they did something wrong. Especially if you are scolding them for it at the same time. If he does it outside, and he knows that is where to go, then he probably thinks nothing of it when it's picked up.
He already knows your his owner and takes care of him, so perhaps he relates that back in some way to you "taking care of him", just in a different way. I doubt they really consider the "why" of you doing it.