Touch not mine annointed

We've all probably heard it said many times, 'touch not mine annointed', and have been taught this translates into 'watch what you say about the man of God'. I am just curious as to any thoughts those on here have about this. Does this apply even if we don't think the man in question is truly called (i.e. it's perceived they are promoting themselves or being promoted by their pastor, etc)? Can you not like the man as a person, and voice your opinions about them as a person, without having crossed the line into 'touching the annointed'? I only ask because throughout my years in the church, I have heard people say derogatory things about a man of God, but they will quickly say "I"m not saying he's not a good preacher, but. . ." As if this justifies the comments they are making. Can you really separate the man from his ministry? I've often wondered where you draw the line. . .

Maybe no one else thinks about stuff like this, but I've often been accused of coming up with some wierd stuff to think about.