My husband has more authority than my pastor, except in two places (which really have nothing to do with my pastor):
1. My personal convictions, which are inherent to my relationship with God and between Him and me.
2. If he tells me to do something that goes against God's Word.
Now, I do realize there are some crazy people out there, but here's where it gets fuzzy:
1. Some men ask their wives to go against their personal convictions, and the woman (perhaps lacking backbone) will blame the church or the pastor, leading to unnecessary resentment between the pastor and husband.
Or
2. Women feel that it is a sin to disobey their pastor but don't feel it's a sin to disobey a non-believing husband. (For whatever reason, usually due to poor teaching). So in that case, they're choosing "God" over their husband; not necessarily "pastor" over husband, even though it comes across that way.
Some churches simply put too much emphasis on pastoral authority and not enough on a man's authority in his own home. Additionally, when the husband is an unbeliever, his authority is compromised anyway, since the whole idea of submission is wrapped up in a biblical concept in which the husband has his own part to play. If his half is missing, the submission part is VERY difficult. (Loving his wife as Christ loved the church) It isn't impossible, and Christian women should submit to their husbands whether they are Christians or not, but it's short sighted to pretend that it isn't difficult.
All that said: NO ONE tells my husband and me when we can use our bed, save God Himself (and He hasn't said anything).

I will never comprehend why pastors involve themselves in stuff like that. Stick to the Word. There's plenty of material there.