If your wife enjoys romance novels and happens to occasionally fantasize about her being a captive princess rescued by some hot young Italian Romeo that looks like an actor on television... it might be lustful. But it's not actual adultery.
If your wife is in the mood to be a "bad girl" and has a fantasy about the pool boy that cleans the pool next door... it might be lustful. It might even an adulterous thought in the heart. But it's not actual adultery.
Sure... for such a woman... some confession, repentance, Spirit filled effort to guide her thoughts better, and hubby going on a diet and being more manly and clean the pool might be in order. But she has not actually committed adultery.
Even the Law distinguishes the two:
Exodus 20:14
14 “You shall not commit adultery."
Exodus 20:17
17 “You shall not covet your neighbor's house; you shall not covet your neighbor's wife, or his male servant, or his female servant, or his ox, or his donkey, or anything that is your neighbor's.”
I'd never accuse my wife of committing adultery if I found a raunchy Cosmo Magazine in her things or discovered she had a trashy fantasy. Now, I might want to talk about it. I might want to encourage her to seek more of Jesus. But I'd not accuse her of adultery and I definitely wouldn't think about divorce. In my mind, this is one of those human issues that might rear its head several times throughout the course of one's lifetime within marriage. It has to addressed... bounderies should be drawn... prayer, confession, and repentance sought. But not a divorce. That's some serious stuff. If I ever remarry, I'd rather live with a wife who gets freaky every now and then... and seek some counseling... than shatter my family and reduce both of us to part time parents and doom our kids to experience the trauma from coming from a broken home.
Just being honest.