The speculation has become pretty humorous. The only reason I can see either of them getting upset over the Super Ayu/Manuel books is because there were provocative photos that: A) Mannie felt pressured into taking, thereby questioning his status in the relationship; B) upset Ayu when Mannie auctioned off a signed copy of his book; or C) upset Mannie when Ayu showed others her book.
Nevertheless, OMG are those pictures of Mannie disturbing.

Those would never turn me on, just freak me out.
I wouldn't be surprised if Ayu didn't give him any money after the divorce, but he's got his own fortune and it's entirely possible that that was part of the prenup. The stuff about him not wanting to go to Japan after the tsunami because of fear of radiation would be really upsetting to anyone, but I'm sure that's absolute crap.
I do wonder if Japan is very spiteful towards him now. Or if they're more spiteful towards Ayu. A celebrity like Ayu marrying a Caucasian man...then splitting up...there's bound to be some jealousy, hatred, conceit, and maybe even gloating floating around amoung the female population of Japan.
Regardless, the talk that we should drop the subject seems silly to me. Few people have said anything malicious, and Ayu wouldn't have told us the situation if she didn't expect us to talk. And to be honest, as international fans, us talking on a forum about it is significantly less damanging than gossiping about it in the streets or in school.