When I was in Japan, I definitely got the sense that young people were not too keen on Ayu. It bummed me out, but then I went to her concert where everyone adored her, and it made me feel better.
People just love to have opinions about celebrities. It would probably be better for your blood pressure if you didn't worry about it too much.