“People just sometimes don’t like some things,” she told Elle ahead of the release of her new album