“I was like, ‘This album is not going to appeal to a lot of people,'” Charli says