I think it can be confusing because pansexuals sometimes give their own meaning to it when they explain what it is (not that there's anything wrong with that). It can be confusing for people unfamiliar with these terms to understand the difference between the general meaning and the personal experience of the individual they're talking with. Though of course, most confusion still comes from people who just don't know what it is or make up their mind about it before listening to explanations.
Btw I've been called pansexual by a guy, even though I'm not pansexual, because his sister identifies as pansexual and according to him my sexual orientation is the same as hers. I've always identified as bisexual though and I personally don't see a reason to change that.