It's really simple: QOS ("Quality Of Service") is about discriminating between different types of traffic based on its characteristics and needs (e.g. low-latency-required stuff like VoIP vs. latency-not-important "bulk data" transfers like BitTorrent). That kind of discrimination is just fine. In contrast, Net Neutrality seeks only to prohibit discrimination based on the origin or destination of the packets; i.e., who sent or requested them. That kind of discrimination is very much not "just fine."
For example, Comcast wanting to prioritize Comcast's video-streaming service above Bittorrent is fine; that's QOS. Comcast wanting to prioritize Comcast's video-streaming service above Netflix is wrong; that violates net neutrality.
In my experience, the only people who disagree with this after having it explained to them are those who are paid to believe otherwise.