Well, you're not doing it wrong, it's more a matter of preference on how to deal with it. Yes, you are correct that if you cancel an order as a seller, it does affect your metrics in a negative way.....but if the buyer wanted it cancelled, as long as they request the cancellation in the first few hours, you can accept their request and get your fees back automatically. This does not affect your metrics.
If they want to cancel it after that time window, they cannot send a cancellation request as that option has been removed. You can contact eBay, they will review the communication and will cancel the order for you ---simply make sure that the cancellation request is from the buyer, not you. You would get your fees back, nothing affects your metrics and you can just move on (perhaps after blocking that buyer).
hope this helps...