I’ve carried the same YubiKey NEO on my keychain for five years. On average it gets used dozens of times per day, via USB, as an OpenPGP card. The YubiKey looks a little worse for wear, but it almost always works flawlessly.
Occasionally, it requires a few insertions to be read. When this happens I clean the contacts by rubbing them gently with a Pentel Clic Eraser, wiping off the dust, spraying them with isopropyl alcohol, and then wiping them dry. Afterwards, the YubiKey is registered immediately on the first insert. I perform this procedure about once or twice per year.
Using the eraser is potentially dangerous, but I’ve had good luck with it over the years. The white vinyl in the Pentel Clic feels very smooth compared to the abrasiveness of the rubber found on the tops of most pencils.