S.M.A.R.T Failure Slowing USB Initialization

For a while now (About 4 years) S.M.A.R.T has been reporting that one of my drives is failing due to having 1,109 (Way above the 140 limit) reallocated sectors. That number hasn't changed in at least a year, so it's safe to say it's not in any immediate danger of completely failing. I also have no plans of replacing this drive any time soon. The problem is that when the BIOS initializes the USB controllers, the immediate next step is to run the S.M.A.R.T. check, which is taking about 15 seconds to complete. It's really annoying.

Is it possible to solve this problem without removing the drive?