ALL the weight should be on the keel rollers. Side bunks are for stability, not to support the weight. If the keel rollers don't touch the keel then loading and launching will be awful.
If you are tearing up rollers, they are either old rubber, sit in the sun too long, or the bushings are shot. Look at them when you are retreiving the boat to be sure they are turning freely. What does your keel look like? Is the metal all torn up from beaching?
Black rubber on a garage kept boat will last 5-10 years. The "yellow" ones ARE more durable and will last considerably longer.