No harm leaving the old bilge pump base in place.
On the starter, were you trying to start the motor and it didn't start? Two things are used to drop the gear. A spring and once the motor starts and speed exceeds the starter speed, reverse force on the gear starts the drop and the spring completes the drop. A sticky, dirty or dry drive assembly can stop the drop and in some cases the rise of the gear. Try putting some light oil on the worm gear on the starter shaft.