I didn't sand too much at all.  The factory crank still fits and works as it should