Depends on the media used. Ground corn cob is very gentle and so are walnut shells. It really only attacks soft stuff.

Sounds like you have a dual drum unit? 
I swap out grits on my single so often that it only takes a few seconds. Surfaces are simple, nooks and crannies on the sides are a nightmare.