Do you have a pin in your router table? That may help you keep the piece stable as use the pattern bit. Instead of just resting against the bearing, having 2 points can make it easier.
--Nathan, TX. Hire the lazy man. He may not do as much work but that's because he will find a better way.