Yup, you sure can mess up moulds using power tools. That's why I do it by hand. Same method of fitting a centered shank to a bullet from that mould, but I have a drill chuck on a handle I hand turn. A spare chuck would do. Fine valve grinding coumpound works fine for me because the bullet is not being turned as fast. I do have a set of Clover Compound in a range of grits for when I just want to polish or remove a very little metal. The advantage to doing it by hand is feel. I can tell when I'm getting out of line.
With the Lee sizers, the easy way is to lube up some cull bullets compound, and shuck them on through like you were sizing them normally. I've lapped a few Lymans in a lathe, but find that more difficult and a limit of about .002 lapping without having to make a new internal rod.