A variation of 4 to 5 grains on a 400-500gn bullet is a bit unusual, but nothing to be too concerned about. It's around a 1% weight variation.
The more important thing is to ensure the bullet fully forms to the shape of the mould. All edges to the driving bands and lube grooves should be as crisp and sharp as the mould. However, the biggest thing to look out for is that the base of the bullet is flat and even. Nothing kills accuracy more (in cast and jacketed bullets), then an uneven base.