I've seen the same thing with 45-70 and .300 win mag. Just swapping from 180 to 200 grain bullets.Think of recoil, the longer it takes the bullet to exit the bore, the more time the bore has to rise. So generaly speaking a faster load will shot lower, on a rifle with sights set for slower loads. Somebody is bound to get real technical, cuz there are exceptions, or situations that can be set up, but this is likely what your experiencing.
Its not really dropping more. The effect your seeing is the differences of the barrel angels when the shot exits the bore.
Even if the recoil of the faster light bullet load seems less, the same or greater, remeber the light bullet at faster speed exited the barrel much sooner than the slower heavy bullet load, and recoil effect (continued muzzle rise) continued long after the light bullet exited the barrel relative to the heavy bullet.
Hope I made sense.
If it were me czfan I'd stick with the 405 grn bullets.