My understanding on this matter is, in leymans terms.
Its to do with the surface area on the back of the bullet, the more surface area there is for the energy(the bang) to push on, ie the larger diameter bullet receives more energy than the smaller one, so goes faster.
Im no rocket scientist, but thats how I understand it works.
Im more than likely wrong, and apoligise for confusing you if I am.
Until someone confirms Im right, or on the right track,dont take it as gospel truth, I failed Science,Phyics school in general