I seem to have found that the secret, for me at least, is not to 'swing' as with a hammer driving a nail When driving a nail the idea is to swing through the head.
With the lil' blue hammer I hold it loosely towards the end of the handle and kind of flip it (hard to explain) so it hits the floor at the 'end' of its travel.
Maybe like cracking a whip or if boxing/bar fight, ya want to make the punch end just as you hit the guy. All the energy is expended at that point.
Also then handle doesn't bend so much.