I know this is SOT; but have you considered using a different tunnel tech or are you completely married to IPSEC?

I have similar experiences in the past - i've found that for a large chunk of use cases switching to OpenVPN on an commodity box sitting at the edge is far more simple and reliable (and has a number of performance gains).