Though should only regenerate when your CA has updated their side if they use openssl anywhere in their pipeline And you also need to revoke your current SSL certificates so they can't be repurposed On 9 April 2014 13:05, Gerard Creamer wrote:

That's what we did - patch then regenerate. Better safe than sorry.

On 9/04/2014 11:47 a.m., Michael Sutton wrote:

NZNOG members:

My apologies but all attempts to send the text content of this PDF this morning have been blackholed until I managed to send the contents to InternetNZ PAG as a PDF which made it through filters which been stopping this content. I have had no problem send other messages etc.

Your comments would be appreciated as I see this as a major issue which may require all certificates to be regenerated and then only used on patched systems whose memory and priv keys cannot read copied by external parties.

Sincerely Michael S Sutton Director - Awacs Communications (NZ) Limited Transit Room The Dominion Observatory 34 Salamanca Road Kelburn Wellington +64 21 305500 Twitter & Skype: Mikiwis http://www.awacs.co.nz https://www.google.co.nz/#q=michael+sutton+nokia+patent http://www.linkedin.com/profile/view?id=16587996&trk=tab_pro

_______________________________________________ NZNOG mailing list NZNOG(a)list.waikato.ac.nz http://list.waikato.ac.nz/mailman/listinfo/nznog

-- Netspace Services Limited http://www.netspace.net.nz Phone +64 4 917 8098 Mobile +64 21 246 2266 Level 4, 191 Thorndon Quay, Thorndon PO Box 12-082, Thorndon, Wellington 6004, New Zealand

_______________________________________________ NZNOG mailing list NZNOG(a)list.waikato.ac.nz http://list.waikato.ac.nz/mailman/listinfo/nznog

A CA who isn't communicating what they have done to address this issue with their customers today needs to be in a different business as well. On Wednesday, April 9, 2014, Nathan Ward wrote:

On 9/04/2014, at 2:47 pm, David Robinson javascript:;> wrote:

...

Though should only regenerate when your CA has updated their side if they use openssl anywhere in their pipeline

I'm not sure that this is really true - The bug lets you read memory in a process that terminates an SSL connection. If your CA has private key material for certificate signing certificates in a process that's network addressable, then surely they should be in a different business, no?

Please correct me if I'm wrong, maybe I haven't thought wide enough.

...

And you also need to revoke your current SSL certificates so they can't be repurposed

On 9 April 2014 13:05, Gerard Creamer javascript:;> wrote:

...

That's what we did - patch then regenerate. Better safe than sorry.

On 9/04/2014 11:47 a.m., Michael Sutton wrote:

NZNOG members:

My apologies but all attempts to send the text content of this PDF this morning have been blackholed until I managed to send the contents to InternetNZ PAG as a PDF which made it through filters which been

...

this content. I have had no problem send other messages etc.

Your comments would be appreciated as I see this as a major issue which may require all certificates to be regenerated and then only used on patched systems whose memory and priv keys cannot read copied by external

stopping parties.

...

Sincerely Michael S Sutton Director - Awacs Communications (NZ) Limited Transit Room The Dominion Observatory 34 Salamanca Road Kelburn Wellington +64 21 305500 Twitter & Skype: Mikiwis http://www.awacs.co.nz https://www.google.co.nz/#q=michael+sutton+nokia+patent http://www.linkedin.com/profile/view?id=16587996&trk=tab_pro

_______________________________________________ NZNOG mailing list NZNOG(a)list.waikato.ac.nz javascript:; http://list.waikato.ac.nz/mailman/listinfo/nznog

-- Netspace Services Limited http://www.netspace.net.nz Phone +64 4 917 8098 Mobile +64 21 246 2266 Level 4, 191 Thorndon Quay, Thorndon PO Box 12-082, Thorndon, Wellington 6004, New Zealand

_______________________________________________ NZNOG mailing list NZNOG(a)list.waikato.ac.nz javascript:; http://list.waikato.ac.nz/mailman/listinfo/nznog

_______________________________________________ NZNOG mailing list NZNOG(a)list.waikato.ac.nz javascript:; http://list.waikato.ac.nz/mailman/listinfo/nznog

_______________________________________________ NZNOG mailing list NZNOG(a)list.waikato.ac.nz javascript:; http://list.waikato.ac.nz/mailman/listinfo/nznog

On 9 April 2014 20:36, Don Stokes wrote:

Is there any indication out there as to how widely this bug has been exploited?

the only thing i've seen that mentions any possible use of the exploit: http://arstechnica.com/security/2014/04/dear-readers-please-change-your-ars-... With Ars servers fully updated, it's time to turn our attention to the next phase of recovery. In the hours immediately following the public disclosure of the so-called Heartbleed vulnerabilityhttp://arstechnica.com/security/2014/04/critical-crypto-bug-in-openssl-opens..., several readers reported their Ars accounts were hijacked by people who exploited the bug and obtained other readers' account passwords. There's no way of knowing if compromises happened earlier than that. Ars has no evidence such hacks did occur, but two years is a long time. There's simply no way of ruling out the possibility.

There is no way to know for sure. The exploit leaves no trace unless you were looking for it with something like very specific network detection signatures. As I've said before, there are 3 required steps. Identify effected equipment. This could be anything linking to OpenSSL. Web servers, mail servers, VPN servers even routers which use TLS to secure configuration sessions. Patch effected equipment. All of it. Do it now or turn it off. Revoke and reissue all key material and Certs. If you use PKI in anger then key/cert rollovers should have been part of an emergency plan anyway. You've all got emergency key rollover procedures in place for DNSSEC as well right :) Do all the steps. It's like people who try and bargain or rationalise their way out of rebuilding servers they know have been compromised. We all know its best practice, just do it. Dean On Wednesday, April 9, 2014, Don Stokes wrote:

Is there any indication out there as to how widely this bug has been exploited? I.e. if you've patched servers in the last 24 hours, how likely is it that your certificate keys have been leaked over the last months / year?

Not looking for accurate numbers, just roughly where on the scale of, "this is possible but no reports of actual use" to "all the black hats have been doing this for years so you're screwed unless you re-issue and revoke your certs" the exploit lies.

Also, last time I worried about this, certificate revocation was, uh, largely unimplemented. That was a while ago. How well does it work now? And with potentially large numbers of revoked certs?

-- don

_______________________________________________ NZNOG mailing list NZNOG(a)list.waikato.ac.nz javascript:; http://list.waikato.ac.nz/mailman/listinfo/nznog

On 2014-04-09 20:36 , Don Stokes wrote:

Also, last time I worried about this, certificate revocation was, uh, largely unimplemented. That was a while ago. How well does it work now? And with potentially large numbers of revoked certs?

From what I've seen of discussion around certification revocation in the last day or so, it's still pretty unevenly implemented/turned on -- CRL lists aren't updated/checked promptly, and the latency, etc, on the online checks is generally high enough that waiting on results tends not to be done. I'd imagine this can only get worse with a batch of brand new certificates (ie, no cached data on them), and high volume of revoked certificates. Eg, from a couple of years ago: https://www.imperialviolet.org/2012/02/05/crlsets.html And apparently, eg, Chrome doesn't check by default: https://twitter.com/jm/status/453594955538501633 (you can turn it on, but the chances of every visitor to every website turning on that obscure feature are... low) Both via a well written blog post today: http://utcc.utoronto.ca/~cks/space/blog/web/SSLPragmaticKeyCompReactions which also references: https://twitter.com/neelmehta/status/453625474879471616 https://twitter.com/tqbf/status/453680389471879168 (the first from one of the finders of the bug), that indicates memory allocation patterns make private key exposure "unlikely". Obviously "unlikely" is not "impossible". So the paranoid/high risk make wish to treat this like a full fire drill. But from a "well SSL is pretty trivial to MITM if you're well connected" point of view (eg, tame CA on speed dial, or own a sign-everything intermediate, or ...), if you were just using SSL to "raise the bar slightly" on being snooped on, for something low value, patching ASAP and carrying on with life may be a risk/benefit trade off you're willing to accept. I'd caution that it should be a deliberately made choice to do that though. Ewen

https://mobile.twitter.com/1njected/status/453797877672706048 Screen dump of private key data in exploit. Also a claim that it's been done on gentoo @thegrugq https://mobile.twitter.com/thegrugq @1njectedhttps://mobile.twitter.com/1njected Cool! I've recovered it from Apache on Gentoo as a bare prime factor in binary, but your demo's a lot clearer. https://mobile.twitter.com/makomk/status/453829416003850240 Seriously guys enough with the equivocation. It's an exploit in the wild that exposes your private key material. Patch, Revoke and Reissue. On Wednesday, April 9, 2014, Dean Pemberton wrote:

And from the same twitter thread saying it was unlikely...

@neelmehta https://mobile.twitter.com/neelmehta @tqbfhttps://mobile.twitter.com/tqbf We can extract the private key successfully on FreeBSD after restarting apache and making the first request with ssltest.py 6:27pm - 9 Apr 14 https://twitter.com/1njected/status/453781230593769472

https://mobile.twitter.com/1njected/status/453781230593769472

On Wednesday, April 9, 2014, Ewen McNeill javascript:_e(%7B%7D,'cvml','nznog(a)ewen.mcneill.gen.nz');> wrote:

...

On 2014-04-09 20:36 , Don Stokes wrote:

...

Also, last time I worried about this, certificate revocation was, uh, largely unimplemented. That was a while ago. How well does it work now? And with potentially large numbers of revoked certs?

From what I've seen of discussion around certification revocation in the last day or so, it's still pretty unevenly implemented/turned on -- CRL lists aren't updated/checked promptly, and the latency, etc, on the online checks is generally high enough that waiting on results tends not to be done. I'd imagine this can only get worse with a batch of brand new certificates (ie, no cached data on them), and high volume of revoked certificates.

Eg, from a couple of years ago:

https://www.imperialviolet.org/2012/02/05/crlsets.html

And apparently, eg, Chrome doesn't check by default:

https://twitter.com/jm/status/453594955538501633

(you can turn it on, but the chances of every visitor to every website turning on that obscure feature are... low)

Both via a well written blog post today:

http://utcc.utoronto.ca/~cks/space/blog/web/SSLPragmaticKeyCompReactions

which also references:

https://twitter.com/neelmehta/status/453625474879471616 https://twitter.com/tqbf/status/453680389471879168

(the first from one of the finders of the bug), that indicates memory allocation patterns make private key exposure "unlikely". Obviously "unlikely" is not "impossible". So the paranoid/high risk make wish to treat this like a full fire drill.

But from a "well SSL is pretty trivial to MITM if you're well connected" point of view (eg, tame CA on speed dial, or own a sign-everything intermediate, or ...), if you were just using SSL to "raise the bar slightly" on being snooped on, for something low value, patching ASAP and carrying on with life may be a risk/benefit trade off you're willing to accept. I'd caution that it should be a deliberately made choice to do that though.

Ewen _______________________________________________ NZNOG mailing list NZNOG(a)list.waikato.ac.nz http://list.waikato.ac.nz/mailman/listinfo/nznog