- NZNOG - lists.nznog.org

Re: [nznog] racks for beer.
by Stuart Pigneguy 10 Jun '15

10 Jun '15

I also have two 19" server racks (with doors) no longer required and can be exchanged for beer. I believe they are 42U, available for pick up in Auckland on Monday 22nd. Otherwise they'll be scrapped! - please reply off list Cheers, Stuart NZNOG lurker since 2001

1 0

DEI bit on Fortigate D series
by Dan BAJARCA 09 Jun '15

09 Jun '15

Hi All, Just sharing a bit of knowledge around issues we've had with Fortigate firewalls on UFB Accelerate plans. Some of our customers reported slow download speeds (generally close to 2Mbps) when using Fortigate D series firewalls on UFB 200/100 (tested with 30, 60 and 90 models). The issue is caused by the units dropping inbound packets with the DEI bit set and showed up on 5.0 and 5.2 software. After some extensive troubleshooting with Fortinet they've eventually acknowledged the fault and moved it to their bug queue. Hopefully a fix isn't too far away. If you're currently battling this issue, a tested workaround would be to configure the circuit as untagged. Chorus should also be able to disable the DEI bit marking on a per service provider basis but better think twice before requesting it. Cheers, Dan

1 0

Assistence with Brocade CNA
by Tristram Cheer 09 Jun '15

09 Jun '15

Hi All, I've run out of options for an issue we've got with some brocade 1020's we're using in a pinch, Is there anyone who's familiar with the 1020's and vmware on the list who's able to drop me a message off-list? I'm hoping it's a simple thing and some beer for your efforts. Cheers Tristram Cheer Network Architect - Most problems are the result of previous solutions... [Visit our Website]<http://www.uber.co.nz/> 09 438 5472 Ext 803 | 022 412 1985 | PO Box 5083, Whangarei, 0140 t(a)uber.co.nz<mailto:t(a)uber.co.nz> | www.ubergroup.co.nz<http://www.ubergroup.co.nz> [http://uber.co.nz/uploads/signatures/fb-s-v4.png]<http://ubergroup.co.nz/fb> [http://uber.co.nz/uploads/signatures/twitter-s-v4.png] <https://twitter.com/#!/ubergroupltd>

1 0

Where can congestion occur on a UFB (GPON) network?
by Joel van Velden 03 Jun '15

03 Jun '15

Hi all, At Auckland Thursday Night Curry I asked what I thought was a simple enough question: "In this this new world of GPON fibre that is UFB, /where can congestion occur/?" Obviously at handovers, no question, but where else? From what I can figure, given a hypothetical GPON network using a 24-way optical split (seems to be number quoted in a NZ context) Standard GPON suggests 2.488 Gbps down, so split 24 ways is 103.7 Mbps (Not sure what packet sizes) So if all customers behind a splitter are provisioned at a maximum of 100Mbps down, there could be /no/ congestion? Obviously this is not the case with 200 and 1000 down being sold. Are there other possible points of congestion between the point of handover and the GPON switch? (forgive me I'm not up with the exact terminology. OLT?) What have I got wrong? -- Regards, Joel van Velden

1 0

RE Legal Text on TICSA - Visit to Wellington/ Seeking input
by Beau Murrah 27 May '15

27 May '15

Hello NZNOG list, Some months ago I emailed to the list about my plan to produce a legal manual/critical to guide to the TICSA legislation for network operators/the list. I got some feedback from people on this list and outside of it that there was interest in such analysis and some highlighted a number of particular issues to look into/affecting them. My original hope was to have it completely edited and ready to share to the list about this time - I know updated guidance was/is planned to be released by the NCSC soon (or has already been released). Well, Im still going at it but it is proving difficult topic to write about and finish into a usable and credible form. My text is currently 6500+ words - and I think there are definitely some genuinely some useful or novel things for consideration but it is also all over the place. Initial feedback on the very rough draft was that it doesn’t seem clear exactly exactly who I am writing for - seeming like a mix of legal academic essay and direct practical consideration for network operators and also partly political content about the nature of bureaucracy in this area (the latter Ive decided to cut out as much as possible. I believe writing about this topic is difficult because I realised I don’t know much about what the worklife of a network operator of a truly large network is actually like at all. I have read/seen most all of the publicly available information - close reading of the legislation and related legislation, NCSC guidance, information from this list and so on. None of the technical people I know are network operators. It is hard to gauge what the level/type of interaction with the Agency is actually like broadly. I really don’t want to inadvertently publish any bullcrap technical claims and there is plenty of cyber-paranoid stuff out there already. So, I digress- I am visiting Wellington later this month (28th-31st May) and I would love to get insight from a network operator(s) over beer/coffee/water. I am actually less interested in what TICSA compliance has been like for you - and I don’t expect or want you to “leak” anything or disclose anything - but to hear more about the worklife of a network operator in general day to day that would come under examination in any potential legal proceeding under TICSA. To give a quick preview of my text, it focuses on the fundamental importance of good faith in the legislation to protecting you from any worst case scenario from alleged non-compliance. It considers what remaining in good faith could look like in a situation of alleged non-compliance and what variety of reasons or events could lead to non-compliance while good faith has been retained by the network operator. This depends on a lot on getting an understanding your workflow and obligations and what kind of people are you expected to report to and way that evidence accumulates in your day - much like in any employment situation or potential examination/dispute. There is very limited legal precedent to what Good Faith actually is from legislation and case law around the issue of employment and law and collective bargaining. How good faith might be interpreted involving network operators dealings with the Agency is a very open question and something all network operators ultimately have an interest in. I will say that the guidance provided to you by the NCSC guidance is not legal binding guidance and makes particular assumptions about the nature of good faith. It is very possible that this entire law will never effectively see a day in court if the narrative that this regulation is intrinsically a piece of national security and good faith flatly means following the Agency’s commands or striving to meet their interpretation of compliance. I do not think this is correct nor is anyway certain. I will add though, an actual Court case under this legislation’s confidentiality and evidential provisions will attempt to turn a civil court into a “Secret Court” like FISA this Country has never really seen and will intimidate the most experienced of lawyers, Judges and Court staff - there is a huge impetus to avoid this situation, and hopefully this can be put more in the network operators favour, rather than to scare them. I invite you to email me off-list if you want to meet, or to email on-list to propose a venue/time on-list for people to chat as a group. Happy Budget day, ___________ Beau Murrah Ph: +64 27 375 7897 Email: bmurrah(a)icloud.com <mailto:bmurrah(a)icloud.com> - PGP: https://keybase.io/airbridge <https://keybase.io/airbridge> Linkedin: https://www.linkedin.com/in/bmorganmurrah <https://www.linkedin.com/in/bmorganmurrah> Enrolled Barrister and Solicitor of the High Court of New Zealand (NB: not a lawyer <https://www.lawsociety.org.nz/for-lawyers/joining-the-legal-profession/admi…>)

2 1

Re: [nznog] Need assistance troubleshooting a DNS issue
by Glen Eustace 27 May '15

27 May '15

To satisfy the curious and to provide some closure on this incident, I have included a statement from Ryan Joe. The incident began approx last Oct/Nov and was resolved last week. I would like to thank Peter Lambrechtsen for getting the right people at Spark to contact me and for their perseverance working through what turned out to be more than one non-trivial issue. The impact, unfortunately, was not wide spread and hence had gone largely un-noticed but had the potential to impact a lot more people. I don’t believe that it is my place to go into the details of the issues, what was found and how they were fixed. Perhaps a Spark engineer would be prepared to do so as I believe it would be of interest to many on this list. "As discussed on Friday, this email is to acknowledge that Spark experienced a problem with our DNS which resulted in some Spark customers being intermittently unable to access content hosted at godzone.net.nz <http://godzone.net.nz/>. I’d like to sincerely apologise for any frustration or inconvenience caused by this – we have now fully resolved the issue” Ryan Joe Senior Product Manager - xDSL -- =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Glen and Rosanne Eustace, GodZone Internet Services, a division of AGRE Enterprises Ltd., P.O. Box 8020, Palmerston North, New Zealand 4446 Ph: +64 6 357 8168 <tel:%2B64%206%20357%208168>, Fax: +64 6 357 8165 <tel:%2B64%206%20357%208165>, Mob: +64 27 542 4015 <tel:%2B64%2027%20542%204015> "Specialising in providing low-cost professional Internet Services since 1997"

1 0

Random number generators
by Nathan Ward 25 May '15

25 May '15

Hi all, As we all know, random numbers are an important part of cryptography, which is required for tools that are important to network operators like the widely deployed RPKI. Most systems today generate random numbers using a PRNG, rather than a dedicated hardware random number generator. A PRNG is only as good as its entropy source - if you have a small amount of entropy, an attacker can start to predict the output of your system’s PRNG, which is obviously bad. There are a number of existing tools for generating entropy: - HAVEGE implementations (i.e. http://www.issihosts.com/haveged/) - Audio/video sources (audio_entropyd and video_entropyd) - Some other things like mouse/keyboard input (i.e. when you wiggle your mouse over that window in whatever that app was that generated keys, puttygen maybe?) - Network interrupts is also a common source A week or two back I was a little hungover and didn’t want to do any real work, and decided to write my own entropy harvesting tool, and have the code available at the below URL: https://github.com/nward/trueentropy I strongly encourage you NOT to run this on production systems, until it has been certified (a good standard to shoot for would be NIST SP 800-90B perhaps), but I would like to get feedback, so please read the code and suggest improvements. Perhaps you have some additional sources of entropy data that would be useful. I would particularly like a good way to “boil down” the entropy generated. The Linux kernel by default can take up to 4096 bits, and this provides far, far more than that, which just seems like a bit of overkill. Please feel free to submit pull requests with ideas for that - it would be a great project to learn a bit of Python. Generally, entropy should not be sourced from a system you don’t control, or sourced over a network you don’t control, for obviously reasons. However, the quality of the data is so good in this case that I believe it negates those concerns. The data is also fetched with HTTPS, so as long as the source doesn't also starting using this as their entropy source (known as entroception) we should be OK - see my recommendation about production systems. -- Nathan Ward (opinions all my own, blah blah blah)

8 14

Dark fibre in the outer wilderness of... Parnell
by Matthew Poole 24 May '15

24 May '15

Hi all I'm getting discouraging information on the availability of dark fibre around the bottom end of St George's Bay Rd in Parnell, near Xero and the Textile Centre. Seems that nobody has any, or at least nobody is mentioning any. If anyone knows of dark fibre capacity in that area could they please contact me off-list. Cheers -- Matthew Poole "The difference between theory and practice is that practice is easier in theory than theory is in practice"

3 3

Re: [nznog] Random number generators
by Nathan Ward 23 May '15

23 May '15

> On 24/05/2015, at 15:12, Dean Pemberton <dean(a)deanpemberton.com> wrote: > > > I can't imagine that if you were an ISP you wouldn't know exactly where any TrueNet probes were in your network. That is possibly true if you’ve paid TrueNet for probes. If your customers have opted to have one without your involvement (which I assume is possible, given ComCom give them money to do just that across NZ) then that’s unlikely. > The only question then is which of the following strategies you employ. > > 1) do nothing. > 2) make this user the best on your network > 3) randomise the results > 4) tweek the results to spell out "what is six times nine” As I’m sure you can imagine, I’m keen to work on solutions to implement (4) if any network providers who have TrueNet portal access are keen for some fun. > Nathan, have you had a look at any other systems (SamKnows) to see if they are any closer to measuring Internet Good-ness? Briefly, but not in a while. My understanding is that SamKnows is much the same as TrueNet - a set of contrived tests, though they do a much better simulation of a web page load than TrueNet and it’s use of “URL Downloads using the WGET command” - but I don’t believe it’s anywhere near perfect in today’s javascript heavy world, where clients cache static content etc. etc. -- Nathan Ward

1 0

Anyone else having GGIS speed problems?
by Nathan Ward 19 May '15

19 May '15

Hi all, Beware, operational content. A couple of hours ago I started noticing weird things getting to stuff overseas. Looks like GGIS have some sort of path flapping that doesn’t care about IP flows. I see RTT to sites in the US jump from ~125ms to ~160ms several times per second, which to me is indicative of going the short and long way around the SXC network, respectively. # hping3 -S -p 80 216.146.46.10 HPING 216.146.46.10 (eth0 216.146.46.10): S set, 40 headers + 0 data bytes len=46 ip=216.146.46.10 ttl=55 DF id=33739 sport=80 flags=SA seq=0 win=65535 rtt=129.1 ms len=46 ip=216.146.46.10 ttl=55 DF id=47972 sport=80 flags=SA seq=1 win=65535 rtt=132.3 ms len=46 ip=216.146.46.10 ttl=53 DF id=48561 sport=80 flags=SA seq=2 win=65535 rtt=159.6 ms len=46 ip=216.146.46.10 ttl=55 DF id=35481 sport=80 flags=SA seq=3 win=65535 rtt=128.1 ms len=46 ip=216.146.46.10 ttl=54 DF id=49791 sport=80 flags=SA seq=4 win=65535 rtt=158.7 ms len=46 ip=216.146.46.10 ttl=55 DF id=36729 sport=80 flags=SA seq=5 win=65535 rtt=132.2 ms len=46 ip=216.146.46.10 ttl=53 DF id=50960 sport=80 flags=SA seq=6 win=65535 rtt=158.8 ms len=46 ip=216.146.46.10 ttl=55 DF id=37839 sport=80 flags=SA seq=7 win=65535 rtt=129.1 ms len=46 ip=216.146.46.10 ttl=55 DF id=38397 sport=80 flags=SA seq=8 win=65535 rtt=129.2 ms len=46 ip=216.146.46.10 ttl=54 DF id=39075 sport=80 flags=SA seq=9 win=65535 rtt=158.8 ms The above example is a bunch of SYN packets so are different L4 flows so it’d be *sort of* reasonable for them to go different ways. However, you see plenty of out of order TCP packets if you do a long running flow (like say trying to load a web page with large objects, such as speedtest.net) Out of order packets means TCP cries a bunch and goes slow. This is the effect I see. Note also the TTL differences and correlation to RTT. I do not see the TTL hopping around when looking at a single TCP stream, though I do see different TTLs on two concurrent streams, so I suspect that’s some other ECMP happening that isn’t a problem. Who knows where that might be, but I don’t believe it’s important. This particular host I have tested to is the speedtest.net main web server. I see the out of order packets when doing a regular speedtest.net test to the Sonic.net endpoint in California (I think San Jose?). Here is what mtr sees when tracing to the same host as above. It is unclear to me why hop 9 doesn’t see the same weird latency. It sees it sometimes, but not all the time. Perhaps there’s something about the way mtr sends packets that means the hashing is different over some sort of aggregate interface/ECMP thing. Maybe that’s where our TTL difference from above happens. I *always* see the frequent latency changes (jitter, I guess) on hop 8. 7.|-- ae1-10.tkbr12.global-gate 0.0% 10 3.3 3.5 2.2 5.3 0.7 8.|-- xe-0-0-7-3.sjbr3.global-g 0.0% 10 130.5 136.1 126.3 163.1 14.2 9.|-- ae0.pabr5.global-gateway. 0.0% 10 154.8 154.7 154.1 155.9 0.5 Here are some more samples that show the differences in later hops, but always hop 8 having jitter: 7.|-- 202.50.232.37 0.0% 10 3.5 3.2 1.5 4.1 0.7 8.|-- 122.56.127.22 0.0% 10 160.2 148.0 127.7 160.2 15.0 9.|-- 122.56.127.25 0.0% 10 158.2 148.6 132.2 166.9 14.3 10.|-- 203.96.120.74 10.0% 10 159.0 159.0 158.2 160.0 0.4 11.|-- 4.28.172.109 0.0% 10 167.6 171.3 166.3 207.7 12.8 7.|-- 202.50.232.37 0.0% 10 2.9 3.3 2.7 4.2 0.3 8.|-- 122.56.127.22 10.0% 10 159.3 147.1 126.6 168.7 17.1 9.|-- 122.56.127.25 10.0% 10 158.7 146.9 132.5 166.3 13.8 10.|-- 203.96.120.74 10.0% 10 159.7 158.8 158.0 159.7 0.0 11.|-- 4.28.172.109 20.0% 10 167.0 167.4 166.8 168.1 0.0 At least two providers have (or are in the process of) logging faults, so I figured it’s appropriate to bring it on to the list. Is anyone else seeing this behaviour? Do you have any feedback from GGIS yet? Have you had any customers complain about poor speeds/user experience? -- Nathan Ward

1 0