I thought I would write out a quick email about modern commodity hardware and Lawful Interception. I promise to try and avoid acronyms or at least explain them.

Lawful intercept appears to be a space where information likes to be scarce, and where it does exist seems purposefully designed to confuse and baffle. This list itself has some examples of this practise. Between the stick of government compliance and vendor rhetoric about how hard Lawful Interception is, operators end up opening not only their networks, but also their wallets to the Lawful Interception theme song.

OpenLI was designed to fit the small to medium sized operator market. It avoids cellular protocols entirely. It��s goal is to reduce the burden on operators and build a community of collaboration around achieving government compliance with operators providing data and voice (SIP) services so we can get on with building networks and services to our customers. If you are a large network or operate a cellular network, OpenLI is not for you in its current form. I��m sure there are a few Lawful Intercept companies that would like to talk to you. If you have had a handful of intercepts to none at all, then you might be interested in the OpenLI project as it stands today. There has been a strong interest expressed in Commercial support to help with specing, deploying and maintain an OpenLI deployment. Although not part of the original plan, this might be able to become a thing, there is some discussion between developers and initial project backers along this front. Maybe with some deployments done, and continued funding, it might grow to support 100��s of gigabits of LI traffic.

I believe that the point some commenters are making is that today technology has moved on to the point that with easy to obtain components and with the arrival of Intel Data Plane Development Kit (DPDK), which allows for software direct access to the hardware paired with smart hardware fanout NIC (which nearly all are these days), you can dedicate cores on a processor to network forwarding and get really impressive performance. I tend to get a multi-socket machine and dedicate an entire processor to being a network processor avoiding all contention on processor cache and memory bandwidth (that processor only deals with packets). A little bit of care with hardware selection and card slot population is required to ensure that the Quick Path Interconnect (QPI or the interface between the two processors) is not a point of contention by ensuring the NIC is served directly by the correct processor and you will have a packet forwarding beast. The performance under this setup is truly impressive and I have done real deployments of packet forwarding engines, switching and routing, on this technology. It works in real life, in the field.

Here is a performance report from Intel showing that only 4 cores of a 2.5G Xeon with 2X40G interfaces can hit line rate at 256 bytes doing L3 routing. This scales out wider with more queues (more processor cores), with core count so if more cores are used this would come down. Seehttps://fast.dpdk.org/doc/perf/DPDK_18_02_Intel_NIC_performance_report.pdf for details. A modern processor has 8-16 cores. In my own development lab, I have easily done line rate 64bit frame size forwarding at 2x10G on 4 cores of an 8 year old Xeon with the code compiled for debug (all compiler optimising off and full debugged enabled.)

OpenLI is based on top of libtrace , which supports Intel DPDK as the base capture interface, therefore utilising Intel��s efforts to bring extreme networking performance to the commodity space. It is well pointed out that some stats would be useful around actual performance of OpenLI, as obviously each intel DPDK program will perform differently and when I get to that point in deploying OpenLI myself, I will share some test stats of my deployment to the list if there is interest.

In my own deployment and at least one other deployment I am aware of, OpenLI will not actually see all frames going through the network anyway. The forwarding hardware will only run the packets past the OpenLI collection point if knows that interception is required (controlled by RADIUS including COA��s to adjust already running sessions). Therefore OpenLI only needs to be able to keep up with my fastest connection I sell multiplied by 2 or 3. Before you flame me about intercept detectability, this is taken into account in the design and is not a factor. My point is smart network design can help lower the requirements on intercept capability.

From: nznog-bounces@list.waikato.ac.nz <nznog-bounces@list.waikato.ac.nz> On Behalf Of brianparsons@subpico.com
Sent: 30 June 2018 17:28
To: "Joel Wir��mu Pauling" <joel@aenertia.net>
Cc: nznog <nznog@list.waikato.ac.nz>; Shane Alcock <shane.alcock@waikato.ac.nz>
Subject: Re: [nznog] Affordable TICSA LI Solution and Support

55MPS will give max ~ 28G << 100G thruput @prema+sdf+64B+ifg. In the other direction, at 100G, with ave 512B your max is 20-25Mpps, in any case 40G is fine and you save your firms $ on the 100G. Actually line capture at these speeds and thruputs have been doable for a good while now, mutli-Tbps is the challenge now. Capture/Filtering and processing are actually two different but interdependent things, as you know speed/thruput are normally qualified in terms of packet sizes and app, nic profile:sustained/peak, For capture, L3 ACLs segment work etc its just fixed and static filtering; different to triggers which need higher state protocol/flow processing, db lookups, markov etc, alot more smarts and CPU CS this lowers the upper limit. CPU stats will be very different.

regards

Brian
---------------------------- Original Message ----------------------------
Subject: Re: [nznog] Affordable TICSA LI Solution and Support
From: Joel Wir��mu Pauling <joel@aenertia.net>
Date: Sat, June 30, 2018 12:40 pm
To: brianparsons@subpico.com
Cc: "Shane Alcock" <shane.alcock@waikato.ac.nz>
"nznog" <nznog@list.waikato.ac.nz>
--------------------------------------------------------------------------

> You can achieve 55MPS from the latest generation of Smartnics - enough to
> do 100GBits ports, with Layer3 ACLs ; with almost no CPU overhead. This is
> running pretty much Mainline Linux (with Vendor Specific Smartnic offloads
> i.e ASAP2 (mellanox), or Netronome). You then obviously need to dump it
> somewhere but with some NVMEE Cards on x8 Bus you can do that too - before
> shuffling it to slow and cheap archival.
>
> So ; no you don't need any real hardware smarts to do this today, there is
> off the shelf bits you can add to off the shelf x86 hardware. Just a little
> planning of your Capture box hardware and knowing what NIC's to use.
>
> -Joel
>
> On 30 June 2018 at 11:39, <brianparsons@subpico.com> wrote:
>
>> Hi Shane,
>>
>> I appreciate your response and it sounds like you and the OpenLI team
>> making great progress, i wish the project and deployments all the best.
>> Writing packet capture isnt hard and anyone with a compiler can do it. Of
>> course a little coding b/gnd helps and depending on what your trying to do,
>> the level of hardware knowhow you need.
>>
>> Take 100% packet capture - 'shouldnt' be a problem at a constant 1 pps,
>> but it will be if you take 2 s to process. Lets say you optimize hw down .5
>> s with a DAG* (i worked their native's ~2004)* your still not going to
>> make it. Timing values are too ilustrate my point and my take away from you
>> response is OpenLI is basically targeting the general usecase here in NZ
>> where say GE or 10G ~50/60 pk. % link utilization is still pretty OK and
>> fit-for-purpose. I assume here your OpenLI app (NWO->IAF/MF
>> ASN.1Enc/MD->HI) is well optimized from your excellent experience too. One
>> suggestion maybe to bench it with XIA/Spirent STC for thruput vs
>> line-rate/packet sizes vs expected from NOC. I dont use cap libraries or
>> anything like that myself, my arch/interface is very different in exo/no
>> net_dev, zero intr,dis flow handling depending on the ICs i got on hand (i
>> have a french foreign legion policy for hardware - survive the bench and
>> your in ). LI started here in NZ but was taken to another level in the US
>> with massive linerates and volumes - vetting starts with corner case
>> burntests, sinking IPGs <9.7ns ..<.97ns (100G+) with 1 Target frame - and
>> if you can catchit send try to send that 1 frame, only after that you get
>> to do something usefull and process it. Sounds easy as you can appreciate
>> back in the day with north/south PCIE this was a massive feat, no SoCs
>> root/pcie qpi we have today.
>>
>> Obviously this level of performance is bit OTT for our general LI small
>> ISP market, but data volumes aint gonna go down - cheers for the heads up.
>> Again wish you and the OpenLI project team much success.
>>
>> regards
>>
>> Brian Parsons BE(E&EE) MIEE | CEO TFE | LI TICSA SIGINT
>>
>> ---------------------------- Original Message ----------------------------
>> Subject: Re: [nznog] Affordable TICSA LI Solution and Support
>>
From: "Shane Alcock" <shane.alcock@waikato.ac.nz>
>> Date: Fri, June 29, 2018 2:38 pm
>> To: brianparsons@subpico.com
>> Cc: nznog@list.waikato.ac.nz
>> --------------------------------------------------------------------------
>>
>> > Hi Brian, all,
>> >
>> > As the primary developer on the OpenLI project, I've been asked to
>> address
>> > some points that you've raised and correct any misconceptions people
>> might
>> > have about OpenLI as a result of this thread.
>> >
>> > For anyone concerned about a possible lack of experience in writing
>> packet
>> > capture software, most of my 14 years working here at the WAND network
>> > research group has been dedicated to both writing and running software
>> for
>> > high speed passive capture and analysis projects. I am one of the primary
>> > authors and maintainers of libtrace [1] [2], which is widely used both in
>> > academia and industry (as well as the OpenLI software itself). Libtrace
>> > supports a variety of capture methods and hardware, including DAG and
>> DPDK,
>> > and has a parallel API for writing multi-threaded packet capture
>> > applications, so I have a reasonable understanding of what is required of
>> > both hardware and software to do high-rate packet capture and encoding.
>> >
>> > OpenLI is not offering a "complete" solution to your TICSA problems. We
>> > provide software that can capture packets and convert them into ETSI
>> > records, wrapped in a centralised provisioning system for managing
>> > intercepts, along with a mediation function that will push the resulting
>> > records to the appropriate agency. Integration of OpenLI into your
>> network
>> > is a matter for the operator; however, we have been working closely with
>> > the organisations that responded to our initial requests for funding to
>> get
>> > their deployments up and running.
>> >
>> > Longer term, we are starting to have conversations about what support
>> > models we want to provide in the future, especially after the feedback I
>> > received at the NZIX AGM last night, so watch this space. WAND has a long
>> > history of maintaining and continually improving any open-source software
>> > that it has released to the public [3] even when it is no longer
>> > specifically funded by our research income (libtrace being a prime
>> > example), so if OpenLI is being used then you can anticipate that WAND
>> will
>> > continue to look after it.
>> >
>> > Finally, the core software components of OpenLI are getting close to
>> > complete. We have provisioning, mediation, multi-threaded and
>> > multi-input-interface distributable collectors. We have support for voice
>> > intercept (SIP + RTP + RTCP) and support for standard IP intercept
>> (RADIUS
>> > + IPv4), with IPv6 and static IP range intercepts not too far away. Most
>> of
>> > the upcoming effort is focused on getting test deployments working within
>> > our partner networks, throwing "real" traffic at them and squashing any
>> > issues that come up. At this stage, it's looking like another month or
>> two
>> > of work to finish the original set of requirements.
>> >
>> > Any other questions about the OpenLI project, feel free to get in touch
>> > with me directly.
>> >
>> > [1] S.Alcock, P. Lorier and R. Nelson, "Libtrace: a packet capture and
>> > analysis library", https://dl.acm.org/citation.cfm?id=2185382
>> > [2] Libtrace Team, "libtrace: C library for working with network packet
>> > traces", https://github.com/LibtraceTeam/libtrace
>> > [3] WAND network research group on GitHub, https://github.com/wanduow
>> >
>> > Thanks,
>> >
>> > Shane
>> >
>> > On Wed, Jun 27, 2018 at 11:49 PM, <brianparsons@subpico.com> wrote:
>> >
>> >> Hi Nathan,
>> >>
>> >> We dont work with the NSA, and you dont need them to decrypt mobile data
>> >> there are many keys used in a heirachy depending on the pairing mobile
>> >> nodes eNB/UE/MME/HSS etc, you just need to be on the right interface to
>> >> catch and process the exchange Control/User Plane (keys vary) is trivial
>> >> for LI vendors like myself who carry key derivation/enc/dec logic and
>> >> protocol sequencing to handle such an exchange. eg UE/HSS:CK,IK - a well
>> >> known mapping is your USIM/AuC:K. As far as knocking on doors, yes
>> >> absolutely probably me, i sell and support what I build and proud of
>> it, but
>> >> NSA lol, but we sell more off shore - why is that?
>> >>
>> >> My post was only intended to inform of a local option that could make a
>> >> material difference to operators trying to meet their obligations under
>> the
>> >> ACT.
>> >>
>> >> As an LI provider i have met a couple of your members stung badly after
>> >> signing offshore and we couldve helped them, also i was encouraged by
>> the
>> >> LEAs that speak at your conference - they obviously cant directly
>> endorse
>> >> but the spirit and intention of my post was not to offend just inform.
>> TFE
>> >> like most LI vendors do not advertise or market, and there are only a
>> >> handfull of us OEMs that build LI solutions the rest are reseller middle
>> >> men.
>> >>
>> >> As far as OpenLI free it is your developing, thats great but till then
>> >> compliance is real, but we wish you all the best. 2050 is when you'l
>> have
>> >> enough field experience in CU/FIbre/hardware with ASN.1 + ETSI/CALEA at
>> >> working at single thread code level. Lets say you do cobble something
>> >> together before then - once you can multithread your kernel (tuplet
>> flows)
>> >> linerate 200g (64B) stream + LI let me know, sorry lets make it easy -
>> >> start with GE @ 1.44 Mpps + LI and ASN enc over HI => garauntee you a
>> DoS
>> >> all day and dont even think about SLAs. But good luck with that, until
>> then
>> >> TICSA 2017
>> >>
>> >> The point is LI is not just about about Software. Hardware, firmware and
>> >> real world field experience is required - how do you handle FEXT/NEXT on
>> >> upstream active/passive TAPs can you run in rx only given the
>> requirements
>> >> and restrictions on spurious noise and traffic, there are many tech and
>> >> capability requirements of the probe/mf which ETSI Parts are you going
>> to
>> >> do first?. HA/asymmetric network deployments (non-collo paths), how will
>> >> you handle this situ? and TIER 4 support on commercial fabrics (Open
>> >> generally means zip support - so i geuss nothing is realtime). If you do
>> >> all this how will you be able to demonstrate at any point in time the
>> two
>> >> core aspects of TICSA 'format usability' and 'nw readiness'.
>> >>
>> >> We do LI not 'network security', ie. we build the 'device' and MF as
>> >> descirbed in the NZ Search and Surveillance Act 2012 and NZ TICSA 2017.
>> >>
>> >> regards
>> >>
>> >> Brian Parsons BE(E&EE) MIEE | CEO TFE | LI TICSA SIGINT
>> >>
>> >>
>> >>
>> >> cheers
>> >>
>> >>
>> >> ---------------------------- Original Message
>> ----------------------------
>> >> Subject: Re: [nznog] Affordable TICSA LI Solution and Support
>> >>
>>
From: "Nathan Ward" <nznog@daork.net>
>> >> Date: Wed, June 27, 2018 10:06 pm
>> >> To: brianparsons@subpico.com
>> >> Cc: nznog@list.waikato.ac.nz
>> >> ------------------------------------------------------------
>> --------------
>> >>
>> >> >
>> >> >> On 27/06/2018, at 11:45 AM, brianparsons@subpico.com wrote:
>> >> >>
>> >> >> We are Telecom Forensics, an OEM of LI solutions based on the Kapiti
>> >> Coast.
>> >> >>
>> >> >> Spam spam spam
>> >> >
>> >> > Are you the chap from Kapiti who a few years ago was knocking about
>> >> claiming that the NSA had given him some sort of secret codes to
>> magically
>> >> decrypt all mobile data, or was that someone else?
>> >> >
>> >> > Not too sure why you��d be pushing this here, when there��s a community
>> >> effort to develop something open source and free.
>> >>
>> >> I��m looking forward to where the OpenLI project goes, and I��d suggest
>> that
>> >> anyone looking for LI tools look towards the OpenLI project, which has
>> been
>> >> discussed on this before, and at the NZNOG conference, and I believe
>> will
>> >> be discussed tomorrow at the NZIX AGM.
>> >> > ps. It��s free, but, go support it with some $, it��ll still be cheaper
>> >> than commercial solutions.
>> >> >
>> >> > NZ: Highest per capita ETSI compliant LI implementors in the world.
>> >> >
>> >> > How long before Fincham pops up saying ��oh also there��s an OpenLI
>> >> project you should look at��? Place your bets.
>> >> >
>> >> > Looking forward to laughing about this one over drinks tomorrow night.
>> >> >
>> >> > --
>> >> > Nathan Ward
>> >> >
>> >> >
>> >>
>> >> _______________________________________________
>> >> NZNOG mailing list
>> >> NZNOG@list.waikato.ac.nz
>> >> https://list.waikato.ac.nz/mailman/listinfo/nznog
>> >>
>> >>
>> >
>>
>>
>> _______________________________________________
>> NZNOG mailing list
>> NZNOG@list.waikato.ac.nz
>> https://list.waikato.ac.nz/mailman/listinfo/nznog
>>
>>
>