To put something vaguely closer to on topic before Andy goes postal. Heres
Bill Norton's ( you may remember him from previous network
conferences) little summary of the Peering BOF at this week's nanog.
The most interesting bit is the "Transit Survey" of prices for Tier1
interconnect. This is roughly how much it'll cost you if you move your
site to the US and send your content to TelstraClear via a large US provider.
--
Simon J. Lyall | Very Busy | Web: http://www.darkmere.gen.nz/
"To stay awake all night adds a day to your life" - Stilgar | eMT.
---------- Forwarded message ----------
From: William B. Norton
Subject: Peering BOF IX Meeting Minutes
Peering BOF IX Meeting Minutes Seattle, WA
May 16, 2005 7:30-9:00PM
...Transcribing my notes here for those who couldn't make the Peering
BOF... comments/corrections to wbn(a)equinix.com or wbn(a)umich.edu
A 5 Minute Plea for Multicast Peering
-------------------------------------------------
Celeste Anderson (ISI/Los Nettos) spoke about the need for greater
peering of native multicast routing. The audience commented that the
volume of multicast traffic was still relatively small (a few T1's
worth) in comparison to unicast traffic, and that there is still not
significant customer demand to do so. We discussed the chicken and egg
problems for a bit.
Transit Survey
------------------------
A few people in the community suggested it would be good to get a
sense of transit prices, to see if they were continuing to fall,
remaining constant, or were rising back up. In previous white papers
I had polled the Peering Coordinator Community to get a sense of
transit prices for comparison against peering costs, so with this
previous data and an audience survey we to take a stab. I asserted
that with transit you often get what you pay for, and we were not
interested in identifying the "lowest price", the "bottom feeder"
price points that have no customer service, and that there is in fact
a qualitative difference between transit services. I was looking for
approximations of "Tier 1 ISP" pricing ? as expected, there was some
debate here. To avoid the rathole of "Tier 1" etc. and to focus on
"quality" of transit services for a quick price measure I pulled
together some questions to disqualify some upstream providers from the
survey:
Q1: When you complain about packet loss to your Upstream, you are told
A) I have Bob working on it, he will call you back in 15 minutes
B) Call this other number (which rings busy)
C) ?Let me put you on hold for an hour(Barry Manilow sings the songs
that make the whole world sing)
D) You should have sent your packets earlier
I told folks if they answered "C" or "D" to please not answer the poll
? we were not interested in bottom feeder pricing for this poll.
Q2: My Upstream Provider's Reliability is
A) 6 9's
B) Close to 6 9's
C) Closer to 9 6's
D) Don't know ? I'll let you know when they stay up long enough to measure.
I told folks if they answered "C" or "D" to please not answer the
poll? we were not interested in bottom feeder pricing for this poll.
Q3: My Upstream Provider's Customer Service Makes Me Feel?
A) Like a VIP
B) Like a Nuisance
C) Very, very unclean.
D) Like I was just traded to another inmate for 2 packs of menthol cigarettes.
I told folks if they answered "C" or "D" to please not answer the
poll? we were not interested in bottom feeder pricing for this poll.
Source: A couple of these questions/answers came from a Wired magazine
article I read on the flight up, which referenced
http://www.5ives.com.
While not a particularly scientific survey, we did get 28 viable data
points with the averages being:
10Mbps commit: $41.75/Mbps
100Mbps commit: $33.83/Mbps
1000Mbps commit: $20.73/Mbps
10000Mbps commit: $13.80/Mbps
About 6 months or so ago I did a similar poll with the Peering
Coordinator Community and the #s were around $85, $60, $45, $30,
indicating that perhaps the transit prices have indeed fallen. If any
of you are interested in the raw data (thanks to Eric Troyer for
putting the data points into a spreadsheet) let me know and I'll email
you the spreadsheet.
Peering at 10G ? introduction to The Great (Public vs. Private) Peering Debate
----------------------------------------------------------------------------------------------------------------
The motivation for the Great Debate was a white paper I am working on
called "The Great (Public vs. Private) Peering Debate: Peering at 10G"
in which I documented the Peering Coordinator Community assertion and
the math that the next best alternative to 10G Public Peering was many
1G Private Peering sessions. I walked through the models for both
cases (10G Public and 1G Private) peering, specifying sample equipment
(thanks ras!), cost estimates, etc. so we could compare the two
alternatives side-by-side. There were many valid points raised during
this discussion including:
1) In the modeling we were not including backbone costs, only IX and
Peering equipment fees at one location,
2) There are other valid configurations of equipment, including
removing the peering router and using switches by themselves (debate
here as well),
3) There were a wide variety of redundancy configurations that
could/should be considered as well in both cases.
The nice thing about modeling this stuff is the ability to identify
and discuss assumptions, to change #'s, to have something on the table
to discuss, and it was a lively discussion before and after the BOF.
The punch line for the current model I shared was that public peering
at 10G and private peering at 1G both are very cost effective
solutions, yielding less than $10/Mbps with a few gigabits of peering.
Also, the more traffic peered, the more the cost of these two models
of peering converged ? so Public vs. Private becomes an architectural
/ religious issue.
This leads to the Great Debate ? when do people prefer Public Peering
and when do they prefer Private Peering, and why?
The Great (Public vs. Private) Peering Debate
-------------------------------------------------------------------
I recruited Maurice Dean (who was the Peering Coordinator at Global
Crossing and now at Google) to present and defend "The Strongest
Arguments for Public Peering", and Peter Cohen (Peering Coordinator
for Telia) to share "The Strongest Arguments for Private Peering". We
used a modified Oxford Style Debate, providing each side three minutes
for opening remarks, three minutes each to attack the other side's
points and defend their own, a few minutes for the audience to ask
clarification questions, and finally two minutes for the debaters to
present summary remarks. The audience would then vote for "Who made
the more compelling case."
Maurice started out by pointing out that IXes today are not the IXEs
of 1995, but rather are large and reliable infrastructure that provide
access to a rich population of potential peers. Public peering
provides instantaneous access to peers, provides aggregation benefits,
ease of administration and is easy to scale peering capacity. He
pointed also to the marketing benefits of public peering at the IXes.
Peter opened by stating that Private Peering provided superior control
over peering infrastructure. He went on that Private Peering scaled
very well, that the same fiber pairs could be reused as both sides
simply upgraded cards. Private Peering is simpler according to Peter
since no NetFlow was needed, that there were not 20 peers all peering
across the same 10G interface. For some he claimed ISPs, the only way
to measure public peering traffic volume to a peer was to turn down a
peering session and see the difference in traffic load.
Maurice countered reinforcing that modern IXes have overcome the head
of line blocking issues of the past and indeed scale very well, and in
fact scale well even across a metro area. He defended the NetFlow
attack by mentioning that some IXes have sFlow, providing public
peering visibility on a per-peer traffic volume basis.
Peter countered that Maurice was a "Stooly for the IXes", and
depending on a single port for peering is asking for trouble. Further,
participation in some IXes requires going to member meetings, and who
has time for that. Finally that the added complexity of adding a
public peering switch was problematic ? he used the analogy of air
travel. "How many of you prefer a direct flight over one with
connections?"
The audience asked a variety of questions next but I did not jot them
all down. A few points did come up though ? that circuit upgrades for
private peering can take 6 months or more depending on which side is
in a hurry to upgrade. Another point was that public peering was seen
by some as easier to justify to their company since it would be used
across many peers.
The closing remarks were a rehash of the points above, but Maurice
introduced a couple additional points ? that 90 day trials could be
effectively done over a public peering fabric without additional costs
to those attached, and that public peering fabrics enabled more
opportunities (for peering? For add'l services? For transit
sales?...not sure what he meant here). Peter reasserted that placing
all the eggs in one public peering port was just asking for trouble.
The audience next voted "Who presented the more compelling case?"
We had to count twice because it was so close. The final results:
Public Peering presented the more compelling case: 37
Private Peering presented the more compelling case: 33
Maurice Dean won the Great Peering Debate.
White Paper Available
--------------------------------
I captured these and other arguments for and against private peering
in the white paper mentioned above ? I would love to have a few more
reviewers give me some feedback/edits/etc. If you are interested in 10
Gigabit Ethernet Peering or the debate discussion, send me an email
(wbn(a)equinix.com) with the Subject: "The Great (Public vs. Private)
Peering Debate ? Peering at 10G" and I'll send you a copy. If you
provide comments on the draft I will add you to the acknowledgements
section of the next version of the document.
We were running out of time so we moved quickly to Peering Personals,
giving Peering Coordinators a chance to introduce themselves to the
group.
Peering Personals ? (jotted down everything on the sheets)
--------------------------------------------------------------------------------------
Korea Telecom ? 4766 ? Bong-Hwa Song ? 40 Gbps Capacity, peering in
Seattle PAIX, Palo Alto PAIX, EQLA, LINX, AMSIX, songbh(a)kt.co.kr
AARNET ? 7575 ? Mark Prior ? Research Net, Seattle SIX, PAIX, LAIIX,
LAAP, FixW, P|Wave, mark.prior(a)aarnet.edu.au
Netflix ? Vish Y ? vish(a)netflix.com ? Selective peer
Steve Gibbard ? PCH ? 42, 3856 ? Data Collection ? Open Peering 0 25
locations all over the world (SIX, EQLA, LAIIX, NYIIX, PAIXNY,
EQASH,NOTA,?)
Steve Gibbard ? Hurricane Electric ? 6939 ? scg(a)stevegibbard.com,
peering(a)he.net ? 2700 routes, public but migrating to private at
EQSJO, EQLA, EQCHI, EQDAL, EQASH, PAIXPA,NYIIX, LINX, AMSIX
Brokaw Price ? Yahoo! ? 10310 ? OPEN ? peering(a)yahoo-inc.com
Todd Underwood ? 64597 ? todd(a)renesys.com ? NOTA, LINX, Goofy peering,
route collection only, store updates,
Google ? Maurice Dean ? 15169 ? selective peering ?
peering(a)google.com, NYIIX, EQ-CHI, EQCHI, EQASH, NOTA,
PAIXPA,PAIXVA,1118th, TelX (60 Hudson)., LINX< AMSIX, LAIIX
Akamai ? 12222 ? Patrick Gilmore ? 20940 outside U.S., OPEN and at
every IX, content heavy, peering(a)akamai.com
ESNET ? 293 ? Yvonne ? ipv4, ipv6, multicast, EQASH, EQSJO, MAEEast
MAEWest, PAIX, AADS, FixW, Pacific NW Gigapop, peering(a)es.net,
selective peering policy with AUP
Celeste Anderson ? AS4, AS27, AS2152/2153, ISI/Los Nettos, research
net, lots of eyeballs, LAIIX, LAAP, Pacific Wave, PAIX LA, Mostly
OPEN~selective, eyeballs, peering(a)pacificwave.net, celestea(a)usc.edu,
Matt Peterson ? SixApart , AS# pending, SIX, EQSJO, LiveJournal.com,
matt(a)sixapart.com
--------------------------------------------------------------------------------------------------------------
Hope this help!
--
//------------------------------------------------
// William B. Norton
// Co-Founder and Chief Technical Liaison, Equinix, Inc.
// GSM Mobile: 650-315-8635
// Skype, Y!IM: williambnorton
