Has anyone got any real world experience of the usefulness of jumbo frames on ethernet (ie, MTU > 1500 bytes). A number of 1000mbps interfaces support it, but it seems to be that it would only be useful if jumbo frames were supported and enabled on all equipment between the sending NIC and final IP destination, so I'm struggling to see how enabling it in just say the core of a network would be all that helpful.
MPLS. VLANs. L2TP with 1500byte payloads. etc. On 28/03/2007, at 3:01 PM, Ian Batterbee wrote:
Has anyone got any real world experience of the usefulness of jumbo frames on ethernet (ie, MTU > 1500 bytes). A number of 1000mbps interfaces support it, but it seems to be that it would only be useful if jumbo frames were supported and enabled on all equipment between the sending NIC and final IP destination, so I'm struggling to see how enabling it in just say the core of a network would be all that helpful.
_______________________________________________ NZNOG mailing list NZNOG(a)list.waikato.ac.nz http://list.waikato.ac.nz/mailman/listinfo/nznog
!DSPAM:22,4609daac107231355811551!
I certainly find Jumbo Frames helpful with an iSCSI SAN in the mix. -----Original Message----- From: Ian Batterbee [mailto:ian.batterbee(a)aut.ac.nz] Sent: Wednesday, 28 March 2007 3:02 p.m. To: NZNOG(a)list.waikato.ac.nz Subject: [nznog] jumbo frames Has anyone got any real world experience of the usefulness of jumbo frames on ethernet (ie, MTU > 1500 bytes). A number of 1000mbps interfaces support it, but it seems to be that it would only be useful if jumbo frames were supported and enabled on all equipment between the sending NIC and final IP destination, so I'm struggling to see how enabling it in just say the core of a network would be all that helpful. _______________________________________________ NZNOG mailing list NZNOG(a)list.waikato.ac.nz http://list.waikato.ac.nz/mailman/listinfo/nznog
On 28/03/2007, at 3:01 PM, Ian Batterbee wrote:
Has anyone got any real world experience of the usefulness of jumbo frames on ethernet (ie, MTU > 1500 bytes). A number of 1000mbps interfaces support it, but it seems to be that it would only be useful if jumbo frames were supported and enabled on all equipment between the sending NIC and final IP destination, so I'm struggling to see how enabling it in just say the core of a network would be all that helpful.
Well if you're after increased TCP session throughput, jumbo frames are nowhere near as might seem (refer to Perry's TCP bandwidth calculator), and in this case end to end large MTU support is required. If you're running any form of encapsulation in your network, then being able to encapsulate complete 1500 byte frames is indeed a useful feature. Usually in this case though, you're simply after something 'a bit bigger' than 1500 bytes. In either case, there is no reason _not_ to enable support for the largest MTU you can support wherever possible. No point in waiting for the entire inter-network to support them otherwise it'll never happen :). Cheers, Jonny.
Jonny Martin wrote:
On 28/03/2007, at 3:01 PM, Ian Batterbee wrote:
Has anyone got any real world experience of the usefulness of jumbo frames on ethernet (ie, MTU > 1500 bytes). A number of 1000mbps interfaces support it, but it seems to be that it would only be useful if jumbo frames were supported and enabled on all equipment between the sending NIC and final IP destination, so I'm struggling to see how enabling it in just say the core of a network would be all that helpful.
Well if you're after increased TCP session throughput, jumbo frames are nowhere near as might seem (refer to Perry's TCP bandwidth calculator), and in this case end to end large MTU support is required.
What my calculator doesn't really show very well is that larger MTUs means you accelerate through slowstart faster. Your top speed is still the same (you're constrained by bandwidth), but you get there earlier. My calculator models "steady state" and ignores slow start. If you have non congestion based loss, you want to get back to sending at full speed as fast as possible, which is where having a larger MTU helps. Then of course theres usually the fact that your per packet overheads go down with MTU increases too. Most OS's let you specify an MTU of 9k on your interface, but then set routes out that interface with a lower MTU on the route. If you're careful you can have your L2 use "192.0.2.0/24". Put all your 1500 MTU'd clients in the bottom half the range. Then on your 9k mtu servers you set a route 192.0.2.0/25 mtu 1500, 192.0.2.128/25 mtu 9k, so all your servers get the benefits of 9k mtu's while talking to each other but use 1500 byte mtu's while talking to more uh limited clients. Beware you need to profile non 1500 MTU's, many applications are tuned towards 1500ish bytes, and some applications/networking stacks/drivers can end up with poorer performance >1500 than at 1500 byte MTU's. Also beware of MTU "black holes" where a 9k MTU host tries to talk to a 1500 byte host and large packets just get dropped leading to confusing "small emails get through but big ones don't" "I can view googles home page, but not the search results" "I can ssh, and ping, but I can't scp" style issues.
Also beware of MTU "black holes" where a 9k MTU host tries to talk to a 1500 byte host and large packets just get dropped leading to confusing "small emails get through but big ones don't" "I can view googles home page, but not the search results" "I can ssh, and ping, but I can't scp" style issues.
Longer Perry: http://www.wand.net.nz/~mjl12/debugging-pmtud.imc2005.pdf http://www.wand.net.nz/scamper/
On 28/03/2007, at 3:28 PM, Jonny Martin wrote:
Well if you're after increased TCP session throughput, jumbo frames are nowhere near as might seem (refer to Perry's TCP bandwidth calculator), and in this case end to end large MTU support is required.
'jumbo frames are nowhere near _as useful_ as might seem...' It's a bad typing day for me! NZNOG 1, Jonny 0.
Jumbo frames aside from the usefulness of bigger payload for mpls/tunnels etc add some value in an environment where there's some packet loss. For example in an environment with say a 1Gbps link where packet loss is .1% with 1500 byte MTU you get 28Mbps througput. With Jumboframes that leaps to approx 162Mbps. Of course bandwidth delay product and default TCP window sizes will catch you well before that. It's probable that your link delay will be greater than 20-30ms before the internetworking is complex or twisty enough so that packet loss is happening (assuming there's a correlation between complex paths and loss (which there usually is). Aside from that, not much. Jamie On Wed, 2007-03-28 at 15:01 +1200, Ian Batterbee wrote:
Has anyone got any real world experience of the usefulness of jumbo frames on ethernet (ie, MTU > 1500 bytes). A number of 1000mbps interfaces support it, but it seems to be that it would only be useful if jumbo frames were supported and enabled on all equipment between the sending NIC and final IP destination, so I'm struggling to see how enabling it in just say the core of a network would be all that helpful.
_______________________________________________ NZNOG mailing list NZNOG(a)list.waikato.ac.nz http://list.waikato.ac.nz/mailman/listinfo/nznog
participants (8)
-
Ian Batterbee
-
jamie baddeley
-
Jonny Martin
-
Jonny Martin
-
Matthew Luckie
-
Nathan Ward
-
Perry Lorier
-
Regan Murphy