Jonny Martin wrote:
On 28/03/2007, at 3:01 PM, Ian Batterbee wrote:
Has anyone got any real world experience of the usefulness of jumbo frames on ethernet (ie, MTU > 1500 bytes). A number of 1000mbps interfaces support it, but it seems to be that it would only be useful if jumbo frames were supported and enabled on all equipment between the sending NIC and final IP destination, so I'm struggling to see how enabling it in just say the core of a network would be all that helpful.
Well if you're after increased TCP session throughput, jumbo frames are nowhere near as might seem (refer to Perry's TCP bandwidth calculator), and in this case end to end large MTU support is required.
What my calculator doesn't really show very well is that larger MTUs means you accelerate through slowstart faster. Your top speed is still the same (you're constrained by bandwidth), but you get there earlier. My calculator models "steady state" and ignores slow start. If you have non congestion based loss, you want to get back to sending at full speed as fast as possible, which is where having a larger MTU helps. Then of course theres usually the fact that your per packet overheads go down with MTU increases too. Most OS's let you specify an MTU of 9k on your interface, but then set routes out that interface with a lower MTU on the route. If you're careful you can have your L2 use "192.0.2.0/24". Put all your 1500 MTU'd clients in the bottom half the range. Then on your 9k mtu servers you set a route 192.0.2.0/25 mtu 1500, 192.0.2.128/25 mtu 9k, so all your servers get the benefits of 9k mtu's while talking to each other but use 1500 byte mtu's while talking to more uh limited clients. Beware you need to profile non 1500 MTU's, many applications are tuned towards 1500ish bytes, and some applications/networking stacks/drivers can end up with poorer performance >1500 than at 1500 byte MTU's. Also beware of MTU "black holes" where a 9k MTU host tries to talk to a 1500 byte host and large packets just get dropped leading to confusing "small emails get through but big ones don't" "I can view googles home page, but not the search results" "I can ssh, and ping, but I can't scp" style issues.