Visual evidence of Amazon EC2 network issues

[Sabre]

Article

Interesting to see how Amazon might be hitting a wall.

[schvin]

it's interesting data. i had seen the original, but not this follow-up, thanks.

i'm not sure which east region he is in, but my ping graphs (smokeping) within the ec2 region i have my stuff in (nor my ec2 to slicehost traffic) show any pattern like this, everything has been pretty quiet recently.

so, i am curious to know more about his setup, and kinda (?) thankful that i'm not in the same region. though, i am tempted to move some stuff to the other us east region to see if i can reproduce.

nice pretty graphs, though. will be curious to hear how amazon follows up.

[avriette]

Article

Interesting to see how Amazon might be hitting a wall.

It's not a wall. It's an inflection point wherein they must update their infrastructure. My guess is they're using the same kind of setup to Google. Tons of tons of tons of lame-ish whitebox pc's. Standardized, even. The problem is I/O is the hardest thing to do on a computer these days (because the cpu's are all plenty fast). There are any number of ways to increase I/O performance, and who am I to second guess them? I know what I'd do, but it wouldn't be cheap, and since they have essentially a legacy environment they need to play nice with, I can't guess at the internals. I'd seriously be looking at 10/100 gig E, and high-speed infiniband, disks with larger caches, procs with larger caches, or even a true realtime setup. (or maybe even a real RTOS... does anyone make things like the NonStop or a Stratus that can keep up with the machines? The thing is, with a mainframe you get giant performance, and they could be used to triage data before being "chunked" like Google's stuff. They're big, and cost their weight in gold, but you might be able to add a single row (six racks or so) per POP where necessary for traffic. (edit: smaller POPs might use memcache clusters or non-relational databases for lookups)

Computing will always run up against "walls." The good news is we know the maximum fidelity we can use via maths. Since the concepts are largely physical rather than logical, it's easy (as in, obviously possible, although not necessarily obtainable RFN) to develop something that works well on your wall. Think of intelligence as a siege engine hammering on Hard Problems. The bigass mainframes and their ilk (mostly ibm today meh) might be your Mehmet II mega-cannon.

Doesn't Seitz work at Amazon? Is he still around?

[Sabre]

Doesn't Seitz work at Amazon? Is he still around?

Not around DCAWD anymore since he sold his STI, but I believe he did mention moving from AOL -> Amazon. It would make sense for him to be part of their EC2 cloud services.

[avriette]

[quote=]Not around DCAWD anymore since he sold his STI, but I believe he did mention moving from AOL -> Amazon. It would make sense for him to be part of their EC2 cloud services.[/quote]

Yeah, we did essentially that when we worked at AOL in 2004-2005. Most of that entire group (maybe twenty people even) has moved to Comcast. I don't know much about the "move," though.

[Mr Kleen]

if anyone sees that guy on a regular basis, I still owe him $20.