Re: Roadrunner Supercomputer using 12,960 CELL Processors Hits 1

On Jun 12, 3:43 am, Robert Myers <rbmyers...@[EMAIL PROTECTED]
> wrote:
> On Jun 11, 8:03 pm, already5cho...@[EMAIL PROTECTED]
 wrote:
>
>
>
> > On Jun 11, 6:10 pm, Robert Myers <rbmyers...@[EMAIL PROTECTED]
> wrote:
>
> > > On Jun 11, 5:22 am, already5cho...@[EMAIL PROTECTED]
 wrote:
>
> > > > I'd agree with mr deo.
> > > > Roadrunner interconnects look like a big step backward from other
PR-
> > > > heavy American supercomputers.
>
> >
>http://www.lanl.gov/orgs/hpc/roadrunner/pdfs/Kerbyson%20-%20RR%20Mode...
>
> > > The predicted worst-case latency is about the same as Blue-Gene. 
Red
> > > Storm routing/switching looks like Blue Gene.  Columbia uses both
> > > Infiniband and Numalink in a fat tree like Roadrunner.
>
> > My understanding of p.10 is that 2-way latency between SPEs on two
> > neighbor triblades is ~8 usec i.e. about the same as node-to-node
> > latency in the whole 65K-node machine.
> > I didn't find any estimates for worst-case latency in the 10K-node
> > configuration. My personal uneducated guess - 10 times worse than BG/L
> > in the worst case and 5 time worse in average loaded case.
>
> I took the "worst case 2-way Infiniband" latency to be the worst case
> for the mesh fabric.

I read it as a latency within triblade that does not include fabric.

>  The advertised worst case for one version of
> Blue Gene was, I think, 5 microseconds.

Full original version was closer to 9  microseconds.  Since the
currently the machine is almost twice bigger than it was back then I'd
guess that today they are at 10 microseconds.

> The latency between the
> Opteron and the Cell Processor is another matter.  In any case, it has
> nothing to do with the mesh fabric.
>

That's the point. I saw nothing about latency/bandwidth
characteristics of the IB switches used in the roadrunner.

> BG/L will do fine on some kinds of problems.  Just not ones requiring
> significant global communication.  That was my beef about Blue Gene
> and Red Storm.
>

It depends on the kind of communication.
If the communication consist mostly of small bi-directional messages
these machines seem to be much better than anything else in existence.
For the large bandwidth-bound messages on BG/L the picture is less
rosy. I didn't see numbers for Crays XTn of comparable size (not sure
they exist); in theory they should be significantly better than BG/L.
However, I do not see why the roadrunner should be any better for
bandwidth-bound global communication. If anything, I expect it to do
worse, esp. in the worst case.

> Robert.