Re: HINT: JOHN GUSTAFSON'S LATEST MEASURE OF PERFORMANCE

["M. Edward Borasky (znmeb@xxxxxxxxxxxxxxxx)" <znmeb@xxxxxxxxxxxxxxxx>]

On Nov 16, 20:44, Penio Penev wrote:
> Subject: HINT: JOHN GUSTAFSON'S LATEST MEASURE OF PERFORMANCE
| I thought, that I would post this info here.
|
| A performance measure for high-performance computing known as HINT that
| evaluates accomplishment in terms of QUIPS has been proposed.
|
| The source is John Gustafson of the Department of Energy-supported Ames
| Laboratory at Iowa State University who caused a stir in the
| performance-evaluation community a few years ago with SLALOM.
|
| In a paper co-authored by Quinn O. Snell, also of the Ames Laboratory,
| Gustafson argues that familiar performance measures like SPEC and
| almost-forgotten ones like the PERFECT benchmark neglect a crucial
| consideration: the quality of work done.
|
| This leads to QUIPS, the HINT measure of performance, which stands for
| Quality Improvement Per Second. The paper asserts that this concept is "based
| rigorously on progress toward solving a problem."
|
| The authors have experimented with HINT on various systems available at the
| Ames Laboratory. Results for a Sun 4/370, Silicon Graphics' original Indigo,
| an Indy without secondary cache, an Indy with the cache, a DEC 5000/240, and
| an Indigo2 provide a fascinating glimpse into real capabilities.
|
| HINT can be downloaded over the Internet via anonymous FTP maintained by
| the Scalable Computing Laboratory at Ames. A Mosaic home page is being
| developed. The ftp server is: <ftp.scl.ameslab.gov>. The HINT directory is
| located within /pub. A paper mailed by the authors to about 350 addresses
| (including this reporter) may be found in the doc subdirectory.
|
| I believe might be interesting to some of you (most definitely it is
| interesting to me :-) because MuP21 and F21 might get a very good benchmark.
| And since the authors argue, that the benchmark measures progress towards an
| answer, it might be very useful in real-life situations.
|
| Of course, one has to write and execute the benchmark first, but my
| feeling is, that some unexpectedly favourable result will show up.
|
| Enjoy,
|
| Penio.
I haven't seen mention of this on "comp.benchmarks", although I'm familiar with
SLALOM and Dr. Gustafson's other work in benchmarking.  The thrust of his,
indeed most benchmarking efforts, unfortunately, tends to measure peak floating
point performance.  The large LINPACK, SPECfp and SLALOM are dominated by this
measurement.  I once coined a law to describe this phenomenon: as the time for
purchase approaches, the benchmark performance approaches the peak floating
point performance of the hardware.  Considering our recent discussion of the
lack of math and signal processing capabilities in MuP21, I am of the opinion
that an "unexpectedly favourable result" is highly unlikely.

I don't have FTP access yet; if someone can e-mail me the Postscript for the
paper I would like to look at it.  If I can find Gustafson's e-mail address, I
can get one from him.


-- 
M. Edward Borasky (znmeb@plaza.ds.adp.com)

I support the right to keep and arm bears.