[Pw_forum] Problem with QE 4.2.1 and AMD Opteron 6200 / 6300

[Fabricio Cannini]

Hi All


I'm facing a very strange situation with the above version of QE, 
compiled with :

- intel 12.1.3
- mkl 10.0,
- fftw 3.2.2 ( OS package )
- openmpi 1.4.5

Running 'prace-medium' benchmark as a test .
http://qe-forge.org/gf/project/q-e/frs/?action=FrsReleaseView&release_id=47

The OS is centos 5.6 x86-64 .

And the results were :

- Intel Xeon E5430 2.66GHz / 8 cores / 16 GB RAM = 00h:38m:21s

- Intel Core i7-2600 @ 3.40GHz / 8 cores / 16 GB = 00h:19m:40s

- AMD Opteron Processor 6276 / 8 cores / 256 GB = more than 8h ( process 
killed )



Then i tried another compilation :

- pgi 12.5
- acml 5.1.0 64
- fftw 3.2.2 ( OS package )
- openmpi 1.4.5

And the results were even worse . None of the machines above were able 
to finish the test in *24h* .




My third attempt was the following :

- intel 13.2
- mkl 11.0
- fftw 3.3.3 ( OS package )
- openmpi 1.6.5

- OS = Ubuntu 12.04 LTS

- AMD Opteron 6380 / 8 cores / 64 GB RAM

- Same "prace medium benchmark" test input.


Result : Also didn't finish in *24h* .




I was suspicious of the intel compiler , so I setup a 4th test :

- gfortran 4.6 ( OS package )
- openblas 0.2.8 ( compiled with gcc 4.6 )
- fftw 3.3.3 ( OS package )
- openmpi 1.4.3 ( OS package )


Same machine as the third test, and the result was the same too, with a 
difference that the binary compiled with gfortran used *much more* 
memory , running into as much as 15GB of swap memory , before i kill the 
process ( it took some 30 min to reach this point )

It should be noted that when running the 'small size' benchmark on the 
Opteron 6380 machine, the gfortran/openblas binary is faster than the 
intel 13.2/mkl binary ( up to a minute on the 3rd and 4th test ) .


Do you have a clue of what could be happening ?

Should i attach the 'make.sys' files to another message or paste it 
somewhere ?

TIA
Fabricio