[QE-users] QE Performance on cluster and AMD

[Lorenzo Paulatto]

Hello,
on most Linux systems there is a virtual ram-disk mounted in /dev/shm 
which is read/writeable to all users. If it is not there, just run the 
command "mount|grep ^tmpfs" to find its location. I'm not sure it is a 
good idea, or appreciated by the system administrators to use it for 
your purpose, but it should not break anything to try.

cheers

On 4/27/20 7:28 PM, Baer, Bradly wrote:
> Paolo,
> 
> I believe there is the capability of creating a local scratch folder 
> that is deleted on the end of the job.  I will try that as a solution as 
> I don't think I am yet brave enough to go under the hood of QE.
> 
> As far as the Ryzen performance, I used the same open source libraries 
> for both systems (OpenMPI, OpenBLAS, FFTW3).  I don't know how to 
> identify if I have a conflict between MPI and multi threading, but I 
> will look into using Intel's MKL and see if that solves the performance 
> issues.  I will also keep searching; the mail you referenced reminded me 
> that there is likely more interest in the performance of epyc rather 
> than ryzen.
> 
> Thanks for the advice.
> 
> -Brad
> 
> --------------------------------------------------------
> Bradly Baer
> Graduate Research Assistant, Walker Lab
> Interdisciplinary Materials Science
> Vanderbilt University
> 
> 
> ------------------------------------------------------------------------
> *From:* users <users-bounces at lists.quantum-espresso.org> on behalf of 
> Paolo Giannozzi <p.giannozzi at gmail.com>
> *Sent:* Saturday, April 25, 2020 10:30 AM
> *To:* Quantum ESPRESSO users Forum <users at lists.quantum-espresso.org>
> *Subject:* Re: [QE-users] QE Performance on cluster and AMD
> On Sat, Apr 25, 2020 at 1:34 AM Baer, Bradly 
> <bradly.b.baer at vanderbilt.edu <mailto:bradly.b.baer at vanderbilt.edu>> wrote:
> 
>     My submitted jobs for phonon calculations keep being canceled
>     because QE is writing large files to the networked filesystem which
>     is both very slow and under a fairly tight quota.
> 
> 
> don't you have anything else to use for scratch files? in the worst-case 
> scenario, a local /tmp or /scratch file system, even if not visible to 
> all processors, may do the job (with some caveats)
> 
>        Files in the _ph0  directory, and appear to be creating one file
>     per processor that I request for my job. Is there some way to have
>     QE load these files in to RAM instead
> 
> 
> there is some way, but it requires some work and some understanding of 
> how the "buffer" machinery used in the self-consistent code works. If 
> you feel brave enough to work with the latest development version and to 
> perform some tests, I can explain what to do.
> 
>     A second question: has anyone had success in using QE on AMD Ryzen
>     processors?  I have a Ryzen system that is struggling to match the
>     performance of an Intel system that is several years older. Both are
>     running QE6.5 on Ubuntu.  From pw.x output, cegterg is 10x slower
>     and cdiaghg is 100x slower on the AMD machine.   I had hoped that
>     going from 4 cores to 16 cores would make it faster, not slower!
> 
> 
> 100x means that something VERY wrong is happening. Are you sure there 
> isn't some conflict between MPI and multithreading? Please also see the 
> following message, that contains an important piece of information:
> https://www.mail-archive.com/users@lists.quantum-espresso.org/msg37148.html 
> <https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.mail-archive.com%2Fusers%40lists.quantum-espresso.org%2Fmsg37148.html&data=02%7C01%7Cbradly.b.baer%40vanderbilt.edu%7Ca98d116614954748e48f08d7e92da395%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C0%7C637234254548330786&sdata=Y%2FcpGdYuEIi2mb5yJBhvBFM4e1RxSTAkebnAZztzCuY%3D&reserved=0>
> 
> Paolo
> 
> 
> 
>     Thanks,
>     Bradly Baer
>     --------------------------------------------------------
>     Bradly Baer
>     Graduate Research Assistant, Walker Lab
>     Interdisciplinary Materials Science
>     Vanderbilt University
> 
> 
>     _________________
>     Quantum ESPRESSO is supported by MaX
>     (www.max-centre.eu/quantum-espresso
>     <https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.max-centre.eu%2Fquantum-espresso&data=02%7C01%7Cbradly.b.baer%40vanderbilt.edu%7Ca98d116614954748e48f08d7e92da395%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C0%7C637234254548340782&sdata=gYD0LRnmRcyRE7ne4lowIVxzc5aNalQxdlI7RRvxFM8%3D&reserved=0>)
>     users mailing list users at lists.quantum-espresso.org
>     <mailto:users at lists.quantum-espresso.org>
>     https://lists.quantum-espresso.org/mailman/listinfo/users
>     <https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.quantum-espresso.org%2Fmailman%2Flistinfo%2Fusers&data=02%7C01%7Cbradly.b.baer%40vanderbilt.edu%7Ca98d116614954748e48f08d7e92da395%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C0%7C637234254548340782&sdata=BlSAR6HXWIHijzqS1BZQ%2FRW%2BXPAmKUzJ14IvSA%2FbgBg%3D&reserved=0>
> 
> 
> 
> -- 
> Paolo Giannozzi, Dip. Scienze Matematiche Informatiche e Fisiche,
> Univ. Udine, via delle Scienze 208, 33100 Udine, Italy
> Phone +39-0432-558216, fax +39-0432-558222
> 
> 
> _______________________________________________
> Quantum ESPRESSO is supported by MaX (www.max-centre.eu/quantum-espresso)
> users mailing list users at lists.quantum-espresso.org
> https://lists.quantum-espresso.org/mailman/listinfo/users
> 

-- 
Lorenzo Paulatto - Paris