[QE-users] Optimal pw command line for large systems and only Gamma point

[Giuseppe Mattioli]

Dear Antonio
Before struggling with parallelization setup, I see that

>      Estimated total dynamical RAM >    1900.41 GB

your calculation requires more or less up to 2TB RAM. I can't see your  
setup (e.g., the supercell containing your cluster) but I suggest that  
you ask yourself if there is something in your setup that can be  
reduced. Moreover, You need at least 10 nodes because of the RAM  
requirements, but you likely don't need 1280 processes. You should try  
to heavily reduce them using 10 (or more) nodes. Have you tried to  
launch pw.x without any indication of parallelization distribution,  
the simpler the better?
HTH
Giuseppe

Quoting Antonio Cammarata via users <users at lists.quantum-espresso.org>:

> Dear all,
>
> I have a silicon nanocluster with 1000 atoms with 1 1 1 k-mesh (only  
> Gamma point). I cannot manage to run the calculation due to memory  
> issue. I use a computational cluster with 128 core/node and 200 GB  
> RAM per node. I am using PWSCF v.7.3. In the input I set ecutwfc= 29  
> and cg diagonalization to save memory. According to
>
> https://www.quantum-espresso.org/Doc/user_guide/node20.html
>
> I tried several command line parameters, the last being
>
> pw.x -nk 1 -nt 1 -nb 1 -nd 768 -inp qe.in > qe.out
>
> for a run on 6 nodes. I tried up to 12 nodes but after 7 nodes I get  
> the warning message "some processors have no G-vectors for  
> symmetrization". Here some info that may be relevant for the issue
>
>      Dense  grid: 30754065 G-vectors     FFT dimensions: ( 400, 400, 400)
>      Dynamical RAM for                 wfc:     153.50 MB
>      Dynamical RAM for     wfc (w. buffer):     153.50 MB
>      Dynamical RAM for           str. fact:       0.61 MB
>      Dynamical RAM for           local pot:       0.00 MB
>      Dynamical RAM for          nlocal pot:    1374.66 MB
>      Dynamical RAM for                qrad:       0.87 MB
>      Dynamical RAM for          rho,v,vnew:       5.50 MB
>      Dynamical RAM for               rhoin:       1.83 MB
>      Dynamical RAM for            rho*nmix:       9.78 MB
>      Dynamical RAM for           G-vectors:       2.60 MB
>      Dynamical RAM for          h,s,v(r/c):       0.25 MB
>      Dynamical RAM for          <psi|beta>:     552.06 MB
>      Dynamical RAM for      wfcinit/wfcrot:     977.20 MB
>      Estimated static dynamical RAM per process >       1.51 GB
>      Estimated max dynamical RAM per process >       2.47 GB
>      Estimated total dynamical RAM >    1900.41 GB
>
> I managed to run the simulation with 512 atoms, cg diagonalization  
> and 3 nodes on the same machine with command line
>
> pw.x -nk 1 -nt 1 -nd 484 -inp qe.in > qe.out
>
> Please, do you have any suggestion on how to set optimal  
> parallelization parameters to avoid the memory issue and run the  
> calculation? I am also planning to run simulations on nanoclusters  
> with more than 1000 atoms.
>
> Thanks a lot in advance for your kind help.
>
> Antonio
>
>
> -- 
> _______________________________________________
> Antonio Cammarata, PhD in Physics
> Associate Professor in Applied Physics
> Advanced Materials Group
> Department of Control Engineering - KN:G-204
> Faculty of Electrical Engineering
> Czech Technical University in Prague
> Karlovo Náměstí, 13
> 121 35, Prague 2, Czech Republic
> Phone: +420 224 35 5711
> Fax:   +420 224 91 8646
> ORCID: orcid.org/0000-0002-5691-0682
> ResercherID: A-4883-2014
>
> _______________________________________________
> The Quantum ESPRESSO community stands by the Ukrainian
> people and expresses its concerns about the devastating
> effects that the Russian military offensive has on their
> country and on the free and peaceful scientific, cultural,
> and economic cooperation amongst peoples
> _______________________________________________
> Quantum ESPRESSO is supported by MaX (www.max-centre.eu)
> users mailing list users at lists.quantum-espresso.org
> https://lists.quantum-espresso.org/mailman/listinfo/users



GIUSEPPE MATTIOLI
CNR - ISTITUTO DI STRUTTURA DELLA MATERIA
Via Salaria Km 29,300 - C.P. 10
I-00015 - Monterotondo Scalo (RM)
Mob (*preferred*) +39 373 7305625
Tel + 39 06 90672342 - Fax +39 06 90672316
E-mail: <giuseppe.mattioli at ism.cnr.it>