[QE-users] Parallelization and Input/Output of wfc in QE

[Paolo Giannozzi]

As a strict minimum, you should write an "unformatted" file, much faster 
than a "formatted" one. I don't understand what you are trying to 
achieve, though. You can read the QE format with a postprocessing code 
(please have a look at pp_example.f90) and do whatever you like with the 
Kohn-Sham orbitals, on as many processors you need, without re-writing 
everything once again.

Paolo

On 23/11/2022 16:11, Riccardo Piombo uniroma1 via users wrote:
> Dear Comunity,
> 
> I modified the local_dos.f90 file (see attached file) stored in the 
> PP/src folder to download the wavefunctions of a system I'm currently 
> studying.
> 
> In particular, I'm running QE on a CINECA supercomputer because my 
> system contains more than 600 atoms, and I need to run my computations 
> on several nodes.
> 
> My minor modification works well, but I'm facing some problems regarding 
> the parallelization during the I/O.
> 
> For example, If I ask my modified pp.x to write down a single wfc 
> computed at gamma at fixed band index (say n=1), I obtain a file with 
> 551981 real and imaginary parts of the coefficients of the Plane waves 
> expansion stored.
> 
> However, it takes more than 4 minutes to accomplish this task using just 
> 1 node, and since the default number of bands is 5747, it takes about 
> 22988 minutes = 383 hours = 15 days to download all the wfcs.
> 
> Therefore, I decided to run pp.x on multiple nodes. I realized the 
> number of coefficients saved in the various files is halved every time 
> the number of nodes is doubled, and the same also happens when the 
> values of n-tasks-per-node and CPU-per-tasks are not 1 and 48, 
> respectively (each CINECA node contains 2 CPU with 24 cores each).
> 
> Since I don't know how the I/O is parallelized in QE, could someone 
> suggest how to modify the file so I can use more nodes and avoid that 
> part of the files is not overwritten?
> 
> The only parts of local_dos.f90 that I have modified are the ones 
> between the following commented lines
> 
> /
> /
> 
> /!##################################################################//
> //!!!NEW: print wave functions in G space and corresponding G vectors//
> /
> 
> /./
> 
> /./
> 
> /./
> 
> /!!!NEW: print wave functions in G space and corresponding G vectors/
> 
> /!----------------------------------------------------------------------------------------------------------/
> 
> 
> So each modification begins with multiple hashes and ends with numerous 
> dashes.
> 
> My concern is that the WRITE statements I have used are not the right 
> ones when performing computations over several nodes.
> 
> Do I have to open multiple files, one for each node involved?
> 
> 
> Best regards,
> 
> Riccardo Piombo
> 
> Post doc researcher in Condensed Matter Physics at Sapienza University 
> of Rome
> 
> 
> 
> 
> 
> _______________________________________________
> The Quantum ESPRESSO community stands by the Ukrainian
> people and expresses its concerns about the devastating
> effects that the Russian military offensive has on their
> country and on the free and peaceful scientific, cultural,
> and economic cooperation amongst peoples
> _______________________________________________
> Quantum ESPRESSO is supported by MaX (www.max-centre.eu)
> users mailing list users at lists.quantum-espresso.org
> https://lists.quantum-espresso.org/mailman/listinfo/users

-- 
Paolo Giannozzi, Dip. Scienze Matematiche Informatiche e Fisiche,
Univ. Udine, via delle Scienze 206, 33100 Udine Italy, +39-0432-558216