[QE-users] Parallelization and Input/Output of wfc in QE

[Riccardo Piombo uniroma1]

Dear Comunity,

I modified the local_dos.f90 file (see attached file) stored in the 
PP/src folder to download the wavefunctions of a system I'm currently 
studying.

In particular, I'm running QE on a CINECA supercomputer because my 
system contains more than 600 atoms, and I need to run my computations 
on several nodes.

My minor modification works well, but I'm facing some problems regarding 
the parallelization during the I/O.

For example, If I ask my modified pp.x to write down a single wfc 
computed at gamma at fixed band index (say n=1), I obtain a file with 
551981 real and imaginary parts of the coefficients of the Plane waves 
expansion stored.

However, it takes more than 4 minutes to accomplish this task using just 
1 node, and since the default number of bands is 5747, it takes about 
22988 minutes = 383 hours = 15 days to download all the wfcs.

Therefore, I decided to run pp.x on multiple nodes. I realized the 
number of coefficients saved in the various files is halved every time 
the number of nodes is doubled, and the same also happens when the 
values of n-tasks-per-node and CPU-per-tasks are not 1 and 48, 
respectively (each CINECA node contains 2 CPU with 24 cores each).

Since I don't know how the I/O is parallelized in QE, could someone 
suggest how to modify the file so I can use more nodes and avoid that 
part of the files is not overwritten?

The only parts of local_dos.f90 that I have modified are the ones 
between the following commented lines

/
/

/!##################################################################//
//!!!NEW: print wave functions in G space and corresponding G vectors//
/

/./

/./

/./

/!!!NEW: print wave functions in G space and corresponding G vectors/

/!----------------------------------------------------------------------------------------------------------/


So each modification begins with multiple hashes and ends with numerous 
dashes.

My concern is that the WRITE statements I have used are not the right 
ones when performing computations over several nodes.

Do I have to open multiple files, one for each node involved?


Best regards,

Riccardo Piombo

Post doc researcher in Condensed Matter Physics at Sapienza University 
of Rome



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.quantum-espresso.org/pipermail/users/attachments/20221123/2d988d57/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: local_dos.f90
Type: text/x-fortran
Size: 22327 bytes
Desc: not available
URL: <http://lists.quantum-espresso.org/pipermail/users/attachments/20221123/2d988d57/attachment.bin>