[Q-e-developers] QE I/O default

[Ye Luo]

I see.
You are right. Just checked on 6.1. No more scratch files during the run.

The large file issue happened in QE 5.3. Some offsets used to access a
record exceed 32bit int and goes negative.
The 6.1 seems still rely on iotk but the svn repo eliminates iotk for
writing WF. Applause!

Best,
Ye


===================
Ye Luo, Ph.D.
Leadership Computing Facility
Argonne National Laboratory

2017-05-25 10:24 GMT-05:00 Paolo Giannozzi <paolo.giannozzi at uniud.it>:

> Dear Ye
>
> > I noticed yesterday that the wf_collect is set true as default. Probably
> you
> > already had a lot of discussion on your side
>
> not really: having realized that I seldom get useful answers when I
> ask opinions about this or that, I no longer ask, just do this or that
> and wait for complaints.
>
> > Do we have confident that wf_collect will not add significant of time on
> > large simulations?
>
> At this stage, performances are not my major concern: correctness and
> maintainability are. The I/O of QE has become a mess beyond control,
> it needs to be simplified and documented, the old stuff must disappear
> ASAP. Note that the possibility to write "distributed" files is still
> there (at least for PW; for CP it can be easily re-added),
>
> > Is the performance good on both lustre and GPFS file system?
>
> No idea.
>
> > I didn't have much experience with the recent added hdf5 feature.
>
> Nor have I (by the way, it is not yet working with new I/O)
>
> > Does the WF collect use  parallel collective I/O? or like the old fashion
> > collect the WF on the master  and write by it.
>
> For the time being: old fashion. Of course, everybody is welcome to
> implement with "the latest and the greatest", but first of all, we
> need something working and that can be modified without hours of
> reverse engineering.
>
> > Is the performance good? Measured bandwidth?
>
> I hope it will be, but I leave to other people measuring bandwidths.
>
> > On the machines I use, the GPFS has 8 aggregators by default and PIO
> > performance is better than creating individual files. The lustre does the
> > opposite and has only 1 OST by default and thus write sequentially with
> PIO.
> > Writing individual files becomes faster. Of course you can tune both of
> > them, just very tricky.
> >
> > Do QE still create the file per MPI rank from the beginning? 4k empty
> file
> > is a bit slow to create and pain to 'ls'. When I do DFT+U basically the
> > number of files doubles or triples I don't remember exactly.
>
> This story of QE opening too many files is becoming an urban legend
> like "QE is 10 times slower than VASP". The default setting, since
> some time, is that QE doesn't open anything while running.
>
> > PS: In the past, I had the experience that QE was not able to ready its
> own
> > collected WF when the record (using IOTK) is very large >100GB.
>
> this doesn't look like a QE problem, though, unless it is a iotk
> problem. IOTK is no longer used to write binaries by the way.
>
> Paolo
>
> > Not collecting the WF was the preferred way for me. It should not be a
> problem
> > with hdf5 since the dataset is per band and much smaller.
> >
> > Thanks,
> > Ye
> > ===================
> > Ye Luo, Ph.D.
> > Leadership Computing Facility
> > Argonne National Laboratory
> >
> > _______________________________________________
> > Q-e-developers mailing list
> > Q-e-developers at qe-forge.org
> > http://qe-forge.org/mailman/listinfo/q-e-developers
> >
>
>
>
> --
> Paolo Giannozzi, Dip. Scienze Matematiche Informatiche e Fisiche,
> Univ. Udine, via delle Scienze 208, 33100 Udine, Italy
> Phone +39-0432-558216, fax +39-0432-558222
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.quantum-espresso.org/pipermail/developers/attachments/20170525/18b4191d/attachment.html>