[Pw_forum] Performance problems due to files in outdir

[Henning Glawe]

On Wed, May 08, 2013 at 09:48:39AM +0200, Axel Kohlmeyer wrote:
> have you tried using a ramdisk? if your machine is linux based, then
> you can usually access up to half of the physical memory via /dev/shm.

Yes, I tried this and it seems to work, in conjunction with tar to
transfer data to the postprocessing code.

> given the ever increasing amount of RAM that current many-core machine
> have, this has become a viable alternative to modifying the source
> code to keep data in core.

Just keeping the data in the core is also no option, due to the necessity
of data exchange with internal and external postprocessing codes.

> > Are there any plans to move away from the 'many-small-files' paradigm? Or is
> > there any way/switch to make QE use another scheme for the scratch files?
> 
> whatever you do, it will require some programming effort. this can be
> done from inside of QE (tedious) or on the outside by using a ramdisk
> (as mentioned above) or faking it, e.g. by wrapping the stdio library
> to write to huge RAM buffers for all files matching a given pattern.
> cray used to provide such a library on their XT series machines to
> work around the same kind of problems that you are describing on their
> lustre scratch file systems.
> 
> if there are some people here on the list that would be interested in
> having such a wrapper and would be volunteering some time testing and
> providing feedback, i might be tempted to give it a try and write such
> a wrapper.

As far as I understand the code, all IO to outdir is performed via IOTK;
maybe this could be an attack vector to the io problem...
Let's see if I can spare a bit of time to look into this.

-- 
Mit freundlichen Grüßen
Henning Glawe

Max-Planck-Institut für Mikrostrukturphysik
Weinberg 2, 06120 Halle (Saale), Germany http://www.mpi-halle.de/~theory
Phone: +49-345-5582-613 Fax: +49-345-5511223  Email: glawe at mpi-halle.de