[Pw_forum] Large files

Mickael Profeta profeta at sissa.it
Fri Apr 23 17:22:21 CEST 2004


> Mickael,
> 
> Support for files larger than 2Gbytes has been the standard feature of the
> Linux 2.4 kernels for THE LAST TWO YEARS. They work with all filesystems
> capable of supporting large files (at least xfs, ext3, reiser, and nfs3
> - and likely a few more). Any reasonably recent distribution will also
> come with large-file support already enabled in all libraries and
> utilities.

Ok, you are right, and you misunderstand my answer.
I did not want to say that linux was not able to handle file large than 
2Gb, I know it is possible, and recent filesystem are just limited by 
the size of the hard limit of the disk...

Yet it is not totally straightforward, and appllications have to use the 
correct LFS API and to compile with appropriate call to libc and 
equivalent to handle files.

As it is the second time I raise the problem with pw, let's say that 
pwscf, and I fear that it is related with fortran compiler on that 
architecture, does not map the good API and are not able to handle large 
files.

> I certainly have no problem with using files as large at 1.5 TERAbytes
> under linux/x86 (using standard, out-of-the-box Suse installation) -
> with the limit coming not from kernel, but from the size of the RAID
> array we can afford.

may you try to produce a file bigger than 2Gb with pw, and tell us if 
you manage or not? I would really like to know where the limitation 
stands? On the library of the linux distribution (I know the problem 
exist on RedHat 7.3) on the compiler version, or on the way pwscf is 
written.

> Blaming Linux kernels for not being able to handle large files is
> simply not a good excuse anymore.

you are right on that point, but you still have to be carefull in order 
to use these files, it is not as easy as it could be.

Mickael



More information about the users mailing list