[QE-developers] Forces are not exact derivatives of the energy for U_projection_type = 'ortho-atomic'
Timrov Iurii
iurii.timrov at epfl.ch
Fri Sep 18 17:04:39 CEST 2020
Dear All,
Just for reference: Max tested the latest development version of QE for DFT+U with ortho-atomic orbitals to compute forces in TiO2. The agreement between the finite differences and the analytical formulas that are implemented in QE is excellent:
# Difference of total energies 0.243038055000000
# Integral force*displacement 0.243038055033344
# Difference 3.334407749910895E-011
This validates the correctness of the implementation, and this is in agreement with my tests. The problem was in the patch that I sent to Max, but all is good in the development version. QE 6.6 is also fine, but there was another bug which was also fixed in the development version: "Bug in Hubbard forces and stress for bands parallelization (when nproc_pool>nbnd)".
Greetings,
Iurii
--
Dr. Iurii TIMROV
Postdoctoral Researcher
STI - IMX - THEOS and NCCR - MARVEL
Swiss Federal Institute of Technology Lausanne (EPFL)
CH-1015 Lausanne, Switzerland
+41 21 69 34 881
http://people.epfl.ch/265334
________________________________
From: developers <developers-bounces at lists.quantum-espresso.org> on behalf of Timrov Iurii <iurii.timrov at epfl.ch>
Sent: Sunday, September 13, 2020 10:58:24 AM
To: General discussion list for Quantum ESPRESSO developers
Subject: Re: [QE-developers] Forces are not exact derivatives of the energy for U_projection_type = 'ortho-atomic'
Dear Max,
> Is the relative error of the same magnitude as the ones form “atomic” or no U at all?
Yes
> What you are suggesting is the central difference. For a particular structure this might give the correct force, but if you do the integral for a completely arbitrary path (and many of them) you can uncover particular configurations where the forces are wrong (e.g., when atoms cross the periodic boundaries), or you accumulate small errors in the forces that show up as significant deviations from the energy difference. On several occasions, I have discovered in this way several bugs in my own implementations of forces that were not easily detectable, and I found that this kind of test is rather robust.
Ok, interesting. I did not know about that.
> I am using the new version with your patch. I am running it on daint, compiled with Intel version 19.1.1.217 and 2020.1.217 MKL libraries.
Ok, thanks for the details. Concerning the patch, it contains three modified routines: initially I uploaded to Google Drive only two routines and one hour latter I realised that I forgot that there is yet another routine, which eventually I also uploaded. I hope you used three new routines and recompiled the code. To be safe, you can try to use the latest development version of QE:
HTTPS: git clone https://gitlab.com/QEF/q-e.git
<https://gitlab.com/QEF/q-e.git>or
SSH: git clone git at gitlab.com:QEF/q-e.git
I hope that this was the reason of the problem. If yes, then sorry about the confusion with the patch.
> I can increase the cutoff and rerun the test. But I am already using very tight parameters (100.00 Ry). I could increase the k-point mesh (which is not fully converged), but then again I would expect that I would see larger errors for No-U and “atomic”.
Yes, indeed the cutoff is already very large. I do not expect that increasing the number of k points will solve the problem, but maybe it is worth to check.
Another test that we can do is to use NCPP instead of USPP, to see if the problem is related to USPP.
But first of all I would make a test with the latest development version (instead of using a patch).
> I can recompile with PGI and see if I see a change in behavior.
Thanks. But to be honest I am not sure that this will solve the problem (even though I proposed to check this).
> Yes, I will perform the test with NiO. Please send me your input files.
Thanks! I will send to you the data and the draft of the manuscript privately in a separate email.
> I have attached the atomic positions for every displacement along the path in ascii format. The PW parameters are given below:
Thanks! I will run some tests tomorrow. A few observations:
> nosym = .true. ,
Is there a reason to disable symmetries? I would keep them (in all my tests I used symmetries).
> conv_thr = 1.0d-10 ,
In my tests I used 1.0d-15 to have total energies and forces very well converged. I guess that in your tests with 500 total energy calculations it is better to use 1.0d-15, because otherwise numerical noise will be accumulated (that's my guess).
> I see. Perhaps it would be helpful to include this kind of pathintegral test to the test set, and I am happy to help implement it in the test suite.
Thanks! The only problem is that the path integral test would require several hundreds of total energy calculations, which is huge. We are trying to keep the test suite as light as possible.
> Thanks again for looking into this issue
Thanks a lot to you too!
Best,
Iurii
--
Dr. Iurii TIMROV
Postdoctoral Researcher
STI - IMX - THEOS and NCCR - MARVEL
Swiss Federal Institute of Technology Lausanne (EPFL)
CH-1015 Lausanne, Switzerland
+41 21 69 34 881
http://people.epfl.ch/265334
________________________________
From: developers <developers-bounces at lists.quantum-espresso.org> on behalf of Max Amsler <amsler.max at gmail.com>
Sent: Saturday, September 12, 2020 9:36:50 PM
To: General discussion list for Quantum ESPRESSO developers
Subject: Re: [QE-developers] Forces are not exact derivatives of the energy for U_projection_type = 'ortho-atomic'
Dear Iurii,
Thanks for your response. Actually, I made a mistake in my previous mail. The energies are in eV, not Ry! But the relative errors are correct.
On 12 Sep 2020, at 15:32, Timrov Iurii <iurii.timrov at epfl.ch<mailto:iurii.timrov at epfl.ch>> wrote:
Dear Max,
Thanks for performing these tests!
I implemented Hubbard forces and stress with ortho-atomic orbitals, and I have thoroughly tested the implementation. I compared forces and stress computed using the exact analytical formulas versus finite differences, and the agreement was up to 10^{-6} Ry/Bohr which validates the implementation. I used the symmetric difference quotient (with the displacement parameter of 5 x 10^{-3} Bohr) to computed derivatives numerically:
https://en.wikipedia.org/wiki/Numerical_differentiation
For example, for NiO I obtained the following results:
Analytical: F = -0.048891 Ry/Bohr
Numerical: F = -0.048891 Ry/Bohr
As you can see forces agree up to 10^{-6} Ry/Bohr. In a similar way I tested also stress.
Is the relative error of the same magnitude as the ones form “atomic” or no U at all?
> It seems that there is a bug in the implementation of the forces for U_projection_type = 'ortho-atomic’, or that some terms have been neglected when computing the derivatives.
The analytical formulas are exact, we did not neglect anything. In fact, it took us a lot of time and effort to derive the exact formulas. Also I do not think that there is a bug, because my tests give excellent agreement between analytical and numerical derivatives for NiO and a couple of other systems.
The path integral method looks to me a bit overkilling. Why not simply to compute the total energy for two atomic configurations and then simply divide the difference of energies by the difference in atomic positions (I mean only one atom must be slightly displaced at a time), i.e. [ E(x + delta) - E(x - delta) ] / (2*delta)? But in any case, since your method works well for standard DFT forces and DFT+U atomic forces, then we should try to understand why it gives weird results for the ortho-atomic case.
What you are suggesting is the central difference. For a particular structure this might give the correct force, but if you do the integral for a completely arbitrary path (and many of them) you can uncover particular configurations where the forces are wrong (e.g., when atoms cross the periodic boundaries), or you accumulate small errors in the forces that show up as significant deviations from the energy difference. On several occasions, I have discovered in this way several bugs in my own implementations of forces that were not easily detectable, and I found that this kind of test is rather robust.
Which version of Quantum ESPRESSO did you use? Was it 6.6 without any modifications, or did you apply the patch which I sent to you in the QE developers mailing list shortly after the release of QE 6.6? Are you running on your workstation or on Piz Daint (should not matter but good to know)? Which compilers and libraries do you use?
I am using the new version with your patch. I am running it on daint, compiled with Intel version 19.1.1.217 and 2020.1.217 MKL libraries.
Perhaps, you can try to increase the kinetic-energy cutoff and see if the agreement for the ortho-atomic case is better. Forces and stress converge slowly with respect to the cutoff (and maybe ortho-atomic case is even slower, but in my tests I did not observe this though).
I can increase the cutoff and rerun the test. But I am already using very tight parameters (100.00 Ry). I could increase the k-point mesh (which is not fully converged), but then again I would expect that I would see larger errors for No-U and “atomic”.
Also, maybe it would useful to try other compilers and libraries.
I can recompile with PGI and see if I see a change in behavior.
If you want I can send you my input files for NiO which I used for the benchmarking and we can see what do you obtain using the path integral approach. Would you be so kind to perform these tests?
Yes, I will perform the test with NiO. Please send me your input files.
On my side I can compute forces with finite differences for your system - could you send to me please the input and output files for reference?
I have attached the atomic positions for every displacement along the path in ascii format. The PW parameters are given below:
&CONTROL
restart_mode = 'from_scratch' ,
outdir='./' ,
pseudo_dir = './' ,
calculation = "scf" ,
tstress = .true. ,
tprnfor = .true. ,
/
&SYSTEM
U_projection_type = 'ortho-atomic'
ibrav= 0,
nat= 6,
ntyp = 2
nosym=.false.
ecutwfc = 100.0 ,
ecutrho = 1200.0 ,
occupations = 'smearing' ,
smearing = 'gaussian' ,
degauss = 0.01 ,
lda_plus_u=.true.
lda_plus_u_kind = 0,
Hubbard_U(1) = 4.7147
nat = 6 ,
ntyp = 2 ,
nosym = .true. ,
/
&ELECTRONS
conv_thr = 1.0d-10 ,
/
ATOMIC_SPECIES
Ti 2.00000 Ti.PSP
O 2.00000 O.PSP
K_POINTS automatic
4 4 6 0 0 0
> In the long run, it might be helpful to include such tests routinely as part of the QE development process.
The QE developers team makes big effort to benchmark various functionalities of QE before the release and during the development process. In particular, we have a test suite with many examples which are used to benchmark various codes and various functionalities. For the DFT+Hubbard ortho-atomic forces, I tested everything using finite differences (as described above), I was convinced that all is good, and only then I added tests to the test suite. So we do care a lot about testing and benchmarking, but of course sometimes there maybe be some bugs which were overlooked or which comes for some other reason.
I see. Perhaps it would be helpful to include this kind of pathintegral test to the test set, and I am happy to help implement it in the test suite.
Greetings,
Iurii
Thanks again for looking into this issue,
Best
Max
--
Dr. Iurii TIMROV
Postdoctoral Researcher
STI - IMX - THEOS and NCCR - MARVEL
Swiss Federal Institute of Technology Lausanne (EPFL)
CH-1015 Lausanne, Switzerland
+41 21 69 34 881
http://people.epfl.ch/265334
________________________________
From: developers <developers-bounces at lists.quantum-espresso.org<mailto:developers-bounces at lists.quantum-espresso.org>> on behalf of Max Amsler <amsler.max at gmail.com<mailto:amsler.max at gmail.com>>
Sent: Saturday, September 12, 2020 12:44:33 PM
To: General discussion list for Quantum ESPRESSO developers
Subject: [QE-developers] Forces are not exact derivatives of the energy for U_projection_type = 'ortho-atomic'
Dear all,
After extensively testing the recently implemented forces for U_projection_type = 'ortho-atomic’, I have come across several cases where the geometry optimizations would fail: the energy would fluctuate without convergence using BFGS, and even damped dynamics would frequently not converge. Such a behavior usually points to a problem with the forces, which are either not exact or noisy.
Based on this observation, I decided to perform a routine test: I compute the integral of the forces times displacement along a path in configurational space, and compare it with the energy difference between the initial and final point. If the forces are exact derivatives of the energy, the path integral must be identical to the energy difference (up to machine precision if the step size of the path integral goes to zero). The method is described in sec 3.3, https://doi.org/10.1016/j.cpc.2020.107415, see also the attachment of the relevant section.
I performed this test for a TiO2 Rutile structure with 6 atoms per cellI, along a path in a random direction with 0.001 Ang step size and with a total of 501 steps. I used 3 different settings: DFT without U, DFT+U with U_projection_type = ‘atomic’ (U=4.7147), and DFT+U with U_projection_type = 'ortho-atomic’ (U=4.7147). In the case of No-U and ‘atomic’, the relative error of the path integral and energy difference is less than1.d-4%, as expected when the forces are correct. But for 'ortho-atomic’ the relative error is a whopping 8%! This indicates that the forces are not the exact derivatives of the energy for U_projection_type = 'ortho-atomic’ (see also plot below, note that the y scale is logarithmic for the last two subplots).
It seems that there is a bug in the implementation of the forces for U_projection_type = 'ortho-atomic’, or that some terms have been neglected when computing the derivatives.
In the long run, it might be helpful to include such tests routinely as part of the QE development process.
Best
Max
_______________________________________________
developers mailing list
developers at lists.quantum-espresso.org<mailto:developers at lists.quantum-espresso.org>
https://lists.quantum-espresso.org/mailman/listinfo/developers
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.quantum-espresso.org/pipermail/developers/attachments/20200918/487ed313/attachment-0001.html>
More information about the developers
mailing list