Exact exchange-correlation potentials from ground-state electron densities
In density functional theory (DFT), the exchange-correlation potentials provide a computationally tractable means to encapsulate the quantum many-electron interactions, without resorting to the complicated many-electron wavefunctions.
The importance of density functional theory (DFT) in quantum chemistry and materials science can scarcely be overstated. For over four decades, it remains the most widely used method for describing electronic states in all manner of nanoscale phenomena, including chemical bonds in molecules, band structures of materials, electron transfer, and reactive metal clusters of proteins. Its prowess lies in providing a formally exact description of the immensely complicated interacting many-electron system in terms of an auxiliary system of non-interacting electrons, called the Kohn-Sham system. In simpler terms, DFT allows us to deal with just single-electron orbitals that dwell in our familiar 3D space, instead of being boggled by the complicated higher-dimensional many-electron wavefunctions. At the heart of this simplification lies the exchange-correlation (XC) potential, which encapsulates the quantum many-electron interactions. In other words, in the Kohn-Sham world, the electrons do not talk to each other directly, but through the XC potential. The XC potential, in turn, is known to be a unique functional of the electron density, thus making DFT, in principle, an exact theory. However, in practice, DFT has remained far from exact due to the unavailability of exact XC potentials, thereby, necessitating the use of approximations. Naturally, the development of accurate XC functionals has, for long, remained the grand challenge in DFT.
This work stems from the above challenge of finding accurate XC functionals. It was in the Fall of 2015 when Vikram Gavini discussed with me about a data-driven approach to constructing exchange-correlation functionals, an idea that goes beyond the conventional approach in the field. The broad idea is to use accurate ground-state densities, from wavefunction-based methods (e.g., quantum Monte-Carlo, full configuration Interaction). The wavefunction-based methods, owing to their exponential computational complexity, remain untenable beyond a few tens of electrons. However, one can utilize the accurate ground-state densities from these small systems that are within the reach of the wavefunction-based methods to develop a machine-learned model for the functional dependence of the XC potential on the density. This involves two distinct steps: (i) generating pairs of density and its corresponding XC potential (both as spatial fields), using densities from wavefunction-based calculations, and (ii) using these density-potential pairs as training data to model the functional dependence of the XC potential on the density, through a machine-learning algorithm. This work addresses the first of the above two challenges of generating the training data. That is, given an accurate ground-state density, we are interested in evaluating the XC potential (as a spatial field) that yields the same density. This is otherwise known as the inverse DFT problem.
We began the work, in earnest, in the beginning of 2017. Although, theoretically, simple to formulate, the inverse DFT problem had remained, heretofore, unresolved because of the numerical challenges associated with it. The first challenge stemmed from the basis used to discretize the problem. Many of the past efforts have suffered from spurious oscillations in the XC potential or from non-unique solutions, largely, owing to the use of incomplete basis to discretize the problem. To put it differently, while the continuous problem is well-posed, the discrete problem can turn ill-posed, if discretized in an incomplete basis. Fortunately, we were cognizant of this challenge from the very beginning, and hence, consciously made the choice of employing the finite-element basis—a systematically improvable and complete basis. We were quickly able to establish the efficacy of the finite-element basis through our verification studies using densities obtained from DFT calculations with a known XC approximation (say local density approximation (LDA)). This verification test allowed for a direct assessment of the accuracy of the XC potential obtained using inversion, by comparing it against the LDA XC potential. As was expected, the use of finite element basis resulted in accurate XC potentials devoid of any spurious oscillations.
With the initial verification established, the next logical step was to evaluate the exact XC potential from correlated ab initio densities (i.e., densities obtained from wavefunction-based methods). Needless to say, it required us to obtain accurate ground-state densities using wavefunction-based methods. To that end, we were fortunate to avail the expertise of Paul Zimmerman. Paul’s incremental full-configuration interaction (iFCI) approach remained vital in efficiently supplying high quality ground-state densities. With these correlated densities at our disposal, it seemed straightforward to extend our finite element based inversion approach to obtain their corresponding exact XC potentials. Contrary to our naive expectations, the XC potentials we obtained were fraught with wild unphysical oscillations. These oscillations, as we learnt the hard way, were the manifestations of the incorrect asymptotics in the Gaussian basis-set densities that were obtained from iFCI calculations. To elaborate, from a practical viewpoint, Gaussian basis remains the only viable means, at present, to obtain all-electron correlated ab initio densities from any CI based calculations. However, it is known that Gaussian basis-set densities, by construction, lack the cusp at the nuclei as well as have incorrect far-field decay. Although seemingly insignificant, these incorrect asymptotics induce wild oscillations in the XC potentials obtained via any inversion procedure. An unambiguous fix to this problem would be to conduct iFCI calculations in a basis that honors the desired asymptotics. However, the use of any such currently available basis remains computationally unviable for iFCI. Thus, we were required to strategize numerical approaches to remedy the artifacts of Gaussian densities. To that end, we developed two crucial methods—(i) to add a small numerical correction to the Gaussian density to correct for the missing cusp at the nuclei, and (ii) to use approximate far-field boundary conditions on the XC potential to insulate the inversion from the incorrect far-field decay in the Gaussian densities. The combination of these two numerical strategies, along with the finite element discretization, did the trick of simultaneously addressing all the unresolved issues in inverse DFT. As a consequence, we were able to obtain exact XC potentials for both weakly and strongly correlated polyatomic molecules, ranging up to 40 electrons (large by inverse DFT standards).
With this significant advance of having resolved the inverse DFT challenge, we hope to embark onto the more consequential challenge of using machine learning to model the functional dependence of the XC potential on the density, by utilizing the density-potential pairs obtained from our inverse calculations.
For more details, check our article: Exact exchange-correlation potentials from ground-state electron densities.