Skip to content

Configuration Interaction

This document provides a clear, simple, and mathematically rigorous explanation of the Configuration Interaction (CI) method in second quantization. Building directly upon the Hartree–Fock reference state and canonical spin-orbitals, this guide explains how many-electron wavefunctions are expanded over excited configurations, how Hamiltonian matrix elements are evaluated using Wick's theorem, and how the variational principle determines correlated ground and excited electronic states.


I. Determinant Space and Second-Quantized Excitations

1. Configuration Expansion and Excitation Hierarchy

While the Hartree–Fock method finds the single Slater determinant that minimizes the mean-field electronic energy, it ignores dynamical and static electron correlation. The Configuration Interaction method provides an exact variational framework to capture this correlation by expanding the many-electron electronic wavefunction \(|\Psi_k\rangle\) for state \(k\) as a linear combination of Slater determinants formed by promoting electrons from occupied orbitals to virtual orbitals as

\[ |\Psi_k\rangle=\sum_n c_{nk}|\Phi_n\rangle=c_{0k}|\Phi_0\rangle+\sum_{i,a}c_i^{a,k}|\Phi_i^a\rangle+\sum_{i<j,a<b}c_{ij}^{ab,k}|\Phi_{ij}^{ab}\rangle+\cdots \]

where \(|\Phi_0\rangle\) is the Hartree–Fock reference state, and \(c_{nk}\) are the linear variational coefficients defining state \(k\). In second quantization, each excited determinant is generated by applying particle creation operators \(a_a^\dagger,a_b^\dagger,\dots\) and hole annihilation operators \(a_i,a_j,\dots\) directly to the reference state, defining single and double excitations as

\[ |\Phi_i^a\rangle=a_a^\dagger a_i|\Phi_0\rangle,\quad|\Phi_{ij}^{ab}\rangle=a_a^\dagger a_b^\dagger a_j a_i|\Phi_0\rangle \]

where canonical index restrictions \(i<j\) and \(a<b\) ensure linear independence among configurations. Truncating this expansion at specific excitation levels defines standard Configuration Interaction models, such as Configuration Interaction Singles (CIS) for electronic excitation spectra, Configuration Interaction Doubles (CID) for electron pair correlation, and Configuration Interaction Singles and Doubles (CISD) for combined ground- and excited-state correlation.

2. Full Configuration Interaction and Combinatorial Scaling

When the expansion includes all possible electron promotions up to the total number of electrons \(N\), the method becomes Full Configuration Interaction (FCI). Because the set of all \(N\)-electron Slater determinants forms a complete basis for the \(N\)-electron Hilbert space spanned by the chosen one-electron orbital basis, FCI yields the exact non-relativistic electronic energy within that basis set.

For a system of \(N\) electrons distributed among \(2M\) spin-orbitals (corresponding to \(M\) spatial molecular orbitals), the total number of determinants in the full expansion is given by the binomial coefficient

\[ N_{\text{det}}=\binom{2M}{N}=\frac{(2M)!}{N!(2M-N)!} \]

When separated into \(N_\alpha\) alpha-spin and \(N_\beta\) beta-spin electrons, the total number of spin-adapted determinants expands as the product of independent binomial factors

\[ N_{\text{det}}=\binom{M}{N_\alpha}\binom{M}{N_\beta} \]

Because this binomial dimension grows factorially with molecular size, solving the exact FCI eigenvalue problem becomes computationally intractable for all but the smallest molecules. Truncated CI approximations like CISD reduce the determinant dimension to a polynomial scaling of \(O(N_{\text{occ}}^2 N_{\text{virt}}^2)\), making correlated electronic calculations feasible for larger chemical systems.


II. Hamiltonian Matrix Elements and the Slater–Condon Rules

3. Second-Quantized Derivation of Matrix Elements

To determine the variational coefficients and electronic energies, we evaluate the matrix representation of the exact electronic Hamiltonian \(\hat{H}=\sum_{pq}h_{pq}a_p^\dagger a_q+\frac{1}{2}\sum_{pqrs}g_{pqrs}a_p^\dagger a_q^\dagger a_s a_r\) in the orthonormal determinant basis. In a canonical Hartree–Fock reference, the Fock operator is diagonal with orbital eigenvalues \(\epsilon_p\). Because the Fock operator decomposes into the one-electron core Hamiltonian and the mean-field Coulomb and exchange potentials as \(\hat{F}=\hat{h}+\sum_i^{\text{occ}}(\hat{J}_i-\hat{K}_i)\), the one-electron core Hamiltonian matrix elements in the canonical spin-orbital basis evaluate directly as

\[ h_{pq}=\epsilon_p\delta_{pq}-\sum_i^{\text{occ}}\left(g_{piqi}-g_{piiq}\right) \]

Applying Wick's theorem to the expectation values \(\langle\Phi_m|\hat{H}|\Phi_n\rangle\) eliminates all uncontracted operator strings, deriving the Slater–Condon rules directly based on the orbital differences between determinants \(m\) and \(n\).

When the two determinants are identical (\(m=n\)), all operators contract within the occupied spin-orbitals of that determinant, giving the diagonal matrix element as

\[ \langle\Phi_m|\hat{H}|\Phi_m\rangle=\sum_{i \in \text{det}_m}h_{ii}+\frac{1}{2}\sum_{i,j \in \text{det}_m}\left(g_{ijij}-g_{ijji}\right) \]

When the two determinants differ by exactly one spin-orbital, where determinant \(m\) contains spin-orbital \(p\) and determinant \(n\) contains spin-orbital \(q\), contracting the one-body and two-body operators over the set of common occupied spin-orbitals \(I_{\text{common}}\) yields

\[ \langle\Phi_m|\hat{H}|\Phi_n\rangle=(-1)^\sigma\left(h_{pq}+\sum_{k \in I_{\text{common}}}\langle pk||qk\rangle\right) \]

where \((-1)^\sigma\) is the phase factor associated with the permutation of fermionic operators required to bring the shared spin-orbitals into matching alignment, and \(\langle pk||qk\rangle=\langle pk|qk\rangle-\langle pk|kq\rangle=g_{pkqk}-g_{pkkq}\) is the antisymmetrized two-electron integral in physicist notation.

When the two determinants differ by exactly two spin-orbitals, where determinant \(m\) contains spin-orbitals \(p,q\) and determinant \(n\) contains spin-orbitals \(r,s\), the one-body core Hamiltonian term vanishes due to the mutual orthogonality of the remaining spin-orbitals, leaving purely the two-body repulsion matrix element as

\[ \langle\Phi_m|\hat{H}|\Phi_n\rangle=(-1)^\sigma\langle pq||rs\rangle=(-1)^\sigma\left(\langle pq|rs\rangle-\langle pq|sr\rangle\right)=(-1)^\sigma\left(g_{pqrs}-g_{pqsr}\right) \]

When the two determinants differ by three or more spin-orbitals, the matrix element vanishes identically because the Hamiltonian contains at most two-body operators, meaning that at least one pair of differing spin-orbitals remains uncontracted and yields zero by orbital orthonormality as

\[ \langle\Phi_m|\hat{H}|\Phi_n\rangle=0\quad(\Delta N\ge3) \]

4. Block Structure of the CISD Hamiltonian Matrix

In the Configuration Interaction Singles and Doubles (CISD) approximation, the determinant basis is partitioned into the reference ground state \(|\Phi_0\rangle\), singly excited determinants \(|\Phi_{\text{S}}\rangle\), and doubly excited determinants \(|\Phi_{\text{D}}\rangle\). Arranging these configurations into subspace blocks reveals the matrix structure

\[ \mathbf{H}_{\text{CISD}}=\begin{pmatrix}H_{00}&\mathbf{H}_{0\text{S}}&\mathbf{H}_{0\text{D}}\\\mathbf{H}_{\text{S}0}&\mathbf{H}_{\text{SS}}&\mathbf{H}_{\text{SD}}\\\mathbf{H}_{\text{D}0}&\mathbf{H}_{\text{DS}}&\mathbf{H}_{\text{DD}}\end{pmatrix} \]

The reference diagonal element reproduces the Hartree–Fock energy, \(H_{00}=E_{\text{HF}}\). By Brillouin's theorem, established for the canonical Hartree–Fock reference, the exact Hamiltonian does not couple the reference ground state directly to any singly excited determinant, causing the reference-singles coupling block to vanish identically as

\[ \mathbf{H}_{0\text{S}}=\mathbf{0} \]

The reference state couples directly only to the doubly excited configurations, with coupling matrix elements given by \(\langle\Phi_0|\hat{H}|\Phi_{ij}^{ab}\rangle=\langle ij||ab\rangle=g_{ijab}-g_{ijba}=\langle ij|ab\rangle-\langle ij|ba\rangle\). Although single excitations do not couple directly to the reference state, they couple to the double excitations through the off-diagonal block \(\mathbf{H}_{\text{SD}}\), allowing singly excited configurations to mix indirectly into the correlated ground-state wavefunction and to describe one-electron polarization effects. The determinant basis and the Hamiltonian block structure \(\mathbf{H}\) established here also serve as the foundation for Møller–Plesset perturbation theory, which evaluates electron correlation by solving the Schrödinger equation perturbatively order by order rather than diagonalizing \(\mathbf{H}\) variationally.


III. Variational Optimization and Matrix Eigenvalues

5. Rayleigh–Ritz Variational Formulation

The linear expansion coefficients \(c_{nk}\) and state energies \(E_k\) are determined by applying the Rayleigh–Ritz variational principle. We minimize the energy expectation value \(E=\frac{\langle\Psi|\hat{H}|\Psi\rangle}{\langle\Psi|\Psi\rangle}\) subject to the normalization constraint \(\langle\Psi|\Psi\rangle=1\) by constructing the Lagrangian functional

\[ \mathcal{L}=\sum_{m,n}c_m^* H_{mn}c_n-E\left(\sum_n c_n^* c_n-1\right) \]

where \(H_{mn}=\langle\Phi_m|\hat{H}|\Phi_n\rangle\) are the Hamiltonian matrix elements evaluated via the Slater–Condon rules. Setting the variation with respect to the complex conjugate coefficients \(c_m^*\) to zero yields the stationary condition

\[ \sum_n H_{mn}c_{nk}=E_k c_{mk} \]

which establishes that the optimal configuration interaction coefficients satisfy a system of linear equations coupled by the Hamiltonian matrix.

6. Secular Equations and Variational Bounds

Collecting the linear equations for all basis determinants into matrix form produces the Hermitian matrix eigenvalue problem

\[ \mathbf{H}_{\text{CI}}\mathbf{c}_k=E_k\mathbf{c}_k \]

where \(\mathbf{H}_{\text{CI}}\) is the Hamiltonian matrix representation in the chosen determinant basis, \(\mathbf{c}_k\) is the normalized eigenvector of configuration coefficients, and \(E_k\) is the corresponding electronic energy eigenvalue. By the Hylleraas–Undheim–MacDonald theorem, each calculated eigenvalue \(E_k\) provides a rigorous variational upper bound to the exact \(k\)-th electronic state energy of the physical Hamiltonian, \(E_k\ge E_k^{\text{exact}}\). Diagonalizing \(\mathbf{H}_{\text{CI}}\) yields the correlated ground state and all electronically excited states simultaneously as mutually orthogonal eigenvectors.


IV. Resolution of the Identity and Analytical Gradients

7. Density Fitting Approximation for CI Matrix Elements

To eliminate the prohibitive four-center two-electron integral storage and transformation bottlenecks, the Resolution of the Identity expands product densities in an auxiliary basis set, expressing two-electron repulsion integrals in physicist notation as

\[ \langle pq|rs\rangle=(pr|qs)\approx\sum_P B_{pr}^P B_{qs}^P \]

where \(B_{pq}^P=\sum_Q(pq|Q)[\mathbf{J}^{-1/2}]_{QP}\) is the three-center fitting tensor in the molecular orbital basis formed by contracting three-center electron repulsion integrals \((pq|Q)\) with the inverse square root of the auxiliary Coulomb metric \(\mathbf{J}\). For restricted spatial references, the spin-orbital integrals \(\langle p_\sigma q_{\sigma'}|r_\sigma s_{\sigma'}\rangle\) are assembled directly from the spatial components, whereas for generalized references the spin-summed molecular orbital three-center intermediate is used. The Hamiltonian matrix elements \(\langle\Phi_m|\hat{H}|\Phi_n\rangle\) are then evaluated via Slater–Condon rules without constructing or storing four-center repulsion tensors.

8. Analytical Nuclear Gradients with Dual Numbers

The analytical nuclear gradient of the \(k\)-th electronic CI state energy \(E_k\) with normalized configuration eigenvector \(\mathbf{c}_k\) with respect to a nuclear coordinate \(X\) is given by Hellmann–Feynman differentiation over the determinant basis as

\[ \frac{d E_k}{d X}=\sum_{m \le n}(2-\delta_{mn})c_{mk}c_{nk}\frac{\partial H_{mn}}{\partial X} \]

where the derivative matrix elements \(\partial H_{mn}/\partial X\) arise from the nuclear coordinate derivatives of the core Hamiltonian \(\mathbf{H}\), molecular orbital coefficients \(\mathbf{C}\), and fitting tensor \(\mathbf{B}\). Using forward-mode automatic differentiation, the unperturbed quantities and their geometric derivatives are packaged into first-order dual numbers as \(\tilde{\mathbf{H}}=\mathbf{H}+\frac{\partial\mathbf{H}}{\partial X}\epsilon\), \(\tilde{\mathbf{C}}=\mathbf{C}+\frac{\partial\mathbf{C}}{\partial X}\epsilon\), and \(\tilde{\mathbf{B}}=\mathbf{B}+\frac{\partial\mathbf{B}}{\partial X}\epsilon\). Transforming the dual core Hamiltonian and three-center fitting tensor into the molecular spin-orbital basis automatically yields the dual determinant Hamiltonian matrix elements \(\tilde{H}_{mn}=H_{mn}+\frac{\partial H_{mn}}{\partial X}\epsilon\), enabling exact nuclear gradients for ground and excited states without computing four-center integral derivatives.