Angular Correlation Function Estimators Accounting for Contamination from Probabilistic Distance Measurements

Using the standard ACF estimator, correlations from known contaminated samples can be corrected for by using the fractions f_αβ as defined in Equation (8); see, e.g., Grasshorn Gebhardt et al. (2018) and Addison et al. (2018) for a similar approach. Formally, this is done by writing the observed correlation functions in terms of the true correlation functions by considering the type of galaxy that contributes to each data pair. Here, we work with two target galaxy samples: Type-A and Type-B. The generalized case is discussed in Appendix D.1.

Since we have two types of galaxies, we aim to calculate two autocorrelations and one cross-correlation from the contaminated sample: ${w}_{{AA}}^{\mathrm{true}}({\theta }_{k}),{w}_{{AB}}^{\mathrm{true}}({\theta }_{k}),{w}_{{BB}}^{\mathrm{true}}({\theta }_{k})$. However, if we calculate the correlations on the subsamples directly, we get ${w}_{{AA}}^{\mathrm{obs}}({\theta }_{k}),{w}_{{AB}}^{\mathrm{obs}}({\theta }_{k}),{w}_{{BB}}^{\mathrm{obs}}({\theta }_{k})$, which differ from the true correlations due to sample contamination. To construct the relation between the two, let us consider ${w}_{{AB}}^{\mathrm{obs}}({\theta }_{k})$ which gets its contributions from four types of pairs: (1) observed Type-A galaxies that are true Type-A, paired with observed Type-B that are true Type-A, contributing ${f}_{{AA}}{f}_{{BA}}{w}_{{AA}}^{\mathrm{true}}({\theta }_{k})$ to the observed correlation; (2) observed Type-A that are true Type-A, paired with observed Type-B that are true Type-B, contributing ${f}_{{AA}}{f}_{{BB}}{w}_{{AB}}^{\mathrm{true}}({\theta }_{k})$; (3) observed Type-B that are true Type-A, paired with observed Type-A that are true Type-B, contributing ${f}_{{AB}}{f}_{{BA}}{w}_{{AB}}^{\mathrm{true}}({\theta }_{k})$; and (4) observed Type-A that are true Type-B, paired with observed Type-B that are true Type-B, contributing ${f}_{{AB}}{f}_{{BB}}{w}_{{BB}}^{\mathrm{true}}({\theta }_{k})$. Therefore, we have

$$\begin{eqnarray}\begin{array}{rcl}{w}_{{AB}}^{\mathrm{obs}}({\theta }_{k}) & = & {f}_{{AA}}{f}_{{BA}}{w}_{{AA}}^{\mathrm{true}}({\theta }_{k})+\left\{{f}_{{AA}}{f}_{{BB}}+{f}_{{BA}}{f}_{{AB}}\right\}\\ & & \times \ {w}_{{AB}}^{\mathrm{true}}({\theta }_{k})+{f}_{{AB}}{f}_{{BB}}{w}_{{BB}}^{\mathrm{true}}({\theta }_{k}).\end{array}\end{eqnarray} \tag{ 10 }$$

The autocorrelations follow similarly, leading us to

$$\begin{eqnarray}&&\begin{array}{l}\left[\begin{array}{l}{w}_{{AA}}^{\mathrm{obs}}({\theta }_{k})\\ {w}_{{AB}}^{\mathrm{obs}}({\theta }_{k})\\ {w}_{{BB}}^{\mathrm{obs}}({\theta }_{k})\end{array}\right]=\left[\begin{array}{ccc}{f}_{{AA}}^{2} & 2{f}_{{AA}}{f}_{{AB}} & {f}_{{AB}}^{2}\\ {f}_{{AA}}{f}_{{BA}} & {f}_{{AA}}{f}_{{BB}}+{f}_{{AB}}{f}_{{BA}} & {f}_{{AB}}{f}_{{BB}}\\ {f}_{{BA}}^{2} & 2{f}_{{BB}}{f}_{{BA}} & {f}_{{BB}}^{2}\end{array}\right]\left[\begin{array}{l}{w}_{{AA}}^{\mathrm{true}}({\theta }_{k})\\ {w}_{{AB}}^{\mathrm{true}}({\theta }_{k})\\ {w}_{{BB}}^{\mathrm{true}}({\theta }_{k})\end{array}\right],\end{array}\end{eqnarray} \tag{ 11 }$$

where we note that the contribution from the true cross-correlation to the observed autocorrelations simplifies (as opposed for that to the observed cross-correlation). We also present a formal derivation of the result above using Equation (1) in Appendix A.1. Now, using these equations, we can construct the Decontaminated estimators ${\widehat{w}}_{{AA}}({\theta }_{k}),{\widehat{w}}_{{BB}}({\theta }_{k}),{\widehat{w}}_{{AB}}({\theta }_{k})$ for the true correlation functions ${w}_{{AA}}^{\mathrm{true}}({\theta }_{k}),{w}_{{BB}}^{\mathrm{true}}({\theta }_{k}),{w}_{{AB}}^{\mathrm{true}}({\theta }_{k})$, given by

$$\begin{eqnarray}&&\begin{array}{l}{\left[{\widehat{w}}_{{AA}}({\theta }_{k}){\widehat{w}}_{{AB}}({\theta }_{k}){\widehat{w}}_{{BB}}({\theta }_{k})\right]}^{T}\\ =\,{\left[{D}_{{\rm{S}}}\right]}^{-1}{\left[{w}_{{AA}}^{\mathrm{obs}}({\theta }_{k}){w}_{{AB}}^{\mathrm{obs}}({\theta }_{k}){w}_{{BB}}^{\mathrm{obs}}({\theta }_{k})\right]}^{T},\end{array}\end{eqnarray} \tag{ 12 }$$

where $[{D}_{{\rm{S}}}]$ is the square matrix in Equation (11), which must be invertible.⁸ Appendix D.1 presents the Decontaminated estimators for the generalized case of working with M target subsamples. We also note that this decontamination formalism could be easily applied to the LS estimator; the decontamination matrix $[{D}_{{\rm{S}}}]$ does not inherently depend on the usage of the DD/RR estimator.

Given their construction, the Decontaminated estimators are unbiased (under the assumption that the contamination fractions are represented by the average classification probabilities); see Appendix A.2 for more details. As for the variance, the decontamination leads to a quadrature sum of the variance of the standard estimators for each of the auto- and cross-correlations in the absence of covariance between the observed correlations; the closed-form expression for the variance as well as the general covariance of the estimators is presented in Appendix A.3. Note that this overarching idea of using contamination fractions is similar to that presented in Benjamin et al. (2010), but their focus is on estimating the contamination fractions from the contaminated correlations—for which they resort to approximating the decontamination matrix as diagonal. Since we expect sufficiently strong correlations across the different target samples (e.g., between the neighboring photo-z bins for a tomographic clustering analysis), the simplification of ignoring some contamination fractions becomes undesirable.

The estimator in Equation (13) is biased, as it considers the entire sample, including contaminants with different correlation functions. In order to estimate the true correlations using unbiased estimators, $\widehat{w}$, we require that their expectation value approach the true correlations. That is, we have

$$\begin{eqnarray}&&\left\langle \left[\begin{array}{l}{\widehat{w}}_{{AA}}({\theta }_{k})\\ {\widehat{w}}_{{AB}}({\theta }_{k})\\ {\widehat{w}}_{{BB}}({\theta }_{k})\end{array}\right]\right\rangle =\left\langle [{D}_{{\rm{W}}}]\left[\begin{array}{l}{\widetilde{w}}_{{AA}}^{\mathrm{obs}}({\theta }_{k})\\ {\widetilde{w}}_{{AB}}^{\mathrm{obs}}({\theta }_{k})\\ {\widetilde{w}}_{{BB}}^{\mathrm{obs}}({\theta }_{k})\end{array}\right]\right\rangle =\left[\begin{array}{l}{w}_{{AA}}^{\mathrm{true}}({\theta }_{k})\\ {w}_{{AB}}^{\mathrm{true}}({\theta }_{k})\\ {w}_{{BB}}^{\mathrm{true}}({\theta }_{k})\end{array}\right],\end{eqnarray} \tag{ 15 }$$

where $[{D}_{{\rm{W}}}]$ is a decontamination matrix, designed to make the estimators unbiased. It is analogous to the decontamination matrix $[{D}_{{\rm{S}}}]$ in Equation (12). Here, we explicitly work with the two-sample case, with only Type-A and Type-B galaxies present in our sample.

As done to decontaminate the ${\mathtt{Standard}}$ estimators in Section 3.1, we calculate the contributions that are coming from each of the true correlation functions to any given weighted correlation function. That is, we have

$$\begin{eqnarray}&&\left\langle {\widetilde{w}}_{\alpha \beta }^{\mathrm{obs}}({\theta }_{k})\right\rangle =\displaystyle \frac{\begin{array}{l}\left({\displaystyle \sum }_{i}^{{N}_{\mathrm{tot}}}{\displaystyle \sum }_{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{q}}}_{i}^{A}{{\mathtt{q}}}_{j}^{A}\right){w}_{{AA}}^{\mathrm{true}}({\theta }_{k})\\ +\,\left({\displaystyle \sum }_{i}^{{N}_{\mathrm{tot}}}{\displaystyle \sum }_{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{\alpha \beta }\left\{{{\mathtt{q}}}_{i}^{A}{{\mathtt{q}}}_{j}^{B}+{{\mathtt{q}}}_{i}^{B}{{\mathtt{q}}}_{j}^{A}\right\}\right){w}_{{AB}}^{\mathrm{true}}({\theta }_{k})\\ +\,\left({\displaystyle \sum }_{i}^{{N}_{\mathrm{tot}}}{\displaystyle \sum }_{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{q}}}_{i}^{B}{{\mathtt{q}}}_{j}^{B}\right){w}_{{BB}}^{\mathrm{true}}({\theta }_{k})\end{array}}{{\displaystyle \sum }_{i}^{{N}_{\mathrm{tot}}}{\displaystyle \sum }_{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{\alpha \beta }}.\end{eqnarray} \tag{ 16 }$$

We present the full derivation of Equation (16) in Appendix B. Consolidating the terms as done in Equation (11), we have

$$\begin{eqnarray}&&\begin{array}{l}\left[\begin{array}{l}\left\langle {\widetilde{w}}_{{AA}}^{\mathrm{obs}}({\theta }_{k})\right\rangle \\ \left\langle {\widetilde{w}}_{{AB}}^{\mathrm{obs}}({\theta }_{k})\right\rangle \\ \left\langle {\widetilde{w}}_{{BB}}^{\mathrm{obs}}({\theta }_{k})\right\rangle \end{array}\right]\,=\,\left[\begin{array}{ccc}\displaystyle \frac{\sum _{i}^{{N}_{\mathrm{tot}}}\sum _{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{{AA}}{{\mathtt{q}}}_{i}^{A}{{\mathtt{q}}}_{j}^{A}}{\sum _{i}^{{N}_{\mathrm{tot}}}\sum _{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{{AA}}} & \displaystyle \frac{\sum _{i}^{{N}_{\mathrm{tot}}}\sum _{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{{AA}}\left\{{{\mathtt{q}}}_{i}^{A}{{\mathtt{q}}}_{j}^{B}+{{\mathtt{q}}}_{i}^{B}{{\mathtt{q}}}_{j}^{A}\right\}}{\sum _{i}^{{N}_{\mathrm{tot}}}\sum _{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{{AA}}} & \displaystyle \frac{\sum _{i}^{{N}_{\mathrm{tot}}}\sum _{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{{AA}}{{\mathtt{q}}}_{i}^{B}{{\mathtt{q}}}_{j}^{B}}{\sum _{i}^{{N}_{\mathrm{tot}}}\sum _{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{{AA}}}\\ \displaystyle \frac{\sum _{i}^{{N}_{\mathrm{tot}}}\sum _{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{{AB}}{{\mathtt{q}}}_{i}^{A}{{\mathtt{q}}}_{j}^{A}}{\sum _{i}^{{N}_{\mathrm{tot}}}\sum _{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{{AB}}} & \displaystyle \frac{\sum _{i}^{{N}_{\mathrm{tot}}}\sum _{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{{AB}}\left\{{{\mathtt{q}}}_{i}^{A}{{\mathtt{q}}}_{j}^{B}+{{\mathtt{q}}}_{i}^{B}{{\mathtt{q}}}_{j}^{A}\right\}}{\sum _{i}^{{N}_{\mathrm{tot}}}\sum _{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{{AB}}} & \displaystyle \frac{\sum _{i}^{{N}_{\mathrm{tot}}}\sum _{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{{AB}}{{\mathtt{q}}}_{i}^{B}{{\mathtt{q}}}_{j}^{B}}{\sum _{i}^{{N}_{\mathrm{tot}}}\sum _{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{{AB}}}\\ \displaystyle \frac{\sum _{i}^{{N}_{\mathrm{tot}}}\sum _{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{{BB}}{{\mathtt{q}}}_{i}^{A}{{\mathtt{q}}}_{j}^{A}}{\sum _{i}^{{N}_{\mathrm{tot}}}\sum _{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{{BB}}} & \displaystyle \frac{\sum _{i}^{{N}_{\mathrm{tot}}}\sum _{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{{BB}}\left\{{{\mathtt{q}}}_{i}^{A}{{\mathtt{q}}}_{j}^{B}+{{\mathtt{q}}}_{i}^{B}{{\mathtt{q}}}_{j}^{A}\right\}}{\sum _{i}^{{N}_{\mathrm{tot}}}\sum _{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{{BB}}} & \displaystyle \frac{\sum _{i}^{{N}_{\mathrm{tot}}}\sum _{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{{BB}}{{\mathtt{q}}}_{i}^{B}{{\mathtt{q}}}_{j}^{B}}{\sum _{i}^{{N}_{\mathrm{tot}}}\sum _{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{{BB}}}\end{array}\right]\left[\begin{array}{l}{w}_{{AA}}^{\mathrm{true}}({\theta }_{k})\\ {w}_{{AB}}^{\mathrm{true}}({\theta }_{k})\\ {w}_{{BB}}^{\mathrm{true}}({\theta }_{k})\end{array}\right].\end{array}\end{eqnarray} \tag{ 17 }$$

Therefore, the ${\mathtt{Decontaminated}}\ {\mathtt{Weighted}}$ estimators are given by

$$\begin{eqnarray}&&\begin{array}{l}{\left[{\widehat{w}}_{{AA}}({\theta }_{k}){\widehat{w}}_{{AB}}({\theta }_{k}){\widehat{w}}_{{BB}}({\theta }_{k})\right]}^{T}\\ =\,{\left[{D}_{{\rm{W}}}\right]}^{-1}{\left[{\widetilde{w}}_{{AA}}^{\mathrm{obs}}({\theta }_{k}){\widetilde{w}}_{{AB}}^{\mathrm{obs}}({\theta }_{k}){\widetilde{w}}_{{BB}}^{\mathrm{obs}}({\theta }_{k})\right]}^{T},\end{array}\end{eqnarray} \tag{ 18 }$$

where $[{D}_{{\rm{W}}}]$ is the square matrix in Equation (17). We note that each row in Equation (18) corresponds to final, unbiased weights on each pair, comprised of a sum of three weights—a fact that can be utilized when optimizing weights for minimum variance. We present an example optimization that decontaminates while estimating the correlation functions in Appendix C.3.

We have checked Equation (18) in various limiting cases to confirm the validity of its form. Specifically, we first divided the total observed sample into subsamples, and then applied the simplifications that reduce the Decontaminated Weighted estimators to Decontaminated estimators (i.e., setting the pair weights for the target subsample to unity and the rest to zero, and approximating the classification probabilities with their averages); we confirm that Equation (18) does indeed reduce to Equation (12), demonstrating that Decontaminated Weighted is the generalized estimator. We then tested the two limiting cases of no contamination and 100% contamination, working with just the observed subsamples and using pair weights that are a linear product of the respective classification probabilities; we confirm that the reduced estimator recovers the truth when there is no contamination, whereas it is indeterminate when there is 100% contamination. Finally, we considered the entire observed sample and tested the limiting cases of no contamination and 100% contamination, with pair weights that are a linear product of the respective classification probabilities, and arrive at true correlations both when there is no contamination and when there is 100% contamination—an advantage of using the full sample. We also present the analytical form of the variance of the Weighted estimator in Appendix C.1; since the variance is a function of a four-point sum and depends nontrivially on the pair weights, we choose to estimate the variance numerically using the bootstrap method as described in Section 5.1. Finally, we present the generalized estimator, i.e., applicable to M target samples, in Appendix D.2.

In order to illustrate the impacts of photo-zs, we consider a toy example: a clustering analysis using only two tomographic bins (0.7 ≤ z < 0.8, 0.8 ≤ z < 0.9) with the true galaxy sample having galaxies only at 0.75 ≤ z ≤ 0.76, 0.85 ≤ z ≤ 0.86, but with the photo-z scatter as mentioned before, i.e., σ_z = 0.03(1 + z). We query the true galaxies in nine 10 × 10 deg² patches along decl. = 0; all patches have a similar number of galaxies (66–78 K) and face similar photo-z contamination rates (22–25% and 18–21% in the two tomographic bins, respectively). To demonstrate the impacts of redshift binning based on photo-z point estimates, we show the true and observed positions of the galaxies in the two redshift bins in Figure 2, where we can see that the two distributions are different, with photo-z uncertainties mixing the LSS between the two bins. Figure 3 shows the distributions of the true and photometric redshifts using one of the patches (with 66,927 galaxies, and 23% and 20% contamination in the two tomographic bins, respectively).

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

Next, using the observed photo-z PDFs, we calculate the classification probabilities as the integral of the PDFs within the target redshift bin. Note that since we are simulating only two bins, we use Gaussian PDFs truncated at z = 0.7 and z = 0.9 to ensure that we conserve the number of true and observed galaxies; this yields a slight bias in the PDF integrations, which we correct to make the overall classification probabilities unbiased, i.e., $\left\langle {q}_{i}^{{AB}}\right\rangle ={f}_{{AB}}$, where the average is checked over redshift intervals with Δz = 0.02, while ensuring the debiased probabilities remain in the range 0–1. For real data, this debiasing should be possible utilizing a limited set of spectroscopic redshifts. Figure 4 shows the distribution of the final classification probabilities for all the galaxies in our observed sample.

Zoom In Zoom Out Reset image size

In order to estimate the various correlation functions (two auto, one cross) and their variance, we consider the nine patches: the mean across the nine samples gives us the mean estimate of the respective correlation function while we calculate the estimator variance as $\left\langle {\left\{{\widehat{w}}_{i}({\theta }_{k})-{w}_{i}^{\mathrm{true}}({\theta }_{k})\right\}}^{2}\right\rangle$ where i runs over all the correlations (both auto and cross) and the expectation value is over all the realizations; note that this variance is not sensitive to the sample variance but only a measure of the estimator variance, which we can calculate explicitly given that we have access to the true CFs in each of the nine patches. Note that for each of the patches, we calculate five types of the three correlation functions: those in the true subsamples; those using the ${\mathtt{Standard}}$ estimator on the contaminated observed subsamples, followed by those from the Decontaminated estimators; and those using the Weighted estimator, followed by the Decontaminated Weighted ones. Also, we use a random catalog that is five times the size of the data catalog, and restrict CF calculation to 0.01–3 deg scales. Figure 5 shows our results, with both the correlation functions and their variance. As expected, the cross-correlations with contamination are non-negligible, taking signal away from the two autocorrelations. Decontamination lowers the amplitude of the cross-correlations, and we find that both estimators correct for the contamination and reduce the bias, leading to estimates closer to the truth. This is more apparent in Figure 6, where we show the bias in the correlation functions—i.e., difference from the truth calculated as $\left\langle {\widehat{w}}_{i}({\theta }_{k})-{w}_{i}^{\mathrm{true}}({\theta }_{k})\right\rangle$, where i runs over all the correlations (both auto and cross) and the expectation value is over all the realizations. We note that the Decontaminated Weighted estimator is unbiased after decontamination—a reassuring result. We also note that our decontaminated estimators reduce the variance on the CF estimates, as indicated by the error bars in Figure 5.

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

Now we consider a more realistic scenario: a true galaxy sample with $0.7\leqslant z\leqslant 1.0$, with three redshift bins (0.7 ≤ z < 0.8, 0.8 ≤ z < 0.9, 0.9 ≤ z < 1.0) for the tomographic clustering analysis. As before, we query the galaxies in nine 10 × 10 deg² patches along decl. = 0, and model their photo-zs assuming Gaussian PDFs for all the galaxies with σ_z = 0.03(1 + z) as discussed at the beginning of Section 5; all patches have a similar number of galaxies (1080–1147 K) and face similar contamination (23–26%, 44–46%, and 19–23% in the three tomographic bins, respectively). Note that our chosen bins are realistic, as a tomographic analysis for 10 redshift bins with Δz = 0.1 is currently planned for dark energy science studies with LSST (The LSST Dark Energy Science Collaboration et al. 2018); our treatment of photo-zs, however, is optimistic in the assumption of Gaussian photo-z PDFs.

Figure 7 shows the distributions of the true redshifts and the photo-zs using one of the patches (with 1095,404 galaxies, and 24%, 45%, and 22% contamination in the three redshift bins, respectively). We note that the middle bin sees the largest and most realistic contamination—the case that will be true for most of the LSST bins, hence making this example a relevant one. Note that the bin edges see the impacts of artificially having contamination from only one side.

Zoom In Zoom Out Reset image size

Figure 8 shows the distribution of the classification probabilities for all the galaxies. Again we note that, given the large contamination rates for the middle bin, the classification probabilities are far from unity, indicating that no observed galaxy has a very high probability to be in any target bin. As before, we calculate the various correlations for each of the nine patches, and estimate the mean and the variance across the calculations. Figure 9 illustrates our results, showing only the estimator bias for brevity, where we see that the Decontaminated Weighted estimator leads to a bias that is comparable to that using the Decontaminated estimator, both of which are smaller than from those without decontamination. We note that the Decontaminated estimator performs similar to Decontaminated Weighted, potentially due to the correlation functions in the three redshift bins being similar. We also note that there is a weak residual bias in the decontaminated estimates, which is likely caused by our simple debiasing of the classification probabilities.

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

As a more comprehensive metric for comparing the various estimators, we consider the covariances in correlation functions across the three redshift bins for an example θ bin. Specifically, given that we have access to the truth here, we first calculate the covariances in the estimators without accounting for the LSS sample variance—this we term as the "estimator covariance" and calculate as $\left\langle \left\{{\widehat{w}}_{i}({\theta }_{k})-{w}_{i}^{\mathrm{true}}({\theta }_{k})\right\}\left\{{\widehat{w}}_{j}({\theta }_{k})-{w}_{j}^{\mathrm{true}}({\theta }_{k})\right\}\right\rangle$ where i, j run over all the correlations (both auto and cross) and the expectation value is over all the realizations;¹¹ note here that the diagonal of this covariance matrix is the estimator variance used to generate uncertainties shown in Figures 5, 6, and 9. We show the estimator covariances for the mock galaxy sample considered here in Figure 10, where we see that without decontamination, the covariances are large, as expected given the strong mixing of the samples. Both decontaminated estimators effectively reduce the covariances, with Decontaminated Weighted outperforming Decontaminated.

Zoom In Zoom Out Reset image size

Next, we consider the covariances accounting for the LSS sample variance—this we term as the "full covariance" and calculate as $\left\langle \left\{{\widehat{w}}_{i}({\theta }_{k})-\left\langle {\widehat{w}}_{i}({\theta }_{k})\right\rangle \right\}\left\{{\widehat{w}}_{j}({\theta }_{k})-\left\langle {\widehat{w}}_{j}({\theta }_{k})\right\rangle \right\}\right\rangle ,$ where i, j again run over all the correlations and the expectation value is over all the realizations; these are shown in Figure 11. We see that without decontamination, the clustering information is smeared across the CF space and is much in contrast from the true covariances. However, both of our decontaminated estimators are able to approximate the true covariances effectively, hence achieving their purpose of correcting for sample contamination. We also note here that decontamination does not simply diagonalize the covariance matrix, but instead reduces off-diagonal elements appropriately; diagonalization would not account for true covariances that exist between auto- and cross-CFs for neighboring bins due to shared LSS. Finally, comparing with Figure 10, we note that LSS sample variance largely dominates over the estimator variance for the 10 × 10 patches considered here—a reassuring result. A comparison between the two sources of variance for larger effective survey area is left for future work.

Zoom In Zoom Out Reset image size

Now we consider a more pessimistic scenario for the true galaxy sample of Section 5.2: instead of having all the galaxies with well-behaved Gaussian photo-z PDFs, we assign half of the galaxies bimodal photo-z PDFs—a scenario where standard N(z) forward modeling might be problematic. Specifically, the Gaussian photo-z PDFs are constructed as described above: by drawing a random number from a Gaussian of width σ = 0.03(1 + z_true), with the observed photo-z PDF being a Gaussian centered at z_draw and with width σ = 0.03(1 + z_draw). In contrast, the bimodal photo-z PDFs are constructed with one mode at the true redshift and another randomly chosen to be ±0.13 away (while ensuring the second mode remains in the redshift range of 0.7–1.0); 0.13 separation mimics a degeneracy arising from Balmer versus 4000 Å decrement at ∼7% separations in 1 + z. This treatment leads to slightly higher contamination rates: 39–42%, 54–57%, 33–36% in the three tomographic bins, respectively. To illustrate the difference between the two cases more explicitly, Figure 12 shows an example set of PDFs for the case of all-Gaussian PDFs versus half-bimodal ones.

Zoom In Zoom Out Reset image size

Figure 13 shows the distributions of the true redshifts and the photo-zs using one of the patches (with 1095,404 galaxies as before, but now with 40%, 55%, and 35% contamination in the three redshift bins, respectively). Comparing it to Figure 7, we see that the distribution is slightly more biased, although the middle redshift bin sees a comparable observed redshift distribution; and as before, the bin edges see the impacts of artificially having contamination from only one side.

Zoom In Zoom Out Reset image size

Figure 14 shows the classification probabilities for all the galaxies here; comparing it to Figure 8, we see that the classification probabilities are now more varied, with more objects in the edge bins with larger classification probabilities, due to the bimodality in some of the photo-z PDFs. As before, we calculate the various correlations for each of the nine patches and estimate the mean CFs and the covariances. Figure 15 shows the residuals in the CF estimates, and we see that the decontaminated estimators are able to reduce the bias significantly. Figure 16 shows the estimator covariance matrices where we see that, as in the all-Gaussian case, our decontaminated estimators lead to lower estimator covariances, with Decontaminated Weighted outperforming Decontaminated slightly more strongly than in Figure 10. Finally, Figure 17 shows the full covariance matrices. Here too, we see that, as in Figure 11 for the all-Gaussian case, our decontaminated estimators approximate the true covariances more effectively than those without decontamination.

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

This completes the demonstration of our new estimators: they provide for a way to decontaminate correlations, while the Weighted estimator specifically allows using the full photo-z PDFs and full observed samples, in a framework that can be extended, e.g., to minimize variance.

Here, we rederive the decontamination equation (Equation (11)) using the definition of angular correlation function. We start with Equation (1), rewriting it as

$$\begin{eqnarray}\begin{array}{rclrcl}{{dP}}_{\alpha \beta }({\theta }_{k}) & = & {\eta }_{\alpha \beta }^{\mathrm{pair}}\left[1+{w}_{\alpha \beta }({\theta }_{k})\right]d{{\rm{\Omega }}}_{\alpha }d{{\rm{\Omega }}}_{\beta } & & = & {{ \mathcal N }}_{\alpha \beta }\left[1+{w}_{\alpha \beta }({\theta }_{k})\right]\displaystyle \frac{d{{\rm{\Omega }}}_{\alpha }}{{V}_{\alpha }}\displaystyle \frac{d{{\rm{\Omega }}}_{\beta }}{{V}_{\beta }},\end{array}\end{eqnarray} \tag{ 19 }$$

where ${\eta }_{\alpha \beta }^{\mathrm{pair}}$ is the observed sky density of Type-$\alpha \beta$ pairs of galaxies while ${{ \mathcal N }}_{\alpha \beta }$ is the observed number of Type-αβ pairs. Assuming that we work with large surveys such that the integral constraint is nearly zero, we have ${{ \mathcal N }}_{\alpha \beta }\to \left\langle {{ \mathcal N }}_{\alpha \beta }\right\rangle$, hence the simplification in the last line in the equation above. Since we consider samples in the same volume, V_α = V_β = V and $d{{\rm{\Omega }}}_{\alpha }=d{{\rm{\Omega }}}_{\beta }=d{\rm{\Omega }}$. Therefore, for the ${\mathtt{Standard}}$ estimator, for the case where we have the correlations measured in the contaminated subsamples, we have

$$\begin{eqnarray}\begin{array}{rclrcl}{{dP}}_{\alpha \beta }({\theta }_{k}) & = & {{ \mathcal N }}_{\alpha \beta ,\mathrm{obs}}\left[1+{w}_{\alpha \beta }^{\mathrm{obs}}({\theta }_{k})\right]\displaystyle \frac{d{\rm{\Omega }}}{V}\displaystyle \frac{d{\rm{\Omega }}}{V} & & = & \displaystyle \sum _{\gamma ,\delta }{{ \mathcal N }}_{\alpha \beta ,\mathrm{obs}}^{\gamma \delta ,\mathrm{true}}[1+{w}_{\gamma ,\delta }^{\mathrm{true}}({\theta }_{k})]\displaystyle \frac{d{\rm{\Omega }}}{V}\displaystyle \frac{d{\rm{\Omega }}}{V},\end{array}\end{eqnarray} \tag{ 20 }$$

where ${w}_{\alpha \beta }^{\mathrm{obs}}({\theta }_{k})$ is the biased correlation function, measured using contaminated samples. Expanding the sum on the right-hand side, we have

$$\begin{eqnarray}&&\begin{array}{l}{{ \mathcal N }}_{\alpha \beta ,\mathrm{obs}}^{\mathrm{tot}}\left[1+{w}_{\alpha \beta }^{\mathrm{obs}}({\theta }_{k})\right]={{ \mathcal N }}_{\alpha \beta ,\mathrm{obs}}^{11,\mathrm{true}}\left[1+{w}_{11}^{\mathrm{true}}({\theta }_{k})\right]\ +\ {{ \mathcal N }}_{\alpha \beta ,\mathrm{obs}}^{12,\mathrm{true}}\left[1+{w}_{12}^{\mathrm{true}}({\theta }_{k})\right]\\ \,+\ {{ \mathcal N }}_{\alpha \beta ,\mathrm{obs}}^{21,\mathrm{true}}\left[1+{w}_{21}^{\mathrm{true}}({\theta }_{k})\right]+{{ \mathcal N }}_{\alpha \beta ,\mathrm{obs}}^{22,\mathrm{true}}\left[1+{w}_{22}^{\mathrm{true}}({\theta }_{k})\right].\end{array}\end{eqnarray} \tag{ 21 }$$

Since we have

$$\begin{eqnarray}&&\displaystyle \frac{{{ \mathcal N }}_{\alpha \beta ,\mathrm{obs}}^{\gamma \delta ,\mathrm{true}}}{{{ \mathcal N }}_{\alpha \beta ,\mathrm{obs}}^{\mathrm{tot}}}={f}_{\alpha \gamma }{f}_{\beta \delta }\end{eqnarray} \tag{ 22 }$$

$$\begin{eqnarray}\begin{array}{lll}\Rightarrow \,\left[1+{w}_{\alpha \beta }^{\mathrm{obs}}({\theta }_{k})\right]={f}_{\alpha 1}{f}_{\beta 1}\left[1+{w}_{11}^{\mathrm{true}}({\theta }_{k})\right] \; +\,\left\{{f}_{\alpha 1}{f}_{\beta 2}+{f}_{\alpha 2}{f}_{\beta 1}\right\}\left[1+{w}_{12}^{\mathrm{true}}({\theta }_{k})\right] & \\+\,{f}_{\alpha 2}{f}_{\beta 2}\left[1+{w}_{22}^{\mathrm{true}}({\theta }_{k})\right].\end{array}\end{eqnarray} \tag{ 23 }$$

Therefore, for α, β = 1, 2, Equation (23) becomes

$$\begin{eqnarray}&&\begin{array}{l}\left[1+{w}_{12}^{\mathrm{obs}}({\theta }_{k})\right]={f}_{11}{f}_{21}\left[1+{w}_{11}^{\mathrm{true}}({\theta }_{k})\right]+\left\{{f}_{11}{f}_{22}+{f}_{12}{f}_{21}\right\}\left[1+{w}_{12}^{\mathrm{true}}({\theta }_{k})\right]\\+{f}_{12}{f}_{22}\left[1+{w}_{22}^{\mathrm{true}}({\theta }_{k})\right].\end{array}\end{eqnarray} \tag{ 24 }$$

Now, since

$$\begin{eqnarray}\begin{array}{rclrcl}{f}_{11}{f}_{21} & + & \left\{{f}_{11}{f}_{22}+{f}_{12}{f}_{21}\right\}+{f}_{12}{f}_{22} & & = & {f}_{11}\left[{f}_{21}+{f}_{22}\right]+{f}_{12}\left[{f}_{21}+{f}_{22}\right]=1,\end{array}\end{eqnarray} \tag{ 25 }$$

we have

$$\begin{eqnarray}\begin{array}{rcl}{w}_{12}^{\mathrm{obs}}({\theta }_{k}) & = & {f}_{11}{f}_{21}{w}_{11}^{\mathrm{true}}({\theta }_{k})+\left\{{f}_{11}{f}_{22}+{f}_{12}{f}_{21}\right\}{w}_{12}^{\mathrm{true}}({\theta }_{k})+{f}_{12}{f}_{22}{w}_{22}^{\mathrm{true}}({\theta }_{k}),\end{array}\end{eqnarray} \tag{ 26 }$$

which agrees with Equation (11). Similar results follow for (α, β) = (1, 1), (2, 2).

We expect that the Decontaminated estimators are unbiased given their construction (i.e., Equation (10)). However, for brevity, we formally show that they are indeed unbiased. By definition, an unbiased estimator is such that

$$\begin{eqnarray}&&\left\langle \widehat{w}\right\rangle ={w}_{\mathrm{true}},\end{eqnarray} \tag{ 27 }$$

where the expectation value is over many realizations of the survey. Now, using Equations (11) and (12), we have

$$\begin{eqnarray}\begin{array}{l}\left\langle {\left[\begin{array}{ccc}{\widehat{w}}_{{AA}}({\theta }_{k}) & {\widehat{w}}_{{AB}}({\theta }_{k}) & {\widehat{w}}_{{BB}}({\theta }_{k})\end{array}\right]}^{T}\right\rangle =\left\langle {\left[{D}_{{\rm{S}}}\right]}^{-1}{\left[\begin{array}{ccc}{w}_{{AA}}^{\mathrm{obs}}({\theta }_{k}) & {w}_{{AB}}^{\mathrm{obs}}({\theta }_{k}) & {w}_{{BB}}^{\mathrm{obs}}({\theta }_{k})\end{array}\right]}^{T}\right\rangle \\ \,=\,{\left[{D}_{{\rm{S}}}\right]}^{-1}[{D}_{{\rm{S}}}]{\left[\begin{array}{ccc}{w}_{{AA}}^{\mathrm{true}}({\theta }_{k}) & {w}_{{AB}}^{\mathrm{true}}({\theta }_{k}) & {w}_{{BB}}^{\mathrm{true}}({\theta }_{k})\end{array}\right]}^{T}\,=\,{\left[\begin{array}{ccc}{w}_{{AA}}^{\mathrm{true}}({\theta }_{k}) & {w}_{{AB}}^{\mathrm{true}}({\theta }_{k}) & {w}_{{BB}}^{\mathrm{true}}({\theta }_{k})\end{array}\right]}^{T},\end{array}\end{eqnarray} \tag{ 28 }$$

where the second equality follows by substituting Equation (11). Hence, the Decontaminated estimators are unbiased. We note here that [D_S] in Equation (12) is effectively a decontamination matrix: it removes the contamination from the biased estimates, ${w}_{\alpha \beta }^{\mathrm{obs}}$, in the presence of sample contamination. A similar argument follows for the case where we have M target samples, using Equation (108). We also note that Equation (28) is valid only when f_αβ are accurate averages of the classification probabilities.

As for the variance of the Decontaminated estimators, we can calculate it by using the variance in our observed correlations. That is, given Equation (12), we have

$$\begin{eqnarray}\begin{array}{ll}{\left[\begin{array}{l}{\sigma }_{{\widehat{w}}_{{AA}}}^{2}({\theta }_{k}){\sigma }_{{\widehat{w}}_{{AB}}}^{2}({\theta }_{k}){\sigma }_{{\widehat{w}}_{{BB}}}^{2}({\theta }_{k})\end{array}\right]}^{T} & \,=\,{\left\{{\left[{D}_{{\rm{S}}}\right]}^{-1}\right\}}_{{ij}}^{2}{\left[\begin{array}{l}{\sigma }_{{w}_{{AA}}^{\mathrm{obs}}}^{2}({\theta }_{k}){\sigma }_{{w}_{{AB}}^{\mathrm{obs}}}^{2}({\theta }_{k}){\sigma }_{{w}_{{BB}}^{\mathrm{obs}}}^{2}({\theta }_{k})\end{array}\right]}^{T},\end{array}\end{eqnarray} \tag{ 29 }$$

where ${\left\{{[{D}_{{\rm{S}}}]}^{-1}\right\}}_{{ij}}^{2}$ denotes that matrix resulting from squaring each individual coefficient in the matrix ${[{D}_{{\rm{S}}}]}^{-1}$. We also note that the above derivation assumes no covariance between the observed correlations (i.e., ${w}_{\alpha \beta }^{\mathrm{obs}}$), which is incorrect for the case of neighboring redshift bins, given the shared LSS between them; this is discussed in more detail when we discuss the covariance matrices in Section 5.2. To consider the covariance matrix for the Decontaminated estimators, we start with Equation (12), which is reproduced here:

$$\begin{eqnarray}\begin{array}{ll}{\left[{\widehat{w}}_{{AA}}({\theta }_{k}){\widehat{w}}_{{AB}}({\theta }_{k}){\widehat{w}}_{{BB}}({\theta }_{k})\right]}^{T} & =\,{\left[{D}_{{\rm{S}}}\right]}^{-1}{\left[{w}_{{AA}}^{\mathrm{obs}}({\theta }_{k}){w}_{{AB}}^{\mathrm{obs}}({\theta }_{k}){w}_{{BB}}^{\mathrm{obs}}({\theta }_{k})\right]}^{T}.\end{array}\end{eqnarray} \tag{ 30 }$$

Given Equation (28), we therefore have

$$\begin{eqnarray}\begin{array}{ll}\left\langle {\left[{\widehat{w}}_{{AA}}({\theta }_{k}){\widehat{w}}_{{AB}}({\theta }_{k}){\widehat{w}}_{{BB}}({\theta }_{k})\right]}^{T}\right\rangle & =\,{\left[{D}_{{\rm{S}}}\right]}^{-1}\left\langle {\left[{w}_{{AA}}^{\mathrm{obs}}({\theta }_{k}){w}_{{AB}}^{\mathrm{obs}}({\theta }_{k}){w}_{{BB}}^{\mathrm{obs}}({\theta }_{k})\right]}^{T}\right\rangle ,\end{array}\end{eqnarray} \tag{ 31 }$$

where we assume that [D_S] is constant across the samples over which the expectation value is calculated. Now, using the above equations, we can write the variations in the estimators from their expectation value ($\equiv {\rm{\Delta }}w\equiv w-\left\langle w\right\rangle$) as

$$\begin{eqnarray}&&\begin{array}{l}{\left[{\rm{\Delta }}{\widehat{w}}_{{AA}}({\theta }_{k}){\rm{\Delta }}{\widehat{w}}_{{AB}}({\theta }_{k}){\rm{\Delta }}{\widehat{w}}_{{BB}}({\theta }_{k})\right]}^{T}\,=\,{\left[{D}_{{\rm{S}}}\right]}^{-1}{\left[{\rm{\Delta }}{w}_{{AA}}^{\mathrm{obs}}({\theta }_{k}){\rm{\Delta }}{w}_{{AB}}^{\mathrm{obs}}({\theta }_{k}){\rm{\Delta }}{w}_{{BB}}^{\mathrm{obs}}({\theta }_{k})\right]}^{T}.\end{array}\end{eqnarray} \tag{ 32 }$$

Now, defining ${C}_{{\widehat{w}}_{}}({\theta }_{k})$ as the covariance matrix for the Decontaminated estimators ${\widehat{w}}_{\alpha \beta }({\theta }_{k})$, we have

$$\begin{eqnarray}&&\begin{array}{l}{C}_{{\widehat{w}}_{}}({\theta }_{k})=\left\langle {\left[{\rm{\Delta }}{\widehat{w}}_{{AA}}({\theta }_{k}){\rm{\Delta }}{\widehat{w}}_{{AB}}({\theta }_{k}){\rm{\Delta }}{\widehat{w}}_{{BB}}({\theta }_{k})\right]}^{T}\right.\,\left.\left[{\rm{\Delta }}{\widehat{w}}_{{AA}}({\theta }_{k})\ \ {\rm{\Delta }}{\widehat{w}}_{{AB}}({\theta }_{k})\ \ {\rm{\Delta }}{\widehat{w}}_{{BB}}({\theta }_{k})\right]\right\rangle .\end{array}\end{eqnarray} \tag{ 33 }$$

Using Equation (32) and its transpose, we then have

$$\begin{eqnarray}&&\begin{array}{l}{C}_{{\widehat{w}}_{}}({\theta }_{k})=\left\langle {\left[{D}_{{\rm{S}}}\right]}^{-1}{\left[{\rm{\Delta }}{w}_{{AA}}^{\mathrm{obs}}({\theta }_{k}){\rm{\Delta }}{w}_{{AB}}^{\mathrm{obs}}({\theta }_{k}){\rm{\Delta }}{w}_{{BB}}^{\mathrm{obs}}({\theta }_{k})\right]}^{T}\right.\left.\,\left[{\rm{\Delta }}{w}_{{AA}}^{\mathrm{obs}}({\theta }_{k})\ \ {\rm{\Delta }}{w}_{{AB}}^{\mathrm{obs}}({\theta }_{k})\ \ {\rm{\Delta }}{w}_{{BB}}^{\mathrm{obs}}({\theta }_{k})\right]{\left[{\left[{D}_{{\rm{S}}}\right]}^{-1}\right]}^{T}\right\rangle \\ \,=\,{\left[{D}_{{\rm{S}}}\right]}^{-1}\left\langle {\left[{\rm{\Delta }}{w}_{{AA}}^{\mathrm{obs}}({\theta }_{k}){\rm{\Delta }}{w}_{{AB}}^{\mathrm{obs}}({\theta }_{k}){\rm{\Delta }}{w}_{{BB}}^{\mathrm{obs}}({\theta }_{k})\right]}^{T}\right.\left.\,\left[{\rm{\Delta }}{w}_{{AA}}^{\mathrm{obs}}({\theta }_{k})\ \ {\rm{\Delta }}{w}_{{AB}}^{\mathrm{obs}}({\theta }_{k})\ \ {\rm{\Delta }}{w}_{{BB}}^{\mathrm{obs}}({\theta }_{k})\right]\right\rangle {\left[{\left[{D}_{{\rm{S}}}\right]}^{-1}\right]}^{T}\\ \,=\,{\left[{D}_{{\rm{S}}}\right]}^{-1}{C}_{{w}^{\mathrm{obs}}}({\theta }_{k}){\left[{\left[{D}_{{\rm{S}}}\right]}^{-1}\right]}^{T},\end{array}\end{eqnarray} \tag{ 34 }$$

where ${C}_{{w}^{\mathrm{obs}}}$ is covariance matrix for the observed correlations, ${w}_{\alpha \beta }^{\mathrm{obs}}$. Note that the second equality is valid only under the assumption that [D_S] is constant.

Both ${C}_{{w}^{\mathrm{obs}}}({\theta }_{k})$ and ${C}_{{\widehat{w}}_{}}({\theta }_{k})$ can be determined via bootstrap, as done for the example considered in Section 5.2, with the estimated covariance matrices presented in Figures 11 and 17. We note that ${C}_{{\widehat{w}}_{}}({\theta }_{k})$ may be calculated using ${C}_{{w}^{\mathrm{obs}}}({\theta }_{k})$ given Equation (34), assuming that $[{D}_{{\rm{S}}}]$ is constant across the bootstrapped samples. We also that one can construct covariance matrices for both ${w}^{\mathrm{obs}}$ and ${\widehat{w}}_{}$ spanning all θ bins via a block combination of the θ-dependent matrices presented here; these larger matrices are only block diagonal to the extent that individual CFs are uncorrelated between neighboring θ bins. Finally, as a simple check of the expression in Equation (34), we note that if ${C}_{{w}^{\mathrm{obs}}}({\theta }_{k})$ is diagonal, i.e., there are no covariances in the observed correlations, Equation (34) leads to the variance in the Decontaminated estimators as given by Equation (29).

We extend the treatment in Appendix A.1 to consider an unweighted full sample. The analog of Equation (20) is then

$$\begin{eqnarray}\begin{array}{rclrcl}{dP}({\theta }_{k}) & = & {{ \mathcal N }}_{{\mathrm{tot}}_{\mathrm{obs}}}\left[1+{w}^{\mathrm{full}}({\theta }_{k})\right]\displaystyle \frac{d{\rm{\Omega }}}{V}\displaystyle \frac{d{\rm{\Omega }}}{V} & & = & \displaystyle \sum _{\gamma ,\delta }{{ \mathcal N }}_{{\mathrm{tot}}_{\mathrm{obs}}}^{\gamma \delta ,\mathrm{true}}[1+{w}_{\gamma ,\delta }^{\mathrm{true}}({\theta }_{k})]\displaystyle \frac{d{\rm{\Omega }}}{V}\displaystyle \frac{d{\rm{\Omega }}}{V}.\end{array}\end{eqnarray} \tag{ 35 }$$

Note that we have dropped the α, β markers since there is only one correlation that can be measured for the unweighted full sample. Expanding the sum, we have

$$\begin{eqnarray}&&\begin{array}{l}{{ \mathcal N }}_{{\mathrm{tot}}_{\mathrm{obs}}}\left[1+{w}^{\mathrm{full}}({\theta }_{k})\right]={{ \mathcal N }}_{{\mathrm{tot}}_{\mathrm{obs}}}^{11,\mathrm{true}}\left[1+{w}_{11}^{\mathrm{true}}({\theta }_{k})\right]+{{ \mathcal N }}_{{\mathrm{tot}}_{\mathrm{obs}}}^{12,\mathrm{true}}\left[1+{w}_{12}^{\mathrm{true}}({\theta }_{k})\right]\\ \,+\,{{ \mathcal N }}_{{\mathrm{tot}}_{\mathrm{obs}}}^{21,\mathrm{true}}\left[1+{w}_{21}^{\mathrm{true}}({\theta }_{k})\right]+{{ \mathcal N }}_{{\mathrm{tot}}_{\mathrm{obs}}}^{22,\mathrm{true}}\left[1+{w}_{22}^{\mathrm{true}}({\theta }_{k})\right].\end{array}\end{eqnarray} \tag{ 36 }$$

Now, if we assume that our classification probabilities are unbiased, we can write

$$\begin{eqnarray}&&\displaystyle \sum _{i}^{{N}_{{\mathrm{tot}}_{\mathrm{obs}}}^{\gamma }}\displaystyle \sum _{j\ne i}^{{N}_{{\mathrm{tot}}_{\mathrm{obs}}}^{\delta }}{{\mathtt{q}}}_{i}^{\gamma }{{\mathtt{q}}}_{j}^{\delta }={\widehat{{ \mathcal N }}}_{{\mathrm{tot}}_{\mathrm{obs}}}^{\gamma \delta ,\mathrm{true}}.\end{eqnarray} \tag{ 37 }$$

Note that technically ${N}_{{\mathrm{tot}}_{\mathrm{obs}}}^{\gamma }={N}_{{\mathrm{tot}}_{\mathrm{obs}}}^{\delta }={N}_{{\mathrm{tot}}_{\mathrm{obs}}}$, but we keep $\gamma ,\delta$ tags just to keep track of samples when reducing to Decontaminated. Now, simplifying the equation above, we have

$$\begin{eqnarray}&&\begin{array}{l}{{ \mathcal N }}_{{\mathrm{tot}}_{\mathrm{obs}}}\left[1+{w}^{\mathrm{full}}({\theta }_{k})\right]=\displaystyle \sum _{i}^{{N}_{{\mathrm{tot}}_{\mathrm{obs}}}^{1}}\displaystyle \sum _{j\ne i}^{{N}_{{\mathrm{tot}}_{\mathrm{obs}}}^{1}}{{\mathtt{q}}}_{i}^{1}{{\mathtt{q}}}_{j}^{1}\left[1+{w}_{11}^{\mathrm{true}}({\theta }_{k})\right]+\displaystyle \sum _{i}^{{N}_{{\mathrm{tot}}_{\mathrm{obs}}}^{1}}\displaystyle \sum _{j\ne i}^{{N}_{{\mathrm{tot}}_{\mathrm{obs}}}^{2}}{{\mathtt{q}}}_{i}^{1}{{\mathtt{q}}}_{j}^{2}\left[1+{w}_{12}^{\mathrm{true}}({\theta }_{k})\right]\\ \,+\,\displaystyle \sum _{i}^{{N}_{{\mathrm{tot}}_{\mathrm{obs}}}^{2}}\displaystyle \sum _{j\ne i}^{{N}_{{\mathrm{tot}}_{\mathrm{obs}}}^{1}}{{\mathtt{q}}}_{i}^{2}{{\mathtt{q}}}_{j}^{1}\left[1+{w}_{21}^{\mathrm{true}}({\theta }_{k})\right]+\displaystyle \sum _{i}^{{N}_{{\mathrm{tot}}_{\mathrm{obs}}}^{2}}\displaystyle \sum _{j\ne i}^{{N}_{{\mathrm{tot}}_{\mathrm{obs}}}^{2}}{{\mathtt{q}}}_{i}^{2}{{\mathtt{q}}}_{j}^{2}\left[1+{w}_{22}^{\mathrm{true}}({\theta }_{k})\right].\end{array}\end{eqnarray} \tag{ 38 }$$

We now check what happens when we reduce the above equation to Decontaminated, i.e., we consider not the full sample but the target subsamples, while all the probabilities are represented by their averages. Thus, for α, β = 1, 2, Equation (38) becomes

$$\begin{eqnarray}&&\begin{array}{l}{N}_{1,\mathrm{obs}}{N}_{2,\mathrm{obs}}\left[1+{w}_{11}^{\mathrm{obs}}({\theta }_{k})\right]=\left(\displaystyle \sum _{i}^{{N}_{1,\mathrm{obs}}}\displaystyle \sum _{j\ne i}^{{N}_{2,\mathrm{obs}}}{{\mathtt{q}}}_{i}^{1}{{\mathtt{q}}}_{j}^{1}\right)\left[1+{w}_{11}^{\mathrm{true}}({\theta }_{k})\right]+\,\left(\displaystyle \sum _{i}^{{N}_{1,\mathrm{obs}}}\displaystyle \sum _{j\ne i}^{{N}_{2,\mathrm{obs}}}{{\mathtt{q}}}_{i}^{1}{{\mathtt{q}}}_{j}^{2}\right)\left[1+{w}_{12}^{\mathrm{true}}({\theta }_{k})\right]\\ \,+\,\left(\displaystyle \sum _{i}^{{N}_{1,\mathrm{obs}}}\displaystyle \sum _{j\ne i}^{{N}_{2,\mathrm{obs}}}{{\mathtt{q}}}_{i}^{2}{{\mathtt{q}}}_{j}^{1}\right)\left[1+{w}_{21}^{\mathrm{true}}({\theta }_{k})\right]+\,\left(\displaystyle \sum _{i}^{{N}_{1,\mathrm{obs}}}\displaystyle \sum _{j\ne i}^{{N}_{2,\mathrm{obs}}}{{\mathtt{q}}}_{i}^{2}{{\mathtt{q}}}_{j}^{2}\right)\,\left[1+{w}_{22}^{\mathrm{true}}({\theta }_{k})\right]\\ \,=\,\left(\displaystyle \sum _{i}^{{N}_{1,\mathrm{obs}}}\displaystyle \sum _{j}^{{N}_{2,\mathrm{obs}}}{{\mathtt{q}}}_{i}^{1}{{\mathtt{q}}}_{j}^{1}\right)\left[1+{w}_{11}^{\mathrm{true}}({\theta }_{k})\right]+\,\left(\displaystyle \sum _{i}^{{N}_{1,\mathrm{obs}}}\displaystyle \sum _{j}^{{N}_{2,\mathrm{obs}}}{{\mathtt{q}}}_{i}^{1}{{\mathtt{q}}}_{j}^{2}\right)\,\left[1+{w}_{12}^{\mathrm{true}}({\theta }_{k})\right]\\ \,+\,\left(\displaystyle \sum _{i}^{{N}_{1,\mathrm{obs}}}\displaystyle \sum _{j}^{{N}_{2,\mathrm{obs}}}{{\mathtt{q}}}_{i}^{2}{{\mathtt{q}}}_{j}^{1}\right)\left[1+{w}_{21}^{\mathrm{true}}({\theta }_{k})\right]+\,\left(\displaystyle \sum _{i}^{{N}_{1,\mathrm{obs}}}\displaystyle \sum _{j}^{{N}_{2,\mathrm{obs}}}{{\mathtt{q}}}_{i}^{2}{{\mathtt{q}}}_{j}^{2}\right)\left[1+{w}_{22}^{\mathrm{true}}({\theta }_{k})\right]\\ \,\mathop{\to }\limits^{{\rm{simplify}}\,q{\rm{s}}}\,\left(\displaystyle \sum _{i}^{{N}_{1,\mathrm{obs}}}\displaystyle \sum _{j}^{{N}_{2,\mathrm{obs}}}{q}_{i,11}{q}_{j,12}\right)\left[1+{w}_{11}^{\mathrm{true}}({\theta }_{k})\right]+\,\left(\displaystyle \sum _{i}^{{N}_{1,\mathrm{obs}}}\displaystyle \sum _{j}^{{N}_{2,\mathrm{obs}}}{q}_{i,11}{q}_{j,22}\right)\left[1+{w}_{12}^{\mathrm{true}}({\theta }_{k})\right]\\ \,+\,\left(\displaystyle \sum _{i}^{{N}_{1,\mathrm{obs}}}\displaystyle \sum _{j}^{{N}_{2,\mathrm{obs}}}{q}_{i,12}{q}_{j,21}\right)\left[1+{w}_{21}^{\mathrm{true}}({\theta }_{k})\right]+\,\left(\displaystyle \sum _{i}^{{N}_{1,\mathrm{obs}}}\displaystyle \sum _{j}^{{N}_{2,\mathrm{obs}}}{q}_{i,12}{q}_{j,22}\right)\left[1+{w}_{22}^{\mathrm{true}}({\theta }_{k})\right]\\ \,\mathop{\to }\limits^{q{\rm{s}}=f{\rm{s}}}\,\left({f}_{11}{f}_{21}\displaystyle \sum _{i}^{{N}_{1,\mathrm{obs}}}\displaystyle \sum _{j}^{{N}_{2,\mathrm{obs}}}\right)\left[1+{w}_{11}^{\mathrm{true}}({\theta }_{k})\right]+\left({f}_{11}{f}_{22}\displaystyle \sum _{i}^{{N}_{1,\mathrm{obs}}}\displaystyle \sum _{j}^{{N}_{2,\mathrm{obs}}}\right)\left[1+{w}_{12}^{\mathrm{true}}({\theta }_{k})\right]\\ \,+\,\left({f}_{12}{f}_{21}\displaystyle \sum _{i}^{{N}_{1,\mathrm{obs}}}\displaystyle \sum _{j}^{{N}_{2,\mathrm{obs}}}\right)\left[1+{w}_{21}^{\mathrm{true}}({\theta }_{k})\right]+\,\left({f}_{12}{f}_{22}\displaystyle \sum _{i}^{{N}_{1,\mathrm{obs}}}\displaystyle \sum _{j}^{{N}_{2,\mathrm{obs}}}\right)\left[1+{w}_{22}^{\mathrm{true}}({\theta }_{k})\right]\\ \,=\,{f}_{11}{f}_{21}{N}_{1,\mathrm{obs}}{N}_{2,\mathrm{obs}}\left[1+{w}_{11}^{\mathrm{true}}({\theta }_{k})\right]+\,{f}_{11}{f}_{22}{N}_{1,\mathrm{obs}}{N}_{2,\mathrm{obs}}\left[1+{w}_{12}^{\mathrm{true}}({\theta }_{k})\right]\\ \,+\,{f}_{12}{f}_{21}{N}_{1,\mathrm{obs}}{N}_{2,\mathrm{obs}}\left[1+{w}_{21}^{\mathrm{true}}({\theta }_{k})\right]+{f}_{12}{f}_{22}{N}_{1,\mathrm{obs}}{N}_{2,\mathrm{obs}}\left[1+{w}_{22}^{\mathrm{true}}({\theta }_{k})\right]\end{array}\end{eqnarray} \tag{ 39 }$$

$$\begin{eqnarray}&&\Rightarrow \,\left[1+{w}_{12}^{\mathrm{obs}}({\theta }_{k})\right]\,=\,{f}_{11}{f}_{21}\left[1+{w}_{11}^{\mathrm{true}}({\theta }_{k})\right]+\left\{{f}_{11}{f}_{22}+{f}_{12}{f}_{21}\right\}\left[1+{w}_{12}^{\mathrm{true}}({\theta }_{k})\right]\\&&+{f}_{12}{f}_{22}\left[1+{w}_{22}^{\mathrm{true}}({\theta }_{k})\right],\end{eqnarray} \tag{ 40 }$$

which agrees with Equation (26). Similar results follow for (α, β) = (1, 1) = (2, 2).

We now extend the analysis above further for the weighted (biased) estimator:

$$\begin{eqnarray}&&d{\widetilde{P}}_{\alpha \beta }({\theta }_{k})={\widetilde{{ \mathcal N }}}_{{\mathrm{tot}}_{\mathrm{obs}}}^{\alpha \beta ,\mathrm{obs}}\left[1+{\widetilde{w}}_{\alpha \beta }({\theta }_{k})\right]\displaystyle \frac{d{\rm{\Omega }}}{V}\displaystyle \frac{d{\rm{\Omega }}}{V},\end{eqnarray} \tag{ 41 }$$

where we introduce $\widetilde{{ \mathcal N }}$ to account for the weighted pair counts, which we define as

$$\begin{eqnarray}&&{\widetilde{{ \mathcal N }}}_{{\mathrm{tot}}_{\mathrm{obs}}}^{\alpha \beta ,\mathrm{obs}}=\displaystyle \sum _{i}^{{N}_{{\mathrm{tot}}_{\mathrm{obs}}}^{\alpha }}\displaystyle \sum _{j\ne i}^{{N}_{{\mathrm{tot}}_{\mathrm{obs}}}^{\beta }}{{\mathtt{w}}}_{{ij}}^{\alpha \beta }.\end{eqnarray} \tag{ 42 }$$

Now, when writing the analog of Equations (20)–(35), we need to account for pair weights, leading us to

$$\begin{eqnarray}\begin{array}{rclrcl}d{\widetilde{P}}_{\alpha \beta }({\theta }_{k}) & = & {\widetilde{{ \mathcal N }}}_{{\mathrm{tot}}_{\mathrm{obs}}}^{\alpha \beta ,\mathrm{obs}}\left[1+{\widetilde{w}}_{\alpha \beta }({\theta }_{k})\right]\displaystyle \frac{d{\rm{\Omega }}}{V}\displaystyle \frac{d{\rm{\Omega }}}{V} & & = & \displaystyle \sum _{\gamma ,\delta }{\widetilde{{ \mathcal N }}}_{{\mathrm{tot}}_{\mathrm{obs}}}^{\gamma \delta ,\mathrm{true}}[1+{w}_{\gamma ,\delta }^{\mathrm{true}}({\theta }_{k})]\displaystyle \frac{d{\rm{\Omega }}}{V}\displaystyle \frac{d{\rm{\Omega }}}{V},\end{array}\end{eqnarray} \tag{ 43 }$$

where we have the analog of Equation (37):

$$\begin{eqnarray}&&\displaystyle \sum _{i}^{{N}_{{\mathrm{tot}}_{\mathrm{obs}}}^{\alpha }}\displaystyle \sum _{j\ne i}^{{N}_{{\mathrm{tot}}_{\mathrm{obs}}}^{\beta }}{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{q}}}_{i}^{\alpha }{{\mathtt{q}}}_{j}^{\beta }={\widehat{\widetilde{{ \mathcal N }}}}_{{\mathrm{tot}}_{\mathrm{obs}}}^{\alpha \beta ,\mathrm{true}}.\end{eqnarray} \tag{ 44 }$$

Now, expanding the sum in Equation (43), we have

$$\begin{eqnarray}\begin{array}{rcl}{\widetilde{{ \mathcal N }}}_{{\mathrm{tot}}_{\mathrm{obs}}}^{\alpha \beta ,\mathrm{obs}}\left[1+{\widetilde{w}}_{\alpha \beta }({\theta }_{k})\right] & = & {\widetilde{{ \mathcal N }}}_{{\mathrm{tot}}_{\mathrm{obs}}}^{11,\mathrm{true}}\left[1+{w}_{11}^{\mathrm{true}}({\theta }_{k})\right]+{\widetilde{{ \mathcal N }}}_{{\mathrm{tot}}_{\mathrm{obs}}}^{12,\mathrm{true}}\left[1+{w}_{12}^{\mathrm{true}}({\theta }_{k})\right]\\ & & +{\widetilde{{ \mathcal N }}}_{{\mathrm{tot}}_{\mathrm{obs}}}^{21,\mathrm{true}}\left[1+{w}_{21}^{\mathrm{true}}({\theta }_{k})\right]\,+{\widetilde{{ \mathcal N }}}_{{\mathrm{tot}}_{\mathrm{obs}}}^{22,\mathrm{true}}\left[1+{w}_{22}^{\mathrm{true}}({\theta }_{k})\right].\end{array}\end{eqnarray} \tag{ 45 }$$

Substituting Equation (37) to estimate the true counts, we have

$$\begin{eqnarray}&&\begin{array}{l}\left(\displaystyle \sum _{i}^{{N}_{{\mathrm{tot}}_{\mathrm{obs}}}^{\alpha }}\displaystyle \sum _{j\ne i}^{{N}_{{\mathrm{tot}}_{\mathrm{obs}}}^{\beta }}{{\mathtt{w}}}_{{ij}}^{\alpha \beta }\right)\left[1+{\widetilde{w}}_{\alpha \beta }^{\mathrm{full}}({\theta }_{k})\right]\,=\,\left(\displaystyle \sum _{i}^{{N}_{{\mathrm{tot}}_{\mathrm{obs}}}^{\alpha }}\displaystyle \sum _{j\ne i}^{{N}_{{\mathrm{tot}}_{\mathrm{obs}}}^{\beta }}{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{q}}}_{i}^{1}{{\mathtt{q}}}_{j}^{1}\right)\left[1+\,{w}_{11}^{\mathrm{true}}({\theta }_{k})\right]\\+\,\left(\displaystyle \sum _{i}^{{N}_{{\mathrm{tot}}_{\mathrm{obs}}}^{\alpha }}\displaystyle \sum _{j\ne i}^{{N}_{{\mathrm{tot}}_{\mathrm{obs}}}^{\beta }}{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{q}}}_{i}^{1}{{\mathtt{q}}}_{j}^{2}\right)\left[1+{w}_{12}^{\mathrm{true}}({\theta }_{k})\right] \,+\,\left(\displaystyle \sum _{i}^{{N}_{{\mathrm{tot}}_{\mathrm{obs}}}^{\alpha }}\displaystyle \sum _{j\ne i}^{{N}_{{\mathrm{tot}}_{\mathrm{obs}}}^{\beta }}{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{q}}}_{i}^{2}{{\mathtt{q}}}_{j}^{1}\right)\left[1+{w}_{21}^{\mathrm{true}}({\theta }_{k})\right]\\ \,+\,\left(\displaystyle \sum _{i}^{{N}_{{\mathrm{tot}}_{\mathrm{obs}}}^{\alpha }}\displaystyle \sum _{j\ne i}^{{N}_{{\mathrm{tot}}_{\mathrm{obs}}}^{\beta }}{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{q}}}_{i}^{2}{{\mathtt{q}}}_{j}^{2}\right)\left[1+{w}_{22}^{\mathrm{true}}({\theta }_{k})\right].\end{array}\end{eqnarray} \tag{ 46 }$$

Note that this equation reduces to Decontaminated as in Equation (39) when weights are set to 1 for target subsample and 0 for the rest, and that we basically have theta-independent decontamination.

Here, we follow the procedure in LS93 to estimate the variance of the Weighted estimator introduced in Equation (13), filling in additional details while accounting for the weights in the data–data pair counts. While the details may be of value to the interested reader, we note that the derivation is lengthy, culminating in the analytical expression for the variance in Appendix C.1.6. Specifically, we write the pair counts, i.e., the unnormalized $\overline{{DD}}$, $\overline{{RR}}$ histograms in terms of the fluctuations about their means, i.e., we have

$$\begin{eqnarray}&&\begin{array}{r}{\left(\overline{{DD}}\right)}_{\alpha \beta }({\theta }_{k})\,=\,\left\langle {\left(\overline{{DD}}\right)}_{\alpha \beta }({\theta }_{k})\right\rangle (1+\eta ({\theta }_{k}))\\ (\overline{{RR}})({\theta }_{k})\,=\,\left\langle (\overline{{RR}})({\theta }_{k})\right\rangle (1+\gamma ({\theta }_{k})),\end{array}\end{eqnarray} \tag{ 47 }$$

where we use the overline to distinguish the unnormalized histograms from the normalized ones (denoted with a tilde). Here, η and γ are the fluctuations in the histograms about their means, which follows

$$\begin{eqnarray}&&\left\langle \eta ({\theta }_{k})\right\rangle =\left\langle \gamma ({\theta }_{k})\right\rangle =0,\end{eqnarray} \tag{ 48 }$$

and hence, we have

where $\left\langle \eta ({\theta }_{k})\gamma ({\theta }_{k})\right\rangle =0$ since the data and random catalogs are not correlated. Note that η here is the same as α in LS93; we choose the former given that the latter letter is already in use here. Thus, given Equations (13) and (47), we have

$$\begin{eqnarray}\begin{array}{l}1+{\widetilde{w}}_{\alpha \beta }({\theta }_{k})&=\displaystyle \frac{{\left(\widetilde{{DD}}\right)}_{\alpha \beta }({\theta }_{k})}{{RR}({\theta }_{k})}\,=\,\displaystyle \frac{{\left(\overline{{DD}}\right)}_{\alpha \beta }({\theta }_{k})}{{\displaystyle \sum }_{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{\alpha \beta }}\displaystyle \frac{{N}_{r}({N}_{r}-1)/2}{(\overline{{RR}})({\theta }_{k})}\\&=\,\displaystyle \frac{{N}_{r}({N}_{r}-1)}{2{\displaystyle \sum }_{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{\alpha \beta }}\displaystyle \frac{\left\langle {\left(\overline{{DD}}\right)}_{\alpha \beta }({\theta }_{k})\right\rangle (1+\eta ({\theta }_{k}))}{\left\langle (\overline{{RR}})({\theta }_{k})\right\rangle (1+\gamma ({\theta }_{k}))},\end{array}\end{eqnarray} \tag{ 50 }$$

where we have collapsed the double sums for brevity, and have defined

$$\begin{eqnarray}&&{RR}({\theta }_{k})=\displaystyle \frac{{\displaystyle \sum }_{i}^{{N}_{r}}{\displaystyle \sum }_{j\gt i}^{{N}_{r}}{\bar{{\rm{\Theta }}}}_{{ij},k}}{{\displaystyle \sum }_{i}^{{N}_{r}}{\displaystyle \sum }_{j\gt i}^{{N}_{r}}}=\displaystyle \frac{{\displaystyle \sum }_{i}^{{N}_{r}}{\displaystyle \sum }_{j\gt i}^{{N}_{r}}{\bar{{\rm{\Theta }}}}_{{ij},k}}{{N}_{r}({N}_{r}-1)/2}\end{eqnarray} \tag{ 51 }$$

$$\begin{eqnarray}&&\begin{array}{l}\Rightarrow 1+\left\langle {\widetilde{w}}_{\alpha \beta }({\theta }_{k})\right\rangle \,=\left\langle \displaystyle \frac{{N}_{r}({N}_{r}-1)}{2{\displaystyle \sum }_{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{\alpha \beta }}\displaystyle \frac{\left\langle {\left(\overline{{DD}}\right)}_{\alpha \beta }({\theta }_{k})\right\rangle (1+\eta ({\theta }_{k}))}{\left\langle (\overline{{RR}})\right\rangle ({\theta }_{k})(1+\gamma ({\theta }_{k}))}\right\rangle \\ \,=\,\displaystyle \frac{{N}_{r}({N}_{r}-1)}{2}\displaystyle \frac{\left\langle {\left(\overline{{DD}}\right)}_{\alpha \beta }({\theta }_{k})\right\rangle }{\left\langle (\overline{{RR}})({\theta }_{k})\right\rangle }\left\langle \displaystyle \frac{1}{{\displaystyle \sum }_{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{\alpha \beta }}\right\rangle \left\langle \displaystyle \frac{(1+\eta ({\theta }_{k}))}{(1+\gamma ({\theta }_{k}))}\right\rangle \\ \,\approx \,\displaystyle \frac{{N}_{r}({N}_{r}-1)}{2{\displaystyle \sum }_{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{\alpha \beta }}\displaystyle \frac{\left\langle {\left(\overline{{DD}}\right)}_{\alpha \beta }({\theta }_{k})\right\rangle }{\left\langle (\overline{{RR}})({\theta }_{k})\right\rangle }\,\left\langle (1+\eta ({\theta }_{k}))(1-\gamma ({\theta }_{k})+{\gamma }^{2}({\theta }_{k}))\right\rangle \\ \,=\,\displaystyle \frac{{N}_{r}({N}_{r}-1)}{2{\displaystyle \sum }_{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{\alpha \beta }}\displaystyle \frac{\left\langle {\left(\overline{{DD}}\right)}_{\alpha \beta }({\theta }_{k})\right\rangle }{\left\langle (\overline{{RR}})({\theta }_{k})\right\rangle }\,\left\langle 1-\gamma ({\theta }_{k})+{\gamma }^{2}({\theta }_{k})+\eta ({\theta }_{k})-\eta ({\theta }_{k})\gamma ({\theta }_{k})+\eta ({\theta }_{k}){\gamma }^{2}({\theta }_{k})\right\rangle ,\end{array}\end{eqnarray} \tag{ 52 }$$

where we only keep the terms up to the second order in fluctuations. Note that the second equality is justified since the weights for individual galaxies are fixed across the different realizations. Now, we calculate the variance of the estimator as

where, again, we only keep the terms up to the second order in fluctuations. Here, as derived from Equation (47), we have the second moments of the fluctuations defined as

$$\begin{eqnarray}&&\left\langle {\eta }^{2}({\theta }_{k})\right\rangle =\displaystyle \frac{\left\langle {\left(\overline{{DD}}\right)}_{\alpha \beta }({\theta }_{k})\cdot {\left(\overline{{DD}}\right)}_{\alpha \beta }({\theta }_{k})\right\rangle -{\left\langle {\left(\overline{{DD}}\right)}_{\alpha \beta }\left({\theta }_{k}\right)\right\rangle }^{2}}{{\left\langle {\left(\overline{{DD}}\right)}_{\alpha \beta }({\theta }_{k})\right\rangle }^{2}},\end{eqnarray} \tag{ 54 }$$

$$\begin{eqnarray}&&\left\langle {\gamma }^{2}({\theta }_{k})\right\rangle =\displaystyle \frac{\left\langle (\overline{{RR}})({\theta }_{k})\cdot (\overline{{RR}})({\theta }_{k})\right\rangle -{\left\langle (\overline{{RR}})({\theta }_{k})\right\rangle }^{2}}{{\left\langle (\overline{{RR}})({\theta }_{k})\right\rangle }^{2}}.\end{eqnarray} \tag{ 55 }$$

In order to evaluate the variance, we calculate the second moments of the fluctuations using the first and second moments of the pair counts. Specifically, we only need $\left\langle (\overline{{RR}})({\theta }_{k})\right\rangle$, $\left\langle {(\overline{{DD}})}_{\alpha \beta }({\theta }_{k})\right\rangle$, and $\left\langle {(\overline{{DD}})}_{\alpha \beta }\cdot {(\overline{{DD}})}_{\alpha \beta }({\theta }_{k})\right\rangle ;$ we do not need the second moment of the random pair counts, since $\left\langle {\gamma }^{2}\right\rangle$ is simply the variance of the random data and hence the variance of the Poisson distribution.

C.1.1. Pair Counts: First and Second Moments

C.1.2. Random Pairs: First Moment

Here, we consider ${N}_{r}$ points distributed randomly over the survey area, which we divide into K cells. The probability of finding the ith random point in any cell is the continuum probability, $\left\langle {\rho }_{j}\right\rangle ={N}_{r}/K$, in the limit of large enough K that we essentially have either zero or one point in each cell. This follows that the number of random pairs is

$$\begin{eqnarray}&&\left\langle (\overline{{RR}})({\theta }_{k})\right\rangle =\left\langle \displaystyle \sum _{j\lt i}^{K}{\rho }_{i}{\rho }_{j}{\bar{{\rm{\Theta }}}}_{{ij},k}\right\rangle =\displaystyle \frac{1}{2}\displaystyle \sum _{i\ne j}^{K}\left\langle {\rho }_{i}{\rho }_{j}\right\rangle {\bar{{\rm{\Theta }}}}_{{ij},k},\end{eqnarray} \tag{ 56 }$$

where we have borrowed the notation introduced in Equation (5) to express the Heavisides. Now, the probability of finding two random points in two cells, chosen without replacement, is

$$\begin{eqnarray}&&\left\langle {\rho }_{i}{\rho }_{j}\right\rangle =\displaystyle \frac{{N}_{r}({N}_{r}-1)}{K(K-1)},\end{eqnarray} \tag{ 57 }$$

and similar to LS93 Equation (10), we have

$$\begin{eqnarray}&&\displaystyle \sum _{i\ne j}^{K}{\bar{{\rm{\Theta }}}}_{{ij},k}=K(K-1){G}_{p}({\theta }_{k}),\end{eqnarray} \tag{ 58 }$$

where ${G}_{p}({\theta }_{k})$ is the probability of finding two random points at separations ${\theta }_{k}\pm d{\theta }_{k}/2$. Hence, ${\sum }_{i\ne j}^{K}{\bar{{\rm{\Theta }}}}_{{ij},k}$ is just the total number of random points with separations between ${\theta }_{\min ,k}$, ${\theta }_{\max ,k}$, as we have $K(K-1)$ cells. Substituting Equations (57) and (58) into Equation (56), we have

$$\begin{eqnarray}\begin{array}{rcl}\left\langle (\overline{{RR}})({\theta }_{k})\right\rangle & = & \displaystyle \frac{1}{2}\displaystyle \frac{{N}_{r}({N}_{r}-1)}{K(K-1)}\left[K(K-1){G}_{p}({\theta }_{k})\right]=\displaystyle \frac{{N}_{r}({N}_{r}-1)}{2}{G}_{p}({\theta }_{k}).\end{array}\end{eqnarray} \tag{ 59 }$$

C.1.3. Data Pairs: First Moment

Here, we have ${N}_{\mathrm{tot}}$ points distributed randomly over the survey area. As in Appendix C.1.2, the probability of finding a galaxy in any cell is $\left\langle \nu \right\rangle ={N}_{\mathrm{tot}}/K$, in the limit of large enough K that we essentially have either no galaxy or one galaxy in each cell. Furthermore, we assign the pair weight to the cells in which the pair falls. It follows, given Equation (14), that

$$\begin{eqnarray}\begin{array}{rclrcl}\left\langle {\left(\overline{{DD}}\right)}_{\alpha \beta }({\theta }_{k})\right\rangle & = & {C}_{{\rm{\Omega }}}\left\langle \displaystyle \sum _{i\ne j}^{K}{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{\nu }_{i}{\nu }_{j}{\bar{{\rm{\Theta }}}}_{{ij},k}\right\rangle & & = & {C}_{{\rm{\Omega }}}\displaystyle \sum _{i\ne j}^{K}\left\langle {{\mathtt{w}}}_{{ij}}^{\alpha \beta }\right\rangle \left\langle {\nu }_{i}{\nu }_{j}\right\rangle {\bar{{\rm{\Theta }}}}_{{ij},k},\end{array}\end{eqnarray} \tag{ 60 }$$

where C_Ω is a normalization constant to ensure that we recover the correct number of pairs, ${\sum }_{i\ne j}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{\alpha \beta }$, when integrating over all angles. Here, the pair weights are assumed to be uncorrelated with the probability of finding galaxies in a particular pair of cells, allowing us to separate their expectation values in the second equality; this assumption is valid since we are assigning pair weights based upon galaxy properties rather than their locations. Now, since data pairs are generally correlated, we must account for the correlation explicitly when considering the probabilities of finding a pair of galaxies in any two cells, chosen without replacement. That is, we have the probability of finding two galaxies in two cells separated by ${\theta }_{k}$, chosen without replacement, as

$$\begin{eqnarray}&&\left\langle {\nu }_{i}{\nu }_{j}\right\rangle \,=\,\displaystyle \frac{{N}_{\mathrm{tot}}({N}_{\mathrm{tot}}-1)}{K(K-1)}\left[1+{w}_{\alpha \beta }({\theta }_{k})\right].\end{eqnarray} \tag{ 61 }$$

Therefore, using Equations (58) and (61), Equation (60) becomes

$$\begin{eqnarray}\begin{array}{rcl}\left\langle {\left(\overline{{DD}}\right)}_{\alpha \beta }({\theta }_{k})\right\rangle & = & {C}_{{\rm{\Omega }}}\left\langle {{\mathtt{w}}}_{{ij}}^{\alpha \beta }\right\rangle {| }_{i\ne j}\displaystyle \frac{{N}_{\mathrm{tot}}({N}_{\mathrm{tot}}-1)}{K(K-1)}\left[1+{w}_{\alpha \beta }({\theta }_{k})\right]\left[K(K-1){G}_{p}({\theta }_{k})\right]\\ & = & {C}_{{\rm{\Omega }}}\left[\displaystyle \frac{{\displaystyle \sum }_{i\ne j}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{\alpha \beta }}{{N}_{\mathrm{tot}}({N}_{\mathrm{tot}}-1)}\right]\,\left[1+{w}_{\alpha \beta }({\theta }_{k})\right]{G}_{p}({\theta }_{k}){N}_{\mathrm{tot}}({N}_{\mathrm{tot}}-1)\\ & = & {C}_{{\rm{\Omega }}}\left[1+{w}_{\alpha \beta }({\theta }_{k})\right]{G}_{p}({\theta }_{k})\displaystyle \sum _{i\ne j}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{\alpha \beta }.\end{array}\end{eqnarray} \tag{ 62 }$$

Now, before finding the normalization constant, we define w_Ω as the mean of ${w}_{\alpha \beta }({\theta }_{k})$ over the sampling geometry, i.e.,

$$\begin{eqnarray}&&{w}_{{\rm{\Omega }}}\equiv {\int }_{{\rm{\Omega }}}{G}_{p}({\theta }_{k}){w}_{\alpha \beta }({\theta }_{k})d{\rm{\Omega }},\end{eqnarray} \tag{ 63 }$$

with ${G}_{p}({\theta }_{k})$ normalized to unity, i.e.,

$$\begin{eqnarray}&&{\int }_{{\rm{\Omega }}}{G}_{p}({\theta }_{k})d{\rm{\Omega }}=1.\end{eqnarray} \tag{ 64 }$$

Therefore, we have

$$\begin{eqnarray}&&\begin{array}{l}\,{\int }_{{\rm{\Omega }}}\left\langle {\left(\overline{{DD}}\right)}_{\alpha \beta }({\theta }_{k})\right\rangle d{\rm{\Omega }}=\displaystyle \sum _{i\ne j}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{\alpha \beta }\\ \Rightarrow \,{\int }_{{\rm{\Omega }}}{C}_{{\rm{\Omega }}}{G}_{p}({\theta }_{k})\left[1+{w}_{\alpha \beta }({\theta }_{k})\right]\displaystyle \sum _{i\ne j}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{\alpha \beta }=\displaystyle \sum _{i\ne j}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{\alpha \beta }\\ \,\Rightarrow \,{C}_{{\rm{\Omega }}}=\displaystyle \frac{1}{1+{w}_{{\rm{\Omega }}}},\end{array}\end{eqnarray} \tag{ 65 }$$

where we make use of Equation (64). Therefore, Equation (62) becomes

$$\begin{eqnarray}&&\left\langle {\left(\overline{{DD}}\right)}_{\alpha \beta }({\theta }_{k})\right\rangle ={G}_{p}({\theta }_{k})\left[\displaystyle \frac{1+{w}_{\alpha \beta }({\theta }_{k})}{1+{w}_{{\rm{\Omega }}}}\right]\displaystyle \sum _{i\ne j}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{\alpha \beta }.\end{eqnarray} \tag{ 66 }$$

C.1.4. Data–Data Pairs

As in LS93, using counts in cells, the second moment is defined as

$$\begin{eqnarray}&&\begin{array}{l}\left\langle {\left(\overline{{DD}}\right)}_{\alpha \beta }\cdot {\left(\overline{{DD}}\right)}_{\alpha \beta }({\theta }_{k})\right\rangle =\left\langle \displaystyle \sum _{j\ne i}^{K}{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{\nu }_{i}{\nu }_{j}{\bar{{\rm{\Theta }}}}_{{ij},k}\displaystyle \sum _{l\ne m}^{K}{{\mathtt{w}}}_{{ml}}^{\alpha \beta }{\nu }_{m}{\nu }_{l}{\bar{{\rm{\Theta }}}}_{{ml},k}\right\rangle \\ \,=\,\displaystyle \sum _{j\ne i}^{K}\displaystyle \sum _{l\ne m}^{K}\left\langle {\nu }_{i}{\nu }_{j}{\nu }_{m}{\nu }_{l}\right\rangle \left\langle {{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{ml}}^{\alpha \beta }\right\rangle {\bar{{\rm{\Theta }}}}_{{ij},k}{\bar{{\rm{\Theta }}}}_{{ml},k}.\end{array}\end{eqnarray} \tag{ 67 }$$

Now, there are three cases to consider, each of which needs to be normalized to give the right total weight from each case (as done in Appendix C.1.3):

• 1.  
  No indices overlap: there are $K(K-1)(K-2)(K-3)$ cases of the sort as we choose each of the four cells without replacement. Since the data pairs are correlated, the probability of finding each of the four galaxies in the four cells, chosen without replacement, is given by

  $$\begin{eqnarray}\left\langle {\nu }_{i}{\nu }_{j}{\nu }_{m}{\nu }_{l}\right\rangle &=&\displaystyle \frac{{N}_{\mathrm{tot}}({N}_{\mathrm{tot}}-1)({N}_{\mathrm{tot}}-2)({N}_{\mathrm{tot}}-3)}{K(K-1)(K-2)(K-3)}\\&\times&\left[1+{w}_{{ij}}({\theta }_{k})+{w}_{{im}}({\theta }_{k})+{w}_{{il}}({\theta }_{k})+{w}_{{jm}}({\theta }_{k})+{w}_{{jl}}({\theta }_{k})+{w}_{{ml}}({\theta }_{k})\right].\end{eqnarray} \tag{ 68 }$$

  Here, given that pairs $i,j$ and $m,l$ are separated by ${\theta }_{k}\pm d{\theta }_{k}/2$, ${w}_{{ij}}({\theta }_{k})={w}_{{ml}}({\theta }_{k})={w}_{\alpha \beta }({\theta }_{k})$, while the rest of the correlations can be approximated as w_Ω. Therefore,

  $$\begin{eqnarray}&&\left\langle {\nu }_{i}{\nu }_{j}{\nu }_{m}{\nu }_{l}\right\rangle \,=\,\displaystyle \frac{{N}_{\mathrm{tot}}({N}_{\mathrm{tot}}-1)({N}_{\mathrm{tot}}-2)({N}_{\mathrm{tot}}-3)}{K(K-1)(K-2)(K-3)}\left[1+2{w}_{\alpha \beta }({\theta }_{k})+4{w}_{{\rm{\Omega }}}\right].\end{eqnarray} \tag{ 69 }$$

  Also, as in LS93, we introduce ${G}_{q}({\theta }_{k})$ as the probability of finding quadrilaterals, i.e., pairs $i,j$ and $m,l$ separated by ${\theta }_{k}\pm d{\theta }_{k}/2$. Thus, the total number of quadrilaterals is

  $$\begin{eqnarray}&&\begin{array}{l}\displaystyle \sum _{\mathrm{unique}\{i,j,l,m\}}^{K}{\bar{{\rm{\Theta }}}}_{{ij},k}{\bar{{\rm{\Theta }}}}_{{ml},k}=K(K-1)(K-2)(K-3){G}_{q}({\theta }_{k}),\,i\ne j,m\ne l.\end{array}\end{eqnarray} \tag{ 70 }$$

  Note that as in Equation (64), ${G}_{q}({\theta }_{k})$ is also normalized to unity, i.e.,

  $$\begin{eqnarray}&&{\int }_{{\rm{\Omega }}}{G}_{q}({\theta }_{k})d{\rm{\Omega }}=1.\end{eqnarray} \tag{ 71 }$$

  Therefore, the contribution to the second moment of the pair counts by the quadrilaterals is given by

  $$\begin{eqnarray}\begin{array}{l}{\left\langle {\left(\overline{{DD}}\right)}_{\alpha \beta }\cdot {\left(\overline{{DD}}\right)}_{\alpha \beta }({\theta }_{k})\right\rangle }_{\mathrm{quad}}&=&\,{C}_{\mathrm{quad}}\displaystyle \sum _{j\ne i\ne l\ne m}^{K}\left\langle {\nu }_{i}{\nu }_{j}{\nu }_{m}{\nu }_{l}\right\rangle \left\langle {{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{ml}}^{\alpha \beta }\right\rangle {\bar{{\rm{\Theta }}}}_{{ij},k}{\bar{{\rm{\Theta }}}}_{{ml},k}\\ \quad &=&\,{C}_{\mathrm{quad}}{N}_{\mathrm{tot}}({N}_{\mathrm{tot}}-1)({N}_{\mathrm{tot}}-2)({N}_{\mathrm{tot}}-3)\\&\times&\left[1+2{w}_{\alpha \beta }({\theta }_{k})+4{w}_{{\rm{\Omega }}}\right]{G}_{q}({\theta }_{k}){\left\langle {{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{ml}}^{\alpha \beta }\right\rangle }_{i\ne j\ne m\ne l}\\ \quad &=&\,{C}_{\mathrm{quad}}{N}_{\mathrm{tot}}({N}_{\mathrm{tot}}-1)({N}_{\mathrm{tot}}-2)({N}_{\mathrm{tot}}-3)\\&\times&\left[1+2{w}_{\alpha \beta }({\theta }_{k})+4{w}_{{\rm{\Omega }}}\right]\ {G}_{q}({\theta }_{k})\\&\times&\left[\displaystyle \frac{{\displaystyle \sum }_{i\ne j\ne m\ne l}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{ml}}^{\alpha \beta }}{{N}_{\mathrm{tot}}({N}_{\mathrm{tot}}-1)({N}_{\mathrm{tot}}-2)({N}_{\mathrm{tot}}-3)}\right]\\ \quad &=&\,{C}_{\mathrm{quad}}\left[1+2{w}_{\alpha \beta }({\theta }_{k})+4{w}_{{\rm{\Omega }}}\right]{G}_{q}({\theta }_{k})\displaystyle \sum _{i\ne j\ne m\ne l}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{ml}}^{\alpha \beta },\end{array}\end{eqnarray} \tag{ 72 }$$

  where ${C}_{\mathrm{quad}}$ is the normalization constant so that we get the correct weight for the quadrilaterals when integrating over all angles, i.e.,

  $$\begin{eqnarray}&&\begin{array}{l}\,\int {\left\langle {\left(\overline{{DD}}\right)}_{\alpha \beta }\cdot {\left(\overline{{DD}}\right)}_{\alpha \beta }({\theta }_{k})\right\rangle }_{\mathrm{quad}}d{\rm{\Omega }}=\displaystyle \sum _{i\ne j\ne m\ne l}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{ml}}^{\alpha \beta }\\ \Rightarrow \,\int \left\{{C}_{\mathrm{quad}}\left[1+2{w}_{\alpha \beta }({\theta }_{k})+4{w}_{{\rm{\Omega }}}\right]{G}_{q}({\theta }_{k})\right\}d{\rm{\Omega }}=1\\ \Rightarrow \,{C}_{\mathrm{quad}}=\displaystyle \frac{1}{1+2\int {w}_{\alpha \beta }({\theta }_{k}){G}_{q}({\theta }_{k})d{\rm{\Omega }}+4{w}_{{\rm{\Omega }}}}=\displaystyle \frac{1}{1\,+\,2{w}_{{\rm{\Omega }},q}+4{w}_{{\rm{\Omega }}}},\end{array}\end{eqnarray} \tag{ 73 }$$

  where we have used Equation (71) and have defined a new mean:

  $$\begin{eqnarray}&&{w}_{{\rm{\Omega }},q}\equiv \int {w}_{\alpha \beta }({\theta }_{k}){G}_{q}({\theta }_{k})d{\rm{\Omega }}.\end{eqnarray} \tag{ 74 }$$

  Therefore,

  $$\begin{eqnarray}\begin{array}{ll}{\left\langle {\left(\overline{{DD}}\right)}_{\alpha \beta }\cdot {\left(\overline{{DD}}\right)}_{\alpha \beta }({\theta }_{k})\right\rangle }_{\mathrm{quad}} & =\,\left[\displaystyle \frac{1+2{w}_{\alpha \beta }({\theta }_{k})+4{w}_{{\rm{\Omega }}}}{1\,+\,2{w}_{{\rm{\Omega }},q}+4{w}_{{\rm{\Omega }}}}\right]{G}_{q}({\theta }_{k})\displaystyle \sum _{i\ne j\ne m\ne l}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{ml}}^{\alpha \beta }.\end{array}\end{eqnarray} \tag{ 75 }$$
• 2.  
  One of the indices is repeated: there are $K(K-1)(K-2)$ cases of the sort, since we choose only three cells without replacement, i.e., we choose two cells for the first $(\overline{{DD}})$ and one for the second $(\overline{{DD}})$. Note that we do not have to account for $m,l$ swap since we consider the two cases explicitly when calculating $\left\langle {\nu }_{i}{\nu }_{j}{\nu }_{m}{\nu }_{l}\right\rangle$ (needed since the swap carries different meaning for the pair weights). As for the probabilities of finding the data points in the chosen cells, we have

  $$\begin{eqnarray}&&\begin{array}{l}\left\langle {\nu }_{i}{\nu }_{j}{\nu }_{m}{\nu }_{l}\right\rangle {| }_{i=m}=\left\langle {\nu }_{i}^{2}{\nu }_{j}{\nu }_{l}\right\rangle =\left\langle {\nu }_{i}{\nu }_{j}{\nu }_{l}\right\rangle \\ \,=\,\displaystyle \frac{{N}_{\mathrm{tot}}({N}_{\mathrm{tot}}-1)({N}_{\mathrm{tot}}-2)}{K(K-1)(K-2)}\left[1+{w}_{{ij}}({\theta }_{k})+{w}_{{il}}({\theta }_{k})+{w}_{{jl}}({\theta }_{k})\right]\\ \,=\,\displaystyle \frac{{N}_{\mathrm{tot}}({N}_{\mathrm{tot}}-1)({N}_{\mathrm{tot}}-2)}{K(K-1)(K-2)}\left[1+3{w}_{\alpha \beta }({\theta }_{k})\right]\\ \left\langle \,,\langle ,{\nu }_{i}{\nu }_{j}{\nu }_{m}{\nu }_{l}\right\rangle {| }_{i=l}=\left\langle {\nu }_{i}^{2}{\nu }_{l}{\nu }_{m}\right\rangle =\left\langle {\nu }_{i}{\nu }_{l}{\nu }_{m}\right\rangle \\ \,=\,\displaystyle \frac{{N}_{\mathrm{tot}}({N}_{\mathrm{tot}}-1)({N}_{\mathrm{tot}}-2)}{K(K-1)(K-2)}\left[1+{w}_{{il}}({\theta }_{k})+{w}_{{im}}({\theta }_{k})+{w}_{{lm}}({\theta }_{k})\right]\\ \,=\,\displaystyle \frac{{N}_{\mathrm{tot}}({N}_{\mathrm{tot}}-1)({N}_{\mathrm{tot}}-2)}{K(K-1)(K-2)}\left[1+3{w}_{\alpha \beta }({\theta }_{k})\right]\\ \left\langle {\nu }_{i}{\nu }_{j}{\nu }_{m}{\nu }_{l}\right\rangle {| }_{j=m}=\left\langle {\nu }_{i}{\nu }_{j}^{2}{\nu }_{l}\right\rangle =\left\langle {\nu }_{i}{\nu }_{j}{\nu }_{l}\right\rangle \\ \,=\,\displaystyle \frac{{N}_{\mathrm{tot}}({N}_{\mathrm{tot}}-1)({N}_{\mathrm{tot}}-2)}{K(K-1)(K-2)}\left[1+{w}_{{ij}}({\theta }_{k})+{w}_{{il}}({\theta }_{k})+{w}_{{jl}}({\theta }_{k})\right]\\ \,=\,\displaystyle \frac{{N}_{\mathrm{tot}}({N}_{\mathrm{tot}}-1)({N}_{\mathrm{tot}}-2)}{K(K-1)(K-2)}\left[1+3{w}_{\alpha \beta }({\theta }_{k})\right]\\ \left\langle \,,\langle ,{\nu }_{i}{\nu }_{j}{\nu }_{m}{\nu }_{l}\right\rangle {| }_{j=l}=\left\langle {\nu }_{i}{\nu }_{j}^{2}{\nu }_{m}\right\rangle =\left\langle {\nu }_{i}{\nu }_{j}{\nu }_{m}\right\rangle \\ \,=\,\displaystyle \frac{{N}_{\mathrm{tot}}({N}_{\mathrm{tot}}-1)({N}_{\mathrm{tot}}-2)}{K(K-1)(K-2)}\left[1+{w}_{{ij}}({\theta }_{k})+{w}_{{im}}({\theta }_{k})+{w}_{{jm}}({\theta }_{k})\right]\\ \,=\,\displaystyle \frac{{N}_{\mathrm{tot}}({N}_{\mathrm{tot}}-1)({N}_{\mathrm{tot}}-2)}{K(K-1)(K-2)}\left[1+3{w}_{\alpha \beta }({\theta }_{k})\right],\end{array}\end{eqnarray} \tag{ 76 }$$

  where we note that $\left\langle \nu \right\rangle =\left\langle {\nu }^{2}\right\rangle ={N}_{\mathrm{tot}}/K$ since we are working in the large-K regime where there is only 0 or 1 galaxy in each cell. Also, as in LS93, we introduce ${G}_{t}({\theta }_{k})$ as the probability of finding triangles, i.e., two galaxies within ${\theta }_{k}\pm d{\theta }_{k}/2$ of a given galaxy. Thus, the total number of triangles is

  $$\begin{eqnarray}\begin{array}{ll}\displaystyle \sum _{\mathrm{unique}\{i,j,m\};l=i}^{K}{\bar{{\rm{\Theta }}}}_{{ij},k}{\bar{{\rm{\Theta }}}}_{{ml},k}=K(K-1)(K-2){G}_{t}({\theta }_{k}), & \quad i\ne j,m\ne i\end{array}\end{eqnarray} \tag{ 77 }$$

  where ${G}_{t}({\theta }_{k})$ is also normalized to unity:

  $$\begin{eqnarray}&&{\int }_{{\rm{\Omega }}}{G}_{t}({\theta }_{k})d{\rm{\Omega }}=1.\end{eqnarray} \tag{ 78 }$$

  Therefore, the contribution to the second moment of the pair counts by the triangles is given by

  $$\begin{eqnarray}&&\begin{array}{l}{\left\langle {\left(\overline{{DD}}\right)}_{\alpha \beta }\cdot {\left(\overline{{DD}}\right)}_{\alpha \beta }({\theta }_{k})\right\rangle }_{\mathrm{tri}}={C}_{\mathrm{tri}}{N}_{\mathrm{tot}}({N}_{\mathrm{tot}}-1)\,({N}_{\mathrm{tot}}-2){G}_{t}({\theta }_{k})\left[1+3{w}_{\alpha \beta }({\theta }_{k})\right]\\ \,\times \,\left\{{\left\langle {{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{ml}}^{\alpha \beta }\right\rangle }_{i=m\ne j\ne l}+{\left\langle {{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{ml}}^{\alpha \beta }\right\rangle }_{i=l\ne j\ne m}\right.\left.\,+\,{\left\langle {{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{ml}}^{\alpha \beta }\right\rangle }_{i\ne j=m\ne l}+{\left\langle {{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{ml}}^{\alpha \beta }\right\rangle }_{i\ne j=l\ne m}\right\}\\ \,=\,{C}_{\mathrm{tri}}{G}_{t}({\theta }_{k})\left[1+3{w}_{\alpha \beta }({\theta }_{k})\right]\displaystyle \sum _{i\ne j\ne l}^{{N}_{\mathrm{tot}}}\left\{{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{il}}^{\alpha \beta }+{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{li}}^{\alpha \beta }+{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{jl}}^{\alpha \beta }+{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{lj}}^{\alpha \beta }\right\},\end{array}\end{eqnarray} \tag{ 79 }$$

  where ${C}_{\mathrm{tri}}$ is the normalization constant so that we get the correct weight for the triangles when integrating over all angles, i.e.,

  $$\begin{eqnarray}&&\begin{array}{l}\,\int {\left\langle {\left(\overline{{DD}}\right)}_{\alpha \beta }\cdot {\left(\overline{{DD}}\right)}_{\alpha \beta }({\theta }_{k})\right\rangle }_{\mathrm{tri}}d{\rm{\Omega }}\,=\,\displaystyle \sum _{i\ne j\ne l}^{{N}_{\mathrm{tot}}}\left\{{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{il}}^{\alpha \beta }+{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{li}}^{\alpha \beta }+{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{jl}}^{\alpha \beta }+{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{lj}}^{\alpha \beta }\right\}\\ \Rightarrow \,\int \left\{{C}_{\mathrm{tri}}\left[1+3{w}_{\alpha \beta }({\theta }_{k})\right]{G}_{t}({\theta }_{k})\right\}d{\rm{\Omega }}=1\\ \Rightarrow \,{C}_{\mathrm{tri}}=\displaystyle \frac{1}{1+3\int {w}_{\alpha \beta }({\theta }_{k}){G}_{t}({\theta }_{k})d{\rm{\Omega }}+3{w}_{{\rm{\Omega }}}}=\displaystyle \frac{1}{1+3{w}_{{\rm{\Omega }},t}},\end{array}\end{eqnarray} \tag{ 80 }$$

  where we have used Equation (78) and have defined a new mean:

  $$\begin{eqnarray}&&{w}_{{\rm{\Omega }},t}\equiv \int {w}_{\alpha \beta }({\theta }_{k}){G}_{t}({\theta }_{k})d{\rm{\Omega }}.\end{eqnarray} \tag{ 81 }$$

  Therefore,

  $$\begin{eqnarray}\begin{array}{ll}{\left\langle {\left(\overline{{DD}}\right)}_{\alpha \beta }\cdot {\left(\overline{{DD}}\right)}_{\alpha \beta }({\theta }_{k})\right\rangle }_{\mathrm{tri}}&=&\left[\displaystyle \frac{1+3{w}_{\alpha \beta }({\theta }_{k})}{1+3{w}_{{\rm{\Omega }},t}}\right] \\&\times& \,{G}_{t}({\theta }_{k})\displaystyle \sum _{i\ne j\ne l}^{{N}_{\mathrm{tot}}}\left\{{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{il}}^{\alpha \beta }+{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{li}}^{\alpha \beta }+{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{jl}}^{\alpha \beta }+{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{lj}}^{\alpha \beta }\right\}.\end{array}\end{eqnarray} \tag{ 82 }$$
• 3.  
  Two of the indices overlap: there are $K(K-1)$ cases, since we choose only two cells. It follows that the probability of finding two galaxies in the chosen cells is

  $$\begin{eqnarray}\begin{array}{rcl}{\left\langle {\nu }_{i}{\nu }_{j}{\nu }_{m}{\nu }_{l}\right\rangle }_{i=m,j=l} & = & \left\langle {\nu }_{i}{\nu }_{j}{\nu }_{i}{\nu }_{j}\right\rangle =\left\langle {\nu }_{i}^{2}{\nu }_{j}^{2}\right\rangle \,=\,\left\langle {\nu }_{i}{\nu }_{j}\right\rangle =\displaystyle \frac{{N}_{\mathrm{tot}}({N}_{\mathrm{tot}}-1)}{K(K-1)}\left[1+{w}_{\alpha \beta }({\theta }_{k})\right]\\ {\left\langle {\nu }_{i}{\nu }_{j}{\nu }_{m}{\nu }_{l}\right\rangle }_{i=l,j=m} & = & \left\langle {\nu }_{i}{\nu }_{j}{\nu }_{j}{\nu }_{i}\right\rangle =\left\langle {\nu }_{i}^{2}{\nu }_{j}^{2}\right\rangle =\left\langle {\nu }_{i}{\nu }_{j}\right\rangle \,=\,\displaystyle \frac{{N}_{\mathrm{tot}}({N}_{\mathrm{tot}}-1)}{K(K-1)}\left[1+{w}_{\alpha \beta }({\theta }_{k})\right].\end{array}\end{eqnarray} \tag{ 83 }$$

  Here, Equation (58) applies, giving us the contribution to the second moment of the pair counts by the pairs as

  $$\begin{eqnarray}\begin{array}{l}{\left\langle {\left(\overline{{DD}}\right)}_{\alpha \beta }\cdot {\left(\overline{{DD}}\right)}_{\alpha \beta }({\theta }_{k})\right\rangle }_{\mathrm{pairs}}&=&{C}_{\mathrm{pairs}}{N}_{\mathrm{tot}}({N}_{\mathrm{tot}}-1){G}_{p}({\theta }_{k})\left[1+{w}_{\alpha \beta }({\theta }_{k})\right]\\&\times&\left\{{\left\langle {{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{ml}}^{\alpha \beta }\right\rangle }_{i=m\ne j=l}+{\left\langle {{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{ml}}^{\alpha \beta }\right\rangle }_{i=l\ne j=m}\right\}\\ \,&=&\,{C}_{\mathrm{pairs}}{N}_{\mathrm{tot}}({N}_{\mathrm{tot}}-1){G}_{p}({\theta }_{k})\left[1+{w}_{\alpha \beta }({\theta }_{k})\right]\\&\times&\left\{{\left\langle {{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{ij}}^{\alpha \beta }\right\rangle }_{i\ne j}+{\left\langle {{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{ji}}^{\alpha \beta }\right\rangle }_{i\ne j}\right\}\\ \,&=&\,{C}_{\mathrm{pairs}}{G}_{p}({\theta }_{k})\left[1+{w}_{\alpha \beta }({\theta }_{k})\right]\displaystyle \sum _{i\ne j}^{{N}_{\mathrm{tot}}}\left\{{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{ij}}^{\alpha \beta }+{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{ji}}^{\alpha \beta }\right\},\end{array}\end{eqnarray} \tag{ 84 }$$

  where ${C}_{\mathrm{pairs}}$ is the normalization constant so that we get the correct weight for the pairs when integrating over all angles, i.e.,

  $$\begin{eqnarray}&&\begin{array}{l}\,\int {\left\langle {\left(\overline{{DD}}\right)}_{\alpha \beta }\cdot {\left(\overline{{DD}}\right)}_{\alpha \beta }\left({\theta }_{k}\right)\right\rangle }_{\mathrm{pairs}}d{\rm{\Omega }}\,=\,\displaystyle \sum _{i\ne j}^{{N}_{\mathrm{tot}}}\left\{{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{ij}}^{\alpha \beta }+{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{ji}}^{\alpha \beta }\right\}\\ \Rightarrow \,\int \left\{{C}_{\mathrm{pairs}}\left[1+{w}_{\alpha \beta }({\theta }_{k})\right]{G}_{p}({\theta }_{k})\right\}d{\rm{\Omega }}=1\\ \,\Rightarrow \,{C}_{\mathrm{pairs}}=\displaystyle \frac{1}{1+{w}_{{\rm{\Omega }}}},\end{array}\end{eqnarray} \tag{ 85 }$$

  where we have used Equation (64); this results matches with Equation (65) as it should. Therefore,

  $$\begin{eqnarray}&&\begin{array}{l}{\left\langle {\left(\overline{{DD}}\right)}_{\alpha \beta }\cdot {\left(\overline{{DD}}\right)}_{\alpha \beta }({\theta }_{k})\right\rangle }_{\mathrm{pairs}}\,=\,{G}_{p}({\theta }_{k})\left[\displaystyle \frac{1+{w}_{\alpha \beta }({\theta }_{k})}{1+{w}_{{\rm{\Omega }}}}\right]\displaystyle \sum _{i\ne j}^{{N}_{\mathrm{tot}}}\left\{{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{ij}}^{\alpha \beta }+{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{ji}}^{\alpha \beta }\right\}.\end{array}\end{eqnarray} \tag{ 86 }$$

Combining the three cases, i.e., Equations (75), (82), and (86), Equation (67) becomes

$$\begin{eqnarray}&&\begin{array}{l}\left\langle {\left(\overline{{DD}}\right)}_{\alpha \beta }\cdot {\left(\overline{{DD}}\right)}_{\alpha \beta }({\theta }_{k})\right\rangle \,=\,\displaystyle \sum _{j\ne i}^{K}\displaystyle \sum _{l\ne m}^{K}\left\langle {\nu }_{i}{\nu }_{j}{\nu }_{m}{\nu }_{l}\right\rangle \left\langle {{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{ml}}^{\alpha \beta }\right\rangle {\bar{{\rm{\Theta }}}}_{{ij},k}{\bar{{\rm{\Theta }}}}_{{ml},k}\\ \,=\,\left[\displaystyle \frac{1+2{w}_{\alpha \beta }({\theta }_{k})+4{w}_{{\rm{\Omega }}}}{1\,+\,2{w}_{{\rm{\Omega }},q}+4{w}_{{\rm{\Omega }}}}\right]{G}_{p}{\left({\theta }_{k}\right)}^{2}\displaystyle \sum _{i\ne j\ne m\ne l}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{ml}}^{\alpha \beta }\\ \,+\,\left[\displaystyle \frac{1+3{w}_{\alpha \beta }({\theta }_{k})}{1+3{w}_{{\rm{\Omega }},t}}\right]{G}_{t}({\theta }_{k})\ \displaystyle \sum _{i\ne j\ne l}^{{N}_{\mathrm{tot}}}\left\{{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{il}}^{\alpha \beta }+{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{li}}^{\alpha \beta }+{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{jl}}^{\alpha \beta }+{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{lj}}^{\alpha \beta }\right\}\\ \,+\,{G}_{p}({\theta }_{k})\left[\displaystyle \frac{1+{w}_{\alpha \beta }({\theta }_{k})}{1+{w}_{{\rm{\Omega }}}}\right]\displaystyle \sum _{i\ne j}^{{N}_{\mathrm{tot}}}\left\{{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{ij}}^{\alpha \beta }+{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{ji}}^{\alpha \beta }\right\},\end{array}\end{eqnarray} \tag{ 87 }$$

where we have used the result ${G}_{q}({\theta }_{k})={G}_{p}^{2}({\theta }_{k})$ from LS93, valid in the large-K limit.

C.1.5. Fluctuations

Now, substituting Equations (66) and (87) in Equation (54), we have

$$\begin{eqnarray}&&\begin{array}{l}\left\langle {\eta }^{2}({\theta }_{k})\right\rangle =\displaystyle \frac{\begin{array}{l}\left[\tfrac{1+2{w}_{\alpha \beta }({\theta }_{k})+4{w}_{{\rm{\Omega }}}}{1\,+\,2{w}_{{\rm{\Omega }},q}+4{w}_{{\rm{\Omega }}}}\right]{G}_{p}{\left({\theta }_{k}\right)}^{2}{\displaystyle \sum }_{i\ne j\ne m\ne l}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{ml}}^{\alpha \beta }\\ +\left[\tfrac{1+3{w}_{\alpha \beta }({\theta }_{k})}{1+3{w}_{{\rm{\Omega }},t}}\right]{G}_{t}({\theta }_{k}){\displaystyle \sum }_{i\ne j\ne l}^{{N}_{\mathrm{tot}}}\left\{{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{il}}^{\alpha \beta }+{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{li}}^{\alpha \beta }+{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{jl}}^{\alpha \beta }+{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{lj}}^{\alpha \beta }\right\}\\ +{G}_{p}({\theta }_{k})\left[\tfrac{1+{w}_{\alpha \beta }({\theta }_{k})}{1+{w}_{{\rm{\Omega }}}}\right]{\displaystyle \sum }_{i\ne j}^{{N}_{\mathrm{tot}}}\left\{{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{ij}}^{\alpha \beta }+{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{ji}}^{\alpha \beta }\right\}\end{array}}{{\left({G}_{p}({\theta }_{k})\left[\tfrac{1+{w}_{\alpha \beta }({\theta }_{k})}{1+{w}_{{\rm{\Omega }}}}\right]{\displaystyle \sum }_{i\ne j}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{\alpha \beta }\right)}^{2}}-1\\ \,=\,\displaystyle \frac{\begin{array}{l}\left[\tfrac{1+2{w}_{\alpha \beta }({\theta }_{k})+4{w}_{{\rm{\Omega }}}}{1\,+\,2{w}_{{\rm{\Omega }},q}+4{w}_{{\rm{\Omega }}}}\right]{\displaystyle \sum }_{i\ne j\ne m\ne l}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{ml}}^{\alpha \beta }\\ +\left[\tfrac{1+3{w}_{\alpha \beta }({\theta }_{k})}{1+3{w}_{{\rm{\Omega }},t}}\right]\tfrac{{G}_{t}({\theta }_{k})}{{G}_{p}^{2}({\theta }_{k})}{\displaystyle \sum }_{i\ne j\ne l}^{{N}_{\mathrm{tot}}}\left\{{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{il}}^{\alpha \beta }+{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{li}}^{\alpha \beta }+{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{jl}}^{\alpha \beta }+{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{lj}}^{\alpha \beta }\right\}\\ +\tfrac{1}{{G}_{p}({\theta }_{k})}\left[\tfrac{1+{w}_{\alpha \beta }({\theta }_{k})}{1+{w}_{{\rm{\Omega }}}}\right]{\displaystyle \sum }_{i\ne j}^{{N}_{\mathrm{tot}}}\left\{{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{ij}}^{\alpha \beta }+{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{ji}}^{\alpha \beta }\right\}\end{array}}{{\left(\left[\tfrac{1+{w}_{\alpha \beta }({\theta }_{k})}{1+{w}_{{\rm{\Omega }}}}\right]{\displaystyle \sum }_{i\ne j}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{\alpha \beta }\right)}^{2}}-1.\end{array}\end{eqnarray} \tag{ 88 }$$

As for $\left\langle {\gamma }^{2}({\theta }_{k})\right\rangle$, given Equation (59), it takes the form

$$\begin{eqnarray}&&\left\langle {\gamma }^{2}({\theta }_{k})\right\rangle =\displaystyle \frac{2}{{N}_{r}({N}_{r}-1){G}_{p}({\theta }_{k})}.\end{eqnarray} \tag{ 89 }$$

C.1.6. Variance

We now go back to Equation (53) and attempt to evaluate it. First, substituting Equations (66) and (59), we have

$$\begin{eqnarray}\begin{array}{rcl}{\sigma }_{{\widetilde{w}}_{\alpha \beta }}^{2}({\theta }_{k}) & = & {\left[\displaystyle \frac{{N}_{r}({N}_{r}-1)}{2{\displaystyle \sum }_{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{\alpha \beta }}\displaystyle \frac{{G}_{p}({\theta }_{k})\left[\tfrac{1+{w}_{\alpha \beta }({\theta }_{k})}{1+{w}_{{\rm{\Omega }}}}\right]{\displaystyle \sum }_{i\ne j}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{\alpha \beta }}{\tfrac{{N}_{r}({N}_{r}-1)}{2}{G}_{p}({\theta }_{k})}\right]}^{2}\left[\left\langle {\gamma }^{2}({\theta }_{k})\right\rangle +\left\langle {\eta }^{2}({\theta }_{k})\right\rangle \right]\\ & = & {\left[\displaystyle \frac{1+{w}_{\alpha \beta }({\theta }_{k})}{1+{w}_{{\rm{\Omega }}}}\right]}^{2}\left[\left\langle {\gamma }^{2}({\theta }_{k})\right\rangle +\left\langle {\eta }^{2}({\theta }_{k})\right\rangle \right].\end{array}\end{eqnarray} \tag{ 90 }$$

Now, in the limit of large ${N}_{r}$, i.e., $\left\langle {\gamma }^{2}\right\rangle \to 0$, we have

$$\begin{eqnarray}&&{\sigma }_{{\widetilde{w}}_{\alpha \beta }}^{2}({\theta }_{k})\mathop{\to }\limits_{\mathrm{large}\ {N}_{r}}{\left[\displaystyle \frac{1+{w}_{\alpha \beta }({\theta }_{k})}{1+{w}_{{\rm{\Omega }}}}\right]}^{2}\left\langle {\eta }^{2}({\theta }_{k})\right\rangle ,\end{eqnarray} \tag{ 91 }$$

where $\left\langle {\eta }^{2}({\theta }_{k})\right\rangle$ is given by Equation (88). The expression can be simplified: we first look at the leading-order term, i.e., the quadrilateral contribution:

$$\begin{eqnarray}&&{\sigma }_{{\widetilde{w}}_{\alpha \beta }}^{2}({\theta }_{k})\underset{\mathrm{order}}{\overset{\mathrm{leading}}{\to }}\displaystyle \frac{\left[\tfrac{1+2{w}_{\alpha \beta }({\theta }_{k})+4{w}_{{\rm{\Omega }}}}{1\,+\,2{w}_{{\rm{\Omega }},q}+4{w}_{{\rm{\Omega }}}}\right]{\displaystyle \sum }_{i\ne j\ne m\ne l}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{ml}}^{\alpha \beta }}{{\left(\left[\tfrac{1+{w}_{\alpha \beta }({\theta }_{k})}{1+{w}_{{\rm{\Omega }}}}\right]{\displaystyle \sum }_{i\ne j}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{\alpha \beta }\right)}^{2}}-1.\end{eqnarray} \tag{ 92 }$$

Then, in the limit of weak correlations, as then $1\ll {w}_{\alpha \beta }({\theta }_{k})\sim {w}_{{\rm{\Omega }}}\lt {w}_{{\rm{\Omega }},t}\lt {w}_{{\rm{\Omega }},q}$, we have

$$\begin{eqnarray}&&{\sigma }_{{\widetilde{w}}_{\alpha \beta }}^{2}({\theta }_{k})\underset{\mathrm{correlations}}{\overset{\mathrm{weak}}{\to }}\displaystyle \frac{{\displaystyle \sum }_{i\ne j\ne m\ne l}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{{\mathtt{w}}}_{{ml}}^{\alpha \beta }}{{\left({\displaystyle \sum }_{i\ne j}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{\alpha \beta }\right)}^{2}}-1,\end{eqnarray} \tag{ 93 }$$

where we note that ${{\mathtt{w}}}_{{ij}}^{\alpha \beta }={{\mathtt{w}}}_{{ji}}^{\beta \alpha }$.

Now, in order to get the analytical expression for the variance of the unbiased estimator, i.e., the Decontaminated Weighted estimator, we must consider not only the variance of each of the biased correlations but also the covariances. As an example, based on Equation (18), which is valid for when there are two galaxy types in our observed sample, we essentially have the unbiased estimator for the AA autocorrelation function as

$$\begin{eqnarray}\begin{array}{ll}{\widehat{w}}_{{AA}}({\theta }_{k})={C}_{{AA}}({\theta }_{k}){\widetilde{w}}_{{AA}}^{\mathrm{obs}}({\theta }_{k})+{C}_{{AB}}({\theta }_{k}){\widetilde{w}}_{{AB}}^{\mathrm{obs}}({\theta }_{k}) & +\,{C}_{{BB}}({\theta }_{k}){\widetilde{w}}_{{BB}}^{\mathrm{obs}}({\theta }_{k}),\end{array}\end{eqnarray} \tag{ 94 }$$

where ${C}_{{AA}}({\theta }_{k}),{C}_{{AB}}({\theta }_{k}),{C}_{{BB}}({\theta }_{k})$ are the elements of the first row of the inverse matrix in Equation (18). Given the dependency of all terms and factors on the pair weights, we have the variance of the unbiased estimator as

$$\begin{eqnarray}&&\begin{array}{l}{\sigma }_{{\widehat{w}}_{{AA}}}^{2}({\theta }_{k})={C}_{{AA}}^{2}({\theta }_{k}){\sigma }_{{\widetilde{w}}_{{AA}}}^{2}({\theta }_{k})+{C}_{{AB}}^{2}({\theta }_{k}){\sigma }_{{\widetilde{w}}_{{AB}}}^{2}({\theta }_{k})+{C}_{{BB}}^{2}({\theta }_{k}){\sigma }_{{\widetilde{w}}_{{BB}}}^{2}({\theta }_{k})\\ \,-\,2\mathrm{cov}\left[{C}_{{AA}}({\theta }_{k}),{\widetilde{w}}_{{AA}}^{\mathrm{obs}}({\theta }_{k})\right]-2\mathrm{cov}\left[{C}_{{AB}}({\theta }_{k}),{\widetilde{w}}_{{AB}}^{\mathrm{obs}}({\theta }_{k})\right]-2\mathrm{cov}\left[{C}_{{BB}}({\theta }_{k}),{\widetilde{w}}_{{BB}}^{\mathrm{obs}}({\theta }_{k})\right]\\ \,-\,2{\widetilde{w}}_{{AA}}^{\mathrm{obs}}({\theta }_{k}){\widetilde{w}}_{{AB}}^{\mathrm{obs}}({\theta }_{k})\mathrm{cov}\left[{C}_{{AA}}({\theta }_{k}),{C}_{{AB}}({\theta }_{k})\right]-2{\widetilde{w}}_{{AA}}^{\mathrm{obs}}({\theta }_{k}){\widetilde{w}}_{{BB}}^{\mathrm{obs}}({\theta }_{k})\mathrm{cov}\left[{C}_{{AA}}({\theta }_{k}),{C}_{{BB}}({\theta }_{k})\right]\\ \,-\,2{\widetilde{w}}_{{AB}}^{\mathrm{obs}}({\theta }_{k}){\widetilde{w}}_{{BB}}^{\mathrm{obs}}({\theta }_{k})\mathrm{cov}\left[{C}_{{AB}}({\theta }_{k}),{C}_{{BB}}({\theta }_{k})\right].\end{array}\end{eqnarray} \tag{ 95 }$$

This expression is unwieldy to evaluate for the general case, even if when we use the leading-order, weak-correlation approximation as in Equation (93). Therefore, we resort to numerical estimation of the variance.

C.2.1. Weighted Data–Data Pair Counts

Here, we note some points that are important when it comes to implementing the Weighted estimator proposed in Equation (13). Specifically considering Equation (14) for the autocorrelation, we have

$$\begin{eqnarray}&&{\left(\widetilde{{DD}}\right)}_{{AA}}({\theta }_{k})=\displaystyle \frac{{\displaystyle \sum }_{i}^{{N}_{\mathrm{tot}}}{\displaystyle \sum }_{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{{AA}}{\bar{{\rm{\Theta }}}}_{{ij},k}}{{\displaystyle \sum }_{i}^{{N}_{\mathrm{tot}}}{\displaystyle \sum }_{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{{AA}}},\end{eqnarray} \tag{ 96 }$$

while for the cross, we have

$$\begin{eqnarray}&&{\left(\widetilde{{DD}}\right)}_{{AB}}({\theta }_{k})=\displaystyle \frac{{\displaystyle \sum }_{i}^{{N}_{\mathrm{tot}}}{\displaystyle \sum }_{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{{AB}}{\bar{{\rm{\Theta }}}}_{{ij},k}}{{\displaystyle \sum }_{i}^{{N}_{\mathrm{tot}}}{\displaystyle \sum }_{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{{AB}}}.\end{eqnarray} \tag{ 97 }$$

It might appear that ${(\widetilde{{DD}})}_{{AB}}\ne {(\widetilde{{DD}})}_{{BA}}$ since ${{\mathtt{w}}}_{{ij}}^{{AB}}\ne {{\mathtt{w}}}_{{ij}}^{{BA}}$, but we must realize that

$$\begin{eqnarray}&&{{\mathtt{w}}}_{{ij}}^{{AB}}={{\mathtt{w}}}_{{ji}}^{{BA}},\end{eqnarray} \tag{ 98 }$$

and since the sums are reindexable, we have

$$\begin{eqnarray}\begin{array}{rclrcl}{\left(\widetilde{{DD}}\right)}_{{BA}}({\theta }_{k}) & = & \displaystyle \frac{{\displaystyle \sum }_{i}^{{N}_{\mathrm{tot}}}{\displaystyle \sum }_{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{{BA}}{\bar{{\rm{\Theta }}}}_{{ij},k}}{{\displaystyle \sum }_{i}^{{N}_{\mathrm{tot}}}{\displaystyle \sum }_{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{{BA}}} & & = & \displaystyle \frac{{\displaystyle \sum }_{i}^{{N}_{\mathrm{tot}}}{\displaystyle \sum }_{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ji}}^{{AB}}{\bar{{\rm{\Theta }}}}_{{ij},k}}{{\displaystyle \sum }_{i}^{{N}_{\mathrm{tot}}}{\displaystyle \sum }_{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ji}}^{{AB}}}={\left(\widetilde{{DD}}\right)}_{{AB}}({\theta }_{k}).\end{array}\end{eqnarray} \tag{ 99 }$$

Therefore, when implementing the weighted data–data histogram, we can work with either ${{\mathtt{w}}}_{{ij}}^{\alpha \beta }$ or ${{\mathtt{w}}}_{{ij}}^{\beta \alpha }$, even though ${{\mathtt{w}}}_{{ij}}^{\alpha \beta }\ne {{\mathtt{w}}}_{{ij}}^{\beta \alpha }$ when $\alpha \ne \beta$.

C.2.2. Pair Weights

While we have used simple pair weights in this work, i.e., ${{\mathtt{w}}}_{{ij}}^{\alpha \beta }={{\mathtt{q}}}_{i}^{\alpha }{{\mathtt{q}}}_{j}^{\beta }$, the Weighted estimator presented in Equation (13) works with general pair weights. In the case where the pair weights are not separable (e.g., they account for a theta-dependence), we must circumvent the problem presented by the normalization of the data–data histogram in Equation (14): it requires summing over all the pair weights—a task that is computationally prohibitive when working with large data sets where standard correlation function algorithms focus on a specified range of separations to reduce compute time. We can address the challenge by two methods: (1) estimating the number of pairs and the average weights for the larger θ bins, and hence still being able to use the all-pairs normalization; and (2) introducing a new, exact normalization, which can be achieved by considering Equation (13) with its full details, i.e.,

$$\begin{eqnarray}\begin{array}{rcl}{\widetilde{w}}_{\alpha \beta }^{\mathrm{obs}}({\theta }_{k})+1 & = & \displaystyle \frac{{\left(\widetilde{{DD}}\right)}_{\alpha \beta }({\theta }_{k})}{{RR}({\theta }_{k})}=\displaystyle \frac{{\displaystyle \sum }_{i}^{{N}_{\mathrm{tot}}}{\displaystyle \sum }_{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{\bar{{\rm{\Theta }}}}_{{ij},k}}{{\displaystyle \sum }_{i}^{{N}_{\mathrm{tot}}}{\displaystyle \sum }_{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{\alpha \beta }}\,\displaystyle \frac{{\displaystyle \sum }_{i}^{{N}_{r}}{\displaystyle \sum }_{j\ne i}^{{N}_{r}}}{{\displaystyle \sum }_{i}^{{N}_{r}}{\displaystyle \sum }_{j\ne i}^{{N}_{r}}{\bar{{\rm{\Theta }}}}_{{ij},k}}\\ & = & \displaystyle \frac{{\displaystyle \sum }_{i}^{{N}_{\mathrm{tot}}}{\displaystyle \sum }_{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{\bar{{\rm{\Theta }}}}_{{ij},k}}{{\displaystyle \sum }_{i}^{{N}_{r}}{\displaystyle \sum }_{j\ne i}^{{N}_{r}}{\bar{{\rm{\Theta }}}}_{{ij},k}}\displaystyle \frac{{\displaystyle \sum }_{i}^{{N}_{r}}{\displaystyle \sum }_{j\ne i}^{{N}_{r}}}{{\displaystyle \sum }_{i}^{{N}_{\mathrm{tot}}}{\displaystyle \sum }_{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{\alpha \beta }},\end{array}\end{eqnarray} \tag{ 100 }$$

where the first fraction in the last line compares the data–data pair weight in bin k with the random–random pairs in the same bins, while the second fraction normalizes the total data–data pair weight with the total random–random pair counts. Now, given that exact numerical calculation of the total data–data pair weight is prohibitive and affects only the overall normalization, we can normalize both the total data–data pair weight and the total random pair counts in a less computationally challenging way, i.e.,

$$\begin{eqnarray}\begin{array}{rcl}{\widetilde{w}}_{\alpha \beta }^{\mathrm{obs}}({\theta }_{k})+1 & = & \displaystyle \frac{{\displaystyle \sum }_{i}^{{N}_{\mathrm{tot}}}{\displaystyle \sum }_{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{\bar{{\rm{\Theta }}}}_{{ij},k}}{{\displaystyle \sum }_{i}^{{N}_{r}}{\displaystyle \sum }_{j\ne i}^{{N}_{r}}{\bar{{\rm{\Theta }}}}_{{ij},k}}\,\displaystyle \frac{{\displaystyle \sum }_{m}^{{N}_{\mathrm{bin}}}{\displaystyle \sum }_{i}^{{N}_{r}}{\displaystyle \sum }_{j\ne i}^{{N}_{r}}{\bar{{\rm{\Theta }}}}_{{ij},m}}{{\displaystyle \sum }_{m}^{{N}_{\mathrm{bin}}}{\displaystyle \sum }_{i}^{{N}_{\mathrm{tot}}}{\displaystyle \sum }_{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{\alpha \beta }{\bar{{\rm{\Theta }}}}_{{ij},m}},\end{array}\end{eqnarray} \tag{ 101 }$$

where have replaced the total counts over all possible scales to those in only the scales of interest.

Here, we attempt to find weights that allow us to decontaminate while estimating the correlations—a step toward optimal weights. To achieve this, we consider Equation (17) which is reproduced here for convenience:

$$\begin{eqnarray}&&\begin{array}{l}\left[\begin{array}{l}\left\langle {\widetilde{w}}_{{AA}}^{\mathrm{obs}}({\theta }_{k})\right\rangle \\ \left\langle {\widetilde{w}}_{{AB}}^{\mathrm{obs}}({\theta }_{k})\right\rangle \\ \left\langle {\widetilde{w}}_{{BB}}^{\mathrm{obs}}({\theta }_{k})\right\rangle \end{array}\right]\,=\,\left[\begin{array}{lll}\displaystyle \frac{\sum _{i}^{{N}_{\mathrm{tot}}}\sum _{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{{AA}}{{\mathtt{q}}}_{i}^{A}{{\mathtt{q}}}_{j}^{A}}{\sum _{i}^{{N}_{\mathrm{tot}}}\sum _{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{{AA}}} & \displaystyle \frac{\sum _{i}^{{N}_{\mathrm{tot}}}\sum _{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{{AA}}\left\{{{\mathtt{q}}}_{i}^{A}{{\mathtt{q}}}_{j}^{B}+{{\mathtt{q}}}_{i}^{B}{{\mathtt{q}}}_{j}^{A}\right\}}{\sum _{i}^{{N}_{\mathrm{tot}}}\sum _{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{{AA}}} & \displaystyle \frac{\sum _{i}^{{N}_{\mathrm{tot}}}\sum _{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{{AA}}{{\mathtt{q}}}_{i}^{B}{{\mathtt{q}}}_{j}^{B}}{\sum _{i}^{{N}_{\mathrm{tot}}}\sum _{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{{AA}}}\\ \displaystyle \frac{\sum _{i}^{{N}_{\mathrm{tot}}}\sum _{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{{AB}}{{\mathtt{q}}}_{i}^{A}{{\mathtt{q}}}_{j}^{A}}{\sum _{i}^{{N}_{\mathrm{tot}}}\sum _{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{{AB}}} & \displaystyle \frac{\sum _{i}^{{N}_{\mathrm{tot}}}\sum _{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{{AB}}\left\{{{\mathtt{q}}}_{i}^{A}{{\mathtt{q}}}_{j}^{B}+{{\mathtt{q}}}_{i}^{B}{{\mathtt{q}}}_{j}^{A}\right\}}{\sum _{i}^{{N}_{\mathrm{tot}}}\sum _{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{{AB}}} & \displaystyle \frac{\sum _{i}^{{N}_{\mathrm{tot}}}\sum _{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{{AB}}{{\mathtt{q}}}_{i}^{B}{{\mathtt{q}}}_{j}^{B}}{\sum _{i}^{{N}_{\mathrm{tot}}}\sum _{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{{AB}}}\\ \displaystyle \frac{\sum _{i}^{{N}_{\mathrm{tot}}}\sum _{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{{BB}}{{\mathtt{q}}}_{i}^{A}{{\mathtt{q}}}_{j}^{A}}{\sum _{i}^{{N}_{\mathrm{tot}}}\sum _{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{{BB}}} & \displaystyle \frac{\sum _{i}^{{N}_{\mathrm{tot}}}\sum _{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{{BB}}\left\{{{\mathtt{q}}}_{i}^{A}{{\mathtt{q}}}_{j}^{B}+{{\mathtt{q}}}_{i}^{B}{{\mathtt{q}}}_{j}^{A}\right\}}{\sum _{i}^{{N}_{\mathrm{tot}}}\sum _{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{{BB}}} & \displaystyle \frac{\sum _{i}^{{N}_{\mathrm{tot}}}\sum _{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{{BB}}{{\mathtt{q}}}_{i}^{B}{{\mathtt{q}}}_{j}^{B}}{\sum _{i}^{{N}_{\mathrm{tot}}}\sum _{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{{BB}}}\end{array}\right]\ \left[\begin{array}{l}{w}_{{AA}}^{\mathrm{true}}({\theta }_{k})\\ {w}_{{AB}}^{\mathrm{true}}({\theta }_{k})\\ {w}_{{BB}}^{\mathrm{true}}({\theta }_{k})\end{array}\right].\end{array}\end{eqnarray} \tag{ 102 }$$

In order to achieve our goal, we would like to find weights ${{\mathtt{w}}}_{{ij},\mathrm{opt}}^{\alpha \beta }$ such that we can write the above equation as

$$\begin{eqnarray}&&\begin{array}{l}\left[\begin{array}{l}\left\langle {\widetilde{w}}_{{AA}}^{\mathrm{obs}}({\theta }_{k})\right\rangle \\ \left\langle {\widetilde{w}}_{{AB}}^{\mathrm{obs}}({\theta }_{k})\right\rangle \\ \left\langle {\widetilde{w}}_{{BB}}^{\mathrm{obs}}({\theta }_{k})\right\rangle \end{array}\right]\\=\,\left[\begin{array}{lll}\displaystyle \frac{\sum _{i}^{{N}_{\mathrm{tot}}}\sum _{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{{AA}}{{\mathtt{q}}}_{i}^{A}{{\mathtt{q}}}_{j}^{A}}{\sum _{i}^{{N}_{\mathrm{tot}}}\sum _{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{{AA}}} & 0 & 0\\ 0 & \displaystyle \frac{\sum _{i}^{{N}_{\mathrm{tot}}}\sum _{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{{AB}}\left\{{{\mathtt{q}}}_{i}^{A}{{\mathtt{q}}}_{j}^{B}+{{\mathtt{q}}}_{i}^{B}{{\mathtt{q}}}_{j}^{A}\right\}}{\sum _{i}^{{N}_{\mathrm{tot}}}\sum _{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{{AB}}} & 0\\ 0 & 0 & \displaystyle \frac{\sum _{i}^{{N}_{\mathrm{tot}}}\sum _{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{{BB}}{{\mathtt{q}}}_{i}^{B}{{\mathtt{q}}}_{j}^{B}}{\sum _{i}^{{N}_{\mathrm{tot}}}\sum _{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{{BB}}}\end{array}\right]\ \left[\begin{array}{l}{w}_{{AA}}^{\mathrm{true}}({\theta }_{k})\\ {w}_{{AB}}^{\mathrm{true}}({\theta }_{k})\\ {w}_{{BB}}^{\mathrm{true}}({\theta }_{k})\end{array}\right].\end{array}\end{eqnarray} \tag{ 103 }$$

To consider a simple scenario, we first assume that the pair weights are a linear product of the weights of individual weights, i.e., ${{\mathtt{w}}}_{{ij},\mathrm{opt}}^{\alpha \beta }={{\mathtt{w}}}_{i,\mathrm{opt}}^{\alpha }{{\mathtt{w}}}_{j,\mathrm{opt}}^{\beta }$, which follows that we only need to find ${{\mathtt{w}}}_{i,\mathrm{opt}}^{\alpha }$ and ${{\mathtt{w}}}_{i,\mathrm{opt}}^{\beta }$ (where we note $\alpha ,\beta$ can be either A or B). Then, we must have the nondiagonal terms in Equation (102) be zero, leading us to specific constraints on the pair weights. To demonstrate the method, we achieved the optimization by assuming a functional form for the optimized weights:

$$\begin{eqnarray}&&{{\mathtt{w}}}_{i,\mathrm{opt}}^{\alpha }={\mu }^{\alpha }+{\nu }^{\alpha }{q}_{i}^{\alpha },\end{eqnarray} \tag{ 104 }$$

where $\mu ,\nu$ are the optimization parameters and are allowed to be negative (which is what allows this method to mimic Decontaminated by automatically subtracting off pairs in which one contributor is likely a contaminant). Using this method, we were able to decontaminate as effectively as Decontaminated for the two-sample case, but without reducing the variance. We note that the equivalence between this direct decontamination with optimized weights and Decontaminated is not guaranteed for larger numbers of samples or for weights that are nonlinear functions of probability, meriting further investigation as part of a larger investigation of optimizing the weights.

As an extension of our derivation for two samples in Section 3.1, we now consider three samples, with galaxies of Types A, B, C present in our sample. For instance, we have

$$\begin{eqnarray}&&\begin{array}{l}{w}_{{AB}}^{\mathrm{obs}}({\theta }_{k})={f}_{{AA}}{f}_{{BA}}{w}_{{AA}}^{\mathrm{true}}({\theta }_{k})+\left\{{f}_{{AA}}{f}_{{BB}}+{f}_{{AB}}{f}_{{BA}}\right\}{w}_{{AB}}^{\mathrm{true}}({\theta }_{k})+{f}_{{AB}}{f}_{{BB}}{w}_{{BB}}^{\mathrm{true}}({\theta }_{k})\\ \,+\,\left\{{f}_{{AB}}{f}_{{BC}}+{f}_{{AC}}{f}_{{BB}}\right\}{w}_{{BC}}^{\mathrm{true}}({\theta }_{k})+{f}_{{AC}}{f}_{{BC}}{w}_{{CC}}^{\mathrm{true}}({\theta }_{k})+\left\{{f}_{{AA}}{f}_{{BC}}+{f}_{{AC}}{f}_{{BA}}\right\}{w}_{{CA}}^{\mathrm{true}}({\theta }_{k}).\end{array}\end{eqnarray} \tag{ 105 }$$

Therefore, similar to the construction of Equation (12), we have

$$\begin{eqnarray}&&\begin{array}{l}\left[\begin{array}{l}{\widehat{w}}_{{AA}}({\theta }_{k})\\ {\widehat{w}}_{{AB}}({\theta }_{k})\\ {\widehat{w}}_{{BB}}({\theta }_{k})\\ {\widehat{w}}_{{BC}}({\theta }_{k})\\ {\widehat{w}}_{{CC}}({\theta }_{k})\\ {\widehat{w}}_{{CA}}({\theta }_{k})\end{array}\right]\,=\,{\left[\begin{array}{cccccc}{\varsigma }_{{AA}}^{{AA}} & 2{\varsigma }_{{AB}}^{{AA}} & {\varsigma }_{{AB}}^{{AB}} & 2{\varsigma }_{{AC}}^{{AB}} & {\varsigma }_{{AC}}^{{AC}} & 2{\varsigma }_{{AC}}^{{AA}}\\ {\varsigma }_{{BA}}^{{AA}} & {\varsigma }_{{BB}}^{{AA}}+{\varsigma }_{{AB}}^{{BA}} & {\varsigma }_{{AB}}^{{BB}} & {\varsigma }_{{BC}}^{{AB}}+{\varsigma }_{{AC}}^{{BB}} & {\varsigma }_{{BC}}^{{AC}} & {\varsigma }_{{BC}}^{{AA}}+{\varsigma }_{{AC}}^{{BA}}\\ {\varsigma }_{{BA}}^{{BA}} & 2{\varsigma }_{{BA}}^{{BB}} & {\varsigma }_{{BB}}^{{BB}} & 2{\varsigma }_{{BC}}^{{BB}} & {\varsigma }_{{BC}}^{{BC}} & 2{\varsigma }_{{BC}}^{{BA}}\\ {\varsigma }_{{CB}}^{{BA}} & {\varsigma }_{{CB}}^{{BA}}+{\varsigma }_{{BB}}^{{CA}} & {\varsigma }_{{CB}}^{{BB}} & {\varsigma }_{{CC}}^{{BB}}+{\varsigma }_{{BC}}^{{CB}} & {\varsigma }_{{CC}}^{{BC}} & {\varsigma }_{{BC}}^{{BA}}+{\varsigma }_{{BC}}^{{BA}}\\ {\varsigma }_{{CA}}^{{CA}} & 2{\varsigma }_{{CB}}^{{CA}} & {\varsigma }_{{CB}}^{{CB}} & 2{\varsigma }_{{CC}}^{{CB}} & {\varsigma }_{{CC}}^{{CC}} & 2{\varsigma }_{{CC}}^{{CA}}\\ {\varsigma }_{{CA}}^{{AA}} & {\varsigma }_{{CB}}^{{AA}}+{\varsigma }_{{AB}}^{{CA}} & {\varsigma }_{{CB}}^{{AB}} & {\varsigma }_{{CC}}^{{AB}}+{\varsigma }_{{AC}}^{{CB}} & {\varsigma }_{{CC}}^{{AC}} & {\varsigma }_{{CC}}^{{AA}}+{\varsigma }_{{AC}}^{{CA}}\end{array}\right]}^{-1}\ \left[\begin{array}{c}{w}_{{AA}}^{\mathrm{obs}}({\theta }_{k})\\ {w}_{{AB}}^{\mathrm{obs}}({\theta }_{k})\\ {w}_{{BB}}^{\mathrm{obs}}({\theta }_{k})\\ {w}_{{BC}}^{\mathrm{obs}}({\theta }_{k})\\ {w}_{{CC}}^{\mathrm{obs}}({\theta }_{k})\\ {w}_{{CA}}^{\mathrm{obs}}({\theta }_{k})\end{array}\right],\end{array}\end{eqnarray} \tag{ 106 }$$

where we have defined the following for brevity:

$$\begin{eqnarray}&&{\varsigma }_{{mn}}^{{ij}}={f}_{{A}_{i}{A}_{j}}{f}_{{A}_{m}{A}_{n}}={\varsigma }_{{ij}}^{{mn}}.\end{eqnarray} \tag{ 107 }$$

Extending the idea to M samples, we can write the analog of the unbiased estimator for ${\mathtt{Decontamination}}$, given by Equation (12), as

$$\begin{eqnarray}&&\begin{array}{c}\left[\begin{array}{c}{\widehat{w}}_{11}({\theta }_{k})\\ {\widehat{w}}_{12}({\theta }_{k})\\ \vdots \\ {\widehat{w}}_{\gamma \gamma }({\theta }_{k})\\ {\widehat{w}}_{\gamma (\gamma +1)}({\theta }_{k})\\ \vdots \\ {\widehat{w}}_{{MM}}({\theta }_{k})\\ {\widehat{w}}_{M1}({\theta }_{k})\\ \end{array}\right]\,=\,{\left[\begin{array}{cccccccc}{\varsigma }_{11}^{11} & 2{\varsigma }_{12}^{11} & ... & {\varsigma }_{1\gamma }^{1\gamma } & 2{\varsigma }_{1(\gamma +1)}^{1\gamma } & ... & {\varsigma }_{1M}^{1M} & 2{\varsigma }_{1M}^{11}\\ {\varsigma }_{21}^{11} & {\varsigma }_{22}^{11}+{\varsigma }_{12}^{21} & ... & {\varsigma }_{2\gamma }^{1\gamma } & {\varsigma }_{2(\gamma +1)}^{1\gamma }+{\varsigma }_{1(\gamma +1)}^{2\gamma } & ... & {\varsigma }_{2M}^{1M} & {\varsigma }_{21}^{1M}+{\varsigma }_{11}^{2M}\\ \vdots & \vdots & ... & \vdots & \vdots & ... & \vdots & \vdots \\ {\varsigma }_{\gamma 1}^{\gamma 1} & 2{\varsigma }_{\gamma 2}^{\gamma 1} & ... & {\varsigma }_{\gamma \gamma }^{\gamma \gamma } & 2{\varsigma }_{\gamma (\gamma +1)}^{\gamma \gamma } & ... & {\varsigma }_{\gamma M}^{\gamma M} & 2{\varsigma }_{\gamma 1}^{\gamma M}\\ {\varsigma }_{(\gamma +1)1}^{\gamma 1} & {\varsigma }_{(\gamma +1)2}^{\gamma 1}+{\varsigma }_{\gamma 2}^{(\gamma +1)1} & ... & {\varsigma }_{(\gamma +1)\gamma }^{\gamma \gamma } & {\varsigma }_{(\gamma +1)(\gamma +1)}^{\gamma \gamma }+{\varsigma }_{\gamma (\gamma +1)}^{(\gamma +1)\gamma } & ... & {\varsigma }_{(\gamma +1)M}^{\gamma M} & {\varsigma }_{(\gamma +1)1}^{\gamma M}+{\varsigma }_{\gamma 1}^{(\gamma +1)M}\\ \vdots & \vdots & ... & \vdots & \vdots & ... & \vdots & \vdots \\ {\varsigma }_{M1}^{M1} & 2{\varsigma }_{M2}^{M1} & ... & {\varsigma }_{M\gamma }^{M\gamma } & 2{\varsigma }_{M(\gamma +1)}^{M\gamma } & ... & {\varsigma }_{{MM}}^{{MM}} & 2{\varsigma }_{M1}^{{MM}}\\ {\varsigma }_{11}^{M1} & {\varsigma }_{12}^{M1}+{\varsigma }_{M2}^{11} & ... & {\varsigma }_{1\gamma }^{M\gamma } & {\varsigma }_{1(\gamma +1)}^{M\gamma }+{\varsigma }_{M(\gamma +1)}^{1\gamma } & ... & {\varsigma }_{1M}^{{MM}} & {\varsigma }_{11}^{{MM}}\end{array}\right]}^{-1}\,\left[\begin{array}{c}{w}_{11}^{\mathrm{obs}}({\theta }_{k})\\ {w}_{12}^{\mathrm{obs}}({\theta }_{k})\\ \vdots \\ {w}_{\gamma \gamma }^{\mathrm{obs}}({\theta }_{k})\\ {w}_{\gamma (\gamma +1)}^{\mathrm{obs}}({\theta }_{k})\\ \vdots \\ {w}_{{MM}}^{\mathrm{obs}}({\theta }_{k})\\ {w}_{M1}^{\mathrm{obs}}({\theta }_{k})\end{array}\right].\end{array}\end{eqnarray} \tag{ 108 }$$

As for the two-sample case, we can get the variance of the estimators for M target samples as

$$\begin{eqnarray}\begin{array}{ll}{\left[\begin{array}{l}{\sigma }_{{\widehat{w}}_{{A}_{1}{A}_{1}}}^{2}{\sigma }_{{\widehat{w}}_{{A}_{1}{A}_{2}}}^{2}\cdots {\sigma }_{{\widehat{w}}_{{A}_{\gamma }{A}_{\gamma }}}^{2}{\sigma }_{{\widehat{w}}_{{A}_{\gamma }{A}_{\gamma +1}}}^{2}\cdots {\sigma }_{{\widehat{w}}_{{A}_{M}{A}_{M}}}^{2}{\sigma }_{{\widehat{w}}_{{A}_{M}{A}_{1}}}^{2}\end{array}\right]}^{T} & \\={\left\{{\left[{D}_{{\rm{S}}}^{\mathrm{gen}}\right]}^{-1}\right\}}_{{ij}}^{2}{\left[\begin{array}{l}{\sigma }_{{w}_{{A}_{1}{A}_{1}}^{\mathrm{obs}}}^{2}{\sigma }_{{w}_{{A}_{1}{A}_{2}}^{\mathrm{obs}}}^{2}\cdots {\sigma }_{{w}_{{A}_{\gamma }{A}_{\gamma }}^{\mathrm{obs}}}^{2}{\sigma }_{{w}_{{A}_{\gamma }{A}_{\gamma +1}}^{\mathrm{obs}}}^{2}\cdots {\sigma }_{{w}_{{A}_{M}{A}_{M}}^{\mathrm{obs}}}^{2}{\sigma }_{{w}_{{A}_{M}{A}_{1}}^{\mathrm{obs}}}^{2}\end{array}\right]}^{T},\end{array}\end{eqnarray} \tag{ 109 }$$

where $[{D}_{{\rm{S}}}^{\mathrm{gen}}]$ is the square matrix in Equation (108), and as in Appendix A.3, ${\left\{{[{D}_{{\rm{S}}}^{\mathrm{gen}}]}^{-1}\right\}}_{{ij}}^{2}$ denotes that matrix resulting from squaring each individual coefficient in the matrix ${[{D}_{{\rm{S}}}^{\mathrm{gen}}]}^{-1}$. The covariance matrix for the M-samples case follows the derivation in Equation (34), with all of its assumptions.

Expanding our derivation for two samples to three samples, with galaxies of Types A, B, C present in our sample, we have

$$\begin{eqnarray}&&\begin{array}{l}\left[\begin{array}{l}{\widehat{w}}_{{AA}}({\theta }_{k})\\ {\widehat{w}}_{{AB}}({\theta }_{k})\\ {\widehat{w}}_{{BB}}({\theta }_{k})\\ {\widehat{w}}_{{BC}}({\theta }_{k})\\ {\widehat{w}}_{{CC}}({\theta }_{k})\\ {\widehat{w}}_{{CA}}({\theta }_{k})\\ \end{array}\right]\,=\,{\left[\begin{array}{cccccc}{\varkappa }_{{AA}}^{{AA}} & 2{\varkappa }_{{AB}}^{{AA}} & {\varkappa }_{{AB}}^{{AB}} & 2{\varkappa }_{{AC}}^{{AB}} & {\varkappa }_{{AC}}^{{AC}} & 2{\varkappa }_{{AC}}^{{AA}}\\ {\varkappa }_{{BA}}^{{AA}} & {\varkappa }_{{BB}}^{{AA}}+{\varkappa }_{{AB}}^{{BA}} & {\varkappa }_{{AB}}^{{BB}} & {\varkappa }_{{BC}}^{{AB}}+{\varkappa }_{{AC}}^{{BB}} & {\varkappa }_{{BC}}^{{AC}} & {\varkappa }_{{BC}}^{{AA}}+{\varkappa }_{{AC}}^{{BA}}\\ {\varkappa }_{{BA}}^{{BA}} & 2{\varkappa }_{{BA}}^{{BB}} & {\varkappa }_{{BB}}^{{BB}} & 2{\varkappa }_{{BC}}^{{BB}} & {\varkappa }_{{BC}}^{{BC}} & 2{\varkappa }_{{BC}}^{{BA}}\\ {\varkappa }_{{CB}}^{{BA}} & {\varkappa }_{{CB}}^{{BA}}+{\varkappa }_{{BB}}^{{CA}} & {\varkappa }_{{CB}}^{{BB}} & {\varkappa }_{{CC}}^{{BB}}+{\varkappa }_{{BC}}^{{CB}} & {\varkappa }_{{CC}}^{{BC}} & {\varkappa }_{{BC}}^{{BA}}+{\varkappa }_{{BC}}^{{BA}}\\ {\varkappa }_{{CA}}^{{CA}} & 2{\varkappa }_{{CB}}^{{CA}} & {\varkappa }_{{CB}}^{{CB}} & 2{\varkappa }_{{CC}}^{{CB}} & {\varkappa }_{{CC}}^{{CC}} & 2{\varkappa }_{{CC}}^{{CA}}\\ {\varkappa }_{{CA}}^{{AA}} & {\varkappa }_{{CB}}^{{AA}}+{\varkappa }_{{AB}}^{{CA}} & {\varkappa }_{{CB}}^{{AB}} & {\varkappa }_{{CC}}^{{AB}}+{\varkappa }_{{AC}}^{{CB}} & {\varkappa }_{{CC}}^{{AC}} & {\varkappa }_{{CC}}^{{AA}}+{\varkappa }_{{AC}}^{{CA}}\end{array}\right]}^{-1}\left[\begin{array}{l}{\widetilde{w}}_{{AA}}^{\mathrm{obs}}({\theta }_{k})\\ {\widetilde{w}}_{{AB}}^{\mathrm{obs}}({\theta }_{k})\\ {\widetilde{w}}_{{BB}}^{\mathrm{obs}}({\theta }_{k})\\ {\widetilde{w}}_{{BC}}^{\mathrm{obs}}({\theta }_{k})\\ {\widetilde{w}}_{{CC}}^{\mathrm{obs}}({\theta }_{k})\\ {\widetilde{w}}_{{CA}}^{\mathrm{obs}}({\theta }_{k})\\ \end{array}\right],\end{array}\end{eqnarray} \tag{ 110 }$$

where we have defined the following for brevity:

$$\begin{eqnarray}&&{\varkappa }_{{mn}}^{{uv}}=\displaystyle \frac{{\displaystyle \sum }_{i}^{{N}_{\mathrm{tot}}}{\displaystyle \sum }_{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{{A}_{u}{A}_{v}}{{\mathtt{q}}}_{i}^{{A}_{m}}{{\mathtt{q}}}_{j}^{{A}_{n}}}{{\displaystyle \sum }_{i}^{{N}_{\mathrm{tot}}}{\displaystyle \sum }_{j\ne i}^{{N}_{\mathrm{tot}}}{{\mathtt{w}}}_{{ij}}^{{A}_{u}{A}_{v}}}.\end{eqnarray} \tag{ 111 }$$

Extending the idea to M samples, we can write the analog of our unbiased estimator for Decontaminated Weighted, given by Equation (18), as

$$\begin{eqnarray}&&\begin{array}{l}\left[\begin{array}{c}{\widehat{w}}_{11}({\theta }_{k})\\ {\widehat{w}}_{12}({\theta }_{k})\\ \vdots \\ {\widehat{w}}_{\gamma \gamma }({\theta }_{k})\\ {\widehat{w}}_{\gamma (\gamma +1)}({\theta }_{k})\\ \vdots \\ {\widehat{w}}_{{MM}}({\theta }_{k})\\ {\widehat{w}}_{M1}({\theta }_{k})\\ \end{array}\right]\\ \,=\,{\left[\begin{array}{cccccccc}{\varkappa }_{11}^{11} & 2{\varkappa }_{12}^{11} & ... & {\varkappa }_{1\gamma }^{1\gamma } & 2{\varkappa }_{1(\gamma +1)}^{1\gamma } & ... & {\varkappa }_{1M}^{1M} & 2{\varkappa }_{1M}^{11}\\ {\varkappa }_{21}^{11} & {\varkappa }_{22}^{11}+{\varkappa }_{12}^{21} & ... & {\varkappa }_{2\gamma }^{1\gamma } & {\varkappa }_{2(\gamma +1)}^{1\gamma }+{\varkappa }_{1(\gamma +1)}^{2\gamma } & ... & {\varkappa }_{2M}^{1M} & {\varkappa }_{21}^{1M}+{\varkappa }_{11}^{2M}\\ \vdots & \vdots & ... & \vdots & \vdots & ... & \vdots & \vdots \\ {\varkappa }_{\gamma 1}^{\gamma 1} & 2{\varkappa }_{\gamma 2}^{\gamma 1} & ... & {\varkappa }_{\gamma \gamma }^{\gamma \gamma } & 2{\varkappa }_{\gamma (\gamma +1)}^{\gamma \gamma } & ... & {\varkappa }_{\gamma M}^{\gamma M} & 2{\varkappa }_{\gamma 1}^{\gamma M}\\ {\varkappa }_{(\gamma +1)1}^{\gamma 1} & {\varkappa }_{(\gamma +1)2}^{\gamma 1}+{\varkappa }_{\gamma 2}^{(\gamma +1)1} & ... & {\varkappa }_{(\gamma +1)\gamma }^{\gamma \gamma } & {\varkappa }_{(\gamma +1)(\gamma +1)}^{\gamma \gamma }+{\varkappa }_{\gamma (\gamma +1)}^{(\gamma +1)\gamma } & ... & {\varkappa }_{(\gamma +1)M}^{\gamma M} & {\varkappa }_{(\gamma +1)1}^{\gamma M}+{\varkappa }_{\gamma 1}^{(\gamma +1)M}\\ \vdots & \vdots & ... & \vdots & \vdots & ... & \vdots & \vdots \\ {\varkappa }_{M1}^{M1} & 2{\varkappa }_{M2}^{M1} & ... & {\varkappa }_{M\gamma }^{M\gamma } & 2{\varkappa }_{M(\gamma +1)}^{M\gamma } & ... & {\varkappa }_{{MM}}^{{MM}} & 2{\varkappa }_{M1}^{{MM}}\\ {\varkappa }_{11}^{M1} & {\varkappa }_{12}^{M1}+{\varkappa }_{M2}^{11} & ... & {\varkappa }_{1\gamma }^{M\gamma } & {\varkappa }_{1(\gamma +1)}^{M\gamma }+{\varkappa }_{M(\gamma +1)}^{1\gamma } & ... & {\varkappa }_{1M}^{{MM}} & {\varkappa }_{11}^{{MM}}\end{array}\right]}^{-1}\left[\begin{array}{c}{\widetilde{w}}_{11}^{\mathrm{obs}}({\theta }_{k})\\ {\widetilde{w}}_{12}^{\mathrm{obs}}({\theta }_{k})\\ \vdots \\ {\widetilde{w}}_{\gamma \gamma }^{\mathrm{obs}}({\theta }_{k})\\ {\widetilde{w}}_{\gamma (\gamma +1)}^{\mathrm{obs}}({\theta }_{k})\\ \vdots \\ {\widetilde{w}}_{{MM}}^{\mathrm{obs}}({\theta }_{k})\\ {\widetilde{w}}_{M1}^{\mathrm{obs}}({\theta }_{k})\end{array}\right].\end{array}\end{eqnarray} \tag{ 112 }$$