Noise Removal

The CBVCorrector

Here we will use CBVs to remove our noise.

Check out our CBV tutorial for more information.

First, we must import the CBVCorrector from lightkurve.correctors, and then we can refer to our generate light curve.

from lightkurve.correctors import CBVCorrector
cbvCorrector = CBVCorrector(lc_S32, interpolate_cbvs=True)
cbvCorrector.cbvs

[TESS CBVs, Sector.Camera.CCD : 32.2.4, CBVType : SingleScale, nCBVS : 16,
 TESS CBVs, Sector.Camera.CCD : 32.2.4, CBVType.Band: MultiScale.1, nCBVs : 8,
 TESS CBVs, Sector.Camera.CCD : 32.2.4, CBVType.Band: MultiScale.2, nCBVs : 8,
 TESS CBVs, Sector.Camera.CCD : 32.2.4, CBVType.Band: MultiScale.3, nCBVs : 5,
 TESS CBVs, Sector.Camera.CCD : 32.2.4, CBVType : Spike, nCBVS : 7]

There are 5 sets of CBVs, all associated with Sector 32, Camera 2, CCD 4. The number of CBVs per type is also given. Let’s plot the Single-Scale CBVs, which contain all systematics combined.

cbvCorrector.cbvs[0].plot()

<matplotlib.axes._subplots.AxesSubplot at 0x7f8dd99f8810>

The first several CBVs contain most of the systematics. The latter CBVs pose a greater risk of injecting more noise than helping. The default behavior in CBVCorrector is to use the first 8 CBVs.

Next, we need to decide what kind of CBV we want to apply. Since we want to try not to overfit, we are going to use the Multi-Scale and Spike options (see above for more details). We can specify the type via an array as indicated below.

cbv_type = ['MultiScale.1', 'MultiScale.2', 'MultiScale.3','Spike']

Since we only want to use the information from the first 8 CBV’s when we use the SingleScale function, and all the Spike CBCs, we must also specify this via an array.

cbv_indices = [np.arange(1,9), np.arange(1,9), np.arange(1,9), 'ALL']

Now we want to check that the application of these vectors will not under or over-fit our data. There is a handy way of checking this via the use of a goodness_metric_scan_plot. A “Goodness Metric” value of 0.8 and above tends to indicate a good fit to the data.

cbvCorrector.goodness_metric_scan_plot(cbv_type=cbv_type, cbv_indices=cbv_indices);
plt.plot([1.5e-1, 1.5e-1], [0,1])

[<matplotlib.lines.Line2D at 0x7f8d8a297290>]

The above plot indicates that a regularization factor alpha value of 1.5e-1 is appropriate for our data. We can now specify this and check it via a diagnose method.

cbvcorrector = CBVCorrector(lc_S32, interpolate_cbvs=True)
cbvcorrector.correct_gaussian_prior(cbv_type=cbv_type, cbv_indices=cbv_indices, alpha=1.5e-1)
pltAxis = cbvcorrector.diagnose()
pltAxis[0].set_ylim(300, 500);
pltAxis[1].set_ylim(300, 500);

cbvcorrector_lc = cbvcorrector.corrected_lc
cbvcorrector_lc.remove_outliers().plot()

<matplotlib.axes._subplots.AxesSubplot at 0x7f8debd37ad0>

We can see that a correction was applied which has removed some of the slope from our data and also labeled the big spike as an outlier.

The above method can be adjusted more, but for now, let’s move onto our next method.

The RegressionCorrector

Remember that this method relies on looking at pixels outside of our previously defined object aperture to determine our systematics. We can apply regression correction as follows.

#Import the packages you need
from lightkurve.correctors import RegressionCorrector, DesignMatrix

Next, we are going to define a design matrix which will contain all the systematics detected in the pixels outside of our aperture.

# Make a design matrix and pass it to a linear regression corrector
dm = DesignMatrix(S32.flux[:, ~aper_new], name='regressors')#.pca(5).append_constant()

We only want the most dominant vectors and so specify the number of principle componant (PCA) as follows,

dm = dm.pca(10)
plt.plot(S32.time.value, dm.values + np.arange(10)*0.2, '.');

From this we can see that 10 is excessive and we should probably use only 5. Let’s do this and plot it up.

dm = DesignMatrix(S32.flux[:, ~aper_new], name='regressors').pca(5)
dm.plot()

<matplotlib.axes._subplots.AxesSubplot at 0x7f8db9fa0750>

Lightkurve’s RegressionCorrector uses linear algebra to find the combination of vectors that makes the input light curve closest to zero. We therefore needed to add one more component - an “offset” term, to be able to fit the mean level of the light curve. We can do this as shown above by appending a“constant” to the design matrix.

dm = DesignMatrix(S32.flux[:, ~aper_new], name='regressors').pca(5).append_constant()

Next we pass this design matrix to the RegressionCorrector which de-trends the light curves against the vectors.

rc = RegressionCorrector(lc_S32)
rc.correct(dm)
pltAxis = rc.diagnose()
pltAxis[0].set_ylim(300, 500);
pltAxis[1].set_ylim(300, 500);

corrected_ffi_lc = rc.correct(dm)

HOWEVER! The RegressionCorrector assumes that you want to remove the trend and set the light curve to the mean level.

This isn’t true for TESS scattered light, as the TESS FFI light curves have an additive background. As such, we want to reduce the flux to the lowest recorded level.

To do this, we can look at the model of the background that RegressionCorrector built and apply that. This model should never go below zero, to ensure that this is the case we only subtract the model flux value at the 5th percentile.

We can then apply all these factors to the FFI light curve using the following code,

# Optional: Remove the scattered light, allowing for the large offset from scattered light
corrected_ffi_lc = lc_S32 - rc.model_lc + np.percentile(rc.model_lc.flux, 5)
corrected_ffi_lc.plot(label='Corrected light curve');

This has removed all the scattered light and the slope, but again could be adjusted further. However, we will move onto our final tool.

The PLDCorrector

PLD is built on top of RegressionCorrector and again works by identifying a set of trends in the pixels surrounding the target star, and performing linear regression to create a combination of these trends that effectively models the noise. This noise model is then subtracted from the uncorrected light curve. The difference between PLDCorrector and RegressionCorrector is that RegressionCorrector asks the user to define the pixels, where as PLDCorrector does not.

We will create a PLDCorrector object, and use the default values for PLDCorrector.correct to remove this scattered light background.

from lightkurve.correctors import PLDCorrector
pld_S32 = PLDCorrector(S32)
pld_S32.correct(pca_components=5)
pltAxis = pld_S32.diagnose()
pltAxis[0].set_ylim(300, 500);
pltAxis[1].set_ylim(300, 500);
pltAxis[2].set_ylim(300, 500);

corrected_pldlc = pld_S32.correct(pca_components=5)

We can also examine the apertures used to perform this correction. For TESS, the dominant source of noise is the scattered light background, so by default only those pixels will be used. In the third panel, we can see that the background_aperture_mask contains only background pixels, reducing the risk of contamination by neighboring stars.

pld_S32.diagnose_masks();

Now lets specifically look at our corrected light curve.

corrected_pldlc.plot()

<matplotlib.axes._subplots.AxesSubplot at 0x7f8d9df0c690>

The large spike from the scattered light has sucessfully been identified and removed, but there is a slope in our data still. We can determine from the diagnostic plots that this is due to the application of a flat background, unlike that which was applied in RegressionCorrection.

Note that there are various componants that can be altered in PLD most of which can be found here.