Astronomers Presented MeerCRAB, A Deep Learning Tool Which Classifies Between Real and Bogus Transients (Astronomy)

Contemporary large-scale optical surveys such as Skymapper, the Palomar Transient Factory (PTF), the Catalina Real-time Transient Survey (CRTS), the Panoramic Survey Telescope and Rapid Response System (Pan-STARRS1), the All-Sky Automated Survey for SuperNova (ASASSN), Gaia, the MeerLICHT telescope and the Zwicky Transient Factory (ZTF) are generating a plethora of transient events originating from a wide range of sources. These instruments enable us to observe and explore changes in millions of sources/candidates, thus unlocking new opportunities for interpreting and understanding large families of sources.

MeerLICHT is an optical wide-field telescope that is operated robotically. The telescope is located at the Sutherland station of the South African Astronomical Observatory (SAAO). It consists of a 65 cm primary mirror and provides a 2.7 square degree field-of-view at a pixel scale of 0.56″/pixel that maximises the volume of astrophysical candidates with brightnesses appropriate for spectroscopic follow-up using current large-aperture optical facilities. Both MeerLICHT and the BlackGEM array (that is currently being installed at the La Silla Observatory in Chile) will yield about 500 observations per night, per telescope, thus generating hundreds of candidate alerts every clear night that could be spectroscopically followed up. BlackGEM’s main focus is on the detection of optical counterparts to gravitational wave events and MeerLICHT is used to co-observe the sky as seen with the MeerKAT radio array. MeerLICHT and BlackGEM are technically identical with MeerLICHT being the prototype for the BlackGEM array.

Transients and variables are sources that vary on all timescales (from milliseconds up to years) and they vary significantly from a reference image – either an increase or decrease in brightness. Transients include phenomena such as supernovae, gamma-ray bursts, tidal disruption events and flare stars, to name a few. A successful transient follow-up program enables the creation of a large database of transient and variable sources. Such large databases are important for future analyses of data collected during upcoming photometric surveys such as those conducted at the Vera C. Rubin observatory (LSST).

“While we possess a reasonable understanding of many transient sources, achieved via consideration of their spectra, the main goal of surveys undertaken with MeerLICHT is to find and select the subset of sources that are not well understood. This will help us to increase our knowledge of the different families of transients and variable stars. Secondly, given that transients are rapidly fading sources due to their often destructive nature, MeerLICHT aims to identify transients rapidly, as they are only visible for a limited amount of time for follow-up.”

In order to have an early and rapid characterisation of these sources, it is fundamentally important to automate several steps within a transient detection pipeline, including the separation of transients/astrophysical events from “bogus” detections, which has become a bottle-neck in fast detection pipelines. So called “bogus” detections can occur as a result of saturated sources, convolution problems, defects in the detector, atmospheric dispersion, unmodeled differences at the subtraction stage and cosmic rays passing through the detector, amongst other things.

Examples of bogus and real transients in the Meer-LICHT database. Each column represents the new (N), reference (R), difference (D) and significance (S) images and the rows are the different fields. © Hosenie et al.

Most surveys use three images for transient event detection and extraction: (i) an early observation of the relevant sky (also known as the template/reference image), (ii) a calibrated recent image (New/Science image), (iii) the difference image which is formed by subtracting the reference image from the new/science image. Using the difference image, one can, in principle, effectively detect transients, however, in many cases, the subtracted image contains bogus sources.

Therefore to be successful, surveys require an automated way to distinguish between real and bogus candidates. To address this challenging task, most of the time-domain surveys mentioned previously have adopted machine learning (ML) algorithms to perform real-bogus classification. Convolutional neural networks (CNNs) have been used in the image domain as feature extractors for automatic vetting algorithms, for example, during the Skymapper Survey, the High cadence Transient Survey (HiTS) and the ZTF similarly utilized deep learning techniques. Other ML techniques such as Random Forest (RF) and k-Nearest Neighbour (k-NN) classification approaches have been employed to classify light curve transients from CRTS.

The classification task in these surveys is usually separated into two distinct steps. Firstly, bogus candidates are filtered out from real sources immediately after acquiring data, that is, the classification between real and bogus. The second stage involves assigning an astrophysical category/class label to each detected transient based on its spectroscopic or photometric information. Now, Z. Hosenie and colleagues, in their recent paper, focused on the automation of the first stage, that is, the classification of sources as either Real or Bogus using deep learning methods developed for the MeerLICHT facility.

Decision tree characterising real and bogus candidates. Vetters used this guide to label each candidate and to construct a large training set for MeerCRAB. © Hosenie et al.

They presented a deep learning pipeline based on the convolutional neural network architecture called MeerCRAB. It is designed to filter out the so called “bogus” detections from true astrophysical sources in the transient detection pipeline of the MeerLICHT telescope.

“In practice, by using MeerCRAB we can significantly reduce the number of missed transients per night and this may have a great impact on detecting and classifying the unknown unknowns of our universe.”

They also detailed the process of developing MeerCRAB. To be able to train a deep neural network, they first, constructed a large, high-quality labelled and representative dataset. To do so, they developed a vetting guidelines for vetters and taught them how real or bogus candidates in the MeerLICHT data appear.

An example of the procedure for selecting the candidates for training the CNN using the thresholding approach, applied on the 5000 candidates. In this example, Atleast 9 (T9) is applied and they note that 623 candidates are discarded. Then, the remaining candidates (4377) are split into training, validation and test set for
training and evaluation processes. © Hosenie et al.

Then, a sample of 5000 candidates were provided to 10 vetters for labelling. Based on the vetters labels, they applied two methods to assign the final labelling to each candidate: (i) the thresholding method (T8, T9 & T10) and (ii) latent class model L𝑙𝑐𝑚. At T9, a source is labelled as real if atleast 9 out of the 10 vetters labelled it as real or vice-versa. They found that T9 is a good threshold criteria to be used since they have enough samples for training and testing the models, hence providing high quality labelled data. They also found that going lower than this (i.e. T7, T8) or using L𝑙𝑐𝑚, added noisy labels. When used to train the network, such data decreased the performance of the models.

The three network architectures considered in this work: MeerCRAB1, MeerCRAB2 and MeerCRAB3. They showed four images grouped together (new, reference, difference and significance) to form the input of the networks, followed by convolutional layers, max-pooling, dropout and dense layers. At the end, the network outputs a probability whether a candidate is either real or bogus during the prediction phase. © Hosenie et al.

Moreover, they demonstrated that by increasing the depth of the network, (MeerCRAB1 to MeerCRAB3), the performance of the model increases as well. McNemar’s statistical test showed that MeerCRAB3 performs better than MeerCRAB1 and MeerCRAB2. In addition, they used a combination of input images (new (𝑁), reference (𝑅), difference (𝐷), significance (𝑆)) as input to the three networks. They found that with only 𝑁 𝑅, they obtained competitive results. They also observed that adding the difference and significance images improves network performance.

Table 1: The results for various labelling methods are presented in terms of precision, recall, accuracy and MCC values using NRD as input to the three models © Hosenie et al.

Finally, they concluded that the best performing model has the following configuration: T9 with MeerCRAB3 having 𝑁𝑅𝐷 as input. This model yields an accuracy of 99.5 % and Matthew’s Correlation Coefficient (MCC) value of 0.989. The best model was integrated to the MeerLICHT transient vetting pipeline, enabling the accurate and efficient classification of detected transients that allows researchers to select the most promising candidates for their research goals.


Reference: Zafiirah Hosenie, Steven Bloemen, Paul Groot, Robert Lyon, Bart Scheers, Benjamin Stappers, Fiorenzo Stoppa, Paul Vreeswijk, Simon De Wet, Marc Klein Wolt, Elmar Körding, Vanessa McBride, Rudolf Le Poole, Kerry Paterson, Daniëlle L. A. Pieterse, Patrick Woudt, “MeerCRAB: MeerLICHT Classification of Real and Bogus Transients using Deep Learning”, Arxiv, pp. 1-15, 2021. https://arxiv.org/abs/2104.13950


Note for editors of other websites: To reuse this article partially or fully kindly give credit to our author S. Aman or either provide a reference link of our article

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s