Strong lens time delay challenge. II. Results of TDC1

Kai Liao, Tommaso Treu, Phil Marshall, Christopher D. Fassnacht, Nick Rumbaugh, Gregory Dobler, Amir Aghamousa, Vivien Bonvin, Frederic Courbin, Alireza Hojjati, Neal Jackson, Vinay Kashyap, S. Rathna Kumar, Eric Linder, Kaisey Mandel, Xiao Li Meng, Georges Meylan, Leonidas A. Moustakas, Tushar P. Prabhu, Andrew Romero-WolfArman Shafieloo, Aneta Siemiginowska, Chelliah S. Stalin, Hyungsuk Tak, Malte Tewes, David Van Dyk

Research output: Contribution to journalArticle

57 Citations (Scopus)

Abstract

We present the results of the first strong lens time delay challenge. The motivation, experimental design, and entry level challenge are described in a companion paper. This paper presents the main challenge, TDC1, which consisted of analyzing thousands of simulated light curves blindly. The observational properties of the light curves cover the range in quality obtained for current targeted efforts (e.g., COSMOGRAIL) and expected from future synoptic surveys (e.g., LSST), and include simulated systematic errors. Seven teams participated in TDC1, submitting results from 78 different method variants. After describing each method, we compute and analyze basic statistics measuring accuracy (or bias) A, goodness of fit χ2, precision P, and success rate f. For some methods we identify outliers as an important issue. Other methods show that outliers can be controlled via visual inspection or conservative quality control. Several methods are competitive, i.e., give |A| < 0.03, P < 0.03, and χ2 < 1.5, with some of the methods already reaching sub-percent accuracy. The fraction of light curves yielding a time delay measurement is typically in the range f = 20%-40%. It depends strongly on the quality of the data: COSMOGRAIL-quality cadence and light curve lengths yield significantly higher f than does sparser sampling. Taking the results of TDC1 at face value, we estimate that LSST should provide around 400 robust time-delay measurements, each with P < 0.03 and |A| < 0.01, comparable to current lens modeling uncertainties. In terms of observing strategies, we find that A and f depend mostly on season length, while P depends mostly on cadence and campaign duration.

Original languageEnglish (US)
Article numberL11
JournalAstrophysical Journal Letters
Volume800
Issue number1
DOIs
StatePublished - Feb 10 2015

Fingerprint

light curve
time lag
lenses
outlier
goodness of fit
quality control
entry
systematic errors
inspection
sampling
statistics
data quality
experimental design
method
curves
estimates
modeling

All Science Journal Classification (ASJC) codes

  • Astronomy and Astrophysics
  • Space and Planetary Science

Cite this

Liao, K., Treu, T., Marshall, P., Fassnacht, C. D., Rumbaugh, N., Dobler, G., ... Van Dyk, D. (2015). Strong lens time delay challenge. II. Results of TDC1. Astrophysical Journal Letters, 800(1), [L11]. https://doi.org/10.1088/0004-637X/800/1/11
Liao, Kai ; Treu, Tommaso ; Marshall, Phil ; Fassnacht, Christopher D. ; Rumbaugh, Nick ; Dobler, Gregory ; Aghamousa, Amir ; Bonvin, Vivien ; Courbin, Frederic ; Hojjati, Alireza ; Jackson, Neal ; Kashyap, Vinay ; Rathna Kumar, S. ; Linder, Eric ; Mandel, Kaisey ; Meng, Xiao Li ; Meylan, Georges ; Moustakas, Leonidas A. ; Prabhu, Tushar P. ; Romero-Wolf, Andrew ; Shafieloo, Arman ; Siemiginowska, Aneta ; Stalin, Chelliah S. ; Tak, Hyungsuk ; Tewes, Malte ; Van Dyk, David. / Strong lens time delay challenge. II. Results of TDC1. In: Astrophysical Journal Letters. 2015 ; Vol. 800, No. 1.
@article{b360f372287d4e179e7b07720733e840,
title = "Strong lens time delay challenge. II. Results of TDC1",
abstract = "We present the results of the first strong lens time delay challenge. The motivation, experimental design, and entry level challenge are described in a companion paper. This paper presents the main challenge, TDC1, which consisted of analyzing thousands of simulated light curves blindly. The observational properties of the light curves cover the range in quality obtained for current targeted efforts (e.g., COSMOGRAIL) and expected from future synoptic surveys (e.g., LSST), and include simulated systematic errors. Seven teams participated in TDC1, submitting results from 78 different method variants. After describing each method, we compute and analyze basic statistics measuring accuracy (or bias) A, goodness of fit χ2, precision P, and success rate f. For some methods we identify outliers as an important issue. Other methods show that outliers can be controlled via visual inspection or conservative quality control. Several methods are competitive, i.e., give |A| < 0.03, P < 0.03, and χ2 < 1.5, with some of the methods already reaching sub-percent accuracy. The fraction of light curves yielding a time delay measurement is typically in the range f = 20{\%}-40{\%}. It depends strongly on the quality of the data: COSMOGRAIL-quality cadence and light curve lengths yield significantly higher f than does sparser sampling. Taking the results of TDC1 at face value, we estimate that LSST should provide around 400 robust time-delay measurements, each with P < 0.03 and |A| < 0.01, comparable to current lens modeling uncertainties. In terms of observing strategies, we find that A and f depend mostly on season length, while P depends mostly on cadence and campaign duration.",
author = "Kai Liao and Tommaso Treu and Phil Marshall and Fassnacht, {Christopher D.} and Nick Rumbaugh and Gregory Dobler and Amir Aghamousa and Vivien Bonvin and Frederic Courbin and Alireza Hojjati and Neal Jackson and Vinay Kashyap and {Rathna Kumar}, S. and Eric Linder and Kaisey Mandel and Meng, {Xiao Li} and Georges Meylan and Moustakas, {Leonidas A.} and Prabhu, {Tushar P.} and Andrew Romero-Wolf and Arman Shafieloo and Aneta Siemiginowska and Stalin, {Chelliah S.} and Hyungsuk Tak and Malte Tewes and {Van Dyk}, David",
year = "2015",
month = "2",
day = "10",
doi = "10.1088/0004-637X/800/1/11",
language = "English (US)",
volume = "800",
journal = "Astrophysical Journal Letters",
issn = "2041-8205",
publisher = "IOP Publishing Ltd.",
number = "1",

}

Liao, K, Treu, T, Marshall, P, Fassnacht, CD, Rumbaugh, N, Dobler, G, Aghamousa, A, Bonvin, V, Courbin, F, Hojjati, A, Jackson, N, Kashyap, V, Rathna Kumar, S, Linder, E, Mandel, K, Meng, XL, Meylan, G, Moustakas, LA, Prabhu, TP, Romero-Wolf, A, Shafieloo, A, Siemiginowska, A, Stalin, CS, Tak, H, Tewes, M & Van Dyk, D 2015, 'Strong lens time delay challenge. II. Results of TDC1', Astrophysical Journal Letters, vol. 800, no. 1, L11. https://doi.org/10.1088/0004-637X/800/1/11

Strong lens time delay challenge. II. Results of TDC1. / Liao, Kai; Treu, Tommaso; Marshall, Phil; Fassnacht, Christopher D.; Rumbaugh, Nick; Dobler, Gregory; Aghamousa, Amir; Bonvin, Vivien; Courbin, Frederic; Hojjati, Alireza; Jackson, Neal; Kashyap, Vinay; Rathna Kumar, S.; Linder, Eric; Mandel, Kaisey; Meng, Xiao Li; Meylan, Georges; Moustakas, Leonidas A.; Prabhu, Tushar P.; Romero-Wolf, Andrew; Shafieloo, Arman; Siemiginowska, Aneta; Stalin, Chelliah S.; Tak, Hyungsuk; Tewes, Malte; Van Dyk, David.

In: Astrophysical Journal Letters, Vol. 800, No. 1, L11, 10.02.2015.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Strong lens time delay challenge. II. Results of TDC1

AU - Liao, Kai

AU - Treu, Tommaso

AU - Marshall, Phil

AU - Fassnacht, Christopher D.

AU - Rumbaugh, Nick

AU - Dobler, Gregory

AU - Aghamousa, Amir

AU - Bonvin, Vivien

AU - Courbin, Frederic

AU - Hojjati, Alireza

AU - Jackson, Neal

AU - Kashyap, Vinay

AU - Rathna Kumar, S.

AU - Linder, Eric

AU - Mandel, Kaisey

AU - Meng, Xiao Li

AU - Meylan, Georges

AU - Moustakas, Leonidas A.

AU - Prabhu, Tushar P.

AU - Romero-Wolf, Andrew

AU - Shafieloo, Arman

AU - Siemiginowska, Aneta

AU - Stalin, Chelliah S.

AU - Tak, Hyungsuk

AU - Tewes, Malte

AU - Van Dyk, David

PY - 2015/2/10

Y1 - 2015/2/10

N2 - We present the results of the first strong lens time delay challenge. The motivation, experimental design, and entry level challenge are described in a companion paper. This paper presents the main challenge, TDC1, which consisted of analyzing thousands of simulated light curves blindly. The observational properties of the light curves cover the range in quality obtained for current targeted efforts (e.g., COSMOGRAIL) and expected from future synoptic surveys (e.g., LSST), and include simulated systematic errors. Seven teams participated in TDC1, submitting results from 78 different method variants. After describing each method, we compute and analyze basic statistics measuring accuracy (or bias) A, goodness of fit χ2, precision P, and success rate f. For some methods we identify outliers as an important issue. Other methods show that outliers can be controlled via visual inspection or conservative quality control. Several methods are competitive, i.e., give |A| < 0.03, P < 0.03, and χ2 < 1.5, with some of the methods already reaching sub-percent accuracy. The fraction of light curves yielding a time delay measurement is typically in the range f = 20%-40%. It depends strongly on the quality of the data: COSMOGRAIL-quality cadence and light curve lengths yield significantly higher f than does sparser sampling. Taking the results of TDC1 at face value, we estimate that LSST should provide around 400 robust time-delay measurements, each with P < 0.03 and |A| < 0.01, comparable to current lens modeling uncertainties. In terms of observing strategies, we find that A and f depend mostly on season length, while P depends mostly on cadence and campaign duration.

AB - We present the results of the first strong lens time delay challenge. The motivation, experimental design, and entry level challenge are described in a companion paper. This paper presents the main challenge, TDC1, which consisted of analyzing thousands of simulated light curves blindly. The observational properties of the light curves cover the range in quality obtained for current targeted efforts (e.g., COSMOGRAIL) and expected from future synoptic surveys (e.g., LSST), and include simulated systematic errors. Seven teams participated in TDC1, submitting results from 78 different method variants. After describing each method, we compute and analyze basic statistics measuring accuracy (or bias) A, goodness of fit χ2, precision P, and success rate f. For some methods we identify outliers as an important issue. Other methods show that outliers can be controlled via visual inspection or conservative quality control. Several methods are competitive, i.e., give |A| < 0.03, P < 0.03, and χ2 < 1.5, with some of the methods already reaching sub-percent accuracy. The fraction of light curves yielding a time delay measurement is typically in the range f = 20%-40%. It depends strongly on the quality of the data: COSMOGRAIL-quality cadence and light curve lengths yield significantly higher f than does sparser sampling. Taking the results of TDC1 at face value, we estimate that LSST should provide around 400 robust time-delay measurements, each with P < 0.03 and |A| < 0.01, comparable to current lens modeling uncertainties. In terms of observing strategies, we find that A and f depend mostly on season length, while P depends mostly on cadence and campaign duration.

UR - http://www.scopus.com/inward/record.url?scp=84922577830&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84922577830&partnerID=8YFLogxK

U2 - 10.1088/0004-637X/800/1/11

DO - 10.1088/0004-637X/800/1/11

M3 - Article

AN - SCOPUS:84922577830

VL - 800

JO - Astrophysical Journal Letters

JF - Astrophysical Journal Letters

SN - 2041-8205

IS - 1

M1 - L11

ER -

Liao K, Treu T, Marshall P, Fassnacht CD, Rumbaugh N, Dobler G et al. Strong lens time delay challenge. II. Results of TDC1. Astrophysical Journal Letters. 2015 Feb 10;800(1). L11. https://doi.org/10.1088/0004-637X/800/1/11