A critical review of the S/L reliability index

The meaning and properties of a commonly used index of reliability, S/L, were examined critically. It was found that the index does not reflect any conventional concept of reliability. When used for an identical behavioral observation session, it is not statistically correlated with other reliability indices. Within an observation session, the standardizing measure of L is beyond the control of the investigator. Furthermore, the reason for the choice of L as the standard is unclear. The role of chance agreement in S/L is not known. The exact interpretation of the index depends on which observer reports L. Overall the conceptual and mathematical meaning of S/L is dubious. It is suggested that the S/L index should not be used until its nature is shown to be a measure of reliability. Other approaches such as the intraclass correlations and generalizability coefficients should be used instead.