Testing Closeness of Discrete Distributions

Given samples from two distributions over an n-element set, we wish to test whether these distributions are statistically close. We present an algorithm which uses sublinear in n, specifically, O(n[superscript 2/3]ε[superscript −8/3] log n), independent samples from each distribution, runs in time l...

وصف كامل

التفاصيل البيبلوغرافية
المؤلفون الرئيسيون: Batu, Tugkan, Fortnow, Lance, Rubinfeld, Ronitt, Smith, Warren D., White, Patrick
مؤلفون آخرون: Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
التنسيق: مقال
اللغة:en_US
منشور في: Association for Computing Machinery (ACM) 2014
الوصول للمادة أونلاين:http://hdl.handle.net/1721.1/90630
https://orcid.org/0000-0002-4353-7639
الوصف
الملخص:Given samples from two distributions over an n-element set, we wish to test whether these distributions are statistically close. We present an algorithm which uses sublinear in n, specifically, O(n[superscript 2/3]ε[superscript −8/3] log n), independent samples from each distribution, runs in time linear in the sample size, makes no assumptions about the structure of the distributions, and distinguishes the cases when the distance between the distributions is small (less than {ε[superscript 4/3]n[superscript −1/3]/32, εn[superscript −1/2]/4}) or large (more than ε) in ℓ[subscript 1] distance. This result can be compared to the lower bound of Ω(n[superscript 2/3]ε[superscript −2/3]) for this problem given by Valiant [2008]. Our algorithm has applications to the problem of testing whether a given Markov process is rapidly mixing. We present sublinear algorithms for several variants of this problem as well.