A kernelised Stein statistic for assessing implicit generative models
Synthetic data generation has become a key ingredient for training machine learning procedures, addressing tasks such as data augmentation, analysing privacy-sensitive data, or visualising representative samples. Assessing the quality of such synthetic data generators hence has to be addressed. As (...
| Main Authors: | , |
|---|---|
| Format: | Conference item |
| Language: | English |
| Published: |
Curran Associates
2023
|
| _version_ | 1826312068404346880 |
|---|---|
| author | Xu, W Reinert, G |
| author_facet | Xu, W Reinert, G |
| author_sort | Xu, W |
| collection | OXFORD |
| description | Synthetic data generation has become a key ingredient for training machine learning procedures, addressing tasks such as data augmentation, analysing privacy-sensitive data, or visualising representative samples. Assessing the quality of such synthetic data generators hence has to be addressed. As (deep) generative models for synthetic data often do not admit explicit probability distributions, classical statistical procedures for assessing model goodness-of-fit may not be applicable. In this paper, we propose a principled procedure to assess the quality of a synthetic data generator. The procedure is a Kernelised Stein Discrepancy-type test which is based on a non-parametric Stein operator for the synthetic data generator of interest. This operator is estimated from samples which are obtained from the synthetic data generator and hence can be applied even when the model is only implicit. In contrast to classical testing, the sample size from the synthetic data generator can be as large as desired, while the size of the observed data that the generator aims to emulate is fixed. Experimental results on synthetic distributions and trained generative models on synthetic and real datasets illustrate that the method shows improved power performance compared to existing approaches. |
| first_indexed | 2024-03-07T08:22:05Z |
| format | Conference item |
| id | oxford-uuid:bae57014-1242-4f01-a308-31ff55d5f788 |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-03-07T08:22:05Z |
| publishDate | 2023 |
| publisher | Curran Associates |
| record_format | dspace |
| spelling | oxford-uuid:bae57014-1242-4f01-a308-31ff55d5f7882024-01-30T11:26:57ZA kernelised Stein statistic for assessing implicit generative modelsConference itemhttp://purl.org/coar/resource_type/c_5794uuid:bae57014-1242-4f01-a308-31ff55d5f788EnglishSymplectic Elements Curran Associates2023Xu, WReinert, GSynthetic data generation has become a key ingredient for training machine learning procedures, addressing tasks such as data augmentation, analysing privacy-sensitive data, or visualising representative samples. Assessing the quality of such synthetic data generators hence has to be addressed. As (deep) generative models for synthetic data often do not admit explicit probability distributions, classical statistical procedures for assessing model goodness-of-fit may not be applicable. In this paper, we propose a principled procedure to assess the quality of a synthetic data generator. The procedure is a Kernelised Stein Discrepancy-type test which is based on a non-parametric Stein operator for the synthetic data generator of interest. This operator is estimated from samples which are obtained from the synthetic data generator and hence can be applied even when the model is only implicit. In contrast to classical testing, the sample size from the synthetic data generator can be as large as desired, while the size of the observed data that the generator aims to emulate is fixed. Experimental results on synthetic distributions and trained generative models on synthetic and real datasets illustrate that the method shows improved power performance compared to existing approaches. |
| spellingShingle | Xu, W Reinert, G A kernelised Stein statistic for assessing implicit generative models |
| title | A kernelised Stein statistic for assessing implicit generative models |
| title_full | A kernelised Stein statistic for assessing implicit generative models |
| title_fullStr | A kernelised Stein statistic for assessing implicit generative models |
| title_full_unstemmed | A kernelised Stein statistic for assessing implicit generative models |
| title_short | A kernelised Stein statistic for assessing implicit generative models |
| title_sort | kernelised stein statistic for assessing implicit generative models |
| work_keys_str_mv | AT xuw akernelisedsteinstatisticforassessingimplicitgenerativemodels AT reinertg akernelisedsteinstatisticforassessingimplicitgenerativemodels AT xuw kernelisedsteinstatisticforassessingimplicitgenerativemodels AT reinertg kernelisedsteinstatisticforassessingimplicitgenerativemodels |