Fuzzy Clustering Methods with Rényi Relative Entropy and Cluster Size
In the last two decades, information entropy measures have been relevantly applied in fuzzy clustering problems in order to regularize solutions by avoiding the formation of partitions with excessively overlapping clusters. Following this idea, relative entropy or divergence measures have been simil...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-06-01
|
| Series: | Mathematics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2227-7390/9/12/1423 |
| _version_ | 1797529650634686464 |
|---|---|
| author | Javier Bonilla Daniel Vélez Javier Montero J. Tinguaro Rodríguez |
| author_facet | Javier Bonilla Daniel Vélez Javier Montero J. Tinguaro Rodríguez |
| author_sort | Javier Bonilla |
| collection | DOAJ |
| description | In the last two decades, information entropy measures have been relevantly applied in fuzzy clustering problems in order to regularize solutions by avoiding the formation of partitions with excessively overlapping clusters. Following this idea, relative entropy or divergence measures have been similarly applied, particularly to enable that kind of entropy-based regularization to also take into account, as well as interact with, cluster size variables. Particularly, since Rényi divergence generalizes several other divergence measures, its application in fuzzy clustering seems promising for devising more general and potentially more effective methods. However, previous works making use of either Rényi entropy or divergence in fuzzy clustering, respectively, have not considered cluster sizes (thus applying regularization in terms of entropy, not divergence) or employed divergence without a regularization purpose. Then, the main contribution of this work is the introduction of a new regularization term based on Rényi relative entropy between membership degrees and observation ratios per cluster to penalize overlapping solutions in fuzzy clustering analysis. Specifically, such Rényi divergence-based term is added to the variance-based Fuzzy C-means objective function when allowing cluster sizes. This then leads to the development of two new fuzzy clustering methods exhibiting Rényi divergence-based regularization, the second one extending the first by considering a Gaussian kernel metric instead of the Euclidean distance. Iterative expressions for these methods are derived through the explicit application of Lagrange multipliers. An interesting feature of these expressions is that the proposed methods seem to take advantage of a greater amount of information in the updating steps for membership degrees and observations ratios per cluster. Finally, an extensive computational study is presented showing the feasibility and comparatively good performance of the proposed methods. |
| first_indexed | 2024-03-10T10:16:45Z |
| format | Article |
| id | doaj.art-a12dad5d886c413d84da94c27247ee00 |
| institution | Directory Open Access Journal |
| issn | 2227-7390 |
| language | English |
| last_indexed | 2024-03-10T10:16:45Z |
| publishDate | 2021-06-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Mathematics |
| spelling | doaj.art-a12dad5d886c413d84da94c27247ee002023-11-22T00:44:44ZengMDPI AGMathematics2227-73902021-06-01912142310.3390/math9121423Fuzzy Clustering Methods with Rényi Relative Entropy and Cluster SizeJavier Bonilla0Daniel Vélez1Javier Montero2J. Tinguaro Rodríguez3Department of Statistics and Operations Research, Universidad Complutense de Madrid, 28040 Madrid, SpainDepartment of Statistics and Operations Research, Universidad Complutense de Madrid, 28040 Madrid, SpainDepartment of Statistics and Operations Research, Universidad Complutense de Madrid, 28040 Madrid, SpainDepartment of Statistics and Operations Research, Universidad Complutense de Madrid, 28040 Madrid, SpainIn the last two decades, information entropy measures have been relevantly applied in fuzzy clustering problems in order to regularize solutions by avoiding the formation of partitions with excessively overlapping clusters. Following this idea, relative entropy or divergence measures have been similarly applied, particularly to enable that kind of entropy-based regularization to also take into account, as well as interact with, cluster size variables. Particularly, since Rényi divergence generalizes several other divergence measures, its application in fuzzy clustering seems promising for devising more general and potentially more effective methods. However, previous works making use of either Rényi entropy or divergence in fuzzy clustering, respectively, have not considered cluster sizes (thus applying regularization in terms of entropy, not divergence) or employed divergence without a regularization purpose. Then, the main contribution of this work is the introduction of a new regularization term based on Rényi relative entropy between membership degrees and observation ratios per cluster to penalize overlapping solutions in fuzzy clustering analysis. Specifically, such Rényi divergence-based term is added to the variance-based Fuzzy C-means objective function when allowing cluster sizes. This then leads to the development of two new fuzzy clustering methods exhibiting Rényi divergence-based regularization, the second one extending the first by considering a Gaussian kernel metric instead of the Euclidean distance. Iterative expressions for these methods are derived through the explicit application of Lagrange multipliers. An interesting feature of these expressions is that the proposed methods seem to take advantage of a greater amount of information in the updating steps for membership degrees and observations ratios per cluster. Finally, an extensive computational study is presented showing the feasibility and comparatively good performance of the proposed methods.https://www.mdpi.com/2227-7390/9/12/1423fuzzy clusteringentropyrelative entropyRényi entropydifferential evolution algorithmGaussian kernel |
| spellingShingle | Javier Bonilla Daniel Vélez Javier Montero J. Tinguaro Rodríguez Fuzzy Clustering Methods with Rényi Relative Entropy and Cluster Size Mathematics fuzzy clustering entropy relative entropy Rényi entropy differential evolution algorithm Gaussian kernel |
| title | Fuzzy Clustering Methods with Rényi Relative Entropy and Cluster Size |
| title_full | Fuzzy Clustering Methods with Rényi Relative Entropy and Cluster Size |
| title_fullStr | Fuzzy Clustering Methods with Rényi Relative Entropy and Cluster Size |
| title_full_unstemmed | Fuzzy Clustering Methods with Rényi Relative Entropy and Cluster Size |
| title_short | Fuzzy Clustering Methods with Rényi Relative Entropy and Cluster Size |
| title_sort | fuzzy clustering methods with renyi relative entropy and cluster size |
| topic | fuzzy clustering entropy relative entropy Rényi entropy differential evolution algorithm Gaussian kernel |
| url | https://www.mdpi.com/2227-7390/9/12/1423 |
| work_keys_str_mv | AT javierbonilla fuzzyclusteringmethodswithrenyirelativeentropyandclustersize AT danielvelez fuzzyclusteringmethodswithrenyirelativeentropyandclustersize AT javiermontero fuzzyclusteringmethodswithrenyirelativeentropyandclustersize AT jtinguarorodriguez fuzzyclusteringmethodswithrenyirelativeentropyandclustersize |