Capacity-Driven Autoencoders for Communications
The autoencoder concept has fostered the reinterpretation and the design of modern communication systems. It consists of an encoder, a channel and a decoder block that modify their internal neural structure in an end-to-end learning fashion. However, the current approach to train an autoencoder reli...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2021-01-01
|
| Series: | IEEE Open Journal of the Communications Society |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9449919/ |
| _version_ | 1818612968731967488 |
|---|---|
| author | Nunzio A. Letizia Andrea M. Tonello |
| author_facet | Nunzio A. Letizia Andrea M. Tonello |
| author_sort | Nunzio A. Letizia |
| collection | DOAJ |
| description | The autoencoder concept has fostered the reinterpretation and the design of modern communication systems. It consists of an encoder, a channel and a decoder block that modify their internal neural structure in an end-to-end learning fashion. However, the current approach to train an autoencoder relies on the use of the cross-entropy loss function. This approach can be prone to overfitting issues and often fails to learn an optimal system and signal representation (code). In addition, less is known about the autoencoder ability to design channel capacity-approaching codes, i.e., codes that maximize the input-output mutual information under a certain power constraint. The task being even more formidable for an unknown channel for which the capacity is unknown and therefore it has to be learnt. In this paper, we address the challenge of designing capacity-approaching codes by incorporating the presence of the communication channel into a novel loss function for the autoencoder training. In particular, we exploit the mutual information between the transmitted and received signals as a regularization term in the cross-entropy loss function, with the aim of controlling the amount of information stored. By jointly maximizing the mutual information and minimizing the cross-entropy, we propose a theoretical approach that a) computes an estimate of the channel capacity and b) constructs an optimal coded signal approaching it. Theoretical considerations are made on the choice of the cost function and the ability of the proposed architecture to mitigate the overfitting problem. Simulation results offer an initial evidence of the potentiality of the proposed method. |
| first_indexed | 2024-12-16T15:54:39Z |
| format | Article |
| id | doaj.art-78c0efb28f5e4053a4473600a35ce72c |
| institution | Directory Open Access Journal |
| issn | 2644-125X |
| language | English |
| last_indexed | 2024-12-16T15:54:39Z |
| publishDate | 2021-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Open Journal of the Communications Society |
| spelling | doaj.art-78c0efb28f5e4053a4473600a35ce72c2022-12-21T22:25:37ZengIEEEIEEE Open Journal of the Communications Society2644-125X2021-01-0121366137810.1109/OJCOMS.2021.30878159449919Capacity-Driven Autoencoders for CommunicationsNunzio A. Letizia0https://orcid.org/0000-0003-1495-4449Andrea M. Tonello1https://orcid.org/0000-0002-9873-2407Institute of Networked and Embedded Systems, Chair of Embedded Communication Systems, University of Klagenfurt, Klagenfurt, AustriaInstitute of Networked and Embedded Systems, Chair of Embedded Communication Systems, University of Klagenfurt, Klagenfurt, AustriaThe autoencoder concept has fostered the reinterpretation and the design of modern communication systems. It consists of an encoder, a channel and a decoder block that modify their internal neural structure in an end-to-end learning fashion. However, the current approach to train an autoencoder relies on the use of the cross-entropy loss function. This approach can be prone to overfitting issues and often fails to learn an optimal system and signal representation (code). In addition, less is known about the autoencoder ability to design channel capacity-approaching codes, i.e., codes that maximize the input-output mutual information under a certain power constraint. The task being even more formidable for an unknown channel for which the capacity is unknown and therefore it has to be learnt. In this paper, we address the challenge of designing capacity-approaching codes by incorporating the presence of the communication channel into a novel loss function for the autoencoder training. In particular, we exploit the mutual information between the transmitted and received signals as a regularization term in the cross-entropy loss function, with the aim of controlling the amount of information stored. By jointly maximizing the mutual information and minimizing the cross-entropy, we propose a theoretical approach that a) computes an estimate of the channel capacity and b) constructs an optimal coded signal approaching it. Theoretical considerations are made on the choice of the cost function and the ability of the proposed architecture to mitigate the overfitting problem. Simulation results offer an initial evidence of the potentiality of the proposed method.https://ieeexplore.ieee.org/document/9449919/Digital communicationsphysical layerstatistical learningautoencoderscoding theorymutual information |
| spellingShingle | Nunzio A. Letizia Andrea M. Tonello Capacity-Driven Autoencoders for Communications IEEE Open Journal of the Communications Society Digital communications physical layer statistical learning autoencoders coding theory mutual information |
| title | Capacity-Driven Autoencoders for Communications |
| title_full | Capacity-Driven Autoencoders for Communications |
| title_fullStr | Capacity-Driven Autoencoders for Communications |
| title_full_unstemmed | Capacity-Driven Autoencoders for Communications |
| title_short | Capacity-Driven Autoencoders for Communications |
| title_sort | capacity driven autoencoders for communications |
| topic | Digital communications physical layer statistical learning autoencoders coding theory mutual information |
| url | https://ieeexplore.ieee.org/document/9449919/ |
| work_keys_str_mv | AT nunzioaletizia capacitydrivenautoencodersforcommunications AT andreamtonello capacitydrivenautoencodersforcommunications |