Sparse Representations Are Most Likely to Be the Sparsest Possible
<p/> <p>Given a signal <inline-formula><graphic file="1687-6180-2006-096247-i1.gif"/></inline-formula> and a full-rank matrix <inline-formula><graphic file="1687-6180-2006-096247-i2.gif"/></inline-formula> with <inline-formula>...
| Main Author: | |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
SpringerOpen
2006-01-01
|
| Series: | EURASIP Journal on Advances in Signal Processing |
| Online Access: | http://dx.doi.org/10.1155/ASP/2006/96247 |
| _version_ | 1811251489959575552 |
|---|---|
| author | Elad Michael |
| author_facet | Elad Michael |
| author_sort | Elad Michael |
| collection | DOAJ |
| description | <p/> <p>Given a signal <inline-formula><graphic file="1687-6180-2006-096247-i1.gif"/></inline-formula> and a full-rank matrix <inline-formula><graphic file="1687-6180-2006-096247-i2.gif"/></inline-formula> with <inline-formula><graphic file="1687-6180-2006-096247-i3.gif"/></inline-formula>, we define the signal's overcomplete representations as all <inline-formula><graphic file="1687-6180-2006-096247-i4.gif"/></inline-formula> satisfying <inline-formula><graphic file="1687-6180-2006-096247-i5.gif"/></inline-formula>. Among all the possible solutions, we have special interest in the sparsest one—the one minimizing <inline-formula><graphic file="1687-6180-2006-096247-i6.gif"/></inline-formula>. Previous work has established that a representation is unique if it is sparse enough, requiring <inline-formula><graphic file="1687-6180-2006-096247-i7.gif"/></inline-formula>. The measure <inline-formula><graphic file="1687-6180-2006-096247-i8.gif"/></inline-formula> stands for the minimal number of columns from <inline-formula><graphic file="1687-6180-2006-096247-i9.gif"/></inline-formula> that are linearly dependent. This bound is tight—examples can be constructed to show that with <inline-formula><graphic file="1687-6180-2006-096247-i10.gif"/></inline-formula> or more nonzero entries, uniqueness is violated. In this paper we study the behavior of overcomplete representations beyond the above bound. While tight from a worst-case standpoint, a probabilistic point of view leads to uniqueness of representations satisfying <inline-formula><graphic file="1687-6180-2006-096247-i11.gif"/></inline-formula>. Furthermore, we show that even beyond this point, uniqueness can still be claimed with high confidence. This new result is important for the study of the average performance of pursuit algorithms—when trying to show an equivalence between the pursuit result and the ideal solution, one must also guarantee that the ideal result is indeed the sparsest.</p> |
| first_indexed | 2024-04-12T16:20:12Z |
| format | Article |
| id | doaj.art-a49e5124c9d747b48d67b0690889a778 |
| institution | Directory Open Access Journal |
| issn | 1687-6172 1687-6180 |
| language | English |
| last_indexed | 2024-04-12T16:20:12Z |
| publishDate | 2006-01-01 |
| publisher | SpringerOpen |
| record_format | Article |
| series | EURASIP Journal on Advances in Signal Processing |
| spelling | doaj.art-a49e5124c9d747b48d67b0690889a7782022-12-22T03:25:35ZengSpringerOpenEURASIP Journal on Advances in Signal Processing1687-61721687-61802006-01-0120061096247Sparse Representations Are Most Likely to Be the Sparsest PossibleElad Michael<p/> <p>Given a signal <inline-formula><graphic file="1687-6180-2006-096247-i1.gif"/></inline-formula> and a full-rank matrix <inline-formula><graphic file="1687-6180-2006-096247-i2.gif"/></inline-formula> with <inline-formula><graphic file="1687-6180-2006-096247-i3.gif"/></inline-formula>, we define the signal's overcomplete representations as all <inline-formula><graphic file="1687-6180-2006-096247-i4.gif"/></inline-formula> satisfying <inline-formula><graphic file="1687-6180-2006-096247-i5.gif"/></inline-formula>. Among all the possible solutions, we have special interest in the sparsest one—the one minimizing <inline-formula><graphic file="1687-6180-2006-096247-i6.gif"/></inline-formula>. Previous work has established that a representation is unique if it is sparse enough, requiring <inline-formula><graphic file="1687-6180-2006-096247-i7.gif"/></inline-formula>. The measure <inline-formula><graphic file="1687-6180-2006-096247-i8.gif"/></inline-formula> stands for the minimal number of columns from <inline-formula><graphic file="1687-6180-2006-096247-i9.gif"/></inline-formula> that are linearly dependent. This bound is tight—examples can be constructed to show that with <inline-formula><graphic file="1687-6180-2006-096247-i10.gif"/></inline-formula> or more nonzero entries, uniqueness is violated. In this paper we study the behavior of overcomplete representations beyond the above bound. While tight from a worst-case standpoint, a probabilistic point of view leads to uniqueness of representations satisfying <inline-formula><graphic file="1687-6180-2006-096247-i11.gif"/></inline-formula>. Furthermore, we show that even beyond this point, uniqueness can still be claimed with high confidence. This new result is important for the study of the average performance of pursuit algorithms—when trying to show an equivalence between the pursuit result and the ideal solution, one must also guarantee that the ideal result is indeed the sparsest.</p>http://dx.doi.org/10.1155/ASP/2006/96247 |
| spellingShingle | Elad Michael Sparse Representations Are Most Likely to Be the Sparsest Possible EURASIP Journal on Advances in Signal Processing |
| title | Sparse Representations Are Most Likely to Be the Sparsest Possible |
| title_full | Sparse Representations Are Most Likely to Be the Sparsest Possible |
| title_fullStr | Sparse Representations Are Most Likely to Be the Sparsest Possible |
| title_full_unstemmed | Sparse Representations Are Most Likely to Be the Sparsest Possible |
| title_short | Sparse Representations Are Most Likely to Be the Sparsest Possible |
| title_sort | sparse representations are most likely to be the sparsest possible |
| url | http://dx.doi.org/10.1155/ASP/2006/96247 |
| work_keys_str_mv | AT eladmichael sparserepresentationsaremostlikelytobethesparsestpossible |