Deep CNN sparse coding analysis: Towards average case
Deep convolutional sparse coding (D-CSC) is a framework reminiscent of deep convolutional neural nets (DCNN), but by omitting the learning of the dictionaries one can more transparently analyse the role of the activation function and its ability to recover activation paths through the layers. Papyan...
| Main Authors: | , |
|---|---|
| Format: | Conference item |
| Published: |
Institute of Electrical and Electronics
2018
|
| _version_ | 1826275214449704960 |
|---|---|
| author | Murray, M Tanner, J |
| author_facet | Murray, M Tanner, J |
| author_sort | Murray, M |
| collection | OXFORD |
| description | Deep convolutional sparse coding (D-CSC) is a framework reminiscent of deep convolutional neural nets (DCNN), but by omitting the learning of the dictionaries one can more transparently analyse the role of the activation function and its ability to recover activation paths through the layers. Papyan, Romano, and Elad conducted an analysis of such an architecture [1], showed the relationship with DCNNs, and proved conditions under which a D-CSC is guaranteed to recover activation paths. A technical innovation of their work highlights that one can view the efficacy of the ReLU nonlinear activation function of a DCNN through the new variant of the tensor’s sparsity, referred to as stripe-sparsity, and by which they can prove that the density of activations can be proportional to the ambient dimension of the data. We extend their uniform guarantees to a slightly modified model and prove that with high probability the desired activation is typically possible to recover for a greater density of activations per layer. Our extension follows from incorporating the prior work on one step thresholding by Schnass and Vandergheynst into the appropriately modified architecture of [1]. |
| first_indexed | 2024-03-06T22:55:18Z |
| format | Conference item |
| id | oxford-uuid:6028ac4c-a330-43d1-961e-df1fe4527ef9 |
| institution | University of Oxford |
| last_indexed | 2024-03-06T22:55:18Z |
| publishDate | 2018 |
| publisher | Institute of Electrical and Electronics |
| record_format | dspace |
| spelling | oxford-uuid:6028ac4c-a330-43d1-961e-df1fe4527ef92022-03-26T17:51:40ZDeep CNN sparse coding analysis: Towards average caseConference itemhttp://purl.org/coar/resource_type/c_5794uuid:6028ac4c-a330-43d1-961e-df1fe4527ef9Symplectic Elements at OxfordInstitute of Electrical and Electronics2018Murray, MTanner, JDeep convolutional sparse coding (D-CSC) is a framework reminiscent of deep convolutional neural nets (DCNN), but by omitting the learning of the dictionaries one can more transparently analyse the role of the activation function and its ability to recover activation paths through the layers. Papyan, Romano, and Elad conducted an analysis of such an architecture [1], showed the relationship with DCNNs, and proved conditions under which a D-CSC is guaranteed to recover activation paths. A technical innovation of their work highlights that one can view the efficacy of the ReLU nonlinear activation function of a DCNN through the new variant of the tensor’s sparsity, referred to as stripe-sparsity, and by which they can prove that the density of activations can be proportional to the ambient dimension of the data. We extend their uniform guarantees to a slightly modified model and prove that with high probability the desired activation is typically possible to recover for a greater density of activations per layer. Our extension follows from incorporating the prior work on one step thresholding by Schnass and Vandergheynst into the appropriately modified architecture of [1]. |
| spellingShingle | Murray, M Tanner, J Deep CNN sparse coding analysis: Towards average case |
| title | Deep CNN sparse coding analysis: Towards average case |
| title_full | Deep CNN sparse coding analysis: Towards average case |
| title_fullStr | Deep CNN sparse coding analysis: Towards average case |
| title_full_unstemmed | Deep CNN sparse coding analysis: Towards average case |
| title_short | Deep CNN sparse coding analysis: Towards average case |
| title_sort | deep cnn sparse coding analysis towards average case |
| work_keys_str_mv | AT murraym deepcnnsparsecodinganalysistowardsaveragecase AT tannerj deepcnnsparsecodinganalysistowardsaveragecase |