CAMixerSR: Only Details Need More "Attention"
To satisfy the rapidly increasing demands on the large image (2K-8K) super-resolution (SR), prevailing methods follow two independent tracks: 1) accelerate existing networks by content-aware routing, and 2) design better super-resolution networks via token mixer refining. Despite directness, they encounter unavoidable defects (e.g., inflexible route or non-discriminative processing) limiting further improvements of quality-complexity trade-off. To erase the drawbacks, we integrate these schemes by proposing a content-aware mixer (CAMixer), which assigns convolution for simple contexts and additional deformable window-attention for sparse textures. Specifically, the CAMixer uses a learnable predictor to generate multiple bootstraps, including offsets for windows warping, a mask for classifying windows, and convolutional attentions for endowing convolution with the dynamic property, which modulates attention to include more useful textures self-adaptively and improves the representation capability of convolution. We further introduce a global classification loss to improve the accuracy of predictors. By simply stacking CAMixers, we obtain CAMixerSR which achieves superior performance on large-image SR, lightweight SR, and omnidirectional-image SR..
| Medienart: |
|
|---|
Preprint| Erscheinungsjahr: |
2024 |
|---|---|
| Erschienen: |
2024 |
| Enthalten in: |
arXiv.org - (2024) vom: 29. Feb. Zur Gesamtaufnahme - year:2024 |
|---|
| Sprache: |
Englisch |
|---|
| Beteiligte Personen: |
Wang, Yan [VerfasserIn] |
|---|
| Links: |
Volltext [kostenfrei] |
|---|
| Themen: |
000 |
|---|
| Förderinstitution / Projekttitel: |
|
|---|
| PPN (Katalog-ID): |
XCH04275237X |
|---|
| LEADER | 01000naa a22002652 4500 | ||
|---|---|---|---|
| 001 | XCH04275237X | ||
| 003 | DE-627 | ||
| 005 | 20240306114436.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 240306s2024 xx |||||o 00| ||eng c | ||
| 035 | |a (DE-627)XCH04275237X | ||
| 035 | |a (chemrXiv)2402.19289 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rakwb | ||
| 041 | |a eng | ||
| 100 | 1 | |a Wang, Yan |e verfasserin |4 aut | |
| 245 | 1 | 0 | |a CAMixerSR: Only Details Need More "Attention" |
| 264 | 1 | |c 2024 | |
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a Computermedien |b c |2 rdamedia | ||
| 338 | |a Online-Ressource |b cr |2 rdacarrier | ||
| 520 | |a To satisfy the rapidly increasing demands on the large image (2K-8K) super-resolution (SR), prevailing methods follow two independent tracks: 1) accelerate existing networks by content-aware routing, and 2) design better super-resolution networks via token mixer refining. Despite directness, they encounter unavoidable defects (e.g., inflexible route or non-discriminative processing) limiting further improvements of quality-complexity trade-off. To erase the drawbacks, we integrate these schemes by proposing a content-aware mixer (CAMixer), which assigns convolution for simple contexts and additional deformable window-attention for sparse textures. Specifically, the CAMixer uses a learnable predictor to generate multiple bootstraps, including offsets for windows warping, a mask for classifying windows, and convolutional attentions for endowing convolution with the dynamic property, which modulates attention to include more useful textures self-adaptively and improves the representation capability of convolution. We further introduce a global classification loss to improve the accuracy of predictors. By simply stacking CAMixers, we obtain CAMixerSR which achieves superior performance on large-image SR, lightweight SR, and omnidirectional-image SR. | ||
| 650 | 4 | |a Electrical Engineering and Systems Science - Image and Video Processing |7 (dpeaa)DE-84 | |
| 650 | 4 | |a Computer Science - Computer Vision and Pattern Recognition |7 (dpeaa)DE-84 | |
| 650 | 4 | |a 620 |7 (dpeaa)DE-84 | |
| 650 | 4 | |a 000 |7 (dpeaa)DE-84 | |
| 700 | 1 | |a Zhao, Shijie |4 aut | |
| 700 | 1 | |a Liu, Yi |4 aut | |
| 700 | 1 | |a Li, Junlin |4 aut | |
| 700 | 1 | |a Zhang, Li |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t arXiv.org |g (2024) vom: 29. Feb. |
| 773 | 1 | 8 | |g year:2024 |g day:29 |g month:02 |
| 856 | 4 | 0 | |u https://arxiv.org/abs/2402.19289 |z kostenfrei |3 Volltext |
| 912 | |a GBV_XCH | ||
| 951 | |a AR | ||
| 952 | |j 2024 |b 29 |c 02 | ||