Study on Algorithms for Image Super-resolution based on Filtering and Learning Methods

Study on Algorithms for Image Super-resolution based on Filtering and Learning Methods

• By Muhammad Haris;
• Tesis
• 322

This thesis focuses on developing theory and algorithms for the single-image

super-resolution problem based on filtering and learning methods. Our pro-

posed methods are divided into three categories.

First part, First-order Derivatives- based Super-resolution is filtering based

method. A single fast super-resolution method based on first-order derivatives

from neighbor pixels is proposed. The basic idea of the proposed method is

to exploit a first-order derivatives component of six edge directions around a

missing pixel; followed by back projection to reduce noise estimated by the

difference between simulated and observed images. Using first-order deriva-

tives as a feature, the proposed method is expected to have low computational

complexity, and it can theoretically reduce blur, blocking, and ringing artifacts

in edge areas compared to previous methods. Experiments were conducted us-

ing 900 natural grayscale images from the USC-SIPI Database. We evaluated

the proposed and previous methods using peak signal-to-noise ratio, structural

similarity, feature similarity, and computation time. Experimental results indi-

cate that the proposed method clearly outperforms other state-of-the-art algo-

rithms such as fast curvature based interpolation.

Second part, Super-Resolution via Adaptive Multiple Sparse Representation

is learning based method. We propose a super-resolution algorithm based on

adaptive sparse representation via multiple dictionaries for images taken by

Unmanned Aerial Vehicles (UAVs). The super-resolution attainable through

the proposed algorithm can increase the precision of 3D reconstruction from

UAV images, enabling the production of high-resolution images for construct-

ing high-frequency time series and for high-precision digital mapping in agri-

culture. The basic idea of the proposed method is to use a field server or

ground-based camera to take training images and then construct multiple pairs

of dictionaries based on selective sparse representations to reduce instability

during the sparse coding process. The dictionaries are classified on the basis

of the edge orientation into five clusters: 0, 45, 90, 135, and non-direction.

The proposed method is expected to reduce blurring, blocking, and ringing

artifacts especially in edge areas. We evaluated the proposed and previous

methods using peak signal-to-noise ratio, structural similarity, feature similar-

ity, and computation time. Our experimental results indicate that the proposed

method clearly outperforms other state-of-the-art algorithms based on qualita-

tive and quantitative analysis. In the end, we demonstrate the effectiveness of

our proposed method to increase the precision of 3D reconstruction from UAV

images.

Last part, Deep Residual Learning Super-resolution is learning based method.

The light and efficient residual network for super-resolution is proposed. We

adopt inception module from GoogLeNet to exploit the features from the low-

resolution images and residual learning to have fast training steps. The pro-

posed network called Deep Residual Learning Super-resolution (DRLSR). The

network is proven to have fast convergence and low computational time. It is

divided into three parts: feature extraction, mapping, and reconstruction. In

the feature extraction, we apply inception module followed by dimensional re-

duction. Then, we map the features using simple convolutional layer. Finally,

we reconstruct the HR component using inception module and 1⇥1 convolu-

tional layer. The experimental results show our proposed method can reduce

more than half of computational time from the-state-of-the-art methods, while

still having clean and sharp images.

• Katakunci
• URL http://
• Penerbit ,

REFERENSI

[1] Michal Aharon, Michael Elad, and Alfred Bruckstein. K-svd: An algorithm for de-

signing overcomplete dictionaries for sparse representation. Signal Processing, IEEE

Transactions on, 54(11):4311–4322, 2006.

[2] Nicola Asuni and Andrea Giachetti. Accuracy improvements and artifacts removal in

edge based image interpolation. VISAPP (1)’08, pages 58–65, 2008.

[3] John Canny. A computational approach to edge detection. Pattern Analysis and

Machine Intelligence, IEEE Transactions on, (6):679–698, 1986.

[4] Turgay Celik and Tardi Tjahjadi. Image resolution enhancement using dual-tree com-

plex wavelet transform. Geoscience and Remote Sensing Letters, IEEE, 7(3):554–

557, 2010.

[5] Dengxin Dai, Yujian Wang, Yuhua Chen, and Luc Van Gool. Is image super-

resolution helpful for other vision tasks? In 2016 IEEE Winter Conference on Appli-

cations of Computer Vision (WACV), pages 1–9. IEEE, 2016.

[6] Aram Danielyan, Alessandro Foi, Vladimir Katkovnik, and Karen Egiazarian. Im-

age upsampling via spatially adaptive block-matching filtering. In Signal Processing

Conference, 2008 16th European, pages 1–5. IEEE, 2008.

[7] Hasan Demirel and Gholamreza Anbarjafari. Discrete wavelet transform-based satel-

lite image resolution enhancement. Geoscience and Remote Sensing, IEEE Transac-

tions on, 49(6):1997–2004, 2011.

[8] Chao Dong, Chen Change Loy, Kaiming He, and Xiaoou Tang. Image super-

resolution using deep convolutional networks. IEEE transactions on pattern analysis

and machine intelligence, 38(2):295–307, 2016.

[9] Chao Dong, Chen Change Loy, and Xiaoou Tang. Accelerating the super-resolution

convolutional neural network. In European Conference on Computer Vision, pages

391–407. Springer, 2016.

68

[10] Michael Elad. Sparse and redundant representation modeling - what next? Signal

Processing Letters, IEEE, 19(12):922–928, 2012.

[11] J Everaerts et al. The use of unmanned aerial vehicles (uavs) for remote sensing and

mapping. The International Archives of the Photogrammetry, Remote Sensing and

Spatial Information Sciences, 37:1187–1192, 2008.

[12] Tokihiro Fukatsu and Masayuki Hirafuji. Field monitoring using sensor-nodes with a

web server. Journal of Robotics and Mechatronics, 17(2):164–172, 2005.

[13] Yasutaka Furukawa and Jean Ponce. Accurate, dense, and robust multiview stereop-

sis. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 32(8):1362–

1376, 2010.

[14] A. Giachetti and N. Asuni. Real time artifact-free image upscaling. Image Processing,

IEEE Transactions on, 20(10):2760–2768, October 2011.

[15] GJ Grenzdorffer, A Engel, and B Teichert. The photogrammetric potential of low-cost  ̈

uavs in forestry and agriculture. The International Archives of the Photogrammetry,

Remote Sensing and Spatial Information Sciences, 31(B3):1207–1214, 2008.

[16] Wei Guo, Tokihiro Fukatsu, and Seishi Ninomiya. Automated characterization of

flowering dynamics in rice using field-acquired time-series rgb images. Plant Meth-

ods, 11(1):1–15, 2015.

[17] Muhammad Haris, Kazuhito Sawase, Muhammad Rahmat Widyanto, and Hajime

Nobuhara. An efficient super resolution based on image dimensionality reduction

using accumulative intensity gradient. Journal of Advanced Computational Intelli-

gence and Intelligent Informatics, 18(4):518–528, 2014.

[18] Muhammad Haris, M Rahmat Widyanto, and Hajime Nobuhara. First-order

derivative-based super-resolution. Signal, Image and Video Processing, 11(1):1–8,

2017.

[19] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning

for image recognition. arXiv preprint arXiv:1512.03385, 2015.

[20] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into recti-

fiers: Surpassing human-level performance on imagenet classification. In Proceedings

of the IEEE International Conference on Computer Vision, pages 1026–1034, 2015.

69

[21] Michal Irani and Shmuel Peleg. Motion analysis for image enhancement: Resolution,

occlusion, and transparency. Journal of Visual Communication and Image Represen-

tation, 4(4):324–335, 1993.

[22] Jiwon Kim, Jung Kwon Lee, and Kyoung Mu Lee. Accurate image super-resolution

using very deep convolutional networks. In The IEEE Conference on Computer Vision

and Pattern Recognition (CVPR Oral), June 2016.

[23] Kwang In Kim and Younghee Kwon. Single-image super-resolution using sparse

regression and natural image prior. Pattern Analysis and Machine Intelligence, IEEE

Transactions on, 32(6):1127–1133, 2010.

[24] Xin Li and Michael T. Orchard. New edge-directed interpolation. IEEE Transactions

on Image Processing, 10:1521–1527, 2001.

[25] Ce Liu and Deqing Sun. A bayesian approach to adaptive video super resolution. In

Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on, pages

209–216. IEEE, 2011.

[26] David G Lowe. Object recognition from local scale-invariant features. In Computer

vision, 1999. The proceedings of the seventh IEEE international conference on, vol-

ume 2, pages 1150–1157. Ieee, 1999.

[27] Stephane Mallat and Guoshen Yu. Super-resolution with sparse mixing estimators.  ́

Image Processing, IEEE Transactions on, 19(11):2889–2900, 2010.

[28] Eric Mjolsness. Fingerprint Hallucination. PhD thesis, California Institute of Tech-

nology, 1985.

[29] Kamal Nasrollahi and Thomas B Moeslund. Super-resolution: a comprehensive sur-

vey. Machine vision and applications, 25(6):1423–1468, 2014.

[30] M.A. Nuno-Maganda and M.O. Arias-Estrada. Real-time fpga-based architecture for

bicubic interpolation: an application for digital image scaling. In Reconfigurable

Computing and FPGAs, 2005. ReConFig 2005. International Conference on, pages 8

pp.–1, Sept 2005.

[31] Sung Cheol Park, Min Kyu Park, and Kang Moon Gi. Super-resolution image re-

construction: A technical overview. IEEE Signal Processing Magazine, 20:21–36,

2003.

70

[32] Santhosh K Seelan, Soizik Laguette, Grant M Casady, and George A Seielstad. Re-

mote sensing applications for precision agriculture: A learning community approach.

Remote Sensing of Environment, 88(1):157–169, 2003.

[33] Milan Sonka, Vaclav Hlavac, and Roger Boyle. Image processing, analysis, and

machine vision. Cengage Learning, 2014.

[34] Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir

Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going

deeper with convolutions. In Proceedings of the IEEE Conference on Computer Vi-

sion and Pattern Recognition, pages 1–9, 2015.

[35] Richard Szeliski. Computer vision: algorithms and applications. Springer Science &

Business Media, 2010.

[36] Hiroyuki Takeda, Sina Farsiu, and Peyman Milanfar. Kernel regression for image

processing and reconstruction. Image Processing, IEEE Transactions on, 16(2):349–

366, 2007.

[37] Hiroyuki Takeda, Peyman Milanfar, Matan Protter, and Michael Elad. Super-

resolution without explicit subpixel motion estimation. Image Processing, IEEE

Transactions on, 18(9):1958–1975, 2009.

[38] Joel A Tropp and Anna C Gilbert. Signal recovery from random measurements via

orthogonal matching pursuit. IEEE Transactions on information theory, 53(12):4655–

4666, 2007.

[39] Qing Wang and Rabab Kreidieh Ward. A new orientation-adaptive interpolation

method. IEEE Transactions on Image Processing, 16(4):889–900, 2007.

[40] Zhaowen Wang, Ding Liu, Jianchao Yang, Wei Han, and Thomas Huang. Deeply

improved sparse coding for image super-resolution. arXiv preprint arXiv:1507.08905,

2015.

[41] Zhou Wang, Alan Conrad Bovik, Hamid Rahim Sheikh, and Eero P Simoncelli. Im-

age quality assessment: from error visibility to structural similarity. Image Process-

ing, IEEE Transactions on, 13(4):600–612, 2004.

[42] GA Watson. Computing helmert transformations. Journal of Computational and

Applied Mathematics, 197(2):387–394, 2006.

[43] Chih-Yuan Yang. Example-Based Single-Image Super-Resolution. PhD thesis, UNI-

VERSITY OF CALIFORNIA, MERCED, 2015.

[44] Jianchao Yang, John Wright, Thomas S Huang, and Yi Ma. Image super-resolution

via sparse representation. Image Processing, IEEE Transactions on, 19(11):2861–

2873, 2010.

[45] Roman Zeyde, Michael Elad, and Matan Protter. On single image scale-up using

sparse-representations. In Curves and Surfaces, pages 711–730. Springer, 2012.

[46] Chunhua Zhang and John M Kovacs. The application of small unmanned aerial sys-

tems for precision agriculture: a review. Precision agriculture, 13(6):693–712, 2012.

[47] Lei Zhang and Xiaolin Wu. An edge-guided image interpolation algorithm via direc-

tional filtering and data fusion. Image Processing, IEEE Transactions on, 15(8):2226–

2238, 2006.

[48] Lin Zhang, Lei Zhang, Xuanqin Mou, and David Zhang. Fsim: a feature similar-

ity index for image quality assessment. Image Processing, IEEE Transactions on,

20(8):2378–2386, 2011.