This thesis focuses on developing theory and algorithms for the single-image
super-resolution problem based on filtering and learning methods. Our pro-
posed methods are divided into three categories.
First part, First-order Derivatives- based Super-resolution is filtering based
method. A single fast super-resolution method based on first-order derivatives
from neighbor pixels is proposed. The basic idea of the proposed method is
to exploit a first-order derivatives component of six edge directions around a
missing pixel; followed by back projection to reduce noise estimated by the
difference between simulated and observed images. Using first-order deriva-
tives as a feature, the proposed method is expected to have low computational
complexity, and it can theoretically reduce blur, blocking, and ringing artifacts
in edge areas compared to previous methods. Experiments were conducted us-
ing 900 natural grayscale images from the USC-SIPI Database. We evaluated
the proposed and previous methods using peak signal-to-noise ratio, structural
similarity, feature similarity, and computation time. Experimental results indi-
cate that the proposed method clearly outperforms other state-of-the-art algo-
rithms such as fast curvature based interpolation.
Second part, Super-Resolution via Adaptive Multiple Sparse Representation
is learning based method. We propose a super-resolution algorithm based on
adaptive sparse representation via multiple dictionaries for images taken by
Unmanned Aerial Vehicles (UAVs). The super-resolution attainable through
the proposed algorithm can increase the precision of 3D reconstruction from
UAV images, enabling the production of high-resolution images for construct-
ing high-frequency time series and for high-precision digital mapping in agri-
culture. The basic idea of the proposed method is to use a field server or
ground-based camera to take training images and then construct multiple pairs
of dictionaries based on selective sparse representations to reduce instability
during the sparse coding process. The dictionaries are classified on the basis
of the edge orientation into five clusters: 0, 45, 90, 135, and non-direction.
The proposed method is expected to reduce blurring, blocking, and ringing
artifacts especially in edge areas. We evaluated the proposed and previous
methods using peak signal-to-noise ratio, structural similarity, feature similar-
ity, and computation time. Our experimental results indicate that the proposed
method clearly outperforms other state-of-the-art algorithms based on qualita-
tive and quantitative analysis. In the end, we demonstrate the effectiveness of
our proposed method to increase the precision of 3D reconstruction from UAV
images.
Last part, Deep Residual Learning Super-resolution is learning based method.
The light and efficient residual network for super-resolution is proposed. We
adopt inception module from GoogLeNet to exploit the features from the low-
resolution images and residual learning to have fast training steps. The pro-
posed network called Deep Residual Learning Super-resolution (DRLSR). The
network is proven to have fast convergence and low computational time. It is
divided into three parts: feature extraction, mapping, and reconstruction. In
the feature extraction, we apply inception module followed by dimensional re-
duction. Then, we map the features using simple convolutional layer. Finally,
we reconstruct the HR component using inception module and 1⇥1 convolu-
tional layer. The experimental results show our proposed method can reduce
more than half of computational time from the-state-of-the-art methods, while
still having clean and sharp images.
Disertasi Lengkap
Ringkasan Disertasi
[1] Michal Aharon, Michael Elad, and Alfred Bruckstein. K-svd: An algorithm for de-
signing overcomplete dictionaries for sparse representation. Signal Processing, IEEE
Transactions on, 54(11):4311–4322, 2006.
[2] Nicola Asuni and Andrea Giachetti. Accuracy improvements and artifacts removal in
edge based image interpolation. VISAPP (1)’08, pages 58–65, 2008.
[3] John Canny. A computational approach to edge detection. Pattern Analysis and
Machine Intelligence, IEEE Transactions on, (6):679–698, 1986.
[4] Turgay Celik and Tardi Tjahjadi. Image resolution enhancement using dual-tree com-
plex wavelet transform. Geoscience and Remote Sensing Letters, IEEE, 7(3):554–
557, 2010.
[5] Dengxin Dai, Yujian Wang, Yuhua Chen, and Luc Van Gool. Is image super-
resolution helpful for other vision tasks? In 2016 IEEE Winter Conference on Appli-
cations of Computer Vision (WACV), pages 1–9. IEEE, 2016.
[6] Aram Danielyan, Alessandro Foi, Vladimir Katkovnik, and Karen Egiazarian. Im-
age upsampling via spatially adaptive block-matching filtering. In Signal Processing
Conference, 2008 16th European, pages 1–5. IEEE, 2008.
[7] Hasan Demirel and Gholamreza Anbarjafari. Discrete wavelet transform-based satel-
lite image resolution enhancement. Geoscience and Remote Sensing, IEEE Transac-
tions on, 49(6):1997–2004, 2011.
[8] Chao Dong, Chen Change Loy, Kaiming He, and Xiaoou Tang. Image super-
resolution using deep convolutional networks. IEEE transactions on pattern analysis
and machine intelligence, 38(2):295–307, 2016.
[9] Chao Dong, Chen Change Loy, and Xiaoou Tang. Accelerating the super-resolution
convolutional neural network. In European Conference on Computer Vision, pages
391–407. Springer, 2016.
68
[10] Michael Elad. Sparse and redundant representation modeling - what next? Signal
Processing Letters, IEEE, 19(12):922–928, 2012.
[11] J Everaerts et al. The use of unmanned aerial vehicles (uavs) for remote sensing and
mapping. The International Archives of the Photogrammetry, Remote Sensing and
Spatial Information Sciences, 37:1187–1192, 2008.
[12] Tokihiro Fukatsu and Masayuki Hirafuji. Field monitoring using sensor-nodes with a
web server. Journal of Robotics and Mechatronics, 17(2):164–172, 2005.
[13] Yasutaka Furukawa and Jean Ponce. Accurate, dense, and robust multiview stereop-
sis. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 32(8):1362–
1376, 2010.
[14] A. Giachetti and N. Asuni. Real time artifact-free image upscaling. Image Processing,
IEEE Transactions on, 20(10):2760–2768, October 2011.
[15] GJ Grenzdorffer, A Engel, and B Teichert. The photogrammetric potential of low-cost ̈
uavs in forestry and agriculture. The International Archives of the Photogrammetry,
Remote Sensing and Spatial Information Sciences, 31(B3):1207–1214, 2008.
[16] Wei Guo, Tokihiro Fukatsu, and Seishi Ninomiya. Automated characterization of
flowering dynamics in rice using field-acquired time-series rgb images. Plant Meth-
ods, 11(1):1–15, 2015.
[17] Muhammad Haris, Kazuhito Sawase, Muhammad Rahmat Widyanto, and Hajime
Nobuhara. An efficient super resolution based on image dimensionality reduction
using accumulative intensity gradient. Journal of Advanced Computational Intelli-
gence and Intelligent Informatics, 18(4):518–528, 2014.
[18] Muhammad Haris, M Rahmat Widyanto, and Hajime Nobuhara. First-order
derivative-based super-resolution. Signal, Image and Video Processing, 11(1):1–8,
2017.
[19] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning
for image recognition. arXiv preprint arXiv:1512.03385, 2015.
[20] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into recti-
fiers: Surpassing human-level performance on imagenet classification. In Proceedings
of the IEEE International Conference on Computer Vision, pages 1026–1034, 2015.
69
[21] Michal Irani and Shmuel Peleg. Motion analysis for image enhancement: Resolution,
occlusion, and transparency. Journal of Visual Communication and Image Represen-
tation, 4(4):324–335, 1993.
[22] Jiwon Kim, Jung Kwon Lee, and Kyoung Mu Lee. Accurate image super-resolution
using very deep convolutional networks. In The IEEE Conference on Computer Vision
and Pattern Recognition (CVPR Oral), June 2016.
[23] Kwang In Kim and Younghee Kwon. Single-image super-resolution using sparse
regression and natural image prior. Pattern Analysis and Machine Intelligence, IEEE
Transactions on, 32(6):1127–1133, 2010.
[24] Xin Li and Michael T. Orchard. New edge-directed interpolation. IEEE Transactions
on Image Processing, 10:1521–1527, 2001.
[25] Ce Liu and Deqing Sun. A bayesian approach to adaptive video super resolution. In
Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on, pages
209–216. IEEE, 2011.
[26] David G Lowe. Object recognition from local scale-invariant features. In Computer
vision, 1999. The proceedings of the seventh IEEE international conference on, vol-
ume 2, pages 1150–1157. Ieee, 1999.
[27] Stephane Mallat and Guoshen Yu. Super-resolution with sparse mixing estimators. ́
Image Processing, IEEE Transactions on, 19(11):2889–2900, 2010.
[28] Eric Mjolsness. Fingerprint Hallucination. PhD thesis, California Institute of Tech-
nology, 1985.
[29] Kamal Nasrollahi and Thomas B Moeslund. Super-resolution: a comprehensive sur-
vey. Machine vision and applications, 25(6):1423–1468, 2014.
[30] M.A. Nuno-Maganda and M.O. Arias-Estrada. Real-time fpga-based architecture for
bicubic interpolation: an application for digital image scaling. In Reconfigurable
Computing and FPGAs, 2005. ReConFig 2005. International Conference on, pages 8
pp.–1, Sept 2005.
[31] Sung Cheol Park, Min Kyu Park, and Kang Moon Gi. Super-resolution image re-
construction: A technical overview. IEEE Signal Processing Magazine, 20:21–36,
2003.
70
[32] Santhosh K Seelan, Soizik Laguette, Grant M Casady, and George A Seielstad. Re-
mote sensing applications for precision agriculture: A learning community approach.
Remote Sensing of Environment, 88(1):157–169, 2003.
[33] Milan Sonka, Vaclav Hlavac, and Roger Boyle. Image processing, analysis, and
machine vision. Cengage Learning, 2014.
[34] Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir
Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going
deeper with convolutions. In Proceedings of the IEEE Conference on Computer Vi-
sion and Pattern Recognition, pages 1–9, 2015.
[35] Richard Szeliski. Computer vision: algorithms and applications. Springer Science &
Business Media, 2010.
[36] Hiroyuki Takeda, Sina Farsiu, and Peyman Milanfar. Kernel regression for image
processing and reconstruction. Image Processing, IEEE Transactions on, 16(2):349–
366, 2007.
[37] Hiroyuki Takeda, Peyman Milanfar, Matan Protter, and Michael Elad. Super-
resolution without explicit subpixel motion estimation. Image Processing, IEEE
Transactions on, 18(9):1958–1975, 2009.
[38] Joel A Tropp and Anna C Gilbert. Signal recovery from random measurements via
orthogonal matching pursuit. IEEE Transactions on information theory, 53(12):4655–
4666, 2007.
[39] Qing Wang and Rabab Kreidieh Ward. A new orientation-adaptive interpolation
method. IEEE Transactions on Image Processing, 16(4):889–900, 2007.
[40] Zhaowen Wang, Ding Liu, Jianchao Yang, Wei Han, and Thomas Huang. Deeply
improved sparse coding for image super-resolution. arXiv preprint arXiv:1507.08905,
2015.
[41] Zhou Wang, Alan Conrad Bovik, Hamid Rahim Sheikh, and Eero P Simoncelli. Im-
age quality assessment: from error visibility to structural similarity. Image Process-
ing, IEEE Transactions on, 13(4):600–612, 2004.
[42] GA Watson. Computing helmert transformations. Journal of Computational and
Applied Mathematics, 197(2):387–394, 2006.
[43] Chih-Yuan Yang. Example-Based Single-Image Super-Resolution. PhD thesis, UNI-
VERSITY OF CALIFORNIA, MERCED, 2015.
[44] Jianchao Yang, John Wright, Thomas S Huang, and Yi Ma. Image super-resolution
via sparse representation. Image Processing, IEEE Transactions on, 19(11):2861–
2873, 2010.
[45] Roman Zeyde, Michael Elad, and Matan Protter. On single image scale-up using
sparse-representations. In Curves and Surfaces, pages 711–730. Springer, 2012.
[46] Chunhua Zhang and John M Kovacs. The application of small unmanned aerial sys-
tems for precision agriculture: a review. Precision agriculture, 13(6):693–712, 2012.
[47] Lei Zhang and Xiaolin Wu. An edge-guided image interpolation algorithm via direc-
tional filtering and data fusion. Image Processing, IEEE Transactions on, 15(8):2226–
2238, 2006.
[48] Lin Zhang, Lei Zhang, Xuanqin Mou, and David Zhang. Fsim: a feature similar-
ity index for image quality assessment. Image Processing, IEEE Transactions on,
20(8):2378–2386, 2011.