You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
81 lines
4.4 KiB
81 lines
4.4 KiB
\section{Background}
|
|
\label{sec:bk_related}
|
|
|
|
This section describes background of Super-resolution technique, and the sensors
|
|
we are using in this work.
|
|
|
|
%\subsection{Background}
|
|
\subsection{Super-Resolution Convolutional Neural Network}
|
|
|
|
C. Dong et al.~\cite{ChaoDong16} proposed a deep learning method for single image
|
|
super-resolution (SR). Their method learns how to directly map the low-resolution
|
|
image to high-resolution image. They show that the traditional
|
|
SR methods can be reformulated into a deep convolutional neural network.
|
|
SRCNN consists three operations.
|
|
The first layer of SRCNN model extracts the patches from low-resolution image.
|
|
The second layer maps the patches from low-resolution to high-resolution. The
|
|
third layer will reconstruct the high-resolution image by the high-resolution
|
|
patches.
|
|
|
|
%% \begin{figure}[htb]
|
|
%% \begin{center}
|
|
%% \includegraphics[width=1\linewidth]{figures/SRCNN_model.pdf}
|
|
%% \caption{Illustration of super-resolution convolutional neural network (SRCNN) model~\cite{ChaoDong16}.}
|
|
%% \label{fig:SRCNN_model}
|
|
%% \end{center}
|
|
%% \end{figure}
|
|
|
|
\subsection{Thermal cameras}
|
|
In this work, we use two different resolution thermal cameras to play the role
|
|
of low-resolution and high-resolution cameras. We use
|
|
Grid-EYE thermal camera.and Lepton 3 as our two different inputs. The
|
|
specification of them are shown in Table~\ref{table:specificationofdevices}.
|
|
|
|
|
|
\begin{table}[tbp]
|
|
\centering
|
|
\footnotesize
|
|
\begin{tabular}{|p{3cm}<{\centering}|p{2cm}<{\centering}|p{2.5cm}<{\centering}|}
|
|
\hline
|
|
Specification & Grid-EYE & Lepton 3\\
|
|
\hline
|
|
Resolution & 64 pixels (8x8) & 120x160\\
|
|
\hline
|
|
FOV-horizontal & $60^{\circ}$ & $49.5^{\circ}$\\
|
|
\hline
|
|
FOV-vertical & $60^{\circ}$ & $61.8^{\circ}$\\
|
|
\hline
|
|
Frame rate & 10Hz & 8.7Hz\\
|
|
\hline
|
|
Detect temperature range & $-20^{\circ}$C to $80^{\circ}$C & $0^{\circ}$C to $120^{\circ}$C\\
|
|
\hline
|
|
Bits per pixel & 12 & 14\\
|
|
\hline
|
|
\end{tabular}
|
|
\caption{Specification of Grid-EYE and Lepton 3.}
|
|
%\caption{Specification of Grid-EYE~\cite{grideye_datasheet} and Lepton 3~\cite{lepton_datasheet}.}
|
|
\label{table:specificationofdevices}
|
|
\end{table}
|
|
|
|
%\subsection{Related Works}
|
|
%X. Chen et al.~\cite{7805509} proposed to use the visible camera as a guidance to super resolution the IR images. They proposed the IR-RBG multi-sensor imaging system. The approach bases on the fact that RGB channels have different correlation with IR images. To avoid wrong texture transfer, the method used cross correlation to find the proper matching between the region of IR image and RGB image. Next, the method applied sub-pixel estimation and guided filtering to remove the noise and discontinuities in IR image. At the last step of the iteration, they used truncated quadric model as the cost function to terminate the algorithm. After the iteration, the method fine tuned the output IR image with outlier detection to remove the black points which only appear at the edge of IR image.
|
|
% \begin{figure}[ht]
|
|
% \begin{center}
|
|
% \includegraphics[width=0.7\linewidth]{figures/color_guided_SR.pdf}
|
|
% \caption{Flow chart of IR-Color multi-sensor imaging system.}
|
|
% \label{fig:color_guided_SR}
|
|
% \end{center}
|
|
% \end{figure}
|
|
|
|
%F. Almsari et al.~\cite{8713356} applied Generative Adversarial Network (GAN) to the super-resoltuion problem called {\it TIGAN}. TIGAN consists of two networks, generator and discriminator, which learn from each other with zero-sum games and try to find an equilibrium state. The generator was trained to transfer low-resolution thermal image into super-resolution thermal image domain. The generated image should be similar to its ground truth high-resolution image domain. While the discriminator was trained to discriminate between generated image and ground truth high-resolution image. Compared to SRCNN model, it preserves the high-frequency details and gave shaper textures.
|
|
|
|
% \begin{figure}[hb]
|
|
% \begin{center}
|
|
% \includegraphics[width=0.7\linewidth]{figures/GAN_SR.pdf}
|
|
% \caption{Network architecture of TIGAN.}
|
|
% \label{fig:GAN_SR}
|
|
% \end{center}
|
|
% \end{figure}
|
|
% % background
|
|
|
|
%The assumption on the resolution of thermal images in above works is at least $120 \times 160$. Compared to the $8 \times 8$ thermal images collected by Grid-EYE sensors, more thermal features are available. In addition, these works use RGB images as reference to train the model. In this work, RGB camera will not be used so as to protect privacy.
|