Walking exercises the nervous, cardiovascular, pulmonary, musculoskeletal and hematologic systems because it requires more oxygen to contract the muscles. Hence, {\it gait velocity}, or called {\it walking speed}~\cite{Middleton2015}, has become a valid and important metric for senior populations~\cite{Middleton2015,studenski2011,Studenski03}.
In 2011, Studenski et al~\cite{studenski2011} published a study that tracked gait velocity of over 34,000 seniors from 6 to 21 years in US. The study found that predicted survival rate based on age, sex, and gait velocity was as accurate as predicted based on age, sex, chronic conditions, smoking history, blood pressure, body mass index, and hospitalization. Consequently, it has motivated the industrial and academia communities to develop the methodology to track and assess the risk based on gait velocity. The following years have led to many papers that point to the importance of gait velocity as a predictor of degradation and exacerbation events associated with various chronic diseases including heart failure, COPD, kidney failure, stroke, etc~\cite{Studenski03, pulignano2016, Konthoraxjnl2015, kutner2015}.
@ -17,7 +19,7 @@ Shih and his colleagues~\cite{Shih17b} proposed a sensing system to be installed
\label{fig:gaitVelocitySmartHome}
\end{figure}
In a IoT environment, many devices will periodically transmit data. Some sensor is use for avoid accidents, so they will have very high sensing frequency. However, most of the data are redundant. Like a temperature sensor on a gas stove, the temperature value is the same as the value from air conditioner and does not change very frequently, but it will have dramatically difference when we are cooking. We can simply make a threshold that when temperature is higher or lower than some degrees, the data will be transmitted, and drop the data that we don't interest. This is a very easy solution if we only have a few devices, but when we have hundreds or thousands devices, it is impossible to manually configure all devices, and the setting may need to change in the winter and summer, or different location.
Many sensors are needed for getting an accurate result. They will periodically send sensing data and let the host server make the final decision. Sensors have high sensing frequency to prevent unexpected events, and many redundant data were produced. For some normal sensors, e.g., temperature sensor on a gas stove, infared sensor for light control, $CO_2$ sensor for avoiding bad ventilation, we can make a threshold and only send the data that is higher or lower than the predefined threshold. This is a very easy solution if we only have a few devices, but when we have hundreds or thousands devices, it is impossible to manually configure all devices, and the setting may need to change in the winter and summer, or different location. Also, some slightly difference between useful data and redundant data will need more computing power or other information from nearby sensors to distinguish.
In this paper, we study the data from Panasonic Grid-EYE, a $8\times8$ pixels infrared array sensor, and FLIR ONE PRO, a $480\times640$ pixels thermal camera. Both are setting on ceiling and taking a video of a person walking under the camera.
First, we study the sensor Panasonic Grid-EYE which is a thermal camera that can output $8\times8$ pixels image with $2.5^\circ C$ accuracy and $0.25^\circ C$ resolution at $10$ frames per second. In normal mode, the current consumption is 4.5mA. It is a low resolution camera and infrared array sensor, so we install it in our house at ease without some privacy issue that may cause by a surveillance camera.
First, we study the sensor Panasonic Grid-EYE which is a thermal camera that can output $8\times8$ pixels images with $2.5^\circ C$ accuracy and $0.25^\circ C$ resolution at $10$ frames per second. In normal mode, the current consumption is 4.5mA. It is a low resolution camera and infrared array sensor, so we install it in our house at ease without some privacy issue that may cause by a surveillance camera.
When someone walks under a Grid-EYE sensor, we will see some pixels with higher temperature than others. Figure~\ref{fig:GridEye} shows an example of image from Grid-EYE sensor. The sensor value will look like a cone shape. The pixel with our head will have the highest temperature, body is lower, and leg is the lowest except background because when the distance from camera to our body is longer, the area cover by the camera will be wider and the ratio of background temperature in the pixel will increase, also our head does not cover by cloth, so the surface temperature will higher than other place. while we are walking in a area, the temperature of air in the area will become warmer, and the shape of human will be harder to recognize.
When someone walks under a Grid-EYE sensor, we will see some pixels with higher temperature than others. Figure~\ref{fig:GridEye} shows an example of image from Grid-EYE sensor. The sensor value will look like a cone shape. The pixel with our head will have the highest temperature, body is lower, and leg is the lowest except background because when the distance from camera to our body is longer, the area cover by the camera will be wider and the ratio of background temperature in the pixel will increase, also our head does not cover by cloth, so the surface temperature will higher than other place. While we are walking in an area, the temperature of air in the area will become warmer, and the shape of human will become harder to recognize.
\begin{figure}[htbp]
\centering
@ -13,11 +13,11 @@ When someone walks under a Grid-EYE sensor, we will see some pixels with higher
\label{fig:GridEye}
\end{figure}
The data we used is from a solitary elder's home. We deployed four Grid-EYE sensors at the corner of her living room, and recorded the thermal video for three weeks at $10$ frames per second data rate.
The data we used is from a solitary elder's home. We deployed four Grid-EYE sensors at the corner of her living room, and recorded the thermal video for three weeks at $10$ frames per second data rate, and the size of raw data is about 17.6GB.
\subsection{FLIR ONE PRO}
FLIR ONE PRO can output a$480\times640$ pixels image with $3^\circ C$ accuracy and $0.01^\circ C$ resolution, and capture video at about 5 FPS. In picture taking mode, it can retrieve the precise data from the header of picture file. However, in the video taking mode, it only store a gray scale video and show the range of temperature on the monitor. Hence, we use $^\circ C$ in picture mode, and gray scale value as the unit to analyze error rate. Since FLIR ONE PRO can offer a image with about $5000$ times number of pixels compare to Grid-EYE. It cannot simply use a Gaussian function to fit it. Hence, we developed a method to compress FLIR images. It can also treat as a normal image and be stored as jpeg, png, etc.
FLIR ONE PRO is a thermal camera that can output $480\times640$ pixels images with $3^\circ C$ accuracy and $0.01^\circ C$ resolution, and capture speed is about 5 frames per second. In picture taking mode, it can retrieve the precise data from the header of picture file. However, in the video taking mode, it only store a gray scale video and show the range of temperature on the monitor. Hence, we use the data from picture taking mode as our test object. The data form FLIR ONE PRO has about $5000$ times resolution compare to Grid-EYE. The shape of object is not just a cone. The temperature in a same object is similar, but an Obvious edge between different objects. Hence, we developed a method to compress FLIR images. It can also treat as a normal image and be stored as jpeg, png, etc.
\subsection{Raspberry Pi 3}
@ -25,14 +25,14 @@ We use Raspberry Pi 3 as our testing environment. It has a 1.2 GHz 64-bit quad-c
\subsection{Simple Data Compressing}
If we save a frame in a readable format, it will take about 380 bytes storage. However, the temperature range of our scenario mostly from $5^\circ C$ to $40^\circ C$ and the resolution is $0.25^\circ C$, so we can easily represent each temperature by one byte. Hence, we only need $64$ bytes to store a frame. We have try several ways to compress the frame.
If we store a frame from Grid-EYE in a readable format, it will take about 380 bytes storage. However, the temperature range of indoor environment mostly from $5^\circ C$ to $40^\circ C$ and the resolution of Grid-EYE is $0.25^\circ C$, so we can easily represent each temperature by one byte. Hence, we only need $64$ bytes to store a frame. We had tried several ways to compress the frame.
\subsubsection{Huffman Coding}
Huffman coding is a lossless data compressing. In average, it can reduce the frame size from $64$ bytes to $40.7$ bytes with $6$ bytes standard deviation.
\subsubsection{Z-score Threshold}
We can only transmit the pixels with higher temperature since thermal sensors are mostly used for detect heat source. Z-score is defined as $z =\frac{\chi-\mu}{\sigma}$, where $\chi$ is the value of the temperature, $\mu$ is the average of the temperature and $\sigma$ is the standard deviation of the temperature. In our earlier work~\cite{Shih17b}, we use Z-score instead of a static threshold to detect human because the background temperature may have a $10^\circ C$ difference between day and night, and when people walk through the sensing area the Grid-EYE, the temperature reading will only increase $2^\circ C$ to $3^\circ C$. Hence, it is impossible to use a static threshold to detect human. In~\cite{Shih17b}, we only use the pixels with the Z-score higher than $2$, so we can reduce the frame size from $64$ bytes to $12.6$ bytes with $2.9$ bytes standard deviation by Z-score threshold $2$ and compress by Huffman coding.
We can only send the pixels with higher temperature since thermal sensors are mostly used for detect heat source. Z-score is defined as $z =\frac{\chi-\mu}{\sigma}$ where $\chi$ is the value of the temperature, $\mu$ is the average of the temperature and $\sigma$ is the standard deviation of the temperature. In our earlier work~\cite{Shih17b}, we use Z-score instead of a static threshold to detect human because the background temperature may have a $10^\circ C$ difference between day and night, and when people walk through the sensing area the Grid-EYE, the temperature reading will only increase $2^\circ C$ to $3^\circ C$. Hence, it is impossible to use a static threshold to detect human. In~\cite{Shih17b}, the pixels with useful data only if the Z-score is higher than $2$, so we can reduce the frame size by dropping all pixels with Z-score lower than $2$. We can reduce the file size from $64$ bytes to $12.6$ bytes with $2.9$ bytes standard deviation by Z-score threshold $2$ and compress by Huffman coding.
\subsubsection{Gaussian Function Fitting}
Since the shape of human in a thermal image looks like a cone, we may use a Gaussian function to fit the image. A Gaussian function $y = Ae^{-(x-B)^2/2C^2}$ has three parameter $A$, $B$ and $C$. The parameter $A$ is the height of the cone, $B$ is the position of the cone's peak and $C$ controls the width of the cone. We let the pixel with highest temperature be the peak of the cone, so we only need to adjust $A$ and $C$ to fit the image. Guo~\cite{guo2011simple} offer a fast way to get the fitting Gaussian function. In our testing, it will be about $0.5^\circ C$ root-mean-square error, and only needs $5$ bytes to store the position of peak and two parameters.
Since the shape of human in a image from Grid-EYE looks like a cone, we may use a Gaussian function to fit the image. A Gaussian function $y = Ae^{-(x-B)^2/2C^2}$ has three parameter $A$, $B$ and $C$. The parameter $A$ is the height of the cone, $B$ is the position of the cone's peak and $C$ controls the width of the cone. We let the pixel with highest temperature be the peak of the cone, so we only need to adjust $A$ and $C$ to fit the image. Guo~\cite{guo2011simple} offer a fast way to get the fitting Gaussian function. In our testing, it will be about $0.5^\circ C$ root-mean-square error, and only needs $5$ bytes to store the position of peak and two parameters.
This section presents the proposed method to outcome a data array than have less size compare to jpeg image when we can tolerate some error of data. We use the image captured by FLIR ONE PRO. In a thermal image, the temperature variation between nearby pixels are very small except the edge of objects. Hence, we can separate an image into several regions, and the pixels in a same region has similar value so we can use the average value to represent it and do not cause too much error. However, precisely separate an image into some polygon region takes a lot of computation time and hard to describe the edge of each region. Also, decide the number of region also a problem. Hence, to effectively describe regions we design that every region most be a rectangle, and every region can only separate into 4 regions by cut in half at the middle of horizontal and vertical. The image will start from only contains one region, and 3 regions will be added per round since we cut a region into 4 pieces.
This section presents the proposed method that can compress the image into a smaller size compare to jpeg image when we can tolerate some error of data. We use the image captured by FLIR ONE PRO. The temperature in a same object is similar, but an Obvious edge between different objects. Hence, we can separate an image into several regions, and the pixels in a same region has similar value so we can use the average value to represent it and do not cause too much error. However, precisely separate an image into some polygon region is very difficult, and need a lot of computation power. Also, decide the number of region also a problem. Hence, to effectively describe regions we design that every region most be a rectangle, and every region can only separate into 4 regions by cut in half at the middle of horizontal and vertical. The image will start from only contains one region, and 3 regions will be added per round since we cut a region into 4 pieces.
Our method is shown in Figure~\ref{fig:SystemArchitecture}. Data structure initialize only need to do once if the size of image doesn't change. A thermal image will be loaded into our data structure and separate into several regions. Finally, output data will be encoded by Huffman coding, and transmit to database. When users want to use the image, they can get the encoded data from database.
@ -24,7 +24,7 @@ For each frame, we can use a context-free language to represent it.
\end{center}
$R$ means a region of image, and it can either use the average $\alpha$ of the pixels in the region to represent whole region or separate into four regions and left a remainder $\beta$. Dependence on the image size we desired, we can choose the amount of separating regions.
The context-free grammar start from a region contain whole image. For each $R$ we calculate a score which is based on the quality of data we can improve by separate it in to smaller regions. After some operation, we can encode the image into a string $\omega$. Figure~\ref{fig:pngImage} shows an example of image which was took by FLIR ONE PRO. One of the possible outcome is Figure~\ref{fig:SeparateImage} if we separate the image 6 times and it will have 19 regions. By this method, we can iteratively separate an image until the number of regions reach our file size requirement.
The context-free grammar start from a region contain whole image. For each $R$ we calculate a score which is based on the data various in the region. After some operation, we can encode the image into a string $\omega$. Figure~\ref{fig:pngImage} shows an example of image which was took by FLIR ONE PRO. One of the possible outcome is Figure~\ref{fig:SeparateImage} if we separate the image 6 times and it will have 19 regions. By this method, we can iteratively separate an image until the number of regions reach our file size requirement.
\begin{figure}[ht]
@ -55,7 +55,7 @@ To help us choose which region to be separated, we give every region a score, an
\end{tabular}
\end{center}
By the equation shows above, we just need to know the sum of squared and the mean of a region, we can get its score. we can use a 4-dimension segment tree to store all possible regions and its scores. For each node, it store the range on both width and height it covered, sum $\sum\limits_{X\in R} X$, and squared sum $\sum\limits_{X\in R} X^2$ of pixels in the region. By the property of segment tree, tree root start from $0$, and each node $X_i$ has four child $X_{i\times4+1}$, $X_{i\times4+2}$, $X_{i\times4+3}$ and $X_{i\times4+4}$. Hence, we only need to allocate a large array and recursively process all nodes form root. Algorithm~\ref{code:SegmentTreePreprocess} shows how we generate the tree.
By the equation shows above, we just need to know the sum of squared and the mean of a region to calculate its score. We use a 4-dimension segment tree to store all possible regions and its scores. Since segment tree is a complete tree, the size of tree is less than 2 times the number if pixels. For each node of segment tree, it records the range on both width and height it covered, sum $\sum\limits_{X\in R} X$, and squared sum $\sum\limits_{X\in R} X^2$ of pixels in the region. By the property of segment tree, tree root start from $0$, and each node $X_i$ has four child $X_{i\times4+1}$, $X_{i\times4+2}$, $X_{i\times4+3}$ and $X_{i\times4+4}$. Hence, we only need to allocate a large array and recursively process all nodes form root. Algorithm~\ref{code:SegmentTreePreprocess} shows how we generate the tree.
\begin{algorithm*}[h]
\caption{Segment Tree Preprocess}
@ -80,7 +80,7 @@ By the equation shows above, we just need to know the sum of squared and the mea
\end{algorithmic}
\end{algorithm*}
For region selection, we use a priority queue to retrieve the region of considerate regions with highest score. The priority queue start with only root of the segment tree. For each round the priority queue pop the item with highest score and push all its child in to the queue. Algorithm~\ref{code:RegionSelection} shows how we select a region by the priority queue. After the selection finished, we will generate the data string to be sent. The regions in $seperatedRegions$ will be $\beta$ and others in $PriorityQueue$ will be the average value, and then compress the string by Huffman Coding.
For region selection, we use a priority queue to retrieve the region of considerate regions with highest score. The priority queue is made by heap, and start with only root of the segment tree. For each round the priority queue pop the item with highest score and push all its child in to the queue. Algorithm~\ref{code:RegionSelection} shows how we select a region by the priority queue. After the selection finished, we will generate the data string to be sent. The regions in $seperatedRegions$ will be $\beta$ and others in $PriorityQueue$ will be the average value, and then compress the string by Huffman Coding.
To evaluate the effectiveness of the proposed method, we do the different ratios of compressing on a thermal image by our method compared to JPEG image using different quality and png image, a lossless bit map image. We set the camera at the ceiling and view direction is perpendicular to the ground, and the image size is $480\times640$ pixels. The JPEG image is generated by OpenCV $3.3.0$, and image quality from $1$ to $99$.
To evaluate the effectiveness of the proposed method, we did the different ratios of compressing on a thermal image by our method compared to JPEG image using different quality and png image, a lossless bit map image. We set the camera at the ceiling and view direction is perpendicular to the ground, and the image size is $480\times640$ pixels. The JPEG image is generated by OpenCV $3.3.0$, and image quality from $1$ to $99$.
Figure~\ref{fig:4KMy} and Figure~\ref{fig:4KJpeg} show the different of JPEG and our method. JPEG image id generated by image quality level $3$, and image of our method does $1390$ rounds of separate and compressed by Huffman Coding. In this case, Huffman Coding can reduce $39\%$ of our image size.
Figure~\ref{fig:compareToJpeg} shows that the size of file can reduce more than $50\%$ compared to JPEG image when both have $0.5\%(0.18^\circ C)$ of root-mean-square error. Our method has $82\%$ less error rate when both size are $4KB$ image. The percentage of file size is compared to PNG image.
\@writefile{toc}{\defcounter{refsection}{0}\relax}\@writefile{toc}{\contentsline{subsection}{\numberline{\unhbox\voidb@x \hbox{II-B}}FLIR ONE PRO}{5}}
\@writefile{toc}{\defcounter{refsection}{0}\relax}\@writefile{toc}{\contentsline{subsection}{\numberline{\unhbox\voidb@x \hbox{II-C}}Raspberry Pi 3}{5}}
\@writefile{toc}{\defcounter{refsection}{0}\relax}\@writefile{toc}{\contentsline{subsection}{\numberline{\unhbox\voidb@x \hbox{II-D}}Simple Data Compressing}{5}}
\@writefile{toc}{\defcounter{refsection}{0}\relax}\@writefile{toc}{\contentsline{subsubsection}{\numberline{\unhbox\voidb@x \hbox{II-D}3}Gaussian Function Fitting}{6}}
\@writefile{lof}{\defcounter{refsection}{0}\relax}\@writefile{lof}{\contentsline{figure}{\numberline{3}{\ignorespaces System Architecture}}{7}}
\newlabel{fig:SystemArchitecture}{{3}{7}}
\@writefile{toc}{\defcounter{refsection}{0}\relax}\@writefile{toc}{\contentsline{subsection}{\numberline{\unhbox\voidb@x \hbox{III-B}}Data Structure and Region Selection Algorithm}{7}}
\@writefile{lof}{\defcounter{refsection}{0}\relax}\@writefile{lof}{\contentsline{figure}{\numberline{5}{\ignorespaces Region separate by CFG}}{8}}
\newlabel{fig:SeparateImage}{{5}{8}}
\@writefile{toc}{\defcounter{refsection}{0}\relax}\@writefile{toc}{\contentsline{subsection}{\numberline{\unhbox\voidb@x \hbox{III-B}}Data Structure and Region Selection Algorithm}{8}}
\@writefile{loa}{\defcounter{refsection}{0}\relax}\@writefile{loa}{\contentsline{algorithm}{\numberline{1}{\ignorespaces Segment Tree Preprocess}}{9}}
\newlabel{code:SegmentTreePreprocess}{{1}{9}}
\@writefile{loa}{\defcounter{refsection}{0}\relax}\@writefile{loa}{\contentsline{algorithm}{\numberline{2}{\ignorespaces Region Selection}}{9}}
\@writefile{lof}{\defcounter{refsection}{0}\relax}\@writefile{lof}{\contentsline{figure}{\numberline{8}{\ignorespaces Proposed method and JPEG comparing}}{12}}
\newlabel{fig:compareToJpeg}{{8}{12}}
\@writefile{lof}{\defcounter{refsection}{0}\relax}\@writefile{lof}{\contentsline{figure}{\numberline{9}{\ignorespaces Computation Time of Separate Regions}}{12}}
Hobe
7 years ago