Neural network-based echocardiogram video classification by incorporating dynamic information and attention mechanism

Echocardiography, the use of ultrasound waves to investigate the heart's action, is the primary physiological test for cardiovascular disease diagnoses. The determination of the probe viewpoint forms an essential step in echocardiographic image analysis. Some of such views are identified as sta...

Full description

Saved in:

Bibliographic Details
Main Author:	Ye, Zi
Format:	Thesis
Language:	English English
Published:	2022
Subjects:	Q Science (General) QA Mathematics
Online Access:	http://eprints.utem.edu.my/id/eprint/26988/1/Neural%20network-based%20echocardiogram%20video%20classification%20by%20incorporating%20dynamic%20information%20and%20attention%20mechanism.pdf http://eprints.utem.edu.my/id/eprint/26988/2/Neural%20network-based%20echocardiogram%20video%20classification%20by%20incorporating%20dynamic%20information%20and%20attention%20mechanism.pdf
Tags:	Add Tag No Tags, Be the first to tag this record!

id	my-utem-ep.26988
record_format	uketd_dc
spelling	my-utem-ep.269882024-01-16T14:47:02Z Neural network-based echocardiogram video classification by incorporating dynamic information and attention mechanism 2022 Ye, Zi Q Science (General) QA Mathematics Echocardiography, the use of ultrasound waves to investigate the heart's action, is the primary physiological test for cardiovascular disease diagnoses. The determination of the probe viewpoint forms an essential step in echocardiographic image analysis. Some of such views are identified as standard views due to the presentation and ease of the evaluations of their major cardiac structures. Finding valid cardiac views has traditionally been a laborious process and interpreted manually by the specialist, so there exists significant interest in providing an automated classification of echocardiograms in order to speed up the diagnosis process. However, the traditional machine learning methods require time-consuming and operator-dependent manual selection and annotation of features. Therefore, this study aims to simplify the diagnosis process by providing an automated classification of standard cardiac views based on deep learning technology. More importantly, our research considers and assesses some new neural network architectures driven by action recognition in video. For this aim, two classes of neural network architectures have been outlined: the CNN+BiLSTM model and the Spatiotemporal-BiLSTM model. It is finally verified that these methods aggregating dynamic information receive a more substantial classification effect. In addition, previous observations concluded that the most significant challenge Hes in distingnishing among the various adjacent views. To this end, our study further aimed to adopt the attention mechanism for designing efficient neural networks. We proposed an ECHOAttention architecture consisting of two parts. We first design an ECHO-ACTION block, which efficiently encodes Spatio-temporal features, channel-wise features, and motion features. Then, we insert this block into existing ResNet architectures, combined with a self-attention module to ensure its echocardiogram classification task-related focus, to form an effective ECHO-Attention network. All of these experiments are implemented on our privately collected dataset of 2693 videos acquired from 267 patients, which trained cardiologists have manually labeled. The evidence from this study showed that all the proposed methods yielded good results. The ECHO-Attention architecture provides the best classification performance (overall accuracy of 94.81 % ) on the entire video sample and achieved significant improvements on the classification of anatomically similar views (precision 88.65% and 81.70% for PSAX-AP and PSAX-MID on 30-frame clips, respectively). 2022 Thesis http://eprints.utem.edu.my/id/eprint/26988/ http://eprints.utem.edu.my/id/eprint/26988/1/Neural%20network-based%20echocardiogram%20video%20classification%20by%20incorporating%20dynamic%20information%20and%20attention%20mechanism.pdf text en public http://eprints.utem.edu.my/id/eprint/26988/2/Neural%20network-based%20echocardiogram%20video%20classification%20by%20incorporating%20dynamic%20information%20and%20attention%20mechanism.pdf text en validuser https://plh.utem.edu.my/cgi-bin/koha/opac-detail.pl?biblionumber=122132 phd doctoral Universiti Teknikal Malaysia Melaka Faculty of Information and Communication Technology Jaya Kumar, Yogan
institution	Universiti Teknikal Malaysia Melaka
collection	UTeM Repository
language	English English
advisor	Jaya Kumar, Yogan
topic	Q Science (General) QA Mathematics
spellingShingle	Q Science (General) QA Mathematics Ye, Zi Neural network-based echocardiogram video classification by incorporating dynamic information and attention mechanism
description	Echocardiography, the use of ultrasound waves to investigate the heart's action, is the primary physiological test for cardiovascular disease diagnoses. The determination of the probe viewpoint forms an essential step in echocardiographic image analysis. Some of such views are identified as standard views due to the presentation and ease of the evaluations of their major cardiac structures. Finding valid cardiac views has traditionally been a laborious process and interpreted manually by the specialist, so there exists significant interest in providing an automated classification of echocardiograms in order to speed up the diagnosis process. However, the traditional machine learning methods require time-consuming and operator-dependent manual selection and annotation of features. Therefore, this study aims to simplify the diagnosis process by providing an automated classification of standard cardiac views based on deep learning technology. More importantly, our research considers and assesses some new neural network architectures driven by action recognition in video. For this aim, two classes of neural network architectures have been outlined: the CNN+BiLSTM model and the Spatiotemporal-BiLSTM model. It is finally verified that these methods aggregating dynamic information receive a more substantial classification effect. In addition, previous observations concluded that the most significant challenge Hes in distingnishing among the various adjacent views. To this end, our study further aimed to adopt the attention mechanism for designing efficient neural networks. We proposed an ECHOAttention architecture consisting of two parts. We first design an ECHO-ACTION block, which efficiently encodes Spatio-temporal features, channel-wise features, and motion features. Then, we insert this block into existing ResNet architectures, combined with a self-attention module to ensure its echocardiogram classification task-related focus, to form an effective ECHO-Attention network. All of these experiments are implemented on our privately collected dataset of 2693 videos acquired from 267 patients, which trained cardiologists have manually labeled. The evidence from this study showed that all the proposed methods yielded good results. The ECHO-Attention architecture provides the best classification performance (overall accuracy of 94.81 % ) on the entire video sample and achieved significant improvements on the classification of anatomically similar views (precision 88.65% and 81.70% for PSAX-AP and PSAX-MID on 30-frame clips, respectively).
format	Thesis
qualification_name	Doctor of Philosophy (PhD.)
qualification_level	Doctorate
author	Ye, Zi
author_facet	Ye, Zi
author_sort	Ye, Zi
title	Neural network-based echocardiogram video classification by incorporating dynamic information and attention mechanism
title_short	Neural network-based echocardiogram video classification by incorporating dynamic information and attention mechanism
title_full	Neural network-based echocardiogram video classification by incorporating dynamic information and attention mechanism
title_fullStr	Neural network-based echocardiogram video classification by incorporating dynamic information and attention mechanism
title_full_unstemmed	Neural network-based echocardiogram video classification by incorporating dynamic information and attention mechanism
title_sort	neural network-based echocardiogram video classification by incorporating dynamic information and attention mechanism
granting_institution	Universiti Teknikal Malaysia Melaka
granting_department	Faculty of Information and Communication Technology
publishDate	2022
url	http://eprints.utem.edu.my/id/eprint/26988/1/Neural%20network-based%20echocardiogram%20video%20classification%20by%20incorporating%20dynamic%20information%20and%20attention%20mechanism.pdf http://eprints.utem.edu.my/id/eprint/26988/2/Neural%20network-based%20echocardiogram%20video%20classification%20by%20incorporating%20dynamic%20information%20and%20attention%20mechanism.pdf
_version_	1794023200287358976

Neural network-based echocardiogram video classification by incorporating dynamic information and attention mechanism

Similar Items