Spectral domain convolutional neural network optimized for computational workload and memory access cost

Conventional convolutional neural networks (CNNs), which are realized in the spatial domain, present a high computationalworkload and memory access cost (CMC). Spectral domain CNNs (SpCNNs) offer a computationally efficient approach for performing CNN training and inference. State-of-the-art SpCNNs...

Full description

Saved in:

Bibliographic Details
Main Author:	Rizvi, Shahriyar Masud
Format:	Thesis
Language:	English
Published:	2023
Subjects:	TK Electrical engineering Electronics Nuclear engineering
Online Access:	http://eprints.utm.my/102817/1/ShahriyarMasudRizviPSKE2023.pdf.pdf
Tags:	Add Tag No Tags, Be the first to tag this record!

id	my-utm-ep.102817
record_format	uketd_dc
spelling	my-utm-ep.1028172023-09-24T03:16:15Z Spectral domain convolutional neural network optimized for computational workload and memory access cost 2023 Rizvi, Shahriyar Masud TK Electrical engineering. Electronics Nuclear engineering Conventional convolutional neural networks (CNNs), which are realized in the spatial domain, present a high computationalworkload and memory access cost (CMC). Spectral domain CNNs (SpCNNs) offer a computationally efficient approach for performing CNN training and inference. State-of-the-art SpCNNs propose activation functions (AFs) that are computationally costly or realize AFs in the spatial domain necessitating multiple and expensive spatial-spectral domain transformations. This work proposes a complex-valued AF for SpCNNs that transmits inputs unaltered or scaled depending on the activation area. This AF is computationally inexpensive and provides sufficient non-linear transformation that ensures high classification accuracy. This work also investigates the CMC of SpCNNs and its contributing components analytically and then proposes a methodology to optimize CMC, under three strategies, to enhance inference and training performance. The strategies involve the reduction of the output feature map (OFM) size, OFM depth, or both under an accuracy constraint to compute performance-optimized CNN inference and training. The proposed AF denoted as complex-valued leaky ReLU (CLReLU), was employed in a LeNet-5 SpCNN architecture and achieves an accuracy gain of up to 3% for MNIST and 8% for Fashion MNIST dataset, while providing up to 2.3 times higher throughput in inference, over state-of-the-art AFs applied to the same model. The proposed CMC reduction methodology was applied to LeNet-5 and AlexNet architectures. For instance, the optimal AlexNet model achieves up to 34 times higher throughput in inference, and up to 16 times greater energy efficiency in training, with a minor accuracy loss of 2%, as compared to related state-of-the-art work. The proposed AF and CMC reduction methodology helps develop an SpCNN model that provides faster and more energyefficient computation as well as high test accuracy. 2023 Thesis http://eprints.utm.my/102817/ http://eprints.utm.my/102817/1/ShahriyarMasudRizviPSKE2023.pdf.pdf application/pdf en public http://dms.library.utm.my:8080/vital/access/manager/Repository/vital:152199 phd doctoral Universiti Teknologi Malaysia Faculty of Engineering - School of Electrical Engineering
institution	Universiti Teknologi Malaysia
collection	UTM Institutional Repository
language	English
topic	TK Electrical engineering Electronics Nuclear engineering
spellingShingle	TK Electrical engineering Electronics Nuclear engineering Rizvi, Shahriyar Masud Spectral domain convolutional neural network optimized for computational workload and memory access cost
description	Conventional convolutional neural networks (CNNs), which are realized in the spatial domain, present a high computationalworkload and memory access cost (CMC). Spectral domain CNNs (SpCNNs) offer a computationally efficient approach for performing CNN training and inference. State-of-the-art SpCNNs propose activation functions (AFs) that are computationally costly or realize AFs in the spatial domain necessitating multiple and expensive spatial-spectral domain transformations. This work proposes a complex-valued AF for SpCNNs that transmits inputs unaltered or scaled depending on the activation area. This AF is computationally inexpensive and provides sufficient non-linear transformation that ensures high classification accuracy. This work also investigates the CMC of SpCNNs and its contributing components analytically and then proposes a methodology to optimize CMC, under three strategies, to enhance inference and training performance. The strategies involve the reduction of the output feature map (OFM) size, OFM depth, or both under an accuracy constraint to compute performance-optimized CNN inference and training. The proposed AF denoted as complex-valued leaky ReLU (CLReLU), was employed in a LeNet-5 SpCNN architecture and achieves an accuracy gain of up to 3% for MNIST and 8% for Fashion MNIST dataset, while providing up to 2.3 times higher throughput in inference, over state-of-the-art AFs applied to the same model. The proposed CMC reduction methodology was applied to LeNet-5 and AlexNet architectures. For instance, the optimal AlexNet model achieves up to 34 times higher throughput in inference, and up to 16 times greater energy efficiency in training, with a minor accuracy loss of 2%, as compared to related state-of-the-art work. The proposed AF and CMC reduction methodology helps develop an SpCNN model that provides faster and more energyefficient computation as well as high test accuracy.
format	Thesis
qualification_name	Doctor of Philosophy (PhD.)
qualification_level	Doctorate
author	Rizvi, Shahriyar Masud
author_facet	Rizvi, Shahriyar Masud
author_sort	Rizvi, Shahriyar Masud
title	Spectral domain convolutional neural network optimized for computational workload and memory access cost
title_short	Spectral domain convolutional neural network optimized for computational workload and memory access cost
title_full	Spectral domain convolutional neural network optimized for computational workload and memory access cost
title_fullStr	Spectral domain convolutional neural network optimized for computational workload and memory access cost
title_full_unstemmed	Spectral domain convolutional neural network optimized for computational workload and memory access cost
title_sort	spectral domain convolutional neural network optimized for computational workload and memory access cost
granting_institution	Universiti Teknologi Malaysia
granting_department	Faculty of Engineering - School of Electrical Engineering
publishDate	2023
url	http://eprints.utm.my/102817/1/ShahriyarMasudRizviPSKE2023.pdf.pdf
_version_	1783729221113217024

Spectral domain convolutional neural network optimized for computational workload and memory access cost

Similar Items