Stochastic computing system hardware design for convolutional neural networks optimized for accuracy area and energy efficiency

Stochastic computing (SC) is an alternative computing paradigm that can lead to designs that oﬀer lower area and power consumption compared to that of the conventional binary-encoded (BE) deterministic computing. In SC, numbers are encoded as a bit-stream of ‘0’s and ‘1’s, where SC computation eleme...

Full description

Saved in:

Bibliographic Details
Main Author:	Hamdan, Hamdan Usamah
Format:	Thesis
Language:	English
Published:	2020
Subjects:	TK Electrical engineering Electronics Nuclear engineering
Online Access:	http://eprints.utm.my/id/eprint/98197/1/HamdanUsamahHamdanPSKE2020.pdf
Tags:	Add Tag No Tags, Be the first to tag this record!

id	my-utm-ep.98197
record_format	uketd_dc
spelling	my-utm-ep.981972022-11-16T02:12:16Z Stochastic computing system hardware design for convolutional neural networks optimized for accuracy area and energy efficiency 2020 Hamdan, Hamdan Usamah TK Electrical engineering. Electronics Nuclear engineering Stochastic computing (SC) is an alternative computing paradigm that can lead to designs that oﬀer lower area and power consumption compared to that of the conventional binary-encoded (BE) deterministic computing. In SC, numbers are encoded as a bit-stream of ‘0’s and ‘1’s, where SC computation elements (or functions) operate on one or more bit-streams. To obtain accurate results, some functions require the bit-streams to be correlated, while others require uncorrelated bit-streams or a combination of both. The relationship between SC function accuracy and correlation is not well studied in previous works. Thus, managing the correlation across the SC system is a key challenge in the eﬀort to achieve optimum accuracy. In addition, to perform SC computation, the input values are converted from BE domain to SC; then on the completion of the computation, back to BE to obtain the results. The conversion processes require circuitry that typically consume over 80% of the overall SC system area, hence this is another key challenge of the problem. To address the above mentioned challenges, this thesis proposes a framework of an end-to-end system design optimized for accuracy and area. The framework provides guidelines to design an eﬀective SC function or system that exploit correlation. This framework is applied in designing the SC functional units and the complete SC system for convolutional neural network (CNN), which is the dominant approach in the implementation of recognition systems. This thesis shows that although CNN is a compute-intensive and resource-demanding algorithm, through the proposed SC design framework, it is possible to implement CNN in an embedded system with limited area and power budget. Several novel SC- based functions are proposed that outperform previous works and obtain signiﬁcant area savings and high accuracy to replace the BE equivalent functions. These functions include inner product, max pooling, ReLU activation function, and average pooling. Then, some training considerations are speciﬁed to enable achieving low error rates for SC-based CNN. Experimental results show that the SC-based CNN attained no or minor accuracy degradation compared to BE counterpart. SC-based CNN achieves 99.6% and 96.25% classiﬁcation accuracy using MNIST digit classiﬁcation and AT&T face recognition datasets, respectively. Moreover, the SC-based CNN of ResNet-20 model achieves 86.5% classiﬁcation accuracy using CIFAR-10 object dataset. To rapidly map an SC system into FPGA, a generic design strategy for high-level synthesis of SC computation engines is proposed. The SC-based CNN hardware on FPGA obtains the lowest resource utilization compared to previous works on FPGA-based CNN accelerators. In addition, the proposed hardware architecture achieves 277.46 GOP/s/W energy eﬃciency, which outperforms previous works. 2020 Thesis http://eprints.utm.my/id/eprint/98197/ http://eprints.utm.my/id/eprint/98197/1/HamdanUsamahHamdanPSKE2020.pdf application/pdf en public http://dms.library.utm.my:8080/vital/access/manager/Repository/vital:144883 phd doctoral Universiti Teknologi Malaysia, Faculty of Engineering - School of Electrical Engineering Faculty of Engineering - School of Electrical Engineering
institution	Universiti Teknologi Malaysia
collection	UTM Institutional Repository
language	English
topic	TK Electrical engineering Electronics Nuclear engineering
spellingShingle	TK Electrical engineering Electronics Nuclear engineering Hamdan, Hamdan Usamah Stochastic computing system hardware design for convolutional neural networks optimized for accuracy area and energy efficiency
description	Stochastic computing (SC) is an alternative computing paradigm that can lead to designs that oﬀer lower area and power consumption compared to that of the conventional binary-encoded (BE) deterministic computing. In SC, numbers are encoded as a bit-stream of ‘0’s and ‘1’s, where SC computation elements (or functions) operate on one or more bit-streams. To obtain accurate results, some functions require the bit-streams to be correlated, while others require uncorrelated bit-streams or a combination of both. The relationship between SC function accuracy and correlation is not well studied in previous works. Thus, managing the correlation across the SC system is a key challenge in the eﬀort to achieve optimum accuracy. In addition, to perform SC computation, the input values are converted from BE domain to SC; then on the completion of the computation, back to BE to obtain the results. The conversion processes require circuitry that typically consume over 80% of the overall SC system area, hence this is another key challenge of the problem. To address the above mentioned challenges, this thesis proposes a framework of an end-to-end system design optimized for accuracy and area. The framework provides guidelines to design an eﬀective SC function or system that exploit correlation. This framework is applied in designing the SC functional units and the complete SC system for convolutional neural network (CNN), which is the dominant approach in the implementation of recognition systems. This thesis shows that although CNN is a compute-intensive and resource-demanding algorithm, through the proposed SC design framework, it is possible to implement CNN in an embedded system with limited area and power budget. Several novel SC- based functions are proposed that outperform previous works and obtain signiﬁcant area savings and high accuracy to replace the BE equivalent functions. These functions include inner product, max pooling, ReLU activation function, and average pooling. Then, some training considerations are speciﬁed to enable achieving low error rates for SC-based CNN. Experimental results show that the SC-based CNN attained no or minor accuracy degradation compared to BE counterpart. SC-based CNN achieves 99.6% and 96.25% classiﬁcation accuracy using MNIST digit classiﬁcation and AT&T face recognition datasets, respectively. Moreover, the SC-based CNN of ResNet-20 model achieves 86.5% classiﬁcation accuracy using CIFAR-10 object dataset. To rapidly map an SC system into FPGA, a generic design strategy for high-level synthesis of SC computation engines is proposed. The SC-based CNN hardware on FPGA obtains the lowest resource utilization compared to previous works on FPGA-based CNN accelerators. In addition, the proposed hardware architecture achieves 277.46 GOP/s/W energy eﬃciency, which outperforms previous works.
format	Thesis
qualification_name	Doctor of Philosophy (PhD.)
qualification_level	Doctorate
author	Hamdan, Hamdan Usamah
author_facet	Hamdan, Hamdan Usamah
author_sort	Hamdan, Hamdan Usamah
title	Stochastic computing system hardware design for convolutional neural networks optimized for accuracy area and energy efficiency
title_short	Stochastic computing system hardware design for convolutional neural networks optimized for accuracy area and energy efficiency
title_full	Stochastic computing system hardware design for convolutional neural networks optimized for accuracy area and energy efficiency
title_fullStr	Stochastic computing system hardware design for convolutional neural networks optimized for accuracy area and energy efficiency
title_full_unstemmed	Stochastic computing system hardware design for convolutional neural networks optimized for accuracy area and energy efficiency
title_sort	stochastic computing system hardware design for convolutional neural networks optimized for accuracy area and energy efficiency
granting_institution	Universiti Teknologi Malaysia, Faculty of Engineering - School of Electrical Engineering
granting_department	Faculty of Engineering - School of Electrical Engineering
publishDate	2020
url	http://eprints.utm.my/id/eprint/98197/1/HamdanUsamahHamdanPSKE2020.pdf
_version_	1776100558314143744

Stochastic computing system hardware design for convolutional neural networks optimized for accuracy area and energy efficiency

Similar Items