Gated recurrent unit for low power wake-word detection

Neural networks made some of the latest state of the art technologies such as speech recognition, language translation and stock prediction possible. Among them, speech recognition is a very popular application which is growing rapidly. It is widely used in applications such as mobile phones and Ama...

Full description

Saved in:

Bibliographic Details
Main Author:	Chin, Jian Qee
Format:	Thesis
Language:	English
Published:	2021
Subjects:	TK Electrical engineering Electronics Nuclear engineering
Online Access:	http://eprints.utm.my/id/eprint/96438/1/JianQeeChinMFABU2021.pdf.pdf
Tags:	Add Tag No Tags, Be the first to tag this record!

id	my-utm-ep.96438
record_format	uketd_dc
spelling	my-utm-ep.964382022-07-24T09:57:40Z Gated recurrent unit for low power wake-word detection 2021 Chin, Jian Qee TK Electrical engineering. Electronics Nuclear engineering Neural networks made some of the latest state of the art technologies such as speech recognition, language translation and stock prediction possible. Among them, speech recognition is a very popular application which is growing rapidly. It is widely used in applications such as mobile phones and Amazon smart speakers in order to enhance user experience. However, neural networks used for speech recognition require a large amount of computations, especially if it is in always-on state. This made it infeasible to be implemented in battery-powered edge devices such as wearables, sensors, and internet-of-things devices, as the battery life will not last long enough to provide a good user experience. To address this issue, this work enhances the recurrent neural network (RNN), or specifically, Gated Recurrent Unit (GRU) for the task ofwake-word detection. Awake-word detector is always powered-on, listening to a specific phrase, the wake-word. Therefore, the power consumption must be low enough to enable long battery usage – a feature that is sought by many end-consumers. This work proposes four modifications to the existing GRU architecture. First, the reset gate is removed as there are researches which implies that it is not needed in application such as speech recognition. Second, the activation function is changed from the conventional sigmoid/hyperbolic tangent function to softsign function. Third, weight quantization is carried out to reduce the memory footprint and speed up calculations. Fourth, fixed point arithmetic is used instead of floating point format. With the above enhancements in architecture, memory and power consumption is reduced while keeping the impact to the accuracy minimal. Furthermore, it is possible to embed this new neural network model to battery-powered edge devices such as wearables. In summary, this work explores the possibility of implementing an improved GRU architecture in batterypowered edge devices to enable low-power usage for speech recognition purpose. 2021 Thesis http://eprints.utm.my/id/eprint/96438/ http://eprints.utm.my/id/eprint/96438/1/JianQeeChinMFABU2021.pdf.pdf application/pdf en public http://dms.library.utm.my:8080/vital/access/manager/Repository/vital:142167 masters Universiti Teknologi Malaysia Faculty of Engineering - School of Electrical Engineering
institution	Universiti Teknologi Malaysia
collection	UTM Institutional Repository
language	English
topic	TK Electrical engineering Electronics Nuclear engineering
spellingShingle	TK Electrical engineering Electronics Nuclear engineering Chin, Jian Qee Gated recurrent unit for low power wake-word detection
description	Neural networks made some of the latest state of the art technologies such as speech recognition, language translation and stock prediction possible. Among them, speech recognition is a very popular application which is growing rapidly. It is widely used in applications such as mobile phones and Amazon smart speakers in order to enhance user experience. However, neural networks used for speech recognition require a large amount of computations, especially if it is in always-on state. This made it infeasible to be implemented in battery-powered edge devices such as wearables, sensors, and internet-of-things devices, as the battery life will not last long enough to provide a good user experience. To address this issue, this work enhances the recurrent neural network (RNN), or specifically, Gated Recurrent Unit (GRU) for the task ofwake-word detection. Awake-word detector is always powered-on, listening to a specific phrase, the wake-word. Therefore, the power consumption must be low enough to enable long battery usage – a feature that is sought by many end-consumers. This work proposes four modifications to the existing GRU architecture. First, the reset gate is removed as there are researches which implies that it is not needed in application such as speech recognition. Second, the activation function is changed from the conventional sigmoid/hyperbolic tangent function to softsign function. Third, weight quantization is carried out to reduce the memory footprint and speed up calculations. Fourth, fixed point arithmetic is used instead of floating point format. With the above enhancements in architecture, memory and power consumption is reduced while keeping the impact to the accuracy minimal. Furthermore, it is possible to embed this new neural network model to battery-powered edge devices such as wearables. In summary, this work explores the possibility of implementing an improved GRU architecture in batterypowered edge devices to enable low-power usage for speech recognition purpose.
format	Thesis
qualification_level	Master's degree
author	Chin, Jian Qee
author_facet	Chin, Jian Qee
author_sort	Chin, Jian Qee
title	Gated recurrent unit for low power wake-word detection
title_short	Gated recurrent unit for low power wake-word detection
title_full	Gated recurrent unit for low power wake-word detection
title_fullStr	Gated recurrent unit for low power wake-word detection
title_full_unstemmed	Gated recurrent unit for low power wake-word detection
title_sort	gated recurrent unit for low power wake-word detection
granting_institution	Universiti Teknologi Malaysia
granting_department	Faculty of Engineering - School of Electrical Engineering
publishDate	2021
url	http://eprints.utm.my/id/eprint/96438/1/JianQeeChinMFABU2021.pdf.pdf
_version_	1747818668544229376

Gated recurrent unit for low power wake-word detection

Similar Items