publications

2020

1. Patent

   Systems and Methods for Radar Based Face Authentication Anti-Spoofing

   Qiu, Wenxun, Sehgal, Abhishek, Gill, Kuldeep, Vutha, VA, and Ng, Boon Loong

   2020

   HTML
2. Patent

   Method and Apparatus for Biometric Authentication using Face Radar Signal

   Nguyen, Khuong N, Qiu, Wenxun, Ng, Boon Loong, Vutha, VA, and Sehgal, Abhishek

   2020

   HTML
3. Patent

   Millimeter Wave Radar and Camera Fusion Based Face Authentication System

   Sehgal, Abhishek, Qiu, Wenxun, and Ng, Boon Loong

   2020

   HTML
4. Patent

   Method and Apparatus for Intelligent Wifi Connection Management

   Qiu, Wenxun, Chen, Hao, Tonnemacher, Matthew, In-Sick, JUNG, Sung, Jihoon, Nguyen, Khuong N, Sehgal, Abhishek, Mo, Jianhua, Zhang, Jianzhong, Jung, Junyeop, and others,

   2020

   HTML
5. Book

   Smartphone-Based Real-Time Digital Signal Processing

   Kehtarnavaz, Nasser, Sehgal, Abhishek, Parris, Shane, and Azarang, Arian

   Synthesis Lectures on Signal Processing 2020

   Abs HTML

   Real-time or applied digital signal processing courses are offered as follow-ups to conventional or theory-oriented digital signal processing courses in many engineering programs for the purpose of teaching students the technical know-how for putting signal processing algorithms or theory into practical use. These courses normally involve access to a teaching laboratory that is equipped with hardware boards, in particular DSP boards, together with their supporting software. A number of textbooks have been written discussing how to achieve real-time implementation on these hardware boards. This book discusses how to use smartphones as hardware boards for real-time implementation of signal processing algorithms, thus providing an alternative to the hardware boards that are used in signal processing laboratory courses.

2019

1. Ph.D. Thesis

   Guidelines and Benchmarks for Deployment of Deep Learning Models on Smartphones as Real-Time Apps

   Sehgal, Abhishek

   2019

   HTML
2. arXiv

   Guidelines and Benchmarks For Deployment of Deep Learning Models On Smartphones As Real-Time Apps

   Sehgal, Abhishek, and Kehtarnavaz, Nasser

   Machine Learning and Knowledge Extraction 2019

   Abs arXiv HTML Code

   Deep learning solutions are being increasingly used in mobile applications. Although there are many open-source software tools for the development of deep learning solutions, there are no guidelines in one place in a unified manner for using these tools toward real-time deployment of these solutions on smartphones. From the variety of available deep learning tools, the most suited ones are used in this paper to enable real-time deployment of deep learning inference networks on smartphones. A uniform flow of implementation is devised for both Android and iOS smartphones. The advantage of using multi-threading to achieve or improve real-time throughputs is also showcased. A benchmarking framework consisting of accuracy, CPU/GPU consumption, and real-time throughput is considered for validation purposes. The developed deployment approach allows deep learning models to be turned into real-time smartphone apps with ease based on publicly available deep learning and smartphone software tools. This approach is applied to six popular or representative convolutional neural network models, and the validation results based on the benchmarking metrics are reported.

3. SPIE

   A Deep Learning-Based Smartphone App for Real-Time Detection of Retinal Abnormalities In Fundus Images

   Wei, Haoran, Sehgal, Abhishek, and Kehtarnavaz, Nasser

   In Real-Time Image Processing and Deep Learning 2019

   Abs HTML

   This paper presents the real-time implementation of two deep neural networks, which are trained for detection of eye retina abnormalities, on smartphones as an app. This app provides a low-cost and universally accessible alternative to fundus cameras since smartphones are widely available and they can be fitted with lenses that are commercially available for examination of the retina. The process of training two convolutional neural networks for retinal abnormality detection based on two publicly available datasets is discussed. Furthermore, it is shown how a smartphone app, both Android and iOS versions, are created from these trained networks. The results obtained indicate that it is possible to carry out the detection of retinal abnormalities on smartphones in an on-the-fly manner as retina images get captured by their cameras in real-time.

2018

1. IEEE Access Featured Article

   A Convolutional Neural Network Smartphone App For Real-Time Voice Activity Detection

   Sehgal, Abhishek, and Kehtarnavaz, Nasser

   IEEE Access 2018

   Abs HTML Code

   This paper presents a smartphone app that performs real-time voice activity detection based on convolutional neural network. Real-time implementation issues are discussed showing how the slow inference time associated with convolutional neural networks is addressed. The developed smartphone app is meant to act as a switch for noise reduction in the signal processing pipelines of hearing devices, enabling noise estimation or classification to be conducted in noise-only parts of noisy speech signals. The developed smartphone app is compared with a previously developed voice activity detection app as well as with two highly cited voice activity detection algorithms. The experimental results indicate that the developed app using convolutional neural network outperforms the previously developed smartphone app.

2. IEEE ICCE

   Utilization of Two Microphones for Real-Time Low-Latency Audio Smartphone Apps

   Sehgal, Abhishek, and Kehtarnavaz, Nasser

   In IEEE International Conference on Consumer Electronics (ICCE) 2018

   Abs HTML

   This paper presents an approach to overcome the limitation imposed by existing smartphone operating systems in using two microphones of smartphones at the same time for real-time low-latency audio apps that require the use of two microphones. This approach involves the use of an external dual-microphone and the processing steps needed to access audio signals from both of the external microphones at the same time. The developed approach is then applied to two example audio applications consisting of noise classification and noise reduction. It is shown that these dual-microphone apps lead to improved noise classification and noise reduction accuracy compared to the apps using a single microphone.

3. IEEE EMBC

   Integrating Signal Processing Modules of Hearing Aids Into A Real-Time Smartphone App

   Chowdhury, Tahsin A, Sehgal, Abhishek, and Kehtarnavaz, Nasser

   In Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) 2018

   Abs HTML

   This paper presents the integration of three major modules of the signal processing pipeline that go into a typical digital hearing aid as a real-time smartphone app. These modules include voice activity detection, noise reduction, and compression. The steps taken to allow the real-time implementation of this integration or signal processing pipeline are discussed. These steps can be utilized to create similar signal processing pipelines or integrated apps to evaluate hearing improvement algorithms. The real-time characteristics of the developed integrated app are reported as well as an objective evaluation of its noise reduction.

2017

1. IEEE ISIE

   Real-Time Implementation of Voice Activity Detector On Arm Embedded Processor of Smartphones

   Sehgal, Abhishek, Saki, Fatemeh, and Kehtarnavaz, Nasser

   In 2017 IEEE International Symposium on Industrial Electronics (ISIE) 2017

   Abs HTML

   This paper presents the real-time implementation of a previously developed voice activity detector (VAD) on ARM embedded processor of smartphones as an app. The implementation is carried out on both Android and iOS smartphones. The real-time detection rates on smartphone platforms in realistic audio environments are reported. As an application of the developed app, it is used to act as an automatic switch between noise and speech sound signals in a noise adaptive speech enhancement pipeline. Objective measures are considered to examine enhanced speech for the switching done automatically and manually as the ground truth. The results obtained indicate that the automatic switching done by the real-time smartphone VAD app performs identical to the manual switching.

2. IEEE ISIE

   Low-Latency Smartphone App for Real-Time Noise Reduction of Noisy Speech Signals

   Bhattacharya, Aditya, Sehgal, Abhishek, and Kehtarnavaz, Nasser

   In 2017 IEEE International Symposium on Industrial Electronics (ISIE) 2017

   Abs HTML

   This papers presents a low-latency smartphone app to achieve real-time noise reduction of speech signals in noisy sound environments. This app overcomes the two shortcomings of high latency and musical noise artifact that are associated with the previously developed apps for the same purpose. The results of both objective and subjective evaluations are reported which show the effectiveness of this app towards reducing noise for speech signals in noisy environments, in particular in babble background noise.

3. IEEE SPMB

   An Unsupervised Noise Classification Smartphone App for Hearing Improvement Devices

   Alamdari, N, Saki, F, Sehgal, A, and Kehtarnavaz, N

   In IEEE Signal Processing in Medicine and Biology Symposium (SPMB) 2017

   Abs HTML

   This paper presents an app for running a previously developed unsupervised noise classifier in realtime on smartphone/tablet platforms. The steps taken to enable the development of this app are discussed. The app is utilized to carry out field testing of the unsupervised classification of actual encountered noise environments without any prior training and without specifying the number of noise classes or clusters. Two objective measures of cluster purity and normalized mutual information are considered to examine the performance of the app in the field with the user acting as the identifier of the ground truth classes. The results obtained indicate the effectiveness of this real-time smartphone app for carrying out the environmental noise classification in an unsupervised manner.

2016

1. IEEE ICASSP

   Smartphone-Based Real-Time Classification of Noise Signals using Subband Features And Random Forest Classifier

   Saki, Fatemeh, Sehgal, Abhishek, Panahi, Issa, and Kehtarnavaz, Nasser

   In IEEE ICASSP 2016

   Abs HTML

   This paper presents the real-time implementation and field testing of an app running on smartphones for classifying noise signals involving subband features and a random forest classifier. This app is compared to a previously developed app utilizing mel-frequency cepstral coefficients features and a Gaussian mixture model classifier. The real-time implementation has been carried out on both the Android and iOS smartphones. The field testing results indicate the superiority of this newly developed app over the previously developed app in terms of classification rates.

2015

1. IEEE SP/SPE

   Using Smartphones As Mobile Implementation Platforms for Applied Digital Signal Processing Courses

   Kehtarnavaz, Nasser, Parris, Shane, and Sehgal, Abhishek

   In IEEE Signal Processing and Signal Processing Education Workshop (SP/SPE) 2015

   Abs HTML

   This paper presents a newly developed laboratory paradigm for teaching applied or real-time digital signal processing courses. It involves the utilization of smartphones to implement digital signal processing algorithms in real-time using ARM processors of smartphones. Representative laboratory experiments together with an application project running on smartphones as apps are discussed in the paper. It is shown that such a paradigm provides a cost-free and a truly mobile laboratory environment for students to learn implementation aspects of signal processing algorithms.

2. MS Thesis

   Real-Time Implementation of Noise Classifiers On Smartphones for Hearing Studies

   Sehgal, Abhishek

   2015

   HTML