PhD Candidate in Electrical and Computer Engineering

University of Southern California

About me

Hi, I am a PhD candidate in the Ming Hsieh Department of Electrical and Computer Engineering at the University of Southern California, working under the guidance of Prof. Salman Avestimehr in the Information Theory and Machine Learning (vITAL) research lab. During my research pursuits, I have also collaborated closely with Prof. Murali Annavaram, Prof. Keith Chugg, and Prof. Ramtin Pedarsani. I have also had the fortune to gain industry experience through multiple internships. I spent Summer 2018 and Summer 2019 as a Research Intern at Intel Labs under Dr. Shilpa Talwar and Dr. Nageen Himayat respectively. During Summer 2021, I was an Applied Scientist Intern at Amazon Alexa AI under Dr. Clement Chung and Dr. Rahul Gupta. Prior to joining the graduate school, I completed my BTech in 2016 in Electrical Engineering from the Indian Institute of Technology Kanpur, where I worked under Prof. Aditya K. Jagannatham, in the Multimedia Wireless Networks (MWN) Group.

As a graduate student, I have been working towards holistically addressing real-world bottlenecks in large-scale distributed computing, including federated learning. My projects are broadly classified into the following three exciting paradigms:

  1. Privacy-Preserving and Robust Machine Learning at the Edge: In many machine learning applications, private training data is distributed across multiple users, such as patient records at multiple hospitals, giving rise to the following multi-dimensional problem: How can individual users jointly train an ML model while (1) keeping their individual datasets private; (2) exploiting the heterogeneity of data across users; and (3) being resilient against straggling and malicious users. For example, a key difficulty in mitigating malicious users when data is non-IID across users is that even the updates from non-malicious users are quite diverse, hence prior outlier based strategies perform poorly. My focus has been to resolve this conundrum both in the federated learning setting (where a central server orchestrates the training), as well as in the serverless decentralized training setting.
  2. Efficient Large-Scale Distributed Learning in the Cloud: In large-scale training tasks, such as pre-training of NLP models with billions of parameters (e.g., GPT-3), straggling nodes adversely impact the performance by increasing the tail latency. A simple way herein is to ignore the computations carried out at the straggling nodes. However, in many industry settings, ignoring straggling tasks is not favored as it reduces the model quality. This is very critical since the model will be used by millions of people and even a slight improvement is quite remarkable in practice. My focus in this domain has been to leverage novel computation redundancy for making distributed training straggler-resilient, leading to a significant improvement in the overall training time while simultaneously preserving the optimal convergence performance.
  3. Foundations of Coded Computing: Coded computing is a nascent transformative framework for injecting computation redundancy in unorthodox encoded forms in order to efficiently deal with communication bottleneck and system disturbances including stragglers, system and statistical heterogeneity, and adversarial computations in distributed systems. Two of the key research problems where I have leveraged as well as advanced the coded computing domain are (1) communication efficient large-scale graph processing, and (2) low-latency federated learning in wireless edge networks.

Outside research, I like hanging out with friends, watching classical Bollywood movies, and listening to Indian classical music.

Interests
  • Security and Privacy in Machine Learning
  • Efficient and Robust Federated Learning
  • Large-Scale Serverless Training
  • Coded Distributed Computing
  • Information and Coding Theory
Education
  • PhD in Electrical and Computer Engineering, 2022 (expected)

    University of Southern California

  • BTech in Electrical Engineering, 2016

    Indian Institute of Technology Kanpur

Professional
Experience

 
 
 
 
 
Graduate Research Assistant
Information Theory and Machine Learning (vITAL) Lab, University of Southern California (USC)
Aug 2016 – Present Los Angeles, CA
 
 
 
 
 
Applied Scientist Intern
Alexa AI, Amazon
Jun 2021 – Aug 2021 Cambridge, MA
 
 
 
 
 
Graduate Technical Intern
Wireless Communication Research, Intel Labs
May 2019 – Aug 2019 Santa Clara, CA
 
 
 
 
 
Graduate Technical Intern
Wireless Communication Research, Intel Labs
May 2018 – Aug 2018 Santa Clara, CA
 
 
 
 
 
International Visiting Student
IUSSTF-Viterbi Program, USC
May 2015 – Jul 2015 Los Angeles, CA
 
 
 
 
 
Undergraduate Research Assistant
Multimedia Wireless Networks (MWN) Group, IIT Kanpur
Aug 2013 – May 2016 Kanpur, India
 
 
 
 
 
Undergraduate Research Intern
Summer Undergraduate Research Grant for Excellence (SURGE), IIT Kanpur
May 2013 – Jul 2013 Kanpur, India

Projects

*
Secure Large-Scale Serveless Training at the Edge

Secure Large-Scale Serveless Training at the Edge

Developed a fast and computationally efficient Byzantine robust algorithm that leverages a sequential, memory assisted and performance criteria for training over a logical ring.

Fast and Robust Large-Scale Distributed Gradient Descent

Fast and Robust Large-Scale Distributed Gradient Descent

Developed a practical algorithm for distributed learning that is both communication efficient and straggler-resilient.

Mitigating Byzantine Attacks in Federated Learning

Mitigating Byzantine Attacks in Federated Learning

Proposed a novel sampling based approach that applies per client criteria for mitigating Byzantines in the general federated learning setting.

Low-Latency Federated Learning in Wireless Edge Networks

Low-Latency Federated Learning in Wireless Edge Networks

Proposed CodedFedL that injects structured coding redundancy into non-linear federated learning for mitigating stragglers and speeding up training procedure in heterogeneous MEC networks.

Hierarchical Decentralized Training at the Edge

Hierarchical Decentralized Training at the Edge

Formulated a problem for decentralized training from data at the edge users, incorporating the challenges of straggling communications and limited communication bandwidth.

Communciation Efficient Large-Scale Graph Processing

Communciation Efficient Large-Scale Graph Processing

Proposed and implemented a practical MapReduce based approach for large-scale graph processing.

Pre-defined Sparsity for Convolutional Neural Networks

Pre-defined Sparsity for Convolutional Neural Networks

Proposed the first approach to reduce footprint of convolutional neural networks via pre-defined sparsity.

Optimal Resource Allocation for Cloud Computing

Optimal Resource Allocation for Cloud Computing

Developed an efficient approach for load allocation in heterogeneous cloud clusters.

Publications

Quickly discover relevant content by filtering publications.

Selected
Talks

TEE-GPU Cooperative Learning: Privacy and Security Without the Price
Federated deep learning: On-device learning for CV and NLP
Coded Computing for Federated Learning at the Edge

Selected
Awards

Qualcomm Innovation Fellowship
Most Novel Research Project Award
Annenberg PhD Fellowship

Contact