Resilient Federated Learning for Trustworthy AI
August 31, 2021
Problem
The effectiveness of machine learning models in operational connective vehicle environments will depend on the ability to certify model’s “operating envelope.” The certification process will provide model based guarantees for safe and resilient operation of IoBT devices in normal or hazard environment. However, current AI techniques are unable to provide assurance if a machine learning model can operate and provide desired performance metrics under dynamic and uncertain conditions.
Approach
In this project, we will develop a transfer learning approach to certify an AI model’s operating envelope under normal and hazardous operational environments. Specifically, this project will develop metrics to quantify the operating envelope using hierarchical transfer learning algorithm and test the proposed methodology in a IoBT environment.
Given a set of data and pre-trained classification models from the source environment, the proposed approach will determine an economical and accurate means to select the useful data and models to meet the performance metrics of the target task. The hierarchical transfer learning approach uses knowledge of the source environment’s system structure and dynamics interaction to automate the process of finding the relevance between source and target environments. This information will guide the feature transfer learning process applied to the data extracted from tactical environments. This process will guide the feature transfer learning in algorithm space. Specifically, this process will help determine the distribution of the hyperparameter and ensure resilience due to unguided parameterization that can lead to incorrect results. We will use statistical approaches, the team will develop metrics to characterize the ability to transfer source models in tactical environments and evaluate the accuracy without retraining. The team will also explore the amount of information needed for the transfer process to achieve optimality and maintain the operating envelope. We will also develop an out-of-band lightweight policy network for fine tuning a source Convolutional Neural Network (CNN) for a given target task from a set of pre-trained CNN models. This approach will be key for mode agnostic fine tuning and selection of optimal source models for a target task. We will also look into automatically ranking source CNNs prior to utilizing them for a target task. As we discussed yesterday, in some scenario, you may prioritize precision over recall. Based on the performance metric expectation, we will automate ranking and selection of the source CNN models.
Funding
DOD Center of Excellence in Machine Learning
Publication
Sayyed Farid Ahamed, Priyanka Aggarwal,Sachin Shetty, Erin Lanus, Laura Freeman, "ATTL: An Automated Targeted Transfer Learning with Deep Neural Network," Big Data Track, IEEE Global Communications Conference (IEEE Globecom 2021).