The changing nature of large scalable AI and ML models – By Aditya Abeysinghe
Distributed AI
Artificial Intelligence (AI) models are often costly to operate due to high Central Processing Unit (CPU) power required to process data. AI models also depend on additional constraints such as volatile memory, secondary memory and power. Most medium and large scale AI models are processed on cloud or remote data center servers which have high CPU power and memory compared to small scale processing devices due to these issues. However, sending data to a remote server is often not viable due to extra costs of data communication, costs of maintaining and monitoring servers in cloud and due to issues with privacy.
Distributed AI is a method to distribute AI models outside a server to a location close to a data source. With distributed AI, AI models are deployed at either the source or at a server closer to the source. Distributing AI models has high privacy compared to models hosted at a central server while reducing latency in responses. However, this is often not an ideal solution especially when there are varied types of devices with varied types of data available. Several methods, e.g., federated inference, to solve this issue with varied need for processing models have been used.
Data extraction before transmission
Today, centralized AI models process data from large number of edge devices. Edge devices generate data of various data formats. Data from these devices need to be often processed in remote servers due to low CPU capacity of edge devices. However, often data that is not useful for analyzing is being transmitted. Therefore, high latency of receiving output from model processing due to processing unnecessary data causes issues in devices which require data to be analyzed with low latency.
Most datasets used for analysis often contain data that is not useful or has redundant values. A solution to this issue with redundancy or less useful datasets is to retain only useful data by removing unnecessary data using extracting only data that could be used for analyzing in AI models. Using data extraction, data with required attributes and that are non-redundant are only transmitted. This reduces the time taken for analysis in AI models.
Model management
Distributing machine learning (ML) models to edge and processing aggregated data from data sources is a new method of computing. With this method, a model is trained using data that will be generated in node devices or using data in a dataset and then tested on data from these nodes. However, outliers to trained data may be used as inputs when these models are being used with data generated from nodes. This is common due to changes in pattern of usage or due to errors or security issues in devices. Model management is often used to manage and update a model when there are changes in data or parameters. This method allows models to be tuned, retrained and retested when changes are observed in generated data.
Several approaches to solve issues with large scale AI and ML models are being used. The most common method is to distribute AI models to edge or devices such that less time and cost are required for processing. However, with diverse data and changes in data over time, and due to the low latency response needed for real-time processes, methods such as data extraction and model management are being deployed for these devices and servers. These methods have enhanced use of large AI and ML models by minimizing drawbacks related with computation, network and time.
Image courtesy: https://www.insurtechinsights.com/
