About EGI
EGI is an open ecosystem for research and innovation that supports data-intensive research with a wide range of advanced computing services.
EGI offers a series of services that offer advanced computing, data spaces and service hosting to support scientists, international projects, research infrastructures and businesses to drive forward their work.
Cooperation between EGI and AI-SPRINT
EGI and AI-SPRINT have started a strong cooperation that brings mutual benefits. Indeed, from one side, EGI offers services that provide online storage and cloud computing, and on the other side, AI-SPRINT can exploit them to build its tools.
In particular, AI-SPRINT used the following two of the main EGI Services to fulfill its objectives:
- EGI Cloud Compute, which offers the opportunity to deploy and scale virtual machines on-demand and to select pre-configured virtual appliances (e.g. CPU, memory, disk, operating system or software) from a catalogue replicated across all EGI cloud providers.
- EGI Online Storage, which allows to store data in a reliable and high-quality environment and share it across distributed teams.
These services helped to develop POPNAS (Pareto-Optimal Progressive Neural Architecture Search), one of the design time tools of the AI-SPRINT project, which is based on Neural Architecture Search, an Auto-ML technique capable of finding optimal neural network architectures for a given task and dataset. The algorithm can consider and optimize multiple objectives, making it easier to deploy the final architectures under potential system constraints. Furthermore, the final architectures are composed of stacking multiple modular units, which makes partitioning into the edge and cloud simple and efficient. This approach makes it possible to generate state-of-the-art neural network models in a single end-to-end process, with minimal AI-expertise requirements, enabling wider adoption of deep learning techniques in the industry.
POPNAS expands PNAS, an established method in the NAS literature, with an additional surrogate ML model, used to estimate the training time required by the selected architecture. Predicting both the accuracy and training time reached by the candidate architectures allows POPNAS to address NAS as a multi-objective optimization problem, solved through Pareto-optimality. This optimization technique selects for training only the neural architectures estimated to reach the best tradeoff between the considered metrics.
Experiments have been performed on four different image classification datasets (CIFAR10, CIFAR100, fashion-MNIST and EuroSAT), executing POPNAS and PNAS with the same configuration parameters, to make a fair comparison. POPNAS algorithm can find architectures with competitive accuracy with PNAS, while drastically reducing the search time by an average 4x speed-up factor. Pareto optimization is the key factor to find simpler architectures with similar accuracy, pruning suboptimal time-consuming architectures from the training selection, drastically improving resource usage and energy requirements.
The work has been published at the WCCI 2022 and the open-source code is available on Zenodo.