Automated Machine Learning Uses and Approaches ExplainedAs organizations seek to use machine learning in more diverse use cases, the amount of pre and post data processing and optimization scales exponentially. The difficulty to hire enough people to do all the tasks associated with advanced machine learning models makes automated tools for machine learning a critical component for the future of AI – which leads us to automated machine learning (AutoML), a quickly growing tool in the AIOps toolkit. AutoML is a way to automate the end-to-end cycle of applying artificial intelligence (AI) to a problem. Data scientists are typically responsible for building ML models and all of the complex tasks that come with that: data pre-processing, feature engineering, model selection, optimization of hyperparameters, and model post-processing. AutoML frameworks complete these steps (or at least some of them) automatically, so that people without data science expertise can build successful ML models. The ability to automate ML processes opens the door to exciting opportunities for companies with limited resources to fully invest in AI. While there’s still much progress to be made toward full automation of ML pipelines, companies are building promising tools for furthering development in this area.
Why Use AutoML Tools?If we examine the current process for building a machine learning model, it typically requires highly-skilled technical experts, a long development process, a lot of money, and a lot of iterations. The push for AutoML is driven by all four of these factors:
Bridge the Skill GapWhen it comes to technical expertise in AI and ML, a skill gap persists. Companies struggle to source candidates who have the domain knowledge and skills to build models, and progress is limited by this restriction. With AutoML, machine learning becomes available for non-experts. Companies don’t need to hire for highly-specialized positions, increasing the acceleration of innovation and ultimately, ML adoption.
Reduce Time-to-MarketIn a field that advances quickly, faster time-to-market offers a significant competitive edge. Automating aspects of the machine learning pipeline decreases time required by humans to build models. It also makes it easier for a company that has never deployed AI before to enter the space and produce a successful solution.
Generate Cost-SavingsBuilding ML models from scratch requires not only extensive time, but a lot of money. Salaries for data scientists and other ML specialists are understandably high. AutoML tools are much more affordable than investing in the skills and effort required to build a model from the ground up.
Produce Better ModelsAutoML iterates through models and hyperparameters more quickly than when done manually. In a set period of time, more iterations will generally lead to the selection of a higher-performing model. AutoML improves the efficiency of the decision process and accelerates model research. Data scientists also struggle to find high-performing architectures for deep neural networks. AutoML will (automatically) search and evaluate architectures – a process known as Neural Architecture Search – to accelerate the development of ML solutions.
Approaches to AutoMLThere are different ways of defining automation when it comes to machine learning. Experts are now categorizing AutoML into levels (much like they have with autonomous vehicles):
- Level 0: No automation. Data scientists code algorithms from scratch.
- Level 1: Use of high-level APIs.
- Level 2: Automatic hyperparameter tuning and model selection.
- Level 3: Automatic feature engineering, feature selection, and data augmentation.
- Level 4: Automatic domain and problem-specific feature engineering, data augmentation, and data integration.
- Level 5: Full automation. No input or guidance required to solve ML problems.
Model Selection and EnsemblingAutoML can iterate through different algorithms trained on the same input data to select the model that performs best. The software may also perform ensembling, which refers to combining several models into one to achieve a better result, often done through techniques like blending and stacking.
Hyperparameter Optimization (HPO)All machine learning algorithms have parameters, or the weights for each variable or feature in the model. A parameter is derived from the training process, while hyperparameter is an adjustable value used to control the learning process. Hyperparameter optimization (HPO) refers to tuning the hyperparameters to improve model performance. AutoML tools can automatically evaluate various hyperparameters to identify the set that results in the highest-performing model.
Feature EngineeringFeature engineering is less common than model selection and HPO in AutoML, although gaining traction due to its ability to improve model predictivity. It’s the constructing of new input features (or explanatory variables) from your existing inputs. It influences model performance as it highlights relevant elements for your model to know and understand as it makes predictions. Data scientists have to manually add features one at a time, but with AutoML tools, that process can be done automatically. These tools extract relevant and meaningful features from a given set of inputs and test different combinations of features to produce the highest performing model.
The Future of AutoMLThe industry still has a very long way to go before ever reaching Level 5, a fully automated solution. Nonetheless, major organizations have invested in AutoML at the lower levels, generally targeting their efforts at model selection and HPO. Advances in feature engineering are likely to be the next stage of innovation in the space. As demands for automation grow and tooling improves, ML adoption will likewise increase as building machine learning becomes more approachable and less resource-intensive.
AutoML Insights from Shambhavi Srivastava, Data Scientist at AppenAt Appen, we work on machine learning model production as a team. My data scientists, machine learning engineers and DevOps colleagues and I work together to build and containerize state-of-the-art (SOTA) models. Productionizing any machine learning models involves multiple steps:
- Understanding the problem from a business perspective
- Preparing data (collecting, cleaning and analysis)
- Building the model
- Evaluating the performance
- Containerize and deploy model to production
- Observe model’s performance on client data in production.