AutoML相关论文-CFANZ编程社区

本文为Awesome-AutoML-Papers的译文。

1、AutoML简介

Machine Learning几年来取得的不少可观的成绩，越来越多的学科都依赖于它。然而，这些成果都很大程度上取决于人类机器学习专家来完成如下工作：

数据预处理 Preprocess the data
选择合适的特征 Select appropriate features
选择合适的模型族 Select an appropriate model family
优化模型参数 Optimize model hyperparameters
模型后处理 Postprocess machine learning models
分析结果 Critically analyze the results obtained

随着大多数任务的复杂度都远超非机器学习专家的能力范畴，机器学习应用的不断增长使得人们对现成的机器学习方法有了极大的需求。因为这些现成的机器学习方法使用简单，并且不需要专业知识。我们将由此产生的研究领域称为机器学习的逐步自动化。

AutoML借鉴了机器学习的很多知识，主要包括：

贝叶斯优化 Bayesian optimization
结构化数据的大数据的回归模型 Regression models for structured data and big data
元学习 Meta learning
迁移学习 Transfer learning
组合优化 Combinatorial optimization.

2、目录

Papers

Automated Feature Engineering

Expand Reduce
Hierarchical Organization of Transformations
Meta Learning
Reinforcement Learning

Architecture Search

Evolutionary Algorithms
Local Search
Meta Learning
Reinforcement Learning
Transfer Learning

Hyperparameter Optimization

Bayesian Optimization
Evolutionary Algorithms
Lipschitz Functions
Local Search
Meta Learning
Particle Swarm Optimization
Random Search
Transfer Learning

Performance Prediction

Performance Prediction

Frameworks
Miscellaneous

Tutorials

Bayesian Optimization
Meta Learning

Articles

Bayesian Optimization
Meta Learning

Slides

Bayesian Optimization

Books

Meta Learning

Projects
Prominent Researchers

Papers

Automated Feature Engineering

Expand Reduce

2017 | AutoLearn — Automated Feature Generation and Selection | Ambika Kaul, et al. | ICDM |PDF
2017 | One button machine for automating feature engineering in relational databases | Hoang Thanh Lam, et al. | arXiv |PDF
2016 | Automating Feature Engineering | Udayan Khurana, et al. | NIPS |PDF
2016 | ExploreKit: Automatic Feature Generation and Selection | Gilad Katz, et al. | ICDM |PDF
2015 | Deep Feature Synthesis: Towards Automating Data Science Endeavors | James Max Kanter, Kalyan Veeramachaneni | DSAA |PDF

Hierarchical Organization of Transformations

2016 | Cognito: Automated Feature Engineering for Supervised Learning | Udayan Khurana, et al. | ICDMW |PDF

Meta Learning

2017 | Learning Feature Engineering for Classification | Fatemeh Nargesian, et al. | IJCAI |PDF

Reinforcement Learning

2017 | Feature Engineering for Predictive Modeling using Reinforcement Learning | Udayan Khurana, et al. | arXiv |PDF
2010 | Feature Selection as a One-Player Game | Romaric Gaudel, Michele Sebag | ICML | PDF

Architecture Search

Evolutionary Algorithms

2017 | Large-Scale Evolution of Image Classifiers | Esteban Real, et al. | PMLR |PDF
2002 | Evolving Neural Networks through Augmenting Topologies | Kenneth O.Stanley, Risto Miikkulainen | Evolutionary Computation |PDF

Local Search

2017 | Simple and Efficient Architecture Search for Convolutional Neural Networks | Thomoas Elsken, et al. | ICLR |PDF

Meta Learning

2016 | Learning to Optimize | Ke Li, Jitendra Malik | arXiv |PDF

Reinforcement Learning

2018 | Efficient Neural Architecture Search via Parameter Sharing | Hieu Pham, et al. | arXiv |PDF
2017 | Neural Architecture Search with Reinforcement Learning | Barret Zoph, Quoc V. Le | ICLR |PDF

Transfer Learning

2017 | Learning Transferable Architectures for Scalable Image Recognition | Barret Zoph, et al. | arXiv | PDF

Frameworks

2017 | Google Vizier: A Service for Black-Box Optimization | Daniel Golovin, et al. | KDD |PDF
2017 | ATM: A Distributed, Collaborative, Scalable System for Automated Machine Learning | T. Swearingen, et al. | IEEE |PDF
2015 | AutoCompete: A Framework for Machine Learning Competitions | Abhishek Thakur, et al. | ICML |PDF

Hyperparameter Optimization

Bayesian Optimization

2016 | Bayesian Optimization with Robust Bayesian Neural Networks | Jost Tobias Springenberg， et al. | NIPS |PDF
2016 | Scalable Hyperparameter Optimization with Products of Gaussian Process Experts | Nicolas Schilling, et al. | PKDD |PDF
2016 | Taking the Human Out of the Loop: A Review of Bayesian Optimization | Bobak Shahriari, et al. | IEEE |PDF
2016 | Towards Automatically-Tuned Neural Networks | Hector Mendoza, et al. | JMLR |PDF
2016 | Two-Stage Transfer Surrogate Model for Automatic Hyperparameter Optimization | Martin Wistuba, et al. | PKDD |PDF
2015 | Efficient and Robust Automated Machine Learning |PDF
2015 | Hyperparameter Optimization with Factorized Multilayer Perceptrons | Nicolas Schilling, et al. | PKDD |PDF
2015 | Hyperparameter Search Space Pruning - A New Component for Sequential Model-Based Hyperparameter Optimization | Martin Wistua, et al. |PDF
2015 | Joint Model Choice and Hyperparameter Optimization with Factorized Multilayer Perceptrons | Nicolas Schilling, et al. | ICTAI |PDF
2015 | Learning Hyperparameter Optimization Initializations | Martin Wistuba, et al. | DSAA |PDF
2015 | Scalable Bayesian optimization using deep neural networks | Jasper Snoek, et al. | ACM |PDF
2015 | Sequential Model-free Hyperparameter Tuning | Martin Wistuba, et al. | ICDM |PDF
2013 | Auto-WEKA: Combined Selection and Hyperparameter Optimization of Classification Algorithms |PDF
2013 | Making a Science of Model Search: Hyperparameter Optimization in Hundreds of Dimensions for Vision Architectures | J. Bergstra | JMLR |PDF
2012 | Practical Bayesian Optimization of Machine Learning Algorithms |PDF
2011 | Sequential Model-Based Optimization for General Algorithm Configuration(extended version) |PDF

Evolutionary Algorithms

2018 | Autostacker: A Compositional Evolutionary Learning System | Boyuan Chen, et al. | arXiv |PDF
2017 | Large-Scale Evolution of Image Classifiers | Esteban Real, et al. | PMLR |PDF

Lipschitz Functions

2017 | Global Optimization of Lipschitz functions | C´edric Malherbe, Nicolas Vayatis | arXiv |PDF

Local Search

2009 | ParamILS: An Automatic Algorithm Configuration Framework | Frank Hutter, et al. | JAIR |PDF

Meta Learning

2008 | Cross-Disciplinary Perspectives on Meta-Learning for Algorithm Selection |PDF

Particle Swarm Optimization

2017 | Particle Swarm Optimization for Hyper-parameter Selection in Deep Neural Networks | Pablo Ribalta Lorenzo, et al. | GECCO |PDF
2008 | Particle Swarm Optimization for Parameter Determination and Feature Selection of Support Vector Machines | Shih-Wei Lin, et al. | Expert Systems with Applications |PDF

Random Search

2016 | Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization | Lisha Li, et al. | arXiv |PDF
2012 | Random Search for Hyper-Parameter Optimization | James Bergstra, Yoshua Bengio | JMLR |PDF
2011 | Algorithms for Hyper-parameter Optimization | James Bergstra, et al. | NIPS |PDF

Transfer Learning

2016 | Efficient Transfer Learning Method for Automatic Hyperparameter Tuning | Dani Yogatama, Gideon Mann | JMLR |PDF
2016 | Flexible Transfer Learning Framework for Bayesian Optimisation | Tinu Theckel Joy, et al. | PAKDD |PDF
2016 | Hyperparameter Optimization Machines | Martin Wistuba, et al. | DSAA |PDF
2013 | Collaborative Hyperparameter Tuning | R´emi Bardenet, et al. | ICML | PDF

Miscellaneous

2018 | Accelerating Neural Architecture Search using Performance Prediction | Bowen Baker, et al. | ICLR |PDF
2017 | Automatic Frankensteining: Creating Complex Ensembles Autonomously | Martin Wistuba, et al. | SIAM |PDF

Tutorials

Bayesian Optimization

2010 | A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning |PDF