Automl

AutoML

keywords for hyperparameter optimization

• Hyper Parameter (Tuning/Optimization/Search)
• Greedy - Stochastic

• Confusion Matrix

• Grid Search, Random Search(Manual Search/Brute-force)

• Bayesian Optimization (BO)

• Genetic Algorithm (Evloutionary Algorithm/Search)

• SMBO
• Sequential Model-based Algorithm Configuration(SMAC)

• Tree-structured Parzen Estimation (TPE)

• Simulated Annealing (SA)

• Particle Swarm Optimization (PSO)

• Reinforcement Learning
• one-armed Bandit
• Multi-armed(one-armed) Bandit
• EarlyStopping(EarlyStop Callback)
• Successive Halving (Algorithm/Pruning)
• Hyperband

---

[reference for hpo]

1. automl의 전반적인 설명 및 이론 정리 https://adioshun.gitbooks.io/deep_learning/content/Auto-ML/Intro.html
2. awsome automl ( paper & code for overall automl) https://github.com/guan-yuan/awesome-AutoML-and-Lightweight-Models
3. automl의 개념 및 manual search http://research.sualab.com/introduction/practice/2019/02/19/bayesian-optimization-overview-1.html
4. 베이지안 추정법 참고 https://sumniya.tistory.com/29
5. surrogate model & acquisition model 참고 논문 https://bi.snu.ac.kr/Publications/Conferences/Domestic/KCC2015_SWYoon.pdf
6. (statistic reference) 빈도론 vs 베이지안 접근방법 비교 정리 https://m.blog.naver.com/61stu01/221277477927
7. 유전알고리즘 참고 https://versusall.tistory.com/entry/%EC%9C%A0%EC%A0%84-%EC%95%8C%EA%B3%A0%EB%A6%AC%EC%A6%98-2-%EC%9C%A0%EC%A0%84-%EC%97%B0%EC%82%B0%EC%9E%90%EC%9D%98-%EC%A2%85%EB%A5%98
8. TPE 참고 https://towardsdatascience.com/a-conceptual-explanation-of-bayesian-model-based-hyperparameter-optimization-for-machine-learning-b8172278050f
9. TPE original paper https://papers.nips.cc/paper/4443-algorithms-for-hyper-parameter-optimization.pdf
10. SMBO, ROAR, SMAC 참고 논문 https://www.cs.ubc.ca/~hutter/papers/10-TR-SMAC.pdf
11. ROAR, SMAC 참고 자료 https://pdfs.semanticscholar.org/2171/35d666e8349ba6d7312a37bd1dd166c098ec.pdf
12. simulated anealing 참고 자료 https://zzsza.github.io/data/2019/04/01/simulated-annealing/
13. PSO참고 https://laonjena227.tistory.com/8
14. Multi-armed bandit 참고 http://sanghyukchun.github.io/96/ https://sumniya.tistory.com/9
15. successive halving & hyperband 논문 참고 https://arxiv.org/pdf/1603.06560.pdf
16. successive halving 논문 참고 https://arxiv.org/pdf/1502.07943.pdf
17. early stopping -> successive halving -> hyperband 참고 https://www.youtube.com/watch?v=eqokKei1aEo