# Automatically Tuning Blackbox Regression Models

## Description

Write a python function that automatically tunes blackbox regression models.

## Function interface

1 | def tune_blackbox_regression_model(algo, data, regularization_method, M, c, K, criterion): |

## Implementation

This function won’t build all algorithms from scratch since there are too many available ones good to use. This is more like a collection of them but with some customized features. Generally it could used as a handy tool for myself. Nothing big, but should be easy to use.

### Regression Algorithms

A category of regression models. We can actually get a sophisticated view of this from scikit-learn.org.

- Linear Models
- Kernel ridge regression
- Support Vector Machines
- Stochastic gradient descent
- Nearest neighbors
- Gaussian Processes
- Decision Trees
- Neural network models (supervised)

### Data

Data format should be uniform.

### Regularization Method

To avoid overfitting by adding extra information to it.

- Lasso
- Ridge
- Dropout
- Data Augmentation
- Noise Addition (Gaussian noise to input variables, or to activation, weights, gradients, outputs)
What’s the difference between adding noise to input and data augmentation?

- Robust (??????????????)

### Cross validation

Necessary

### Criterion to evaluate the model

- MSE (Mean Square Error)
- MAD (Mean Absolute Deviation)

## Development steps

### Criterion

[ ] MSE

[ ] MAD

### Linear models

Use different regularization methods

[ ] Lasso

[ ] Ridge

[ ] Noise Augmentation

### Nonlinear models

[ ] NN

[ ]