{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.6"
    },
    "colab": {
      "name": "hyperparameter-tuning.ipynb",
      "provenance": [],
      "collapsed_sections": []
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eEgJmb-iw_q3",
        "colab_type": "text"
      },
      "source": [
        "# **Tuning Hyperparameters of Machine Learning Model**\n",
        "\n",
        "Chanin Nantasenamat\n",
        "\n",
        "<i>Data Professor YouTube channel, http://youtube.com/dataprofessor </i>\n",
        "\n",
        "In this Jupyter notebook, we will be tuning hyperparameters of a classification model built by random forest algorithm using scikit-learn package in Python."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pjiFK_EFw_rH",
        "colab_type": "text"
      },
      "source": [
        "## **1. Make synthetic dataset**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2EAYJIgEw_rx",
        "colab_type": "text"
      },
      "source": [
        "### **1.1. Generate the dataset**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "m_pxnncew_r2",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from sklearn.datasets import make_classification\n",
        "\n",
        "X, Y = make_classification(n_samples=200, n_classes=2, n_features=10, n_redundant=0, random_state=1)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3R29htzPw_sH",
        "colab_type": "text"
      },
      "source": [
        "### **1.2. Let's examine the data dimension**\n",
        "\n",
        "We can see that there are 100 rows (samples) and 5 columns (features) for the **X** variable and 100 rows and 1 column (class label) for the **Y** variable."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7Nwtk2C6w_sM",
        "colab_type": "code",
        "outputId": "0388b7ff-0133-4d52-ee1a-67cc88993989",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "X.shape, Y.shape"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "((200, 10), (200,))"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 2
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HaINcGrNw_s2",
        "colab_type": "text"
      },
      "source": [
        "## **2. Data split (80/20 ratio)**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZcBSucptw_tT",
        "colab_type": "text"
      },
      "source": [
        "### **2.1. Data split**\n",
        "\n",
        "A ratio of 80/20 is used for data splitting such that 80% goes to the training subset and 20% to the testing subset."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bWItU-xbw_tY",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from sklearn.model_selection import train_test_split\n",
        "\n",
        "X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size=0.2)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-58VLXQsw_tq",
        "colab_type": "text"
      },
      "source": [
        "### **2.2. Let's examine the data dimension**\n",
        "\n",
        "Here we see that the **training set** has 160 rows and 10 columns while there are 160 rows and 1 column for the **Y** variable."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "D7UxtTrJw_tw",
        "colab_type": "code",
        "outputId": "56fa6fa1-7d77-433a-c3f5-402bd013b329",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "X_train.shape, Y_train.shape"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "((160, 10), (160,))"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "x6l7J5PqBFnp",
        "colab_type": "text"
      },
      "source": [
        "The **testing set** has 40 rows and 10 columns for the **X** variable while there are 40 rows and 1 column for the **Y** variable."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Jvt4See2w_uE",
        "colab_type": "code",
        "outputId": "9b87742a-0408-484b-e0dc-75c9573a7292",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "X_test.shape, Y_test.shape"
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "((40, 10), (40,))"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IuzvrUrCDZyr",
        "colab_type": "text"
      },
      "source": [
        "# **3. Building a simple machine learning model using Random Forest**\n",
        "\n",
        "In the following blocks of codes, we will first start with building a random forest model. Finally, we will explore how to tune the hyperparameters (e.g. **n_estimators** and **max_features**) of the random forest algorithm. \n",
        "\n",
        "We first start by importing the necessary libraries and assigning the random forest classifier to the **rf** variable."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5JzOz1gU1OSu",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from sklearn.ensemble import RandomForestClassifier\n",
        "from sklearn.metrics import accuracy_score\n",
        "\n",
        "rf = RandomForestClassifier(max_features=5, n_estimators=100)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-5b-q8kVGsRL",
        "colab_type": "text"
      },
      "source": [
        "Now, we will be applying the random forest classifier to build a classification model using the **rf.fit()** function on the training data (e.g. **X_train** and **Y_train**)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "J5207y6A1Vax",
        "colab_type": "code",
        "outputId": "106f8a36-5243-4483-e514-562be5e048f9",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 153
        }
      },
      "source": [
        "rf.fit(X_train, Y_train)"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "RandomForestClassifier(bootstrap=True, ccp_alpha=0.0, class_weight=None,\n",
              "                       criterion='gini', max_depth=None, max_features=5,\n",
              "                       max_leaf_nodes=None, max_samples=None,\n",
              "                       min_impurity_decrease=0.0, min_impurity_split=None,\n",
              "                       min_samples_leaf=1, min_samples_split=2,\n",
              "                       min_weight_fraction_leaf=0.0, n_estimators=100,\n",
              "                       n_jobs=None, oob_score=False, random_state=None,\n",
              "                       verbose=0, warm_start=False)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3ppx4mdtHzDy",
        "colab_type": "text"
      },
      "source": [
        "The **rf.score()** function will be used to calculate the accuracy score of the RF model in predicting the *test data* (**X_test**)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "RZ7vmyEJ1iNV",
        "colab_type": "code",
        "outputId": "cf493dc7-d7e4-4872-d670-4894d23659fc",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "rf.score(X_test, Y_test)"
      ],
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.875"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6281UxefI1Jm",
        "colab_type": "text"
      },
      "source": [
        "The following 2 code cells also calculate the accuracy score of the RF model in predicting the test data (X_test) but performs it in 2 steps using **rf.predict()** and **accuracy_score()** functions."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "G19rlbJ41nd1",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "Y_pred = rf.predict(X_test)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UCCS1JORaut5",
        "colab_type": "code",
        "outputId": "e99e80dc-85b2-407e-8c4f-4ac05d840281",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "accuracy_score(Y_pred, Y_test)"
      ],
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.875"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "J6r_AY5bKfZu",
        "colab_type": "text"
      },
      "source": [
        "The advantage of using this latter approach is that you have access to the predicted data values."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "E4XPCkdxJccQ",
        "colab_type": "code",
        "outputId": "a14a38b3-2dbb-4096-cdc1-fbe6acae4a83",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 85
        }
      },
      "source": [
        "Y_pred, Y_test"
      ],
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(array([0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0, 1, 1, 0, 0,\n",
              "        0, 1, 0, 1, 1, 0, 0, 1, 0, 0, 1, 0, 1, 1, 0, 1, 0, 1]),\n",
              " array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0,\n",
              "        0, 1, 0, 1, 1, 0, 0, 1, 0, 0, 1, 0, 1, 1, 0, 0, 0, 1]))"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yfdK3TpVUH0t",
        "colab_type": "text"
      },
      "source": [
        "# **4. Hyperparameter Tuning**\n",
        "\n",
        "Now we will be performing the tuning of hyperparameters of Random forest model. The hyperparameters that we will tune includes **max_features** and the **n_estimators**.\n",
        "\n",
        "Note: Some codes modified from [scikit-learn](http://scikit-learn.org/stable/auto_examples/svm/plot_rbf_parameters.html)\n",
        "\n",
        "Firstly, we will import the necessary modules.\n",
        "\n",
        "The **GridSearchCV()** function from scikit-learn will be used to perform the hyperparameter tuning. Particularly, **GridSearchCV()** function can perform the typical functions of a classifier such as ***fit***, ***score*** and ***predict*** as well as ***predict_proba***, ***decision_function***, ***transform*** and ***inverse_transform***.\n",
        "\n",
        "Secondly, we define variables that are necessary input to the GridSearchCV() function.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "lMI5UTrUUgwR",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from sklearn.model_selection import GridSearchCV\n",
        "import numpy as np\n",
        "\n",
        "max_features_range = np.arange(1,6,1)\n",
        "n_estimators_range = np.arange(10,210,10)\n",
        "param_grid = dict(max_features=max_features_range, n_estimators=n_estimators_range)\n",
        "\n",
        "rf = RandomForestClassifier()\n",
        "\n",
        "grid = GridSearchCV(estimator=rf, param_grid=param_grid, cv=5)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "l6bOyGrgVb5H",
        "colab_type": "code",
        "outputId": "d977fa14-7055-4bc0-cf49-bbd17c4d0aa9",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 391
        }
      },
      "source": [
        "\n",
        "grid.fit(X_train, Y_train)"
      ],
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "GridSearchCV(cv=5, error_score=nan,\n",
              "             estimator=RandomForestClassifier(bootstrap=True, ccp_alpha=0.0,\n",
              "                                              class_weight=None,\n",
              "                                              criterion='gini', max_depth=None,\n",
              "                                              max_features='auto',\n",
              "                                              max_leaf_nodes=None,\n",
              "                                              max_samples=None,\n",
              "                                              min_impurity_decrease=0.0,\n",
              "                                              min_impurity_split=None,\n",
              "                                              min_samples_leaf=1,\n",
              "                                              min_samples_split=2,\n",
              "                                              min_weight_fraction_leaf=0.0,\n",
              "                                              n_estimators=100, n_jobs=None,\n",
              "                                              oob_score=False,\n",
              "                                              random_state=None, verbose=0,\n",
              "                                              warm_start=False),\n",
              "             iid='deprecated', n_jobs=None,\n",
              "             param_grid={'max_features': array([1, 2, 3, 4, 5]),\n",
              "                         'n_estimators': array([ 10,  20,  30,  40,  50,  60,  70,  80,  90, 100, 110, 120, 130,\n",
              "       140, 150, 160, 170, 180, 190, 200])},\n",
              "             pre_dispatch='2*n_jobs', refit=True, return_train_score=False,\n",
              "             scoring=None, verbose=0)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 13
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6B4hjETWWcXC",
        "colab_type": "code",
        "outputId": "951347b7-068b-4830-db5c-4be883cc31e4",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "print(\"The best parameters are %s with a score of %0.2f\"\n",
        "      % (grid.best_params_, grid.best_score_))"
      ],
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "The best parameters are {'max_features': 1, 'n_estimators': 120} with a score of 0.90\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "p98hoS9GCjl2",
        "colab_type": "text"
      },
      "source": [
        "# **5. Dataframe of Grid search parameters and their Accuracy scores**\n",
        "\n",
        "Finally, we will be exporting the grid search parameters and their resulting accuracy scores into a dataframe."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vMdtuunZdBt-",
        "colab_type": "code",
        "outputId": "36713988-8f16-4793-b7cd-62b96c34234a",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        }
      },
      "source": [
        "import pandas as pd\n",
        "\n",
        "grid_results = pd.concat([pd.DataFrame(grid.cv_results_[\"params\"]),pd.DataFrame(grid.cv_results_[\"mean_test_score\"], columns=[\"Accuracy\"])],axis=1)\n",
        "grid_results.head()"
      ],
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>max_features</th>\n",
              "      <th>n_estimators</th>\n",
              "      <th>Accuracy</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1</td>\n",
              "      <td>10</td>\n",
              "      <td>0.78750</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1</td>\n",
              "      <td>20</td>\n",
              "      <td>0.83750</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>1</td>\n",
              "      <td>30</td>\n",
              "      <td>0.86875</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>1</td>\n",
              "      <td>40</td>\n",
              "      <td>0.86250</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>1</td>\n",
              "      <td>50</td>\n",
              "      <td>0.85625</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   max_features  n_estimators  Accuracy\n",
              "0             1            10   0.78750\n",
              "1             1            20   0.83750\n",
              "2             1            30   0.86875\n",
              "3             1            40   0.86250\n",
              "4             1            50   0.85625"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 15
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "6s-R4ekaCC6-"
      },
      "source": [
        "# **6. Preparing data for making contour plots**\n",
        "\n",
        "Prior to making contour plots, we will have to reshape the data into a compatible format that will be recognized by the contour plot functions.\n",
        "\n",
        "Firstly, we will be using Pandas' **groupby()** function to segment the data into groups based on the 2 hyperparameters: **max_features** and **n_estimators**."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LbzWmXXQ09cT",
        "colab_type": "code",
        "outputId": "7359d147-1386-469e-a90c-67d7ad1f3645",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 450
        }
      },
      "source": [
        "grid_contour = grid_results.groupby(['max_features','n_estimators']).mean()\n",
        "grid_contour"
      ],
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th>Accuracy</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>max_features</th>\n",
              "      <th>n_estimators</th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th rowspan=\"5\" valign=\"top\">1</th>\n",
              "      <th>10</th>\n",
              "      <td>0.78750</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>20</th>\n",
              "      <td>0.83750</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>30</th>\n",
              "      <td>0.86875</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>40</th>\n",
              "      <td>0.86250</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>50</th>\n",
              "      <td>0.85625</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th rowspan=\"5\" valign=\"top\">5</th>\n",
              "      <th>160</th>\n",
              "      <td>0.88750</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>170</th>\n",
              "      <td>0.88750</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>180</th>\n",
              "      <td>0.86875</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>190</th>\n",
              "      <td>0.87500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>200</th>\n",
              "      <td>0.86875</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>100 rows × 1 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "                           Accuracy\n",
              "max_features n_estimators          \n",
              "1            10             0.78750\n",
              "             20             0.83750\n",
              "             30             0.86875\n",
              "             40             0.86250\n",
              "             50             0.85625\n",
              "...                             ...\n",
              "5            160            0.88750\n",
              "             170            0.88750\n",
              "             180            0.86875\n",
              "             190            0.87500\n",
              "             200            0.86875\n",
              "\n",
              "[100 rows x 1 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 16
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FxGvc5lvEH7U",
        "colab_type": "text"
      },
      "source": [
        "## **Pivoting the data**\n",
        "\n",
        "Data is reshaped by pivoting the data into an m by n matrix where rows and columns correspond to the **max_features** and **n_estimators**, respectively."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ynbIgQf_1IL8",
        "colab_type": "code",
        "outputId": "8130e761-21c1-46be-ca7a-ee6d1f331e6b",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 266
        }
      },
      "source": [
        "grid_reset = grid_contour.reset_index()\n",
        "grid_reset.columns = ['max_features', 'n_estimators', 'Accuracy']\n",
        "grid_pivot = grid_reset.pivot('max_features', 'n_estimators')\n",
        "grid_pivot"
      ],
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead tr th {\n",
              "        text-align: left;\n",
              "    }\n",
              "\n",
              "    .dataframe thead tr:last-of-type th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr>\n",
              "      <th></th>\n",
              "      <th colspan=\"20\" halign=\"left\">Accuracy</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>n_estimators</th>\n",
              "      <th>10</th>\n",
              "      <th>20</th>\n",
              "      <th>30</th>\n",
              "      <th>40</th>\n",
              "      <th>50</th>\n",
              "      <th>60</th>\n",
              "      <th>70</th>\n",
              "      <th>80</th>\n",
              "      <th>90</th>\n",
              "      <th>100</th>\n",
              "      <th>110</th>\n",
              "      <th>120</th>\n",
              "      <th>130</th>\n",
              "      <th>140</th>\n",
              "      <th>150</th>\n",
              "      <th>160</th>\n",
              "      <th>170</th>\n",
              "      <th>180</th>\n",
              "      <th>190</th>\n",
              "      <th>200</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>max_features</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>0.78750</td>\n",
              "      <td>0.83750</td>\n",
              "      <td>0.86875</td>\n",
              "      <td>0.86250</td>\n",
              "      <td>0.85625</td>\n",
              "      <td>0.84375</td>\n",
              "      <td>0.88125</td>\n",
              "      <td>0.87500</td>\n",
              "      <td>0.86250</td>\n",
              "      <td>0.86875</td>\n",
              "      <td>0.89375</td>\n",
              "      <td>0.90000</td>\n",
              "      <td>0.88125</td>\n",
              "      <td>0.86250</td>\n",
              "      <td>0.86875</td>\n",
              "      <td>0.87500</td>\n",
              "      <td>0.88125</td>\n",
              "      <td>0.88125</td>\n",
              "      <td>0.88125</td>\n",
              "      <td>0.86250</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>0.81250</td>\n",
              "      <td>0.86875</td>\n",
              "      <td>0.86875</td>\n",
              "      <td>0.88125</td>\n",
              "      <td>0.86875</td>\n",
              "      <td>0.88750</td>\n",
              "      <td>0.90000</td>\n",
              "      <td>0.86875</td>\n",
              "      <td>0.89375</td>\n",
              "      <td>0.88125</td>\n",
              "      <td>0.87500</td>\n",
              "      <td>0.89375</td>\n",
              "      <td>0.88750</td>\n",
              "      <td>0.90000</td>\n",
              "      <td>0.88125</td>\n",
              "      <td>0.88750</td>\n",
              "      <td>0.88750</td>\n",
              "      <td>0.88125</td>\n",
              "      <td>0.88750</td>\n",
              "      <td>0.88750</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>0.86875</td>\n",
              "      <td>0.85625</td>\n",
              "      <td>0.86250</td>\n",
              "      <td>0.88750</td>\n",
              "      <td>0.88750</td>\n",
              "      <td>0.86250</td>\n",
              "      <td>0.89375</td>\n",
              "      <td>0.87500</td>\n",
              "      <td>0.87500</td>\n",
              "      <td>0.88750</td>\n",
              "      <td>0.88125</td>\n",
              "      <td>0.86875</td>\n",
              "      <td>0.88125</td>\n",
              "      <td>0.88125</td>\n",
              "      <td>0.88750</td>\n",
              "      <td>0.88750</td>\n",
              "      <td>0.88125</td>\n",
              "      <td>0.88125</td>\n",
              "      <td>0.88125</td>\n",
              "      <td>0.88125</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>0.83750</td>\n",
              "      <td>0.85000</td>\n",
              "      <td>0.88750</td>\n",
              "      <td>0.89375</td>\n",
              "      <td>0.86875</td>\n",
              "      <td>0.89375</td>\n",
              "      <td>0.87500</td>\n",
              "      <td>0.88750</td>\n",
              "      <td>0.86250</td>\n",
              "      <td>0.88750</td>\n",
              "      <td>0.88750</td>\n",
              "      <td>0.86875</td>\n",
              "      <td>0.88750</td>\n",
              "      <td>0.86250</td>\n",
              "      <td>0.88125</td>\n",
              "      <td>0.89375</td>\n",
              "      <td>0.90000</td>\n",
              "      <td>0.87500</td>\n",
              "      <td>0.88125</td>\n",
              "      <td>0.87500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>0.83125</td>\n",
              "      <td>0.83125</td>\n",
              "      <td>0.85625</td>\n",
              "      <td>0.86250</td>\n",
              "      <td>0.87500</td>\n",
              "      <td>0.86875</td>\n",
              "      <td>0.87500</td>\n",
              "      <td>0.88125</td>\n",
              "      <td>0.88125</td>\n",
              "      <td>0.89375</td>\n",
              "      <td>0.88750</td>\n",
              "      <td>0.88125</td>\n",
              "      <td>0.86250</td>\n",
              "      <td>0.88125</td>\n",
              "      <td>0.87500</td>\n",
              "      <td>0.88750</td>\n",
              "      <td>0.88750</td>\n",
              "      <td>0.86875</td>\n",
              "      <td>0.87500</td>\n",
              "      <td>0.86875</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "             Accuracy                    ...                           \n",
              "n_estimators      10       20       30   ...      180      190      200\n",
              "max_features                             ...                           \n",
              "1             0.78750  0.83750  0.86875  ...  0.88125  0.88125  0.86250\n",
              "2             0.81250  0.86875  0.86875  ...  0.88125  0.88750  0.88750\n",
              "3             0.86875  0.85625  0.86250  ...  0.88125  0.88125  0.88125\n",
              "4             0.83750  0.85000  0.88750  ...  0.87500  0.88125  0.87500\n",
              "5             0.83125  0.83125  0.85625  ...  0.86875  0.87500  0.86875\n",
              "\n",
              "[5 rows x 20 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 17
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3qT_UoWpEeo4",
        "colab_type": "text"
      },
      "source": [
        "Finally, we assign the pivoted data into the respective ***x***, ***y*** and ***z*** variables."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gdO54DZK2NMc",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "x = grid_pivot.columns.levels[1].values\n",
        "y = grid_pivot.index.values\n",
        "z = grid_pivot.values"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "WFCz6WSjBy-j"
      },
      "source": [
        "# **2D Contour Plot**\n",
        "\n",
        "Now, comes the fun part, we will be visualizing the landscape of the 2 hyperparameters that we are tuning and their influence on the accuracy score."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fglg5T62z74a",
        "colab_type": "code",
        "outputId": "2cd41f66-0ca4-495d-c7ed-1afa5ad0841f",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 517
        }
      },
      "source": [
        "import plotly.graph_objects as go\n",
        "\n",
        "# X and Y axes labels\n",
        "layout = go.Layout(\n",
        "            xaxis=go.layout.XAxis(\n",
        "              title=go.layout.xaxis.Title(\n",
        "              text='n_estimators')\n",
        "             ),\n",
        "             yaxis=go.layout.YAxis(\n",
        "              title=go.layout.yaxis.Title(\n",
        "              text='max_features') \n",
        "            ) )\n",
        "\n",
        "fig = go.Figure(data = [go.Contour(z=z, x=x, y=y)], layout=layout )\n",
        "\n",
        "fig.update_layout(title='Hyperparameter tuning', autosize=False,\n",
        "                  width=500, height=500,\n",
        "                  margin=dict(l=65, r=50, b=65, t=90))\n",
        "\n",
        "fig.show()"
      ],
      "execution_count": 19,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<html>\n",
              "<head><meta charset=\"utf-8\" /></head>\n",
              "<body>\n",
              "    <div>\n",
              "            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax) {MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>\n",
              "                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
              "        <script src=\"https://cdn.plot.ly/plotly-latest.min.js\"></script>    \n",
              "            <div id=\"fe2d2a5a-a6ce-42fe-a232-616e6a1c1788\" class=\"plotly-graph-div\" style=\"height:500px; width:500px;\"></div>\n",
              "            <script type=\"text/javascript\">\n",
              "                \n",
              "                    window.PLOTLYENV=window.PLOTLYENV || {};\n",
              "                    \n",
              "                if (document.getElementById(\"fe2d2a5a-a6ce-42fe-a232-616e6a1c1788\")) {\n",
              "                    Plotly.newPlot(\n",
              "                        'fe2d2a5a-a6ce-42fe-a232-616e6a1c1788',\n",
              "                        [{\"type\": \"contour\", \"x\": [10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190, 200], \"y\": [1, 2, 3, 4, 5], \"z\": [[0.7875, 0.8375, 0.86875, 0.8625, 0.85625, 0.84375, 0.88125, 0.875, 0.8625, 0.86875, 0.89375, 0.9, 0.88125, 0.8625, 0.86875, 0.875, 0.88125, 0.88125, 0.88125, 0.8625], [0.8125, 0.86875, 0.86875, 0.88125, 0.86875, 0.8875, 0.9, 0.86875, 0.89375, 0.88125, 0.875, 0.89375, 0.8875, 0.9, 0.88125, 0.8875, 0.8875, 0.88125, 0.8875, 0.8875], [0.86875, 0.85625, 0.8625, 0.8875, 0.8875, 0.8625, 0.89375, 0.875, 0.875, 0.8875, 0.88125, 0.86875, 0.88125, 0.88125, 0.8875, 0.8875, 0.88125, 0.88125, 0.88125, 0.88125], [0.8375, 0.85, 0.8875, 0.89375, 0.86875, 0.89375, 0.875, 0.8875, 0.8625, 0.8875, 0.8875, 0.86875, 0.8875, 0.8625, 0.88125, 0.89375, 0.9, 0.875, 0.88125, 0.875], [0.83125, 0.83125, 0.85625, 0.8625, 0.875, 0.86875, 0.875, 0.88125, 0.88125, 0.89375, 0.8875, 0.88125, 0.8625, 0.88125, 0.875, 0.8875, 0.8875, 0.86875, 0.875, 0.86875]]}],\n",
              "                        {\"autosize\": false, \"height\": 500, \"margin\": {\"b\": 65, \"l\": 65, \"r\": 50, \"t\": 90}, \"template\": {\"data\": {\"bar\": [{\"error_x\": {\"color\": \"#2a3f5f\"}, \"error_y\": {\"color\": \"#2a3f5f\"}, \"marker\": {\"line\": {\"color\": \"#E5ECF6\", \"width\": 0.5}}, \"type\": \"bar\"}], \"barpolar\": [{\"marker\": {\"line\": {\"color\": \"#E5ECF6\", \"width\": 0.5}}, \"type\": \"barpolar\"}], \"carpet\": [{\"aaxis\": {\"endlinecolor\": \"#2a3f5f\", \"gridcolor\": \"white\", \"linecolor\": \"white\", \"minorgridcolor\": \"white\", \"startlinecolor\": \"#2a3f5f\"}, \"baxis\": {\"endlinecolor\": \"#2a3f5f\", \"gridcolor\": \"white\", \"linecolor\": \"white\", \"minorgridcolor\": \"white\", \"startlinecolor\": \"#2a3f5f\"}, \"type\": \"carpet\"}], \"choropleth\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"choropleth\"}], \"contour\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"contour\"}], \"contourcarpet\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"contourcarpet\"}], \"heatmap\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"heatmap\"}], \"heatmapgl\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"heatmapgl\"}], \"histogram\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"histogram\"}], \"histogram2d\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"histogram2d\"}], \"histogram2dcontour\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"histogram2dcontour\"}], \"mesh3d\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"mesh3d\"}], \"parcoords\": [{\"line\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"parcoords\"}], \"pie\": [{\"automargin\": true, \"type\": \"pie\"}], \"scatter\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatter\"}], \"scatter3d\": [{\"line\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatter3d\"}], \"scattercarpet\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattercarpet\"}], \"scattergeo\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattergeo\"}], \"scattergl\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattergl\"}], \"scattermapbox\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattermapbox\"}], \"scatterpolar\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterpolar\"}], \"scatterpolargl\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterpolargl\"}], \"scatterternary\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterternary\"}], \"surface\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"surface\"}], \"table\": [{\"cells\": {\"fill\": {\"color\": \"#EBF0F8\"}, \"line\": {\"color\": \"white\"}}, \"header\": {\"fill\": {\"color\": \"#C8D4E3\"}, \"line\": {\"color\": \"white\"}}, \"type\": \"table\"}]}, \"layout\": {\"annotationdefaults\": {\"arrowcolor\": \"#2a3f5f\", \"arrowhead\": 0, \"arrowwidth\": 1}, \"coloraxis\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"colorscale\": {\"diverging\": [[0, \"#8e0152\"], [0.1, \"#c51b7d\"], [0.2, \"#de77ae\"], [0.3, \"#f1b6da\"], [0.4, \"#fde0ef\"], [0.5, \"#f7f7f7\"], [0.6, \"#e6f5d0\"], [0.7, \"#b8e186\"], [0.8, \"#7fbc41\"], [0.9, \"#4d9221\"], [1, \"#276419\"]], \"sequential\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"sequentialminus\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]]}, \"colorway\": [\"#636efa\", \"#EF553B\", \"#00cc96\", \"#ab63fa\", \"#FFA15A\", \"#19d3f3\", \"#FF6692\", \"#B6E880\", \"#FF97FF\", \"#FECB52\"], \"font\": {\"color\": \"#2a3f5f\"}, \"geo\": {\"bgcolor\": \"white\", \"lakecolor\": \"white\", \"landcolor\": \"#E5ECF6\", \"showlakes\": true, \"showland\": true, \"subunitcolor\": \"white\"}, \"hoverlabel\": {\"align\": \"left\"}, \"hovermode\": \"closest\", \"mapbox\": {\"style\": \"light\"}, \"paper_bgcolor\": \"white\", \"plot_bgcolor\": \"#E5ECF6\", \"polar\": {\"angularaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"bgcolor\": \"#E5ECF6\", \"radialaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}}, \"scene\": {\"xaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}, \"yaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}, \"zaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}}, \"shapedefaults\": {\"line\": {\"color\": \"#2a3f5f\"}}, \"ternary\": {\"aaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"baxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"bgcolor\": \"#E5ECF6\", \"caxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}}, \"title\": {\"x\": 0.05}, \"xaxis\": {\"automargin\": true, \"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\", \"title\": {\"standoff\": 15}, \"zerolinecolor\": \"white\", \"zerolinewidth\": 2}, \"yaxis\": {\"automargin\": true, \"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\", \"title\": {\"standoff\": 15}, \"zerolinecolor\": \"white\", \"zerolinewidth\": 2}}}, \"title\": {\"text\": \"Hyperparameter tuning\"}, \"width\": 500, \"xaxis\": {\"title\": {\"text\": \"n_estimators\"}}, \"yaxis\": {\"title\": {\"text\": \"max_features\"}}},\n",
              "                        {\"responsive\": true}\n",
              "                    ).then(function(){\n",
              "                            \n",
              "var gd = document.getElementById('fe2d2a5a-a6ce-42fe-a232-616e6a1c1788');\n",
              "var x = new MutationObserver(function (mutations, observer) {{\n",
              "        var display = window.getComputedStyle(gd).display;\n",
              "        if (!display || display === 'none') {{\n",
              "            console.log([gd, 'removed!']);\n",
              "            Plotly.purge(gd);\n",
              "            observer.disconnect();\n",
              "        }}\n",
              "}});\n",
              "\n",
              "// Listen for the removal of the full notebook cells\n",
              "var notebookContainer = gd.closest('#notebook-container');\n",
              "if (notebookContainer) {{\n",
              "    x.observe(notebookContainer, {childList: true});\n",
              "}}\n",
              "\n",
              "// Listen for the clearing of the current output cell\n",
              "var outputEl = gd.closest('.output');\n",
              "if (outputEl) {{\n",
              "    x.observe(outputEl, {childList: true});\n",
              "}}\n",
              "\n",
              "                        })\n",
              "                };\n",
              "                \n",
              "            </script>\n",
              "        </div>\n",
              "</body>\n",
              "</html>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "w8IA_12Y82W5",
        "colab_type": "text"
      },
      "source": [
        "# **3D Surface Plot**\n",
        "\n",
        "Let's add an extra dimension to the plot and we now have a 3D surface plot. The cool thing about this plot is that you can rotate the graph."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "50yp2yb2vDK0",
        "colab_type": "code",
        "outputId": "b9bf1bc1-fd7b-4d8f-c014-bac299d74394",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 817
        }
      },
      "source": [
        "import plotly.graph_objects as go\n",
        "\n",
        "\n",
        "fig = go.Figure(data= [go.Surface(z=z, y=y, x=x)], layout=layout )\n",
        "fig.update_layout(title='Hyperparameter tuning',\n",
        "                  scene = dict(\n",
        "                    xaxis_title='n_estimators',\n",
        "                    yaxis_title='max_features',\n",
        "                    zaxis_title='Accuracy'),\n",
        "                  autosize=False,\n",
        "                  width=800, height=800,\n",
        "                  margin=dict(l=65, r=50, b=65, t=90))\n",
        "fig.show()"
      ],
      "execution_count": 20,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<html>\n",
              "<head><meta charset=\"utf-8\" /></head>\n",
              "<body>\n",
              "    <div>\n",
              "            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax) {MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>\n",
              "                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
              "        <script src=\"https://cdn.plot.ly/plotly-latest.min.js\"></script>    \n",
              "            <div id=\"c9f9e6c5-442e-4f7c-a138-b84304dc3c69\" class=\"plotly-graph-div\" style=\"height:800px; width:800px;\"></div>\n",
              "            <script type=\"text/javascript\">\n",
              "                \n",
              "                    window.PLOTLYENV=window.PLOTLYENV || {};\n",
              "                    \n",
              "                if (document.getElementById(\"c9f9e6c5-442e-4f7c-a138-b84304dc3c69\")) {\n",
              "                    Plotly.newPlot(\n",
              "                        'c9f9e6c5-442e-4f7c-a138-b84304dc3c69',\n",
              "                        [{\"type\": \"surface\", \"x\": [10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190, 200], \"y\": [1, 2, 3, 4, 5], \"z\": [[0.7875, 0.8375, 0.86875, 0.8625, 0.85625, 0.84375, 0.88125, 0.875, 0.8625, 0.86875, 0.89375, 0.9, 0.88125, 0.8625, 0.86875, 0.875, 0.88125, 0.88125, 0.88125, 0.8625], [0.8125, 0.86875, 0.86875, 0.88125, 0.86875, 0.8875, 0.9, 0.86875, 0.89375, 0.88125, 0.875, 0.89375, 0.8875, 0.9, 0.88125, 0.8875, 0.8875, 0.88125, 0.8875, 0.8875], [0.86875, 0.85625, 0.8625, 0.8875, 0.8875, 0.8625, 0.89375, 0.875, 0.875, 0.8875, 0.88125, 0.86875, 0.88125, 0.88125, 0.8875, 0.8875, 0.88125, 0.88125, 0.88125, 0.88125], [0.8375, 0.85, 0.8875, 0.89375, 0.86875, 0.89375, 0.875, 0.8875, 0.8625, 0.8875, 0.8875, 0.86875, 0.8875, 0.8625, 0.88125, 0.89375, 0.9, 0.875, 0.88125, 0.875], [0.83125, 0.83125, 0.85625, 0.8625, 0.875, 0.86875, 0.875, 0.88125, 0.88125, 0.89375, 0.8875, 0.88125, 0.8625, 0.88125, 0.875, 0.8875, 0.8875, 0.86875, 0.875, 0.86875]]}],\n",
              "                        {\"autosize\": false, \"height\": 800, \"margin\": {\"b\": 65, \"l\": 65, \"r\": 50, \"t\": 90}, \"scene\": {\"xaxis\": {\"title\": {\"text\": \"n_estimators\"}}, \"yaxis\": {\"title\": {\"text\": \"max_features\"}}, \"zaxis\": {\"title\": {\"text\": \"Accuracy\"}}}, \"template\": {\"data\": {\"bar\": [{\"error_x\": {\"color\": \"#2a3f5f\"}, \"error_y\": {\"color\": \"#2a3f5f\"}, \"marker\": {\"line\": {\"color\": \"#E5ECF6\", \"width\": 0.5}}, \"type\": \"bar\"}], \"barpolar\": [{\"marker\": {\"line\": {\"color\": \"#E5ECF6\", \"width\": 0.5}}, \"type\": \"barpolar\"}], \"carpet\": [{\"aaxis\": {\"endlinecolor\": \"#2a3f5f\", \"gridcolor\": \"white\", \"linecolor\": \"white\", \"minorgridcolor\": \"white\", \"startlinecolor\": \"#2a3f5f\"}, \"baxis\": {\"endlinecolor\": \"#2a3f5f\", \"gridcolor\": \"white\", \"linecolor\": \"white\", \"minorgridcolor\": \"white\", \"startlinecolor\": \"#2a3f5f\"}, \"type\": \"carpet\"}], \"choropleth\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"choropleth\"}], \"contour\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"contour\"}], \"contourcarpet\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"contourcarpet\"}], \"heatmap\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"heatmap\"}], \"heatmapgl\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"heatmapgl\"}], \"histogram\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"histogram\"}], \"histogram2d\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"histogram2d\"}], \"histogram2dcontour\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"histogram2dcontour\"}], \"mesh3d\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"mesh3d\"}], \"parcoords\": [{\"line\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"parcoords\"}], \"pie\": [{\"automargin\": true, \"type\": \"pie\"}], \"scatter\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatter\"}], \"scatter3d\": [{\"line\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatter3d\"}], \"scattercarpet\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattercarpet\"}], \"scattergeo\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattergeo\"}], \"scattergl\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattergl\"}], \"scattermapbox\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattermapbox\"}], \"scatterpolar\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterpolar\"}], \"scatterpolargl\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterpolargl\"}], \"scatterternary\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterternary\"}], \"surface\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"surface\"}], \"table\": [{\"cells\": {\"fill\": {\"color\": \"#EBF0F8\"}, \"line\": {\"color\": \"white\"}}, \"header\": {\"fill\": {\"color\": \"#C8D4E3\"}, \"line\": {\"color\": \"white\"}}, \"type\": \"table\"}]}, \"layout\": {\"annotationdefaults\": {\"arrowcolor\": \"#2a3f5f\", \"arrowhead\": 0, \"arrowwidth\": 1}, \"coloraxis\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"colorscale\": {\"diverging\": [[0, \"#8e0152\"], [0.1, \"#c51b7d\"], [0.2, \"#de77ae\"], [0.3, \"#f1b6da\"], [0.4, \"#fde0ef\"], [0.5, \"#f7f7f7\"], [0.6, \"#e6f5d0\"], [0.7, \"#b8e186\"], [0.8, \"#7fbc41\"], [0.9, \"#4d9221\"], [1, \"#276419\"]], \"sequential\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"sequentialminus\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]]}, \"colorway\": [\"#636efa\", \"#EF553B\", \"#00cc96\", \"#ab63fa\", \"#FFA15A\", \"#19d3f3\", \"#FF6692\", \"#B6E880\", \"#FF97FF\", \"#FECB52\"], \"font\": {\"color\": \"#2a3f5f\"}, \"geo\": {\"bgcolor\": \"white\", \"lakecolor\": \"white\", \"landcolor\": \"#E5ECF6\", \"showlakes\": true, \"showland\": true, \"subunitcolor\": \"white\"}, \"hoverlabel\": {\"align\": \"left\"}, \"hovermode\": \"closest\", \"mapbox\": {\"style\": \"light\"}, \"paper_bgcolor\": \"white\", \"plot_bgcolor\": \"#E5ECF6\", \"polar\": {\"angularaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"bgcolor\": \"#E5ECF6\", \"radialaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}}, \"scene\": {\"xaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}, \"yaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}, \"zaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}}, \"shapedefaults\": {\"line\": {\"color\": \"#2a3f5f\"}}, \"ternary\": {\"aaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"baxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"bgcolor\": \"#E5ECF6\", \"caxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}}, \"title\": {\"x\": 0.05}, \"xaxis\": {\"automargin\": true, \"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\", \"title\": {\"standoff\": 15}, \"zerolinecolor\": \"white\", \"zerolinewidth\": 2}, \"yaxis\": {\"automargin\": true, \"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\", \"title\": {\"standoff\": 15}, \"zerolinecolor\": \"white\", \"zerolinewidth\": 2}}}, \"title\": {\"text\": \"Hyperparameter tuning\"}, \"width\": 800, \"xaxis\": {\"title\": {\"text\": \"n_estimators\"}}, \"yaxis\": {\"title\": {\"text\": \"max_features\"}}},\n",
              "                        {\"responsive\": true}\n",
              "                    ).then(function(){\n",
              "                            \n",
              "var gd = document.getElementById('c9f9e6c5-442e-4f7c-a138-b84304dc3c69');\n",
              "var x = new MutationObserver(function (mutations, observer) {{\n",
              "        var display = window.getComputedStyle(gd).display;\n",
              "        if (!display || display === 'none') {{\n",
              "            console.log([gd, 'removed!']);\n",
              "            Plotly.purge(gd);\n",
              "            observer.disconnect();\n",
              "        }}\n",
              "}});\n",
              "\n",
              "// Listen for the removal of the full notebook cells\n",
              "var notebookContainer = gd.closest('#notebook-container');\n",
              "if (notebookContainer) {{\n",
              "    x.observe(notebookContainer, {childList: true});\n",
              "}}\n",
              "\n",
              "// Listen for the clearing of the current output cell\n",
              "var outputEl = gd.closest('.output');\n",
              "if (outputEl) {{\n",
              "    x.observe(outputEl, {childList: true});\n",
              "}}\n",
              "\n",
              "                        })\n",
              "                };\n",
              "                \n",
              "            </script>\n",
              "        </div>\n",
              "</body>\n",
              "</html>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yVnh22--8y9l",
        "colab_type": "text"
      },
      "source": [
        "---"
      ]
    }
  ]
}