Skip to content

Commit f819812

Browse files
author
Github Actions
committed
Matthias Feurer: Merge pull request #1232 from openml/develop
1 parent ace66bb commit f819812

280 files changed

Lines changed: 9022 additions & 19378 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎main/.buildinfo‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
11
# Sphinx build info version 1
22
# This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done.
3-
config: 884c0728f1dea38019eaffe6df15f82c
3+
config: 977121ba2ad02efffcbb2ee6874bcd8d
44
tags: 645f666f9bcd5a90fca523b33c5a78b7

‎main/_downloads/006bd0fc770c8918e54252890f1e023e/study_tutorial.py‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -51,7 +51,9 @@
5151
# And we can use the evaluation listing functionality to learn more about
5252
# the evaluations available for the conducted runs:
5353
evaluations = openml.evaluations.list_evaluations(
54-
function="predictive_accuracy", output_format="dataframe", study=study.study_id,
54+
function="predictive_accuracy",
55+
output_format="dataframe",
56+
study=study.study_id,
5557
)
5658
print(evaluations.head())
5759

‎main/_downloads/27f49b0e36fba2fe65360adcf060e098/2015_neurips_feurer_example.ipynb‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -51,14 +51,14 @@
5151
"cell_type": "markdown",
5252
"metadata": {},
5353
"source": [
54-
"The dataset IDs could be used directly to load the dataset and split the data into a training set\nand a test set. However, to be reproducible, we will first obtain the respective tasks from\nOpenML, which define both the target feature and the train/test split.\n\n<div class=\"alert alert-info\"><h4>Note</h4><p>It is discouraged to work directly on datasets and only provide dataset IDs in a paper as\n this does not allow reproducibility (unclear splitting). Please do not use datasets but the\n respective tasks as basis for a paper and publish task IDS. This example is only given to\n showcase the use of OpenML-Python for a published paper and as a warning on how not to do it.\n Please check the `OpenML documentation of tasks <https://docs.openml.org/#tasks>`_ if you\n want to learn more about them.</p></div>\n\n"
54+
"The dataset IDs could be used directly to load the dataset and split the data into a training set\nand a test set. However, to be reproducible, we will first obtain the respective tasks from\nOpenML, which define both the target feature and the train/test split.\n\n<div class=\"alert alert-info\"><h4>Note</h4><p>It is discouraged to work directly on datasets and only provide dataset IDs in a paper as\n this does not allow reproducibility (unclear splitting). Please do not use datasets but the\n respective tasks as basis for a paper and publish task IDS. This example is only given to\n showcase the use of OpenML-Python for a published paper and as a warning on how not to do it.\n Please check the [OpenML documentation of tasks](https://docs.openml.org/#tasks) if you\n want to learn more about them.</p></div>\n\n"
5555
]
5656
},
5757
{
5858
"cell_type": "markdown",
5959
"metadata": {},
6060
"source": [
61-
"This lists both active and inactive tasks (because of ``status='all'``). Unfortunately,\nthis is necessary as some of the datasets contain issues found after the publication and became\ndeactivated, which also deactivated the tasks on them. More information on active or inactive\ndatasets can be found in the `online docs <https://docs.openml.org/#dataset-status>`_.\n\n"
61+
"This lists both active and inactive tasks (because of ``status='all'``). Unfortunately,\nthis is necessary as some of the datasets contain issues found after the publication and became\ndeactivated, which also deactivated the tasks on them. More information on active or inactive\ndatasets can be found in the [online docs](https://docs.openml.org/#dataset-status).\n\n"
6262
]
6363
},
6464
{
@@ -89,7 +89,7 @@
8989
"name": "python",
9090
"nbconvert_exporter": "python",
9191
"pygments_lexer": "ipython3",
92-
"version": "3.8.10"
92+
"version": "3.8.16"
9393
}
9494
},
9595
"nbformat": 4,

‎main/_downloads/296bc5731c400ca6e06e54ecb9b84b5c/configure_logging.ipynb‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@
2222
"cell_type": "markdown",
2323
"metadata": {},
2424
"source": [
25-
"Openml-python uses the `Python logging module <https://docs.python.org/3/library/logging.html>`_\nto provide users with log messages. Each log message is assigned a level of importance, see\nthe table in Python's logging tutorial\n`here <https://docs.python.org/3/howto/logging.html#when-to-use-logging>`_.\n\nBy default, openml-python will print log messages of level `WARNING` and above to console.\nAll log messages (including `DEBUG` and `INFO`) are also saved in a file, which can be\nfound in your cache directory (see also the\n`sphx_glr_examples_20_basic_introduction_tutorial.py`).\nThese file logs are automatically deleted if needed, and use at most 2MB of space.\n\nIt is possible to configure what log levels to send to console and file.\nWhen downloading a dataset from OpenML, a `DEBUG`-level message is written:\n\n"
25+
"Openml-python uses the [Python logging module](https://docs.python.org/3/library/logging.html)\nto provide users with log messages. Each log message is assigned a level of importance, see\nthe table in Python's logging tutorial\n[here](https://docs.python.org/3/howto/logging.html#when-to-use-logging).\n\nBy default, openml-python will print log messages of level `WARNING` and above to console.\nAll log messages (including `DEBUG` and `INFO`) are also saved in a file, which can be\nfound in your cache directory (see also the\n`sphx_glr_examples_20_basic_introduction_tutorial.py`).\nThese file logs are automatically deleted if needed, and use at most 2MB of space.\n\nIt is possible to configure what log levels to send to console and file.\nWhen downloading a dataset from OpenML, a `DEBUG`-level message is written:\n\n"
2626
]
2727
},
2828
{
@@ -53,7 +53,7 @@
5353
"name": "python",
5454
"nbconvert_exporter": "python",
5555
"pygments_lexer": "ipython3",
56-
"version": "3.8.10"
56+
"version": "3.8.16"
5757
}
5858
},
5959
"nbformat": 4,

‎main/_downloads/4076733b22158deda2a79e57d217b001/2018_kdd_rijn_example.ipynb‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -82,7 +82,7 @@
8282
"name": "python",
8383
"nbconvert_exporter": "python",
8484
"pygments_lexer": "ipython3",
85-
"version": "3.8.10"
85+
"version": "3.8.16"
8686
}
8787
},
8888
"nbformat": 4,

‎main/_downloads/42ecf9b9ca30a385452934aeb1a420d5/2018_neurips_perrone_example.ipynb‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -147,7 +147,7 @@
147147
"name": "python",
148148
"nbconvert_exporter": "python",
149149
"pygments_lexer": "ipython3",
150-
"version": "3.8.10"
150+
"version": "3.8.16"
151151
}
152152
},
153153
"nbformat": 4,

‎main/_downloads/4e36450a7c3d3fe8c9f4e71689c8e677/plot_svm_hyperparameters_tutorial.ipynb‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -136,7 +136,7 @@
136136
"name": "python",
137137
"nbconvert_exporter": "python",
138138
"pygments_lexer": "ipython3",
139-
"version": "3.8.10"
139+
"version": "3.8.16"
140140
}
141141
},
142142
"nbformat": 4,

‎main/_downloads/4ff7bd62b4b1b9012f431cd3c21d497d/study_tutorial.ipynb‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@
1515
"cell_type": "markdown",
1616
"metadata": {},
1717
"source": [
18-
"\n# Benchmark studies\nHow to list, download and upload benchmark studies.\nIn contrast to `benchmark suites <https://docs.openml.org/benchmark/#benchmarking-suites>`_ which\nhold a list of tasks, studies hold a list of runs. As runs contain all information on flows and\ntasks, all required information about a study can be retrieved.\n"
18+
"\n# Benchmark studies\nHow to list, download and upload benchmark studies.\nIn contrast to [benchmark suites](https://docs.openml.org/benchmark/#benchmarking-suites) which\nhold a list of tasks, studies hold a list of runs. As runs contain all information on flows and\ntasks, all required information about a study can be retrieved.\n"
1919
]
2020
},
2121
{
@@ -123,7 +123,7 @@
123123
},
124124
"outputs": [],
125125
"source": [
126-
"evaluations = openml.evaluations.list_evaluations(\n function=\"predictive_accuracy\", output_format=\"dataframe\", study=study.study_id,\n)\nprint(evaluations.head())"
126+
"evaluations = openml.evaluations.list_evaluations(\n function=\"predictive_accuracy\",\n output_format=\"dataframe\",\n study=study.study_id,\n)\nprint(evaluations.head())"
127127
]
128128
},
129129
{
@@ -190,7 +190,7 @@
190190
"name": "python",
191191
"nbconvert_exporter": "python",
192192
"pygments_lexer": "ipython3",
193-
"version": "3.8.10"
193+
"version": "3.8.16"
194194
}
195195
},
196196
"nbformat": 4,

‎main/_downloads/51e951fb65905058a563b0f066ec5771/2018_ida_strang_example.ipynb‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -118,7 +118,7 @@
118118
"name": "python",
119119
"nbconvert_exporter": "python",
120120
"pygments_lexer": "ipython3",
121-
"version": "3.8.10"
121+
"version": "3.8.16"
122122
}
123123
},
124124
"nbformat": 4,

‎main/_downloads/55fed401604e83ceb83e08221c96c779/fetch_evaluations_tutorial.ipynb‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -98,7 +98,7 @@
9898
},
9999
"outputs": [],
100100
"source": [
101-
"from matplotlib import pyplot as plt\n\n\ndef plot_cdf(values, metric=\"predictive_accuracy\"):\n max_val = max(values)\n n, bins, patches = plt.hist(values, density=True, histtype=\"step\", cumulative=True, linewidth=3)\n patches[0].set_xy(patches[0].get_xy()[:-1])\n plt.xlim(max(0, min(values) - 0.1), 1)\n plt.title(\"CDF\")\n plt.xlabel(metric)\n plt.ylabel(\"Likelihood\")\n plt.grid(b=True, which=\"major\", linestyle=\"-\")\n plt.minorticks_on()\n plt.grid(b=True, which=\"minor\", linestyle=\"--\")\n plt.axvline(max_val, linestyle=\"--\", color=\"gray\")\n plt.text(max_val, 0, \"%.3f\" % max_val, fontsize=9)\n plt.show()\n\n\nplot_cdf(evals.value, metric)\n# This CDF plot shows that for the given task, based on the results of the\n# runs uploaded, it is almost certain to achieve an accuracy above 52%, i.e.,\n# with non-zero probability. While the maximum accuracy seen till now is 96.5%."
101+
"from matplotlib import pyplot as plt\n\n\ndef plot_cdf(values, metric=\"predictive_accuracy\"):\n max_val = max(values)\n n, bins, patches = plt.hist(values, density=True, histtype=\"step\", cumulative=True, linewidth=3)\n patches[0].set_xy(patches[0].get_xy()[:-1])\n plt.xlim(max(0, min(values) - 0.1), 1)\n plt.title(\"CDF\")\n plt.xlabel(metric)\n plt.ylabel(\"Likelihood\")\n plt.grid(visible=True, which=\"major\", linestyle=\"-\")\n plt.minorticks_on()\n plt.grid(visible=True, which=\"minor\", linestyle=\"--\")\n plt.axvline(max_val, linestyle=\"--\", color=\"gray\")\n plt.text(max_val, 0, \"%.3f\" % max_val, fontsize=9)\n plt.show()\n\n\nplot_cdf(evals.value, metric)\n# This CDF plot shows that for the given task, based on the results of the\n# runs uploaded, it is almost certain to achieve an accuracy above 52%, i.e.,\n# with non-zero probability. While the maximum accuracy seen till now is 96.5%."
102102
]
103103
},
104104
{
@@ -154,7 +154,7 @@
154154
"name": "python",
155155
"nbconvert_exporter": "python",
156156
"pygments_lexer": "ipython3",
157-
"version": "3.8.10"
157+
"version": "3.8.16"
158158
}
159159
},
160160
"nbformat": 4,

0 commit comments

Comments
 (0)