Skip to content

Feature/give possibility to not download the dataset qualities #1017

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
PGijsbers merged 11 commits into from
Feb 11, 2021
Merged

Feature/give possibility to not download the dataset qualities #1017

Changes from 1 commit
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
48653e2
update getdatasets function to give possibility to not download the d…
Jan 7, 2021
6cc1952
make download qualities defaulted to True
Jan 7, 2021
4ec995f
Using cahced version if exist
Jan 8, 2021
611d2bf
Updated the comments for get_dataset and get_datasets to include new …
Jan 12, 2021
d5d3a15
Update openml/datasets/functions.py
a-moadel Jan 13, 2021
08a3c41
Update openml/datasets/functions.py
a-moadel Jan 13, 2021
c3d9684
update get_dataset_qualities to have consistent output regardless the…
madel0093 Jan 17, 2021
84b9a8f
run pre-commit
madel0093 Jan 17, 2021
63b791d
fix parameter passing
Jan 26, 2021
55c7196
Merge branch 'feature/give_possibility_to_not_download_the_dataset_qu…
Jan 26, 2021
1b6467d
Updated the comments for get_dataset and get_datasets to include new …
Feb 10, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Using cahced version if exist
  • Loading branch information
Mohamed Adel committed Jan 8, 2021
commit 4ec995f891d803fa038d11ecbbbb3db0e5bd9290
19 changes: 8 additions & 11 deletions openml/datasets/functions.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1004,21 +1004,18 @@ def _get_dataset_qualities_file(did_cache_dir, dataset_id, download_qualities=Tr
str
Path of the cached qualities file
"""
if download_qualities:
# Dataset qualities are subject to change and must be fetched every time
qualities_file = os.path.join(did_cache_dir, "qualities.xml")
try:
with io.open(qualities_file, encoding="utf8") as fh:
qualities_xml = fh.read()
except (OSError, IOError):
# Dataset qualities are subject to change and must be fetched every time
qualities_file = os.path.join(did_cache_dir, "qualities.xml")
try:
with io.open(qualities_file, encoding="utf8") as fh:
qualities_xml = fh.read()
except (OSError, IOError):
if download_qualities:
url_extension = "data/qualities/{}".format(dataset_id)
qualities_xml = openml._api_calls._perform_api_call(url_extension, "get")

with io.open(qualities_file, "w", encoding="utf8") as fh:
fh.write(qualities_xml)
return qualities_file
else:
pass
return qualities_file


def _create_dataset_from_description(
Expand Down