Skip to content

Making some unit tests work #1000

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
PGijsbers merged 33 commits into from
Dec 24, 2020
Merged

Making some unit tests work #1000

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
e0af15e
Making some unit tests work
Neeratyoy Nov 10, 2020
14aa11d
Waiting for dataset to be processed
Neeratyoy Nov 16, 2020
31d48d8
Minor test collection fix
Neeratyoy Nov 16, 2020
431447c
Template to handle missing tasks
Neeratyoy Nov 30, 2020
cc3199e
Accounting for more missing tasks:
Neeratyoy Nov 30, 2020
8a29668
Fixing some more unit tests
Neeratyoy Nov 30, 2020
405e03c
Simplifying check_task_existence
Neeratyoy Nov 30, 2020
caf4f46
black changes
Neeratyoy Dec 4, 2020
b308e71
Minor formatting
Neeratyoy Dec 8, 2020
436a9fe
Handling task exists check
Neeratyoy Dec 9, 2020
ddd8b04
Testing edited check task func
Neeratyoy Dec 14, 2020
74ae622
Merge branch 'fix_unit_tests' of https://github.com/openml/openml-pyt…
Neeratyoy Dec 14, 2020
50ce90e
Flake fix
Neeratyoy Dec 15, 2020
56cd639
More retries on connection error
Neeratyoy Dec 16, 2020
8e8ea2e
Adding max_retries to config default
Neeratyoy Dec 17, 2020
d518beb
Update database retry unit test
Neeratyoy Dec 17, 2020
37d9f6b
Print to debug hash exception
Neeratyoy Dec 17, 2020
9bd4892
Fixing checksum unit test
Neeratyoy Dec 17, 2020
dc41b5d
Retry on _download_text_file
Neeratyoy Dec 18, 2020
396cb8d
Update datasets_tutorial.py
mfeurer Dec 21, 2020
8f380de
Update custom_flow_tutorial.py
mfeurer Dec 21, 2020
bc1745e
Update test_study_functions.py
mfeurer Dec 21, 2020
d95b5e6
Update test_dataset_functions.py
mfeurer Dec 21, 2020
91c6cf5
more retries, but also more time between retries
mfeurer Dec 21, 2020
a9430b3
allow for even more retries on get calls
mfeurer Dec 21, 2020
e9cfba8
Catching failed get task
Neeratyoy Dec 21, 2020
c13f6ce
Merge branch 'fix_unit_tests' of https://github.com/openml/openml-pyt…
Neeratyoy Dec 21, 2020
3d7abc2
undo stupid change
mfeurer Dec 21, 2020
94576b1
Merge branch 'fix_unit_tests' of https://github.com/openml/openml-pyt…
Neeratyoy Dec 21, 2020
b5e1242
fix one more test
mfeurer Dec 21, 2020
f5e4a3e
Refactoring md5 hash check inside _send_request
Neeratyoy Dec 21, 2020
07ce722
Fixing a fairly common unit test fail
Neeratyoy Dec 22, 2020
82e1b72
Reverting loose check on unit test
Neeratyoy Dec 23, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
Making some unit tests work
  • Loading branch information
@Neeratyoy
Neeratyoy committed Nov 10, 2020
commit e0af15e751b4c74d68940da2d41c7714694b2e94
21 changes: 19 additions & 2 deletions tests/test_datasets/test_dataset_functions.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@
DATASETS_CACHE_DIR_NAME,
)
from openml.datasets import fork_dataset, edit_dataset
from openml.tasks import TaskType, create_task


class TestOpenMLDataset(TestBase):
Expand Down Expand Up @@ -1350,7 +1351,7 @@ def test_data_edit_errors(self):
"original_data_url, default_target_attribute, row_id_attribute, "
"ignore_attribute or paper_url to edit.",
edit_dataset,
data_id=564,
data_id=64,
Comment thread
Neeratyoy marked this conversation as resolved.
Outdated
)
# Check server exception when unknown dataset is provided
self.assertRaisesRegex(
Expand All @@ -1360,15 +1361,31 @@ def test_data_edit_errors(self):
data_id=999999,
description="xor operation dataset",
)

# Need to own a dataset to be able to edit meta-data
# Will be creating a forked version of an existing dataset to allow the unit test user
# to edit meta-data of a dataset
did = fork_dataset(1)
TestBase._mark_entity_for_removal("dataset", did)
# Need to upload a task attached to this data to test edit failure
task = create_task(
task_type=TaskType.SUPERVISED_CLASSIFICATION,
dataset_id=did,
target_name="class",
estimation_procedure_id=1,
)
task = task.publish()
TestBase._mark_entity_for_removal("task", task.task_id)
# Check server exception when owner/admin edits critical fields of dataset with tasks
self.assertRaisesRegex(
OpenMLServerException,
"Critical features default_target_attribute, row_id_attribute and ignore_attribute "
"can only be edited for datasets without any tasks.",
edit_dataset,
data_id=223,
data_id=did,
default_target_attribute="y",
)

# Check server exception when a non-owner or non-admin tries to edit critical fields
self.assertRaisesRegex(
OpenMLServerException,
Expand Down
2 changes: 1 addition & 1 deletion tests/test_tasks/test_task_functions.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ def _check_task(self, task):
self.assertIn(task["status"], ["in_preparation", "active", "deactivated"])

def test_list_tasks_by_type(self):
num_curves_tasks = 200 # number is flexible, check server if fails
num_curves_tasks = 198 # number is flexible, check server if fails
ttid = TaskType.LEARNING_CURVE
tasks = openml.tasks.list_tasks(task_type=ttid)
self.assertGreaterEqual(len(tasks), num_curves_tasks)
Expand Down