Skip to content

store uint8 data in uint8 not float #892

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
amueller wants to merge 4 commits into from
Closed

store uint8 data in uint8 not float #892

Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
164173a
store uint8 data in uint8 not float
amueller Dec 3, 2019
3f0fac1
don't be strict on boundaries, mnist has constant features
amueller Dec 3, 2019
6dd7581
add try block in case of missing data, add test
amueller Dec 3, 2019
3225588
fix expected dtypes
amueller Dec 3, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
store uint8 data in uint8 not float
  • Loading branch information
@amueller
amueller committed Dec 3, 2019
commit 164173a9ee3c53442c5fe466e3e467c234b00124
9 changes: 9 additions & 0 deletions openml/datasets/dataset.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -388,6 +388,15 @@ def _parse_data_from_arff(
'boolean'):
col.append(self._unpack_categories(
X[column_name], categories_names[column_name]))
elif attribute_dtype[column_name] in ('floating',
'integer'):
X_col = X[column_name]
if X_col.min() == 0 and X_col.max() == 255:
X_col_uint = X_col.astype('uint8')
if (X_col == X_col_uint).all():
col.append(X_col_uint)
continue
col.append(X[column_name])
else:
col.append(X[column_name])
X = pd.concat(col, axis=1)
Expand Down