Skip to content

[CMake] Add sanity tests for python file lists#15670

Merged
rmlarsen merged 3 commits into
tensorflow:masterfrom
Androbin:sanity
Jan 22, 2018
Merged

[CMake] Add sanity tests for python file lists#15670
rmlarsen merged 3 commits into
tensorflow:masterfrom
Androbin:sanity

Conversation

@Androbin

@Androbin Androbin commented Dec 27, 2017

Copy link
Copy Markdown
Contributor

Replaces #15166
See discussion in #15368
@gunan @martinwicke Friendly ping.

@tensorflow-jenkins

Copy link
Copy Markdown
Collaborator

Can one of the admins verify this patch?

@drpngx drpngx requested a review from martinwicke December 27, 2017 22:13
@drpngx drpngx added the awaiting review Pull request awaiting review label Dec 27, 2017
test.entries = []
test.whitelist = []

for line in lines:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I find this logic confusing and (maybe because I don't understand it) it looks brittle -- are the comments becoming meaningful?

@martinwicke

Copy link
Copy Markdown
Member
  • @mrry for CMAKE-fu to check whether this is testing what it should be.

@yifeif @gunan with the aim of making all_opensource_files redundant, I think this is great. But I'm not sure about the specifics.

@martinwicke martinwicke requested a review from mrry December 29, 2017 22:19
@gunan

gunan commented Dec 31, 2017

Copy link
Copy Markdown
Contributor

Why should we add the new syntax? Why not make the test slightly more flexible, and just comment out the lines we use the special syntax instead?

My concern is the syntax is specific to this, and will be alien to anyone adding new modules. So they will just "hack" the test instead of trying to write the special syntax.

@gunan

gunan commented Dec 31, 2017

Copy link
Copy Markdown
Contributor

Why not make the test slightly more flexible, and just comment out the lines we use the special syntax instead?

@gunan

gunan commented Dec 31, 2017

Copy link
Copy Markdown
Contributor

This can be as simple as "If some folder is left out from this list, we do not care about them" and "if something is commented, we know these should exist in our python packages, but there are bugs with these packages, and they will be added to the pip package as soon as possible.

@mrry

mrry commented Jan 3, 2018

Copy link
Copy Markdown
Contributor

The CMake code looks fine to me, but I'll defer to @martinwicke or @gunan about what we'd like to support in the build.

@martinwicke martinwicke left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See, e.g., line 292 of python_modules.txt. Will a plain comment work with the new logic? I suppose it's added to the whitelist, which is harmless?

I'm ok with this, although it's a tiny bit ugly. I would like this behavior to be documented somewhere though. The python_modules.txt would be ideal (if we can make that work, given comments are meaningful now), and the test should, if it fails, explains how one can fix it.

@Androbin

Copy link
Copy Markdown
Contributor Author
  • Only lines starting with tensorflow/ will be whitelisted
  • Behavior is documented in python_modules.txt and python_sanity_test.py
  • Descriptions of problem and solution are printed respectively

@martinwicke

Copy link
Copy Markdown
Member

Great. The only question left is runtime, but I don't expect problems for this.

@martinwicke martinwicke added awaiting testing (then merge) and removed awaiting review Pull request awaiting review labels Jan 22, 2018
@rmlarsen rmlarsen merged commit 510713a into tensorflow:master Jan 22, 2018
@Androbin Androbin deleted the sanity branch January 23, 2018 17:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants