Skip to content

Changed logic used to determine validity of debugger executable#2708

Merged
DonJayamanne merged 2 commits into
microsoft:masterfrom
DonJayamanne:fixPySparkSubmit
Sep 28, 2018
Merged

Changed logic used to determine validity of debugger executable#2708
DonJayamanne merged 2 commits into
microsoft:masterfrom
DonJayamanne:fixPySparkSubmit

Conversation

@DonJayamanne
Copy link
Copy Markdown

For #2351

  • Pull request represents a single change (i.e. not fixing disparate/unrelated things in a single PR)
  • Title summarizes what is changing
  • [n/a] Has a news entry file (remember to thank yourself!)
  • Unit tests & system/integration tests are added/updated
  • [n/a] Any new/changed dependencies in package.json are pinned (e.g. "1.2.3", not "^1.2.3" for the specified version)
  • [n/a] package-lock.json has been regenerated by running npm install (if dependencies have changed)

@DonJayamanne DonJayamanne requested a review from d3r3kk September 27, 2018 15:34
@DonJayamanne
Copy link
Copy Markdown
Author

@d3r3kk
Please review, you reviewed the previous PR.
The logic had to change.
One can debug using either Python Executable or the PySpark-submit executable.
To validate the executable, we were using the command python -c print('1').
However that same mechanism doesn't work for pysparm-submit.
I thought using python one.py would work. However when you execute a python file with spark-submit it outputs other stuff as well.

So the new logic is as follows:

  • If either one of the following is true, then assume the executable exits:
  • Check if python -c print("1") returns the expected output
  • Check if the executable file exists

I've created a separate file with targeted tests for spark-submit so hopefully the logic doesn't change without considering the impact on debugging with spark-submit

Comment thread src/client/debugger/executableValidator.ts Outdated
Comment thread src/client/debugger/executableValidator.ts
@DonJayamanne
Copy link
Copy Markdown
Author

@d3r3kk Please re-review

@DonJayamanne DonJayamanne merged commit 3bceb5d into microsoft:master Sep 28, 2018
@DonJayamanne DonJayamanne deleted the fixPySparkSubmit branch October 2, 2018 22:45
@lock lock Bot locked as resolved and limited conversation to collaborators Jul 31, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants