Skip to content

Hook up kernel selection to the kernel finder (part 1)#11505

Merged
IanMatthewHuff merged 6 commits into
microsoft:masterfrom
IanMatthewHuff:dev/ianhu/rawKernelSelector
Apr 30, 2020
Merged

Hook up kernel selection to the kernel finder (part 1)#11505
IanMatthewHuff merged 6 commits into
microsoft:masterfrom
IanMatthewHuff:dev/ianhu/rawKernelSelector

Conversation

@IanMatthewHuff

Copy link
Copy Markdown
Member

This is the first part of making kernel selection independent from jupyter.

  • Pull request represents a single change (i.e. not fixing disparate/unrelated things in a single PR).
  • Title summarizes what is changing.
  • Has a news entry file (remember to thank yourself!).
  • Appropriate comments and documentation strings in the code.
  • Has sufficient logging.
  • Has telemetry for enhancements.
  • Unit tests & system/integration tests are added/updated.
  • Test plan is updated as appropriate.
  • package-lock.json has been regenerated by running npm install (if dependencies have changed).
  • The wiki is updated with any design decisions/details.

}

// Provider for searching for installed kernelspecs on disk without using jupyter to search
export class InstalledRawKernelSelectionListProvider implements IKernelSelectionListProvider {

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

KernelSelections was structured pretty nicely for making a new SelectionListProvider without having to change the higher level services. So kudos @DonJayamanne .

@sonarqubecloud

Copy link
Copy Markdown

Kudos, SonarCloud Quality Gate passed!

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities (and Security Hotspot 0 Security Hotspots to review)
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

Comment thread src/client/datascience/jupyter/kernels/kernelSelections.ts Outdated
Comment thread src/client/datascience/jupyter/kernels/kernelSelector.ts
Comment thread src/client/datascience/jupyter/kernels/kernelSwitcher.ts Outdated
}

// Search all our local file system locations for installed kernel specs and return them
public async listKernelSpecs(_cancelToken?: CancellationToken): Promise<IJupyterKernelSpec[]> {

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was the part 2 of this work. @DavidKutu was still working on some finder changes. So I didn't want to collide here hence just doing the routing up to here in this PR. Also there is a question of who should actually create the kernelspec. I believe that the kernelFinder is now caching just file paths, so since the kernelSelection code does it's own caching it might make sense to return back the paths here an have the json read on the selector side.

@@ -356,6 +360,7 @@ suite('Data Science - Kernel Switcher', () => {
anything(),

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The all anything() verify check are a bit painful since they both need to updated along with the when, and due to optional parameters they only break at test run time. Would be nice if we could combine the when and verify into one call definition, but not sure if that can be done.

@IanMatthewHuff IanMatthewHuff added the no-changelog No news entry required label Apr 29, 2020
// Lets pre-warm the list of local kernels.
this.selectionProvider.getKernelSelectionsForLocalSession(resource, sessionManager, cancelToken).ignoreErrors();
this.selectionProvider
.getKernelSelectionsForLocalSession(resource, 'jupyter', sessionManager, cancelToken)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

jupyter [](start = 59, length = 7)

Seems like it would be better to have the calling code 'know' that an IJupyterSessionManager is a 'jupyter' kernel instead of having that knowledge explicit to this function.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rchiodo the tricky bit here, which I didn't actually realize until I was tracing this out, is that there are jupyter only paths currently that don't pass down a session manager. JupyterExecution does it in connectToNotebookServer. Only the kernel switching paths have session managers, so we can't rely on seeing if we have one to know if we are in a jupyter or non jupyter scenario.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you're misunderstanding. This function should have an argument of 'type'. the caller of that function should make a determination of whether or not the kernels should come from jupyter or raw.

This function is only called from the jupyter code, so it would always be 'jupyter'

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, I think I did misunderstand. So instead of the explicit hardcode to 'jupyter' here this just takes a type and type gets pushed up another level to getKernelForLocalConnection. Then were getKernelForLocalConnection is called (jupyterExecution and jupyterServer) put the 'jupyter' hardcode there since those are jupyter specific classes? @rchiodo was that what you were saying?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is what you mean, but I'll add this on part 2 so I can keep it flowing.

Comment thread src/test/datascience/jupyter/kernels/kernelSelections.unit.test.ts Outdated

@rchiodo rchiodo left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

cancelToken?: CancellationToken
): Promise<IKernelSpecQuickPickItem[]> {
const items = await this.kernelFinder.listKernelSpecs(cancelToken);
return items

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think its best to sort based on some algorithm.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@DonJayamanne after the classes return their ipykernels they get sorted at line 287 regardless of the source. That seems the right spot so that both jupyter and raw kernel specs get the same sorting applied.

@DonJayamanne DonJayamanne left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Happy to discuss unknonw/liveShare.

@IanMatthewHuff

Copy link
Copy Markdown
Member Author

Happy to discuss unknonw/liveShare.

@DonJayamanne . I see your point. 'unknown' is never a great option. My only slight counter point would be that while live share is the one exception that I know about, there could be something else in the future. All that we know is that the given notebook does not have connection information. "liveshare" sounds like a specific type of connection when what we have it the absence of a connection.

How do you feel about 'raw' | 'jupyter' | 'noConnection'? It think that might express it better. Either you are connected via raw kernel connection, connected via jupyter connection, or not connected.

@DonJayamanne

Copy link
Copy Markdown

'noConnection'

👍

@codecov-io

Copy link
Copy Markdown

Codecov Report

Merging #11505 into master will decrease coverage by 0.28%.
The diff coverage is 84.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #11505      +/-   ##
==========================================
- Coverage   60.63%   60.35%   -0.29%     
==========================================
  Files         613      614       +1     
  Lines       33507    33553      +46     
  Branches     4727     4734       +7     
==========================================
- Hits        20318    20250      -68     
+ Misses      12739    12303     -436     
- Partials      450     1000     +550     
Impacted Files Coverage Δ
.../datascience/interactive-common/interactiveBase.ts 5.61% <ø> (-0.02%) ⬇️
src/client/datascience/jupyter/jupyterExecution.ts 47.79% <ø> (ø)
...client/datascience/kernel-launcher/kernelFinder.ts 59.64% <0.00%> (-1.62%) ⬇️
src/client/datascience/kernel-launcher/types.ts 23.07% <ø> (ø)
...ient/datascience/jupyter/kernels/kernelSelector.ts 76.37% <66.66%> (ø)
...nt/datascience/jupyter/kernels/kernelSelections.ts 90.52% <94.11%> (+0.52%) ⬆️
...ient/datascience/jupyter/kernels/kernelSwitcher.ts 86.84% <100.00%> (+1.31%) ⬆️
...cience/ipywidgets/cdnWidgetScriptSourceProvider.ts 15.51% <0.00%> (-68.97%) ⬇️
src/datascience-ui/react-common/arePathsSame.ts 75.00% <0.00%> (-12.50%) ⬇️
src/client/common/utils/platform.ts 64.70% <0.00%> (-11.77%) ⬇️
... and 155 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 04ab539...747de2e. Read the comment docs.

@IanMatthewHuff

Copy link
Copy Markdown
Member Author

windows tests were just a timeout

@IanMatthewHuff IanMatthewHuff merged commit f08993b into microsoft:master Apr 30, 2020
@IanMatthewHuff IanMatthewHuff deleted the dev/ianhu/rawKernelSelector branch April 30, 2020 13:20
@lock lock Bot locked as resolved and limited conversation to collaborators May 20, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

no-changelog No news entry required

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants