Add torch.profiler submodule telemetry#15830
Conversation
| } | ||
| } | ||
| if (s && TorchProfilerImportRegEx.test(s)) { | ||
| sendTelemetryEvent(EventName.TENSORBOARD_TORCH_PROFILER_IMPORT); |
There was a problem hiding this comment.
This seems unnecessary as we already have import telemetry? Why is tensorboard profiler special?
There was a problem hiding this comment.
Because our existing import telemetry only tracks top level modules, i.e. we know how many people are importing torch, but not the submodules within torch. This is a request from Rong and Jeffrey so they can measure the PyTorch profiler user base size.
There was a problem hiding this comment.
Suppose that we don't worry about PII like with the package hashing since it's just a single known package and module. Might be worth generalizing at some point, but we can worry about that if we have more submodules that we want to look for in the future. No need to generalize now.
There was a problem hiding this comment.
@karrtikr @karthiknadig do you have any concerns about modifying the existing import regex and changing the HASHED_PACKAGE_NAME telemetry event to send both the top-level module name as well as the specific submodules that the user imported?
This would let us reuse an existing event rather than using a separate regex and separate telemetry event, and would be more generalizable to future requests to track imports with submodule granularity. Are there any implications of doing this from the Python extension team's perspective (either PM or dev)?
There was a problem hiding this comment.
This might impact existing telemetry. I don't have any concerns from the extension perspective. @luabud Do you use this information in an way that might impact analysis in any way.
There was a problem hiding this comment.
@joyceerhl Sorry didn't mean to complicate things now. Was more of a question for the future if we want to do more of this. I'm fine with the one-off for this.
There was a problem hiding this comment.
Just to be clear, right now the regular telemetry will gather this:
import torch as pyt
but won't gather this
import torch.profiler as pyt
In fact on the second one, the import is ignored.
Joyce's suggestion would leave the original import in place, but then add a new hash for the 'torch.profiler' one.
There was a problem hiding this comment.
Elaborating on that, I'm proposing that for something like from torch import profiler or import torch.profiler, we'd have the following two attached properties on the existing telemetry event:
{
hashedName: string // Existing property representing the hashed top-level package name, e.g. `torch`
hashedFullyQualifiedName: string // Hash of `torch.profiler`
}
In importTracker.ts we would construct the fully qualified, '.'-delimited import name, hash that, and send it as the second property.
A limitation is that this won't be able to distinguish submodules from functions, e.g. for an import like from torch.utils.tensorboard import SummaryWriter the fully qualified name is torch.utils.tensorboard.SummaryWriter. The PMs would have to account for all possible submodule import paths as distinct hashes in their queries.
There was a problem hiding this comment.
The PMs would have to account for all possible submodule import paths as distinct hashes in their queries.
Only if they wanted to look at the hashedFullyQualifedName. Otherwise the current query would remain the same.
rchiodo
left a comment
There was a problem hiding this comment.
Given that this is the second special case for import telemetry, I'd rather we made it more general now, but if there's a rush, seems okay for now.
For #15825