Feature Request: Enter-PSSession: -CustomTransport argument

Paul Higinbotham (@PaulHigin)

Summary of the new feature/enhancement

Currently the ssh implementation is "hard coded" as whatever local command ssh (or ssh.exe in windows) comes back with.

The enter-pssession -hostname doesn't actually have any internal SSH logic, and relies on the executable, and as such it only expect stdin and stdout back from the executable and its relevant "powershell" subsystem.

As such, it would be straightforward to add a -CustomTransport argument that specifies the path to an executable, calls that executable, and transmits/receives the PSRP XML messages via stdio in the same way it does to SSH. This would make developing and implementing custom transports (e.g. over websockets, azure relay, etc.) much simpler, and wouldn't require those custom transports to be merged into the Powershell code base like current transport handlers would (since the class is internal and sealed)

I've never actually tried it before but you could look into NamedPipeConnectionInfo through the .NET functions. It has a constructor that takes in a pipe name which can be a named pipe that you've set up in another process to read/write with.

Part of the runspace creation with this type of connection is that it just connects to the pipe and starts the thread to read from it. This should be simpler to maintain than having a custom process that reads/writes to it's own stdout/stdin that the ssh transport expects as now it's just a single pipe to read/write from. In saying this I've not really played around with it too much but based on the docs and the code it should be possible today to do what you want.

I actually have a full implementation of that working
https://gist.github.com/JustinGrote/866536458bfa097cb18a4b82181e5f16

The issue is that I want users of my script to have an easy, built-in way to access powershell anywhere without needing a separate module, hence the request. They're used to enter-pssession, this would be an easy way for clients to connect.

Gist
Powershell Debug Anywhere Design
Powershell Debug Anywhere Design. GitHub Gist: instantly share code, notes, and snippets.

I actually have a full implementation of that working

Nice

They're used to enter-pssession, this would be an easy way for clients to connect.

That's a fair request, it sounds like you are way ahead of me, just thought I would share the NamedPipeConnectionInfo stuff in case you weren't aware of it :P

This is a very interesting idea. But one problem with sdtin/stdout/stderror is that it doesn't provide a very good user experience. stdError is often used by applications for non-error messages. And there is no good way to prompt the user for different kinds of authentication. I wonder if a more general C# class based plug-in model would be better, allowing implementers access to PSHost for any kind of authentication. The implementation could wrap an exe (like SSH.exe) if need be.

Paul Higinbotham (@PaulHigin)

Paul Higinbotham (@PaulHigin) all I'm doing is passing the PSRP XML messages back and forth as a bytestream, the authentication/etc. still happens at the higher level (or within the application). I have it working great right now but for the client to connect you currently have to start "my" client and then enter-pshostsession to the named pipe it creates on the local machine. That's not a great user experience vs. something like just enter-pssession -customtransport websocat -argumentlist @{port=18522;host=my.websocat.server}
https://gist.github.com/JustinGrote/866536458bfa097cb18a4b82181e5f16

This is just a way to make it easily implemented and backwards compatible to 5.1. I suppose a powershell module might be able to implement a transport handler but I don't know how the scoping works there, ideas?

Gist
Powershell Debug Anywhere Design
Powershell Debug Anywhere Design. GitHub Gist: instantly share code, notes, and snippets.

It sounds like both an abstraction for client (connection initiation) and target (endpoint creation/configuration/hosting) is needed. We are actually planning to do work on the latter (endpoint) in the next sprint. But I like the idea of looking at this from both ends. The fact that you already have need of this helps with motivation. I need to spend some time thinking more about it.

Sounds good Paul. Currently the "simplest" approach has been to use an SSH server in golang (no good ssh server implementations exist in C# that I can find or I would have kept it all-native, and I'm not in the mood to write an SSH server using SSH.NET as a base right now...) and wire the runspace pipe to the server and forward out the port. Then the client experience is very simple: Enter-PSSession -Hostname -Port

If I had more flexibility in the client transport layer abstraction, that would be huge, in order to support all kinds of services like Azure Relay, Localtunnel, ngrok, etc.

The server-side abstraction is not as important, since I can just attach to the named pipe and then create a "server" on whatever I want (example in the gist linked above)

EDIT: Also as to why I'm not using openssh for the server, it's difficult to find a configuraiton that works well in userspace, openssh generally expects to run as root and that's not ideal when trying to remote a GIthub Action, Azure Automation, or other "locked down" environment. Plus I can't count on it being there, and pulling down a 3MB tiny golang ssh server is easy and simple :)

Ideas are:

Unseal and make public the PSRemoting transport layer classes, so that additional transports can be inherited and implemented as C# powershell modules (assuming there aren't any scoping issues that interfere with this). Then Enter-PSSession could also look for "plugin" transport layers either via an enum parameter or some sort of autodiscovery mechanism (a special uri perhaps?)
Just allow the SSH.EXE portion to be abstracted to any executable (or scriptblock???) that accepts stdin/out. Easier than the above.

Paul Higinbotham (@PaulHigin)

So currently the least-friction path would be another parameterset to *-pssession, since all the work is pretty much already done via the SSH transport.

Example:
New-PSSession -CustomTransport -CustomTransportPath "my/custom/websocketserver.exe" -CustomTransportArgs @('-hostname','my.websocket.relay','-port','10443')

And it would just call that program, tie into stdio, and expect and send PSRP xml in the same way it does to ssh.exe today. That would enable a ton of new scenarios without requiring client-side Powershell updates, they could be implemented as Powershell Modules outside of the core code (e.g. a New-PSWebsocketSession cmdlet that just wraps new-pssession and supplies the customtransportpath information)

Paul Higinbotham (@PaulHigin) thoughts? Obviously a C#-level extensible transport handler may be nice but that may take a while to develop, whereas the surface area of the customtransportpath option above is very low and already covered well by the SSH remoting framework/test surface.

Why is the transport always external app but not API (like WSMan)?

Ilya (@iSazonov)

Ilya (@iSazonov) not quite sure your question, this would just enable an easy way to enable additional PSRP transports. I actually prefer that the PSRP is not encrypted so I can "bring my own" encryption at the transport level, that's not something that should be part of the PSRP messaging in-and-of-itself IMHO.

As to why the transport isn't an API, making the transport layer more generalized would be great but the SSH transport as currently implemented basically just calls "ssh.exe" and then ties into stdin/out to exchange the PSRP messages to the process (the pwsh process is started with -sshs which is basically the same as -s so that it sends PSRP messages via stdio).

This is all how it exists today

What I want is basically be able to replace "ssh.exe" which is hardcoded into the code with any generic handler. Today it is possible to do this by replacing ssh.exe with your own executable as outlined in this blog:
https://blog.devolutions.net/2020/07/powershell-remoting-over-ssh-without-ssh

That's what gave me the inspiration, and looked into the code to find that ssh.exe was hardcoded when it should be a user-specifiable variable (maybe I don't want to use openssh for example for my ssh transport)

As mentioned before, making the transport handler class inheritable and having it work within the scope of the powershell module is a viable option, but it's a lot more effort for the already-stretched-thin Powershell team, whereas this change is a very straightforward modification that already has a strong testing surface. I can submit a PR if you want.

The Devolutions Blog
PowerShell Remoting Over SSH, Without SSH!
A few months ago, PowerShell 7 came out, finally bridging the gap between Windows PowerShell and PowerShell Core. It is now possible to use the same PowerShell everywhere (Windows, macOS, Linux!) whil...

Justin Grote (@JustinGrote)

Justin Grote (@JustinGrote) I see your point in allowing SSH remoting cmdlets to take arbitrary 'exe' path to act as the transport process via stdin/out/error. But after all of the problems we had with SSH, I would hesitate to do this. I would much prefer to support an abstract class or interface implementation. I also want an endpoint design so that a transport solution can safely establish a remote session with configuration options. But haven't really thought about that deeply yet.

Ilya (@iSazonov) PSRP never provided encryption, except for transferring SecureString objects (which is now supported x-plat with .NET crypto APIs). It is up to the transport mechanism such as WinRM or SSH to provide over the wire encryption.

@PowerShell/powershell-committee discussed this, we agreed that it makes sense to have a pluggable model for remoting transports, but based on #13186 and generalized so that any potential remoting transport could be used and not just when calling out to native commands.

I've been playing around with NamedPipeConnectionInfo to see how that works and it seems like you can just specify a custom named pipe as the bidirectional traffic for PSRP. This allows you to create your own custom transport mechanism that PowerShell talks to through the named pipe and then your transport does whatever is needed to talk to the server.

I created a POC New-PSSessionLogger that demonstrates this in a way. It's actually being used to log the PSRP traffic to a file for later analysis but it shows how you can inject custom code to sit in between the client and server using what's available today in PowerShell. If all you are wanting is to create a named pipe target then you can ignore that spaghetti C# code and just do

# All incoming and outgoing PSRP traffic will be sent through this pipe
$connInfo = [Management.Automation.Runspaces.NamedPipeConnectionInfo]::new('My Pipe')
$rs = [RunspaceFactory]::CreateRunspace($connInfo, $Host, $null)
$rs.Open()

$cstr = [Management.Automation.Runspaces.PSSession].GetConstructor(
    'NonPublic, Instance', $null, [type[]]$rs.GetType(), $null)
$session = $cstr.Invoke(@($rs))  # This is the PSSession

Invoke-Command -Session $session -ScriptBlock { echo 'look at me now' }

While it is possible there are some downsides

You need to track the state of the named pipe separately
- Remove-PSSession won't close the pipe or any other resources PowerShell is managing
- Maybe we could extend PSSession to accept a custom dispose() ScriptMethod that is called with Remove-PSSession
It uses an internal constructor to make the PSSession
- This might be something the PowerShell team is willing to open up in the future
- The argument to the constructor is a public type that you can build with CreateRunspace so there's not much hackery going on there
You are limited to a single bidirectional named pipe
- While not a major issue, other transports (like SSH) uses stdout for output and stdin for input which is nice if integrating it directly into a single process
Invoke-Command can fan-out in parallel, i.e. -ComputerName host1, host2 but sessions must be created synchronously beforehand
- You still have the fanout behaviour of running the commands on each session in parallel, but you need to build and connect to the session synchronously beforehand
- I was hoping Invoke-Command would connect any PSSession's specified that are in the BeforeOpen state but it errors instead
- You could use ForEach-Object -Parallel to build and connect to the pipes in parallel but it's not as nice or as seemless
- Having a -NamedPipe parameter taking an array of strings which does the managing internally could work

Having a provider like mechanism for transport would be nice and opens up more possibilities but I feel like this could take a long time to be delivered. Potentially improving the named pipe experience will be an easier task and give a more public interface for custom transports with PSRP.

Jordan Borean (@jborean93)

Jordan Borean (@jborean93) I've demonstrated this with PSAnywhere but there are a ton of limitations as you describe especially around runspace debugging that being able to specify a proper custom transport handler would solve.
https://gist.github.com/JustinGrote/866536458bfa097cb18a4b82181e5f16

Gist
Powershell Debug Anywhere Design
Powershell Debug Anywhere Design. GitHub Gist: instantly share code, notes, and snippets.

Jordan Borean (@jborean93)

I have successfully implemented an alternative PSRemoting transport by replacing the "ssh.exe" executable that makes the bridge between standard input/output streams and a WebSocket URL in rendezvous mode. For reference, here is how it is used in my product, and here is the blog post detailing the proof-of-concept technique. I have taken this approach because I don't want to ship custom PowerShell builds modified to expose internal interfaces.

I think what Jordan Borean (@jborean93) suggests is very sound and on the right track. Let me add a few thoughts on how we could get alternative PSRemoting transports implemented in a very generic and extensible manner.

Subprocess standard input/output interface: this is exactly how PowerShell currently interfaces with OpenSSH. I was surprised to realize how PowerShell barely "relies" on OpenSSH for SSH remoting other than the fact that it is hardcoded to use it. Having completely replaced it in my product, I can attest to the fact that the only thing preventing extensibility using a subprocess + standard input/output streams limited to the following:

Ability to specify which executable to launch for new connections (other than ssh.exe)
Ability to specify command-line parameters to be passed to the subprocess (other than OpenSSH)

In other words, all that stands in the way is to register and pass pretty much the same kind of arguments one would pass to the Start-Process command, except we need to somehow feed them to New-PSSession / Enter-PSSession to override which executable will be launched, and what command-line parameters will be used. For my implementation, I implemented the parsing of the OpenSSH client command-line parameters, but I passed my "real" command-line (the one with my WebSocket URL to connect to) through an environment variable.

In-process C# input/output stream interface: this is closer to what Jordan Borean (@jborean93) suggests, except I would expand on it. The named pipe transport is the closest right now to a generic input/output stream interface, which uses at its core System.IO.Pipes.PipeStream. I think we could expose a generic named pipe interface, just like we could expose a generic subprocess interface, but the real deal would be to make it possible to implement a generic type derived from System.IO.Stream, a parent of PipeStream. Here is a good documentation page covering all the specialized System.IO.Stream classes to use as reference.

Here is my suggestion: either find an existing type derived from System.IO.Stream, or define a new one with extended functions meant to cover all the required functionality for PSRemoting. How about "PSRemotingStream" type? Just make sure it contains everything that would need to be implemented for a generic input/output stream used for PSRemoting and we have a deal.

Because in the end, all PSRemoting needs is a generic transport interface that can be implemented by anyone and easily injected in class like New-PSSession / Enter-PSSession. Since authentication is often done out-of-band (it's not baked into the PSRP protocol itself - the transport itself requires authentication to be opened, after which no second authentication occurs). This part appears to have always been handled either implicitly (the transport can be opened and authenticated automatically) or by explicit prompting at the time of opening the transport. In the case of the OpenSSH client, it is entirely done inside the child subprocess, PowerShell doesn't do a thing on its side. That password prompt you see is done by ssh.exe, not pwsh.exe.

I say let's just get generic PSRemoting transport interfaces done experimentally, and then move on from there. I don't see a reason why we couldn't take both approaches: subprocess like with ssh, and in-process with a type derived from System.IO.Stream. While a subprocess can be a pain, it has a major upside: you don't care what the subprocess is. In our case, it is written in Rust, and we didn't have to bother figuring out ways to make Rust play well with .NET. The in-process approach has the advantage of not requiring an external executable, but you have to write a bit of C# to make it work. I see advantages to both approaches so I would definitely not settle for one or the other, I'd just go for both.

Marc-André Moreau (@awakecoding)

Marc-André Moreau (@awakecoding) yeah definitely replacing the hardcoded 'ssh.exe' with a public constructor is a small non-breaking change and should be the initial experimental way to go about this, because all we need is for pwsh to send us xml messages back and forth.

I know enough C# to be dangerous so I may attempt a PR of this, I'll bring it up on the community call today.

Justin Grote (@JustinGrote)

Justin Grote (@JustinGrote) we just need two parameters, which we can "steal" from Start-Process. The names can be changed from the original ones, but this is just to get the idea:

-FilePath: the file path to the executable to launch instead of ssh.exe (either just the file name, or full path)
-ArgumentList: the command-line arguments to pass to the executable (not the OpenSSH CLI parameters)

Give me those two and I can get rid of my two ugly workarounds that I am using today in my product:
-FilePath: I ship a special "psremoting" directory with a hardlink or symlink called "ssh.exe" that points to "jetsocat.exe" (my real executable). I prepare the environment by prepending this directory to PATH before calling New-PSSession or Enter-PSSession.
-ArgumentList: my executable (jetsocat.exe) parses the OpenSSH command-line arguments but doesn't do much with it. I pass the real command-line arguments meant for jetsocat through a "JETSOCAT_ARGS" environment variable and parse that.

A subprocess PSRemoting interface doesn't have to go beyond that: launch this executable with these parameters, hook the standard input/output streams of that subprocess, and just use it for PSRP, exactly like SSH remoting works with OpenSSH.

For reference, here are my helper functions that modify the environment prior to calling the real New-PSSession / Enter-PSSession functions. I made wrapper functions called New-WaykPSSession / Enter-WaykPSSession that do it all at once.

Marc-André Moreau (@awakecoding)

Marc-André Moreau (@awakecoding) I can't promise the ArgumentList one but filepath you can just point to a batch file or whatnot and then call your program with those arguments.

This was implemented with the custom transport feature (more or less) in Powershell 7.3 so I am closing as completed.

Navigation Menu

Repository navigation

Feature Request: Enter-PSSession: -CustomTransport argument #13282

Summary of the new feature/enhancement

3 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Feature Request: Enter-PSSession: -CustomTransport argument #13282

Description

Summary of the new feature/enhancement

Activity

jborean93 commented on Jul 26, 2020

JustinGrote commented on Jul 26, 2020

jborean93 commented on Jul 27, 2020

PaulHigin commented on Jul 27, 2020

JustinGrote commented on Jul 27, 2020

PaulHigin commented on Jul 28, 2020

JustinGrote commented on Jul 28, 2020

JustinGrote commented on Jul 28, 2020

JustinGrote commented on Jul 29, 2020

iSazonov commented on Jul 29, 2020

iSazonov commented on Jul 29, 2020

JustinGrote commented on Jul 29, 2020

PaulHigin commented on Jul 29, 2020

3 remaining items

SteveL-MSFT commented on Jul 29, 2020

jborean93 commented on Dec 17, 2020

JustinGrote commented on Dec 17, 2020

awakecoding commented on Dec 17, 2020

JustinGrote commented on Dec 17, 2020

awakecoding commented on Dec 17, 2020

JustinGrote commented on Dec 17, 2020

JustinGrote commented on Nov 16, 2023

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions