Repository navigation
New in PowerShell 7.3: Expressions are evaluated but aren't substituted with the results #18668
Description
Prerequisites
- Write a descriptive title.
- Make sure you are able to repro it on the latest released version
- Search the existing issues.
- Refer to the FAQ.
- Refer to Differences between Windows PowerShell 5.1 and PowerShell.
Steps to reproduce
In certain contexts PowerShell 7.3 evaluates expressions (i.e. $(Do-Something)) but doesn't replace the expressions with the results of the evaluation.
Accepted behaviors are:
-
Evaluate an expression AND replace the expression with the result of the evaluation.
This is what happens usually and in expandable strings:A string enclosed in double quotation marks is an expandable string. Variable names preceded by a dollar sign (
$) are replaced with the variable's value before the string is passed to the command for processing.
Also, in a double-quoted string, expressions are evaluated, and the result is inserted in the string. -
Not to evaluate an expression at all (and not to replace the expression because there's nothing to replace it with).
This is what happens in verbatim strings:A string enclosed in single quotation marks is a verbatim string. The string is passed to the command exactly as you type it. No substitution is performed.
However evaluating an expressing and not performing substitution is unacceptable.
Evaluating expressions in contexts where they are not replaced makes as much sense - that is, none - as the second line in the following code actually destroying the computer:
PS C:\> "$(Get-Location)"
C:\
PS C:\> '$(Destroy-Computer)'
$(Destroy-Computer)
PS C:\>This is wrong, dangerous and unacceptable.
This issue is related to #18502 but distinct from it.
That issue deals with commas preventing substitution.
Regardless of whether not performing substitution is appropriate in contexts other than verbatim strings and whether arguments that contains commas to external commands is one of those context, in no case whatsoever should expressions that are not substituted be evaluated.
Since the verbal explanation in that issue wasn't clear enough to elicit the proper response I have also prepared a visual:
Expected behavior
PS C:\> #$PSNativeCommandArgumentPassing = "Legacy"
PS C:\> cmd /c echo foo,cd=$(Write-Host "completely insane"; "return")
completely insane
foo,cd=returnActual behavior
PS C:\> cmd /c echo foo,cd=$(Write-Host "completely insane"; "return")
completely insane
foo,cd=$(Write-Host"completelyinsane";"return")
PS C:\>Error details
No response
Environment data
PS C:\> $PSVersionTable
Name Value
---- -----
PSVersion 7.3.0
PSEdition Core
GitCommitId 7.3.0
OS Microsoft Windows 10.0.22621
Platform Win32NT
PSCompatibleVersions {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion 2.3
SerializationVersion 1.1.0.1
WSManStackVersion 3.0
PS C:\>Visuals
No response
Activity
237dmitry commentedon Nov 28, 2022
conioh commentedon Nov 28, 2022
Did you not see the part titled "environment data"?
kilasuit commentedon Nov 28, 2022
conioh your steps to reproduce could have been a little clearer
however 237dmitry was showing that when you hadn't commented this out
$PSNativeCommandArgumentPassing = "Legacy"prior to runningcmd /c echo foo,cd=$(Write-Host "completely insane"; "return")that you do indeed get the desired result as can be shown in this screenshot on my system237dmitry commentedon Nov 28, 2022
My Environment Data is the same.
kilasuit commentedon Nov 28, 2022
237dmitry - I thought it would be but to many a prompt with
$in it is often seen to be a unix like system.237dmitry commentedon Nov 28, 2022
What does it matter?
conioh commentedon Nov 28, 2022
Ryan Yates (@kilasuit), what's unclear? Please explain and I will attempt to improve the report.
So if you perform different actions than those I have described you get different results?
That's hardly surprising and I don't see even a bit of relevance.
There are lots of other ways to "get the desired result" if you think the result is a string printed to the console:
This too "get[s] the desired result", but only with a malicious interpretation of the desired result.
The example is an example. The issue is "expressions are evaluated but aren't substituted with the results", not "I am unable to send to the standard output the string
foo,cd=return".Finding a different way to print that string is irrelevant. The issue is that PowerShell evaluates an expression where it shouldn't.
mklement0 commentedon Nov 28, 2022
To provide some context:
To me, this issue is a duplicate of PowerShell 7.3 doesn't pass variable values / cmdlet results arguments to child processes when comma is present #18502 (which is also mentioned in the initial post).
That the behavior exhibited there (and elaborated on here) should never occur is a given, and doesn't need discussion as a general hypothetical.
By contrast, if there are other actual bugs where this behavior manifests, they should be called out specifically, in their own bug reports.
The behavior at hand is clearly a bug, and it was introduced in the context of introducing
$PSNativeCommandArgumentPassingin 7.3.0$PSNativeCommandArgumentPassingvalue.conioh commentedon Nov 28, 2022
237dmitry, no offence but I don't believe you.
A few minutes later:
I see that I was right.
On Windows the default value for
$PSNativeCommandArgumentPassingis"Windows". If you somehow changed it you're not using the same environment and if you did that intentionally that's dishonest.Obviously I did not specify in the bug report "don't run any other command before the one I provided" and I also did not specify "use and empty PowerShell profile file" because that's implied in any bug report.
Perhaps your
Profile.ps1file executesEnter-PSSessionand everything entered after that happens in a remote computer against PowerShell 7.0? There's no way to anticipate all of this and it is generally understood that this is not to be done.mklement0 commentedon Nov 28, 2022
In this case it actually should evaluate the expression - as it always has - and therefore, of course, also perform expansion (i.e. replace the subexpression with its (stringified) evaluation result in the argument); the latter part is now broken.
conioh commentedon Nov 28, 2022
Michael Klement (@mklement0), I believe this not to be a duplicate for the reason states at the bottom of the report:
These are two separate issues:
They are distinct and independent. It is possible to fix this issue by not evaluating the expression without fixing the other one and having commas cause the odd treatment of command-line arguments.
Edit:
I'm trying to be realistic here. The clever folks here perhaps really have decided that:
But even given that decision the evaluation is wrong.
jhoneill commentedon Nov 28, 2022
Ryan Yates (@kilasuit)
It took me multiple readings are true
Yes. When legacy is set "return" and not "$(Write-Host "completely insane"; "return")" is passed to the command all good.
When it is not set
$(Write-Host "completely insane"; "return")is sent. That is also expectedWhat is not expected is as well as sending the literal string, to the command, any evaluation still happens. In other words
cmd /c echo foo,cd=$(Write-Host "completely insane"; "return")first does what $() tells it to do and gets the value of what is inside the brackets. conioh Write-Host doesn't write to standard output, it bypasses std-out and goes to the console. So we do a write host then send some text back.
Now: if the value is set to legacy
returnis sent,but if the value is set to Windows
$(Write-Host "completely insane"; "return")is sent,Evaluating what's inside $() and sending the result makes sense.
Not evaluating what's inside and sending $(blah blah) also makes sense
Evaluating what's inside and the sending $(blah blah) is at best wildly counter intutitve, and as Michael Klement (@mklement0) has just said while I've been typing, looks like a bug.
mklement0 commentedon Nov 28, 2022
conioh
The issues are the same, because the symptom only arises in the context of passing arguments to external programs, and it is an unwanted side effect of how argument-passing to external programs has changed in 7.3.0 (
,is not a metacharacter when passing arguments to external programs, and should not cause any modification in behavior - an unquoted argument containing,should be treated like any other implicit expandable string).Again, if you know of specific other contexts where a similar symptom manifests, please create separate reports.
The comment by James Truher (@JamesWTruher) you link to (which is the one I also linked to and called "bewilderingly cavalier") should not be construed as a decision on how to handle your bug report - and in this particular case I certainly hope that the behavior will ultimately be recognized for the - serious - regression bug that it is.
In short:
No official action has been taken on PowerShell 7.3 doesn't pass variable values / cmdlet results arguments to child processes when comma is present #18502 as of this writing. It hasn't even been triaged yet.
As you state, even if - God forbid - the non-expansion aspect should be declared to be the (breaking) new behavior, there would be no justification whatsoever for evaluation taking place nonetheless, and at least that aspect would need fixing.
To me, there is no justification for the breaking change, which seems to be entirely accidental, and fixing this regression bug strikes me as the only sensible resolution.
Either way, resolution of PowerShell 7.3 doesn't pass variable values / cmdlet results arguments to child processes when comma is present #18502 should settle the issue.
mklement0 commentedon Nov 28, 2022
James O'Neill (@jhoneill)
No -
$PSNativeCommandArgumentPassingis unrelated to expansion behavior, so its value shouldn't matter.I think conioh is aware of that and deliberately used
Write-Hostto demonstrate that despite the non-expansion, evaluation still took place - which never makes sense and can be dangerous.conioh commentedon Nov 28, 2022
Michael Klement (@mklement0), the actionable thing is to immediately stop evaluating expressions in those comma-containing command-line arguments contexts while we consider whether commas should affect argument parsing at all and try to find out what JamesWTruher meant, what is the mysterious bug he referred to etc.
I agree with you that the answer to this should be "no, commas should not affect this", but while that issue it being investigated this issue should be immediately fixed.
kilasuit commentedon Nov 28, 2022
conioh - the team is predominantly US West coast based so only recently having got up and gotten online - so I hope that Steve Lee (@SteveL-MSFT) and team will look into this in the coming hours & hope that we will see a PR that can fix this soon enough
But please be realistic on timeframes as immediately just isn't possible here
237dmitry commentedon Nov 28, 2022
I think you misunderstand
Legacymode. It allows you to avoid escaping service symbols that could be interpreted by PowerShell as a part of its own syntax. But that doesn't preclude the use of variables. If you want to get the command line as it is, you must use the--%operator.Linux:
The last command via
--%outputs empty string because$stringis not bash variable.conioh commentedon Nov 28, 2022
Ryan Yates (@kilasuit), I didn't mean "immediately" to be understood as "within the hour" but more like "as soon as reasonably possible". PowerShell 7.3.1 is a good place for this fix. PowerShell 7.4 is bad.
jhoneill commentedon Nov 28, 2022
I think you've quoted the wrong half of what I said. If it is set to
legacythe expanded value is sent., if not the expansion still happens (because as you sayPSNativeCommandArgumentPassingdoesn't stop expansion happening) but the expanded result isn't sent. Which is hard to rationalize.It was a comment made about std output
mklement0 commentedon Nov 28, 2022
I'm not sure if there's still a disagreement, and maybe this is just about words, but to be clear:
Expansion - meaning replacing the source code of the subexpression in the original, raw argument with the result of its evaluation - is not happening in the case at hand - only evaluation is (with the (success-stream-output) result getting discarded).
And it is the lack of expansion that constitutes the bug.
As an aside: I think it's better to use the term success (output) stream, so as to distinguish it from stdout, which only applies to external programs.
My comment was meant to point out the intentional bypassing of the success-output stream with
Write-Host, as simple proof that evaluation happened.conioh commentedon Nov 28, 2022
Michael Klement (@mklement0), I mentioned "standard output" in passing in another comment, not in the root post and that's what James O'Neill (@jhoneill) referred to.
I thought it was immaterial to the matter at hand so I didn't reply to James O'Neill (@jhoneill)'s comment about it and just fixed that comment.
But you're both technically right that
Write-Hostwrites directly to the console (regardless of stream redirection) and that PowerShell has multiple output streams (success/error/debug/...) and not just stdout and stderr.I fixed that so we can move on.
Regarding the substantial issue here I think we're basically in agreement.
jhoneill commentedon Nov 29, 2022
Michael Klement (@mklement0) yes, reading this last night my brain was swapping "expand" (the string) and "evaluate" (the expression) freely.
The expression is being evaluated, (it runs) whether or not the string is expanded (text of expression replaced by its result as a string). I think we've all got that.
microsoft-github-policy-service commentedon Nov 15, 2023
This issue has not had any activity in 6 months, if this is a bug please try to reproduce on the latest version of PowerShell and reopen a new issue and reference this issue if this is still a blocker for you.
microsoft-github-policy-service commentedon Nov 15, 2023
This issue has not had any activity in 6 months, if this is a bug please try to reproduce on the latest version of PowerShell and reopen a new issue and reference this issue if this is still a blocker for you.
microsoft-github-policy-service commentedon Nov 15, 2023
This issue has not had any activity in 6 months, if this is a bug please try to reproduce on the latest version of PowerShell and reopen a new issue and reference this issue if this is still a blocker for you.
microsoft-github-policy-service commentedon Nov 22, 2023
This issue has been marked as "No Activity" as there has been no activity for 6 months. It has been closed for housekeeping purposes.