Skip to content

[Java] CWE-348: Using a client-supplied IP address in a security check#5631

Merged
smowton merged 31 commits into
github:mainfrom
haby0:UseOfLessTrustedSource
Apr 30, 2021
Merged

[Java] CWE-348: Using a client-supplied IP address in a security check#5631
smowton merged 31 commits into
github:mainfrom
haby0:UseOfLessTrustedSource

Conversation

@haby0

@haby0 haby0 commented Apr 8, 2021

Copy link
Copy Markdown
Contributor

If an application trusts an HTTP request header like X-Forwarded-For to accurately specify the remote IP address of the connecting client.

@haby0 haby0 requested a review from a team as a code owner April 8, 2021 09:15
@Target({ElementType.TYPE})
@Retention(RetentionPolicy.RUNTIME)
@Documented
@Component

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for reminding

Comment thread java/ql/src/experimental/Security/CWE/CWE-348/UseOfLessTrustedSource.qhelp Outdated
Comment thread java/ql/src/experimental/Security/CWE/CWE-348/UseOfLessTrustedSource.qhelp Outdated
/** A data flow sink of method return or log output or local print. */
class UseOfLessTrustedSink extends DataFlow::Node {
UseOfLessTrustedSink() {
exists(ReturnStmt rs | rs.getResult() = this.asExpr())

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we're treating returning as a sink we might as well flag any use of X-Forwarded-For except splitting and taking the rightmost value.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The result of return as a sink, I think the method is a util method, this usage is common.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, but you don't care about what is done with it subsequently? This is so general that there's almost no point checking how it is used at all.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If so, I think there is a problem as long as there is a place to call this method. If you consider what to do with the result of the client ip method, I think there will be a false negative rate. What do you think?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What uses do you actually consider dangerous, besides logging?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In addition to local output or log records, there are also user authentication based on client ip, database storage, etc., but their use is not unique, so I think it is impossible to standardize the definition.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we let the security-lab review it together?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

E.g:

  • The client ip detects whether there is login permission: UserServiceImpl
  • Limit the number of logins for the client ip, which is common in scenarios such as blasting and ticketing: CaptchaService
  • Insert the database as a record: LogDB
  • Save it in JavaBean and insert into the database: MenuController

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can, but as this stands I would have to warn them that the query will produce really strange results:

  • Directly used in logging call: positive
  • Directly used in a rate-limiter: negative
  • Directly returned: positive
  • Returned in a field: negative

A user of this query would be very confused, because their expectation is that the way the value is used is the important thing, not trivial aspects of dataflow like is it returned from a function or used in function-local scope.

I would recommend you find some e.g. rate-limiting utility that would constitute a dangerous way to use this identifier. Alternatively you can choose to send this to the security lab with the health warning attached.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, I agree to let them know the current situation, and I also want to listen to their suggestions, so as to implement a changed query.

Comment thread java/ql/src/experimental/Security/CWE/CWE-348/UseOfLessTrustedSourceLib.qll Outdated
.getASupertype*()
.hasQualifiedName("javax.servlet", "ServletRequest") and
this.getArgument(0).toString().toLowerCase() in [
"\"wl-proxy-client-ip\"", "\"proxy-client-ip\"", "\"http_client_ip\"",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps provide a link regarding how you chose this list of header values?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reference:

}

/** The first one in the method to get the ip value through `x-forwarded-for`. */
predicate xffIsFirstGet(Node node) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Explain what you're intending to do here? Should this suppress some warnings completely, or just prevent repeated warnings in the same function?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ensure that x-forwarded-for is the first to be obtained in the method of obtaining client ip.

.hasQualifiedName("javax.servlet", "ServletRequest") and
this.getArgument(0).toString().toLowerCase() in [
"\"wl-proxy-client-ip\"", "\"proxy-client-ip\"", "\"http_client_ip\"",
"\"http_x_forwarded_for\"", "\"x-real-ip\""

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Explain why you're using these headers for xffIsFirstGet but only X-Forwarded-For in the main query?

@haby0 haby0 Apr 8, 2021

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is related to the xffIsFirstGet predicate.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, but why are you checking for all these possibilities here, but only X-Forwarded-For in UseOfLessTrustedSource? What is special about that header that means you trust it less than X-Real-IP for instance?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, and X-Forwarded-For has a high usage rate.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are you only considering X-Forwarded-For a source of error? Isn't X-Real-IP exactly the same? I don't understand why you consider all 5 for the purposes of finding out if this is the first check in this function, but only one for the source.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

X-Forwarded-For is a superposition process when used, format: client, proxy1, proxy2. X-Real-IP is a process of coverage when in use.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, which is why you suppose that taking the right-most comma-separated piece of X-Forwarded-For might be safer than the left-most. If X-Real-IP doesn't get used that way then it's simply always dangerous to trust, right? But at the moment we treat it as always safe?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for reviewing my pr.

You are right. But I still find it weird, regarding the questions here and above, I would like to hear the opinions of security-lab. Hope you can understand.

haby0 and others added 3 commits April 8, 2021 19:17
…SourceLib.qll

Co-authored-by: Chris Smowton <smowton@github.com>
…Source.qhelp

Co-authored-by: Chris Smowton <smowton@github.com>
…Source.qhelp

Co-authored-by: Chris Smowton <smowton@github.com>
exists(ArrayAccess aa, MethodAccess ma | aa.getArray() = ma |
ma.getQualifier() = node.asExpr() and
ma.getMethod() instanceof SplitMethod and
not aa.getIndexExpr().toString() = "0"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
not aa.getIndexExpr().toString() = "0"
not aa.getIndexExpr().(CompileTimeConstantExpr).getIntValue() = 0

@smowton

smowton commented Apr 13, 2021

Copy link
Copy Markdown
Contributor

After discussing with colleages, I have decided I will reject this PR unless you can replace the highly-general ReturnValue sink with something more precise which you believe represents a hazardous use of an untrustworthy header. Otherwise this query is not much better than flagging any getHeader("X-Forwarded-For"), which can be achieved with much simpler tools than CodeQL.

@haby0

haby0 commented Apr 13, 2021

Copy link
Copy Markdown
Contributor Author

After discussing with colleages, I have decided I will reject this PR unless you can replace the highly-general ReturnValue sink with something more precise which you believe represents a hazardous use of an untrustworthy header. Otherwise this query is not much better than flagging any getHeader("X-Forwarded-For"), which can be achieved with much simpler tools than CodeQL.

I will try to find the accurate sink from github. In addition, I want to know if the source needs to be changed here?

@smowton

smowton commented Apr 14, 2021

Copy link
Copy Markdown
Contributor

I would strongly advise to use the same list of headers that specify a remote endpoint identifier (X-Forwarded-For, X-Real-IP, ...) as you used elsewhere in this PR. X-Forwarded-For only differs in that it is sometimes possible to comma-split it and so retrieve a safe(r) version of it.

@haby0

haby0 commented Apr 15, 2021

Copy link
Copy Markdown
Contributor Author

I would strongly advise to use the same list of headers that specify a remote endpoint identifier (X-Forwarded-For, X-Real-IP, ...) as you used elsewhere in this PR. X-Forwarded-For only differs in that it is sometimes possible to comma-split it and so retrieve a safe(r) version of it.

Thank you for your suggestion, I will reflect it in the next version.

@haby0

haby0 commented Apr 20, 2021

Copy link
Copy Markdown
Contributor Author

There are several changes in this revision:

  • Modify source, add remote endpoint identifier.
  • Modify sink. Including: log operation, if condition (excluding the null value judgment of client ip), sql operation (using QueryInjectionSink directly), local output.

lgtm result for reference.

@haby0 haby0 requested review from smowton and removed request for a team April 20, 2021 03:22
Comment thread java/ql/src/experimental/Security/CWE/CWE-348/UseOfLessTrustedSource.qhelp Outdated
Comment thread java/ql/src/experimental/Security/CWE/CWE-348/UseOfLessTrustedSource.qhelp Outdated
Comment thread java/ql/src/experimental/Security/CWE/CWE-348/UseOfLessTrustedSource.qhelp Outdated
Comment thread java/ql/src/experimental/Security/CWE/CWE-348/UseOfLessTrustedSource.qhelp Outdated
Comment thread java/ql/src/experimental/Security/CWE/CWE-348/UseOfLessTrustedSource.ql Outdated
Comment thread java/ql/src/experimental/Security/CWE/CWE-348/UseOfLessTrustedSource.ql Outdated
Comment thread java/ql/src/experimental/Security/CWE/CWE-348/UseOfLessTrustedSource.ql Outdated
* `if (remoteAddr == null || "".equals(remoteAddr)) {...` judging whether the client ip is a null value,
* it needs to be excluded
*/
private class IfConditionSink extends UseOfLessTrustedSink {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is still a very very broad sink. I would much rather we looked for specific patterns we think are dangerous (for example, the startsWith condition you describe in the comment than to just exclude 7 or 8 cases you think are probably FPs and flag every other if statement.

If we look positively for suspicious usage, we can also avoid insisting the expression occurs directly with an if statement -- instead the sink is simply any startsWith(...) call, or an .equals with a non-blank right-hand side, or similar.

Comment thread java/ql/src/experimental/Security/CWE/CWE-348/UseOfLessTrustedSourceLib.qll Outdated
haby0 and others added 5 commits April 20, 2021 19:31
…SourceLib.qll

Co-authored-by: Chris Smowton <smowton@github.com>
…Source.qhelp

Co-authored-by: Chris Smowton <smowton@github.com>
…Source.qhelp

Co-authored-by: Chris Smowton <smowton@github.com>
…Source.qhelp

Co-authored-by: Chris Smowton <smowton@github.com>
…Source.qhelp

Co-authored-by: Chris Smowton <smowton@github.com>
@haby0

haby0 commented Apr 28, 2021

Copy link
Copy Markdown
Contributor Author

It works for structr, it's just such an unprincipled discriminator that I don't really want to commit this to our codebase. It wouldn't make any sense to anyone using the query -- this one is regarded as bad, and this one as good, because you used a String or Object-typed variable somewhere?

The Object type restriction on Node is currently only for the structr project. I haven't found it in other projects for the time being.

@kevinbackhouse

Copy link
Copy Markdown
Contributor

My recommendation for improving the query is to restrict it so that it only flags cases like this one:

https://lgtm.com/projects/g/heathcare/heathcare/snapshot/a76e38b839a1fd02a12996c07bb70d33c1a38915/files/src/main/java/com/glaf/base/filter/PermissionFilter.java?sort=name&dir=ASC&mode=heatmap#x762b35b7412f98f6:1

In other words, only flag if-conditions that contain a hard-coded IP address: "192.168.0", "127.0.0", etc. Those are the places where an attacker has a higher chance of an authentication bypass by sending a spoofed IP address in the X-Forwarded-For field.

@haby0

haby0 commented Apr 29, 2021

Copy link
Copy Markdown
Contributor Author

My recommendation for improving the query is to restrict it so that it only flags cases like this one:

https://lgtm.com/projects/g/heathcare/heathcare/snapshot/a76e38b839a1fd02a12996c07bb70d33c1a38915/files/src/main/java/com/glaf/base/filter/PermissionFilter.java?sort=name&dir=ASC&mode=heatmap#x762b35b7412f98f6:1

In other words, only flag if-conditions that contain a hard-coded IP address: "192.168.0", "127.0.0", etc. Those are the places where an attacker has a higher chance of an authentication bypass by sending a spoofed IP address in the X-Forwarded-For field.

This will have a false negative.

@smowton What is your suggestion on this?

@smowton

smowton commented Apr 29, 2021

Copy link
Copy Markdown
Contributor

That looks pretty precise. Implement it and let's test it?

@haby0

haby0 commented Apr 29, 2021

Copy link
Copy Markdown
Contributor Author

That looks pretty precise. Implement it and let's test it?

@haby0 haby0 closed this Apr 29, 2021
@haby0 haby0 reopened this Apr 29, 2021
@haby0

haby0 commented Apr 29, 2021

Copy link
Copy Markdown
Contributor Author

That looks pretty precise. Implement it and let's test it?

ok

@haby0

haby0 commented Apr 29, 2021

Copy link
Copy Markdown
Contributor Author

That looks pretty precise. Implement it and let's test it?

Please review.

https://lgtm.com/query/4289318833195816514/

@smowton

smowton commented Apr 29, 2021

Copy link
Copy Markdown
Contributor

Made an all-projects run of this for the security lab to look at

@haby0

haby0 commented Apr 29, 2021

Copy link
Copy Markdown
Contributor Author

@kevinbackhouse

Copy link
Copy Markdown
Contributor

Thanks. The results are looking much better now. There is just one false positive pattern that seems to come up a lot:

"0:0:0:0:0:0:0:1".equals(ip) ? "127.0.0.1" : ip;

Examples:

So it might be worth adding special case for that pattern, but otherwise I am happy with this now.

@haby0

haby0 commented Apr 30, 2021

Copy link
Copy Markdown
Contributor Author

Thanks. The results are looking much better now. There is just one false positive pattern that seems to come up a lot:

"0:0:0:0:0:0:0:1".equals(ip) ? "127.0.0.1" : ip;

Examples:

So it might be worth adding special case for that pattern, but otherwise I am happy with this now.

I made a modification to this false positive, please review the lgtm result.

https://lgtm.com/query/2916758049910443110/
https://lgtm.com/query/2367742774803333177/

@smowton smowton left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rename "UseOfLessTrustedSource" -> "ClientSuppliedIpUsedInSecurityCheck" everywhere, including filenames and class names.

Comment thread java/ql/src/experimental/Security/CWE/CWE-348/UseOfLessTrustedSource.qhelp Outdated
Comment thread java/ql/src/experimental/Security/CWE/CWE-348/UseOfLessTrustedSourceLib.qll Outdated
ma.getAnArgument()
.(CompileTimeConstantExpr)
.getStringValue()
.regexpMatch("^((10\\.((1\\d{2})?|(2[0-4]\\d)?|(25[0-5])?|([1-9]\\d|[0-9])?)(\\.)?)|(192\\.168\\.)|172\\.(1[6789]|2[0-9]|3[01])\\.)((1\\d{2})?|(2[0-4]\\d)?|(25[0-5])?|([1-9]\\d|[0-9])?)(\\.)?((1\\d{2})?|(2[0-4]\\d)?|(25[0-5])?|([1-9]\\d|[0-9])?)$")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Factor the regex out into something like

string getIpAddressRegex() { result = ... }

@smowton smowton changed the title [Java] CWE-348: Use of less trusted source [Java] CWE-348: Using a client-supplied IP address in a security check Apr 30, 2021
haby0 and others added 4 commits April 30, 2021 16:54
…Source.qhelp

Co-authored-by: Chris Smowton <smowton@github.com>
…SourceLib.qll

Co-authored-by: Chris Smowton <smowton@github.com>
…Source.java

Co-authored-by: Chris Smowton <smowton@github.com>
@haby0 haby0 requested a review from smowton April 30, 2021 09:44
@haby0

haby0 commented Apr 30, 2021

Copy link
Copy Markdown
Contributor Author

@smowton smowton merged commit b2c0259 into github:main Apr 30, 2021
@haby0

haby0 commented May 5, 2021

Copy link
Copy Markdown
Contributor Author

@smowton @kevinbackhouse thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants