Skip to content

Add lgamma#2417

Merged
gwhitney merged 23 commits into
josdejong:developfrom
yifanwww:gammaln
Apr 12, 2022
Merged

Add lgamma#2417
gwhitney merged 23 commits into
josdejong:developfrom
yifanwww:gammaln

Conversation

@yifanwww
Copy link
Copy Markdown
Contributor

@yifanwww yifanwww commented Feb 11, 2022

I noticed that there was already a PR #320 for gammaln, but it's too old. Since mathjs has refactored itself several times, the code in that PR cannot be merged any more. And it seems that @nfoti is rarely active on Github over the past few years.

So I created a new PR to support gammaln.

This implementation is the same as that in that PR, using Lanczos method to compute the logarithm of the gamma function since log(gamma(x)) can result in numerical problems.

I don't know how gammaln works when it receives Complex or BigNumber, so I only write the implementation for numbers. If an array or matrix only contains numbers, gammaln can also work.

Compare to the results calculated from https://keisan.casio.com/exec/system/1180573442, the precision of this implementation is high enough.

Comment thread src/plain/number/probability.js Outdated
@gwhitney
Copy link
Copy Markdown
Collaborator

gwhitney commented Feb 22, 2022

Thank you very much for reviving an old PR, yifanwww! It will be great to get this in. Some feedback/questions before we review this PR in more detail:

  1. The prior PR notwithstanding, since this function is the natural logarithm of the gamma function of its argument, and most commonly in math compositions are written in that same order left to right, this needs to be called either lngamma or lgamma (as it is in all other 'CAS's). Probably lgamma is better here since the mathjs notation for natural log is log, not ln, so lgamma looks more like log gamma. Thanks for making that change.

  2. Since lgamma is naturally defined on the complex plane (and in fact, one of the benefits of it is that it has a simpler branch-cut structure than it would normally inherit from directly taking ln of gamma of a complex number), and since gamma and ln are both already defined on complex numbers, this function will naturally be expected to operate on complex numbers. Therefore, we need at least to collaborate on providing an implementation for complex numbers. Here are some options: (A) I believe that the Lanczos method you are using is actually valid for all complex arguments with real part > 0.5, so you can follow the template of the complex version of the regular gamma function, but with your Lanczos method for log Gamma, using the reflection principle to ensure the condition on the real part and then just following the same steps as you currently use for real x. But that should then definitely be verified for sufficient accuracy against a reference implementation of log Gamma for a variety of complex numbers! (B) If that doesn't seem to work properly, next I would try the Stirling series for log Gamma shown in the first answer in https://math.stackexchange.com/questions/1338753/how-do-i-calculate-values-for-gamma-function-with-complex-arguments. You shouldn't need any additional terms to get sufficient accuracy as long as you use the recurrence log Gamma(z) = log Gamma(z+1) - log z, possibly several times in the proper direction, to ensure that |re(z)| >= 6. (C) a very extensive survey of calculating log Gamma and other gamma-related functions is given by https://hal.inria.fr/hal-03346642/document -- but that may be much deeper than you want or need to dive in. If you were interested in adding this function for BigNumbers, there would likely be little choice other than to understand this paper or something along the same lines. (If you don't implement lgamma for BigNumbers, please make sure that trying to compute it on one produces an intelligible error message, as currently happens for Gamma of all non-integer BigNumbers.)

  3. The LaTeX for this function is incorrect, there is no LaTeX command \Gammaln. So the beginning of the LaTeX translation of this function should be changed to \\ln\\Gamma (and then pick up with the \\left etc.)

Thanks for making these changes and then we will go through the PR in detail. If you need further assistance with the calculation for complex inputs, please just let us know.

Finally, I note that the prior PR had a stub for computing the log of the multivariate Gamma function. Did you have any interest in including that as well? If not, there is not a particular need to incorporate it into this PR -- I just thought I would bring that up in case it is a function that's of interest to you, since there is some code for it in the old PR.

@gwhitney
Copy link
Copy Markdown
Collaborator

Excellent changes, @yifanwww -- thank you so much for the updates. The last major ingredient needed is a decent implementation for complex numbers. Please let us know if you are able to take a stab at that, yifanwww (which would likely be quicker), or if you're hoping that a maintainer can supply a complex implementation (will probably take a bit longer). Thanks!

@yifanwww
Copy link
Copy Markdown
Contributor Author

Well, I'd like to try to provide an implementation for complex numbers, but I don't think that would be quicker because I'm not very good at math and not familiar with the mathematical calculations in computer languages. Anyway, I want to challenge it.

I'll leave comments if I need further help for complex implementation.

For the multivariate Gamma function, I don't want to incorporate it into this PR, since in this PR I only want to supply lgamma. But if I have time, I think I may submit a new PR to supply it. So what's the best name for the multivariate Gamma function under mathjs naming convention?

@gwhitney
Copy link
Copy Markdown
Collaborator

Well, I'd like to try to provide an implementation for complex numbers, but I don't think that would be quicker because I'm not very good at math and not familiar with the mathematical calculations in computer languages. Anyway, I want to challenge it.

That's great. We'll look forward to your efforts and hold off on further action on this PR until you make another commit.

For the multivariate Gamma function, I don't want to incorporate it into this PR, since in this PR I only want to supply lgamma.

That's perfectly reasonable.

But if I have time, I think I may submit a new PR to supply it. So what's the best name for the multivariate Gamma function under mathjs naming convention?

I think mvgamma would be a very typical name for this, and/or lmvgamma for the log of the multivariate gamma.

Thanks for all of your efforts.

@gwhitney
Copy link
Copy Markdown
Collaborator

@yifanwww are you still planning to try to provide an implementation for complex numbers? @kmdrGroch given your digamma submission, any chance you'd be interested in lending a hand?

@yifanwww yifanwww changed the title Add gammaln Add lgamma Mar 18, 2022
Conflicts:
- src/expression/embeddedDocs/embeddedDocs.js
@yifanwww
Copy link
Copy Markdown
Contributor Author

@gwhitney yeah... I'm still trying, but cannot pass all the tests.

@josdejong
Copy link
Copy Markdown
Owner

@yifanwww I see there is only one test failing, it related to testing NaN === NaN, which is not reliable. You can fix it by testing assert(isNaN(...)) instead.

@yifanwww
Copy link
Copy Markdown
Contributor Author

@josdejong ah not that one, I fixed it but haven't pushed yet.

I'm trying to write an implementation for complex numbers, but cannot pass all the tests. Some results are very different from the reference results.

@josdejong
Copy link
Copy Markdown
Owner

I'm trying to write an implementation for complex numbers, but cannot pass all the tests. Some results are very different from the reference results.

Ah ok, so that needs some careful examination whether the old or the new results are wrong. I regularly use https://www.wolframalpha.com/ for example to double check what the right result should be.

@yifanwww
Copy link
Copy Markdown
Contributor Author

yifanwww commented Mar 27, 2022

At my first try, I chose Lanczos method to implement log gamma for complex numbers, but somehow it didn't give the correct results. So I changed the solution, now the implementation is based on Stirling series and it seems to work fine.

But there is a problem in complex.js, new Complex(-9.87654321e50, 1.23456789e70).log() produces a wrong result { re: NaN, im: 1.5707963267948966 }, causing the unit test to fail.

If we try it in Python

>>> import cmath
>>> cmath.log(-9.87654321e50 + 1.23456789e70j) 
(161.39167753179885+1.5707963267948966j)

Do we have any alternate utilities to get the logarithm of complex numbers to work correctly?

@gwhitney
Copy link
Copy Markdown
Collaborator

Yes, you've uncovered something very wrong with the mathjs implementation of complex logarithm. I will file an issue and mention that this PR is blocked by it. Hopefully then someone involved can track down the problem and fix it and thereby get this great PR back on track -- thanks for all of your work on this!

@gwhitney
Copy link
Copy Markdown
Collaborator

The difficulty turns out to be in the complex library mathjs is using; I have filed upstream issue rawify/Complex.js#40. Hopefully Robert Eisele will be able to be responsive.

@gwhitney
Copy link
Copy Markdown
Collaborator

gwhitney commented Mar 27, 2022

OK, if you want to experiment you can try this on top of branch fix/complex_log (or just wait until @josdejong has a chance to review and merge that, and then rebase on develop again).

@yifanwww
Copy link
Copy Markdown
Contributor Author

yifanwww commented Mar 28, 2022

@gwhitney That's ok to wait that PR, since we already have a PR #2505 to wait and lgamma for BigNumber is not finished.

UPDATED:
It seems that I misunderstand your meaning, so you can just ignore this comment.

@josdejong
Copy link
Copy Markdown
Owner

Note that if you simply do not provide an implementation for BigNumber, then mathjs will just throw a TypeError if someone tries to compute lgamma on a BigNumber -- maybe that's sufficient?

Indeed. It is fine if there just is no BigNumber implementation if that is a step to far for now. The user will get an error, and can convert to a regular number him/her self if that is acceptable for him/her.

@yifanwww
Copy link
Copy Markdown
Contributor Author

I don't know what's wrong, now I cannot get a TypeError if I try to compute lgamma on a BigNumber.

console.log(math.lgamma(new BigNumber(0)))
console.log(math.lgamma(math.bignumber(0)))

// output:
// { re: Infinity, im: 0 }
// { re: Infinity, im: 0 }

@gwhitney
Copy link
Copy Markdown
Collaborator

There's an automatic conversion from BigNumber to Complex. So if you have a Complex implementation and want BigNumber to throw, you have to make a BigNumber implementation that throws. Sorry I didn't realize this before.

@yifanwww
Copy link
Copy Markdown
Contributor Author

Ready for merge, please check it.

Comment thread src/expression/embeddedDocs/function/probability/lgamma.js
Comment thread src/function/probability/lgamma.js
Comment thread AUTHORS
Comment thread src/function/probability/lgamma.js Outdated
Comment thread src/function/probability/lgamma.js
Comment thread src/function/probability/lgamma.js
Comment thread src/function/probability/lgamma.js Outdated

return signa ^ signb ? -a : a
}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@josdejong does this not exist already somewhere in mathjs? If not, is there a utility module it belongs in rather than here?

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think there is already such a util function. We could indeed move it to a util file like src/utils/number.js.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved.

Comment thread test/unit-tests/function/probability/lgamma.test.js Outdated
Comment thread types/index.d.ts Outdated
Comment thread test/unit-tests/function/probability/lgamma.test.js
Comment thread src/function/probability/lgamma.js
@gwhitney
Copy link
Copy Markdown
Collaborator

My apologies for doing all those as individual comments rather than a cohesive review, I was distracted by another task and did not realize that there would be several comments. Thanks for understanding. One last item that coudn't be attached to a file: please add a test or test in types/index.ts that lgamma types and executes correctly. Thanks so much for taking care of all of these details! The submission is now looking good and we are looking forward to getting it in. (We will just need feedback from Jos when he has a chance on the best location for the copysign utility function, or maybe something equivalent already exists somewhere in mathjs.)

@yifanwww
Copy link
Copy Markdown
Contributor Author

One last item that coudn't be attached to a file: please add a test or test in types/index.ts that lgamma types and executes correctly.

Thanks for reminding. I added a type test for lgamma at the end of types/index.ts, but don't know if it's correct since we don't yet have similar type test cases for probability functions that I can refer to.

Comment thread src/function/probability/lgamma.js
@gwhitney
Copy link
Copy Markdown
Collaborator

The TypeScript test looks fine to me. Basically anything that does anything substantive with the function is good. And yes, they are missing for other functions, but that doesn't mean we should compound the problem when a new function is added, so thank you for your understanding on that. I just had one last documentation point I just commented above, and we are just waiting for feedback from Jos on a couple of small points for you to act on if necessary, and then this should be good to merge. Thank you so much for all of your efforts and seeing it through the long process, it's much appreciated.

@josdejong
Copy link
Copy Markdown
Owner

Thanks guys for all the hard work, this PR is looking very good now!

I've commented on the conversations where you tagged me @gwhitney.

@gwhitney gwhitney mentioned this pull request Apr 12, 2022
@gwhitney gwhitney merged commit b3f564f into josdejong:develop Apr 12, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants