I set up a simple multinomial log-likelihood with a known solution, and attempted to solve it by minimizing the negative log-likelihood using Powell's (10.7 3rd ed). The negative log-likelihood is as follows:


negativelogLikelihood = -( 1000*log(1 - s + s*(1-p)*(1-k)) +
500*log(s*(1-p)*k) + 750*log(s*p*(1-k)) +
1100 * log(s*p*k) )


where s, p, and k are parameters to be estimated.

The closed-form solution is:
s = 0.80,
p = 0.69,
k = 0.59

Powell gives the solution:
p = 0.94
p = 0.60
k = 0.52

When I used the same likelihood (the very same code) with the Downhill simplex method (10.5), it gives the correct solution.

Does anyone have any ideas, why Powell is arriving at the incorrect minimum?

Thanks for any help.

I recently coded up the simplex method as a general fitter, the powell method and the Levenberg Marqaurdt using the NR3 header files and with all the problems I have used them for I have obtained the same minimums. I have made them as general tools for fitting any function in Matlab.

So did you use the exact same functor for both the simplex and Powell minimizations, and was it set up the same? A typo could lead to a different solution. Could you show us the code that you wrote to use the powell and simplex methods, it might be possible to spot the bug with a bit more information?

Will Handler

...Does anyone have any ideas...

1. What value of tol are you using to initialize your Powell object?

2. In your function evaluation, do you have some tests that return a very large value in case any of the expressions in terms like log(expression) are <= 0?

3. What initial values are you using?

Here are a few runs that I get:

All runs are for tol = 1.00000e-16

Initially, value = 4.57046e+03, at: 9.40000e-01 6.00000e-01 5.20000e-01
Powell minimum = 4.50751e+03, at: 8.03256e-01 6.87500e-01 5.94595e-01
Number of Powell iterations = 5


Initially, value = 4.50816e+03, at: 8.00000e-01 7.00000e-01 6.00000e-01
Powell minimum = 4.50751e+03, at: 8.03256e-01 6.87500e-01 5.94595e-01
Number of Powell iterations = 4


Initially, value = 1.35059e+04, at: 1.00000e-01 1.00000e-01 1.00000e-01
Powell minimum = 4.50751e+03, at: 8.03256e-01 6.87500e-01 5.94595e-01
Number of Powell iterations = 14


Initially, value = 1.06840e+05, at: 1.00000e-08 1.00000e-08 1.00000e-08
Powell minimum = 4.50751e+03, at: 8.03256e-01 6.87500e-01 5.94595e-01
Number of Powell iterations = 47


Initially, value = 3.10849e+04, at: 9.99999e-01 9.99999e-01 9.99999e-01
Powell minimum = 4.50751e+03, at: 8.03256e-01 6.87500e-01 5.94595e-01
Number of Powell iterations = 7


Regards,

Dave