I am trying to optimize a function under certain constraint using Lagrange Multiplier. Is there anyone who can help me finding the physical meaning of the coefficient? Suggestion will be highly appreciated.

By the coefficient, I assume you mean the multiplier introduced in the Lagrange multiplier method. The multiplier is the derivative of the obtained maximum or minimum value with respect to the constraint.

To see this, let x0 be the solution to the problem "maximize f(x) subject to the constraint g(x)=k0." One solves this by introducing the multiplier L and solving the system of equations

grad f(x) = L grad g(x)
g(x) = k0

Now replace k0 by the variable k, and let x(k) be the corresponding solution as a function of k. Then

d f(x(k))/dk = (grad f) dot dx/dk
= L (grad g) dot dx/dk
= L d g(x(k))/dk
= L dk/dk
= L

Thank you, Mr. rbrowne. I appreciate!