matrix not positive definite

Unfortunately we have only limited control over the behavior of the optimizer as we are not using one that we developed ourselves (that may change in the future). In the early 1990's I visited the good people at Stanford to try to address this problem, with partial success. There are essentially two phases to optimization: i) figuring out the first and second derivatives of the likelihood function with respect to each of the parameters; and ii) moving to a new location based on the information gathered in i). Built into the optimizer there is a mechanism that when a ii) occurs if there is non-positive definiteness (or zero likelihood for some other reason such as outlier observations or really bad parameter estimates) then it will back out of the region to which it moved. It does this reducing the size of the step it takes towards the new best guess as to where the solution is. Several previously problematic cases ran smoothly after this change was introduced. However, two factors limit the effectiveness of this strategy. One is that during the phase i) part incomputable likelihoods may be encountered, which really stops the optimizer in its tracks because it no longer has the information needed to decide where to go next. The second is that, perhaps due to the nature of the data or the model, even a very small step taken in a phase ii) still ends up in non-positive definite (or otherwise incomputable) land. Usually the solution in such cases is to start optimization from some other location. Sometimes the model needs to be revised, and other times one may have to go and get some more data. Although this last seems rather drastic, it is perforce the case if one has more variables (columns) than subjects (rows) in a data set. Optimization would head towards a non-positive definite region in a quite determined fashion. If the model had been specified (e.g. via Cholesky) to only generate positive definite matrices then optimization would likely end up at the boundary. If it had not been specified to stay positive definite, the optimizer would likely try parameter estimates which generate non-positive definite predicted covariance matrices, and - since the log-likelihood is not computable in such instances - fail rather unhappily.