Non-recursive model and negative SMC

Depends on your model. In many models, squared multiple correlations may be defined in terms of variance reduction for a given variable, with the squared multiple correlation defined as total variance (from either a saturated model or from the model expected variance) minus the residual variance for any variable. There are two things to note here. First, this squared multiple correlation is model dependent; not only is the SMC in this case tied to all predictor variables, constraints and other aspects of the model can affect the appropriate model parameters and thus the SMC. Second, this definition breaks when the variable of interest predicts other variables or is in a non-recursive model, as an estimated residual variance term is an incomplete estimate of the variance unique to any variable in this case.

If you provide code and more information about what you're trying to do, we can be of more help.