Convergence issues with "mixed" parametrization of multivariate models

Hello,
in the context of multivariate twin models, I often encounter convergence issues when "mixing" different parametrizations of multivariate twin models. One example: When I do not want to establish a causal order between some of the decomposed variables, I choose a "correlated factors" approach while building on the Cholesky parametrization to model the covariances between the "independent" decomposed variables and the "dependent" decomposed variable with cross paths heading to the dependent variable coming from the independent variable. When I don't use this mixed approach, e.g. just using the Cholesky parametrization for all variables, there are no convergence problems.
I could use the correlated factors solution, or alternatively the variance component solution with the latent factors variance as a free parameter, for all variables but I want to moderate some paths and I am not sure whether it is recommended to moderate a correlation or (co)variance.
Are there any a priori reasons that the mixed approach might fail? If so, what parametrization do you recommend?

Edit: I attached a path diagram as an example of what I'm referring to with a "mixed" parametrization: The covariance between e.g. Fx1 and Fx1 is decomposed via the correlated factors approach but the covariance between Fx1 and Fx2 via a directed paths.

Edit: After adjusting some of the starting values, the model now runs. But I'm still interested in hearing your opinions whether there are some a priori reasons to expect problems with a mixed parametrization.