Tutorials start too fast
Steve
Joined: 07/30/2009
The tutorial currently starts with far too much assumed knowledge. I think we will see that as we begin to hear comments from our beta testers. If we don't change our style from the current Mx (high level, lots of features, but no concessions for beginners), there is not much point in having written OpenMx. We will continue to appeal to a niche audience, primarily BG researchers who are already willing to learn Mx.
The main complaint I hear about Mx, and I hear this alot: "It must be a great program, but it's too hard for someone like me." I hear this from PhDs who are successfully running and publishing SEM. I even have heard it from people who teach SEM. I think we need to remember that on a day-to-day basis we operate in a rarified atmosphere. We could increase the user base of OpenMx a great deal by keeping its introductory tutorials simple enough that people who will may never hear of a Cholesky decomposition in their entire research career will not think "This is just like Mx, it's too hard for me".
The R philosophy is "make
Ross Ihaka
I agree the learning curve needs to be flatter for standard things.
Part of that could be fixed for sure by slowing down and re-ordering. Beginning a tutorial on an SEM program with an example of matrix algebra and inner and outer products doesn't seem helpful to me.
Why not start with the idea of the variance of one column of data, then a correlation, with path diagrams at each point, and full downloadable and runnable cut and paste usable scripts, each of which answers a real question - the value of the desired parameter and its significance or confidence interval.
And side-by-side comparisons to getting the same thing done in R using var() and cor() to help people make connections...
With our new instrument, let people play songs they already know how to hum, and which they find useful.
Also, endeavor not to call things names like "q3" when "Matrix A.B" would be more informative. Characters are no longer valuable - we can spend them freely.
I'd add too that this assumes a ton of knowledge about matrices (like what an inner and outer product are) without defining them, giving any references, or mentioning why, if ever, someone doing an SEM would need to know about them.
Perhaps solution is to have an OpenMX tutorial based around getting SEM done, and then, somewhere else, a tutorial on using OpenMx for doing Matrix algebra and/or learning OpenMx if you happen to already know Mx.
Log in or register to post comments
In reply to The R philosophy is "make by tbates
I disagree with Tim about the
Tim's final suggestion for a straight sem example right off the top with a separate section on how to do matrix algrebra makes better sense.
I would like a simple model for a congeneric problem. Then I would like to solve it. Then do it for two groups. Then .... ?
Having path diagrams for what one is doing is a great suggestion.
Jim Steiger's suggestion to show the commands and then show the output is very good (at least from an R perspective). I want to make sure that what I am doing is producing what it should produce.
Bill
Log in or register to post comments
In reply to I disagree with Tim about the by wrevelle
Hi Bill, did mean you agree
did mean you agree (rather than disagree?) with me about speed? I think we are on the same page...
Log in or register to post comments
In reply to I disagree with Tim about the by wrevelle
Sounds to me like Bill and
I also agree with Jim Steiger's suggestion about always including output. It helps people to see what should be produced.
Log in or register to post comments
After meeting with several
Log in or register to post comments
In reply to After meeting with several by pdeboeck
Along these lines: As an
As an extremely naive R user but a somewhat experienced SEM software user...can I make a suggestion on the quick start tutorial.
I installed OpenMx and it went through fine. Then I went to the quick start tutorial and of course pasted in the first command to see how it ran...
mxMatrix(
type="Full",
nrow=3,
ncol=1,
values=c(1,2,3),
name='A'
)
mxMatrix(
type="Full",
nrow=3,
ncol=1,
values=c(1,2,3),
name='A'
)
The problem is...it didn't run. I got this...
Error: could not find function "mxMatrix"
> mxMatrix(
+ type="Full",
+ nrow=3,
+ ncol=1,
+ values=c(1,2,3),
+ name='A'
+ )
Error: could not find function "mxMatrix"
> mxAlgebra(
+ A %*% t(A),
+ name='q1'
+ )
Error: could not find function "mxAlgebra"
> mxAlgebra(
+ t(A) %*% A,
+ name='q2'
+ )
Error: could not find function "mxAlgebra"
> mxAlgebra(
+ A * A,
+ name='q3'
+ )
Error: could not find function "mxAlgebra"
> mxAlgebra(
+ A + B,
+ name='q4'
+ )
Error: could not find function "mxAlgebra"
>
So that made me think that OpenMx didn't install properly. So I spent a while trying to re-install it and kept getting the same error message. Frustrating. Then...I found the library of commands in my OpenMx files that were installed and open one up and it ran. So it looks like I need to type in the require(OpenMx) command. So, just a thought that the first set of code in the users manual might be confusing to R-newbies such as myself.
Log in or register to post comments
In reply to Along these lines: As an by msturgeapple
Good point!
Log in or register to post comments
In reply to Good point! by Steve
I've added an OpenMx
Log in or register to post comments
In reply to I've added an OpenMx by mspiegel
Beta testers: please try the
Log in or register to post comments
In reply to Beta testers: please try the by mspiegel
Since I'm not versed in
Comments on the new guide:
1. For me, the pass by value section is a little redundant because i do get some of the code there, but for someone not versed in R, that could be helpful.
2. The section after that is great. This might sound really needy...but I wonder if some sort of diagram to go with the five factor model might be helpful. Particularly since this is the first tutorial that a new user will encounter - the figure might help to solidify the concepts in the code.
What is an "s4 class"?
Finally...this thought might get towards what the purpose of OpenMx is. If the purpose is to be accessible to SEM users who typically might try AMOS, MPlus, ect...then I do get lost in this section of the quick users guide. Primarily because these users are probably not versed in the language of matrices and SEM. However, if OpenMx is for users who are more advanced in the workings of SEM and get matrices, etc..then the section reads fine for me. What type of person do you want the quick users guide to appeal to?
Log in or register to post comments
In reply to Since I'm not versed in by msturgeapple
Good feedback! I am hearing
S4 objects and S4 classes are two related terms that are used to describe the dominant implementation of object-oriented programming in R. I am not much of an R historian, but the name "S4 objects" is somehow related to "S3 objects" which was an earlier implementation of object-oriented programming in R. One of the more concise descriptions I've seen of S4 is called "S4 Classes in 15 pages, more or less." Sadly, I have not found a good stand-alone introduction to S4 objects. The presentations online tend to assume the reader is familiar with some sort of object-oriented implementation.
Log in or register to post comments
In reply to Since I'm not versed in by msturgeapple
I agree with msturgeapple's
My opinion is that the quick users guide should be primarily introducing users to the applications of the statistical package, rather than explaining the logic behind the SEM & Matrices.
I would prefer to see an agenda for the wiki documentation so that users can begin providing some expository and interactive (i.e. using OpenMX) instruction on the logic and basics of SEM/Matrices. Perhaps some of this can be using OpenMX while other portions are just instructive. However, I think it is going to be important to set an agenda for the wiki so that users can begin to contribute in an orderly fashion.
Log in or register to post comments
In reply to I agree with msturgeapple's by smcquillin
Good idea!!
Log in or register to post comments
In reply to I agree with msturgeapple's by smcquillin
I'd say just dig in:make some
I think it's a barrier that the wiki seems limited to html rather than mediawiki syntax - it's a lot more work to work on pages when everything has to have closing tags and such.
Can we get something here that does wiki syntax?
Log in or register to post comments
In reply to I'd say just dig in:make some by tbates
Does the quick start users
Also, the first section of the quick start guide - honestly, I have no idea what that means or how to follow it and in reading it, I get the feeling that I need to read up on R before I can even attempt OpenMx. Which might be a bit discouraging. I'm happy to work with someone on the users guild outside of the boards so not to take up board space - try to make it more user friendly from the non-R sophisticated perspective, if that's the goal of the team. Let me know.
Also, I'm not sure if this is something I need to program in my computer or if it's a function of the website - but is it possible to have some of the documentation links open to a new page when I click on them? Otherwise I have to go back and forth when it would be easier to have a new page and the old page. I suppose I could just paste the URL in my browser...but if it's an easy thing just thought I would mention it.
Log in or register to post comments
In reply to Does the quick start users by msturgeapple
hi, > does the quick start
> does the quick start repeat things?
Can you point out the specific pieces that were repeated? (note that things are covered twice: once for writing a path-based model, and once for a matrix-based model, so those sections are similar, not repetitive by design)
> is it possible to have some of the documentation links open to a
> new page when I click on them?
Just right-click (or cntrl-click on a mac) the link, and say "open in new page"
Log in or register to post comments
In reply to Does the quick start users by msturgeapple
> the first section of the
The quick-start dives into "pass by value" from the get go, which might be off-putting.
I wonder if we could say:
A key insight into using OpenMx is that you assemble your model out of a set of pieces (paths or matrices, data, perhaps some algebra), and only subsequently run the model using mxRun().
Because of this, it becomes important to understand how R treats the parts of a model that you are assembling. In particular, that R uses a style of information passing called "pass by value" as opposed to "pass by reference". You can proceed with the quick start guide following how to get a basic model run, but you should also spend some time reading the "Guide to Pass by value" in order to get the most out of OpenMx and avoid surprises that might trip you up later on.
... a nice model that can be typed in and run to get a feeling for how OpenMx works
Log in or register to post comments
My opinion is that since
Log in or register to post comments
In reply to My opinion is that since by mspiegel
Perhaps this should a
Log in or register to post comments
In reply to My opinion is that since by mspiegel
Hi Michael, I think you're
Hi Michael, I think you're right about pass-by-value not being a hand-wave issue, but ideally the quick start shouldn't have much of a learning curve.
So my comment was in line with helping people get to issues as they need to and to avoid setting things up as hurdles when people don't even know enough to make an error.
For the quick start (as opposed to an in-depth study guide), I'd begin by cutting and pasting in the 6-line factor analysis and getting people to push return: then people know this thing works and motivation to learn goes up.
With something working under their belt, where name="innerName" is used in and algebra (say) while name=object is used to bring the object into a model, People will be thinking (if previous comments are a guide) that that seems a bit unwieldy - so a good time to explain that that's how it's done and is worth making into a habit - perhaps that the easiest way to get used to this is to adopt a personal rule for naming objects based on the "name" parameter (like pre-pending the type to the internal name to make memorable objectName: matrix "A" --> "matrixA".
Then have some pages or sidebars in the pdf on design philosophy, constraints imposed by pass-by-value, the name !="name" issue, and pass-by-value in depth.
I'm sure much of this is something where there is no easy answer, and the same issues come up no matter what, so no big deal.
Log in or register to post comments
In reply to My opinion is that since by mspiegel
Your point about common
Your point about common errors made me wonder if we know what the causes of error are?
For me it was (is :-) ) a whole set of things. I'd say
Then a pretty flat space of things like
Just the normal learning of a complex system :-) but very very rewarding!
Log in or register to post comments
In reply to My opinion is that since by mspiegel
I'd say that there is a
I'd say that there is a difference between "Hand Wave", which carries a negative connotation and "Minimally Sufficient in Order to Prevent Beginner Errors". Like it or not, most of our target users just want to know the minimum thing they need to memorize and don't care why it is the case.
So, I'd recommend that we have something like:
We have found that many people make error X. Here is all you need to remember in order to avoid X: "blah blah".
For those of you who want to know why this is the way it is: "blah blah blah blah blah blah blah.... etc. etc. etc. Blah Blah"
Wish everyone were programmers and cared why, but I'd estimate that 90% don't care. However, the 10% are more likely to post long posts on the forums...
Log in or register to post comments
As someone who has a rather
I am working through it slowly and annotating heavily for my own benefit and that of my students.
If someone can let me have a Word version of the documentation that I can edit, I'd be happy to make suggestions of how make it more friendly for the complete novice - and also to flag up points where I don't understand it.
Log in or register to post comments
In reply to As someone who has a rather by Dorothy Bishop
Unfortunately our
Log in or register to post comments
In reply to Unfortunately our by mspiegel
I think that using HTML->txt
I will attach a document with suggested changes that I generated this afternoon. I could continue if people think this kind of additional material would be useful - I guess it could be irritating for the experts.
I am not sure whether I will persist - I had hoped that OpenMx would be more straightforward than Mx, and that I'd be able to get quickly up to speed, but my experience today suggests not - I'm only up to page 5 and had to spend a lot of time googling to work out R commands. I do find R pretty frustrating after working in Matlab. Colleagues tell me it's good once you get used to it, so I am willing to persevere a bit longer, but I'll wait to see what you think of the attached.
Log in or register to post comments
In reply to I think that using HTML->txt by Dorothy Bishop
Your comments are really
As I see it, we have three audiences for this documentation. Some, like you, are new to R but expert in Mx and SEM modeling in general.
A second group is familiar with R, but is new to SEM and/or Mx. Maybe they've used Amos or MPlus a little. Or maybe they don't know SEM modeling at all.
The third group knows neither R nor SEM.
I hope you continue with your comments, but it is very valuable that you make these comments as you go through the frustration of learning. It is hard for an expert to remember what frustrated them as a novice. And if you keep going, you'll be an expert soon.
One other comment. It is clear from your documentation that you are running on Windows and using the R GUI. We have another challenge, and that is for the documentation to work just as well for PC users as Mac users as Linux users.
I think you'll find that as you run R from the R GUI, it will be a good habit to do the following: First create a project directory that will contain all of your scripts, data, and workspace for a project. Second, copy your data and any pre-existing scripts into that directory. Third, start up a session of the R GUI just for that project and use the "Change Working Directory" option to that directory. Now, as you work, you won't need to use any paths in your read.table() commands or write.table() commands. Plus, when you quit, you can Save Workspace and you will be able to take up where you left off. Also, your script will continue to work even if you move your project directory.
Log in or register to post comments
In reply to Your comments are really by Steve
Thanks Steve. All advice most
I'll press on working through the manual and see if this is going to be a realistic thing to aim for. I have to make the decision whether to move to R for my own analyses and those of a couple of students and am not yet sure it will be worth the investment of effort.
Log in or register to post comments
In reply to Thanks Steve. All advice most by Dorothy Bishop
Hi, Dorothy, Thanks for your
Thanks for your input thus far. If you would like to start writing documentation that people can use (and check), you could create a page in our Wiki. We would love to have user-contributed documentation/tutorials.
I hope you don't give up on R quite yet. It does take some time to get the hang of, but here are some references that may be of value to you and your students:
http://cran.r-project.org/other-docs.html
Log in or register to post comments
In reply to Hi, Dorothy, Thanks for your by Jeff
I've nearly finished some
The example of using a simulated dataset with correlated data is nice from the point of view of introducing people to simulations and to model-fitting in a relatively simple situation, but I'm having trouble explaining to the uninitiated exactly what it is that we are doing with the likelihood estimation, I think because I am confused myself. I can follow how to do it, but not why.
I think lots of novices will find it odd that you generate a dataset with a specified intercorrelation, but then evaluate the likelihood of the data. Obviously, it makes sense if you want to compare fit to models with different covariance between X and Y, but the initial ML estimation seems odd. I have no trouble with the notion that there will be a standard error around an estimate of a correlation etc - should this example be introduced in that kind of context? Advice would be welcome on this.
Log in or register to post comments
In reply to I've nearly finished some by Dorothy Bishop
The basic idea behind using
I'm hoping I'm understanding your question.
If the data are simulated, we know what the correct answer is, but OpenMx doesn't know the answer. OpenMx is estimating an answer, just the same as if it were given data from a real experiment.
Does that clear things up? Hopefully I'm not just further muddying the waters.
Log in or register to post comments
In reply to The basic idea behind using by Steve
Just to say that I have
Log in or register to post comments
I'd be very grateful for some
Log in or register to post comments