[Limdep Nlogit List] Model selection Criteria

[Noel Roy]

At 01:21 PM 04/03/2008, you wrote:
>Santa Dutta wrote:
>
> > We generally do model selection based on the log-likelihood ratio and
> > chi-square values of different models and their degrees of freedom. But in
> > case of models with same number of estimated parameters, this 
> cannot be done.
> > So, Can we just use the log-likelihood values and the rho-square values of
> > two models and compare them and comment on which model is the better one
> > instead of doing through some other complex tests like vuong test.
> > Or is their any other method of best model identification for models with
> > same number of estimated parameters? Regards, S.S.Dutta
> > ________________________________
>
>The tests you describe are only valid for *nested* models (one is a parametric
>restriction of the other). Models with the same number of parameters cannot be
>nested.
>
>There is a sizeable literature on testing non-nested models, but no
>all-purpose, simple methods based on the sort of asymptotic theory that the
>chi-square test is. One is simply to compare some synthetic measure of
>in-sample fit, like AIC or BIC (although these propose penalties for number of
>parameters, which in your application are the same in the two models), or MAD
>(mean absolute devation), etc. If "your" model does better on pretty much all
>of these, that's considered convincing evidence.

I would add here the Pollock-Wales paper (JEcon 47 (1991)) on the 
likelihood dominance criterion. On this criterion, if two non-nested 
models have the same number of parameters and same dependent variable 
(or, if the dependent variables are functionally related, the 
appropriate Jacobian is incorporated into the likelihood function), 
the model with the highest log-likelihood dominates.


>If you are a Bayesian, there is an all-purpose solution: calculating 
>integrated
>likelihoods and doing model comparison via Bayes Factors. But this is highly
>non-trivial, even for simple models (see Chib's papers on the topic, for
>starters).
>
>A last possibility is to compute a variety of measures on a hold-out sample,
>which is really the gold standard, and not only in-sample. A bit of Googling
>should take you to resources describing how. But likelihood ratios and
>Chi-squares just won't work in this situation.
>
>Fred
>
>=====
>
>Fred Feinberg
>Hallman Fellow and Professor
>Ross School of Business
>University of Michigan
>feinf at umich.edu
>
>
> > _______________
> > Limdep site list
> > Limdep at limdep.itls.usyd.edu.au
> > http://limdep.itls.usyd.edu.au
>
>
>
>_______________________________________________
>Limdep site list
>Limdep at limdep.itls.usyd.edu.au
>http://limdep.itls.usyd.edu.au




-----
Dr. Noel Roy
Professor and Head
Department of Economics	
Memorial University of Newfoundland
St. John's, Newfoundland, Canada A1C 5S7

tel: (709) 737-8245
fax: (709) 737-2094
WWW: http://www.ucs.mun.ca/~noelroy