[Limdep Nlogit List] Model selection Criteria

[Fred Feinberg]

Santa Dutta wrote:

> We generally do model selection based on the log-likelihood ratio and
> chi-square values of different models and their degrees of freedom. But in
> case of models with same number of estimated parameters, this cannot be done.
> So, Can we just use the log-likelihood values and the rho-square values of
> two models and compare them and comment on which model is the better one
> instead of doing through some other complex tests like vuong test.
> Or is their any other method of best model identification for models with
> same number of estimated parameters? Regards, S.S.Dutta
> ________________________________

The tests you describe are only valid for *nested* models (one is a parametric
restriction of the other). Models with the same number of parameters cannot be
nested.

There is a sizeable literature on testing non-nested models, but no
all-purpose, simple methods based on the sort of asymptotic theory that the
chi-square test is. One is simply to compare some synthetic measure of
in-sample fit, like AIC or BIC (although these propose penalties for number of
parameters, which in your application are the same in the two models), or MAD
(mean absolute devation), etc. If "your" model does better on pretty much all
of these, that's considered convincing evidence.

If you are a Bayesian, there is an all-purpose solution: calculating integrated
likelihoods and doing model comparison via Bayes Factors. But this is highly
non-trivial, even for simple models (see Chib's papers on the topic, for
starters).

A last possibility is to compute a variety of measures on a hold-out sample,
which is really the gold standard, and not only in-sample. A bit of Googling
should take you to resources describing how. But likelihood ratios and
Chi-squares just won't work in this situation.

Fred

=====

Fred Feinberg
Hallman Fellow and Professor
Ross School of Business
University of Michigan
feinf at umich.edu


> _______________
> Limdep site list
> Limdep at limdep.itls.usyd.edu.au
> http://limdep.itls.usyd.edu.au