Linear Model and Statistical Tests

STATISTICAL TESTS

The traditional approach is based on a number of statistical tests where every single test has its own formula and tests a very specific null-hypothesis. The approach of linear modelling works with first specifiying a linear model for the data. For the inference regarding statistical significance it is not important which approach is used. However, there are two main advantages of the linear model. First, the family of linear models is so large that you can test infinitely more hypotheses with a linear model than with a specially-designed test-statistic. Second, the syntaxes of SPSS and codes used in R all follow the same structure which makes it more accessible and understandable. 

Linear model

At the University of Twente we encourage students to use linear models for their statistical analyses. Also we as the Methodology Shop would like to advise you to focus on linear models. 

Generally, linear models follow the structure of y=b0 + b1x1 + b2x2 + ... + e. This is often recognized as a regression model which is one form of a linear model. By including categorical variables you can extend the model and cover ANOVA-type of analyses. The linear models themselves can also be extended to linear mixed models to model dependencies in the data, e.g. when you used repeated measures, clustered data, multilevel etc. Also, generalized linear models can be used to model non-numeric dependent variables, such as dichotomous (e.g. logistic regression) or when your data consist of counts (poisson regression). 

BELOW YOU CAN FIND AN OVERVIEW OF WHAT IS THE BEST LINEAR APPROACH TO ANSWER YOUR RESEARCH QUESTION

To be able to choose the right approach you need to know what you dependent and independent variables are. You also need to know what type of variables you have (e.g. categorical, numerical etc.). Please write this down first and have a look at this overview afterwards.

The mentioned chapters refer to the following book.

  • Models with a Numeric Dependent Variable without a Clustering Variable

     

    Old approach

    New approach

    Relevant chapters

    Comparing two means from two independent samples

    Independent samples t-test

    Linear model with

    • Numeric dependent variable
    • Dichotomous independent variable

    Chapter 6

    Comparing more than two means from independent samples

    One-way analysis of variance

    Linear model with

    • Numeric dependent variable
    • Categorical independent variable

    Chapter 6

    Testing the interaction effect of two categorical variables on  a numeric dependent variable

    Factorial analysis of variance

    Linear model with

    • Numeric dependent variable
    • Two independent categorical variables

    Chapter 9

    Testing the interaction effect of two numeric variables on a numeric dependent variable

    Regression analysis

    Linear model with

    • Numeric dependent variable
    • Two numeric independent variables

    Chapter 9

    Testing the interaction effect of one independent numeric variable and one numeric dependent variable

    Not possible

    Linear model with

    • Numeric dependent variable
    • One numeric independent variable
    • One categorical independent variable

    Chapter 9 

    Regression

    Linear regression analysis

    Linear model with

    • Numeric dependent variable
    • Numeric or dummy independent variable

    Chapter 4 and Chapter 6.5

  • Models with a Numeric Dependent Variable and a Clustering Variable (due to Repeated Measures or ESM)


    Old approach

    New approach

    Relevant chapters

    Comparing two means from two related samples

    Paired samples t-test

    Linear mixed model with

    • Numeric dependent variable
    • Dichotomous independent variable
    • Clustering variable “id”

    Chapter 12

    Comparing more than two means from related samples

    Repeated measures ANOVA

    Linear mixed model with

    • Numeric dependent variable
    • Categorical independent variable
    • Clustering variable “id”

    Chapter 13

    Analysing data from ESM studies


    Linear mixed model with

    • Numeric dependent variable
    • Categorical & numeric independent variable
    • Clustering variable “id”

    Chapter 12 & Chapter 13


    Additional information for analysing data from ESM studies:

    To analyse your data, make sure to have your data in a long-format (one observation per row). 

    Here you find information how to do the analysis in SPSS. 

    Person Mean (PM) and Person-mean centering (PMC)

    In case you want to distinguish between- and within-person effects it is important to apply person-mean centering. This article explains in detail how to do this and what the rationale behind it is. 

  • Models with a Dummy or a Count Dependent Variable


    Old approach

    New approach

    Relevant chapters

    Logistic regression

    Logistic regression analysis

    Generalized linear model with

    • Dummy dependent variable
    • Numeric/dummy/categorical independent variable

    Chapter 15

    Testing the independence of two categorical variables

    Pearson chi-square test

    Generalized linear model with

    • Dependent variable that represents counts
    • Independent categorical variables

    Chapter 16