History

RSA first introduced by Box and Wilson in 1951 to determine optimal conditions (the response, or the outcome variable) in chemical investigations and understand the interplay between various factors (the predictors). One way to determine what combinations of factors can produce the optimum outcome is to simply try out every possible combination of factors. A second approach, introduced via RSA, is to select a relatively small number of combinations of the factors (e.g., temperature, pressure, proportions of the reactants) and measure the outcome (i.e., experimental attainment) to estimate a response surface. By doing so, one could identify a sub-region of the whole surface to approximate the optimum outcome using a limited number of combinations of factors located within the sub-region (i.e., “sequential experimentation”).

For example, Trink and Kang (2010) used response surface methods to study the optimization of coagulation tests by estimating the following 3D surface using coagulation pH and Alum dose to predict turbidity removal. By fitting a quadratic model using RSA, they concluded that the removal of maximum amounts of 92.5% turbidity (i.e., optimal outcome) was found with 44 mg/L Alum dose at Coagulation pH 7.6.

Figure 1

Three dimensional response surface for predicting turbidity removal using Coagulation pH and Alum dose (reproduced using the regression coefficients from Figure 6 in Trinh & Kang, 2015).

Similarly, social scientists are sometimes interested in how two variables work together to predict one outcome. One area of research that finds RSA particularly interesting and useful is the investigation of the congruence hypothesis - the congruence (i.e., fit, match, similarity, or agreement) between two predictor variables can produce the optimum condition of an outcome variable.

For instance, Humberg and colleagues (2018) examined whether well-being is maximized when self-perceived ability is congruent with objective ability (i.e., the congruence hypothesis). They constructed the following response surface predicting well-being using objective vocabulary and self-estimated vocabulary. By fitting a quadratic model using RSA, they did not find evidence for the congruence hypothesis. Instead, they found that well-being is dominantly predicted by self-perceived ability - higher self-perceived ability can predict higher well-being at all levels of objective ability.

Figure 2

Three dimensional response surface for predicting well-being using objective vocabulary and self-estimated vocabulary (reproduced using the regression coefficients from Figure 5 in Humberg et al., 2018).

Three dimensional response surface for predicting wellbeing using objective vocabulary and self-estimated vocabulary (Figure 5 reproduced using the regression coefficients from Humberg et al., 2018).

Other example questions where RSA is useful include “the effects of the fit between a job and an individual on job performance” (person-environment fit, Caldwell & O’Reilly, 1990) and “the effect of the difference between the perceived harm of media on others and on oneself on the likelihood of endorsing media use restrictions” (third-person effect, Feng et al., 2017).

Why RSA?

Before RSA, researchers often used difference scores (or its adaptations like absolute values) and similarity measures (e.g., correlation coefficient) to provide statistical testings (e.g., the significance level of the coefficient of the difference score variable in the regression) for the congruence hypothesis. RSA is adapted from chemical investigations to overcome multiple problems with such approaches including reduced reliability, ambiguous interpretation, untested constraints on the regression estimation, and oversimplification through dimension reduction (for reviews, see Kenny et al., 2006; Edwards, 2002). RSA untangles those problems by restoring the one component measure (e.g., difference score or similarity measures) to its initial two predictors and retaining a three-dimensional surface (i.e., two predictors plus one outcome).

This tutorial provides a demonstration of fitting, visualizing, and interpreting a basic RSA model for continuous data, as well as its comparison to the results obtained using difference score.

Load necessary libraries

If you are running macOS 10.9 or later, you will need to install XQuartz in order to use the RSA package. R will prompt you to do so the first time you install RSA, and you can also access XQuartz via their site.

library(psych)            # data descriptives
library(RSA)              # Response Surface Analysis
library(plot3D)           # visualize response surfaces
library(plotly)           # visualize response surfaces
library(scatterplot3d)    # visualize 3D scatter plot
library(rgl)              # visualize 3D scatter plot
library(tidyverse)        # data structures & visualization

Introduction to iSAHIB data

We use data from the Intraindividual Study of Affect, Health and Interpersonal Behavior (iSAHIB), which is an intensive longitudinal study that collected rich repeated measures of its namesake variables at multiple time-scales. Participants, 150 adults (age 18-89 years, 51% women) from the Pennsylvania State University and surrounding community, completed three 3-week “measurements bursts” during which they reported on their social interactions, thoughts, feelings, and behaviors.

To request the iSAHIB data, please have a look at the study materials and documentary - and send a note to Nilam Ram.

Variables of Interest

Three variables are of interest here in constructing a response surface. First, Self Agency was measured by asking participants to rate how you acted in the social interaction in terms of agency on a 0-100 slider scale, with 0 tagged Submissive and 100 tagged Dominant. Second, Other Agency was measured by asking participants to rate how the other person acted in the corresponding dyadic social interaction on the same slider scale. Both were adapted from Moskowitz and colleagues (2005). Third, Happiness was measured by asking participants to rate how happy do you feel right now? on a 0 to 100 slider with 0 tagged not at all happy and 100 tagged extremely happy.

Research Question

The current demonstration of RSA explores the effect of the fit between the perceived agency of the self (predictor 1) and the perceived agency of the interaction partner (predictor 2) in dyadic interpersonal interactions on happiness (the outcome). We here test the congruent hypothesis - the more congruent between the perceived self agency and the perceived other agency, the happier oneself would be.

#set filepath for data file
filepath <- "https://raw.githubusercontent.com/The-Change-Lab/collaborations/refs/heads/main/iSAHIB_RSA/isahib_RSA.csv"

#read in the .csv file using the url() function
isahib <- read.csv(file=url(filepath), header=TRUE)

#look at the data
head(isahib, 10)

##    SelfAgency_X OtherAgency_Y Happy_Z
## 1            82            22      57
## 2            97            21      66
## 3            55            51      67
## 4            82            13      73
## 5            32            61      94
## 6            14            95      83
## 7            56            52      78
## 8            34            71      74
## 9            50            50      67
## 10           51            48      77

#describe the data
describe(isahib)

##               vars   n  mean    sd median trimmed   mad min max range  skew
## SelfAgency_X     1 866 51.51 15.76   53.0   51.85 14.83   2  97    95 -0.22
## OtherAgency_Y    2 866 55.93 17.07   57.5   56.45 17.05   4  99    95 -0.28
## Happy_Z          3 866 81.41 11.33   84.0   83.44  5.93   5  98    93 -2.91
##               kurtosis   se
## SelfAgency_X     -0.19 0.54
## OtherAgency_Y    -0.14 0.58
## Happy_Z          11.77 0.39

#plot the distribution of each variable and the correlation between them
psych::pairs.panels(isahib)

#plot raw data scatter plot in three dimensions
plot_ly(x = isahib$SelfAgency_X, y = isahib$OtherAgency_Y, z = isahib$Happy_Z, 
        type = "scatter3d", mode = "markers", color = isahib$Happy_Z) %>%
   layout(scene = list(xaxis=list(title = "Self Agency", nticks=10, range=c(0,100)),
                       yaxis=list(title = "Other Agency", nticks=10, range=c(0,100)),
                       zaxis=list(title = "Happiness", nticks=10, range=c(0,100))))

Line of Congruence

The line of congruence (LOC) is the collection of points where \(Y = X\).

In the plot above, the LOC is shown as the dotted black line. We can see when looking at the projection of the LOC onto the flat surface at the bottom of the cube, that this is the line where \(Y = X\).

The hypothesis for “perfect” congruence is that the outcome Z should be maximized when \(Y = X\), or in regression for when which can be written as \(Y = 0 + 1X\).

#visualizing the surface
plot(rsa_happy_agency,
     rotation = list(x = -63, y = 32, z = 15), # graph position
     surface = "predict",           # surface based on the predicted values of outcome
     param = FALSE,                 # TRUE to display RSA hypothesis testing parameters
     coefs = FALSE,                 # TRUE to display polynomial coefficients
     axes = c("LOC"),               # display the line of congruence and incongruence
     project = c("LOC", "PA1"),     # projections onto the bottom
     points = list(show = FALSE),   # default to display raw scatter points
     hull = FALSE,                  # TRUE to display a bag plot on the surface
     # you can also use type = "contour" here to obtain a contour plot
     type = "3d",                   # or use type = "interactive" for an interactive surface
     legend = TRUE,                 # TRUE to display color legend
     main = ""                      # title
     )

To describe the LOC where \(Y = X\) is on the response surface (i.e., the corresponding hyperbola for LOC), we could substitute this into the polynomial regression:

\[ \begin{split} Z_{i} & = b_{0} + b_{1}X_{i} + b_{2}Y_{i} + b_{3}X^{2}_{i} + b_{4}X_{i}Y_{i} + b_{5}Y^{2}_{i} + e_{i} \\ & = b_{0} + b_{1}X_{i} + b_{2}X_{i} + b_{3}X^{2}_{i} + b_{4}X_{i}X_{i} + b_{5}X^{2}_{i} + e_{i} \\ & = b_{0} + (b_{1}+b_{2})X_{i} + (b_{3}+b_{4}+b_{5})X^{2}_{i} + e_{i} \\ \end{split} \]

First Principal Axis

The first principal axis, along which the upward curvature is the greatest when the surface is convex (i.e., the current example) or saddle-shaped, and which the downward curvature is the least when the surface is concave (Edwards, 2002).

In the plot above, the LOC is shown as the blue diagonal line. To describe the principal axes in the X, Y plane for hypothesis testing (for a detailed transformation to the 2D X, Y plane, or using the contour plot, see Khuri & Cornell, 1996), the equation for the first principal axis is:

\[ Y_i = p_{10} + p_{11}X_i \]

To solve this equation, one first needs to obtain the Hessian matrix of the polynomial equation, then calculate the eigenvectors by solving each eigenvalue of the Hessian matrix. For the purpose of this tutorial we did not present the mathematical process of obtaining \(p_{10}\) and \(p_{11}\) here but anyone interested could consult the book by Khuri and Cornell (1996) for a detailed derivation process (with some adaptations on expression the eigenvectors and stationary point adjustment to correspond to their application in psychology).

Here \(p_{10}\) and \(p_{11}\) are given by:

\[ \begin{split} p_{11} & = \frac{b_{5}-b_{3}+\sqrt[]{(b_{3}-b_{5})^{2}+b_{4}^{2}}}{b_{4}} \\ \\ p_{10} & = Y_{0} - p_{11}X_{0} \end{split} \]

where \((X_{0}, Y_{0})\) is the stationary point of the surface, which is the point at which the slope of the surface is zero in all directions, given by:

\[ \begin{split} X_{0} & = \frac{b_{2}b_{4}-2b_{1}b_{5}}{4b_{3}b_{5}-b_{4}^2} \\ \\ Y_{0} & = \frac{b_{1}b_{4}-2b_{2}b_{3}}{4b_{3}b_{5}-b_{4}^2} \end{split} \]

Line of InCongruence

The line of incongruence (LOIC) is the collection of points where \(Y = -X\).

We now change the project = in plotting to visualize the LOIC as the blue diagonal line. Its associated parabola on the surface is visualized as a blue parabola by changing axes =.

#visualizing the surface
plot(rsa_happy_agency,
     rotation = list(x = -63, y = 32, z = 15), # graph position
     surface = "predict", # surface based on the predicted values of outcome
     param = FALSE, # TRUE to display RSA hypothesis testing parameters
     coefs = FALSE, # TRUE to display polynomial coefficients
     axes = c("LOIC"), # display the line of congruence and incongruence
     project = c("LOIC"), # projections onto the bottom
     points = list(show = FALSE), # default to display raw scatter points
     hull = FALSE, # TRUE to display a bag plot on the surface
     # you can also use type = "contour" here to obtain a contour plot
     type = "3d", # or use type = "interactive" for an interactive surface
     legend = TRUE, # TRUE to display color legend
     main = "" # title
     )

To describe the LOIC where \(Y = -X\) on the response surface (i.e., the corresponding hyperbola for LOIC), we could substitute this into the polynomial regression:

\[ \begin{split} Z_{i} & = b_{0} + b_{1}X_{i} + b_{2}Y_{i} + b_{3}X^{2}_{i} + b_{4}X_{i}Y_{i} + b_{5}Y^{2}_{i} + e_{i} \\ & = b_{0} + b_{1}X_{i} + b_{2}(-X_{i}) + b_{3}X^{2}_{i} + b_{4}X_{i}(-X_{i}) + b_{5}(-X^{2}_{i}) + e_{i} \\ & = b_{0} + (b_{1}-b_{2})X_{i} + (b_{3}-b_{4}+b_{5})X^{2}_{i} + e_{i} \\ \end{split} \]

Broad Congruence Hypothesis

The first statistical significance testing tests that the first principle axis should not significantly differ from the LOC. Visually, this means that the ridge of the surface (or its projection on the XY plane) should not be significantly different than the diagonal of the XY plane (the LOC, extending from the lower left to the upper right). The properties \(p_{10}\approx0\) and \(p_{11}\approx1\) are the first two tests for congruence effects.

The congruence hypothesis suggests that the surface should predict the highest outcome (i.e., happiness) for people with congruent predictors (\(Y = X\)). In incongruent cases where \(Y \neq X\), the predictors deviate from the LOC. All such deviated combinations of X and Y should render lower outcome Z values than the values obtained with numerically equal predictors. If the first principal axis is significantly different from the LOC, then the highest outcome Z value will be obtained with numerically non-equal predictors \(Y \neq X\).

The estimation and statistical significance testing of \(p_{10}\approx0\) are provided in the last section, Principal Axes for the Model, of the model output using the RSA() function.

In the current demonstration, this first conditions are fulfilled with \(p_{10}\) not significantly different from 0 (\(CI: [-293.542, 322.580]\), \(p=.926\)).

This section only provides significance testing of whether \(p_{11}\) is significantly different from 0 instead of 1. To test whether \(p_{11}\) is significantly from 1, we conduct a z-test:

#z-test the p-value (p11 = 2.399, se = 17.740)
z_p11 <- (2.399 - 1) / 17.740
#which is equivalent to obtaining the z-statistics of p11 against the null hypothesis value of 1
p_value <- 2 * (1 - pnorm(z_p11))
p_value

## [1] 0.9371429

The results indicate that the current demonstration data also fulfilled the second condition of the congruent hypothesis that \(p_{11}\) is not significantly different from 1 (\(p = .937\)).

For the third and forth significance testing of the broad sense of congruent hypothesis, we looked at the line of incongruence and its corresponding parabola on the surface. In the RSA package, they set \(a_{3} = b_{1} - b_{2}\) (\(a_{3}X_{i}\)) and \(a_{4} = b_{3}-b_{4}+b_{5}\) (\(a_{4}X_{i}^2\)) to describe LOIC’s parabola for brevity. For any congruence effects to occur, the parabola we just obtained needs to reach its maximum at the point where \(X = Y\) (perfect congruence) and have an inverted U-shape (Humberg et al., 2019; Parry & Edwards, 1993). Those two conditions correspond to the assumptions of the congruence hypothesis that the outcome \(Z\) is the largest only at the numerically aligned predictors combination (\(Y = X\)) and that more deviation from this point would predict lower outcome (the inverted U-shape parabola).

We thus obtain the third and forth significance testing of the congruence effect:

3. \(a_{3}\), the slope of tangent line at the vertex (\(X=0\), \(Y=0\)) on the response surface, should not be significantly different from \(0\) as a parabola is maximized at its vertex of which the slope of the tangent line is 0 and

4. \(a_{4}\) should be significantly negative for the inverted U-shape.

Those results are included in the Surface tests (a1 to a5) for model section of the RSA model output. As in the current demonstration, those two conditions are fulfilled with \(a_{3}\) not significantly different from 0 (\(CI: [-.084, .052]\), \(p = .648\)) and \(a_{4}\) significantly negative (\(CI: [-.011, -.006]\), \(p <.001\)).

In sum, the current demonstration data indicates a congruence effect in a broad sense, meaning that the more similar the perceived self agency and the perceived other agency are, the happier one would report.

To further optimize the equations that satisfy those four conditions for a broad sense of congruence hypothesis, we could get see that only three conditions are required.

\[ \begin{split} p_{11} & = \frac{b_{5}-b_{3}+\sqrt[]{(b_{3}-b_{5})^{2}+b_{4}^{2}}}{b_{4}} \\ \\ & = 1\\ \text{thus} \qquad \quad & \\ \\ \quad b_{4} & + b_{3} - b_{5} = \sqrt[]{(b_{3}-b_{5})^{2}+b_{4}^{2}} \\ \\ b_{3} & = b_{5} \end{split} \]

Similarly,

\[ \begin{split} p_{10} & = Y_{0} - p_{11}X_{0}\\ \\ & = 0 \\ \\ \text{thus} \qquad \quad & \\ \\ \quad \frac{b_{1}b_{4}-2b_{2}b_{3}}{4b_{3}b_{5}-b_{4}^2} & = 1 * \frac{b_{2}b_{4}-2b_{1}b_{5}}{4b_{3}b_{5}-b_{4}^2}\\ \\ b_{1}b_{4}-2b_{2}b_{3} & = b_{2}b_{4}-2b_{1}b_{5}\\ \\ \\ \text{since} \qquad \quad & \\ \\ \quad a_{3} & = b_{1} - b_{2} \\ \\ & = 0 \\ \\ \text{thus} \qquad \quad & \\ \\ \quad b_{3} & = b_{5} \end{split} \]

We could thus consider the four significance tests of broad sense of congruence (i.e., \(p_{10}=0\), \(p_{11}=1\), \(a_{3} = b_{1} - b_{2} = 0\), and \(a_{4} = b_{3}-b_{4}+b_{5} < 0\)) are essentially three tests: \(a_{5} = b_{3} - b_{5} = 0\) (corresponding to \(p_{10}=0\) and \(p_{11}=1\)), \(b_{1} = b_{2}\) (or \(a_{3} = 0\)), and \(b_{3}-b_{4}+b_{5} < 0\) (or \(a_{4} < 0\)). Those are all implemented in the RSA package following (Schönbrodt et al., 2018).

Strict Congruence Hypothesis

We have achieved the broad sense of congruence effects in the demonstration data, while allowing for the fact that the predictors can have “main effects”.

For example, as we can see from the response surface with the demonstration data, the predicted outcome happiness (\(Z=56.85\), \(CI:[47.67, 66.03]\)) is lower when the predictors perceived self agency and perceived others agency are congruent on its high end (e.g., \(X=Y=50\)) than the predicted happiness (\(Z=84.30\), \(CI: [83.25, 85.35]\)) when the two predictors are congruent at the center of the scale (e.g., \(X=Y=0\)). Both sets of predictors are congruent (numerically equal) but the predicted outcome is different with a negative main effect that higher predictors are associated with lower outcome.

Such congruence effects combined with main effects might seem theoretically sound at its first sight: adolescents at stages with higher desired separation and actual separation from family reported higher interpersonal and academic adjustment than themselves at stages with lower desired separation and actual separation from family (Hoffman, 1984).

However, this could also violate one basic assumption for congruence hypothesis that we might observe cases where incongruent predictors can predict better outcome than congruent predictors due to main effects. This is also the case with our demonstration data here that the predicted happiness (\(Z=56.85\), \(CI:[47.67, 66.03]\)) is lower with congruent and higher perceived self agency and other agency (\(X=Y=50\)) than the predicted happiness (\(Z=75.32\), \(CI:[72.70, 77.94]\)) with incongruent and lower predictor combinations (\(X = 30\), \(Y = 20\)).

To avoid such violation of the congruent hypothesis when desired, one could consider adopting the last two significance testing for the strict sense of congruence by looking at the LOC and its corresponding parabola on the response surface. In the RSA package, they set \(a_{1} = b_{1} + b_{2}\) (\(a_{1}X_{i}\)) and \(a_{2} = b_{3}+b_{4}+b_{5}\) (\(a_{2}X_{i}^2\)) to describe LOC’s parabola for brevity. For the strict sense of congruence to occur, the line along all congruent combinations of predictors should be constant. The corresponding statistical testings include two conditions that both \(a_{1}\) and \(a_{2}\) should not be significantly different from 0 so there are no estimated main effects of the predicted surface. Those results can also be found in the Surface tests (a1 to a5) for model section of the RSA model output.

For our current demonstration data, both \(a_{1}\) (\(β = -0.14\), \(CI: [-0.20, -0.08]\), \(p < .001\)) and \(a_{2}\) (\(β = -0.01\), \(CI: [-0.012, -0.004]\), \(p < .001\)) are significantly negative. We did observe the negative linear and curvilinear association between the congruent predictors combination and the outcome, visualized as the corresponding parabola of the LOC on the response surface.

With that being said, we also wanted to note that the broad sense of congruence hypothesis might be more applicable in psychology even with the violation the assumptions of “congruence” as we often expect “main” effects of the combination of the predictors in psychological theories (for example in studying person-environment fit).

In the following table, we report all six hypothesis tests for examining congruence hypothesis using RSA as well as the results for the current empirical example. As shown in the table, LOIC’s parabola has an inverted U-shape (\(a_{4} = -0.01\), \(p < .001\)) and reaches the maximum happiness when perceived self agency is congruent with (numerically equal to) perceived other agency (\(a_{3} = -0.02\), \(p = .648\)). Correspondingly, the first principal axis is not significantly different from the line of congruence (\(p_{10} = 14.52\), \(p = .926\), \(p_{11} = 2.40\), \(p = .937\); or \(a_{5} = 0.00\), \(p = .911\)), meaning that the highest happiness scores, located on the ridge of the surface, are predicted when perceived self agency and perceived other agency are congruent (numerically equal). In addition, the congruent combinations of perceived self agency and perceived other agency are linearly (\(a_{1} = -0.14\), \(p < .001\)) and curvilinearly (\(a_{2} = -0.01\), \(p < .001\)) associated with happiness, meaning that happiness is maximized with a moderate self agency and other agency (\(X = Y = -7\), \(Z = 84.79\)) and either lower or higher congruent combinations of self agency and other agency would predict lower happiness.

Table 1

Results from Response Surface Analysis Examining How the Congruence of Perceived Self Agency and Perceived Other Agency Influences Happiness

Note. SE: Standard Error, CI: Confidence Interval

So far, we have achieved specifying, fitting, visualizing, and interpreting a polynomial regression model with the RSA package for both the broad and strict sense of congruence hypothesis.

Next, we provide a comparison with the results obtained with the more conventional difference score approach.

Why RSA?

While both the absolute difference score approach and the full polynomial model using RSA indicated the existence of congruence hypothesis, RSA offers at least three advantages in examining the congruence hypothesis:

1. RSA can capture possible nonlinear associations beyond the linear assumption with difference scores;
   
2. RSA makes distinctions between the broad and strict sense of congruence, allowing for interpretations of the congruence combined with linear and/or curvilinear “main” effects of the combined predictors. The difference score approach only involves one testing about whether the congruence is statistically significant or not;
   
3. In cases of incongruence, multiple hypotheses testing in the five steps of RSA could help diagnose the source of incongruence with the visualization of the surface and facilitate further exploration of the surface (the estimated stationary point as the maximum/minimum of the outcome).