{"id":489,"date":"2024-10-18T02:02:04","date_gmt":"2024-10-18T02:02:04","guid":{"rendered":"https:\/\/pressbooks.ccconline.org\/mat1260\/?post_type=chapter&p=489"},"modified":"2024-12-11T21:22:49","modified_gmt":"2024-12-11T21:22:49","slug":"3-4-linear-regression-equation","status":"publish","type":"chapter","link":"https:\/\/pressbooks.ccconline.org\/mat1260\/chapter\/3-4-linear-regression-equation\/","title":{"raw":"3.4: Linear Regression Equation","rendered":"3.4: Linear Regression Equation"},"content":{"raw":"

Linear Regression: Summarizing the Pattern of the Data with a Line<\/span><\/h2>\r\nSo far we\u2019ve used the scatterplot to describe the relationship between two quantitative variables, and in the special case of a linear relationship, we have supplemented the scatterplot with the correlation (r). The correlation, however, doesn\u2019t fully characterize the linear relationship between two quantitative variables\u2014it only measures the strength and direction. We often want to describe more precisely how one variable changes with the other (by \u201cmore precisely,\u201d we mean more than just the direction), or\u00a0predict\u00a0<\/em>the value of the response variable for a given value of the explanatory variable. In order to be able to do that, we need to summarize the linear relationship with a line that best fits the linear pattern of the data. In the remainder of this section, we will introduce a way to find such a line, learn how to interpret it, and use it (cautiously) to make predictions.\r\n\r\nAgain, let\u2019s start with a motivating example:\r\n\r\nEarlier, we examined the linear relationship between the age of a driver and the maximum distance at which a highway sign was legible, using both a scatterplot and the correlation coefficient. Suppose a government agency wanted to predict the maximum distance at which the sign would be legible for 60-year-old drivers, and thus make sure that the sign could be used safely and effectively.\r\n

How would we make this prediction?<\/p>\r\nhttps:\/\/youtube.com\/watch?v=8hf3dMf59cI\r\n

\r\n
\r\n
How and why did we pick this particular line (the one shown in red in the above walkthrough) to describe the dependence of the maximum distance at which a sign is legible upon the age of a driver? What line exactly did we choose? We will return to this example once we can answer that question with a bit more precision.<\/span><\/div>\r\n<\/div>\r\n<\/div>\r\nThe technique that specifies the dependence of the response variable on the explanatory variable is called\u00a0regression<\/em>. When that dependence is linear (which is the case in our examples in this section), the technique is called\u00a0linear regression<\/em>. Linear regression is therefore the technique of finding the line that best fits the pattern of the linear relationship (or in other words, the line that best describes how the response variable linearly depends on the explanatory variable).\r\n

To understand how such a line is chosen, consider the following very simplified version of the age-distance example (we left just 6 of the drivers on the scatterplot):\"The<\/span><\/span><\/p>\r\nThere are many lines that look like they would be good candidates to be the line that best fits the data:\"The<\/span><\/span>\r\n\r\nIt is doubtful that everyone would select the same line in the plot above. We need to agree on what we mean by \u201cbest fits the data\u201d; in other words, we need to agree on a criterion by which we would select this line. We want the line we choose to be close to the data points. In other words, whatever criterion we choose, it had better somehow take into account the vertical deviations of the data points from the line, which are marked with blue arrows in the plot below:\"The<\/span><\/span>\r\n

The most commonly used criterion is called the\u00a0least squares<\/em>\u00a0criterion. This criterion says: Among all the lines that look good on your data, choose the one that has the smallest sum of squared vertical deviations. Visually, each squared deviation is represented by the area of one of the squares in the plot below. Therefore, we are looking for the line that will have the smallest total yellow area.\"The<\/span><\/span>This line is called the\u00a0least-squares regression line<\/em>, and, as we\u2019ll see, it fits the linear pattern of the data very well.<\/p>\r\n

For the remainder of this lesson, you\u2019ll need to feel comfortable with the algebra of a straight line. In particular you\u2019ll need to be familiar with the\u00a0slope\u00a0<\/em>and the\u00a0intercept\u00a0<\/em>in the equation of a line, and their interpretation.<\/p>\r\n

Like any other line, the equation of the least-squares regression line for summarizing the linear relationship between the response variable(Y) and the explanatory variable (X) has the form:\u00a0Y<\/span><\/span>\u00a0<\/span><\/span>=<\/span><\/span>\u00a0<\/span><\/span>a<\/span><\/span>\u00a0<\/span><\/span>+<\/span><\/span>\u00a0<\/span><\/span>b<\/span><\/span>X<\/span><\/span><\/span><\/span><\/span><\/p>\r\n

All we need to do is calculate the intercept\u00a0a<\/em>, and the slope\u00a0b<\/em>, which is easily done if we know:<\/p>\r\n\r\n

    \r\n \t
  • \r\n

    \u00af<\/span>\u00af<\/span>\u00af<\/span><\/span><\/span><\/span>X<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>\u2014the mean of the explanatory variable\u2019s values<\/p>\r\n<\/li>\r\n \t

  • \r\n

    SX<\/sub>\u2014the standard deviation of the explanatory variable\u2019s values<\/p>\r\n<\/li>\r\n \t

  • \r\n

    \u00af<\/span>\u00af<\/span>\u00af<\/span><\/span><\/span><\/span>Y<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>\u2014the mean of the response variable\u2019s values<\/p>\r\n<\/li>\r\n \t

  • \r\n

    SY<\/sub>\u2014the standard deviation of the response variable\u2019s values<\/p>\r\n<\/li>\r\n \t

  • \r\n

    r\u2014the correlation coefficient<\/p>\r\n<\/li>\r\n<\/ul>\r\n

    Given the five quantities above, the slope and intercept of the least squares regression line are found using the following formulas:<\/p>\r\n[latex]\\mathcal{b}=\\mathcal{r}\\left(\\frac{\\mathcal{S}_\\mathcal{y}}{\\mathcal{S}_\\mathcal{x}}\\right)\\\\\r\n\\mathcal{a}=\\bar{\\mathcal{Y}}-\\mathcal{b}\\bar{\\mathcal{X}}[\/latex]\r\n

    \r\n
    \r\n

    Comments<\/span><\/h2>\r\n
      \r\n \t
    1. \r\n

      Note that since the formula for the intercept\u00a0a<\/em>\u00a0depends on the value of the slope,\u00a0b<\/em>, you need to find\u00a0b<\/em>\u00a0first.<\/p>\r\n<\/li>\r\n \t

    2. \r\n

      The slope of the least squares regression line can be interpreted as the average change in the response variable when the explanatory variable increases by 1 unit.<\/p>\r\n<\/li>\r\n<\/ol>\r\n<\/div>\r\n<\/div>\r\n

      \r\n
      \r\n
      \r\n

      Example<\/h3>\r\n<\/header>\r\n
      \r\n

      Age-Distance<\/h4>\r\n
      \r\n

      Let\u2019s revisit our age-distance example, and find the\u00a0least-squares regression line<\/em>. The following output will be helpful in getting the 5 values we need:<\/p>\r\n\r\n

      \r\n
      \r\n\r\n\"Output<\/span><\/span>\r\n
        \r\n \t
      • \r\n

        The\u00a0slope\u00a0<\/em>of the line is [latex]\\mathcal{b}=\\left(-0.793\\right)*\\left(\\frac{82.8}{21.78}\\right)=-3[\/latex]. This means that for every 1-unit increase of the explanatory variable, there is, on average, a 3-unit decrease in the response variable. The interpretation\u00a0in context<\/em>\u00a0of the slope being -3 is, therefore: For every year a driver gets older, the maximum distance at which he\/she can read a sign decreases,\u00a0on average<\/em>, by 3 feet.<\/p>\r\n<\/li>\r\n \t

      • \r\n

        The\u00a0intercept<\/em>\u00a0of the line is\u00a0a<\/span><\/span>=<\/span><\/span>423<\/span><\/span>\u2212<\/span><\/span>(<\/span><\/span>\u22123<\/span><\/span>\u2217<\/span><\/span>51<\/span><\/span>)<\/span><\/span>=<\/span><\/span>576<\/span><\/span><\/span><\/span><\/span><\/span>\u00a0and therefore the\u00a0least-squares regression line<\/em>\u00a0for this example is<\/p>\r\n\r\n\r\n\r\n\r\n
        Distance = 576 + (\u22123 * Age)<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n<\/li>\r\n<\/ul>\r\n<\/div>\r\n<\/div>\r\n

        Here is the regression line plotted on the scatterplot:\"The<\/span><\/span>As we can see, the regression line fits the linear pattern of the data quite well.<\/p>\r\n\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n

        \r\n
        \r\n

        Comment<\/span><\/h2>\r\n

        As we mentioned before, hand-calculation is not the focus of this course. We wanted you to see one example in which the least squares regression line is calculated by hand, but in general we\u2019ll let a statistics package do that for us.<\/p>\r\n\r\n\r\n

        \r\n

        Let\u2019s go back now to our motivating example, in which we wanted to predict the maximum distance at which a sign is legible for a 60-year-old. Now that we have found the least squares regression line, this prediction becomes quite easy:\"The<\/span><\/span><\/p>\r\nPractically, what the figure tells us is that in order to find the predicted legibility distance for a 60-year-old, we plug Age = 60 into the regression line equation, to find that:\r\n\r\n\r\n\r\n
        Predicted distance = 576 + (- 3 * 60) = 396<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n396 feet is our best prediction for the maximum distance at which a sign is legible for a 60-year-old.\r\n
        \r\n

        Did I get this?<\/h3>\r\n<\/header>\r\n
        \r\n

        Background:\u00a0<\/em>A statistics department is interested in tracking the progress of its students from entry until graduation. As part of the study, the department tabulates the performance of 10 students in an introductory course and in an upper-level course required for graduation. The scatterplot below includes the least squares line (the line that best explains the upper-level course average based on the lower-level course average), and its equation:<\/p>\r\n\r\n

        \"The<\/div>\r\n
        [h5p id=\"49\"]<\/div>\r\n<\/div>\r\n<\/div>\r\n
        \r\n
        \r\n

        Comment About Predictions<\/span><\/h2>\r\n

        Suppose a government agency wanted to design a sign appropriate for an even wider range of drivers than were present in the original study. They want to predict the maximum distance at which the sign would be legible for a 90-year-old. Using the least squares regression line again as our summary of the linear dependence of the distances upon the drivers' ages, the agency predicts that 90-year-old drivers can see the sign at no more than 576 + (- 3 * 90) = 306 feet:\"<\/span><\/span><\/p>\r\n

        (The green segment of the line is the region of ages beyond 82, the age of the oldest individual in the study.)<\/p>\r\n\r\n

        \r\n
        Question:\u00a0<\/em>Is our prediction for 90-year-old drivers reliable?<\/div>\r\n
        <\/div>\r\n
        Answer:\u00a0<\/em>Our original age data ranged from 18 (youngest driver) to 82 (oldest driver), and our regression line is therefore a summary of the linear relationship\u00a0in that age range only.\u00a0<\/em>When we plug the value 90 into the regression line equation, we are assuming that the same linear relationship extends beyond the range of our age data (18-82) into the green segment.\u00a0There is no justification for such an assumption.<\/em>\u00a0It might be the case that the vision of drivers older than 82 falls off more rapidly than it does for younger drivers. (i.e., the slope changes from -3 to something more negative). Our prediction for age = 90 is therefore\u00a0not reliable.<\/em><\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n
        \r\n
        \r\n

        In General<\/span><\/h2>\r\n

        Prediction for ranges of the explanatory variable that are not in the data is called\u00a0extrapolation<\/em>. Since there is no way of knowing whether a relationship holds beyond the range of the explanatory variable in the data, extrapolation is not reliable, and should be avoided. In our example, like most others, extrapolation can lead to very poor or illogical predictions.<\/p>\r\n\r\n

        Let\u2019s Summarize<\/span><\/h2>\r\n
          \r\n \t
        • A special case of the relationship between two quantitative variables is the\u00a0linear\u00a0<\/em>relationship. In this case, a straight line simply and adequately summarizes the relationship.<\/li>\r\n \t
        • When the scatterplot displays a linear relationship, we supplement it with the\u00a0correlation coefficient (r)<\/em>, which measures the\u00a0strength<\/em>\u00a0and direction of a linear relationship between two quantitative variables. The correlation ranges between -1 and 1. Values near -1 indicate a strong negative linear relationship, values near 0 indicate a weak linear relationship, and values near 1 indicate a strong positive linear relationship.<\/li>\r\n \t
        • The correlation is only an appropriate numerical measure for linear relationships, and is sensitive to outliers. Therefore, the correlation should only be used as a supplement to a scatterplot (after we look at the data).<\/li>\r\n \t
        • The most commonly used criterion for finding a line that summarizes the pattern of a linear relationship is \u201cleast squares.\u201d The\u00a0least squares regression line\u00a0<\/em>has the smallest sum of squared vertical deviations of the data points from the line.<\/li>\r\n \t
        • The slope of the least squares regression line can be interpreted as the average change in the response variable when the explanatory variable increases by 1 unit.<\/li>\r\n \t
        • The least squares regression line predicts the value of the response variable for a given value of the explanatory variable.\u00a0Extrapolation<\/em>\u00a0is prediction of values of the explanatory variable that fall outside the range of the data. Since there is no way of knowing whether a relationship holds beyond the range of the explanatory variable in the data, extrapolation is not reliable, and should be avoided.<\/li>\r\n<\/ul>\r\n

          Causation<\/h2>\r\nSo far we have discussed different ways in which data can be used to explore the relationship (or association) between two variables. To frame our discussion we followed the role-type classification table:\"It<\/span><\/span>\r\n\r\nand we have now completed learning how to explore the relationship in cases C\u2192Q, C\u2192C, and Q\u2192Q. (As noted before, case Q\u2192C will not be discussed in this course.) When we explore the relationship between two variables, there is often a temptation to conclude from the observed relationship that changes in the explanatory variable\u00a0cause<\/em>\u00a0changes in the response variable. In other words, you might be tempted to interpret the observed association as causation. The purpose of this part of the course is to convince you that this kind of interpretation is often\u00a0wrong!<\/em>\u00a0The motto of this section is one of the most fundamental principles of this course:\r\n\r\n\r\n\r\n\r\n\r\n
          Principle<\/th>\r\n<\/tr>\r\n<\/thead>\r\n
          \r\n
          \r\n
          \r\n\r\nAssociation\u00a0does not<\/em>\u00a0imply causation!\r\n\r\n<\/div>\r\n<\/div><\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n
          Let\u2019s start by looking at the following example:<\/div>\r\n
          \r\n

          Example<\/h3>\r\n<\/header>\r\n
          \r\n
          \r\n

          Fire Damage<\/h4>\r\n
          \r\n
          \r\n

          The scatterplot below illustrates how the number of firefighters sent to fires (X) is related to the amount of damage caused by fires (Y) in a certain city.\"A<\/span><\/span><\/p>\r\n

          The scatterplot clearly displays a fairly strong (slightly curved)\u00a0positive<\/em>\u00a0relationship between the two variables. Would it, then, be reasonable to conclude that sending more firefighters to a fire causes more damage, or that the city should send fewer firefighters to a fire, in order to decrease the amount of damage done by the fire? Of course not! So what is going on here?<\/p>\r\nThere is a\u00a0third variable in the background<\/em>\u2014the seriousness of the fire\u2014that is responsible for the observed relationship. More serious fires require more firefighters, and also cause more damage.\r\n\r\nThe following figure will help you visualize this situation:\"A<\/span><\/span>\r\n

          Here, the seriousness of the fire is a\u00a0lurking variable.\u00a0<\/em>A\u00a0lurking variable<\/em>\u00a0is a variable that is not among the explanatory or response variables in a study, but could substantially affect your interpretation of the relationship among those variables.<\/p>\r\n\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n

          In particular, as in our example, the lurking variable might have an effect on\u00a0both<\/em>\u00a0the explanatory and the response variables. This common effect creates the observed association between the explanatory and response variables, even though there is no causal link between them. This possibility, that there might be a lurking variable (which we might not be thinking about) that is responsible for the observed relationship leads to our principle:<\/p>\r\n\r\n\r\n\r\n\r\n\r\n\r\n
          Principle<\/th>\r\n<\/tr>\r\n<\/thead>\r\n
          \r\n
          \r\n
          \r\n\r\nAssociation\u00a0does not<\/em>\u00a0imply causation!\r\n\r\n<\/div>\r\n<\/div><\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\nThe next example will illustrate another way in which a lurking variable might interfere and prevent us from reaching any causal conclusions.\r\n
          \r\n
          \r\n
          \r\n

          Example<\/h3>\r\n<\/header>\r\n
          \r\n

          SAT Test<\/h4>\r\n
          \r\n

          For U.S. colleges and universities, a standard entrance examination is the SAT test. The side-by-side boxplots below provide evidence of a relationship between the student\u2019s country of origin (the United States or another country) and the student\u2019s SAT Math score.<\/p>\r\n\"A<\/span><\/span>\r\n

          The distribution of international students\u2019 scores is higher than that of U.S. students. The international students\u2019 median score (about 700) exceeds the third quartile of U.S. students\u2019 scores. Can we conclude that the country of origin is the\u00a0cause<\/em>\u00a0of the difference in SAT Math scores, and that students in the United States are weaker at math than students in other countries?<\/p>\r\nNo, not necessarily. While it\u00a0might<\/em>\u00a0be true that U.S. students differ in math ability from other students\u2014i.e. due to differences in educational systems\u2014we can\u2019t conclude that a student\u2019s country of origin is the cause of the disparity. One important\u00a0lurking variable<\/em>\u00a0that might explain the observed relationship is the educational level of the two populations taking the SAT Math test. In the United States, the SAT is a standard test, and therefore a broad cross-section of all U.S. students (in terms of educational level) take this test. Among all international students, on the other hand, only those who plan on coming to the U.S. to study, which is usually a more selected subgroup, take the test.\r\n\r\nThe following figure will help you visualize this explanation:\r\n\r\n\"A<\/span><\/span>\r\n

          Here, the explanatory variable (X)\u00a0may<\/em>\u00a0have a causal relationship with the response variable (Y), but the lurking variable might be a contributing factor as well, which makes it very hard to isolate the effect of the explanatory variable and prove that it has a causal link with the response variable. In this case, we say that the lurking variable is\u00a0confounded<\/em> with the explanatory variable, since their effects on the response variable cannot be distinguished from each other.<\/p>\r\n\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\nNote that in each of the above two examples, the lurking variable interacts differently with the variables studied. In example 1, the lurking variable has an effect on both the explanatory and the response variables, creating the illusion that there is a causal link between them. In example two, the lurking variable is confounded with the explanatory variable, making it hard to assess the isolated effect of the explanatory variable on the response variable.\r\n\r\nThe distinction between these two types of interactions is not as important as the fact that in either case, the observed association can be at least partially explained by the lurking variable. The most important message from these two examples is therefore:\u00a0An observed association between two variables is not enough evidence that there is a <\/em>causal relationship between them.<\/em>\r\n

          In other words \u2026<\/p>\r\n\r\n\r\n\r\n\r\n\r\n\r\n
          Principle<\/th>\r\n<\/tr>\r\n<\/thead>\r\n
          \r\n
          \r\n
          \r\n

          Association\u00a0does not<\/em>\u00a0imply causation!<\/p>\r\n\r\n<\/div>\r\n<\/div><\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n

          \r\n
          \r\n

          Did I get this?<\/h3>\r\n<\/header>\r\n
          \r\n\r\n[h5p id=\"50\"]\r\n\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n

          So far, we have:<\/p>\r\n\r\n

            \r\n \t
          • discussed what lurking variables are,<\/li>\r\n \t
          • demonstrated different ways in which the lurking variables can interact with the two studied variables, and<\/li>\r\n \t
          • understood that the existence of a possible lurking variable is the main reason why we say that association does not imply causation.<\/li>\r\n<\/ul>\r\nAs you recall, a lurking variable, by definition, is a variable that was not included in the study, but could have a substantial effect on our understanding of the relationship between the two studied variables.\r\n\r\nWhat if we\u00a0did<\/em>\u00a0include a lurking variable in our study? What kind of effect could that have on our understanding of the relationship? These are the questions we are going to discuss next.\r\n\r\nLet\u2019s start with an example:\r\n
            \r\n

            Example<\/h3>\r\n<\/header>\r\n
            \r\n
            \r\n
            \r\n

            Hospital Death Rates<\/h4>\r\n
            \r\n

            Background:<\/em>\u00a0A government study collected data on the death rates in nearly 6,000 hospitals in the United States. These results were then challenged by researchers, who said that the federal analyses failed to take into account the variation among hospitals in the severity of patients\u2019 illnesses when they were hospitalized. As a result, said the researchers, some hospitals were treated unfairly in the findings, which named hospitals with higher-than-expected death rates. What the researchers meant is that when the federal government explored the relationship between the two variables\u2014hospital and death rate\u2014it also should have included in the study (or taken into account) the lurking variable\u2014severity of illness.<\/em><\/p>\r\nWe will use a simplified version of this study to illustrate the researchers\u2019 claim, and see what the possible effect could be of including a lurking variable in a study. (Reference: Moore and McCabe (2003).\u00a0Introduction to the Practice of Statistics<\/em>.)\r\n

            Consider the following two-way table, which summarizes the data about the status of patients who were admitted to two hospitals in a certain city (Hospital A and Hospital B). Note that since the purpose of the study is to examine whether there is a \u201chospital effect\u201d on patients\u2019 status, \u201cHospital is the explanatory variable, and \u201cPatient\u2019s Status\u201d is the response variable.\"A<\/span><\/span><\/p>\r\n

            When we supplement the two-way table with the conditional percents within each hospital:\"A<\/span><\/span><\/p>\r\nwe find that Hospital A has a higher death rate (3%) than Hospital B (2%). Should we jump to the conclusion that a sick patient admitted to Hospital A is 50% more likely to die than if he\/she were admitted to Hospital B?\u00a0Not so fast \u2026<\/em>\r\n

            Maybe Hospital A gets most of the severe cases, and that explains why it has a higher death rate. In order to explore this, we need to\u00a0include (or account for) the lurking variable \u201cseverity of illness\u201d in our analysis.<\/em>\u00a0To do this, we go back to the two-way table and split it up to look separately at patents who are severely ill, and patients who are not.\"The<\/span><\/span><\/p>\r\n

            As we can see, Hospital A\u00a0did<\/em>\u00a0admit many more severely ill patients than Hospital B (1,500 vs. 200). In fact, from the way the totals were split, we see that in Hospital A, severely ill patients were a much higher proportion of the patients\u20141,500 out of a total of 2,100 patients. In contrast, only 200 out of 800 patients at Hospital B were severely ill. To better see the effect of including the lurking variable, we need to supplement each of the two new two-way tables with its conditional percentages:\"Two<\/span><\/span><\/p>\r\n

            Note that despite our earlier finding that overall Hospital A has a higher death rate (3% vs. 2%), when we take into account the lurking variable, we find that actually it is Hospital B that has the higher death rate both among the severely ill patients (4% vs. 3.8%) and among the not severely ill patients (1.3% vs. 1%).\u00a0Thus, we see that adding a lurking variable can change the direction of an association.<\/em><\/p>\r\n

            Whenever including a lurking variable causes us to rethink the direction of an association, this is called\u00a0Simpson\u2019s paradox.<\/em><\/p>\r\n

            The possibility that a lurking variable can have such a dramatic effect is another reason we must adhere to the principle:<\/p>\r\n\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n\r\n\r\n\r\n\r\n\r\n
            Principle<\/th>\r\n<\/tr>\r\n<\/thead>\r\n
            \r\n
            \r\n
            \r\n

            Association\u00a0does not<\/em>\u00a0imply causation!<\/p>\r\n\r\n<\/div>\r\n<\/div><\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n<\/div>\r\n<\/div>\r\nIt is\u00a0not<\/em> always the case that including a lurking variable makes us rethink the direction of the association. In the next example we will see how including a lurking variable just helps us gain a deeper understanding of the observed relationship.\r\n

            \r\n
            \r\n
            \r\n

            Example<\/h3>\r\n<\/header>\r\n
            \r\n

            College Entrance Exams<\/h4>\r\n
            \r\n\r\nAs discussed earlier, in the United States, the SAT is a widely used college entrance examination, required by the most prestigious schools. In some states, a different college entrance examination is prevalent, the ACT.\r\n\r\nhttps:\/\/youtube.com\/watch?v=Nnj1YlqzkX4\r\n
            \r\n
            \r\n
            <\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\nThe last two examples showed us that including a lurking variable in our exploration may\r\n
              \r\n \t
            • Lead us to rethink the direction of an association (as in the Hospital\/Death Rate example).<\/li>\r\n \t
            • Help us to gain a deeper understanding of the relationship between variables (as in the SAT\/ACT example)<\/li>\r\n<\/ul>\r\n
              \r\n

              Did I get this?<\/h3>\r\n<\/header>\r\n
              \r\n\r\n[h5p id=\"51\"]\r\n\r\n<\/div>\r\n<\/div>\r\n

              Let\u2019s Summarize<\/span><\/h2>\r\n
                \r\n \t
              • A\u00a0lurking variable<\/em>\u00a0is a variable that was not included in your analysis, but that could substantially change your interpretation of the data if it were included.<\/li>\r\n \t
              • Because of the possibility of lurking variables, we adhere to the principle that\u00a0association does not imply causation<\/em>.<\/li>\r\n \t
              • Including a lurking variable in our exploration may:\r\n
                  \r\n \t
                • Help us to gain a deeper understanding of the relationship between variables.<\/li>\r\n \t
                • Lead us to rethink the direction of an association.<\/li>\r\n<\/ul>\r\n<\/li>\r\n \t
                • Whenever including a lurking variable causes us to rethink the direction of an association, this is an instance of\u00a0Simpson\u2019s paradox<\/em>.<\/li>\r\n<\/ul>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>","rendered":"

                  Linear Regression: Summarizing the Pattern of the Data with a Line<\/span><\/h2>\n

                  So far we\u2019ve used the scatterplot to describe the relationship between two quantitative variables, and in the special case of a linear relationship, we have supplemented the scatterplot with the correlation (r). The correlation, however, doesn\u2019t fully characterize the linear relationship between two quantitative variables\u2014it only measures the strength and direction. We often want to describe more precisely how one variable changes with the other (by \u201cmore precisely,\u201d we mean more than just the direction), or\u00a0predict\u00a0<\/em>the value of the response variable for a given value of the explanatory variable. In order to be able to do that, we need to summarize the linear relationship with a line that best fits the linear pattern of the data. In the remainder of this section, we will introduce a way to find such a line, learn how to interpret it, and use it (cautiously) to make predictions.<\/p>\n

                  Again, let\u2019s start with a motivating example:<\/p>\n

                  Earlier, we examined the linear relationship between the age of a driver and the maximum distance at which a highway sign was legible, using both a scatterplot and the correlation coefficient. Suppose a government agency wanted to predict the maximum distance at which the sign would be legible for 60-year-old drivers, and thus make sure that the sign could be used safely and effectively.<\/p>\n

                  How would we make this prediction?<\/p>\n