class: center, middle, inverse, title-slide # 3.3 — Omitted Variable Bias ## ECON 480 • Econometrics • Fall 2021 ### Ryan Safner<br> Assistant Professor of Economics <br> <a href="mailto:safner@hood.edu"><i class="fa fa-paper-plane fa-fw"></i>safner@hood.edu</a> <br> <a href="https://github.com/ryansafner/metricsF21"><i class="fa fa-github fa-fw"></i>ryansafner/metricsF21</a><br> <a href="https://metricsF21.classes.ryansafner.com"> <i class="fa fa-globe fa-fw"></i>metricsF21.classes.ryansafner.com</a><br> --- # Review: u .pull-left[ .smallest[ `$$Y_i=\beta_0+\beta_1X_i+u_i$$` - `\(u_i\)` includes .hi-purple[all other variables that affect `\\(Y\\)`] - Every regression model always has .hi[omitted variables] assumed in the error - Most are unobservable (hence “u”) - .green[**Examples**:] innate ability, weather at the time, etc - Again, we *assume* `\(u\)` is random, with `\(E[u|X]=0\)` and `\(var(u)=\sigma^2_u\)` - *Sometimes*, omission of variables can **bias** OLS estimators `\((\hat{\beta_0}\)` and `\(\hat{\beta_1})\)` ] ] .pull-right[ .center[  ] ] --- # Omitted Variable Bias I .pull-left[ - .hi[Omitted variable bias (OVB)] for some omitted variable `\(\mathbf{Z}\)` exists if two conditions are met: ] .pull-right[ ] --- # Omitted Variable Bias I .pull-left[ - .hi[Omitted variable bias (OVB)] for some omitted variable `\(\mathbf{Z}\)` exists if two conditions are met: **1. `\(Z\)` is a determinant of `\(Y\)`** - i.e. `\(Z\)` is in the error term, `\(u_i\)` ] .pull-right[ <!-- --> ] --- # Omitted Variable Bias I .pull-left[ - .hi[Omitted variable bias (OVB)] for some omitted variable `\(\mathbf{Z}\)` exists if two conditions are met: **1. `\(Z\)` is a determinant of `\(Y\)`** - i.e. `\(Z\)` is in the error term, `\(u_i\)` **2. `\(Z\)` is correlated with the regressor `\(X\)`** - i.e. `\(cor(X,Z) \neq 0\)` - implies `\(cor(X,u) \neq 0\)` - implies .hi-purple[X is endogenous] ] .pull-right[ <!-- --> ] --- # Omitted Variable Bias II .pull-left[ - Omitted variable bias makes `\(X\)` .hi-purple[endogenous] - Violates **zero conditional mean assumption** `$$E(u_i|X_i)\neq 0 \implies$$` - knowing `\(X_i\)` tells you something about `\(u_i\)` (i.e. something about `\(Y\)` *not* by way of `\(X\)`)! ] .pull-right[ <!-- --> ] --- # Omitted Variable Bias III .pull-left[ - `\(\hat{\beta_1}\)` is .hi-purple[biased]: `\(E[\hat{\beta_1}] \neq \beta_1\)` - `\(\hat{\beta_1}\)` systematically over- or under-estimates the true relationship `\((\beta_1)\)` - `\(\hat{\beta_1}\)` “picks up” *both* pathways: 1. `\(X\rightarrow Y\)` 2. `\(X \leftarrow Z\rightarrow Y\)` ] .pull-right[ <!-- --> ] --- # Omited Variable Bias: Class Size Example .bg-washed-green.b--dark-green.ba.bw2.br3.shadow-5.ph4.mt5[ .green[**Example**]: Consider our recurring class size and test score example: `$$\text{Test score}_i = \beta_0 + \beta_1 \text{STR}_i + u_i$$` ] - Which of the following possible variables would cause a bias if omitted? -- 1. `\(Z_i\)`: time of day of the test -- 2. `\(Z_i\)`: parking space per student -- 3. `\(Z_i\)`: percent of ESL students --- # Recall: Endogeneity and Bias .smaller[ - (Recall): the true expected value of `\(\hat{\beta_1}\)` is actually:^{.magenta[†]} `$$E[\hat{\beta_1}]=\beta_1+cor(X,u)\frac{\sigma_u}{\sigma_X}$$` ] -- .smallest[ 1) If `\(X\)` is exogenous: `\(cor(X,u)=0\)`, we're just left with `\(\beta_1\)` ] -- .smallest[ 2) The larger `\(cor(X,u)\)` is, larger .hi-purple[bias]: `\(\left(E[\hat{\beta_1}]-\beta_1 \right)\)` ] -- .smallest[ 3) We can .hi-purple[“sign”] the direction of the bias based on `\(cor(X,u)\)` - .hi-purple[Positive] `\(cor(X,u)\)` overestimates the true `\(\beta_1\)` `\((\hat{\beta_1}\)` is too large) - .hi-purple[Negative] `\(cor(X,u)\)` underestimates the true `\(\beta_1\)` `\((\hat{\beta_1}\)` is too small) ] .footnote[.quitesmall[ ^{.magenta[†]} See [2.4 class notes](/content/2.4-content) for proof.] ] --- # Endogeneity and Bias: Correlations I - Here is where checking correlations between variables helps: .pull-left[ .code50[ ```r # Select only the three variables we want (there are many) CAcorr <- CASchool %>% select("str","testscr","el_pct") # Make a correlation table cor_table <- cor(CAcorr) cor_table # look at it ``` ``` ## str testscr el_pct ## str 1.0000000 -0.2263628 0.1876424 ## testscr -0.2263628 1.0000000 -0.6441237 ## el_pct 0.1876424 -0.6441237 1.0000000 ``` ] ] .pull-right[ - `el_pct` is strongly (negatively) correlated with `testscr` (Condition 1) - `el_pct` is reasonably (positively) correlated with `str` (Condition 2) ] --- # Endogeneity and Bias: Correlations II - Here is where checking correlations between variables helps: .pull-left[ ```r # Make a correlation plot library(corrplot) corrplot(cor_table, type="upper", method = "circle", order="original") ``` ] .pull-right[ <img src="3.3-slides_files/figure-html/unnamed-chunk-6-1.png" width="504" /> ] - `el_pct` is strongly correlated with `testscr` (Condition 1) - `el_pct` is reasonably correlated with `str` (Condition 2) --- # Look at Conditional Distributions I .smallest[ .code50[ ```r # make a new variable called EL # = high (if el_pct is above median) or = low (if below median) CASchool <- CASchool %>% # next we create a new dummy variable called ESL mutate(ESL = ifelse(el_pct > median(el_pct), # test if ESL is above median yes = "High ESL", # if yes, call this variable "High ESL" no = "Low ESL")) # if no, call this variable "Low ESL" # get average test score by high/low EL CASchool %>% group_by(ESL) %>% summarize(Average_test_score = mean(testscr)) ``` <div data-pagedtable="false"> <script data-pagedtable-source type="application/json"> {"columns":[{"label":["ESL"],"name":[1],"type":["chr"],"align":["left"]},{"label":["Average_test_score"],"name":[2],"type":["dbl"],"align":["right"]}],"data":[{"1":"High ESL","2":"643.9591"},{"1":"Low ESL","2":"664.3540"}],"options":{"columns":{"min":{},"max":[10]},"rows":{"min":[10],"max":[10]},"pages":{}}} </script> </div> ] ] --- # Look at Conditional Distributions II .pull-left[ .code50[ ```r ggplot(data = CASchool)+ aes(x = testscr, fill = ESL)+ geom_density(alpha=0.5)+ labs(x = "Test Score", y = "Density")+ ggthemes::theme_pander( base_family = "Fira Sans Condensed", base_size=20 )+ theme(legend.position = "bottom") ``` ] ] .pull-right[ <img src="3.3-slides_files/figure-html/unnamed-chunk-8-1.png" width="504" /> ] --- # Look at Conditional Distributions III .pull-left[ .code50[ ```r esl_scatter <- ggplot(data = CASchool)+ aes(x = str, y = testscr, color = ESL)+ geom_point()+ geom_smooth(method = "lm")+ labs(x = "STR", y = "Test Score")+ ggthemes::theme_pander( base_family = "Fira Sans Condensed", base_size=20 )+ theme(legend.position = "bottom") esl_scatter ``` ] ] .pull-right[ <img src="3.3-slides_files/figure-html/unnamed-chunk-9-1.png" width="504" /> ] --- # Look at Conditional Distributions III .pull-left[ ```r esl_scatter+ * facet_grid(~ESL)+ * guides(color = F) ``` ] .pull-right[ <img src="3.3-slides_files/figure-html/unnamed-chunk-10-1.png" width="504" /> ] --- # Omitted Variable Bias in the Class Size Example .center[ `$$E[\hat{\beta_1}]=\beta_1+bias$$` `\(E[\hat{\beta_1}]=\)` .red[`\\(\beta_1\\)`] `\(+\)` .blue[`\\(cor(X,u)\\)`] `\(\frac{\sigma_u}{\sigma_X}\)` ] - .blue[`\\(cor(STR,u)\\)`] is positive (via `\(\%EL\)`) - `\(cor(u, \text{Test score})\)` is negative (via `\(\%EL\)`) - .red[`\\(\beta_1\\)`] is negative (between Test score and STR) - .blue[Bias] is positive - But since `\(\color{red}{\beta_1}\)` is negative, it’s made to be a *larger* negative number than it truly is - Implies that `\(\color{red}{\beta_1}\)` *over*states the effect of reducing STR on improving Test Scores --- # Omitted Variable Bias: Messing with Causality I - If school districts with higher Test Scores happen to have both lower STR **AND** districts with smaller `\(STR\)` sizes tend to have less `\(\%EL\)` ... -- - How can we say `\(\hat{\beta_1}\)` estimates the .hi[marginal effect] of `\(\Delta STR \rightarrow \Delta \text{Test Score}\)`? - (We can’t.) --- # Omitted Variable Bias: Messing with Causality II .pull-left[ - Consider an ideal .hi-turquoise[random controlled trial (RCT)] - .hi-turquoise[Randomly] assign experimental units (e.g. people, cities, etc) into two (or more) groups: - .hi[Treatment group(s)]: gets a (certain type or level of) treatment - .hi-purple[Control group(s)]: gets *no* treatment(s) - Compare results of two groups to get .hi-slate[average treatment effect] ] .pull-right[ .center[  ] ] --- # RCTs Neutralize Omitted Variable Bias I .bg-washed-green.b--dark-green.ba.bw2.br3.shadow-5.ph4.mt5[ .green[**Example**]: Imagine an ideal RCT for measuring the effect of STR on Test Score ] .pull-left[ - School districts would be .hi-turquoise[randomly assigned] a student-teacher ratio - With random assignment, all factors in `\(u\)` (%ESL students, family size, parental income, years in the district, day of the week of the test, climate, etc) are distributed *independently* of class size ] .pull-right[ .center[  ] ] --- # RCTs Neutralize Omitted Variable Bias II .bg-washed-green.b--dark-green.ba.bw2.br3.shadow-5.ph4.mt5[ .green[**Example**]: Imagine an ideal RCT for measuring the effect of STR on Test Score ] .pull-left[ - Thus, `\(cor(STR, u)=0\)` and `\(E[u|STR]=0\)`, i.e. .hi-purple[exogeneity] - Our `\(\hat{\beta_1}\)` would be an .hi-turquoise[unbiased estimate] of `\(\beta_1\)`, measuring the .hi-slate[true causal effect] of STR `\(\rightarrow\)` Test Score ] .pull-right[ .center[  ] ] --- # But We Rarely, if Ever, Have RCTs .pull-left[ .smallest[ - But we **didn't** run an RCT, it's observational data! - “Treatment” of having a large or small class size is **NOT** randomly assigned! - `\(\%EL\)`: plausibly fits criteria of O.V. bias! 1. `\(\%EL\)` is a determinant of Test Score 2. `\(\%EL\)` is correlated with STR - Thus, “control” group and “treatment” group differ systematically! - Small STR also tend to have lower `\(\%EL\)`; large STR also tend to have higher `\(\%EL\)` - .hi-orange[Selection bias]: `\(cor(STR, \%EL) \neq 0\)`, `\(E[u_i|STR_i]\neq 0\)` ] ] .pull-right[ .pull-left[ .center[  Treatment Group ] ] .pull-right[ .center[  Control Group ] ] ] --- # Another Way to Control for Variables .pull-left[ - Pathways connecting str and test score: 1. str `\(\rightarrow\)` test score 2. str `\(\leftarrow\)` ESL `\(\rightarrow\)` testscore ] .pull-right[ <!-- --> ] --- # Another Way to Control for Variables .pull-left[ - Pathways connecting str and test score: 1. str `\(\rightarrow\)` test score 2. str `\(\leftarrow\)` ESL `\(\rightarrow\)` testscore - DAG rules tell us we need to .hi-purple[control for ESL] in order to identify the causal effect of str `\(\rightarrow\)` test score - So now, .hi-turquoise[how *do* we control for a variable]? ] .pull-right[ <!-- --> ] --- # Controlling for Variables .pull-left[ - Look at effect of STR on Test Score by comparing districts with the **same** %EL - Eliminates differences in %EL between high and low STR classes - “As if” we had a control group! Hold %EL constant - The simple fix is just to .hi-purple[not omit %EL]! - Make it *another* independent variable on the righthand side of the regression ] .pull-right[ .pull-left[ .center[  Treatment Group ] ] .pull-right[ .center[  Control Group ] ] ] --- # Controlling for Variables .pull-left[ - Look at effect of STR on Test Score by comparing districts with the **same** %EL - Eliminates differences in %EL between high and low STR classes - “As if” we had a control group! Hold %EL constant - The simple fix is just to .hi-purple[not omit %EL]! - Make it *another* independent variable on the righthand side of the regression ] .pull-right[ <!-- --> ] --- class: inverse, center, middle # The Multivariate Regression Model --- # Multivariate Econometric Models Overview .smallest[ `$$Y_i = \beta_0 + \beta_1 X_{1i} + \beta_2 X_{2i} + \cdots + \beta_kX_{ki} +u_i$$` ] -- .smallest[ - `\(Y\)` is the .hi[dependent variable] of interest - AKA "response variable," "regressand," "Left-hand side (LHS) variable" ] -- .smallest[ - `\(X_1\)` and `\(X_2\)` are .hi[independent variables] - AKA "explanatory variables", "regressors," "Right-hand side (RHS) variables", "covariates" ] -- .smallest[ - Our data consists of a spreadsheet of observed values of `\((X_{1i}, X_{2i}, Y_i)\)` ] -- .smallest[ - To model, we .hi-turquoise["regress `\\(Y\\)` on `\\(X_1\\)` and `\\(X_2\\)`"] ] -- .smallest[ - `\(\beta_0, \beta_1, \cdots, \beta_k\)` are .hi-purple[parameters] that describe the population relationships between the variables - We estimate `\(k+1\)` parameters (“betas”)^{.magenta[†]} ] .footnote[^{.magenta[†]} Note Bailey defines k to include both the number of variables plus the constant.] --- # Marginal Effects I `$$Y_i= \beta_0+\beta_1 X_{1i} + \beta_2 X_{2i}$$` - Consider changing `\(X_1\)` by `\(\Delta X_1\)` while holding `\(X_2\)` constant: `$$\begin{align*} Y&= \beta_0+\beta_1 X_{1} + \beta_2 X_{2} && \text{Before the change}\\ \end{align*}$$` --- # Marginal Effects I `$$Y_i= \beta_0+\beta_1 X_{1i} + \beta_2 X_{2i}$$` - Consider changing `\(X_1\)` by `\(\Delta X_1\)` while holding `\(X_2\)` constant: `$$\begin{align*} Y&= \beta_0+\beta_1 X_{1} + \beta_2 X_{2} && \text{Before the change}\\ Y+\Delta Y&= \beta_0+\beta_1 (X_{1}+\Delta X_1) + \beta_2 X_{2} && \text{After the change}\\ \end{align*}$$` --- # Marginal Effects I `$$Y_i= \beta_0+\beta_1 X_{1i} + \beta_2 X_{2i}$$` - Consider changing `\(X_1\)` by `\(\Delta X_1\)` while holding `\(X_2\)` constant: `$$\begin{align*} Y&= \beta_0+\beta_1 X_{1} + \beta_2 X_{2} && \text{Before the change}\\ Y+\Delta Y&= \beta_0+\beta_1 (X_{1}+\Delta X_1) + \beta_2 X_{2} && \text{After the change}\\ \Delta Y&= \beta_1 \Delta X_1 && \text{The difference}\\ \end{align*}$$` --- # Marginal Effects I `$$Y_i= \beta_0+\beta_1 X_{1i} + \beta_2 X_{2i}$$` - Consider changing `\(X_1\)` by `\(\Delta X_1\)` while holding `\(X_2\)` constant: `$$\begin{align*} Y&= \beta_0+\beta_1 X_{1} + \beta_2 X_{2} && \text{Before the change}\\ Y+\Delta Y&= \beta_0+\beta_1 (X_{1}+\Delta X_1) + \beta_2 X_{2} && \text{After the change}\\ \Delta Y&= \beta_1 \Delta X_1 && \text{The difference}\\ \frac{\Delta Y}{\Delta X_1} &= \beta_1 && \text{Solving for } \beta_1\\ \end{align*}$$` --- # Marginal Effects II `$$\beta_1 =\frac{\Delta Y}{\Delta X_1}\text{ holding } X_2 \text{ constant}$$` -- Similarly, for `\(\beta_2\)`: `$$\beta_2 =\frac{\Delta Y}{\Delta X_2}\text{ holding }X_1 \text{ constant}$$` -- And for the constant, `\(\beta_0\)`: `$$\beta_0 =\text{predicted value of Y when } X_1=0, \; X_2=0$$` --- # You Can Keep Your Intuitions...But They're Wrong Now .pull-left[ - We have been envisioning OLS regressions as the equation of a line through a scatterplot of data on two variables, `\(X\)` and `\(Y\)` - `\(\beta_0\)`: “intercept” - `\(\beta_1\)`: “slope” - With 3+ variables, OLS regression is no longer a “line” for us to estimate... ] .pull-right[ <div id="htmlwidget-1a746e01b3f3b85ada95" style="width:504px;height:504px;" class="plotly html-widget"></div> <script type="application/json" data-for="htmlwidget-1a746e01b3f3b85ada95">{"x":{"visdat":{"c2ac2485476e":["function () ","plotlyVisDat"]},"cur_data":"c2ac2485476e","attrs":{"c2ac2485476e":{"x":{},"y":{},"z":{},"alpha_stroke":1,"sizes":[10,100],"spans":[1,20],"type":"scatter3d","mode":"markers","alpha":0.75,"inherit":true},"c2ac2485476e.1":{"x":[14,15,16,17,18,19,20,21,22,23,24,25],"y":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85],"z":[[670.614106240256,669.512810351389,668.411514462521,667.310218573654,666.208922684786,665.107626795918,664.006330907051,662.905035018183,661.803739129316,660.702443240448,659.60114735158,658.499851462713],[669.964329463603,668.863033574736,667.761737685868,666.660441797001,665.559145908133,664.457850019265,663.356554130398,662.25525824153,661.153962352663,660.052666463795,658.951370574927,657.85007468606],[669.31455268695,668.213256798083,667.111960909215,666.010665020347,664.90936913148,663.808073242612,662.706777353745,661.605481464877,660.504185576009,659.402889687142,658.301593798274,657.200297909407],[668.664775910297,667.563480021429,666.462184132562,665.360888243694,664.259592354827,663.158296465959,662.057000577091,660.955704688224,659.854408799356,658.753112910489,657.651817021621,656.550521132753],[668.014999133644,666.913703244776,665.812407355909,664.711111467041,663.609815578173,662.508519689306,661.407223800438,660.305927911571,659.204632022703,658.103336133835,657.002040244968,655.9007443561],[667.365222356991,666.263926468123,665.162630579256,664.061334690388,662.96003880152,661.858742912653,660.757447023785,659.656151134918,658.55485524605,657.453559357182,656.352263468315,655.250967579447],[666.715445580338,665.61414969147,664.512853802602,663.411557913735,662.310262024867,661.208966136,660.107670247132,659.006374358264,657.905078469397,656.803782580529,655.702486691662,654.601190802794],[666.065668803685,664.964372914817,663.863077025949,662.761781137082,661.660485248214,660.559189359347,659.457893470479,658.356597581611,657.255301692744,656.154005803876,655.052709915008,653.951414026141],[665.415892027031,664.314596138164,663.213300249296,662.112004360429,661.010708471561,659.909412582693,658.808116693826,657.706820804958,656.605524916091,655.504229027223,654.402933138355,653.301637249488],[664.766115250378,663.664819361511,662.563523472643,661.462227583775,660.360931694908,659.25963580604,658.158339917173,657.057044028305,655.955748139437,654.85445225057,653.753156361702,652.651860472835],[664.116338473725,663.015042584857,661.91374669599,660.812450807122,659.711154918255,658.609859029387,657.50856314052,656.407267251652,655.305971362784,654.204675473917,653.103379585049,652.002083696182],[663.466561697072,662.365265808204,661.263969919337,660.162674030469,659.061378141601,657.960082252734,656.858786363866,655.757490474999,654.656194586131,653.554898697264,652.453602808396,651.352306919528],[662.816784920419,661.715489031551,660.614193142684,659.512897253816,658.411601364948,657.310305476081,656.209009587213,655.107713698346,654.006417809478,652.90512192061,651.803826031743,650.702530142875],[662.167008143766,661.065712254898,659.96441636603,658.863120477163,657.761824588295,656.660528699428,655.55923281056,654.457936921692,653.356641032825,652.255345143957,651.15404925509,650.052753366222],[661.517231367113,660.415935478245,659.314639589377,658.21334370051,657.112047811642,656.010751922775,654.909456033907,653.808160145039,652.706864256172,651.605568367304,650.504272478437,649.402976589569],[660.867454590459,659.766158701592,658.664862812724,657.563566923857,656.462271034989,655.360975146122,654.259679257254,653.158383368386,652.057087479519,650.955791590651,649.854495701783,648.753199812916],[660.217677813806,659.116381924939,658.015086036071,656.913790147203,655.812494258336,654.711198369468,653.609902480601,652.508606591733,651.407310702866,650.306014813998,649.20471892513,648.103423036263],[659.567901037153,658.466605148286,657.365309259418,656.26401337055,655.162717481683,654.061421592815,652.960125703948,651.85882981508,650.757533926212,649.656238037345,648.554942148477,647.45364625961],[658.9181242605,657.816828371632,656.715532482765,655.614236593897,654.51294070503,653.411644816162,652.310348927294,651.209053038427,650.107757149559,649.006461260692,647.905165371824,646.803869482956],[658.268347483847,657.167051594979,656.065755706112,654.964459817244,653.863163928376,652.761868039509,651.660572150641,650.559276261774,649.457980372906,648.356684484039,647.255388595171,646.154092706303],[657.618570707194,656.517274818326,655.415978929458,654.314683040591,653.213387151723,652.112091262856,651.010795373988,649.90949948512,648.808203596253,647.706907707385,646.605611818518,645.50431592965],[656.968793930541,655.867498041673,654.766202152805,653.664906263938,652.56361037507,651.462314486203,650.361018597335,649.259722708467,648.1584268196,647.057130930732,645.955835041865,644.854539152997],[656.319017153888,655.21772126502,654.116425376152,653.015129487285,651.913833598417,650.81253770955,649.711241820682,648.609945931814,647.508650042947,646.407354154079,645.306058265212,644.204762376344],[655.669240377234,654.567944488367,653.466648599499,652.365352710632,651.264056821764,650.162760932896,649.061465044029,647.960169155161,646.858873266294,645.757577377426,644.656281488558,643.554985599691],[655.019463600581,653.918167711714,652.816871822846,651.715575933978,650.614280045111,649.512984156243,648.411688267376,647.310392378508,646.209096489641,645.107800600773,644.006504711905,642.905208823038],[654.369686823928,653.26839093506,652.167095046193,651.065799157325,649.964503268458,648.86320737959,647.761911490722,646.660615601855,645.559319712987,644.45802382412,643.356727935252,642.255432046384],[653.719910047275,652.618614158407,651.51731826954,650.416022380672,649.314726491805,648.213430602937,647.112134714069,646.010838825202,644.909542936334,643.808247047467,642.706951158599,641.605655269731],[653.070133270622,651.968837381754,650.867541492887,649.766245604019,648.664949715151,647.563653826284,646.462357937416,645.361062048549,644.259766159681,643.158470270813,642.057174381946,640.955878493078],[652.420356493969,651.319060605101,650.217764716233,649.116468827366,648.015172938498,646.913877049631,645.812581160763,644.711285271895,643.609989383028,642.50869349416,641.407397605293,640.306101716425],[651.770579717316,650.669283828448,649.56798793958,648.466692050713,647.365396161845,646.264100272978,645.16280438411,644.061508495242,642.960212606375,641.858916717507,640.75762082864,639.656324939772],[651.120802940662,650.019507051795,648.918211162927,647.81691527406,646.715619385192,645.614323496324,644.513027607457,643.411731718589,642.310435829722,641.209139940854,640.107844051986,639.006548163119],[650.471026164009,649.369730275142,648.268434386274,647.167138497407,646.065842608539,644.964546719671,643.863250830804,642.761954941936,641.660659053069,640.559363164201,639.458067275333,638.356771386466],[649.821249387356,648.719953498489,647.618657609621,646.517361720753,645.416065831886,644.314769943018,643.213474054151,642.112178165283,641.010882276415,639.909586387548,638.80829049868,637.706994609813],[649.171472610703,648.070176721835,646.968880832968,645.8675849441,644.766289055233,643.664993166365,642.563697277497,641.46240138863,640.361105499762,639.259809610895,638.158513722027,637.057217833159],[648.52169583405,647.420399945182,646.319104056315,645.217808167447,644.116512278579,643.015216389712,641.913920500844,640.812624611977,639.711328723109,638.610032834241,637.508736945374,636.407441056506],[647.871919057397,646.770623168529,645.669327279662,644.568031390794,643.466735501926,642.365439613059,641.264143724191,640.162847835324,639.061551946456,637.960256057588,636.858960168721,635.757664279853],[647.222142280744,646.120846391876,645.019550503008,643.918254614141,642.816958725273,641.715662836406,640.614366947538,639.51307105867,638.411775169803,637.310479280935,636.209183392068,635.1078875032],[646.57236550409,645.471069615223,644.369773726355,643.268477837488,642.16718194862,641.065886059753,639.964590170885,638.863294282017,637.76199839315,636.660702504282,635.559406615414,634.458110726547],[645.922588727437,644.82129283857,643.719996949702,642.618701060835,641.517405171967,640.416109283099,639.314813394232,638.213517505364,637.112221616497,636.010925727629,634.909629838761,633.808333949894],[645.272811950784,644.171516061917,643.070220173049,641.968924284181,640.867628395314,639.766332506446,638.665036617579,637.563740728711,636.462444839843,635.361148950976,634.259853062108,633.158557173241],[644.623035174131,643.521739285263,642.420443396396,641.319147507528,640.217851618661,639.116555729793,638.015259840926,636.913963952058,635.81266806319,634.711372174323,633.610076285455,632.508780396588],[643.973258397478,642.87196250861,641.770666619743,640.669370730875,639.568074842007,638.46677895314,637.365483064272,636.264187175405,635.162891286537,634.06159539767,632.960299508802,631.859003619934],[643.323481620825,642.222185731957,641.12088984309,640.019593954222,638.918298065354,637.817002176487,636.715706287619,635.614410398752,634.513114509884,633.411818621016,632.310522732149,631.209226843281],[642.673704844172,641.572408955304,640.471113066436,639.369817177569,638.268521288701,637.167225399834,636.065929510966,634.964633622098,633.863337733231,632.762041844363,631.660745955496,630.559450066628],[642.023928067519,640.922632178651,639.821336289783,638.720040400916,637.618744512048,636.517448623181,635.416152734313,634.314856845445,633.213560956578,632.11226506771,631.010969178843,629.909673289975],[641.374151290865,640.272855401998,639.17155951313,638.070263624263,636.968967735395,635.867671846528,634.76637595766,633.665080068792,632.563784179925,631.462488291057,630.361192402189,629.259896513322],[640.724374514212,639.623078625345,638.521782736477,637.420486847609,636.319190958742,635.217895069874,634.116599181007,633.015303292139,631.914007403272,630.812711514404,629.711415625536,628.610119736669],[640.074597737559,638.973301848692,637.872005959824,636.770710070956,635.669414182089,634.568118293221,633.466822404354,632.365526515486,631.264230626618,630.162934737751,629.061638848883,627.960342960016],[639.424820960906,638.323525072038,637.222229183171,636.120933294303,635.019637405436,633.918341516568,632.8170456277,631.715749738833,630.614453849965,629.513157961098,628.41186207223,627.310566183362],[638.775044184253,637.673748295385,636.572452406518,635.47115651765,634.369860628782,633.268564739915,632.167268851047,631.06597296218,629.964677073312,628.863381184445,627.762085295577,626.660789406709],[638.1252674076,637.023971518732,635.922675629864,634.821379740997,633.720083852129,632.618787963262,631.517492074394,630.416196185526,629.314900296659,628.213604407791,627.112308518924,626.011012630056],[637.475490630947,636.374194742079,635.272898853211,634.171602964344,633.070307075476,631.969011186609,630.867715297741,629.766419408873,628.665123520006,627.563827631138,626.462531742271,625.361235853403],[636.825713854294,635.724417965426,634.623122076558,633.521826187691,632.420530298823,631.319234409956,630.217938521088,629.11664263222,628.015346743353,626.914050854485,625.812754965618,624.71145907675],[636.17593707764,635.074641188773,633.973345299905,632.872049411038,631.77075352217,630.669457633302,629.568161744435,628.466865855567,627.3655699667,626.264274077832,625.162978188964,624.061682300097],[635.526160300987,634.42486441212,633.323568523252,632.222272634384,631.120976745517,630.019680856649,628.918384967782,627.817089078914,626.715793190046,625.614497301179,624.513201412311,623.411905523444],[634.876383524334,633.775087635466,632.673791746599,631.572495857731,630.471199968864,629.369904079996,628.268608191128,627.167312302261,626.066016413393,624.964720524526,623.863424635658,622.76212874679],[634.226606747681,633.125310858813,632.024014969946,630.922719081078,629.821423192211,628.720127303343,627.618831414475,626.517535525608,625.41623963674,624.314943747873,623.213647859005,622.112351970137],[633.576829971028,632.47553408216,631.374238193293,630.272942304425,629.171646415557,628.07035052669,626.969054637822,625.867758748955,624.766462860087,623.665166971219,622.563871082352,621.462575193484],[632.927053194375,631.825757305507,630.724461416639,629.623165527772,628.521869638904,627.420573750037,626.319277861169,625.217981972301,624.116686083434,623.015390194566,621.914094305699,620.812798416831],[632.277276417722,631.175980528854,630.074684639986,628.973388751119,627.872092862251,626.770796973384,625.669501084516,624.568205195648,623.466909306781,622.365613417913,621.264317529046,620.163021640178],[631.627499641068,630.526203752201,629.424907863333,628.323611974466,627.222316085598,626.12102019673,625.019724307863,623.918428418995,622.817132530128,621.71583664126,620.614540752392,619.513244863525],[630.977722864415,629.876426975548,628.77513108668,627.673835197813,626.572539308945,625.471243420077,624.36994753121,623.268651642342,622.167355753475,621.066059864607,619.964763975739,618.863468086872],[630.327946087762,629.226650198895,628.125354310027,627.024058421159,625.922762532292,624.821466643424,623.720170754557,622.618874865689,621.517578976821,620.416283087954,619.314987199086,618.213691310219],[629.678169311109,628.576873422241,627.475577533374,626.374281644506,625.272985755639,624.171689866771,623.070393977903,621.969098089036,620.867802200168,619.766506311301,618.665210422433,617.563914533565],[629.028392534456,627.927096645588,626.825800756721,625.724504867853,624.623208978985,623.521913090118,622.42061720125,621.319321312383,620.218025423515,619.116729534647,618.01543364578,616.914137756912],[628.378615757803,627.277319868935,626.176023980067,625.0747280912,623.973432202332,622.872136313465,621.770840424597,620.66954453573,619.568248646862,618.466952757994,617.365656869127,616.264360980259],[627.72883898115,626.627543092282,625.526247203414,624.424951314547,623.323655425679,622.222359536812,621.121063647944,620.019767759076,618.918471870209,617.817175981341,616.715880092474,615.614584203606],[627.079062204496,625.977766315629,624.876470426761,623.775174537894,622.673878649026,621.572582760159,620.471286871291,619.369990982423,618.268695093556,617.167399204688,616.06610331582,614.964807426953],[626.429285427843,625.327989538976,624.226693650108,623.125397761241,622.024101872373,620.922805983505,619.821510094638,618.72021420577,617.618918316903,616.517622428035,615.416326539167,614.3150306503],[625.77950865119,624.678212762323,623.576916873455,622.475620984587,621.37432509572,620.273029206852,619.171733317985,618.070437429117,616.969141540249,615.867845651382,614.766549762514,613.665253873647],[625.129731874537,624.028435985669,622.927140096802,621.825844207934,620.724548319067,619.623252430199,618.521956541332,617.420660652464,616.319364763596,615.218068874729,614.116772985861,613.015477096994],[624.479955097884,623.378659209016,622.277363320149,621.176067431281,620.074771542413,618.973475653546,617.872179764678,616.770883875811,615.669587986943,614.568292098076,613.466996209208,612.36570032034],[623.830178321231,622.728882432363,621.627586543496,620.526290654628,619.42499476576,618.323698876893,617.222402988025,616.121107099158,615.01981121029,613.918515321422,612.817219432555,611.715923543687],[623.180401544578,622.07910565571,620.977809766842,619.876513877975,618.775217989107,617.67392210024,616.572626211372,615.471330322505,614.370034433637,613.268738544769,612.167442655902,611.066146767034],[622.530624767925,621.429328879057,620.328032990189,619.226737101322,618.125441212454,617.024145323587,615.922849434719,614.821553545851,613.720257656984,612.618961768116,611.517665879249,610.416369990381],[621.880847991271,620.779552102404,619.678256213536,618.576960324669,617.475664435801,616.374368546934,615.273072658066,614.171776769198,613.070480880331,611.969184991463,610.867889102595,609.766593213728],[621.231071214618,620.129775325751,619.028479436883,617.927183548015,616.825887659148,615.72459177028,614.623295881413,613.521999992545,612.420704103678,611.31940821481,610.218112325942,609.116816437075],[620.581294437965,619.479998549098,618.37870266023,617.277406771362,616.176110882495,615.074814993627,613.97351910476,612.872223215892,611.770927327024,610.669631438157,609.568335549289,608.467039660422],[619.931517661312,618.830221772444,617.728925883577,616.627629994709,615.526334105842,614.425038216974,613.323742328106,612.222446439239,611.121150550371,610.019854661504,608.918558772636,607.817262883768],[619.281740884659,618.180444995791,617.079149106924,615.977853218056,614.876557329188,613.775261440321,612.673965551453,611.572669662586,610.471373773718,609.370077884851,608.268781995983,607.167486107115],[618.631964108006,617.530668219138,616.429372330271,615.328076441403,614.226780552535,613.125484663668,612.0241887748,610.922892885933,609.821596997065,608.720301108197,607.61900521933,606.517709330462],[617.982187331353,616.880891442485,615.779595553617,614.67829966475,613.577003775882,612.475707887015,611.374411998147,610.273116109279,609.171820220412,608.070524331544,606.969228442677,605.867932553809],[617.3324105547,616.231114665832,615.129818776964,614.028522888097,612.927226999229,611.825931110362,610.724635221494,609.623339332626,608.522043443759,607.420747554891,606.319451666023,605.218155777156],[616.682633778046,615.581337889179,614.480042000311,613.378746111444,612.277450222576,611.176154333708,610.074858444841,608.973562555973,607.872266667106,606.770970778238,605.66967488937,604.568379000503],[616.032857001393,614.931561112526,613.830265223658,612.72896933479,611.627673445923,610.526377557055,609.425081668188,608.32378577932,607.222489890452,606.121194001585,605.019898112717,603.91860222385],[615.38308022474,614.281784335872,613.180488447005,612.079192558137,610.97789666927,609.876600780402,608.775304891534,607.674009002667,606.572713113799,605.471417224932,604.370121336064,603.268825447196]],"alpha_stroke":1,"sizes":[10,100],"spans":[1,20],"type":"surface","inherit":true}},"layout":{"margin":{"b":40,"l":60,"t":25,"r":10},"scene":{"xaxis":{"title":"Class Size"},"yaxis":{"title":"Percent EL"},"zaxis":{"title":"Test Scores"}},"hovermode":"closest","showlegend":false,"legend":{"yanchor":"top","y":0.5}},"source":"A","config":{"showSendToCloud":false},"data":[{"x":[17.8899097442627,21.5246639251709,18.6972255706787,17.3571434020996,18.671329498291,21.40625,19.5,20.8941173553467,19.9473686218262,20.8055553436279,21.238094329834,21,20.6000003814697,20.0082168579102,18.0277786254883,20.2519607543945,16.9778690338135,16.5098037719727,22.7040233612061,19.911111831665,18.3333339691162,22.619047164917,19.4482765197754,25.0526313781738,20.6754379272461,18.68235206604,22.8455295562744,19.2666664123535,19.25,20.5454540252686,20.6069660186768,21.072681427002,21.5358142852783,19.9039993286133,21.1940689086914,21.8653545379639,18.3296451568604,16.2285709381104,19.1785717010498,20.2773666381836,22.9861373901367,20.4444446563721,19.8208465576172,23.2052230834961,19.2669677734375,23.301887512207,21.1882858276367,20.8717956542969,19.0174942016602,21.9193801879883,20.1012382507324,21.4765110015869,20.065788269043,20.3750953674316,22.4464817047119,22.8952388763428,20.4979705810547,20,22.2565803527832,21.5643634796143,19.4773712158203,17.6700210571289,21.9475612640381,21.7833938598633,19.1399993896484,18.1104965209961,20.6824245452881,22.623607635498,21.7864990234375,18.5829315185547,21.5454540252686,21.1528911590576,16.6333332061768,21.1443824768066,19.7818183898926,18.9837284088135,17.667667388916,17.7549915313721,15.2727270126343,14,20.596134185791,16.3116874694824,21.1279621124268,17.4880123138428,17.8867931365967,19.3067588806152,20.8923072814941,21.2868385314941,20.1955986022949,24.9500007629395,18.1304340362549,20,18.7295093536377,18.25,18.9925689697266,19.8876419067383,19.3789482116699,20.4625854492188,22.2915725708008,20.7047386169434,19.0600528717041,20.2324695587158,19.6901226043701,20.3625392913818,19.754222869873,19.3797664642334,22.9235095977783,19.3733978271484,19.1551551818848,21.2999992370605,18.3035717010498,21.0792560577393,18.7912082672119,19.6266174316406,19.5901641845703,20.8718719482422,21.1150016784668,20.0845203399658,19.9104881286621,17.8128509521484,18.1333332061768,19.2222118377686,18.6607151031494,19.6000003814697,19.283842086792,22.8181819915771,18.8092193603516,21.3736324310303,20.0204086303711,21.4986152648926,15.4285717010498,22.3999996185303,20.1270866394043,19.0379753112793,17.3421573638916,17.0186328887939,20.7999992370605,21.1538467407227,18.4583339691162,19.1408214569092,19.407657623291,19.5689640045166,21.50119972229,17.529411315918,16.4301719665527,19.7965393066406,17.1861343383789,17.615894317627,20.1253719329834,22.1666660308838,19.9615383148193,19.0394515991211,15.2243595123291,21.1447505950928,19.6438999176025,21.0486888885498,20.1754379272461,21.3913040161133,20.0083255767822,20.2913722991943,17.6666660308838,18.2205505371094,20.2710018157959,20.1989459991455,21.3842449188232,20.9736843109131,20,17.153284072876,22.3497714996338,22.1700706481934,18.1818180084229,18.9571437835693,19.7453308105469,16.4262294769287,16.6253967285156,16.3817672729492,20.0741577148438,17.9954433441162,19.3913040161133,16.4285717010498,16.7294864654541,24.4134521484375,18.2641506195068,18.955041885376,21.0389614105225,20.7407398223877,18.1000003814697,19.8461532592773,21.6000003814697,22.4424209594727,23.0143756866455,17.7489185333252,18.2866401672363,19.2654418945312,22.6666660308838,19.294116973877,17.3636360168457,19.8214282989502,20.4337787628174,21.0372085571289,19.9246234893799,19.0098571777344,23.8222217559814,19.3690853118896,19.8285713195801,15.2588548660278,17.1612911224365,21.8133335113525,19.0747127532959,25.7851238250732,18.2126140594482,18.1660633087158,16.972972869873,21.5008735656738,20.6000003814697,16.990291595459,20.7795448303223,15.5124664306641,19.8850574493408,21.3988227844238,20.4975128173828,19.3637580871582,17.659574508667,21.0179557800293,19.0556488037109,22.5384616851807,21.1078720092773,20.0513534545898,14.2017631530762,18.4768676757812,18.6354160308838,20.9459457397461,21.0854816436768,18.6928806304932,20.8680782318115,19.8255767822266,19.75,19.5,18.3908042907715,18.7867641448975,19.7701797485352,19.3333339691162,21.4639167785645,23.0849227905273,21.0629920959473,18.6868686676025,20.7702350616455,19.3055553436279,20.1328029632568,20.6696357727051,22.2815532684326,20.6002712249756,20.8273391723633,19.2249240875244,17.6547698974609,17,16.4977283477783,19.7826080322266,22.3021583557129,17.730770111084,20.4483604431152,20.3716945648193,20.1647872924805,21.6153774261475,20.5614280700684,19.9555110931396,21.1838703155518,18.8104228973389,20.5783824920654,18.3246078491211,18.8206272125244,20.8163261413574,20,19.6818180084229,19.3901767730713,20.927318572998,19.9443664550781,20.7910938262939,19.2035388946533,19.0243911743164,17.6205787658691,20.237154006958,19.2937393188477,18.8299808502197,20.3394927978516,19.2289962768555,17.8913040161133,19.5188102722168,19.0845069885254,19.9354839324951,18.8732566833496,20.1417846679688,23.5563716888428,21.4647884368896,19.191011428833,20.1308002471924,25.7999992370605,18.7777404785156,19.1098155975342,19.7010860443115,18.6159420013428,20.9972114562988,20,20.9832515716553,21.6426239013672,20.0296726226807,19.8113975524902,18,19.3581085205078,20.1791210174561,21.1198635101318,23.3897361755371,22.1818180084229,19.9428272247314,17.7882556915283,14.7058820724487,19.0407676696777,20.8919544219971,19.8385066986084,19.5219116210938,20.6862182617188,18.1818180084229,18.8922424316406,24.8888893127441,18.5806446075439,18.0400009155273,17.7339897155762,21.4545459747314,19.9234256744385,20.3394203186035,22.5460758209229,21.103443145752,18.1974258422852,20.1076812744141,19.1598358154297,19.5454540252686,20.8888893127441,18.3915023803711,19.1799030303955,19.3977069854736,21.6782722473145,19.2888889312744,20.3492736816406,20.9641609191895,19.4603900909424,19.2857151031494,20.9197940826416,20.9002132415771,20.5957450866699,19.375,19.9512233734131,18.8497333526611,18.1178703308105,19.1834087371826,22,21.5841579437256,20.3888893127441,16.2931022644043,18.2777824401855,19.3747158050537,18.9090900421143,16.406925201416,15.5913972854614,18.7069416046143,18.3298530578613,17.9023513793945,18.9115657806396,20.3249664306641,20.0245685577393,24,17.6078433990479,19.3485336303711,19.6784648895264,18.7286052703857,15.8823528289795,20.0549125671387,17.9882526397705,16.9662933349609,19.2393741607666,19.195858001709,19.5990619659424,20.543478012085,18.5884838104248,15.6041860580444,15.2930402755737,17.655366897583,17.579761505127,22.3333339691162,18.75,18.102409362793,20.2564105987549,18.802074432373,18.7723045349121,20.4052104949951,18.6507930755615,20.7070713043213,22,17.6997756958008,21.4832897186279,16.7010307312012,19.5756721496582,17.2580642700195,17.3752555847168,17.3493118286133,16.2622852325439,17.7004547119141,20.1288146972656,18.2653942108154,14.5421361923218,19.1526107788086,17.3657417297363,15.1389846801758,17.8426609039307,15.4070415496826,18.8653392791748,16.4741344451904,17.8626251220703,21.885856628418,20.2000007629395,19.0364017486572],"y":[0,4.58333349227905,30.0000019073486,0,13.8576774597168,12.4087591171265,68.7179489135742,46.9594612121582,30.0791568756104,40.2759208679199,52.9147987365723,54.6099319458008,42.7184448242188,20.5338802337646,80.1232604980469,49.413143157959,85.5397186279297,58.9073638916016,77.0058135986328,49.8139877319336,40.6818199157715,16.2105255126953,45.0748634338379,39.0756301879883,76.6652526855469,40.4911842346191,73.7202301025391,70.0115356445312,55.962215423584,11.0619468688965,80.4200897216797,63.1303558349609,65.1218566894531,53.4164009094238,49.823070526123,35.4653701782227,56.1255264282227,32.3943672180176,65.5121002197266,53.0554428100586,49.6425323486328,45.1086959838867,30.3204593658447,52.2431259155273,36.8013153076172,30.2833995819092,49.8645668029785,13.7592134475708,28.8637046813965,52.8047866821289,44.0850601196289,35.25,37.4944610595703,50.3905906677246,31.0725784301758,18.2612323760986,34.7002067565918,33.288948059082,33.4877090454102,38.1587524414062,36.9294624328613,32.9894905090332,58.2166557312012,17.0036468505859,17.6593532562256,7.32153749465942,31.1981563568115,16.6174449920654,58.0810356140137,55.8925437927246,5.48523235321045,14.4291934967041,22.8456916809082,38.7833366394043,64.2463226318359,25.225564956665,5.38243627548218,6.07902765274048,0,0,34.8891792297363,13.0573244094849,36.0251235961914,34.6534652709961,28.6919822692871,1.79533219337463,30.6332855224609,59.0334548950195,13.5593223571777,5.01001977920532,0.95923262834549,0.666666686534882,38.512035369873,0,17.1739139556885,9.88700580596924,19.3916358947754,36.4893608093262,39.3999977111816,28.6776218414307,13.6986303329468,11.3498363494873,3.44262313842773,15.4302673339844,17.973783493042,22.7177333831787,18.4362449645996,16.1717147827148,2.02212882041931,9.62441349029541,41.4634132385254,9.89903450012207,16.0818710327148,43.4949378967285,8.78661060333252,39.0243873596191,53.8903045654297,41.1332931518555,1.38328528404236,35.4430389404297,8.63970565795898,15.3497743606567,19.8564586639404,3.06122446060181,9.87318801879883,16.0956172943115,43.4989776611328,45.0379524230957,18.1957187652588,15.4876947402954,0.92592591047287,7.5892858505249,29.0858383178711,0.132978722453117,50.8576316833496,11.4963502883911,13.4615392684937,4.72727251052856,21.444694519043,12.3536806106567,30.1488838195801,15.8590297698975,43.75,0,0,16.6522121429443,3.22033905982971,0,48.5245895385742,0.751879692077637,1.92678236961365,13.5135126113892,0,16.7146053314209,5.94353628158569,36.1743774414062,44.971866607666,16.6666679382324,15.1872396469116,34.3377990722656,0,13.0673999786377,4.27807474136353,39.5946311950684,21.6683731079102,3.2622332572937,7.85714292526245,39.5744667053223,10.1278028488159,27.5217914581299,14,8.84200000762939,6.40584659576416,2.39520978927612,5.95744705200195,8.17391300201416,15.4589366912842,0,0,0,31.8245830535889,12.8009166717529,0,3.84615397453308,2.46913576126099,10.7142858505249,0,1.93798446655273,4.62962961196899,5.25059652328491,27.5403823852539,14.9825782775879,47.8793754577637,0,1.76470601558685,0,0,7.28910732269287,17.028003692627,10.4355516433716,2.26986122131348,28.2479133605957,8.76865673065186,7.49185657501221,9.79827117919922,12.5,0,31.2041549682617,1.36919319629669,9.61538505554199,0,24.2572898864746,9.55414009094238,5.92532444000244,0.970873773097992,18.2857131958008,4.9843487739563,4.28571462631226,3.35260105133057,32.7074966430664,9.46601963043213,6.26078748703003,13.2530126571655,3.16913533210754,16.9911499023438,0.853242337703705,6.64956569671631,0.116414435207844,0,6.93374443054199,9.55841255187988,0.903225779533386,1.15830111503601,40.1080169677734,16.6632118225098,21.955358505249,8.86075973510742,0.854700922966003,0,3.91389417648315,11.9494590759277,3.99274039268494,22.9394817352295,2.48563003540039,9.87149524688721,0.54054057598114,8.95663166046143,6.47481966018677,20.7254314422607,0.187265917658806,13.9869289398193,8.74172115325928,20.0345420837402,1.38339924812317,2.06270623207092,28.5714302062988,0.169491529464722,13.7362642288208,0,0.216919749975204,3.12103939056396,47.3684234619141,22.7096786499023,29.1207408905029,4.43262434005737,4.81421661376953,5.24920463562012,1.89944136142731,8.66293334960938,10.4761905670166,1.24223601818085,0,9.375,1.38568139076233,35.3421211242676,9.64071846008301,2.23152017593384,0.366403609514236,0.576036870479584,0,0.851581513881683,5.52455377578735,0,0,2.43722319602966,20.766773223877,2.18712043762207,13.6261758804321,13.2841329574585,1.94174754619598,23.0615653991699,0.621118009090424,3.25551223754883,0,2.22482442855835,18.5583763122559,6.20155048370361,18.4574813842773,32.0725517272949,3.96551704406738,15.3367071151733,27.6652278900146,12.537314414978,12.6801748275757,6.90804433822632,14.9629621505737,11.2693252563477,9.72222232818604,3.31588125228882,0.693481266498566,5.14112901687622,4.96916532516479,6.96721315383911,7.99606513977051,0.48192772269249,2.5,3.84131002426147,1.32042253017426,4.91228055953979,0,0.222386956214905,0,17.8540592193604,4.4642858505249,9.54861164093018,3.99113082885742,20.6666660308838,7.6271185874939,27.4536724090576,10.8576259613037,5.02573394775391,14.2361688613892,0.235849052667618,2.0706889629364,8.55614948272705,0,4.78723430633545,3.96039605140686,1.57935285568237,3.5939085483551,8.72622680664062,0,0.264061272144318,1.44792151451111,0.147492617368698,0,3.70500636100769,0.268528461456299,0,4.51612901687622,2.44479489326477,0,0,5.40540552139282,0,6.42201805114746,6.53950929641724,14.2857151031494,3.67879748344421,0.502176105976105,0,22.6912937164307,0,0.141643062233925,2.73348522186279,1.68350172042847,0,0.143609374761581,0,11.0465116500854,0,0,0.0633312240242958,0,0,15.0598974227905,1.47959184646606,2.64900660514832,1.47991538047791,4.46989297866821,0.326441794633865,2.53968262672424,0.54200541973114,11.5853662490845,11.3772449493408,0,1.37488543987274,2.2388060092926,5.33333349227905,2.77315592765808,0,17.7257518768311,0,0,0.425531893968582,6.06707334518433,10.1275911331177,3.37552738189697,8.6485424041748,12.3456792831421,1.58959531784058,1.4953271150589,10.3855228424072,6.13402032852173,2.2662889957428,2.47863245010376,0.893629252910614,2.42165231704712,3.76569032669067,1.96286475658417,0.578034639358521,2.80728387832642,1.36363637447357,1.16448330879211,2.05040574073792,5.99593496322632,4.72610092163086,24.2630386352539,2.97029709815979,5.00562429428101],"z":[690.799987792969,661.200012207031,643.599975585938,647.700012207031,640.849975585938,605.550048828125,606.75,609,612.5,612.650024414062,615.75,616.299987792969,616.299987792969,616.299987792969,616.450012207031,617.349975585938,618.050048828125,618.300048828125,619.799987792969,620.299987792969,620.5,621.400024414062,621.75,622.050048828125,622.599975585938,623.099975585938,623.200012207031,623.450012207031,623.599975585938,624.150024414062,624.550048828125,624.950012207031,625.299987792969,625.849975585938,626.099975585938,626.800048828125,626.900024414062,627.099975585938,627.25,627.299987792969,628.25,628.400024414062,628.550048828125,628.650024414062,628.75,629.800048828125,630.349975585938,630.400024414062,630.549987792969,630.549987792969,631.050048828125,631.400024414062,631.849975585938,631.900024414062,631.950012207031,632,632.200012207031,632.25,632.449951171875,632.849975585938,632.950012207031,633.049987792969,633.150024414062,633.650024414062,633.900024414062,634,634.050048828125,634.099975585938,634.099975585938,634.150024414062,634.199951171875,634.400024414062,634.549987792969,634.700012207031,634.900024414062,634.949951171875,635.049987792969,635.199951171875,635.450012207031,635.599975585938,635.599975585938,635.75,635.950012207031,636.099975585938,636.5,636.599975585938,636.699951171875,636.900024414062,636.950012207031,637,637.099975585938,637.349975585938,637.650024414062,637.949951171875,637.950012207031,638,638.200012207031,638.300048828125,638.300048828125,638.349975585938,638.549987792969,638.700012207031,639.25,639.300048828125,639.349975585938,639.5,639.75,639.799987792969,639.849975585938,639.900024414062,640.099975585938,640.150024414062,640.5,640.75,640.900024414062,641.099975585938,641.449951171875,641.449951171875,641.549987792969,641.800048828125,642.199951171875,642.200012207031,642.400024414062,642.75,643.049987792969,643.199951171875,643.25,643.400024414062,643.400024414062,643.5,643.5,643.699951171875,643.700012207031,644.199951171875,644.200012207031,644.400024414062,644.450012207031,644.450012207031,644.5,644.549987792969,644.699951171875,644.950012207031,645.099975585938,645.25,645.549987792969,645.550048828125,645.599975585938,645.75,645.75,646,646.200012207031,646.349975585938,646.400024414062,646.5,646.549987792969,646.700012207031,646.900024414062,646.949951171875,647.049987792969,647.25,647.299987792969,647.599975585938,647.599975585938,648,648.200012207031,648.25,648.349975585938,648.700012207031,648.949951171875,649.150024414062,649.300048828125,649.5,649.699951171875,649.849975585938,650.449951171875,650.549987792969,650.599975585938,650.650024414062,650.900024414062,650.900024414062,651.150024414062,651.200012207031,651.349975585938,651.400024414062,651.450012207031,651.800048828125,651.849975585938,651.900024414062,652,652.099975585938,652.099975585938,652.299987792969,652.300048828125,652.349975585938,652.400024414062,652.400024414062,652.5,652.849975585938,653.099975585938,653.400024414062,653.5,653.549987792969,653.550048828125,653.699951171875,653.799987792969,653.849975585938,653.949951171875,654.099975585938,654.199951171875,654.199951171875,654.299987792969,654.599975585938,654.849975585938,654.849975585938,654.900024414062,655.049987792969,655.050048828125,655.050048828125,655.199951171875,655.300048828125,655.349975585938,655.349975585938,655.400024414062,655.549987792969,655.699951171875,655.799987792969,655.849975585938,656.400024414062,656.5,656.550048828125,656.650024414062,656.700012207031,656.800048828125,656.800048828125,657,657,657.150024414062,657.400024414062,657.5,657.550048828125,657.650024414062,657.75,657.799987792969,657.900024414062,658,658.349975585938,658.599975585938,658.799987792969,659.050048828125,659.150024414062,659.349975585938,659.400024414062,659.400024414062,659.799987792969,659.900024414062,660.050048828125,660.099975585938,660.199951171875,660.299987792969,660.75,660.949951171875,661.349975585938,661.450012207031,661.599975585938,661.599975585938,661.849975585938,661.849975585938,661.849975585938,661.900024414062,661.900024414062,661.950012207031,662.400024414062,662.400024414062,662.450012207031,662.5,662.550048828125,662.550048828125,662.650024414062,662.700012207031,662.75,662.900024414062,663.349975585938,663.449951171875,663.5,663.849975585938,663.849975585938,663.900024414062,664,664,664.150024414062,664.150024414062,664.299987792969,664.400024414062,664.449951171875,664.700012207031,664.75,664.949951171875,664.950012207031,665.099975585938,665.200012207031,665.349975585938,665.650024414062,665.900024414062,665.950012207031,666,666.050048828125,666.099975585938,666.150024414062,666.150024414062,666.449951171875,666.550048828125,666.599975585938,666.650024414062,666.650024414062,666.699951171875,666.849975585938,666.849975585938,667.150024414062,667.199951171875,667.450012207031,667.450012207031,667.599975585938,668,668.099975585938,668.400024414062,668.599975585938,668.650024414062,668.799987792969,668.900024414062,668.950012207031,669.099975585938,669.300048828125,669.300048828125,669.349975585938,669.349975585938,669.799987792969,669.849975585938,669.950012207031,670,670.699951171875,671.25,671.299987792969,671.599975585938,671.599975585938,671.650024414062,671.699951171875,671.75,671.900024414062,671.900024414062,671.949951171875,672.049987792969,672.050048828125,672.299987792969,672.349975585938,672.450012207031,672.550048828125,672.699951171875,673.049987792969,673.25,673.299987792969,673.549987792969,673.549987792969,673.900024414062,674.25,675.400024414062,675.700012207031,676.150024414062,676.549987792969,676.599975585938,676.849975585938,676.949951171875,677.25,677.950012207031,678.050048828125,678.400024414062,678.799987792969,679.400024414062,679.5,679.650024414062,679.75,679.800048828125,680.050048828125,680.450012207031,681.299987792969,681.299987792969,681.599975585938,681.900024414062,682.150024414062,682.450012207031,682.549987792969,682.650024414062,683.349975585938,683.400024414062,684.300048828125,684.349975585938,684.800048828125,684.950012207031,686.050048828125,686.699951171875,687.549987792969,689.099975585938,691.049987792969,691.349975585938,691.900024414062,693.950012207031,694.25,694.800048828125,695.199951171875,695.299987792969,696.550048828125,698.199951171875,698.25,698.449951171875,699.099975585938,700.300048828125,704.300048828125,706.75,645,672.200012207031,655.75],"type":"scatter3d","mode":"markers","marker":{"color":"rgba(31,119,180,0.75)","line":{"color":"rgba(31,119,180,1)"}},"error_y":{"color":"rgba(31,119,180,0.75)"},"error_x":{"color":"rgba(31,119,180,0.75)"},"line":{"color":"rgba(31,119,180,0.75)"},"frame":null},{"colorbar":{"title":"testscr","ticklen":2,"len":0.5,"lenmode":"fraction","y":1,"yanchor":"top"},"colorscale":[["0","rgba(68,1,84,0.75)"],["0.041666666666666","rgba(70,19,97,0.75)"],["0.0833333333333338","rgba(72,32,111,0.75)"],["0.125","rgba(71,45,122,0.75)"],["0.166666666666666","rgba(68,58,128,0.75)"],["0.208333333333334","rgba(64,70,135,0.75)"],["0.25","rgba(60,82,138,0.75)"],["0.291666666666667","rgba(56,93,140,0.75)"],["0.333333333333333","rgba(49,104,142,0.75)"],["0.374999999999999","rgba(46,114,142,0.75)"],["0.416666666666667","rgba(42,123,142,0.75)"],["0.458333333333333","rgba(38,133,141,0.75)"],["0.500000000000001","rgba(37,144,140,0.75)"],["0.541666666666667","rgba(33,154,138,0.75)"],["0.583333333333333","rgba(39,164,133,0.75)"],["0.625000000000001","rgba(47,174,127,0.75)"],["0.666666666666667","rgba(53,183,121,0.75)"],["0.708333333333333","rgba(79,191,110,0.75)"],["0.75","rgba(98,199,98,0.75)"],["0.791666666666666","rgba(119,207,85,0.75)"],["0.833333333333332","rgba(147,214,70,0.75)"],["0.875","rgba(172,220,52,0.75)"],["0.916666666666666","rgba(199,225,42,0.75)"],["0.958333333333334","rgba(226,228,40,0.75)"],["1","rgba(253,231,37,0.75)"]],"showscale":true,"x":[14,15,16,17,18,19,20,21,22,23,24,25],"y":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85],"z":[[670.614106240256,669.512810351389,668.411514462521,667.310218573654,666.208922684786,665.107626795918,664.006330907051,662.905035018183,661.803739129316,660.702443240448,659.60114735158,658.499851462713],[669.964329463603,668.863033574736,667.761737685868,666.660441797001,665.559145908133,664.457850019265,663.356554130398,662.25525824153,661.153962352663,660.052666463795,658.951370574927,657.85007468606],[669.31455268695,668.213256798083,667.111960909215,666.010665020347,664.90936913148,663.808073242612,662.706777353745,661.605481464877,660.504185576009,659.402889687142,658.301593798274,657.200297909407],[668.664775910297,667.563480021429,666.462184132562,665.360888243694,664.259592354827,663.158296465959,662.057000577091,660.955704688224,659.854408799356,658.753112910489,657.651817021621,656.550521132753],[668.014999133644,666.913703244776,665.812407355909,664.711111467041,663.609815578173,662.508519689306,661.407223800438,660.305927911571,659.204632022703,658.103336133835,657.002040244968,655.9007443561],[667.365222356991,666.263926468123,665.162630579256,664.061334690388,662.96003880152,661.858742912653,660.757447023785,659.656151134918,658.55485524605,657.453559357182,656.352263468315,655.250967579447],[666.715445580338,665.61414969147,664.512853802602,663.411557913735,662.310262024867,661.208966136,660.107670247132,659.006374358264,657.905078469397,656.803782580529,655.702486691662,654.601190802794],[666.065668803685,664.964372914817,663.863077025949,662.761781137082,661.660485248214,660.559189359347,659.457893470479,658.356597581611,657.255301692744,656.154005803876,655.052709915008,653.951414026141],[665.415892027031,664.314596138164,663.213300249296,662.112004360429,661.010708471561,659.909412582693,658.808116693826,657.706820804958,656.605524916091,655.504229027223,654.402933138355,653.301637249488],[664.766115250378,663.664819361511,662.563523472643,661.462227583775,660.360931694908,659.25963580604,658.158339917173,657.057044028305,655.955748139437,654.85445225057,653.753156361702,652.651860472835],[664.116338473725,663.015042584857,661.91374669599,660.812450807122,659.711154918255,658.609859029387,657.50856314052,656.407267251652,655.305971362784,654.204675473917,653.103379585049,652.002083696182],[663.466561697072,662.365265808204,661.263969919337,660.162674030469,659.061378141601,657.960082252734,656.858786363866,655.757490474999,654.656194586131,653.554898697264,652.453602808396,651.352306919528],[662.816784920419,661.715489031551,660.614193142684,659.512897253816,658.411601364948,657.310305476081,656.209009587213,655.107713698346,654.006417809478,652.90512192061,651.803826031743,650.702530142875],[662.167008143766,661.065712254898,659.96441636603,658.863120477163,657.761824588295,656.660528699428,655.55923281056,654.457936921692,653.356641032825,652.255345143957,651.15404925509,650.052753366222],[661.517231367113,660.415935478245,659.314639589377,658.21334370051,657.112047811642,656.010751922775,654.909456033907,653.808160145039,652.706864256172,651.605568367304,650.504272478437,649.402976589569],[660.867454590459,659.766158701592,658.664862812724,657.563566923857,656.462271034989,655.360975146122,654.259679257254,653.158383368386,652.057087479519,650.955791590651,649.854495701783,648.753199812916],[660.217677813806,659.116381924939,658.015086036071,656.913790147203,655.812494258336,654.711198369468,653.609902480601,652.508606591733,651.407310702866,650.306014813998,649.20471892513,648.103423036263],[659.567901037153,658.466605148286,657.365309259418,656.26401337055,655.162717481683,654.061421592815,652.960125703948,651.85882981508,650.757533926212,649.656238037345,648.554942148477,647.45364625961],[658.9181242605,657.816828371632,656.715532482765,655.614236593897,654.51294070503,653.411644816162,652.310348927294,651.209053038427,650.107757149559,649.006461260692,647.905165371824,646.803869482956],[658.268347483847,657.167051594979,656.065755706112,654.964459817244,653.863163928376,652.761868039509,651.660572150641,650.559276261774,649.457980372906,648.356684484039,647.255388595171,646.154092706303],[657.618570707194,656.517274818326,655.415978929458,654.314683040591,653.213387151723,652.112091262856,651.010795373988,649.90949948512,648.808203596253,647.706907707385,646.605611818518,645.50431592965],[656.968793930541,655.867498041673,654.766202152805,653.664906263938,652.56361037507,651.462314486203,650.361018597335,649.259722708467,648.1584268196,647.057130930732,645.955835041865,644.854539152997],[656.319017153888,655.21772126502,654.116425376152,653.015129487285,651.913833598417,650.81253770955,649.711241820682,648.609945931814,647.508650042947,646.407354154079,645.306058265212,644.204762376344],[655.669240377234,654.567944488367,653.466648599499,652.365352710632,651.264056821764,650.162760932896,649.061465044029,647.960169155161,646.858873266294,645.757577377426,644.656281488558,643.554985599691],[655.019463600581,653.918167711714,652.816871822846,651.715575933978,650.614280045111,649.512984156243,648.411688267376,647.310392378508,646.209096489641,645.107800600773,644.006504711905,642.905208823038],[654.369686823928,653.26839093506,652.167095046193,651.065799157325,649.964503268458,648.86320737959,647.761911490722,646.660615601855,645.559319712987,644.45802382412,643.356727935252,642.255432046384],[653.719910047275,652.618614158407,651.51731826954,650.416022380672,649.314726491805,648.213430602937,647.112134714069,646.010838825202,644.909542936334,643.808247047467,642.706951158599,641.605655269731],[653.070133270622,651.968837381754,650.867541492887,649.766245604019,648.664949715151,647.563653826284,646.462357937416,645.361062048549,644.259766159681,643.158470270813,642.057174381946,640.955878493078],[652.420356493969,651.319060605101,650.217764716233,649.116468827366,648.015172938498,646.913877049631,645.812581160763,644.711285271895,643.609989383028,642.50869349416,641.407397605293,640.306101716425],[651.770579717316,650.669283828448,649.56798793958,648.466692050713,647.365396161845,646.264100272978,645.16280438411,644.061508495242,642.960212606375,641.858916717507,640.75762082864,639.656324939772],[651.120802940662,650.019507051795,648.918211162927,647.81691527406,646.715619385192,645.614323496324,644.513027607457,643.411731718589,642.310435829722,641.209139940854,640.107844051986,639.006548163119],[650.471026164009,649.369730275142,648.268434386274,647.167138497407,646.065842608539,644.964546719671,643.863250830804,642.761954941936,641.660659053069,640.559363164201,639.458067275333,638.356771386466],[649.821249387356,648.719953498489,647.618657609621,646.517361720753,645.416065831886,644.314769943018,643.213474054151,642.112178165283,641.010882276415,639.909586387548,638.80829049868,637.706994609813],[649.171472610703,648.070176721835,646.968880832968,645.8675849441,644.766289055233,643.664993166365,642.563697277497,641.46240138863,640.361105499762,639.259809610895,638.158513722027,637.057217833159],[648.52169583405,647.420399945182,646.319104056315,645.217808167447,644.116512278579,643.015216389712,641.913920500844,640.812624611977,639.711328723109,638.610032834241,637.508736945374,636.407441056506],[647.871919057397,646.770623168529,645.669327279662,644.568031390794,643.466735501926,642.365439613059,641.264143724191,640.162847835324,639.061551946456,637.960256057588,636.858960168721,635.757664279853],[647.222142280744,646.120846391876,645.019550503008,643.918254614141,642.816958725273,641.715662836406,640.614366947538,639.51307105867,638.411775169803,637.310479280935,636.209183392068,635.1078875032],[646.57236550409,645.471069615223,644.369773726355,643.268477837488,642.16718194862,641.065886059753,639.964590170885,638.863294282017,637.76199839315,636.660702504282,635.559406615414,634.458110726547],[645.922588727437,644.82129283857,643.719996949702,642.618701060835,641.517405171967,640.416109283099,639.314813394232,638.213517505364,637.112221616497,636.010925727629,634.909629838761,633.808333949894],[645.272811950784,644.171516061917,643.070220173049,641.968924284181,640.867628395314,639.766332506446,638.665036617579,637.563740728711,636.462444839843,635.361148950976,634.259853062108,633.158557173241],[644.623035174131,643.521739285263,642.420443396396,641.319147507528,640.217851618661,639.116555729793,638.015259840926,636.913963952058,635.81266806319,634.711372174323,633.610076285455,632.508780396588],[643.973258397478,642.87196250861,641.770666619743,640.669370730875,639.568074842007,638.46677895314,637.365483064272,636.264187175405,635.162891286537,634.06159539767,632.960299508802,631.859003619934],[643.323481620825,642.222185731957,641.12088984309,640.019593954222,638.918298065354,637.817002176487,636.715706287619,635.614410398752,634.513114509884,633.411818621016,632.310522732149,631.209226843281],[642.673704844172,641.572408955304,640.471113066436,639.369817177569,638.268521288701,637.167225399834,636.065929510966,634.964633622098,633.863337733231,632.762041844363,631.660745955496,630.559450066628],[642.023928067519,640.922632178651,639.821336289783,638.720040400916,637.618744512048,636.517448623181,635.416152734313,634.314856845445,633.213560956578,632.11226506771,631.010969178843,629.909673289975],[641.374151290865,640.272855401998,639.17155951313,638.070263624263,636.968967735395,635.867671846528,634.76637595766,633.665080068792,632.563784179925,631.462488291057,630.361192402189,629.259896513322],[640.724374514212,639.623078625345,638.521782736477,637.420486847609,636.319190958742,635.217895069874,634.116599181007,633.015303292139,631.914007403272,630.812711514404,629.711415625536,628.610119736669],[640.074597737559,638.973301848692,637.872005959824,636.770710070956,635.669414182089,634.568118293221,633.466822404354,632.365526515486,631.264230626618,630.162934737751,629.061638848883,627.960342960016],[639.424820960906,638.323525072038,637.222229183171,636.120933294303,635.019637405436,633.918341516568,632.8170456277,631.715749738833,630.614453849965,629.513157961098,628.41186207223,627.310566183362],[638.775044184253,637.673748295385,636.572452406518,635.47115651765,634.369860628782,633.268564739915,632.167268851047,631.06597296218,629.964677073312,628.863381184445,627.762085295577,626.660789406709],[638.1252674076,637.023971518732,635.922675629864,634.821379740997,633.720083852129,632.618787963262,631.517492074394,630.416196185526,629.314900296659,628.213604407791,627.112308518924,626.011012630056],[637.475490630947,636.374194742079,635.272898853211,634.171602964344,633.070307075476,631.969011186609,630.867715297741,629.766419408873,628.665123520006,627.563827631138,626.462531742271,625.361235853403],[636.825713854294,635.724417965426,634.623122076558,633.521826187691,632.420530298823,631.319234409956,630.217938521088,629.11664263222,628.015346743353,626.914050854485,625.812754965618,624.71145907675],[636.17593707764,635.074641188773,633.973345299905,632.872049411038,631.77075352217,630.669457633302,629.568161744435,628.466865855567,627.3655699667,626.264274077832,625.162978188964,624.061682300097],[635.526160300987,634.42486441212,633.323568523252,632.222272634384,631.120976745517,630.019680856649,628.918384967782,627.817089078914,626.715793190046,625.614497301179,624.513201412311,623.411905523444],[634.876383524334,633.775087635466,632.673791746599,631.572495857731,630.471199968864,629.369904079996,628.268608191128,627.167312302261,626.066016413393,624.964720524526,623.863424635658,622.76212874679],[634.226606747681,633.125310858813,632.024014969946,630.922719081078,629.821423192211,628.720127303343,627.618831414475,626.517535525608,625.41623963674,624.314943747873,623.213647859005,622.112351970137],[633.576829971028,632.47553408216,631.374238193293,630.272942304425,629.171646415557,628.07035052669,626.969054637822,625.867758748955,624.766462860087,623.665166971219,622.563871082352,621.462575193484],[632.927053194375,631.825757305507,630.724461416639,629.623165527772,628.521869638904,627.420573750037,626.319277861169,625.217981972301,624.116686083434,623.015390194566,621.914094305699,620.812798416831],[632.277276417722,631.175980528854,630.074684639986,628.973388751119,627.872092862251,626.770796973384,625.669501084516,624.568205195648,623.466909306781,622.365613417913,621.264317529046,620.163021640178],[631.627499641068,630.526203752201,629.424907863333,628.323611974466,627.222316085598,626.12102019673,625.019724307863,623.918428418995,622.817132530128,621.71583664126,620.614540752392,619.513244863525],[630.977722864415,629.876426975548,628.77513108668,627.673835197813,626.572539308945,625.471243420077,624.36994753121,623.268651642342,622.167355753475,621.066059864607,619.964763975739,618.863468086872],[630.327946087762,629.226650198895,628.125354310027,627.024058421159,625.922762532292,624.821466643424,623.720170754557,622.618874865689,621.517578976821,620.416283087954,619.314987199086,618.213691310219],[629.678169311109,628.576873422241,627.475577533374,626.374281644506,625.272985755639,624.171689866771,623.070393977903,621.969098089036,620.867802200168,619.766506311301,618.665210422433,617.563914533565],[629.028392534456,627.927096645588,626.825800756721,625.724504867853,624.623208978985,623.521913090118,622.42061720125,621.319321312383,620.218025423515,619.116729534647,618.01543364578,616.914137756912],[628.378615757803,627.277319868935,626.176023980067,625.0747280912,623.973432202332,622.872136313465,621.770840424597,620.66954453573,619.568248646862,618.466952757994,617.365656869127,616.264360980259],[627.72883898115,626.627543092282,625.526247203414,624.424951314547,623.323655425679,622.222359536812,621.121063647944,620.019767759076,618.918471870209,617.817175981341,616.715880092474,615.614584203606],[627.079062204496,625.977766315629,624.876470426761,623.775174537894,622.673878649026,621.572582760159,620.471286871291,619.369990982423,618.268695093556,617.167399204688,616.06610331582,614.964807426953],[626.429285427843,625.327989538976,624.226693650108,623.125397761241,622.024101872373,620.922805983505,619.821510094638,618.72021420577,617.618918316903,616.517622428035,615.416326539167,614.3150306503],[625.77950865119,624.678212762323,623.576916873455,622.475620984587,621.37432509572,620.273029206852,619.171733317985,618.070437429117,616.969141540249,615.867845651382,614.766549762514,613.665253873647],[625.129731874537,624.028435985669,622.927140096802,621.825844207934,620.724548319067,619.623252430199,618.521956541332,617.420660652464,616.319364763596,615.218068874729,614.116772985861,613.015477096994],[624.479955097884,623.378659209016,622.277363320149,621.176067431281,620.074771542413,618.973475653546,617.872179764678,616.770883875811,615.669587986943,614.568292098076,613.466996209208,612.36570032034],[623.830178321231,622.728882432363,621.627586543496,620.526290654628,619.42499476576,618.323698876893,617.222402988025,616.121107099158,615.01981121029,613.918515321422,612.817219432555,611.715923543687],[623.180401544578,622.07910565571,620.977809766842,619.876513877975,618.775217989107,617.67392210024,616.572626211372,615.471330322505,614.370034433637,613.268738544769,612.167442655902,611.066146767034],[622.530624767925,621.429328879057,620.328032990189,619.226737101322,618.125441212454,617.024145323587,615.922849434719,614.821553545851,613.720257656984,612.618961768116,611.517665879249,610.416369990381],[621.880847991271,620.779552102404,619.678256213536,618.576960324669,617.475664435801,616.374368546934,615.273072658066,614.171776769198,613.070480880331,611.969184991463,610.867889102595,609.766593213728],[621.231071214618,620.129775325751,619.028479436883,617.927183548015,616.825887659148,615.72459177028,614.623295881413,613.521999992545,612.420704103678,611.31940821481,610.218112325942,609.116816437075],[620.581294437965,619.479998549098,618.37870266023,617.277406771362,616.176110882495,615.074814993627,613.97351910476,612.872223215892,611.770927327024,610.669631438157,609.568335549289,608.467039660422],[619.931517661312,618.830221772444,617.728925883577,616.627629994709,615.526334105842,614.425038216974,613.323742328106,612.222446439239,611.121150550371,610.019854661504,608.918558772636,607.817262883768],[619.281740884659,618.180444995791,617.079149106924,615.977853218056,614.876557329188,613.775261440321,612.673965551453,611.572669662586,610.471373773718,609.370077884851,608.268781995983,607.167486107115],[618.631964108006,617.530668219138,616.429372330271,615.328076441403,614.226780552535,613.125484663668,612.0241887748,610.922892885933,609.821596997065,608.720301108197,607.61900521933,606.517709330462],[617.982187331353,616.880891442485,615.779595553617,614.67829966475,613.577003775882,612.475707887015,611.374411998147,610.273116109279,609.171820220412,608.070524331544,606.969228442677,605.867932553809],[617.3324105547,616.231114665832,615.129818776964,614.028522888097,612.927226999229,611.825931110362,610.724635221494,609.623339332626,608.522043443759,607.420747554891,606.319451666023,605.218155777156],[616.682633778046,615.581337889179,614.480042000311,613.378746111444,612.277450222576,611.176154333708,610.074858444841,608.973562555973,607.872266667106,606.770970778238,605.66967488937,604.568379000503],[616.032857001393,614.931561112526,613.830265223658,612.72896933479,611.627673445923,610.526377557055,609.425081668188,608.32378577932,607.222489890452,606.121194001585,605.019898112717,603.91860222385],[615.38308022474,614.281784335872,613.180488447005,612.079192558137,610.97789666927,609.876600780402,608.775304891534,607.674009002667,606.572713113799,605.471417224932,604.370121336064,603.268825447196]],"type":"surface","frame":null}],"highlight":{"on":"plotly_click","persistent":false,"dynamic":false,"selectize":false,"opacityDim":0.2,"selected":{"opacity":1},"debounce":0},"shinyEvents":["plotly_hover","plotly_click","plotly_selected","plotly_relayout","plotly_brushed","plotly_brushing","plotly_clickannotation","plotly_doubleclick","plotly_deselect","plotly_afterplot","plotly_sunburstclick"],"base_url":"https://plot.ly"},"evals":[],"jsHooks":[]}</script> ] --- # The “Constant” - Alternatively, we can write the population regression equation as: `$$Y_i=\beta_0\color{#e64173}{X_{0i}}+\beta_1X_{1i}+\beta_2X_{2i}+u_i$$` - Here, we added `\(X_{0i}\)` to `\(\beta_0\)` - `\(X_{0i}\)` is a .hi[constant regressor], as we define `\(X_{0i}=1\)` for all `\(i\)` observations - Likewise, `\(\beta_0\)` is more generally called the .hi[“constant”] term in the regression (instead of the “intercept”) - This may seem silly and trivial, but this will be useful next class! --- # The Population Regression Model: Example I .bg-washed-green.b--dark-green.ba.bw2.br3.shadow-5.ph4.mt5[ .green[**Example**:] .smaller[ `$$\text{Beer Consumption}_i=\beta_0+\beta_1Price_i+\beta_2Income_i+\beta_3\text{Nachos Price}_i+\beta_4\text{Wine Price}+u_i$$` ] ] - Let's see what you remember from micro(econ)! -- - What measures the **price effect**? What sign should it have? -- - What measures the **income effect**? What sign should it have? What should inferior or normal (necessities & luxury) goods look like? -- - What measures the **cross-price effect(s)**? What sign should substitutes and complements have? --- # The Population Regression Model: Example I .bg-washed-green.b--dark-green.ba.bw2.br3.shadow-5.ph4.mt5[ .green[**Example**:] .smaller[ `$$\widehat{\text{Beer Consumption}_i}=20-1.5Price_i+1.25Income_i-0.75\text{Nachos Price}_i+1.3\text{Wine Price}_i$$` ] ] - Interpret each `\(\hat{\beta}\)` --- # Multivariate OLS in R .left-code[ .code50[ ```r # run regression of testscr on str and el_pct school_reg_2 <- lm(testscr ~ str + el_pct, data = CASchool) ``` ] ] .right-plot[ .smaller[ - Format for regression is ```r lm(y ~ x1 + x2, data = df) ``` - `y` is dependent variable (listed first!) - `~` means “is modeled by” or “is explained by” - `x1` and `x2` are the independent variable - `df` is the dataframe where the data is stored ] ] --- # Multivariate OLS in R II .left-code[ .code60[ ```r # look at reg object school_reg_2 ``` ``` ## ## Call: ## lm(formula = testscr ~ str + el_pct, data = CASchool) ## ## Coefficients: ## (Intercept) str el_pct ## 686.0322 -1.1013 -0.6498 ``` ] ] .right-plot[ - Stored as an `lm` object called `school_reg_2`, a `list` object ] --- # Multivariate OLS in R III .code50[ ```r summary(school_reg_2) # get full summary ``` ``` ## ## Call: ## lm(formula = testscr ~ str + el_pct, data = CASchool) ## ## Residuals: ## Min 1Q Median 3Q Max ## -48.845 -10.240 -0.308 9.815 43.461 ## ## Coefficients: ## Estimate Std. Error t value Pr(>|t|) ## (Intercept) 686.03225 7.41131 92.566 < 2e-16 *** ## str -1.10130 0.38028 -2.896 0.00398 ** ## el_pct -0.64978 0.03934 -16.516 < 2e-16 *** ## --- ## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 ## ## Residual standard error: 14.46 on 417 degrees of freedom ## Multiple R-squared: 0.4264, Adjusted R-squared: 0.4237 ## F-statistic: 155 on 2 and 417 DF, p-value: < 2.2e-16 ``` ] --- # Multivariate OLS in R IV: broom .left-column[ .center[  ] ] .right-column[ .quitesmall[ .code50[ ```r # load packages library(broom) # tidy regression output tidy(school_reg_2) ``` <div data-pagedtable="false"> <script data-pagedtable-source type="application/json"> {"columns":[{"label":["term"],"name":[1],"type":["chr"],"align":["left"]},{"label":["estimate"],"name":[2],"type":["dbl"],"align":["right"]},{"label":["std.error"],"name":[3],"type":["dbl"],"align":["right"]},{"label":["statistic"],"name":[4],"type":["dbl"],"align":["right"]},{"label":["p.value"],"name":[5],"type":["dbl"],"align":["right"]}],"data":[{"1":"(Intercept)","2":"686.0322487","3":"7.41131248","4":"92.565554","5":"3.871501e-280"},{"1":"str","2":"-1.1012959","3":"0.38027832","4":"-2.896026","5":"3.978056e-03"},{"1":"el_pct","2":"-0.6497768","3":"0.03934255","4":"-16.515879","5":"1.657506e-47"}],"options":{"columns":{"min":{},"max":[10]},"rows":{"min":[10],"max":[10]},"pages":{}}} </script> </div> ] ] ] --- # Multivariate Regression Output Table .pull-left[ .code50[ ```r library(huxtable) huxreg("Model 1" = school_reg, "Model 2" = school_reg_2, coefs = c("Intercept" = "(Intercept)", "Class Size" = "str", "%ESL Students" = "el_pct"), statistics = c("N" = "nobs", "R-Squared" = "r.squared", "SER" = "sigma"), number_format = 2) ``` ] ] .pull-right[ .quitesmall[ <table class="huxtable" style="border-collapse: collapse; border: 0px; margin-bottom: 2em; margin-top: 2em; ; margin-left: auto; margin-right: auto; " id="tab:unnamed-chunk-21"> <col><col><col><tr> <th style="vertical-align: top; text-align: center; white-space: normal; border-style: solid solid solid solid; border-width: 0.8pt 0pt 0pt 0pt; padding: 6pt 6pt 6pt 6pt; font-weight: normal;"></th><th style="vertical-align: top; text-align: center; white-space: normal; border-style: solid solid solid solid; border-width: 0.8pt 0pt 0.4pt 0pt; padding: 6pt 6pt 6pt 6pt; font-weight: normal;">Model 1</th><th style="vertical-align: top; text-align: center; white-space: normal; border-style: solid solid solid solid; border-width: 0.8pt 0pt 0.4pt 0pt; padding: 6pt 6pt 6pt 6pt; font-weight: normal;">Model 2</th></tr> <tr> <th style="vertical-align: top; text-align: left; white-space: normal; padding: 6pt 6pt 6pt 6pt; font-weight: normal;">Intercept</th><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0pt 0pt; padding: 6pt 6pt 6pt 6pt; font-weight: normal;">698.93 ***</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0pt 0pt; padding: 6pt 6pt 6pt 6pt; font-weight: normal;">686.03 ***</td></tr> <tr> <th style="vertical-align: top; text-align: left; white-space: normal; padding: 6pt 6pt 6pt 6pt; font-weight: normal;"></th><td style="vertical-align: top; text-align: right; white-space: normal; padding: 6pt 6pt 6pt 6pt; font-weight: normal;">(9.47)   </td><td style="vertical-align: top; text-align: right; white-space: normal; padding: 6pt 6pt 6pt 6pt; font-weight: normal;">(7.41)   </td></tr> <tr> <th style="vertical-align: top; text-align: left; white-space: normal; padding: 6pt 6pt 6pt 6pt; font-weight: normal;">Class Size</th><td style="vertical-align: top; text-align: right; white-space: normal; padding: 6pt 6pt 6pt 6pt; font-weight: normal;">-2.28 ***</td><td style="vertical-align: top; text-align: right; white-space: normal; padding: 6pt 6pt 6pt 6pt; font-weight: normal;">-1.10 ** </td></tr> <tr> <th style="vertical-align: top; text-align: left; white-space: normal; padding: 6pt 6pt 6pt 6pt; font-weight: normal;"></th><td style="vertical-align: top; text-align: right; white-space: normal; padding: 6pt 6pt 6pt 6pt; font-weight: normal;">(0.48)   </td><td style="vertical-align: top; text-align: right; white-space: normal; padding: 6pt 6pt 6pt 6pt; font-weight: normal;">(0.38)   </td></tr> <tr> <th style="vertical-align: top; text-align: left; white-space: normal; padding: 6pt 6pt 6pt 6pt; font-weight: normal;">%ESL Students</th><td style="vertical-align: top; text-align: right; white-space: normal; padding: 6pt 6pt 6pt 6pt; font-weight: normal;">       </td><td style="vertical-align: top; text-align: right; white-space: normal; padding: 6pt 6pt 6pt 6pt; font-weight: normal;">-0.65 ***</td></tr> <tr> <th style="vertical-align: top; text-align: left; white-space: normal; padding: 6pt 6pt 6pt 6pt; font-weight: normal;"></th><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0.4pt 0pt; padding: 6pt 6pt 6pt 6pt; font-weight: normal;">       </td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0.4pt 0pt; padding: 6pt 6pt 6pt 6pt; font-weight: normal;">(0.04)   </td></tr> <tr> <th style="vertical-align: top; text-align: left; white-space: normal; padding: 6pt 6pt 6pt 6pt; font-weight: normal;">N</th><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0pt 0pt; padding: 6pt 6pt 6pt 6pt; font-weight: normal;">420       </td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0pt 0pt; padding: 6pt 6pt 6pt 6pt; font-weight: normal;">420       </td></tr> <tr> <th style="vertical-align: top; text-align: left; white-space: normal; padding: 6pt 6pt 6pt 6pt; font-weight: normal;">R-Squared</th><td style="vertical-align: top; text-align: right; white-space: normal; padding: 6pt 6pt 6pt 6pt; font-weight: normal;">0.05    </td><td style="vertical-align: top; text-align: right; white-space: normal; padding: 6pt 6pt 6pt 6pt; font-weight: normal;">0.43    </td></tr> <tr> <th style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0.8pt 0pt; padding: 6pt 6pt 6pt 6pt; font-weight: normal;">SER</th><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0.8pt 0pt; padding: 6pt 6pt 6pt 6pt; font-weight: normal;">18.58    </td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0.8pt 0pt; padding: 6pt 6pt 6pt 6pt; font-weight: normal;">14.46    </td></tr> <tr> <th colspan="3" style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0.8pt 0pt 0pt 0pt; padding: 6pt 6pt 6pt 6pt; font-weight: normal;"> *** p < 0.001; ** p < 0.01; * p < 0.05.</th></tr> </table> ] ]