Comments and forum for a non-linear regression on the Crew (SAS)

[taylor]

I don't know an incredible amount about this sort of thing, but thought I'd offer up a suggestion.

Have you actually got in contact with someone from the Crew/MLS and asked them if they could provide any of the data you require? You never know they might suprise you and be very helpful.

A friend of mine did his masters thesis on something involving sports (can't quite remember what, but it wasn't financial or math based), and got in contact with the Galaxy and they set him up to meet with the team and some of the players, and where overall very helpful, so it can't hurt to ask.

A couple of years ago, a guy from my undergrad worked there and was able to help me out (that is why I am using the Crew over DC). Now that he is gone, I don't have any inside person.

After your post I decided to try. I was forwarded to a different person (Mr. Wuerth) and left a message. I will try calling back after class. But, I am skeptical of this guy disclosing info because he is the PR director. I, therefore, doubt he is familiar with regressions, but it is worth a shot.

Guihno, I am skepitical that they would understand how important and expensive a tool this could be for them, if they had to pay for it.

I will post later today about my new contact.
Cheers,
Taylor

[taylor]

I left a couple of messages. He never called me back. I will try again tomorrow (Thursday).

[Auxodium]

and what will this acheive? better marketing? :o

[taylor]

Well, I have had quite some success. I was able to take out the autoco by using a yule-walker function and the multico is within workable parameters...(barely on a couple). Below are the estimates for the Crew. For those who don't know how to read this, I will offer a quick synopsis.
First however, let me offer the caveat that I opperating with a 7% level of significance.
The first thing I would like to say is that the model is obviously underfit, so the results should be intepreted with a high degree of flexibility when thinking of the results.

Now to the juicy stuff. The variables below explain 35% of why fans attend games.
The most obvious significant variables are price and stadium. E.G. a $1 price increase causes a -2269 in fan attendance. The new stadium resulted in 6459 new fans per game. A surprising variable is televised games in Spanish (1600 people don't show up for televised spanish games). The role of a high quality opponent on attendance does not play a role in the conducted model. That is to say, a winning team does not draw more than another team. Population is also found not to be significant. Finally, winning matters. If the Crew won the previous game, 3146 people attended the current game.

So there you go folks. As I said before, the numbers should be viewed as estimations. That is all they are. Since I do not have all the information available to me, one MUST interpret the values with some salt. But they are sure as hell more scientific than anything else I have seen, IMHO. If you are interested in the details or if you would be willing to collect a lot of data for your team to see the results, send me an email.
Cheers,
Taylor


The AUTOREG Procedure

Yule-Walker Estimates

SSE 936985352 DFE 84
MSE 11154588 Root MSE 3340
SBC 1804.49284 AIC 1781.69944
Regress R-Square 0.2732 Total R-Square 0.3540
Durbin-Watson 1.7457


Standard Approx
Variable DF Estimate Error t Value Pr > |t|

Intercept 1 -38411 42814 -0.90 0.3722
won_last game1 3146 853.9676 3.68 0.0004
price 1 -2296 694.2424 -3.31 0.0014
population 1 77.3345 47.3620 1.63 0.1062
opponent 1 700.1484 719.6596 0.97 0.3334
engtv 1 -28.3462 925.6405 -0.03 0.9756
spantv 1 -1696 922.1362 -1.84 0.0694
newsta 1 6459 2212 2.92 0.0045

[mpruitt]

Yeah I'd have no idea how to interpret these results but how'd you finally come about to yoru success? Were you able to contact someone with the Crew, were they helpful?

[ChrisE]

Taylor, can you explain to me what your population variable means?

edit: neat results, by the way.

[taylor]

Taylor, can you explain to me what your population variable means?

edit: neat results, by the way.


It's population of the columbus metro area per year. Once finals die down I will explain some more.

[numerista]

A few thoughts ...
1) It's hard to interpret "won last game" as a predictor without also including a measure of the team's overall strength. Is the real conclusion that good Crew teams draw more than bad ones, or do winning streaks inducing attendance (or both)?
2) Rather than (inaccurately) claiming to be "working with a 7% significance level," you should really just state that the Spanish-language TV coefficient came out to be borderline significant.
3) Kenn (kenn.com) has shown a huge weekday/weeknight effect. IIRC, he has also shown that holidays and opening day tend to have a large impact on attendance. These are easy predictors to collect, and because they aren't in the model, it's hard to place much faith in the results.
4) It would be interesting to see the same analysis carried out on logged attendance. The model would then be interpretable in terms of % change, and you would probably come closer to satisfying your underlying normality assumptions.

[taylor]

First of all I made a LABELING MISTAKE in my rush. THE "Wonlastgame" VARIABLE IS ACTUALLY A BINARY WEEKEND VARIABLE. To be clear, the "wonlastgame" variable is actually whether the Crew played on the weekend or not.
I was excited about finally getting rid of the autoco and made a silly mistake. Forgive me econometrics gods. I apologize for my hasty labeling. When I copied the results over on to the BS page, I renamed the variables in a way I thought people could easily understand the variable labels. I obviously overestimated my own abilities in remembering my labels.

Also, IMHO, there is no accurate or inaccurate level of significance. There are standards, but one level over another is not "inaccurate". Particularly when dealing with a one tailed test it is quite plausible to consider spantv significant. But then again, there is no correct level of significance, so interpret as you will.

Second, as I cleary stated, the model is underfit, so take your salt.

Third, I want to correct my R square value statement. That was an R square using a transformed matrix value for attendance. I've learned that I can't do that. Under a linear reg, the r square is around 25%.

Fourth, once I get some time, I will try to log everything. The problem is that SAS is esoteric and labor intensive. Both of which I don't have a lot of right now.

[taylor]

Yeah I'd have no idea how to interpret these results but how'd you finally come about to yoru success? Were you able to contact someone with the Crew, were they helpful?


No the Crew were not helpful at all. I think they thought I was working for the players union or something. That or they really didn't know what the hell I was trying to do.

ps, sorry for not responding sooner. I to do a couple papers over the past week.

[numerista]

For the record, Taylor ran a statistical test and got back a p-value of 0.0694. Then he/she changed the significance cut-off from 0.05 to 0.07, just so that he/she could claim that the result was below the cutoff.

Here's the problem:
If you're willing to change your cutoff after the fact, you invalidate the probabilistic formulation that gives a significance level its meaning. For that reason, I use the word "inaccurate."

In any case, the good news is that the weekend/weekday predictor is in the model. It'd really be nice to get seasonal effects (which Kenn has documented) in there, as well as opening day and holidays.

[ur_land]

I agree with numerista. The "industry standard" for significance is .05--saying you're working with a significance level of .07, while technically not incorrect, is odd, and takes away from the impact of your other significant results. Just call the spanish-tv result "marginally significant" and move on.

Another question--are these all one-tailed tests? If so, why?

[taylor]

[QUOTE=numerista]For the record, Taylor ran a statistical test and got back a p-value of 0.0694. Then he/she changed the significance cut-off from 0.05 to 0.07, just so that he/she could claim that the result was below the cutoff.
QUOTE]

Num, what are you talking about?

For the record, how did I (he) "change a cut-off" from .05 to .07? One can not change levels, if you don't have a starting level. I never "claimed" or said what level I was working with (and I do really think this to be a petty point, btw).

If you read my statement I offer a 'caveat of using a .07 level'. I said one needs to read the estimations with some 'salt' because the model is underfit. Don't make spurious remarks about what I never said or claimed.

If your question is when I decided to use that level, it is once I saw that the spantv was a .069.

As far as using a cut off of a .07, I figured it would be an acceptable level of significance because of it being a one tailed test (due to an "expected negative attendance effect", therefore only needing a one sided test and hence a .0349 level). The second point is that a P-.1 value is also an "industry standard" acceptable test level. So why such grief?

The point I am making about sig levels is that they are discretionary. Yes, .05 is the most commonnly used level, but that certianly does not prohibit the application of other levels. It simply depends on your risk aversion. Some people use a .001, some others a .5.

If you are interested in a lit review on the subject, we can do one.

Yet, if some people feel the .05 is the risk level they want, there is no problem in doing so. Done.

The greatest point not made about the spantv variable is its standard error. It's so is so large that it could be argued to be "marginal", but not because of its mean estimate.

Finally, as I said before, I am not finished with the project. I need to regress a simultanoues nonlinear model and feel that will be a "better fit model".

Phew.

[taylor]

ok, Num after reading your message again, I may of come accross a bit too strong. Sorry about that.

[numerista]

I agree with numerista. The "industry standard" for significance is .05--saying you're working with a significance level of .07, while technically not incorrect ...

It is incorrect -- a significance level is defined as the probability one would reject in the case where there is no signal. By claiming to be working with a significance level of .07, Taylor is claiming that he/she would not have rejected if a p-value had come out to be .0704. Since Taylor admits having chosen a cutoff after seeing the results, it doesn't make sense to talk about a significance level at all.

This is a technical point, but it's also the reason why it's important to have an industry standard, regardless of what particular standard is used.

[mpruitt]

PEOPLE! PLEASE? We're all here for the love of numbers, and horriably esoteric analysis of them! Don't fight! We're all in this together!!! Lol you guys are having a pretty heated disagreement and believe me when I say, I have absolutely no idea what you're talking about. This thread has turned extremely weird.... Kudos to the both of you though for mixing it up on a topic like this.

[numerista]

PEOPLE! PLEASE? We're all here for the love of numbers, and horriably esoteric analysis of them! Don't fight!.

:)

(For the sake of clarity, let me state that I'm not here to fight with anybody, and that I try to avoid making posts with that intent. But as an expert in statistics, I am prepared to be finicky about studies being done reasonably well. There's no point in breaking out heavy statistical machinery if it isn't used proficiently.)

[mpruitt]

It's just cracking me up. As I just pmed you when we started this form it was with the idea of taking sabermetric principles and statistical analysis and using it to learn more about American soccer. The stuff you guys are intellectually sparring about is just so beyond anything I'd envisioned. It's cracking me up but is oddly validating. I'm wondering though for those of us with a slightly shall we say, 'less then proficient grasp of stats and economics' would the two of you mind explaining some of this stuff further in English. If you don't feel the want or need to break it down for my 5th grade grasp of this stuff then that's totally cool. It's just far far beyond me :)

[taylor]

Maxim, I am in the middle of my finals period. After the 8th... well after 10th (I will be drinking heavily for my sister's graduation party)
I/we/whoever can spend some time extrapolating this stuff. Part of the problem is that this stuff is interpretative. There are different camps when it come to this stuff. Some are more risk averse than others and that's where the debate begins.

Finally, I learned along time ago not to fully trust statistician's/econometrician's presenations because of how much manipulaton can occur.

Any person claiming their analysis to be a perfect representation of data should raise eyebrows.

Hence, my attempted humble "salt comment".

Now, to Num.

:)
But as an expert in statistics, I am prepared to be finicky about studies being done reasonably well. There's no point in breaking out heavy statistical machinery if it isn't used proficiently.)


First, the t tests the null hypothesis if mean estimate equals zero. I have no idea what you mean when you say "signal"

I tried to be nice, but are you trolling this? If so, this is an unprecedented bigsoccer moment. Seriously.

I have a year of SAS under my belt and would never want to call myself an "expert", hell even after five years.

I still feel you still haven't substantively made a point. As I said before, the .1 level is also an "industry standard" level is it not? Check yes or no. If no, please cite why the .1 level is an unacceptable standard ( can you include several citations), because I can cite several too at the .1 level, albeit after the 12th.

To be clear one more time, I do understand and agree that a majority of academia uses a .05 level. That, however, does not make a substantive point. Academia still applies a .1 and less level test. If I were to publish something, I would probably use a .05, but it really depends on how risk averse one is (i.e. type 1 or type 2 error), hence a .1 or even greater could be applicable, depending on the study.

Just because the .05 level is used a majority of the time does not, however, mean one can not use other levels, to implicate otherwise is specious.

You clearly feel some people really should only work with a .05 level, but the degree you are are harping on this, if indeed you aren't trolling, is what I really don't understand. People use other levels, as an "expert", you undoubtedly know this.

I'm sure as an "expert" you recognize this stuff to be far more of interpretation than a binary anwser, so add some salt to make it more digestable for you.

Even following your "expert" opinion, is it not reasonable to apply a one tailed test and have a 3.49 value, therefore satisfying your .05 level?

Just so we can try to settle this, can you please anwser "yes" or "no" to a couple questions.

Does the "industry" publish .1 levels of sig, therefore implying a .07 is also an acceptable level? Remember the cyber soccer Gods still hold hontesy to be a virtue.

Will you except a one tailed test, if yes does it not therefore make the .07 level significant under a .05 test? If not please explain.

The only real substantive critque SO FAR (because, again, I am not finished with the project) is the spantv standard error.

Num, as the "expert" would you like to comment on why the SE would cause it to be "marginal"?

The reason I have parathesized the expert is because you haven't, imho, demonstrated anything substatively, other that saying because you are an industry "expert" I am misinforming people by using a .07 level.

To be clear, if I am wrong and you are right on anything, I have no problem stating it. I would prefer a collective effort over competitive anyday.

[taylor]

I forgot one more point. To me, and implicitly the world, using any particuliar level is discretionary. We make these discretionary decision everyday.

One can use whatever level they want, it merely depends on the risk aversion. EG in terms of industry, the EPA uses a different level than the OMB.

For example on a personal level Maxim, if it comes to risk aversion to the life of your child, you would be as risk averse as possible. Therefore, using a .000000000001 level would be the only way you would risk your child's life. You would want this level to ensure yourself that there was as little chance as possible that your child could die. A .05 level would be far too great a risk level for you, because that would mean 1 out of 20 chances your child would die.
On the other hand, if the test was to offer a life saving vaccine to your terminally ill child (assuming no cost), you would not care what the failure rate was, hence using .5 level or less, in the hope of saving your child (i.e. 1 out of 2 failure).

I hope this has illustrated why it is really an interpretational issue when it comes to the t-test.