Jacob Klerman writes:
I noticed you've been stressing the importance of measurement lately (e.g. „There are several ways to make your research reproducible…“). For reasons unrelated here, I was rereading his Leamer (1983). Let's get rid of the shortcomings of econometrics–Currently 40 years old. It's a bit old, but as you've noticed, it's a fun paper.
Leamer also indicates measurement points (emphasis added).
Once the sampling uncertainty S becomes small compared to the misspecification uncertainty M, it is time to look for other forms of evidence, experimental or non-experimental. Suppose you are interested in measuring the width of a coin. And I will provide a ruler for the volunteer room. After each volunteer reported their measurements, we calculated the mean and standard deviation and concluded that the width of the coin was 1.325 millimeters and the standard error was 0.013. I don't like this level of uncertainty, so I suggest finding three other rooms filled with volunteers, multiplying the sample size by 4, and dividing the standard error by 2. This is a stupid way to get more accurate measurements. This is because we have already reached the point where the sampling uncertainty S is very small compared to the misspecification uncertainty M. If you want to increase the true accuracy of your estimates, it's time to consider using a micrometer. The same goes for diet and heart disease. Medical researchers have more or less exhausted their vein of non-experimental evidence, and it is time to switch to the more expensive but richer vein of experimental evidence.
interesting. It is good to see examples where the ideas we will talk about today have already been discussed in classical literature.I certainly am measurement is important And it's not really discussed in statistics. Economists have a wide range of problems both theoretically (textbooks routinely discuss the great challenges of defining, let alone measuring, important microeconomic quantities such as the „money supply“) and practically (data collection is (which can often be a big problem) and are well aware of the importance of measurement. , even if this unfortunately involves archival research, data quality checks, etc. not always done) However, once the data is entered, issues of data quality and measurement bias and variance often seem to be forgotten. For example, consider: this infamous paper At no stage during the research, writing, reviewing, revising, or editing process did anyone seem to be concerned about regions where life expectancy is said to be 91 years (see graph above), which is odd. It doesn't even apply. regression curve. However, p is less than her 0.05. Publishing and promoting such results based on p-values represents a kind of culmination of trusting implausible theories over realistic measurements.
Also, if you want a good story about why it's a mistake to think that the uncertainty should be like 1/sqrt(n), here's how: Check out this story This is also included in the upcoming book Active Statistics.