This forum is about wrong numbers in science, politics and the media. It respects good science and good English.
I found an old book on the subject of time series analysis from my undergraduate days. It was not on the syllabus, which was all historical engineering rather than the electronic engineering that I hoped to pursue. The book mark in it was a piece of paper, on which I had typed the following:
Time series analysis
Is a compound of fallacies
By subtle catalysis
Causing mental paralysis.
GRRR This fits into our authors poem. The linked video cause my blood pressure to increase. It wouldn't take much to cause me to tantrum. I plot the raw data for the red regions and can't find jack @()#$ in the way of increasing temperatures. There are folks out there with real hats on that are also saying the same thing, but how can these people be so arrogant?
I thought I'd update this thread as I have finally got round to calculating the correlation coefficient myself for the sea level change and sunspot number pair of time series.
The method I used was:
a) digitise the graphs from a screen capure of Fig 2 in the Solheim paper using 'PlotDigitizer'. PlotDigitizer only takes about 5 minutes to produce pretty accurate x, y coordinates for the two graphs, whereas it would take at least several hours if you tried to estimate the coordinate values by scaling from an enlarged printed out copy of Fig 2 using a ruler.
b) interpolate the points at uniform time intervals using a short Fortran program. With this I turned each time series into 911 data points at 0.1 year intervals for the year range 1909 to 2000.
c) run the two 911 point time series through another short Fortran program called 'corrcheck' I wrote about twenty years ago which calculates the absolute value of the correlation coefficient. This program uses the text book definition of R, the same one as in the Wikipedia article.
The value for the correlation coefficient I worked out was 0.35842, or 0.36 to two decimal places. This agrees very well with the value quoted by Willis Eschenbach [I was expecting to prove that he had significantly underestimated it].
So this example does raise an interesting point, two time series which look pretty well correlated by eyeball have quite a poor R value, only just above the acceptance criterion of 0.3 for them being taken as correlated.
Yes. I think your example illustrates the fragility of trying to use just one number to summarise the relationship between two waveforms. Consider a sine and a cosine (possibly derived in a physical system from the same source). The cross-correlation function is clearly periodic, but its average is zero. Even more difficult is the case where there is clearly oscillatory behaviour, but the periodic time is wandering, as happens in many common physical processes. This is clearly visible to the eye, but not to the single number. Even the cross-correlation function might blur the effect to relative invisibility.