The Information Commissioner's Office has ordered the University of East Anglia to release a portion of a weather dataset. The University's Climatic Research Unit had shared the data with Georgia Tech but refused to release it more widely. A leading Oxford physicist, Professor Jonathan Jones, made the successful request, which …
When things don't make sense
Start by following the money. Our money.
In fact, it would be far cheaper to pay the CRU 'scientists' to take an extended vacation, rather than to believe all their results and throw mind-boggling sums at solving the climate change 'problem'.
And after all, if some low-lying islands are going to become uninhabitable, the islanders should migrate over to Siberia, which apparently will soon be quite pleasant. Remember, it's 'change', not the end of the World, and not all change is bad.
I guess they would be more credible if they could agree whether Britain is going to get warmer or colder.
Re: When things don't make sense
"Start by following the money. Our money."
You'd be surprised, not merely that it doesn't get shovelled by the tonne into scientific research (which most people doing research would already be able to tell you, but to whom you don't want to listen), but about where it actually goes.
"And after all, if some low-lying islands are going to become uninhabitable, the islanders should migrate over to Siberia, which apparently will soon be quite pleasant."
Do you know anything at all about Siberia? It's not frozen wasteland all year that could do with a bit of warming; in summer it gets unbearably hot in places. The bits that aren't pleasant now will only become unpleasant in various other ways.
Sure, the way science is done could be improved by applying proven engineering principles. That should be the real story here, if people muster the stamina to actually read and digest even the messages reproduced in the article, rather than tittering at the naughty words and basking in the gossip and peripheral sensationalism.
@AC @AC from another AC
One does not need to be a fashion designer to see that there is something wrong with the Emperor's wardrobe. I have read enough of the Climategate e-mails (and the README_HARRY.TXT) to be quite sure that they show some pretty junk science. And I pay for these folk through my taxes so I reckon that should give me a say.
Is climate change real? Of course it is, climate changes (medieval warm period, little ice age, Vikings in Greenland, etc). The questions are more like: how much is it caused by human CO2 emissions, how many degrees of temperature change will all the "Climate Change" legislation prevent and at what cost? From the limited research I've done into the subject, the answers seem to be: not much, very little and a mind bogglingly huge amount (which will come from our pockets and go into the pockets of carbon traders, windmill manufacturers, Government taxes, climate scientist research grants, etc).
Re: Yet another AC - Wikipedia reckons Graham Stringer MP has a BSc (Hons) in Chemistry and worked as an analytical chemist. If so, and having been part of one of the inquiries into UEA, he should have the basic skill set to give a meaningful opinion on the validity or otherwise of UEA's work.
Re @ AC
"Or it could simply be both and that weather becomes more erratic"
Yes, good point... consider my post amended accordingly: "I guess they would be more credible if they could agree whether Britain is going to get warmer, colder, or suffer a more erratic climate."
Now I guess I touched on a raw nerve somewhere, so sorry about that.
I try to use measured language in all my postings, but when AGW proponents start reaching for terms like "idiot", "dumb" and "retarded", I think we know we're not going to be in for a particularly productive debate here.
As to why I dismiss the CRU people, it's not because of their beliefs, but because they did science badly, and in doing so have brought both climate science, and the broader scientific community, into disrepute.
Now you sound like a pretty intelligent chap, with a good handle on the science here. Help us less fortunate souls understand your insights, and lead us unto wisdom. Just lose the attitude, and bring on the science.
It is however, entirely acceptable to dismiss those with expert knowledge who then abuse their position of trust to pervert the course of expanding said knowledge.
Blindly trusting those "in the know" to be imbued with any sense of honesty or integirity is what let these guys irreperably damage the climate change debate, now they've given all the critics out there another excuse to dismiss the arguement altogether.
Personally I feel bad for the all the honest folks at the CRU whose tireless work and reputation has been forever tainted by Phil Jones and his corrupt cronies.
Re: RE: Re @ AC
"Well therein lies the problem,..."
Thank you, most eloquent. As it happens I don't read the Daily Mail, or base my opinions on TV personalities' humour. In fact, I wouldn't place myself in the 'deniers' camp. Nor do I think it's so 'easy' it can be explained in a short posting here, though I appreciate your trying.
The point is, I do have pretty high expectations of scientists, particularly when I'm forced (a) to pay for their research and (b) to pay for our public policy changes based on that research.
Here, I fear the CRU people were so sure of the veracity of their position, that to explain it to everyone was too much of a hassle.
What you/they don't seem to understand is that asking scientists to prove their results is not the same as saying they are wrong, it is a method of giving them an opportunity to communicate their results. (Think of it like a politician being interviewed: the journalist makes a series of opposing statements, allowing them to use the rebuttals as a way of explaining their position).
When the scientist/politician resorts to insult in response, the audience is left with the suspicion that the question was too hard to answer. And they wonder why, and sometimes they figure perhaps it's because the science isn't as valid as it should be. But that may not be the case at all. But by then it's a bit late.
So let's try to agree some common ground. Here's where's I'm at, at the moment...
1. AGW is happening.
2. AGW will be a problem for a lot of people.
3. AGW won't be a problem for a lot of other people.
4. (non-A)GW has happened many times in the past, causing quite a bit of hassle. It's been a lot hotter, and colder (oh, and erratic, I suspect too :-)
On the climate scientists, CRU in particular:
1. They know a lot about AWG
2. They are struggling with poor data
3. They are embarrassed they can't easily prove their conjectures using the data
4. They need to realise the science is the easy bit, compared to communicating their findings
5. They need to realise the communication really matters, as the general population need to be educated and convinced that their money should be spent on this. Because we're talking trillions of dollars here.
If you disagree on any of that, let us know what and why.
Bizarre though it might seem, I'm not trying to take pot-shots for the fun of it, but am actually trying to help you here.
If the data, methods, decisions, and all other items necessary
for a third party to duplicate the results are not PUBLICLY available, it isn't science - it's either a scam, a patent, or a trade secret. And there seem to a fair few who regard at least one of those as a redundancy.
RE : RE: Re: RE: Re @ AC → #
what has wanting to believe in a theory got to do with science ?
WANT -- that is part of the problem; it belongs with belief - out side the lab.
Here are some facts :
Science requires a new hypothesis to be proven before being accepted in preference to a previous theory - those advocating that the AGW hypothesis have to prove it. Those denying its veracity have to do NOTHING (in the way of proving the pre-existing theory).
To prove a theory you make falsifiable predictions. You collect real world data and analyze said data to discover if your predictions are true . You may not use the out put of models as evidence that your theory is correct. You may however use models to attempt to make your predictions.
Example - the general climate models ALL predict there should be heating in the troposphere. No such heating has been found. (Some heating *may* be present; but statistically insignificant compared to the prediction).
This falsification all on its own says there is something very wrong with the AGW theory.
The AGW brigade continue with the statistically totally discredited hockey stick curve - no the rebuttal was rebutted piecemeal and in detail. There is no hockey stick temperature curve.
Results can not be replicated - by the very person who carried out the original experiment - go to Science 101 - ALL experiments must be capable of being replicated AND the results from such experiments MUST agree with the results of the preceding experiments. If you will not put the data AND methodology into public domain; how can other people replicate your experiment ?
Climate changes; the current theory that it is mostly due to natural changes has yet to be disproved; the proposed replacement - AGW - has been falsified in several important areas.
It doesn't matter what *I WANT* the answer to be; until the AGW proponents repair their hypothesis so it repeatedly and accurately makes true predictions it is fatally broken.
A bit more information on the Climate models : they are attempting to model a chaotic system. We do not know how many input variables there are. We do not know how the input variables interact.
The models used are predicated on CO2 being a major input variable.
There is no proof for this supposition. It is not surprising that the Climate Models show extreme sensitivity to variations in CO2 - THAT WAS THE WAY THE MODELS HAVE BEEN BUILT.
Now; when the models can make accurate predictions 1 year; 5 years; 10 years into the future; we *may* have grounds for believing they *may* be able to approximate conditions in 100 years time. They can't. Yet most climate 'science' relies on the output of these models; despite the fact that they are incapable of accurately modeling the climate.
Note - there is no belief in any of the above; there is an expectation that the science should be done properly and in the full light of day.
Re: RE: Re: RE: Re @ AC
I apologise in advance for picking some quotes from your posting out of context, but hopefully they advance the debate...
"you're focussing on the CRU scientists"
- because that's what the article was about. But I'm happy to criticise the IPCC too if you like.
"Why is the AGW camp being held to such impossibly high standards"
- because they are suggesting we spend vast amounts of money and change our entire lifestyles. It matters to us, a lot, so we want to be really sure it's worth it.
"Why isn't the climate skeptic crowd held to the same standards"
- because they aren't suggesting we spend vast amounts of money and change our entire lifestyles.
"why do you think climate science is questionable"
- because discussions like this where the AGW sceptics question the science tend to be met not with straight factual answers, but with undisguised annoyance and insult by the AGW camp.
Here's a tip. If you want to be taken seriously, start engaging with the 'ignorant', if you can bear to lower yourself that far, and then, directly, answer and debate the issues they want to discuss. If your facts are solid, they'll work down there too, and you'll end up with some more supporters. If you don't, or can't, then don't be surprised if your views/facts are ignored, and we face the climatic consequences.
Giant RIGHT WING conspiracy?
"Really, people don't want to believe in man made climate change because it's an inconvenience to them and grasping at straws like plucking out examples of a few bad apples and applying that to the practice as a whole is utterly pathetic"
Your problem is that you haven't looked at the evidence critically and you think most people are more stupid than you. That's like crossing the road with your eyes shut - eventually you'll get run over.
People don't find the arguments convincing BECAUSE the arguments aren't convincing - simples. It isn't get warmer (0.7C in 120 years), CO2 is out of phase with temperature, the "missing heat" can't be found, the solar influence is simply not scientifically well understood. The state of the science is roughly on a par with phrenology. There are not "thousands" but only a couple of dozen physicists doing climate physics - studying "the effect of climate change on blue tit migrations" does not count.
This is NOT to say humans don't influence the climate - my bet is we probably do in many ways (particulates from wood fires have a significant local effect) - we want to know how much.
Your ranting and raving about right-wing conspiracies or God or irrationality doesn't make a shaky scientific hypothesis any stronger.
Re: RE: Giant RIGHT WING conspiracy?
"Putting the AGW argument aside I'm just intrigued to know why the UK viewpoint contrasts so radically with that of much of the rest of the world."
Someone should do some research into that. Three conjectures to kick things off:
1. We're famous for talking about the weather, so everyone here considers themselves an expert. (and, ironically, being sceptical is a good scientific trait)
2. We're generally more sceptical, not just of AGW, but other things too.
3. We're fed-up with our traditional cold damp climate, and the chance of a warmer one is actually rather appealing.*
* So being asked to pay extra, to stop that, is somewhat galling!
I'm not sure I understood this correctly...
... but is this apparently quite buggy data set the basis (or one of the basis data sets) on which a lot of climate science (and the IPCC reults) rests?
Good thing the data has been released, at least now we can see whether the model results were replicable or not. Let's wait and see what other external scientists make of the dataset.
(Not much hope that the people on the forum here will wait and see, I'm expecting total nuclear forum warfare to break out)
<ducks, runs for cover, grabs a beer from a safe distance>
This is not the data you're looking for
No, this data set is not used as the basis for any GW predictions. I have that on the authority of the (in) famous Harry himself, in fact.
Now you have done it..
.. mentioned the work 'newcleer' in a Reg forum post.
The end of the world is nigh
No title needed
> When modelling chaotic systems it's quite normal for there to be an element of randomness, precisely because that's what's require to model chaos.
And what precisely does that have to do with Briffa's results?
> The real question is whether repeated experiments tell the same story, not whether they come to precisely the same final state-
What experiment has Briffa performed?
Briffa specialises in reconstructing past climate using various climate proxies (mainly tree rings). There should be no randomness in his method, he doesn't use computer models that have an element of randomness in them.
Stringer isn't talking about Briffa taking new measurements, he is talking about him using the same data and methods to reproduce the same results. Which, apparently, he is unable to do.
> But then, this is also why MPs have no business in scientific debate..
Stringer worked as an analytical chemist for a number of years.
Let me be more specific.
If you run the same programs over the *same* datasets do you get the same results, results in the same *range* or completely unreproducible results?
BTW as this is *modeling* and *not* simulation the software should write out *all* hidden values to archive as SOP. Otherwise how could you know if run #8 (sea level rises by 10m, flooding *all* UK ports. That would make the UK news would it not) was a data entry mistake, all internal random numbers going to worst case values at the *same* time (finite probability, not impossible), corrupted data file or something else.
Note that climate change measures have bills in the 10s of *billions*.
You don't get it
Modelling != science
You mean if my iPhone says I was near a Tower 100 miles away it could be lying ?
(points) Witch! (averts eyes)
If you can't feed the same data into your model and get the same result
it is NOT science. They have a journal out there for stuff like that: The Journal of Irreproducible Results. I hear it's even peer reviewed.
The main issue
That most of us that get categorised as 'Deniers' have is that the dataset and methodology wasn't released to allow the other scientists to validate the results.
This when further compounded by quotes such as "When I asked Oxburgh if [Keith] Briffa [CRU academic] could reproduce his own results, he said in lots of cases he couldn't," for me means that this is most definitely not science up to this point.
If however the results become repeatable and are validated by external scientists then it'll be science, until then it's just playing with numbers.
My personal gripe is that this 'science' is being used to govern serious and far reaching policy decisions as is it was incontrovertible fact.
Just to be clear before I get flamed to hell, I don't adamantly deny that humans are having any impact on our climate, nor do I ascribe to the hypothesis that we are solely or largely responsible for the change in the world climate, or even that the climate is changing in the manner we are told. I would just like some actual proper transparent science to be done by non political organisations to attempt to understand what actually is happening and what may actually be causing it.
Is that too much to ask?
It's not at all too much to ask...
....it's just that, on the "denier" side, the vast majority is in the business of spreading FUD, and not in the business of actual science.
It's a lot like the way the tobacco companies attempted to hide or discredit the facts about the health issues of cigarettes (and still do, come to that), or the way the Christian Right is trying to "teach the controversy" about Intelligent Design.
There are very many scientists trying to do actual proper transparent science. The vast majority of them have been convinced - by the science - that we really are likely to have a problem in the fairly near future. They can't ALL be deliberately misleading us. On the other hand, there are a lot of non-scientists (and a few scientists, to be fair) who do not believe the science is convincing. And there are a large number of corporations who have a vested interest in persuading people that it's not really happening. And the science is hard, and not by any means exact.
Hence, it's very easy to spread FUD, and El Reg ought, frankly, to be ashamed of the help that it gives the deniers.
When I run stress tests I get slightly different numbers each time. According to this article this means what I am doing is just not science. Even though the stress test results are pretty much consistant. It's machine and network "weather" creating noise that make the results slightly different each time.
I can imagine a kind of (admittedly unfunny) dilbert cartoon where the Boss demands the test results are replicated. Completed in 203.1 seconds? Reproduce that result! You can't?? Incompetence!
> When I run stress tests I get slightly different numbers each time.
If you were asked "On 2nd March your report showed that this part was faulty." Would you be able to go back to your measurements from the 2nd March (not take new measurements) and feed them all through your equations again and come up with the same result?
This is what Briffa is unable to do. To use your stress test anology, in some cases he does not know where the stress test measurements are or what version of the measurements he used and in others he doesn't know what calculations he used to get the final result.
That is the only polite word I can use - especially for others who must have been aware of this shambles but still exploited it.
Shut them down or set them to work doing something useful like clearing sewers. On minimum wage of course.
Land Surface Temperatures
An initiative, largely unfunded at present, has begun with its aim to improve the state of the instrumental land surface temperature record. The new data base will be completely open, and all data traceable to its source. The 'climate community' have gone out of their way to invite temperature metrologists from NIST and NPL (Me) and statisticians to look closely at their work, how the 'data products' are compiled and how they are tested. What more do you expect people to do?
Andrew, Cynicism is an easy stance but actually doesn't reflect the reality that I have experienced in the 9 months I have been working in this area. The possibility of climate change is very real and worth investigating. And yes, people need to do a better job. And they are trying really hard to do just that:
Dr. Michael de Podesta
"An initiative, largely unfunded at present, has begun with its aim to improve the state of the instrumental land surface temperature record. "
The cynicism comes from the fact that what you describe seems so admirable it should be funded as a matter of course, if for no other reason than a general house clearing of what seems to be *highly* suspect data.
Except it isn't.
And the issues of data management seem so bad they should have been picked up and resolved *decades* ago.
Except they weren't.
I *fully* support this proposal. But boy has it taken them a while to get round to it.
I think you missed, and keep missing, the point.
'...reflect the reality that I have experienced in the 9 months I have...' Does not refect on the use of mangled, dataless datasets going back years, decades or centuries. The reason for all of the mistrust is that you (or at least your colleagues) published results and hid the data and methods used to generate those results. If you/they had published the results, the data and the methods, NO ONE WOULD HAVE CARED how wildly in/accurate your results were because they would have been able to paste it all into Excel and click print...and come to a conclusion, perhaps the same as yours, for themselves. That was not what happened, was it? What you did was (I was about to say immoral, but scientists aren't supposed to be moral) dishonest.
I'm not sure about the statement about not getting the same results twice in a row. If you feed a database full of numbers through a non-RNG based program twice in a row and get different results, how can you trust anything written by those programmers ever again? Or those people (I almost said scientists) who publish said numbers?
In all likelyhood, there were several scientists behind the scenes working feverishly to come up with an equation (or set thereof) to match the existing data that could then be used to predict future trends. If, however, in removing outliers and other 'oddities,' the dataset that was supposed to be raw data turned into one that contained 'data that proves our model' then that dataset became worse than suspect, it became a useless, and the model a lie.
"In all likelyhood, there were several scientists behind the scenes working feverishly to come up with an equation (or set thereof) to match the existing data that could then be used to predict future trends"
That isn't remotely how it's done. The datasets are not input into models. The models are not merely extrapolating datasets.
the datasets are used to validate the HINDCASTING capability of the models
the models which are attempting to model a chaotic system with a large number of unknown variables interacting in a complex and chaotic way; which is not understood (and therefore can not be modeled). The same models which are used as *proof* of AGW (read the IPCC papers to see how many times the proof is the output of a model). The same models which assume CO2 is a primary driver of climate change.
They run the models against historic readings and see if the resulting 'climate' agrees with the historic record. They 'tune' the input parameters (none of which they have empirical proof for; which they therefore don't understand; apart from CO2 MUST be one of the important ones) until the hindcast agrees with history. They then run the model forwards in time; using parameters designed to get from a to b in a chaotic system; and pronounce the outcome as valid. ITS A CHAOTIC SYSTEM. That means the hind cast matters as much as the last spin of the roulette wheel matters when placing your £1000000000000000000000000 bet on the next spin.
The models are a busted flush. And judging by Harry's readme - there were several scientists behind the scenes working feverishly to come up with an equation (or set thereof) to match the existing data that could then be used to predict future trends.
Then what do the models do? And why do they do it?
UEA - A Missed opportunity?
The whole point of this isn't anything to do with climate change being real or not, it's the idiotic steps taken by UEA (apparently with the collusion of senior staff members) to avoid sending to perfectly innocuous data to people who were entitled to see it.
The fact is that their behaviour has cost them a lot of our money and done nothing except generate more sceptics and conspiracy theories.
I thought the 'inquiries' into climategate said that all the data was publicly available. This judgement appears to contradict that assertion...
The worst part of all of this is that it's distracted effort from trying to work out the important stuff, like how much the planet has actually warmed and why. Also, where Trenberths missing heat has gone (unless you accept it really was Mt Pinatubo what done it)...
Why let lack of evidence get in the way of a conspiracy theory ?
The emails quoted out of context are likely to seem pretty banal to anyone who works in a messy research environment. Part of the process has always to involve critically reviewing the validity of your data and methodology - and that's what we see some evidence of here. Wish I could say the same for the approach taken by many journalists.
As someone who has observed how long it can take once a corporation decides to open source a significant in-house developed program involving code provided by multiple contributing parties, I can also well imagine why copyright can block public domain release of any substantial data collection coming from very many independent sources. This is very likely to be the case even if any one of the external contributors may be thought unlikely to object to repurposing the data or code provided (i.e. having the data made available to a third party for a purpose not covered in the original agreement) but one or more of very many data providers might, and the cost of obtaining permission from all sources attested sufficiently to convince corporate lawyers that the exercise is litigation risk free is substantial. So what benefit to UEA in this expensive exercise ?
It also seems pretty rich that a journalist who expends so many words arguing against any sensible reform of the outdated copyright system is willing to argue conspiratorial and fraudulent intent behind these logical consequences of the very system he supports. I'm sure UEA are fine about having the data released into the public domain once it's going to be the ICO which risks being sued. Now that the ICO has ordered the release, UEA can wash their hands of the potential risk and tell any copyright objector amongst the data originators to go sue the ICO.
Re: Why let lack of evidence get in the way of a conspiracy theory ?
A man confused by an "abuse" of copyright, and in quite a quandry because the "good" guys invoked it.
Much poorly-informed speculation of motives follows.
Your taxes at work.
Hypocrisy and abuse of Copyright
Hunting with the hounds when it's about creators getting paid. Running with the fox when he can get away with liberally quoting private emails illegally put into the public domain following an offence under the Computer Misuse Act. Clearly Mr Orlowski wants to have his own cake and eat other people's.
Re: Hypocrisy and abuse of Copyright
Your time would be better spent reading the ICO ruling, rather than speculation, red herrings and repetitive ad hominems on the author.
copsewood: "private emails illegally put into the public domain following an offence under the Computer Misuse Act."
So much for "Open data" :-)
Re: Hypocrisy and abuse of Copyright
> Hunting with the hounds when it's about creators getting paid.
From the actual decision:
" Given all of the above, it is not clear to the Commissioner how UEA might have planned to commercially exploit the specific information requested and how disclosure might have impacted on any plans that it might have developed or been in the process of developing. He is consequently not satisfied that it is more probable than not that disclosure would adversely affect its intellectual property rights. He has therefore determined that regulation 12(s)(c) is not engaged."
In other words, the "creators" were not even attempting to commercially exploit the requested data.
> he can get away with liberally quoting private emails illegally put into the public domain following an offence under the Computer Misuse Act.
1. The emails where not private. They where sent and received using UEA equipment and had UEA email addresses on them. As such they belonged to the UEA. None of the senders or recipients of the emails therefore has any expectation of privacy with the emails.
2. The UEA is a public authority as defined within the FOIA. As such all documents, including email, held by the UEA are subject to FOI requests. The UEA therefore can not have any expectation of privacy with any email. This is irrespective of whether a FOI request is granted or refused.
3. Norfolk police have yet to determine if a criminal, civil or even if any offence occurred.
4. If the emails were released by an UEA employee then the release is covered under the Public Interest Disclosure Act 1998. The UEA would not even be able to terminate the whistleblowers employment, let alone prosecute.
@ac 29th June 2011 08:53 GMT
'In other words, the "creators" were not even attempting to commercially exploit the requested data.'
This is unlikely to have been the blocking issue. It is more likely that the data was provided by very many originators and aggregators, and copyright had been granted to UEA for specific purposes and uses, as opposed to for any use or onwards distribution. Prior to the overriding instruction by the ICO transferring potential liability for this to the government, it seems unlikely that the UEA could have provided data to other parties, assuming that they wanted to, without risking breaching copyright based on the agreements under which they have received this data, because it wasn't theirs to give. So how much student fee and research grant money do you think UEA should have spent in attempting to trace and contact thousands of data providers, paying their legal advice costs in attesting consent to the data transfer suggested to enable further peer review, when peer review had already occurred ?
"1. The emails where not private. They where sent and received using UEA equipment and had UEA email addresses on them. As such they belonged to the UEA. None of the senders or recipients of the emails therefore has any expectation of privacy with the emails."
You assume that Universities are like other empoyers in this respect. They are not. Many of my emails concern student assessment or involve counselling students about matters very personal to them, where they have reasonable expectation of privacy as well as constitutional rights guaranteeing this. The same issues cover research supervision especially where that supervision covers research degree activity. In many cases someone who is at the start of a research career is both a customer and an employee of the University in the sense they pay student fees and collect a reseach assistant salary. Universities consequently do not have the right to intrude into or publish staff or student emails without permission.
Many University staff contracts include free speech clauses, where part of the job involves developing independent views. As a consequence, private correspondence is also expected to stay private, because while a lecturer is accountable for what is said in public or published through normal peer review and critical channels, for private conversations to be able to result in the independent formation of views these have to carry an expectation of privacy.
> many originators and aggregators, and copyright had been granted to UEA
You are talking complete bollocks. Please point me a single instance of where one of the "originators and aggregators" had granted any form of copyright to the UEA. All agreements have been FOIA'ed so if they exist you should be able to find one.
> it seems unlikely that the UEA could have provided data to other parties, assuming that they wanted to, without risking breaching copyright based on the agreements..
Yet that is exactly what they did. They provided the information (free of restriction) to third parties that the UEA approved of and refused those they disapproved of.
> So how much student fee and research grant money do you think UEA should have spent in attempting to trace and contact thousands of data providers
You exaggerate. It was not thousands of data providers. There should have been no need to trace anybody since if there were agreements in place, they would have copies of these agreements and know exactly who to contact.
> You assume that Universities are like other empoyers in this respect. They are not. Many of my emails concern student assessment or involve counselling students about matters very personal to them, where they have reasonable expectation of privacy as well as constitutional rights guaranteeing this.
Universities are exactly like other employers in this respect. The University does not have to seek legal remedy to read your email. If, for example, an allegation is made against you then the University can read your email. You have no expectation of privacy.
> Many University staff contracts include free speech clauses, where part of the job involves developing independent views.
> As a consequence, private correspondence is also expected to stay private,
Then use a private means of communication such as your own personal email account.
BTW. I have worked for a University. I have signed employment contracts with them. I have also agreed to the terms and conditions on using their computer facilities. I have never seen a "free speech" clause or a clause restricting free speech. I have never seen a clause stating that emails I send and receive using their resources are private and the University has no right of access.
"BTW. I have worked for a University. "
I am not surprised to find one useless end of academia defending another useless end of academia.
Try RTFA: the material requested by Professor Jonathan Jones was not email, it was not source code, it was station data.
The scientific method requires work to be replicated. Spending hours defending attempts to prevent this is is disgraceful. Were you thrown out of your University for incompetence, corpsewood, or did you have a non-academic job?
Please get me as far away ...
... from these fools as possible.
Pass the fainting salts, please
""OH FUCK THIS. It's Sunday evening, I've worked all weekend, and just when I thought it was done I'm hitting yet another problem that's based on the hopeless state of our databases. There is no uniform data integrity, it's just a catalogue of issues that continues to grow as they're found.""
I'm failing to see how this doesn't reflect what most of us have said at one or point or another. Perhaps St. Orlowski can show us the contents of his own email account?
Re: Pass the fainting salts, please
"Data integrity - who needs it?"
DO you have a better data set?
If so it may be worth disclosing.
If each set of people who set up a data collecting weather station backl in the 17/18/19/20th century had knwon what form the data should be kept in so that it would all be handled easily now they might well have done so, but I suspect they did not.
I also suspect there is a large element of the historian and archeologist in the approach to the data sets, which is irritating enough when it is the sort of old records I sometimes have to try t interpret, and must be more so with this.
However, if someone has a gold standard set of readings to compare the data against, lets see them.
Actually there were only two files
From the judgement it's quite clear that almost all the data used in CRUTEM3 was already publicly available elsewhere. The FOIA request related to a relatively small quantity of data that wasn't.
All UEA had to do was hand over two files. Nothing else. It would have been no more than days work for one person. Instead, they chose to make a stand, as a point of principle, to fight off the 'barbarians'.
Originally, there were (I think) 2 FOIA request for the data. When UEA refused on (among other grounds) breach of confidentiality agreements, it generated a flurry of new FOIA requests to see those agreements because the requesters thought they were bogus. In fact most of the requests the UEA received were actually to see those confidentiality agreements, and not for data or anything else...
UEA have no excuse. It's a massive FAIL on their part and was probably the trigger for whoever helped themselves to the content of the UEA email server.
Just because there isn't a better data set
doesn't give them the right to bollux it up and then proclaim it is "good". Moreover, if it IS the definitive data set, all the more reason access to it should be given to all and not restricted to the high preisthood of the Cult of AGW.