I'm sure that, for a price,
google will be more than happy to provide HMG with any info they ask for.
Sorted.
The UK's Office for National Statistics is under pressure. Every decade since 1801, it has carried out one of the world's most comprehensive statistical undertakings, the census. Now, it has until 2021 to prove it can do so without the massive surveys it still relies upon. The population survey, which was sent out to 25 …
Quite honestly they could do that or just make it up based on a quick survey in the office because the government never acts on the data its got.
The don't provide enough schools even when they could see a growth in the number of school age children coming. They act all surprised about an ageing population even though that's been obvious for decades.
The just look at the short term and what will make them popular and if it goes wrong the usual cookbook is something like this.
1. Leak some stories showing that s small group of people are at fault, immigrants are always a popular target.
2. If it's not immigrants then it's some kind of "cheat" who's depriving the rest of us.
3. Publish some distorted statistics to prove they're spending more than ever before.
4. Publish another made up number to show the problem sin't as big as it is.
5. Publish another story showing that it's our fault as we're not using the service correctly.
6. Blame it on the previous government.
7. Pretend that the big boys (EU) mad them do it.
8. If this doesn't work then stick your fingers in your ears and repeat some of the above in a loud voice.
Never spend long defending any of the above, just move on to the next one in the hope that no-one will notice their lies.
One of the primary uses of a census is as a reality check : this is the actual state of the country.
You cannot have that reality check if you are just scraping databases for info. In that case, you're just compiling existing data and you'll miss the possibility of including pointed questions to uncover some specific behavioural or societal change.
Do it online, by all means, supplement the census with administrative data if you wish, but do not reduce the census to just perusing databases.
Agree that re-using administrative data is likely to become a turd-polishing exercise.
As a secondary consideration, I do relish the chance - once every 10 years - to declare my faith as "Jedi" and fear this may be lost in my on-grid footprint (unless they also trawl the comments section of El Reg.
I do relish the chance - once every 10 years - to declare my faith as "Jedi" and fear this may be lost in my on-grid footprint
That's a bit last century, don't you think? How about a social media campaign to encourage people to assert Dawkinism as their religion? That should leave almost nobody unoffended.
Dawkinsism. Me, I claim pedanticism as my religion.
Blasphemer!
As an adherent of a different abominable religion, you are not entitled to make this claim! Curse you, infidel! It is and always will be Dawkinism, until Saint Richard both dies and is then reborn in the perfect form - that is, without the "s".
And then it still will be Dawkinism
The article's initial premise is wrong. I'm sure that government would like to save money - and that's a driver of what's going on.
But one of the initial reasons that it was thought 2011 would be the last census is that people just weren't filling out the forms. Worse it was the poorest people most unrepresented in the data - and they're the ones who need government services most. There's a particular problem with people in temporary or multi-occupancy accommodation - again likely to be the poorest.
A lot of money was spent on paying people to knock on doors and go round checking this stuff - and we're talking tens of thousands of people here, so actual serious money (even by government standards). Plus the forms then come back hand-written to had to be scanned electronically and lots of it input by hand. Again that costs serious money.
So online makes sense, but again, it's the poorest who're least likely to get spotted.
Another huge problem is people with poor english. Again, likely to be poorer, and more in need of government services. Much less likely to fill out a form, and often not likely to answer the door to government surveyors - or let them in even if they do open up. This is a hugely significant part of the population in some places. Especially ones with high levels of immigration.
This then ties in with my final point. Time. The data gets increasingly out-of-date - so doesn't pick up changes in patterns of population movement.
Take the example of Slough (has all the above problems). They were begging the Labour government for more cash because they had high levels of immigration from both EU and non-EU sources during the last decade. Plus got a lot of the asylum seekers who come through Heathrow (and need much more government support than most immigrants). But their local government grant from the Treasury was based on census data from 2001 - when immigration levels were much lower.
I would have thought that information collected by (say councils) can only be used for the original purposes they were collected for. they would have to get permission from the invidivial users to re-use that data for other purposes. Unless government (local and national) can just ignore these rules!
"I would have thought that information collected by (say councils) can only be used for the original purposes they were collected for. they would have to get permission from the invidivial users to re-use that data for other purposes. Unless government (local and national) can just ignore these rules!"
They would simply post on their local gov.uk portion of the councils website that they are making use of the data opt-in by default, and then bury that info as deeply as they can - if you don't want your info to be collated, simply send in your details to this P.O. box, on the back of a smutty 1960's postcard from Blackpool, with the simple message "opt-out" written in lemon juice, using a penny-black stamp! We'll make sure it's not used! Promise!!
Why do everyone say Ooo GDPR won't allow that....
Exceptions.
-other important public interests, in particular economic or financial interests, including budgetary and taxation matters, public health and security;
Quite easily would a census stat come under that, as the census is used to predict and validate government budgets, health provision and education as well as a thousand other thing.
Also the raw census data doesn't get published for 100 years - at least that's how the law always used to be. I believe the last census to go public was 1911. Great if you're a historian, economist or into family history.
In the meantime anonymised data sets will be released in varying levels of detail.
"Also the raw census data doesn't get published for 100 years - at least that's how the law always used to be. I believe the last census to go public was 1911. Great if you're a historian, economist or into family history."
the next one will go live in 2021 (1921 census) but then we'll have to wait a while: the 1931 census (at least for England and Wales) was destroyed in a fire, and the 1941 census didn't happen because the UK had more important things to be getting on with at the time.
>>Great if you're a historian, economist or into family history...
Except when, for some reason, a huge American commercial enterprise [ie. Ancestry. com] seems to have somehow managed to be appointed gatekeepers of this info and are allowed to charge us for access to it —profiting from selling us back the data which our ancestors were legally obliged to provide.
[One of the reasons I refuse to fill in any of these census or census-like forms]
"Except when, for some reason, a huge American commercial enterprise [ie. Ancestry. com] seems to have somehow managed to be appointed gatekeepers of this info and are allowed to charge us for access to it —profiting from selling us back the data which our ancestors were legally obliged to provide."
Anyone can go to Kew and look at it. It appears, although I'm not sure, that Ancestry.com digitized the lot, and so they have access to it. It's not clear to me whether they paid for the privilege of doing this, I suspect so. If another company wants to do this, I would guess that the ONS not only would be happy to allow it, but would legally be required to do so.
That exemption is a Member State Derogation. ie. Member states can execute discretion to legislate over these areas. The UK won't be a member state, so we won't be entitled to derogations of our own making unless we obtain an agreement to the contrary, dispite the fact we're pretty likely to stiill have to comply with thme.
As usual, government only interested in saving money, and creating Big Brother.
The paper census has a lot going for it. I can still look at the schedules from 1841, without having to wonder how I can get a copy of Windows -37 or Office 1848 to read it. Censuses are a primary source for studying and understanding our history, albeit imperfect - not that our governments are famous for wanting to learn from the past. Live in it? Yes, but not learn from it. And given their habit of totally ignoring evidence that disagrees with their knee-jerk prejudices, why do they want any data at all?
We need to keep the paper census. Yes it costs, but it's worth it. Add a few more questions each time, and keep the data completely separate from other government databases. And in a hundred years our great-grand-children can look at them, see how bad great-grandma's handwriting was, and marvel at great-grandpa being a Jedi Knight!
And in a hundred years our great-grand-children can look at them, see how bad great-grandma's handwriting was, and marvel at great-grandpa being a Jedi Knight!
Way before that ever happens, some civil servant type will have left the document on a train/pub/hookers nightstand.
The main advantage of digital that I see is that it removes one leg of incomoeptence from our overssers. On the other hand, remind me again when lessons will actually be learned about keeping our digital data secure too?
We need to keep the paper census. Yes it costs, but it's worth it. ... And in a hundred years our great-grand-children can look at them, see how bad great-grandma's handwriting was, and marvel at great-grandpa being a Jedi Knight!
But the paper census is already damaged. The 2011 Census was the first at which you could fill in the questionnaire online instead of on paper. The percentage of online submissions was in the mid-teens.
Yes, the paper forms were scanned. and stored on micro-film, so if anyone is still around in 2111, they will be available to all.* But the information submitted online will never be seen.
* Pro tip - one way of sending a message to your descendants is to scrawl it on your Census form. If it's not obscene or defamatory, it will appear in 100 years' time!
Pen-y-gors,
See my (over-long) post above for some reasons why this got changed, other than money. Obviously saving money may have also been a major factor.
But to cut a long post story short, the census was becoming increasingly unreliable. Far fewer people were filling the form in, paritcularly amongst poorer people in temporary/shared occupancy accommodation. And most particularly amongst immigrants, becoming a much larger group in recent years and often concentrated in specific areas.
That was making the census much less useful for its most important job, planning local services, and that use trumps the historical one.
One of the really important points of the census is to validate all the other data excercises going on day to day. ie. is the admin data any good, or does government miss big sections of the population (always shows that it does..).
This move threatens to render missed groups off the radar indefinately, which has all kinds of knock on in service provision, democracy etc.
Doing bits of it online to save postage is seems fine form that this point of view, but not doing it at all is a huge failing.
Strongly agree - the increased use of digitally enabled services and data collection on government funded service provision, and infrastructure can make use of the administratively collected records to reduce the reliance on another centrally managed detailed survey (census).
The reality is that there are data quality issues within each and every database and data collection. The art in this digitally enabled world is to leverage off available data sources and importantly make use of Data Science with consistent and considered quality reviews and processes, in order to obtain an integrated data asset for the community that is fit for purpose.
"Once it goes through the anonymisation process... our researchers ... link the data together to produce the aggregate insights"
Otherwise known as deanonymisation.
If you aggregate information, it becomes even more anonymous. Instead of knowing who's a Jedi, you just know how many there are in whatever container you aggregated down to,
Unfortunately, the usefulness of a survey in the future is greater if the actual data can be used. The 2021 release of the 1921 survey will be interesting to me as I will be able to find out about my grandparents. Aggregated data would tell me that one area had more people but the other was still down following WWI and the Spanish flu.
As someone who has recently been putting together a bit of family history... the census is an absolute boon. Sure, you can go through the Parish and Civil records for Birth/Death/Marriage/Baptism but particularly in the 1800s some of them are vague, hard to find or can be ambiguous where you have two John Smiths both getting married in the same year and you're not sure which one is yours. A household census that shows the whole family (with servants, lodgers, occupations and the rest) paints a broader social picture.
The government could produce reports based on benefits or tax data for internal usage, but unless they use that to publish a derived dataset (household, dependents, etc) in a Census-like format, then that data is unlikely going to be searchable in the future.
It would be a great loss to sociology and social history (both today and for future researchers) to abandon any sort of survey. The 2011 census cost £485m which (at £48m/yr) is chump change and - IMO - worth every penny.
Paper forms will last hundreds of years under the right storage conditions. CDs won't. I know ONS are probably acutely aware of this, but digital data will need to be re-stored (and perhaps restored) every so often, and we all know what happens to any funds that are supposed to be dedicated to this.
Apparently Elon Musk's roadster is carrying a quartz disk with Asimov's Foundation Trilogy on it. Something like that may possibly be the future of long-term storage, as long as the design of the reader is stored in some more basic long-term medium!
"[...] so it might represent religion even if it doesn't capture religion.""
CofE and RCC parish rolls are notorious for listing as members everyone who was baptised as a baby. They are also now making it very difficult for people to remove their names from those registers.
That will inflate their perceived numbers for those still alive who were born before the 1970s when baptism was effectively mandatory. Official forms until the end of the 20th century still insisted on a tick box for an organised religion. It was a long time before a tick box for "none" caught up with the actuality in society.
"Are you sure about that? "
Not totally. I was going on reports that both Churches - but particularly the RCC - make it difficult to achieve. It is possible that there are two parallel counts. One from the baptismal archives - and the other as estimates of attending congregations every so often.
I know of one church where it is reckoned that non-religious visitors regularly outnumber the sparse congregation - on account of the quality of the choir.
The beauty of a census was that it was a frozen point in time.
Analyses of different sets of dynamic data will not necessarily give the same correlation between sets. Whether that is better or worse than a single point in each decade is an interesting question.
The single point census gave a snapshot. The new proposals seem likely to facilitate regular tracking of people and their activities. Even if the ONS themselves use anonymised data - other fingers in the pie may not be so scrupulous. Theresa will just love that.
It's been really hard for local councils, in particular, to plan in an era when we've had 300,000 - 500,000 net immigration a year for over a decade. Obviously data from 5 years ago is bugger-all good if you get a disproportionate amount of that immigration to your area. Not so much of a problem if you have an ID card system of course, but we don't. So I think they've decided the needs of government planning are higher than those of historians.
Getting people to fill out the forms is increasingly difficult too.
Of course it doesn't help that other government data might be pants. It was discovered about 5 years ago that HMRC had created one or two million extra National Insurance numbers - basically they had duplicates for some people and were randomly assigning some of their contributions between them. I think it emerged when calculating pension contributions, and people complaining they'd paid in more than the computer said.
This will be the census of the Data Dark Ages. There are too many people who wont trust an online survey. Im just thinking of my parents, their mates, my mates parents and their mates. "Oh online survey, wont be trusting that", and I cannot blame them.
Most of my mates have real fancy smart phones but are generally inept at using them. They have traded their laptops/desktops for smart phones and wont be spending any time filling in an online form like the census.
Yes it costs money but after the Car Tax** someone needs to the give the Mandarins a good shake and tell them its too early for this type of survey.
Im no luddite but there are quite a few folk out there that are when it comes to this type of thing.
** http://www.bbc.co.uk/news/business-42009111
Hi
I had to go and have a look and for the 2001 census there was a 98% return rate but did not find anything for the last census. I would be surprised if there was a massive drop in return rates though.
https://www.ons.gov.uk/census/2001censusandearlier/dataandproducts/qualityofthecensusdata/responserates
I can't speak for other government departments, but in the past I've worked with people from the ONS and their "paranoid" approach to any one being able to individually identify any person or household from data they release is impressive.
If other data holders were like them our data would be a lot safer.
I can't help but wonder if they've got past the old storage issue that put this one on the back burner in 2011 after Francis Maude realized that the proposed aggregation of data would require IL6 storage (aka the same used by the spooks) to prevent national embarrassment, outrage and so on.
Anonymising data after it has been received is well and good, but you have to store the original data somewhere ... and last I checked, the cost of adequately securing the data was higher than the cost of the paper census.
On this side of the pond, the census decides a lot of government resources, such as health "care", housing/city planning, emergency police/fire funding, welfare, roads, and other important stuff.
And yes, we have morons that don't fill it out. I threatened to evict a roommate over that. You wanna be that kind of idiot, don't do it in my house.