* Posts by Nigel Whitfield.

1049 publicly visible posts • joined 12 Jun 2009

Dry those eyes, ad blockers are unlikely to kill the internet

Nigel Whitfield.

Re: Lazy/greedy publishers are the reason ad-blockers exist

Not being anything more than freelance, I can't comment on how exactly any particular site organises it, but even in print, ads were never checked for quality by editors, and that was in the good old days of the 90s when we had decent sized editorial teams, and only had to produce one thumping great magazine a month.

Publishing now typically has much smaller teams of people, producing far more content because it's not enough to have a batch of material once a month. There must be things going live all day, every day to ensure the traffic never stops.

(Whether or not that's a good thing is, at least in part, a separate discussion).

But the downsizing of editorial teams means that the idea that a site could produce the amount of content people want, and vet every advert, without needing even more staff, is quite fanciful in my view - certainly if you want to pay the people who write enough to actually live on.

Many sites do have their own ad team, as well as using ad networks - they're the people who will do things like the 'site takeovers' for big clients from time to time. But to get all that low hanging fruit, it almost certainly is more efficient, given staffing levels, to contract this out.

Where the problem lies, really, is in the ad networks and the way in which, thanks to brokers, once you put a box on your site from a particular network, you have no real say in what might appear there, save for some category tweaks and the ability to block something after the fact, if it causes you a lot of aggro.

Ideally, instead of these largely automated networks of ads, we'd have real people curating them, but that would probably require cooperation between various publishers to create their own 'trust ad' network with much stricter rules for what could be accepted.

It might well be possible for a large publishing group - the Condé Nasts of this world - to do something like that, and only accept ads via their own in-house network. But I suspect the economics would be strongly against it for a smaller company with only a couple of online titles.

(Caveat: as I say, I'm a freelance these days, and not privy to any information about the financials; I simply know that on the editorial side it's a case of fewer people producing a lot more work than in the past)

Nigel Whitfield.

Re: New TV

If you go for the latest Panasonics, they have Freeview Play integrated, so backwards EPG for access to the catch up services and so on.

I've not seen the EPG in those myself, but I think it's similar to the Play EPG in the Humax box, which I did see

Nigel Whitfield.

Re: If advertisers want to reach me...

But what would that fee bee? Unless it's incredibly small, you automatically bias these types of reviews towards the big companies with a large marketing budget, because small ones may have an innovative product, but they won't have the money to ensure widespread coverage.

If it's too small, however, you're not going to get people doing a really thorough job as a reviewer, if that's their day job. You're only going to get people who can afford to do it for pocket money while they have a 'real' job that pays, or those who will rush through as many reviews as they can to make enough to pay the mortgage.

Not to mention, I think a lot of people would be pretty distrustful of a situation in which the manufacturers were directly paying the reviewers. That's exactly the sort of thing that does open up people to accusations of bias, and it would take a big mindshift to change people's views.

Five years ago, I did a series of blog posts about How reviews work and I don't think much has changed (except, of course, the rate for the job still hasn't kept up with inflation).

Meaningful gesture: Thalmic Labs Myo motion sensing armband

Nigel Whitfield.
Thumb Up

Re: Leg

I don't see why not - all it's doing is reading the electrical signals beneath the skin. Obviously the calibration is set up with arms in mind, but you can create your own profile or even access the raw EMG data.

So, you might need to do some fiddling to come up with appropriate gestures, but I don't see why not. Maybe I'll have a play later and see what I can make it do.

Nigel Whitfield.

Maybe; I was chatting about it in the pub last night with someone who manages band tours, and we thought it could be handy for some of that - giving another way of controlling some aspects of a performance, without having to step away from the mic, for example.

Nigel Whitfield.

Re: Masturbation

The presentation mode does include an on screen pointer. But yes, while it's a neat idea I have mixed feelings about how useful it could be. I suspect that it may be a boon to people with some conditions who may find using other types of controller difficult, but for a lot of us you're entirely right that a normal remote is likely to be simpler for a presentation.

PHONE me if you feel DIRTY: Yanks and 'Nadians wave bye-bye to magstripe

Nigel Whitfield.

Bang!

Perhaps now the americans too can enjoy the boom in gas-powered ATM thefts. Essentially, in the last couple of years, filling ATMs with gas and blowing them up has become a popular way to rob them. And one of the reasons for that is increasing use of chip cards - no longer can you simply clone a card and use it to withdraw lots of money from an ATM.

In January this Bloomberg article looked at the phenomenon. Back then, no US machine had been robbed that way, and it was largely a European phenomenon. Now that the chip cards have been rolled out, I wonder how long they'll have to wait.

4K catches fire with OTT streamers, while broadcasters burn

Nigel Whitfield.

Re: Waste of time and money, you can't break the rules of biology.

We covered the maths here a couple of years ago:

http://www.theregister.co.uk/2011/05/19/tv_sizes_deconstructed/

Nigel Whitfield.

Re: "a decent fibre network connection (around 15Mbps)"

In my last Breaking Fad, I included a graph from Netflix, with their ISP speed index. You can see the August figures on their blog

This is the speed that they are managing to stream during prime time. Top of that list is Virgin, with 4.02.

You might have a headline speed much more than that, but that's the average that they're able to pump out IPTV. There's a hell of a long way to go before it's a reasonable delivery mechanism that's capable of replacing broadcast.

Nigel Whitfield.

Broadcasters will have to spend a huge amount of money - production and playout chains will need updating, as well a just cameras. Some of them won't even have finished their HD upgrades, let alone written off the costs of the equipment.

Sure, some tech-heads will lament that people like the BBC aren't rushing to bring 4K to their brand new TV. But imagine the glee with which others would turn that into "BBC replaces almost new HD kit just to bring 4K pictures to homes of super rich execs."

As I said in my last Breaking Fad, there are still standards to sort out, and I think it's much more realistic for broadcasters to be looking a little further down the line, for when those are ready, and IP-based production systems have replaced SDI.

Even the OTT bunch are going to have a job on their hands getting 4K to anything more than a very small subset of their customers for a few years anyway. Sure, the TV makers want to flog us new sets and get as much volume out of their new panel factories as they can.

But, as far as I'm concerned, the sensible thing is still to wait, for both consumers and broadcasters - especially the public service ones.

Only a CNUT would hold back the waves of the sharing economy

Nigel Whitfield.

Re: Very good article, would read again

London doesn't have the arbitrary hard limit on numbers that cities with a medallion system impose, so in that regard, anyone could become a taxi driver.

However, the Knowledge takes a while to learn, so to a degree that does slow down the potential for growth in the number of drivers.

Private hire vehicles (aka mini-cabs) don't have to learn the knowledge, and again there is no fixed limit on the number of people who can register for those.

Nigel Whitfield.

Re: Self Employment [was Repeated Typo!]

When it comes to Uber and similar services, the issue isn't just self employment - as others have said, many of us have done that for years. But in a lot of cases, it's been the case that people are self employed within specific fields (like journalism, say, or being an electrician) which may have qualifications, skillsets or a need for equipment which serves as a barrier to entry. So, while self employment in that sense can be hard work, it can also be quite rewarding too, both financially and in other ways.

However Uber, Fiverr, People Per Hour and so on are not just about self employment, but casualisation, and sometimes the atomisation of work into ever smaller units. Much as politicians always say "I met someone who was overjoyed to have a zero hours contract, because it fit with their lifestyle," there will be some for whom this model does indeed work, and who will be held up as the poster boys. But there will be a great many for whom it represents nothing more than a decline in wages, standards of living, and security.

Nigel Whitfield.

Re: TFL protecting users?

The five minute wait period is, as far as I can see, an attempt to ensure that private hire vehicles (and that's surely the category into which Uber falls) all play by the same rules.

Black cabs have certain privileges, like plying for trade on the streets, being able to use a taximeter to charge based on distance/time, and being able to drive in bus lanes.

However, in return for that, they have obligations too, including wheelchair accessibility, and the Knowledge.

I think some within TfL view the 'near instant' availablility (or promise of it) within apps like Uber as an end run around the rule about plying for trade on the streets, attempting to grab that market, without shouldering any of the responsibilities that have been associated with it.

It would be interesting, indeed, if TfL were to also propose that the wait period would be waived for vehicles that were certified wheelchair accessible.

It is odd that the concept of balancing rights and responsibilities, so beloved of right wing politicians when it comes to people doesn't seem anywhere near as important to them when it's companies involved.

Nigel Whitfield.

Re: Repeated Typo!

I think it's extremely relevant, because it goes to the heart of the sort of society we want to have.

Personally, I think that a world where people have some job security, and some basic idea of what they'll be earning, the right to be ill without suffering massive financial penalties, the right to paid holiday and health care. All those are, to me, fundemental to a decent society where as many people as possible can have a good life.

The increasing casualisation of labour that things like Uber, Fiverr and so on are bringing about is not something that helps build a stable society for all. The very rich love the idea of being able to hire and fire at will, and treating people as assets. But if you're one of those assets, and you actually want a stable environment in which to bring up a family, it's not so much fun.

Nigel Whitfield.

Re: Repeated Typo!

I think the value is from all those types with money salivating at the thought of an "on demand" economy where the actual workers aren't employed, so you don't have to provide them with health care, holiday, rights, and so on, and they're eternally grateful for the apps that direct them to whatever scraps they can pick up from day to day. Scraps, naturally, on which you take a hefty commission.

Uber is aiming to become the 21st century version of a foreman at the docks, deciding which of the many people queuing up each day are going to get work

Slander-as-a-service: Peeple app wants people to rate and review you – whether you like it or not

Nigel Whitfield.

Great for lion killers, apparently.

One of the fools behind this was interviewed on the World Service around 4am today, and I happened to catch it.

She was trying to explain that it would help give a rounded view of people, and dug up the example of the lion-killing dentist, saying that if their app had been around, people would have a fairer view, because it would include comments from before his 'problems'.

I think that if you have to try and justify your horrific web service by saying that it would help present a better view of a sadist who shoots wild animals for fun, you're on pretty dodgy ground.

The last post: Building your own mail server, Part 3

Nigel Whitfield.

Re: Citadel - takes about 20 minutes

Well, quite; you can install OpenBSD and add the packages used here in a pretty short space of time. And though some people will find using vi take a bit longer to edit config files than filling in some web forms, it doesn't actually take an awfully long time.

I don't think the process we showed here is massively involved really - but we did show it in stages, where pretty much each one leaves you with a working system, and hopefully at least an understanding of how everything fits together.

It may well be quicker to grab an off the shelf package that wraps everything up with wizards, but I think learning how the parts fit together is always a worthwhile exercise.

Nigel Whitfield.

@TheVogon: Thanks for that; when I set up SPF myself originally, I used the basic wizard on the OpenSPF site, and I see that's now gone away.

I agree that it's pretty straightforward to create a record, especially using a wizard like that. And it's easy enough to turn on in most MTAs or spam filters.

Of course, it's also pretty easy for senders of marketing emails to create their own SPF records, or sign messages with DKIM. So SPF is probably most useful against zombies and compromised systems.

I have a hard fail (-all) in my SPF record, is also partly why I have the submission port set up on the mail server, to ensure that wherever I am, outbound messages always come from that address. I don't see the point of anything other than the hard fail option, really - it seems on the face of it to tremendously reduce the utility of SPF.

Nigel Whitfield.

Re: Wot no SPF?

Spamassassin will check SPF records as part of the config described here, and I'd recommend that you set them up for any server you're running; a few other people mentioned that in the comments last week as well.

I didn't really have a lots of space to cover all the network side of the config - there are only so many pages long an article can be before it's really annoying clicking through, sadly.

Anyway, for people not familiar with it, SPF is a fairly straightforward DNS entry - you create a TXT entry for your domain, using a specific syntax, and it essentially allows you to say "Only these specific hosts are allows to send mail from my domain. If you get mail from nigelwhitfield.com that comes from anywhere else, reject it"

Checking isn't mandatory, but many sites do use it. You can learn more at OpenSPF but bear in mind two things

1. Set it up wrong and you may cause lots of servers to reject your messages

2. If you host mailing lists or forward mail for people, SPF can cause problems with that

Nigel Whitfield.

Re: linux email distribution ?

Most of what has been described in these articles will work just fine on Linux too. The first part explains a little bit about the choice of OpenBSD.

There will obviously be a few differences when it comes to things like package installation, start-up scripts and file paths.

Nigel Whitfield.

Here's an example of using Postscreen with the config from the article, to do preliminary checks against DNSBL services. As I say, I've only recently been tinkering with this, so I'm not going to recommend specific block lists to use until I have more experience.

Using postscreen to filter connections

Nigel Whitfield.

Re: a third way?

Off the top of my head, I'd suggest looking at some of the packages for a NAS. For instance, if you have a Synology then their Mail Server package allows you to host IMAP mailboxes for people, and calendars can be handled with CalDAV via the control panel's WebDav settings

This tutorial covers the basics: https://www.synology.com/en-global/knowledgebase/tutorials/463 though note that on DSM 5.2 you'll find WebDAV settings not at the top level of the control panel but in the 'File Services' section

Nigel Whitfield.

Re: rsync, security?

In that particular instance, I'm usually zapping stuff to my Synology via the second (private) network port on the box, so I don't worry too.much about it.

Yes, if I were sending stuff over the public net, it would be wise to use encryption, SSH and distract the agencies with a squirrel too.

Nigel Whitfield.

I've been experimenting with using Postscreen to do that, and it looks promising. The reason it didn't make it into the config in the article is that I've not been running it for more than a few weeks so far. But if there's interest, I can summarise the changes you need to do for that.

Nigel Whitfield.

Yes, I think if it's just a couple of people and you have the wits to recognise a phishing email, then just a combination of greylisting and filtering based on DNSBLs will very likely cut things down to manageable proportions. Certainly, greylisting is the one thing that keeps the biggest load off my servers.

On the other hand, if you have less technically aware people, then something that can spot things like phishing and viruses is probably a very good idea.

Mostly, you can build up from each of the parts of this tutorial and stop when you've got to the level of protection that feels right for you. Or indeed you can drive yourself made with tweaks to Amavis settings and custom rules for SpamAssassin if you want to crack down even more.

Nigel Whitfield.

Re: sanesecurity signatures for clamav

Thanks for that; looks very useful.

NEW ERA for HUMANITY? NASA says something 'major' FOUND ON MARS

Nigel Whitfield.

The cynic in me thinks ...

How about a movie tie in? With The Martian opening this week, what better time to talk about the science of growing stuff in your own shit.

Perhaps that would be too cynical, even for a cash-strapped NASA and an eager marketing team. It would, however, be hilarious if they wheel out Matt Damon.

Look out, Silicon Valley – Don Basile is back to kick ass and chew gum

Nigel Whitfield.

Re: Get the saying right!!

Here's the trailer; at 1:09 it's definitely an AND

https://youtu.be/iJC4R1uXDaE?t=1m9s

If you got Netflix for Miss Marple, you're out of luck (and a bit odd)

Nigel Whitfield.

Re: Dating myself somewhat...

Have a nice lie down and a lovely tune.

Nigel Whitfield.

Re: Disc rental

I remember using ScreenSelect a long time ago (a dive into the email folders reveals it was 2004). Although it was a nice enough idea, I think what killed it for me at the time was the horribly erratic service from the post office

Nigel Whitfield.

Well, that would work, but I doubt you'll get the movie companies to agree to it, sadly.

UltraViolet is, I think, perhaps best viewed as an adjunct to buying a real disc. A nice to have, but I won't buy just the digital copy again.

Nigel Whitfield.

Re: Elysium

You mean a bit like the end of BSG ?

I don't know why they have to go and ruin things with all that mystical woo.

As for Elysium, I like to think of it as the story of one man's fight for socalised healthcare.

Break from the future: Hold the new stuff and fix the web first

Nigel Whitfield.

Re: >Geolocation API, possibly the most useful HTML API out there

The geolocation API does ask for permission, and does it on a per site basis, so in that regard, it's not really any different from allowing an app to access your position, as far as I can see.

And sometimes, it is just simpler to let the site work out where you are, especially if it's somewhere unfamiliar, or just to save time.

On one of my sites, we have a 'Visiting' option so people can update their profiles when travelling. You can enter city and country, or just choose 'Geolocation' and we'll work it out for you. As far as I know, there's no way a site can grab your details without a pop-up the first time you visit.

We saw the future: Apart from the bath apps it looks like the past

Nigel Whitfield.

I do enjoy a good skewering ...

... but I fear Mr Caplin may have raised the bar for cynicism so high I'll never be able to top that.

The last post: Building your own mail server, part 2

Nigel Whitfield.

Re: Problem with running your own mail server now is the big 3 do not trust your email by default

I have to say that that's not my experience; I have SPF, but not DKIM, and other than the very occasional wobbly with Hotmail (which has a habit of vanishing mail into black holes, though I suspect more to do with user prefs than anything else), I don't have any problems.

I have a server set up in a dedicated hosting centre as well, as we send out newsletters to around 1500 people a week, and again largely no problems, nor indeed with the various welcome messages when people sign up to that site.

If what you're suggesting is true, how would anyone ever be able to launch a new service that requires verification of users' email addresses?

Nigel Whitfield.

Re: Smarthost likely required

I'd certainly always recommend delivering to the right place, rather than via a smarthost, and that's what the setup in the articles will do, and why I kept the smarthost tips for the comments.

Clearly, from the comments this week and last, a lot of people are concerned about whether or not they're able to do this on their home connection, and it's worth providing some options - and tips about when they may turn out not to be so helpful.

But, yes, ideally, I'd recommend that you get yourself a fixed IP from a provider that will also let you set up rDNS for it

Nigel Whitfield.

Re: Used to run my own mail server 6 or 7 years ago, but stopped

There are also plenty of config options in Postfix to control how the smtp daemon will behave. For example, maximum simultaneous connections per client, maximum connection rate per client, and also options to slow down the smtp chat (effectively tarpitting) based on the number of errors.

Some options in the config can be set to be 'stress dependent' too, so you set a limit to the number of SMTP processes you want to run (eg 50) and if that's reached, then the stress config is used.

Nigel Whitfield.

Re: Smarthost likely required

The Postfix docs cover quite a bit about these issues. Setting up a host for outbound relaying of all mail is simple:

relayhost = [mail.isp.com]:587

would send via port 587 on your ISP, for example. It's also quite easy (described in full here) to use SASL on that connection.

You can, if you want to get really fancy, have different outbound hosts for different email, and differed SASL credentials for each person too.

Nigel Whitfield.

Re: Used to run my own mail server 6 or 7 years ago, but stopped

I agree;on the whole, I don't find that there's a huge number of attempts to do anything other than send mail through a mail server, and robust filtering can manage that pretty well.

While a system with a handy web-based config server is appealing, especially to the novice, the web server itself means there are more things open to attack. By starting with OpenBSD and adding only the things we want, there's far less of a surface for people to attack.

I don't believe I"ve ever seen a DoS against the mail server, and it's quite easy to limit concurrent connections if you do need to. The good thing about mail, of course, is that a properly configured sender will retry anyway.

Certainly, compared to the hammering you'll see the moment you have something listening on port 5060 (SIP) accessible from the internet, a mail system suffers very little. In theory, my phones are set up so people can call using my main email address via SIP. In practise, it lasted a couple of hours before I had to give that idea up an allow calls only via my SIP trunk provider

Nigel Whitfield.

Re: smtputf8_enable is true, but EAI support is not compiled in

Don't panic - it's only a warning and you can just add

smtputf8_enable = no

to the config file to avoid it, which won't cause any problems unless you want to use extended characters in domain and user names

Nigel Whitfield.

Re: Push email for mobile?

Dovecot does, but certainly some older versions of iOS didn't support it in the mail client. I don't know what the current version is like, as my only iDevice is on iOS 7. Or possibly 6.

Nigel Whitfield.

Well, there are a few other tools I didn't mention - and as I've said before, this isn't the definitive solution, because there's no such thing. It does, however, work for me with these tools. Though it doesn't do the tarpitting, Postfix's postscreen can effectively do the same job too. I'm actually experimenting with that at the moment, using it to drop connections based on RBLs before passing them on to the PostGray and then amavisd; there are an awful lot of ways to skin this particular cat.

With regard to the companies that don't play nicely with greylisting, the Postgrey package installs a list of the major offenders, who are automatically whitelisted, and you can of course tweak that yourself.

Nigel Whitfield.

If you use Pocket, that seems to manage to collate all the content into one chunk - certainly when I save a multi-page Reg article to Pocket and then read it on my Kobo, it comes out as a single piece.

Nigel Whitfield.

Re: Push email for mobile?

A quick read around suggests that z-push may be worth a look at too; it can use Maildir as a backend, and as far as I can see the notifications are done via long polling. z-push.org - but not played with it myself, as I'm happy with Maildroid.

Nigel Whitfield.

Re: Push email for mobile?

The plan for next week, so far, is to go through the config for spam and virus filtering (the original plan was to do it all in this part, but I felt I'd have to compress things so much it wouldn't be helpful).

In terms of notifications, a few things come to mind off the top of my head, at least as far as Android is concerned.

Firstly, choose your mail client. For example, MailDroid has a various options to control polling, including control over whether or not connections are maintained when you exit a mailbox, and whether or not mail should be checked when the device is asleep. For me, that's fine.

Secondly, you can sign up with Google to send notifications over their network to Android devices via GCM. It's quite easy, having registered for that, to send messages from the command like, for instance via a PHP script or curl.

So, in terms of getting notifications out, you could use a Procmail recipe to send a push notification for new messages.

On the client side, I'd then knock up a very simple listener to register with GCM and display the notifications, and start the mail client if required.

If you don't want to do any coding yourself, you could use something like PushOver.net to deliver notifications, though it's not completely free.

Nigel Whitfield.

Re: Good luck to "vi" something - if you're not used to it

If you're not on a system with pkg_add, yes, you'll get that error.

If, however, you're following the steps in the article, and using OpenBSD, then that's exactly the command to install nano, and you can do that right at the beginning before you need to edit any config files. Then, throughout, instead of "vi main.cf" or whatever, type "nano main.cf"

Nigel Whitfield.

Re: Why a PC?

I depends on how you view email; for me, as I mentioned in part 1, it's part of an audit trail of discussions about work and so on, so I do keep it.

I'll take a look at Prayer (though since I use IMAP, it's easy to sync a phone to the same set of mail folders); I've played with Horde in the past, which does the job, but is a big of a big beast to install and set up

Nigel Whitfield.

Re: Good luck to "vi" something - if you're not used to it

If you want to use nano, just

pkg_add nano

will do the trick; likewise for other favourite editors. We got taught vi in pretty much the first week at uni. Mind you, they didn't teach us C programming in those days!

The last post: Building your own mail server, part 1

Nigel Whitfield.

Re: Fail2Ban?

Ooops; correct link to the patch is http://www.djs.to/2013/10/1-postfix-sasl-support-for-sshguard/

Nigel Whitfield.

Re: Fail2Ban?

sshguard will do similar for you, and works nicely with the OpenBSD pf firewall. It's available as a package, and if you want to use it to catch SASL login attempts, there's a patch for that here