Feeds

* Posts by Trevor_Pott

4341 posts • joined 31 May 2010

How I poured a client's emails straight into the spam bin – with one Friday evening change

Trevor_Pott
Gold badge

Re: ^^3 day SLA?

For issues where it is creating an outage, I do. Although even paid incident support offered - for the best instance - 18 hour resolution. It's ultimately what has ended up driving most of my clients to Google Apps.

Gmail is nowhere near as feature rich or awesome as Office 365...but it fucking works, and most SMBs simply don't use 99.9% of the features in Exchange anyways.

0
0
Trevor_Pott
Gold badge

Re: What's with the Google fascination?

I'd agree with you, for organizations willing to invest in the full stack. Exchange needs more than just exchange to get the benefits you speak of...and that stack needs a dedicated full time admin. Not an admin who is also doing storage, networking, applications, desktop support, websites, Linux, etc.

It was one thing to be the gneeralist who lumped in "and exchange" back in the Exchange 2000 or 2003 days. It's another thing entirely to try to keep up with e-mail today. Even for "basic" MTAs, there is so much to configure, and so many "conventions" on configuration you have to abide by to stay off greylists that it's crazy.

I agree exchange is amazing. I rather like it for many things...but only in cases where you're willing to pay the tithe. That means proper hosted AS. It also means keeping up to date on clients and all ancillary applications that tie into it.

As a unified communications stack, Exchange/Lync/Sharepoint/etc can be very powerful. But they aren't wrapper-ware and they aren't particularly good past their "best before" dates.

Where exchange truly shines is in things like retention rules, archiving, and all related stuff. If you need to do things like legal holds, in-depth content scanning, Exchange is pretty goddamned hard to beat.

The problem is that most companies absolutely don't need that stuff. They never use it, but they're sold on the idea that they "need" etiher the top-end collaboration stuff or the in-depth retention/legal policy framework, despite never actually wanting to engage any of it.

Worse, you sometimes get a CIO who thinks it's all really, really cool and wants everyone to use it, but simply can't get buy-in from the staff. Usually they'll try everything, including outright threats and bullying, but the staff have non-technological ways of communicating and getting things done that are simply faster and far more efficient for them.

The biggest thing I see with my SMBs is people wanting to use the full Microsoft stack to be "more efficient" at communications because one or two people (who typically telework for some or all of their day) feel "out of the loop." They try to impose a technological solution on a human problem and it fails every single time. The problem isn't that people don't use the relevant technology, it's typically that they're an asshole, or that they simply choose not to give a fuck about $issue until it there's a problem.

Exchange isn't - and can't be - a replacement for human beings taking responsibility for their actions, taking the time out to think about the various projects that needs be done, or actually taking the time to answer the various and sundry e-mails and communications that need answers. Making communications "more efficient" doesn't force people to actually acknowledge one another, keep eachother in the loop or convince the powers that be to make a fucking decision about something.

It absolutely doesn't force overworked people to sort their crap and "properly file" digital data. If you have problems with people using a single public share as a catch-all wastebin where they store everything "because everyone has access and it's more convenient" then public folders and/or sharepoint are just going to look the exact same. The issue there is the people, their habits and their workload, not the technological tools available to them.

When and where exchange can make a difference, I absolutely champion it's use. Exchange is one piece in the best groupware and productivity stack on the planet. Period.

But I do not champion it's use in most SMBs. I think that's ridiculous overkill. Hell, even Office 365 which is designed to be simple to administer (compared to Exchange) and offers only a subset of features is something where 98% of all SMBs I've worked with that use it simply don't change anything past defaults.

So, while I think Exchange is grand, I can't and don't recommend it for SMBs, unless the SMB has a definable need for it and they're willing to pay for it. Regular updates, proper amounts of sysadmin time, proper hosted AS and enough server licenses and hardware to make it all go.

I will never do another exchange install that doesn't have Exchange Enterprise Cal Suite for each user and hosted AS. There will also be a minimum of three server licenses involved: one dedicated hub transport server and at least two storage servers in a cluster. They will also be backed up using Data Protection Manager and monitored using System Center.

The floor cost for this is simply higher than most SMBs are willing to pay, to say nothing of the ongoing costs of keeping it ticking along.

Here's a great example: try running Update Rollup 3 if you'd disabled IPv6. Whole thing goes pear-shaped. Worked fine without IPv6 until then, then *bam*, implosion upon update. There are various reasons why IPv6 had to be disabled in one of the environments. Update happens along, murders exchange. Figuring out what went wrong, then applying the fix takes a proper sysadmin.

Ideally, you never encounter the error because everything exists in a test environment, all patches are vetted, etc. How often do you think that happens in an SMB where you don't have things like "dedicated Microsoft communications stack admins" or even "dedicated Microsoft admins?"

And so we get to the heart of it: Exchange is an example of a service that should never be run by an in-house SMB sysadmin. It needs to be outsourced. If you are going to run Exchange in-house then the sysadmins should have access to an MSP with a hell of a lot more experience, time and resources to do proper labbing of patches for that SMB's config and so forth. It is an application in a stack for which specialists should be used.

...or where it makes damned good sense to simply pack the whole thing up and go "cloud".

If Microsoft had "Office 365" for service providers and/or could make their own offering reliable enough that it isn't constantly experiencing outages, I'd say "use O365 service provider" and be done with it. MS refuses to release O365 to SPs and it can't keep it's own version working.

That leaves me with Gmail as the most stable offering for SMBs, followed by the more expensive hosted Exchange (assuming you can meet the floor cost), or simply hosted e-mail using open source MTAs without all the groupware faffery.

But the issue, 99.9% of the time isn't that "groupware will magically make things better." It is that there are bigger business and communication issues that need to be dealt with that no software can make better.

Anywho. Long ramble...

0
0
Trevor_Pott
Gold badge

Re: Too much info in too few X-Spam headers

Actually, I have to disagree with you here. The reason for moving towards an X-Spam-Status header is that it is an industry standard. If the system is set up to accept these then it can be used with AS devices or services from any number of providers. Not all providers allow you to change the headers you are working with, so X-Spam-Status makes the most sense to stay with.

Now, the ability to change the old server to pop it's stupid BAYES info into a different header, that would be great...

0
0
Trevor_Pott
Gold badge

Re: OPE vs FOPE . . .

Oh, I lied. The MPN support people only solved part of the problem within 32 hours of picking up the ticket. They fixed the part that was preventing me from generating quotes for new seats. They didn't fix the licensing issue with my MAPS. They *just* e-mailed me about that.

This makes it 48 hours to pick up the ticket and we're 72 hours past that point without the ticket fully resolved. And the ticket in this case not being some niggly complex technical problem, it's a billing/administrative issue that stemmed from a years-back uncaught authentication system screwup on their side.

I.E. the damned thing autogenerated me an Office 365 account without informing me, then assigned my MPN account to it. I was then able to create another Office 365 account that was somehow also attached to the same MPN account, but which couldn't get at the partner section, but which would accept my MAPS keys.

They fixed the bizzare double-attachment bit 32 hours after picking up the ticket, but solving the "regenerating me a new MAPS key" part of the ticket is two days past that and counting...

I think I'd have more confidence in Office 365 - which, from a technical standpoint is actulaly quite a good solution - if only authentication ever fucking worked. MPN never works the first time. Even straight-up .onmicrosoft.com Office 365 IDs never seem to work, requiring me to login two or even three times, sometimes requiring a log-out in between. There's something about session cookies they can't every get right.

Beyond that, I have all sorts of issues with Azure Active Directory. Sometimes it says it works, but isn't. Other times for reasons incomprehensible it just stops working, despite nothing having changed (and no reported outages on the MS side.) This makes hybrid setups very frustrating.

Microsoft is so close. Their hybrid solution will one day be the solution. But to be perfectly honest it's another 1-2 years away from being ready for primetime. Maybe when Server 9 comes out, they'll have added in the bits required to make it go reliably.

0
0
Trevor_Pott
Gold badge
Angel

Re: Typical Geek Whingeing...

On the off chance that you might be right, I ran a series of tests against my own Google Apps domain, egeek.ca. Here are the results.

Attempting to sent to an address that doesn't exist from a Telus-based e-mail account provided me this bounce message:

Reporting-MTA: dns; cmta4.telus.net [209.171.16.77]

Received-From-MTA: dns; Impella [108.181.21.61]

Arrival-Date: Wed, 04 Jun 2014 18:10:33 -0600

Final-recipient: rfc822; bob@egeek.ca

Action: failed

Status: 5.1.1

Diagnostic-Code: smtp; 550 5.1.1 http://support.google.com/mail/bin/answer.py?answer=6596 v7si6012708qad.84 - gsmtp

Last-attempt-Date: Wed, 04 Jun 2014 18:10:33 -0600

Similarly, attempting to send from a legitimate eGeek.ca account to an Astlor.ca (which runs on sendmail) account that doesn't exist let the NDR through to my eGeek account. It didn't get caught up in spam or trash; Gmail sent it straight through to my Inbox. Here is that e-mail:

Delivery to the following recipient failed permanently:

Bob@astlor.ca

Technical details of permanent failure:

Google tried to deliver your message, but it was rejected by the server for the recipient domain astlor.ca by astlor.ca. [64.141.126.154].

The error that the other server returned was:

550 5.1.1 <Bob@astlor.ca>... User unknown

----- Original message -----

DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;

d=egeek.ca; s=google;

h=from:to:subject:date:message-id:mime-version:content-type

:thread-index:content-language;

bh=stXngne3UrZepo/myHRVcSj4pEeKGAcgHsgoYbGKzkI=;

b=Y5T94txWG8KxY2DgzDuCHomK+vBIqnyKjTXdBpOMSzPCcF3Dcjh9LC3rAboEEMTlhc

0c0q/g5uzKBguhzfehD1IsFoRhZkAoSTW51I8xW3eUCinyhVENHBGxtwg+X3WWJf6Coc

ioDEGLMb0LUJz07bkAuqtpv6lN9ey698Hzvr0=

X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;

d=1e100.net; s=20130820;

h=x-gm-message-state:from:to:subject:date:message-id:mime-version

:content-type:thread-index:content-language;

bh=stXngne3UrZepo/myHRVcSj4pEeKGAcgHsgoYbGKzkI=;

b=iu6A0TLCPfGtwcUnD2FBh7LJOI3nAhbRZaumLMOZwKxkin9XjutfZvj66Js7ALupUA

+A52iq2TbIqaUv7N7kyN+0um6pa0jn0GWpsygwKn5ACVYvOf74D8vUqKHmsFkfmNoKMa

wJEn4URuLWrB1gLIUg1Q1gbTPzrQqGMuWKC6jyAkVTI+mO+pfYIRiUvOdp69K1sVmoDD

AnxAov02u6sABPVS2Y+vLD6V3Z+SgABUT+oy6vi9Y8kXc30nTvKJyBOK9GNmbij7esdV

4BohEl5QoevwwXFxqj5Xfzv4fLpXJsCV1G2T7TEfkAtYZ054EG28nnRBDJIQ88p/W048

m6hQ==

X-Gm-Message-State: ALoCoQnrR4fNM2MLTt+cTlUi3sJ7W/wrA1rtU6u5WkhKAzxc5vL1uO8QtLfap95CLWh1q5g5hTOQ

X-Received: by 10.50.13.4 with SMTP id d4mr13139985igc.11.1401927652048;

Wed, 04 Jun 2014 17:20:52 -0700 (PDT)

Return-Path: <trevor.p@egeek.ca>

Received: from Impella ([108.181.21.61])

by mx.google.com with ESMTPSA id q2sm400463ign.2.2014.06.04.17.20.51

for <Bob@astlor.ca>

(version=TLSv1 cipher=ECDHE-RSA-AES128-SHA bits=128/128);

Wed, 04 Jun 2014 17:20:51 -0700 (PDT)

From: Trevor Pott <trevor.p@egeek.ca>

X-Google-Original-From: "Trevor Pott" <Trevor.P@egeek.ca>

To: <Bob@astlor.ca>

Subject: Test

Date: Wed, 4 Jun 2014 18:20:44 -0600

Message-ID: <021701cf8053$fe803650$fb80a2f0$@egeek.ca>

MIME-Version: 1.0

Content-Type: multipart/alternative;

boundary="----=_NextPart_000_0218_01CF8021.B3E689A0"

X-Mailer: Microsoft Outlook 14.0

Thread-Index: Ac+AU/3dyjJXVzIqTei0po4Bz7aTVQ==

Content-Language: en-ca

Test

I also tried a series of additional tests (mailbox full and so forth) and found that Gmail allows all standard SMTP NDRs that I can think of to reach the Inbox and returns most of them.

Now, IIRC, this wasn't always the case; quite some time ago they had disabled NDRs for a while in order to cope with backscatter - quite frankly, backscatter is a huge problem for a lot of MTAs - but they seem to have gotten around the backscatter issue through a combination of blacklisting known bad senders (thus not sending them NDRs) and greylisting.

Interestingly enough, this is exactly what I am trying to achieve with the chained X-SPAM-STATUS filters: reduce backscatter. I need something that will do proper LDAP lookups against active directory and thus not accept mail for users that don't exist. That said, I also need something that wil both blacklist the known baddies (and not NDR them) as well as greylist new users so that known badguys can't just probe the directory.

E-mail isn't simple, and it's getting harder. It's a heck of a lot more complicated today than it was even two years ago, and it's nightmarishly fiendish compared to a decade ago.

Google does it well. Better, quite frankly, than anyone else I've seen. It seems we will remain starkly divergent in our opinions on this topic.

Also: just FYI, Peter had raised the issue to me before you did. I simply didn't check my e-mail until late Sunday afternoon because I was enjoying a wonderful blissful day of sleeping in, followed by spending time with my wife.

Cheers.

2
0
Trevor_Pott
Gold badge

Re: ^^3 day SLA?

I get 48 hours for responses to queries calling the MS partner support network. Then up to 32 hours for them to fix it. I get similar responses for average customers with E1 and E3 licenses. Multiple events now, same timeframes for each.

0
0
Trevor_Pott
Gold badge

Re: Its a regex !

It can use regex?

...must test this.

0
0
Trevor_Pott
Gold badge

Re: Your mistake was...

Thanks for that. In addition to everything I have to remember about the hundreds of applications I manage, I'll just run along now and memorize every PowerShell command. It's not like money was paid so that there would be a reasonably easy to use and modestly intuitive GUI. Nope, rote memorization of more data than a human mind can actually hold for every application is absolutely the best possible path forward for systems administration.

0
0
Trevor_Pott
Gold badge

Re: Precise language and vague language

There is no "exactly matches" condition. You would have to do "matches" then "except when". Still working on the details.

0
0
Trevor_Pott
Gold badge

Re: OPE vs FOPE . . .

The Office 365 advisor program and I are having a disagreement. Specifically, I've been fighting with MS for the past five days to even make my bloody partner page work. MPN and O365 both hate me. I hate them right back in turn.

Office 365 is something I'll revisit when they A) beef up the reporting to levels that aren't complete ass. [Insert 8-page reporting rant here]. and B) Make the fucking thing work. When Microsoft can achieve Google Apps levels of uptime, we'll talk again.

As for SPLA; fuck SPLA. I refuse to host Exchange in my cloud. The hosted e-mail I offer my clients is Qmail, Postfix or Zimbra, front-ended by Barracuda and/or Netgear UTM. OPE can be got by the customer for their own site...but it's more expensive than competing solutions and not as good.

0
0
Trevor_Pott
Gold badge

Re: What's with the Google fascination?

Oh, I just really, really hate exchange. E-mail in general, but exchange in particular. Loathe it with the burning passion of 10,000 suns. Most of my clients you Google Apps, Zimbra or a hosted exchange solution (that I don't have to manage, hee hee!)

If there's e-mail to manage I just want it to be a nice IMAP server. Postfix + Dovecot on virtualmin works like a hot damn. Or Qmail. For the love of $deity, why can't I just use Qmail? But no; exchange! Exchange, destroyer of souls. Exchange, the eraser of sanity. Exchange the requirer of resources 80x that of any other MTA.

And the cloudy alternatives? Well there are Linux-based IMAP mails...but I could run those in house, if allowed, with no real problems. There Google Apps with Just Works and works better than any hosted e-mail solution I've ever used. And then there's Office 365, which is the only solution I've used that makes me piss away more hours solving pointless problems (or waiting for Microsoft to do so) than Exchange itself.

Maybe I wouldn't have Office 365 so much if it weren't for the 48-hour lag on support calls, followed by 32 hours to resolve issues, but this is what it is. And when it's a "client down" scenario, 3+ days to get them back online isn't okay.

So yeah, Google Apps, when possible. Because it just works. If you read these pages, you know I'm not a big public cloud fan...but I trust Google to keep the e-mail working. Because they have a hell of a track record of doing so.

The solution, to my mind, is "have a critical service be bulletproof." I cannot offer that running on 10-year-old hardware using overly complicated MTAs with no funding for proper spam and antivirus scanning software. I am not convinced that Office 365 can offer it either. The only things I trust are Qmail, Zimbra and Postfix (which the client is allergic to) and Google Apps (which at least has something sort of like public folders, though you have to use a web UI to access them.)

Hence the desire to convince them that's the way to go.

When someone says "do this" and you aren't sure you can, the bigger mistake, I think, is spending your life just saying "yes". I've started to say "no", and this is a source of a lot of tension and conflict. "No, I can't do that" or "I don't think that will work." A decade ago I would fucking make it work...but a decade ago I only needed 2 hours a night of sleep...and I was only responsible for about 12 applications.

Now I am responsible for hundreds of applications, and I'm getting old. I need 8 hours of sleep or I am worthless the next morning. That young punk who could solve any technological problem using spit and bailing wire and sheer force of will is dead and buried. I used to know all there was to know within my sphere...but IT now encompasses a hell of a lot more than it did then. I could spend my entire day just trying to keep track of which companies exist in our industry, let alone what they do and how to implement their technologies.

So the scope of the project is beyond just software needs or desires for one vendor or feature. Who is going to look after this stuff? Especially once I'm no longer there to keep it ticking along? How will it all interact with everything else, and should it even interact with anything else?

The more I ask these questions, the more I want to pull core services off the local network. Some things need to be in house. But e-mail doesn't. There's already too much there for one person to handle; I'd prefer to pull everything that doesn't need be on-prem off, just so that it's feasible that one person with next-to-no budget can keep that place going for another decade.

Even if that means feeding the advertising behemoth of Mountain View.

1
2
Trevor_Pott
Gold badge

Re: DailyWTF?

You know, I find this whole "never go live on a Friday" thing idiotic. I went live for a brief period of testing on a Friday. Someone found the error I missed on a Sunday. It was fixed before Monday. Staff came to work with a weekend of low-volume traffic where they had to check through the junk-email folder for (on average) about 15 e-mails to see if they were false positives. Not the end of the fucking worked.

If I had run that thing at 8am Monday morning, it would have taken about 4 hours for someone to notice that something was up. In that time an average of about 100 e-mails would have hit each person's box that they needed to check through.

And I'd rather work a weekend than have 50 people screeching at me demanding to know when the fix will be in, "How could I possibly have let this happen" and telling me how shit I am because I can't design a network that's more reliable than Google while being more accurate than Microsoft and more capable than Amazon, all for free.

Buncha great choices there.

2
0
Trevor_Pott
Gold badge

Re: DailyWTF?

1) My spam servers worked just fine for years.

2) Putting things live during the day risks outages during working hours which has been emphatically affirmed to be an absolute no-no. There isn't much choice.

3) Exim? Really? I'm a bit of a QMail fan myself, though I have to admit that Postfix has come a long way. Honestly though, I've been working more and more with Zimbra and liking it.

I loathe exchange with the burning passion of 10,000 suns.

1
0
Trevor_Pott
Gold badge

Re: Hosted AS?

@Vic; I'm on the long path to getting rid of a decade's worth of bandaids and nudging the client along towards a proper (though significantly more expensive/year) IT setup ahead of leaving. It's a long fight.

1
0
Trevor_Pott
Gold badge

@Sampler

Oh, I tried that argument. I believe the response was "so we can take the cost of the hosted AS out of your salary?"

0
0
Trevor_Pott
Gold badge

Re: Quote to long to put int title

I'm not in the UK/EU. I'm in Canada. And for this class of customer you cheerily can put them in Google Apps without consequence.

1
1
Trevor_Pott
Gold badge

You get an exchange licence and 10 free CALs with your Action Pack. That's about $400/year. When you already have to have a virtual infrastructure to deal with all the other stuff you do....yeah, it's cheaper.

Internal IT doesn't exist just to support one app.

0
0
Trevor_Pott
Gold badge

Re: actually..

I did. I fed it simulated data for days. Of course, the one thing I hadn't thought of was that the X-SPAM-HEADER info would be a problem, so the simulated data all had X-SPAM-HEADER data of either "yes" or "no".

0
0
Trevor_Pott
Gold badge

@peteur

I try to discourage people working late, or on weekends. I have few enough maintenance windows as it is. If you work during off hours, well, I have no sympathy. There isn't a 24/7 global team of nerds to implement changes and patch things. So we have to sleep some time. If I have to be up for the 9-5 grind, then I'm not waiting until 3am to patch.

Besides, some folks start getting in a 4am...

3
3
Trevor_Pott
Gold badge

Re: Friday ...

For a full "this is live and will stay that way", I agree. For a pre-permenant, data-gathering exercise that needs to run on live...this I prefer on the Friday EOB. Remeber, the goal here was not a permenant run, just a very brief test on live with just enough traffic to find bugs.

Found one.

1
3
Trevor_Pott
Gold badge

Re: Speaking of language........

>_>

<_<

:(

//sads

2
0
Trevor_Pott
Gold badge

Re: Hosted AS?

Hosted AS is ultimately where I want to go. The history is as follows:

1) Until recently, hosted AS was along the lines of "a few dollars per user per month" not "a few dollars per user per year." Which is more than the client would pay.

2) Until recently, relatively simple in-house open source AS systems worked just fine.

3) Having used the simple open source AS systems for so long transitioning away from them takes time. The existing system, for example, injects [SPAM ASSASSIN DETECTED SPAM] into the subject, rather than adding X-SPAM-STATUS

My goal is to get them using an in-house AS system that uses X-SPAM-STATUS for the rest of the year and then have them transition to a hosted AS system at the end of the year. This will be possible because both the system I'm trying to deploy for the in-house option and virtually all hosted AS systems use X-SPAM-STATUS.

Now, getting them to accept hosted AS will require getting them accept paying a subscription for an AS service when they're used to using free in-house stuff AND getting them to overcome their innate paranoia regarding having their e-mail hit servers in the states. I honestly don't know if I can "sell" that...and I'm pretty sure I don't care enough to try.

What I can do is get them migrated to a solution that uses X-SPAM-STATUS instead of subject injection which will make the transition to a proper hosted AS a heckofalot easier in that mythical future when the decide to just pay the tithe like everyone else.

That's the goal, anyways...

4
0
Trevor_Pott
Gold badge

Re: Friday ...

Do it on a weekday and they'll have your hide if anything goes wrong. Do it on a weekend and there's not enough traffic to make it go 'ping'. Do it on a Friday, right after EOB and you have a few good hours of decent incoming traffic flow, a handful of folks who work late and are used to minor changes and an entire weekend to fix things if you bork them really badly.

5
4
Trevor_Pott
Gold badge

Re: actually..

I watched it for about two hours. Nothing bizarre jumped out at me. I figured if something was going to go splonk, it would do so in a two hour timeframe. Guess I was wrong.

4
2

Got VDI questions? Fire them at our expert panel

Trevor_Pott
Gold badge

Re: Virtual VDI deployments ..

I can see about getting that included; is there anything specific you'd like to see in the demo?

0
0
Trevor_Pott
Gold badge

Re: "Our VDI panelists are diverse."

Agree 100%. I have a second webinar in the planning to address that. I am looking to do these in stages; start from the bottom (infrastructure) and work towards the top. The LoginVSI guys (and Eric, for that matter) know a fair amount about the user virtualisation issues, enough to hold their own, but user virtualisation is, of itself, a separate field entirely that transcends "just VDI".

User issues are present in non-persistent VDI, but also in "hoteling" desktop setups, multi-device/multi-OS setups and more. That's why that discussion will happen separately.

0
0

Russians turn Raspberry Pi into fully-fledged autopilot

Trevor_Pott
Gold badge

Re: Weird

I believe that Moar Power will come, in the Next Generation...

0
0

Amazing never-seen-before photo of colourful hot young stars (Thanks Hubble)

Trevor_Pott
Gold badge

Re: For all the believers out there

Can you god make an argument so circular even he can't believe it?

1
1

China puts Windows 8 on TV, screams: 'SECURITY, GET IT OUT OF HERE!'

Trevor_Pott
Gold badge

Re: You ignore China to your peril

No source, no sale, now ship off, Shirley.

3
0

VMware hits back at Amazon cloud Trojan Horse with ... a blog post

Trevor_Pott
Gold badge

If they were Swiss? Yes.

1
0
Trevor_Pott
Gold badge

Re: roach motel

"he system PSOD'd and the HP firmware auto rebooted the box. Still waiting on support to analyze the logs first indication is it is a General protection fault (13)."

$20 it's a bad SPD chip on a single DIMM. Virtually impossible to isolate. The error manifests as a PSOD/BSOD only under certain very specific conditions. The issue is that the BIOS is configured to clock each DIMM (or at least each bank) independently. The bad SPD chip reports an incorrect speed for the capabilities of the DIMM. The result: an overclocked DIMM that goes squirrely seemingly at random, but especially when the temperature goes up.

The error will often show up as a set of ECC errors within your system, but when you go to memtest the DIMMs individually they're all fine. Alternately, you could have a system wherein timings are set per bank, not per DIMM, and the bad SPD chip is in fact on a DIMM that absolutely can handle the higher speed, but one of the other DIMMs in the bank can't, and that (perfectly fine, if tested on it's own) DIMM is the one that errors out.

Solution: attempt to isolate DIMM (hard) and RMA - or - manually set the speed and timings of all DIMMs in the system. (I typically downclock to just below rated anyways, just to avoid minor manufacturing defect issues.)

You can also use that Intel technology that allows you to RAID 1 your RAM. For file servers, this is what I do: RAID 1 the RAM and downclock it. Then the things run like a tank.

1
0
Trevor_Pott
Gold badge

Jack, buddy, I have huge respect for you...but I feel I must disagree with your assessment of the blog post being aught but marketing malarkey. The fellow behind that blog post raises some damned good points, especially around bringing workloads back into your on-prem datacenter once you're done.

Cloud bursting with VMware/vCHS isn't great, but it's a heck of a lot better than this Amazon connector allows. Beyond that, to be blunt, VMware has some great next-gen technologies in QA around vCHS that will make cloudbursting easier. I'm sure you've seen the same NDAed slides we all have at this point; it's all an open secret by now.

All of which leads me to: Amazon's move is the desperate one. Bullshitting saved for another day, Microsoft has the best hybrid cloud. This is followed by both companies that have deployed Openstack internally; there are hundreds of Openstack public providers and Openstack to Openstack actually works quite well.

VMware is next up, their technology is immature, but they are dumping amazing resources into it. The people working on the hybrid cloud offering at VMware are some of the brightest on the planet and I promise you they will be at an Azure level by the end of the year. They'll probably pull away from MS and have the best damned hybrid offering (at the highest price!) of all contenders by VMworld 2015.

All of which leaves Amazon, where? As the poster child for voluntarily handing your data to the US government? The embodiment of the inability to even attempt data sovereignty or control over your own workloads in a superficial way? Amazon is great for SaaS developers who make pointless tat or who work in industries where America basically sets global law anyways. (See: Netflix.) It's rather less awesome for the man - many - high-value industries that are either regulated, or where innovation occurs at a such a pace that economic/industrial espionage* is something that companies worry about.

The public cloud isn't safe for some workloads. On-premises isn't cost effective or fast enough for other workloads. That makes hybrids cloud an absolute necessity and it is Amazon - not VMware - that doesn't have a story here.

There are some very valid concerns about picking up your workloads and putting them on a public cloud, regardless of which cloud you choose. But when the workloads can't come back easily, or your VMs are converted, or you are integrating with management tools/using software with weird licensing restrictions then things get a hell of a lot more messy than "this is technically possible."

We could always take our workloads and put them into Amazon's cloud. The thing that was holding us back was never an integration tool. It was all the myriad reasons listed in that blog post, and more besides.

When Amazon develops the ability to truly move workloads from on-prem to the cloud and back again, with conversion headaches, networking issues and management/agent integration tools dealt with on the fly, then VMware should start sweating. Until then, I'm pretty sure that VMware's best path forward is to make a dmaned good hybrid solution of their own...and lower the prices for service providers dramatically.

If they don't, Microsoft is going to win. Microsoft has a hybrid cloud that is not just on-prem and public cloud, it's "service provider"...and that's critical. Data sovereignty means a lot of people want cloudbursting...but only within their own legal jurisdiction. Microsoft has an answer to this. VMware doesn't**.

VMware has about a year, maybe a year and a half to get that sorted before even large enterprises are willing to use the abomination that is SCVMM*** in exchange for a proper multi-teir cloud.

*Ask yourself: if you had the cure for cancer, the formula for room temperature superconducters or the plans for a machine that could cancel gravity in a localized field would you store that information with an American cloud provider? If you would, please e-mail me and we can discuss a fantastic opportunity I have regarding some riverside real estate that provides access for individuals wishing to cross.

**Because most cloud providers won't pay VMware's exorbitant fees and are still miffed that VMware is competing against them.

***Fuck SCVMM.

5
0

My my, Intel, that's one speedy NVMe flash card you have there

Trevor_Pott
Gold badge

They fail to "read-only" mode. When the write limit's up, then you can still read your data off.

0
0
Trevor_Pott
Gold badge

Re: Shut up take my money!

The 1.2TB Micron p420m is pretty much the most amazing thing that's ever entered my lab. There are not enough nice things to say about proper enterprise flash.

0
0

What's that you say? HP's going to do WHAT to 3PAR StoreServs?

Trevor_Pott
Gold badge

Oh look, an anonymous coward with a product/vendor fetish. How boring.

0
1

DARPA crazytech crew want to create HUMAN-FREE cyber defence systems

Trevor_Pott
Gold badge

If I had software that could automatically detect vulnerabilities in software and suggest methods of fixing or defending against it I would also have software that could automatically detect vulnerabilities in software and suggest methods of attack. The last people on earth I'd give that to are the Americans.

The ability to defend better than modern infosec experts is also the ability to completely and utterly overwhelm modern infosec experts. Under no circumstances should the American government have that kind of power.

2
0

Brunner does a runner: Beats designer must hand the brand to Apple

Trevor_Pott
Gold badge

Re: "Cool things are the things that don't even know they are cool."

I think the precise set of clothing and accoutrements changes with time, but the general rule is "desperately attempting not to look mainstream by dressing exactly like everyone else who is desperately attempting not to look mainstream." At least, if you are attempting to judge based solely on appearance. Which is somewhat valid in this case...

0
0
Trevor_Pott
Gold badge

Re: "Cool things are the things that don't even know they are cool."

Run-of-the-mill twats I can cope with. Hipsters need to be liquified into series of short-chain polymers.

0
0

Linux users at risk as ANOTHER critical GnuTLS bug found

Trevor_Pott
Gold badge

Re: Open source was supposed to be secure

That would solve rather a lot of problems in short order...

2
0

Samsung, with this new 3D NAND SSD, you're really spoiling us ... or perhaps a rival?

Trevor_Pott
Gold badge

Re: As a 1TB flash drive ....

"satisfied your need demonstrate how much better you are"

You really don't get it, do you? My faith in our species remains shattered.

0
1
Trevor_Pott
Gold badge

Re: As a 1TB flash drive ....

"The fact that a euphemism has been in place for ages doesn't mean I can't criticise its use does it?"

Rewritten

"Ain't ain't a word because it ain't in the dictionary."

Yes it fucking is. PULL! *Shotgun blast*

1
0
Trevor_Pott
Gold badge

Re: As a 1TB flash drive ....

Egads you people. You do realize that I can rip off a 500 word "rant" like the above in less than two minutes, no? It isn't exactly a burden requiring much though or a huge emotional input. Use some big words and a little bit of florid prose and it's a "meltdown". If I wave my hands and make the bunny disappear is it magic, too?

And you wonder why linguistic repression makes me grumpy.

1
0
Trevor_Pott
Gold badge

Re: As a 1TB flash drive ....

Because you're bitching about a common euphemism on a site infamous for it's euphemisms. I hold your gripe in even less regard than the dandies who trip in here bitching and moaning about use of the word "boffin".

At least I can excuse the odd nublet for not grokking the local lingo, but for $deity's sake man, "spinning rust" is a decade+ old, and used throughout the industry. It's not a Registerism. It's not a Trevorism. Yet you come in here and accuse me of "trying to look smart" through it's use.

The fuck, what?

That pisses me off. Why? Because I have to control my vocabulary in order to write here, or just about anywhere else. I hate doing it, but normals just don't have a vocabulary of more than about 20,000 words. 20,000 words! That's like chiseling the future on a fucking stone tablet using a quantum singularity as a source of granularity. It's restrictive and irksome.

The last time I ran the tests it was estimated that I probably had a vocabulary of around 50K words. That didn't include a lifetime's worth of popular culture references or a dedicated study of memetics. I post that Darmok and Jalad at Tenagra demonstrates that memological phrases should be counted as "words" in a vocabulary as they convey an atomic sense of meaning.

So we go back to "spinning rust." I absolutely don't use it to "sound smart". Quite the opposite, I use it because it conveys a bit of industry humour, along with a sense of contempt as well as a relegation of the technology to the past. It does the job in a very Darmok and Jalad fashion that should be accessible to every practicing member of our industry. It is an expression whose use is so broad that it shouldn't give me the kinds of issues I have when attempting to freely communicate using a wider knowledge base.

Do you have any idea how frustrating it is to only have two, maybe three people in the entire world that you know you can talk to without restraint and be perfectly confident that they'll understand you? To have to constantly ask yourself "will they even understand that word, or that reference?"

I bloody well expect you to get "spinning rust". I also expect that you'll understand the industry culture behind it. I expect it because you're here. You read this magazine. Because you're supposed to be that little bit better than normals, damn it, and I should be able to loosen the vocabulary restrictions by just that tiniest of fractional quanta.

I have to work all day with my brain not allowed to actually run free. So you're damned right if I take offense to someone telling my I'm trying to "sound smart" by using what amounts to a pedestrian industry-specific colloquialism! Am I "dressing up" by wearing a pair of clean blue jeans, by chance? Or having a fancy dinner because I marinated the steak for a few hours before cooking it?

I want to wear a monacle and a $5K custom suit while riding a horse on a boat, damn it! To tell me I'm putting on airs because I wore a polo shirt to the hockey game just makes me want to stuff you into a coil gun and fire you into Sol!

So yes. Ferro-magnetic princess. Wear it. It's yours now.

7
2
Trevor_Pott
Gold badge

Re: As a 1TB flash drive .... @Pott

You don't understand, the issue is not "that there were bugs." I don't care about bugs. I care about how you handle bugs. I.E. that you man up, admit them publicly and deploy the fix openly.

Most SSD - and HDD - vendors just list the firmware on the site. You can download it, and rotate the disks through your internal lab, applying the new firmware and then back into service. It's abotu as much work as filling out RMA paperwork, but you get the drive back into service ASAP.

WD, otoh, never admitted the bug. They just denied and denied and denied. They demanded RMAs for any drive "affected by the issue that doesn't exist", and legally pursued people who had posted the firmware online so that those of us with affected disks didn't have to dick around with the horrific RMA process to get a fix.

That's not quite OCZ levels of fail, but it is really, really bad.

Now, it doesn't put me off WD altogether, but it absolutely does make me steer clear of the raptor division, and of their enterprise disks. I'd rather work with HGST or Seagate, simply because I don't get this kind of dicking around from their enterprise teams.

Every bit of software has bugs; how you handle them is the bit that matters.

3
0
Trevor_Pott
Gold badge

Re: As a 1TB flash drive ....

Aye, that's where it started, but it was ported to hard disks about 10 years ago and has been used there ever since. Originally, it was used lovingly (a-la "rust never sleeps",) but as newer alternatives come around, "rust" took on a far more negative connotation in the industry.

Now it's 2014 and I want rust in my racks about as much as I want rust on my car.

0
1
Trevor_Pott
Gold badge

Re: As a 1TB flash drive ....

Seriously, mate...how long have you been reading The Register? This is not only a common euphemism for magnetic platter storage throughout the industry, it fits perfectly with the tongue-in-cheek nature of the publication.

To put it more bluntly: it's called "spinning rust" because it both evokes thoughts a spinning metal disk (which hard drives are) while also evoking thoughts of "old and horribly outdated"...which hard drives also are. If you have a personal attraction to spinning rust drives, I feel sorry for you.

But shite they are, and shite they'll remain...and "spinning rust" they shall be called, throughout the entirety of the IT domain.

Suck it up, ferro-magnetic princess...

4
2

CSIRO claims milestone in solar-powered steam turbines

Trevor_Pott
Gold badge

So these people will all be taken out back and shot by the conservative Oz gov't then?

2
0

How Bitcoin could become a super-sized Wayback Machine

Trevor_Pott
Gold badge

Good idea. I like it.

1
0

'GODZILLA WORLD' of the DRAGON CONSTELLATION - scientists

Trevor_Pott
Gold badge

Don't buy the solar storms = doom thing one bit. Earth has organisms that withstand high levels of ionizing radiation...and organisms on a tidally locked planetary body would be living on the terminus anyways; no direct "brunt force stellar trauma."

Natural selection can do amazing things, we'd be fools to write off red dwarf stars as potential sources for habitable life. Especially when you consider that exomoons won't have many of these problems. Heck, you could even get a little farther out from the star if the planetary parent were big enough to be emitting substantial infrared.

1
0

Supreme Court nixes idea of 'indirect' patent infringement

Trevor_Pott
Gold badge

Good.

20
1