back to article A sysadmin in telco hell

At the best of times, telecommunications is a complicated field to navigate. Putting aside the technical difficulties of creating and maintaining a modern telecommunications system, customer support and regulatory compliance can be challenging burdens for organisations of any size to cope with. Even with the best and brightest …

COMMENTS

This topic is closed for new posts.
  1. Christian Berger

    Cooperation between companies never works

    Unless of course it's against the customer.

    That's why one should always try to rely on as short of company chains as possible. If one of them messes up, you're screwed.

    This is something you should also keep in mind when relying on cloud services.

  2. This post has been deleted by its author

    1. Oor Nonny-Muss

      Re: Phew!

      Its exactly the same palaver in the UK (and as far as I have experienced - wider Europe)...

      1. JohnG

        Re: Phew!

        Yeah - the European dimension adds language/translation errors and different national legislation to the fun.

    2. Martijn Bakker
      Happy

      Re: Phew!

      These problem are not particular to the US of freephone numbers. My experiences when porting Dutch mobile and landline numbers were much the same.

      In my experience, some telcos are more likely than not to mess up a numbers transfer.

      Maybe it's a very complicated subject.. or maybe it's the telco's way of getting back at you. Anyway.. the advice is sound.

    3. Anonymous Coward
      Anonymous Coward

      Re: Phew!

      Well for some of us, this is our daily jobs.

      I'm just going through this hell of sub-ownership and multiple carrier migration.

      Gah..

    4. Anonymous Coward
      Anonymous Coward

      Re: Phew! (ahem...)

      Actually, Canada is not part of the U.S.

      1. Tom Maddox Silver badge
        Joke

        Re: Phew! (ahem...)

        WHAT?! Since when?

        1. Michael Wojcik Silver badge

          Re: Phew! (ahem...)

          I'm amused that three people have bothered to downvote an apparently innocuous three-word joke. Well, that'll show 'im.

  3. Alex C

    The failings of ported numbers

    The issue here highlights the downsides of porting numbers. Wherever it is possible it is worth phasing out old numbers (which some businesses will find easier than others).

    There are two main reasons for this - telcos are required (certainly in the UK) to come to agreement with other telcos over creating a porting agreement within 90 days, but that regulation does not go into any financial specifics. Generally the company who's number it was originally will require that they continue to make in the region of a penny / minute from it. Move the number around over a few companies and the charges can stack up. There's only one person left holding the bill and that's the customer.

    The other issue is the one you particularly ran into here - that once they've hardwired the number to it's new destination (or think they have) you are no longer their customer and they care not one whit about you. This is far more true of larger outfits where the person you initially talk to is often a peon going through a script and if you have no valid account number (etc) their systems are often designed to keep you out. (usually more to prevent phishing / fraud than for this case but the effect is the same).

    Where you're able to it's worth moving numbers - most people are smart enough to look for a contacts page on a website if they number they're trying seems broken, and even the most catalogue focused companies can usually migrate within a few months, using number change announcements and a bit of brain on the website, and handled sensibly it is a pretty soft process. I'm surprised it took Planet as long as it seems to have done to offer you their own free phone numbers - they'd have been itching to do so from the outset - as their margins will be better on the non-ported numbers.

    1. Orv Silver badge
      Alert

      Yes, but...

      If you *do* switch numbers, there's a good chance the old one will end up pointing to a phone sex line. There's a company in the US that specializes in the 800 number equivalent of domain squatting -- they acquire available 800 numbers and forward them to phone sex services. They now own about 25% of the 800 number space:

      http://www.msnbc.msn.com/id/42658653/ns/business-small_business/t/phone-sex-company-amassing---numbers/#.T1UWlXJWqw4

  4. Anonymous Coward
    Anonymous Coward

    For those that have this sort of problem in the UK...

    ..this may help:

    www.ofcom.org.uk/static/numbering/index.htm

    It lists who owns what range. This will probably not include already ported numbers though.

  5. G2
    FAIL

    i'm in the same porting hell with Vodafone ever since 2009 !!!!

    i'm stuck in a similar porting hell with Vodafone (in Romania) since 2009 and they won't do ANYTHING about it.

    in october 2009 i ported my number from Vodafone to another carrier (RDS DigiMobil) and to this day i CANNOT RECEIVE any international call or sms.

    i tested with a friend in Canada: while we were both on skype video chat i had him call my mobile phone number from his mobile phone and after he started to get a ringing tone (and my phone wasn't ringing at all) i used the very same phone that he was trying to reach to call his home land line number.

    His landline started to ring immediately and at the same time his mobile phone was still trying to ring my mobile number (and did a very good job of faking the ringing, there was no indication that the call was not connected properly at all, he was hearing a standard ringing tone)

    i filed complaints all the way up to the National Authority for Communications but all my complaints end up in the same stink-hole that is vodafone+rds support.

    RDS says that the porting has been completed from their end... Vodafone says the same and i'm stuck with having to use a secondary prepaid sim card (from vodafone since they won't unlock that handset even if the contract is finished) so that i can receive international calls and sms.

    1. Matt Bryant Silver badge
      Boffin

      Re: i'm in the same porting hell with Vodafone ever since 2009 !!!!

      ".....and did a very good job of faking the ringing....." That's not your friend's phones fault, it wasn't pretending anything, it was the inter-telco connection. The phone makes the ringing sound the minute the call it is making is accepted, which is actually the first telco system that picks it up as "owning" the number, even if it has subsequently been transferred. The problem is the chain that it can then get passed through to get to the final exchange connection, and this can be so many hops you actually give up before the eventual connection gets made, though in your case it sounds like the eventual connection was never going to get made.

      We had a similar problem with a subsidiary we bought. They had changed their carrier several times, each time failling to notice the longer time it took for incoming calls to get connected. Callers would complain that they had to let the phone ring seven rings before it was picked up, whereas the company staff swore blind they were picking it up on the first ring. Amusingly, it did cut down on unwanted salescalls, as the average telesales is usually trained to let the phone ring four times and then cancel the call! Our eventual fix was to change all the numbers to new ones with our carrier.

    2. Aaron Em

      Re: i'm in the same porting hell with Vodafone ever since 2009 !!!!

      Too bad that's got nothing to do with the subject of the article, though -- that, and for all the sound and fury you've no doubt expended on the issue "since 2009 !!!!", surely it would've been simpler just to tell everyone your new phone number...

  6. Anonymous Coward
    Anonymous Coward

    I've said it before and I'll say it again,....

    I have never met a network techician that is both competent and helpful.

    1. Anonymous Coward
      Anonymous Coward

      Re: I've said it before and I'll say it again,....

      You did indeed; I remember you saying that at the time and, to be honest, I wasn't convinced. How wrong was I?! I'm sure you had some other nuggets of wisdom; any chance you could say them again, in case anyone else missed them. It would be terribly terribly helpful.

    2. Anonymous Coward
      Anonymous Coward

      Re: I've said it before and I'll say it again,....

      Actually, I have met several who are both competent and helpful, but never at an ILEC.

      A PBX installation was going wrong one evening, with the telco provided T1 configured as E&M Wink start, but we could not get the thing to make outbound calls properly. After many hours of fiddling with configuration parameters, the telco agreed to convert the Wink Start T1 into a PRI on the spot. In 20 minutes things were up and running. I've asked other ILECs if they can do that, 30 business days is their usual answer.

      This was not a big telco, rather a CLEC in the NY area, that I would certainly recommend and do business again with in the future. The CLECs are hungry for business and will go out of their way to satisfy the customer.

  7. jake Silver badge

    ::shrugs::

    If the cognizant sysadmin doesn't grok the tariff for any given link, said sysadmin will have no end of trouble.

    This is the sysadmin's fault, not $TELCO's ...

    1. Matt Bryant Silver badge
      Happy

      Re: ::shrugs::

      Has to be said: "Caller, can you please confirm you're holding the handset correctly?"

    2. Hungry Sean
      Thumb Down

      Re: ::shrugs::

      Unfortunately, sysadmins are only human in the scope of their knowledge. Having previously worked at a very small telco, I can only sympathize with the hassles and potential for error involved. I'm going to go out on a limb and guess that porting 800 numbers isn't something Trevor (or most sysadmins) does commonly, and as with many things that sound simple in practice, there are a lot of potential pitfalls. How on earth do you expect him to know them all intimately?

      I don't think it's unreasonable to expect the vendors, who should in fact be domain experts, to hide the complexity from their customers and get things right, or failing that, put things right. Some might even say that this is what distinguishes a good vendor from a bad one.

      1. jake Silver badge

        My point is that ...

        ... if you don't grok the guts of wide area networking, you shouldn't be (attempting to) make a living at it. For a start, how many of all y'all understand how to provide seamless service via multiple peering points?

        Not understanding the contract for any given link is YOUR fault, not the telco's fault.

        If you know how, and the outfit you are working for doesn't allow you to provision the line (including redundancy), that's a second level of problem.

        Kids these days ...

  8. David 14

    Always have a non-800 number available!

    A lesson from my own past... again, this one is in the Canadian telecom marketplace...

    I do not know the exact technical configuration of this today, as my story did occur back in the late 90s, during the run-up to the Y2K milestone. During this time, not only was there a lot of coding, system upgrades, and new deployments all ongoing at breakneck speed, but the need for "Business Continuity" plans was top of mind to most businesses.

    Sometime in the 1998 or 1999 timeframe, there was an outage of the 800-number lookup service in this part of Canada (East Coast). It turns out that at the time (and maybe even today) each 800 number dialed in this large geographic region (and covering about 22million people) all used a single lookup system running in a Montreal-based datacentre.

    For about 24 hours, no 800 lines worked. And it became painfully apparent during that time that almost every contact number we had for IT support vendors were of the toll-free type.

    While maybe the system is now more robust, I still, to this day, always demand a direct-dial number alternative for any critical contact information alongside any 800 line service.

    Cheers!

  9. Dibbles

    It may be the same in the UK but...

    ...it does seem that customer service in Canada falls into the passive-aggressive cliche of old. Yes of course I'd like to help, but no, I actually can't help, and no, I don't see that as being a big problem...

  10. b166er

    Let's do away with the telephone numbering system and have a DNS for phones!

    1. Trevor_Pott Gold badge

      Ummm...NO

    2. Joe User
      Facepalm

      Re: DNS for phones

      b166er: "Let's do away with the telephone numbering system and have a DNS for phones!"

      You do realize that numbers are still required in a DNS, don't you?

      1. P. Lee

        Re: DNS for phones

        I assume he means to use something like MX records with priorities and multiple "real" numbers hidden from the user interface.

        However, its rare to find a company that can be bothered to do DNS properly, which is odd, seeing how fundamental it is.

        1. This post has been deleted by its author

        2. Trevor_Pott Gold badge

          Re: DNS for phones

          Also: see ENUM, something for which there is crazy stupid powerful resistance amongst NA Telcos...

    3. garetht t

      Enum

      http://en.wikipedia.org/wiki/Enum

  11. Anonymous Coward
    Anonymous Coward

    I have worked for a well known carrier in the UK. If you think voice provisioning can get screwed up, wait until you get data provisioning problems.

    They lost a contract a few years back that was so massive they had to swallow the telco that won it instead of them (think massive profits, as it gives them a monopoly). I am certain that all of this was actually because they had (yet again) managed to massively piss off the upper levels of probably their largest UK customer.

    They did so by totally screwing up the provisioning of a tiny, simple single leased line for a critically important project. The circuit was there, the money was there, yet after MONTHS (yes, really) they still hadn't managed and why? Because they ran out of routers. Never mind that they had warehouses full with spares - they were allocated as spares and none shall touch them, even if it screwed over the one client they really could not afford to lose.

    So the client took notice, and borked their bid off the table. Deservedly so, in my opinion.

    1. Anonymous Coward
      Anonymous Coward

      Sounds familiar

      Did the big customer by any chance have a 4-letter abbreviation?

      1. Anonymous Coward
        Anonymous Coward

        Re: Sounds familiar

        No, this big customer uniquely exists in every country of the world. And that customer *never* runs out of money to waste..

  12. Andrew Dunning
    Unhappy

    Porting/Bell Canada

    The porting process needs major improvements. I had a similar issue when an ex-employee left and was able to port their Cell number to another courier without the companies consent.

    Bell Canada is a completely different issue. We; as a company; had our phone lines down for 3.5 business days. Bell didn't believe me the issue was theirs and 2nd of all didn't send a technician out for 3 days. I had to tell Bell how to test their own equipment. The only good thing was the tech that come out was someone how was with the company for 25+ years and knew his crap.

    We left Bell after this. Bell doesn't care at all about the small businesses.

  13. Anonymous Coward
    Anonymous Coward

    Porting Numbers is the same in Canada as it is in the US. When you port a number it goes like this. you are porting a number from company a to b. Company B sends a request A. company sends an ack of the request and creates a new route. Company B gets the ACK and puts the number into there switch and creates route in there system. Company A gives the new route to company B and from then on forwards the calls to company B. Company A still owns the number. In theory if the person cancels service with company B the number is suppose to go back to company A. Now more fun with porting fun. If the number does not go back to to company A and company B recycles it creates a 3 way port and those are fun. Ok you want to port the number from company B to C. Company C sends the request to company B. Company B says we don't own it so company B releases the number back to company A. Now Company A calls company C and the port goes as normal. Company C can not get involved with company A till company B tells A that we don't want this number. Now if company B does not want to do there job or releases the number before informing company A it creates havoc for company C and there customers

  14. Maty

    aaah ... Bell Canada

    Bell Canada has a certain reputation among Canadian consumers.

    Articles like this explain how the company got that reputation.

  15. Anonymous Coward
    Headmaster

    I pinged Planet Telecom...

    And how many ms did the the reply take?

    A genuine techie misuses "ping" --- all is lost, the end of the world is nigh ...and all that!

  16. b0llchit Silver badge
    Pint

    OSI layers 8..10

    My education has learned me that OSI layers 8, 9 and 10 are Money, Friends and Politics.

    1. Trevor_Pott Gold badge

      Re: OSI layers 8..10

      Bruce Schneier, RSA says

      Layer 8: The individual person.

      Layer 9: The organisation.

      Layer 10: Government or legal compliance

    2. jake Silver badge

      Anyone discussing OSI seriously ...

      Is immediately fired.

  17. Chris 3

    Sounds like Planet Telecom are the people you should be cross with...

    You signed a contract with them, they sub-contracted to someone who screwed up, and then apparently failed to even entertain the question of whether their subcontractor had screwed up.

    If they had simply checked with ThinkTel, this would have been a much shorter story.

    1. Trevor_Pott Gold badge

      Re: Sounds like Planet Telecom are the people you should be cross with...

      Planet Telecom did check with ThinkTel. Numerous times. By phone and email. Now, if you ask me how cross I am with ThinkTel, that would be a whole other article. One I can't write right now, because I haven't been able to adequately emotionally distance myself from the topic.

      I've got nothing but all sorts of awesome happy for the Planet Telecom guys. They have done us a solid more than once throughout this particular event. Good people.

    2. ElReg!comments!Pierre

      Re: Sounds like Planet Telecom are the people you should be cross with...

      In Canada, in the case of an unexplained telecom mishap assuming that Bell has the answer but won't tell you is always the safest bet. Not always true, but still statistically the safest.

      That's true for home users as well as corporate ones. My BFF has been waiting for 3 month without her home internet connection because, in order:

      1) Tech support told her that everything is alright from their side, datalink up and ADSL modem responding as it should, so please reboot everything and call us back.

      2) (several days later, due to Bell support being mostly unreachable) tech support admits that no, the modem doesn respond someone must have made a mistake, and it's probably down to the modem, please make an appointment with the service department.

      3) (several days later, due to the whole service department in town consisting of a "recycled" pizza delivery boy on his moped, working 9-5. Or so I must suppose given the delays and unpredictability of the visit): Yes, that´s the modem alright. The lights don't go blinky-blinky as they should, and let me refer that back home (appart from the fact that they rather obviously sent a sales rep to reboot her computer for her, one wonders how the no-blinky-blinky fact has been overlooked after more than 1 week of back-and-forth communication)

      4) several days later, after arguing back and forth because all traces of her previous communications had seemingly disappeared, she is told that it's probably down to the power brick, because this model tends to go "poof" every so often which is a known issue, and don't worry my dear, just go to any of our salespoints in your town (of which there are many) with your faulty unit, and they will replace it.

      5) several days laters, after several visits to several salespoints in town, it appears that Bell mothership had ceased to provide replacement units for the faulty model, _several month_ prior the whole kerfuffle, but that wasn't made clear to either the service dept, the customer support dept, or even the retailers (who kept sending my friends to other outlets in case they had a unit left). The solution is to mail in the deffective unit, and the replacement unit should arrive shortly after reception of the faulty one. After some arguing along the lines of "I'm not paying 20 bucks to send you back your broken stuff after several of your staff confirmed it was broken, onsite and off-site" / "tough luck then dear, we can't do anyting without it", it is agreed that the replacement be sent. With the threat of just leaving for the competition providing some, erm, lubrication. The delivery adress being set as the customer's _work_ adress, to avoid wasting to much time.

      6) several days later, nothing being delivered, it appears that the customer's adress was not updated, and that the part was shipped to a previous _home_ adress, and returned to sender. Obviously. After much arguing, the home customer adress is reset and a delivery is agreeed on. at the customer's _work_ adress, again.

      7) several days and a couple of delivery notices at the customer's _home_ adress later, followed by delivery to the wrong Post Canada pick-up point (the sole blunder here that is not blamable on Bell Canada), and after threats of bringing the matter to regulatory instances, an agreement is reached for reimbursment of the 2+ month of paid-for -but undelivered- internet service and termination of the contract.

      And they lived happy ever after.

      As also demonstrated by Trevor's story: when you suspect a problem could be down to Bell Canada, don't do chit-chat. Directly go for the official complaint. You'll save heaps of time.

  18. vgrig_us

    The more things change...

    I saw a similar problem in NYC few year sback - my accounttant switched his local phone service from Verizon (local baby bell) to AT&T (i told him not to btw).

    So, his phones went dead - can't receive calls.

    Hew in the states local bells actually share equipment at CO's, so coming into the switch - Verizon, going out - AT&T. A trigger on AT&T's side was misconfigured - totally AT&T's fault. After full day of finger-pointing - problem solved in 2 minutes.

    1. Anonymous Coward
      Anonymous Coward

      Re: The more things change...

      Share equipment @ COs? Are you high? Like hell they do!

  19. ElReg!comments!Pierre
    Unhappy

    The OTHER lesson

    Don't get involved with Bell Canada. Their support (be it customer support, tech support, for home or enterprise users, ...) is atrocious. As long as everything is OK they are great; come the slightest glitch, and suddently they can't find your references anywhere, their departments put the blame on each other and interminably forward you from one to the other to the next to the first. Resolution can take month for home users; usually faster for enterprise accounts, but still unacceptably slow. I can't begin to imagine the mess that their internal working must be, to allow for that crap.

  20. Christian Berger

    BTW, why 800 numbers?

    I mean its fine to have a normal number _and_ an 800 number, but barely any customer needs an 800 number now as many have a flat rate for phone calls. I'd simply advertise the normal phone number, plus the 800 one in smaller print.

    1. Trevor_Pott Gold badge

      Re: BTW, why 800 numbers?

      We did and do have non 1-800 numbers. But it's complicated. We had shut down two stores covering "the parts of Canada that are not Alberta." The remaining stores had only Alberta-local numbers. Also: many of our customers had the 1-800s on speed dial.

      Oh, and it also came the same week as the website contact page was completely redone. ;/

      Annnnd we had just recently sent out email newsletters and physical pricelists that had the 1-800 numbers on them.

      So, folk were able to call the local numbers, sure. If they knew about them, and didn't immediately freak out when the 1-800s stopped working.

      1. jake Silver badge

        Re: Re: BTW, why 800 numbers?

        So, basically Trevor, your manglement sucks, and has absolutely no concept when it comes to keeping customers happy & employees gruntled?

        Why are you whining here? Find a new job. You'll be a lot happier.

        1. Trevor_Pott Gold badge
          Stop

          Re: BTW, why 800 numbers?

          I think your comment and questions are rather wildly off topic, and I really recommend this be taken elsewhere. E-mail is probably the most appropriate venue for such a personal discussion, if you feel it must be persued.

          1. jake Silver badge

            Re: BTW, why 800 numbers?

            Uh, Trevor, you're the one who brought the subject up. I'm just commentarding, based on my reality. Sorry if that scares you.

            Nice to see that you've (apparently) created a FanBoiPage for little ol' me ... I won't be visiting. Too many years of Usenet says "no".

            1. Trevor_Pott Gold badge

              Re: BTW, why 800 numbers?

              Sir, someone asked me a question. It was actually raised a couple of times in the thread. I responded with as much detail I reasonably could. From this you somehow decided to extrapolate that I "was unhappy" and felt the need to inject your uninformed viewpoint. This doesn’t “scare me.” It makes me irritated feel sorry for you that you feel this level of unwarranted and undesired personal dialogue is requested or required in such a setting.

              The rest of us are trying to have an on topic conversation about the article. Kindly restrict yourself to such, or take the conversation - and your uninformed speculation - to a more appropriate medium.

  21. Anonymous Coward
    Anonymous Coward

    With a certain major telco here in the UK

    the company I work for found out last year that on-site engineers have to go through the same Indian call-centre monkeys as the customers. Lost count of the number of times the guy tried to explain that he's the engineer and yes, the filter is plugged in.

  22. atsak

    Welcome to my hell

    This is why I'm always trying to drive the costs as close to zero as I can; if I'm going to get jerked around I may as well not pay for it. It does not matter who you use (I work with Rogers, Bell, Allstream, VOIP Innovations, Vitelity, voip.ms and Telus to carry our various telecom services) they all make mistakes like this; Rogers did a very similar thing the author describes when porting our numbers from Bell to Primus in a recent drive to save themselves some money, I assume, broke the lines for three days; only a LOT of sleuthing on my part resolved the issue. All supposedly transparent to me, and everyone denying responsibility all the way along.

    The number port management in Canada and the US is broken; it's too hard to find out who "owns" the number so you can actually get things fixed.

    1. kain preacher

      Re: Welcome to my hell

      There are tools out there that will tell you which telco is the parent company. But that does you no good because you can't or are not suppose to talk to that company. The rules are you are suppose to talk to the company that is porting the numbers. The only time you are allowed to talk to the losing telco is if they pull the number early. When that happens the losing telco will often lie and said we released it so the company should be able to just pick it up . That is of course wrong . The process is automated. When orders fall out like this is not easy to just flip a switch. Just because they released the number does not mean they put the route in yet to forward the number to the new telco. The new route may not have a chance to propagate in the losing telco and or the new telco switches yet . When that happens the number can randomly fail.

  23. James 100
    FAIL

    BT fun

    About 4 years ago now, I put in a tariff change on a single business line. Pretty trivial really, you'd think ... until the line went dead. BT Faults said "oh, but you asked for the line to be disconnected! (check) Oh, whoops. You asked for a change of tariff, but it went through as a disconnection instead. Nothing we can do, you'll have to pay a huge early termination fee and order a "new" service to replace the line we cut off by mistake." Weeks later, they had refunded all the hundreds of pounds of wrongly-imposed penalties and reconnected the line, but it took a LOT of calls to many different BT people to fix.

    Then there was a domain registered through BT, being transferred to another registrar. Without even confirming receipt of the transfer request, they changed the DNS entries on their end to point MX records at 127.0.0.1 - causing every incoming message to bounce. That didn't make people very happy either, as we waited several days for the new delegation to propagate and cached entries to expire. Of course their answer was "not our customer any more, go away". If they had *stopped* serving DNS records, I could understand, but deliberately causing all mail to bounce with new records of their own? Downright malicious.

  24. kain preacher

    Beware of porting numbers out of Verizon . They have a habit of pulling the plug early . Usually with in 48 hours of the port date. Why 48 hours ? Because there is this regulation that says the losing telco can not have direct contact with the customer 48 prior to the port . So it makes the new phone company look bad. Oh any one want to discuss caller ID hell ?

    1. Trevor_Pott Gold badge

      Might be a bit poking the nest with a stick, but...caller ID hell? Now I'm curious.

  25. kain preacher

    Because each phone company is responsible for maintaining there own caller ID data base . Lets say I'm small telco . My customers shows up wrong to Comacast customers . All I can do is update with a national data base and hope that Comacast will look there with in 90 days . Comcast insists it's my fault even though I can't touch their caller ID data base . Oh it shows up right to ATT customers

    1. Trevor_Pott Gold badge

      ...

      That's terrible.

      1. Anonymous C0ward
        Headmaster

        What is?

        His punctuation and grammar, or the state of caller ID?

        1. Trevor_Pott Gold badge
          Pint

          Re: What is?

          ...the person using a number as a replacement for a letter in their nickname is being a language pedant?

          Also: the state of caller ID is terrible.

This topic is closed for new posts.

Other stories you might like