Re: The old quote
It's a Bank. So fool with someone else money, most likely taxpayers if things go south.
29 posts • joined 11 Oct 2013
Here are some larger traffic sites (actually bad word) most what is build with Django or Flask count as applications/application backends rather than websites.
Maybe your experience is different, but I've used both strongly and weakly typed languages and most bugs I've created couldn't be avoided with typing.
Typing gives better developer tooling and maybe makes it easier to understand the code in larger projects, but preventing bugs not so sure. My feeling is more that, I write a different kind of bugs in JS compared to bugs in Java. The absolute number of bugs seems constant...
Typescript I feel is the wrong solution to what might be a real problem for some larger web projects. Why I say this. Each compiler step or added indirection step added adds complexity and potential of hard to understand bugs. What really had made my code more reliable is moving to much simple programming paradigms.
Moving to class-based OOP or even procedural code with very little abstraction creates more lines of code, but also seems hugely more reliable. Especially so if its maintained by less experience/cheaper developer resources.
I think there is much cheaper optimization that a lot of people tend to forget. Look at which part of that data actually is needed for real-time.
I worked on a few SAP HANA enterprise projects recently and I managed to convert 2 supposed 4TB in-memory databases to 512gb in-memory databases simply asking business which data is a need for interactive analyses and cross-referencing that to database IT side of the house was trying to load in memory. 512 is still a good chunk of RAM but now I have a DB that can fit any decent x86 box rather than these enterprise models which take whole SMB shop server budget per unit...
What Amazon has and I mean retail arm is an enormous amount of customer trust gained by experience. Its kinda sad that in eCommerce you great simply by delivering what you promised, when you promised and compensate customers without making them hop extra hoops when you failed either of the first two.
This trust is a huge thing and gives them enormous revenue opportunities going future. ECommerce accounts still only 10% of the total retails dollars but growing at 17% space.
This growth is also accelerating. If Amazon can keep even close to their current market share they will be absolutely gigantic company once eCommerce hits 50% of the retail and that day will come. While not as dominant in Europe Amazon takes 0.4 dollars for each dollar spent on eCommerce in North America.
So you are saying people who build highly successful web/mobile applications aren't professional developers? If you make a living on something is that not your profession?
That being said there are a lot of people without a clue or with limited industry experience going round StackOverflow but that's fine. It's a place to ask questions after all.
Python is an important language. It's dynamic and therefore limited when used with large teams, but it has really great support on data science and machine learning side. Furthermore, for a small team, it's a great way to get your web app or API up quickly while building something that is also secure and maintainable. Just check Django's out of the box security & project structure. Beats most ways of building same things with JVM languages with the exception of modern versions of Spring. Somewhat surprisingly writing Python also pays better than Java for the same job.
That said, I agree with that if you have large long live applications they should be written with a compiled language, simply because tooling support where the project is too large to understand for anyone developer entirely. Does that have to be Java? Not so sure.
Big things are being built with Go as we speak cool opensource examples being Docker, Kubernetes, Ethereum, Terraform, and Vault. For-profit world besides Google, Github, IBM, AWS, Microsoft and Pivotal some financial institutions and most Chinese mega Corps are picking the language up as less verbose/complex alternative to enterprise Java. Here is a good bit what the language is about https://www.youtube.com/watch?v=k9Zbuuo51go
The whole thing is to build for maintainability and software engineering at large scale. Not something Java was ever purposely designed for. Will it make sense to migrate some old core banking stack to it? Probably not. But it will be a really good tool to build new better one. Look at Monzo https://monzo.com/ in the UK they did it.
All that rant aside I don't hate Java(wrote some yesterday) I'm just saying keep up with new stuff to keep your options open rather than fortifying on the old tech stack. All it takes is a new company CTO/CIO and your job could be gone with that mentality.
Don't know how its in UK universities but elsewhere that has somewhat changed already.
That being said it's going to take time before it actually impacts hiring.
If you are looking some senior +5 years in the belt having more diversity in 3 rd year students is not going to make a dent until 8 years time...
Working on IBM Cloud this would be a good time to blame competitor but in fact, the end customer is at fault here. S3 bucket ACL is fully private by default allowing access to bucket owner only (not even other admins on the same account). You need to explicitly change policy to get into mess like this one.
I have been sorting a mess on few customer cases with badly configured buckets.When customers change their buckets open to all they usually don't understand they are doing that, and that is because they aren't generally directly applying the change to the portal or with CLI.
What actually happens is customers start to use Python, Java or Js library in their application to use S3 directly as storage backend. A good example I know would be Django-Storage https://simpleisbetterthancomplex.com/tutorial/2017/08/01/how-to-setup-amazon-s3-in-a-django-project.html these libraries expect you to pass AWS console API keys as env variables and do "required changes on buckets they create..."
As these libs where most parts designed as storage for website static assets like user uploaded public profile pics security has never been much of a design point. Next some dev figures these libs are also pretty handy for storing more sensitive content. I mean once configured with the backend, they are just tag you can use directly in web forms, a security review is not part of CI/CD test suite and the rest is history...
Plumbing is always a good skill to have... I was working as a welder in Norwegian gas fields during my years in university. That said if you know OpenStack you probably know at least a bit Python.
Start from there. Cloud or on-prem what I really find is companies are looking to automate(they often move to cloud as it's seems easier to automate). If your job is to manage Linux boxes using SSH your job perspectives are going down the drain sooner or later. So based on Python learn Ansible/Salt to automate infrastructure then learn python APIs for 1 or more of the cloud vendors. All have a Python SDK (AWS is called Boto for some reason...)
Companies are cutting admin staff but same time ready to pay top money for people who can automate admin tasks using code. Other things that seem hot right now are Docker containers (you can run these with Ansible as well) and Kubernetes for managing a huge number of containers. Kubernetes yml file does pretty much the same thing as OpenStack heat chart except it uses containers and seems to gain popularity much faster.
I agree with this. Move or be moved! .Tech keeps moving. Always been that way.
There are plenty of clouds projects out there that could be considered missteps, especially all the let's move our legacy ERP to cloud projects, but practically all greenfield projects get to build for one of the major cloud providers.
A lot of legacy software also runs great in cloud especially opensource side of the fence. I think we are seeing the move to pay as you go cloud infra, but at the same time, we are also seeing a major move towards FOSS software.
I have experience both on-prem and cloud systems. There are few things to that make the cloud shine.
Tooling. Monitoring, automation, access control and security are an order of magnitude better than anything you could get in-house unless you work at Google.
Automation is built in! This is big. I no longer need SSH and manually build everything or hope my "golden image" updates.
It's not all unicorns and rainbows, however. Lof of this tooling works really badly with legacy software bought from some regional ISV... For this look, if they are moving to SaaS model or at least do they provide premade IaaS images or Cloud deployment guides.
If not stay away from these projects. They are never on time, on budget and you get the blame....
A lot of software usage moves to cloud either by existing vendors SaaS offering or new comers taking the market. Initially, these saas solutions are built on 1 of the 3-5 big players in the field.
Some point however many these cloud offers to grow to a size where price scalability and automation benefits from cloud vendors no longer weight as much as possible to have purpose engineered hardware together with purpose build low-level software to get absolute max out of everything. This will result in players who build integrated HW + SW + network stacks servicing single SaaS apps globally from distributed systems. First movers will be capacity heavy players in the consumer markets. Many enterprise SaaS solutions will never reach these loads & and will stay within public cloud vendors.
Dropbox is a good example of this and Netflix has also moved the majority of their stream to what is essentially purpose build, global, single tenant CDN. https://www.wired.com/2017/04/building-ai-chip-saved-google-building-dozen-new-data-centers/
I cannot see enterprise HW vendors ever returning as the whole problem with that kit is always been its desire to be something for everyone and including a lot of extra HW + system SW cost in the package. Someone like SuperMicro or even Dell DCS might manage, however.
Dropbox story : https://www.wired.com/2016/03/epic-story-dropboxs-exodus-amazon-cloud-empire/
Google purpose build HW: https://www.wired.com/2017/04/building-ai-chip-saved-google-building-dozen-new-data-centers/
Facebook Open HW : http://www.opencompute.org/
I use both cloud version and a local version of Office if you have a subscription you can also install it locally. Note that you need to do a little setup in order to also have/save your files or some of them locally.
I also use Gmail or G-Suite like they call it in some client projects. No local version available but have to give it to Google. The platform has never been unavailable or even experiences small performance degradation that I would have noticed.
The bottom line is both platforms do what it says on the tin and most users know how to use them. They use exchange at work but mostly Gmail privately. Excel is still better than sheets, but G-Suite is cheaper if you don't have complex needs.
I presume that is sarcasm, but anyway plot is to expand Amazon's retail business. What tech Whole Foods use or used is probably the least relevant thing in this. Generally, I think right now Amazon should just try to buy everything. Why their own highly inflated stock can be used as "cash" in these acquisitions and it must be clear to Bezos that stock price won't stay at its current level so instead use the stock to buy other businesses while you can.
They also just bought Souq.com allowing them to enter middle east and Arabian speaking markets
I always encrypt my photos and reason is that along with normal photos I tend to have a copy of my passport pages, drivers license, pics of all my payment cards (for their numbers) proof of address and various other important documents that could be potentially be abused if stolen.
I have traveled a lot and after getting my wallet stolen in Iran I have figured I always have digital copies of my documentation just in case. Besides my own USB copy and phone, I do also hold copies in my google account where I have 2-factor auth.
This is for extra security which I know is trouble if I lose both my wallet and phone, but my partner's phone can unlock my google account as well. I don't trust that Google would not use my private documents for marketing purposes, but I do trust they won't leak them out as there would be no profit for them and it would come with a huge reputational cost.
Why do I see this comment every time there is cloud anything. Most companies I ever worked for trust, 3rd parties access their data anyway and probably have a poorer track record on availability done in-house than any cloud company.
Questions for an anonymous coward:
1. Do you use colocation or ISP for internet access? There are someone else's computers are handling critical part of your infra if you are.
2. Security side, do you have better resources & skills against for-profit hackers or DDoS: ers then let's say Amazon, Microsoft, Google or IBM? It's their business to keep hackers away anything else would be marketing disaster.
3. If NSA/CIA/FSB/MI6 etc. wants to get your data do you really have means to secure it? Legally or physically. I mean UK government is not any better than US or Chinese when it comes to data privacy. http://www.reuters.com/article/us-usa-britain-surveillance-idUSKCN0VE2B7
All above does not mean that cloud computing is the answer to all the ills at least not yet. I have a really good reference point from back home Finland. There are a lot of paper companies there. They run a huge amount of electricity on the cellulose process. When Finland started to build national grid there was a huge outcry from people running electric power plants for these paper mills, claiming that grip power is more expensive, it's not reliable, out of your control.
Took 2 decades and paper mills now operate on grid power but have bought a huge share of the business of the major electric companies so they can affect the placement of next nuclear power plant, for example, to make sure that the grid of Northen Finland is not depended on just 1 -2 main lines.
What I want to say is that if you don't want your business side of the company to outsource your IT department completely to the cloud vendor. You need to show why it's not a good idea. Remember you are talking to a business person who knows he is already outsourcing your critical data. Your company's accounting company for example probably has access to almost anything in your company, so does your contracted legal company or that company that consults your upper management with business management. How would cloud vendors be less trustworthy?
You need to show it's not financially viable yet, or there is a type of skill in your department that is a genuine competitive asset to your company. Going back to my example below. There are paper mills that never moved to grid power in Finland.
Why? They had their own water power plants that generate basically free energy they still connected to grid as backup. Maybe most IT departments should look at the cloud the same way.
You can build secure (data access secure systems) on-prem or in Cloud ( call it hosting if you want). Everything is going towards internet accessible systems so you can no longer just build perimeter protection and allow access only from company LAN.
If you still work in one of these LAN companies those LAN's are more often than not connected using MPSL links (other people's routers and switches) you do not control. Many people do not realise this and run MPSL links without GRE or any other type of encryption. Many of these on-prem heavy enterprises usually have a fleet of MS2003 or even NT servers still running on the same network to provide some legacy application no one even uses... You just get in from one place and you quickly everywhere ...
I am not saying that you should stop using firewall or antivirus. What you need to do is to compliment existing protective measures with encryptions networks connections across the system. Using encryption in database and filesystems using keys you control. Most importantly you need a good process and mitigation systems if and when attacks happen.
Basically a good system that tells what you have, where, generating what type of traffic patterns and firing up large red light soon as something behaves way not expected.
Additionally you are going to need to protect yourself from DDoS attacks and update your system maintenance policies you have to update critical security patches soon as they are out not 6 - 10 weeks from it.
This should protect you from most evil actors as most attackers you get are for-profit hackers not NSA. Why? This does not make you unhackable but there are many easier targets for these guys to pick.
Outside of Telco, defence and large finance there is simply very little NSA, FSB or Chinese want or cannot access easier way than hacking you.
** In past I served Finnish Army Electronic Warfare Unit (counter hacking and protection of nationally important infrastructure). We have such a lovely eastern neighbours I even got some practice. To know what ability even a small country like Finland has in this fields just proves that against NSA or China, if you are resource wise anything smaller than Google, Apple or Microsoft you have not hope of protection anyway.
There are some merits for lock-in statement especially when end customer has minimal depth in its own engineering department and trusts most work to be done by 3th party ISVs.
What I have witnessed from cloud partners for every vendor are poorly documented deployments with high levels of clouds own internal API integration on the automation side. This does not mean that you cannot migrate but it will probably mean it will cost you. I think they mention RedShift on purpose as that is one of the hardest migrate from targets in AWS. RedShift, after all, is proprietary to AWS and missing any clear 1to1 featured product on the market. Similar story with DynomoDB, but that you can easily drop to MongoDB.
But Larry Ellison throwing FUD on price raises and vendor lock-in!!! I mean Larry of all the people. If Larry feels there is need to trough this type of statements, clearly OracleDB customers are actually leaking to other vendors.
Larry if you reading this remember your core business is basically squeezing every last penny from your existing clients by fearing them with impossible migration (which is actually pretty easy at least to -> MSSQL or PostgSQL) and by suing them after they break incomprehensible license agreements you changed after the contract was signed.
I cannot see need / room more than 5 - 10 (is counting Asia, and Russia) players on the general-purpose IaaS / PaaS on the global market. Out of those 2 -3 top players end up running on the the size of all the put others together.
There might be room for few niche cloud vendors for Telco clouds, Financial institutions and potentially someone professionally fitting country size public sector clouds. That said most applications even from these areas can now be services from larger players locations and portfolio.
The reality is providing cloud on a global scale requires the kinda capital and diverse organizational skill base that very few organizations can master in IT industry. It's not only about infra management, its support organizations, logistics, software development, reliability engineering, digital business channels and streamlined business processes.
This kinda news only the make demise faster for the smaller players as potential customers levitate towards bigger players for a better feature set and lower risk, allowing them further investments on their platforms.
in my view, no-backend do work but only in greenfield. It's not that hard to build app to solve a problem people have even in an environment with many constraints like, AWS Lambda, Firebase or what used to be Parse.com (RIP).
But that is greenfield only. Try to integrate that with an existing technology stack if you have one and I'm sure you not gonna have a good time.
Only feasible way (does not break your mind or budget) Is to add rest end point to existing data sources and only integrate them at the view layer. Meaning just present the data in the UI like it was coming from one source while it actually is not.
I fully agree that IT can be core part of company's competitiveness, BUT that competetiveness is surely not coming from racks full of big vender hardware running standardiced enterprise software and impletement with common industry datamodels!
That just gets you to same line as anyone else. If you host the same thing in cloud again you are just on same line as anyone else.
In my mind competetiveness what IT brings must come from inhouse written code, analytics and automation scrips for menial backoffice tasks. Why? Because here you can drive real difference in how business operates.
For example I see most financial institutions still filling compliance docs manually with hundrads of staff to a word docs before submitting to review in Luxemburg. Now few organisations have automated this together with autocollecting data from their fraud deparments to used in these reports. This is faster, less error prone and much cheaper. Its the software that makes a difference not if you run it on cloud or on prem.
Why cloud is gaining popularity is that it is making easier to build something like this compared to on premise alternative. You get SQL services, API gateways, NoSQL, MQ services, , object storage with map reduce and many other things.
When I talk to on prem folk I just get VM and some storage, then spend 80% of the project time wiring up stuff that cloud already gives with nice API... If on prem vendors actually figure that this where they lose to the cloud and start offers services rather than VM's from booting the box I have no issues with on premise. If not on prem IT can go the way of the dinosaurus...
I think benefits of cloud gives you depends a lot of what is been deployed there, and how company operates.
1. Who benefits most. Born in a web mobile and SaaS app companies less than 5 years old with no legasy infra or team to run it. Why they use open source software (no licensing constraints) can design the deployment for cloud in mind. Why they benefit. No data centers and more importantly minimal ops staff (cost). Can scale fast if become successful (scalability). Have the bandwidth needed to serve high traffic web apps. (bandwidth)
2. Second group who usually makes returns unless hampered with huge bandwidth costs is more traditional companies entering new markets and projects. Do you really want to build data center in South America as German company just to serve niche market with responsive web commerce solution(global)? Do you want to try figure out how to put together IoT platform rather than use PaaS solution (time to market ). Business-driven IT projects. Do you really want to go through 12 months security review and provisioning process with IT department when you are delivering marketing site that renders HTML with no customer data (time to market)
3. Group who I rarely see to get any benefit or cost savings in the cloud. Enterprise company who deploys traditional enterprise apps with licensing agreements(usually blogs scaling and is not designed to constrain bandwidth usage) to cloud creating these weird VPN accessed mutants(perimeter security) that consume huge amount of bandwidth (cost) do not add any agility or any business value. These are generally done because someone wants to use "cloud". Only industry where I see positive cost cases for these are public sector and financial clients. Why because their own IT cost structure is so bad that anything is better by a mile.
Above opinions are mine, but they been formulated after hundreds of production cloud deployments across industrues, market's and country borders.
I work as engineer on one of the vendors listed here. Maybe we are talking about a different things but most of the systems we see moving first are external customer facing or partner facing. This means that the actual users come over the internet regardless where the system is hosted.
Compared to on premise using any of the big 5 cloud vendors you get none of the bigger players put data center anywhere that does not give connection to at least 2 physical power networks and 3 telco networks for redundancy.
Now when it comes to internal applications yes can connect your on prem to cloud with fiber cable if you need to / want to, but reality is changing there as well. I routinely work from remote sites, home (we have some who come to office on once every 3 months) or from customer premises so internet just has to work thinking that it's ok as long as connection to working to local premise is no longer true anyways.
Having a local apps to avoid internet related issues might still be true for a hospital, manufacturing site or a university but not for most other environments. Even there internet down pretty fast effects on communication, supply chain etc.
Agree completely. I have seen how many of the "in house" data centers are run especially outside of the absolutely largest organisations. Many of the people running these are first to point raise huge noise on cloud downtime event but cannot even provide real uptime figures of their own bit barn.
Or when they do those figures do not include "scheduled maintenance" that usually consist of every situation where IT was first to figure DC is not responding.
To be honest I have never understood the concept in reality. I can see that maybe if you are big enough with many enough departments like government or really large corporate building a private cloud might make sense for 2 things. 1. Billing split between departments. 2. If departments have actually highly different needs on when they need capacity this could lead to higher utilization and bit smaller HW bill.
However to be honest for the second part. You can already do that simply using over provisioning with your hypervisors.
Anyone outside of large organizations I find it really hard to see what could archive with spending money building private clouds. If you need pay as you go just go public cloud.
Even if you take something free like Openstack to build private cloud yourself maintaining it will be extra burden and potentially initially calls for outside help (cost) at least for design and build.
Well if you refer to performance comparison done outside of Redmond I would like to see it. If KVM does not scale why is it use by Google?
If Xen is bad why AWS runs on Xen or Softlayer(my company) dont get me wrong we support Hyper-V as well but I really don't see it a solution outside of MS land even with the linux support. If you are full MS house with limited size this might well be the right chose.
If you want it easy use VMware and pay the premium. Soon I think that Openstack + KVM compo is doing orchestration so well that you don't have to.
Main question is: Do you soon need hypervisor at all? Openstack is getting direct Linux container hook soon and you could run CoreOS based semi PaaS directly from baremel from your own DC. We can do it today but at moment it is outside of capabilities of many IT departments.
There is real reason why business does this. IT department often acts as cost center not a source of innovation. I mean 10 years in business and I have never seen internal IT come up with idea how to run business better, faster or more effectively.
I agree cloud is marketing construction but it gives the power to the business departments to do IT when they like how they like it.
Biting the hand that feeds IT © 1998–2020