Deliverable rather than deliver
There are many issues with Agile. There is no "proper" way to do it. Proper way to do SCRUM etc yes, but these are just rigid methodologies bolted onto something that was never inteded to have such formality. Even then SCRUM and the like are fine if just used as a guide and forget the "thou must do X" rules.
The key problems I see though are
1) *All* stakeholders which often goes right to CEO level, must be signed up to it and understand it.
2) Stop trying to *release* every week, two weeks.
The latter doesn't mean you stop doing your sprints or whatever you like to call them and you should still have a tested build that you *could* release if necessary. It's just that rolling it out every couple of weeks when it's a buildable and tested product but incomplete, clearly a 'beta' to the user, and may be time consuming to release, and fills your backlog with more issues each time is not ideal. You just end up spending half the 'sprint' planning for the next one.
Just be in a position where you *could* release if you need to, but rather wait until you have the finished polished goods. Ideally don't let the user and more importantly the CEO see the product until you're happy. Else they'll want to make changes! ;)
I've seen Agile done fairly well, I've seen it done badly, but I prefer some attempt at it rather than the opposite to Agile, the "bugs and feature requests keep rolling in, and we keep developing endlessly and never releasing, never having time to test properly, and then suddenly rush a release when the boss demands it that's an utter mess" approach.