Agile is often viewed as the way that small organizations and teams can build product quickly and in fact I believe that to be true. Being quick and nimble allows smaller/growing organizations the ability to get into a market quickly and deliver features that larger organizations haven't delivered yet or haven't thought of.
This very ability to deliver new features to a market that desires them is the very reason that larger organizations need to change the way that they deliver their product. As an organization becomes larger, the entropy that comes from that growth causes the organization to stop being a market leader and start being a market follower.
In the technological world we live in, you can go from market leader to out of business in just a few short years. I read an article recently that suggests that Walmart is starting to show signs that its dominance in retail may be coming to an end. I wouldn't be surprised, as I think they have lost sight of the fact that low prices don't always solve the things we as consumers value. Walmart isn't a market leader anymore, they may just not have come to that realization.
It's this lag in reality when big companies start falling behind Somewhere along the way almost every organization will lose its way, lose site of the very thing that caused them to be great in the first place.
When you look around in the Agile learning space you see many of the original creators of Agile and Scrum trying to figure out how to help large organizations get past their own entropy. I don't think that Agile can help, it can highlight where your planning and delivery processes are inefficient but it doesn't provide the language that senior management needs to hear in order to effect real change.
If we want to help organizations transform and be 'agile' then we need to speak in the langauge of management, which is money. I've yet to have a cogent conversation with management regarding how their lack of focus costs them money and more importantly fails to deliver the speed to market that they need in order to stay market leaders.
If you think that Agile provides you with a means to quickly change product direction in order to play defense or catchup when new features that you have been planning on are released by a competitor, then you are missing the point. Agile does accept that change will happen, but we expect that the change will take the form of measured change in how we can make the product more valuable, not wholesale changes in direction.
Changing direction within a Sprint is something I've seen several times and it's almost always driven by management. Whether you have too many idea's/opportunities or are just trying to stay relevant give you team time to work on one thing at a time. Context switching reduces productivity and that you can translate into money.
The example I've been using recently is this:
If you managed your Sprints the way you manage your money you would go broke. Take this example for consideration (note this is what good Agile teams should be forcing into the planning conversation).
Consider a team that has planned 20 points for the current sprint and someone (Product Owner, Sr. Management) has decided that something else is more important, like completing a feature for a key client. The appropriate response from your Agile team is 'Great, let the team review the request and provide an estimate'. Right there you have probably lost 2-3 points in analysis and estimation by the team. Once they have completed their estimation, they come back to you and tell you that it's a 5 point effort.
You think to yourself great we can do that in this sprint, let's do it. A good Agile team will tell you yes we can do it but you need to remove 2-3 points for the estimation effort AND 5 points for the new work that you want us to take on in the current sprint. So you have LOST 7-8 points of productivity, aka value) to get 5 points of value.
If you invested 8 dollars into a business and only were able to get 5 dollars back you would quickly realize that you can't sustain that monetarily, right?
Put this to your executives next time you have a priority conversation. I didn't even bring in the technical debt we typically take on to hack something together and the lack of testing that will happen because we weren't ready to take on the work (read no automation).
As a Finance guy I would say that this is a bad investment. Yes I want to satisfy my customer but if they are a good customer you should be able to have an ability to manage their expectations so that the team can take on the work in the next Sprint. Depending on when the request came in they may only have to wait 2 weeks.
Effective teams I've worked with use this process as a first line of defense against changing priorities within a Sprint.