Agile – The Ugly Truth

If you are an organization that is considering moving to Agile, especially a larger one there is an Ugly Truth that Agile consulting firms don’t want you know:

You don’t need coaches to come in full-time to help you on your journey……

Sure you can spend a ton of money on people who will come in and tell you how you should be doing standup, scrum, retrospectives, etc….Try to get everyone to do the same thing across your organization.  The Truth? In successful Agile organizations everyone can be different, that’s ok.  Yes you want some consistency but doing things by rote doesn’t get you where you want to go.

In the coaching world we talk about organizations Doing Agile who never get to Being Agile.  What’s the difference??  Everything.

Yes the basics of Agile include things such as standup, retros, sprint planning and release planning, but those are the basics that can make you feel like you are Agile, but that’s about it.

Agile IS a transformative process and one that if you as an organization by into, will receive great value from in the form of valued delivered in a more consistent manner.

I remember years ago when I was managing a technical project that required that I purchase some hardware and though I had found a better deal with a different vendor my boss told me to buy IBM hardware (at a much higher price) with the advice – ‘You don’t get fired for buying IBM’.

I think that many large organizations fall into this mindset and when trying to move to Agile  They fall into the mistaken belief that if you hire one of the large Agile coaching groups or hire Agile consultants that you will ensure success or at the very least not get fired because you hired ‘Agile professionals’.

Yes you do need people who can provide support in the important areas that Agile truly needs to have in order to bring the value you desire from it:

  1. Automation – Develop, implement and improve your test automation frameworks.  This includes unit, integration and functional areas.  You MUST make this investment and you need to get to high levels of automation, because without it you cannot go fast, time for testing will hold you back.
  2. Planning and Estimation – An area that is very much overlooked when moving your product development teams to Agile is the need for these teams to form a strong working relationship and learn how to plan in estimate in Agile.  The notion, especially for larger organizations, that you don’t need to do some level of up front planning is a sure way to fail and fast.
  3. Organizational change – Probably more than anything you need to understand that your organization will change, significantly in many ways, when you move to Agile.  You need to prepare your managers to let go and let their teams own their deliverables (trust me if you give this to people they will own it).  You as an organization need to understand that your management group will change, you will need fewer managers (sorry folks that is a reality).  Though in reality many of your current managers were placed in these roles as more as a retention means over a true management track that the organization has defined.  These people need to move into your Tech Lead roles, they run their scrum teams, own the technical and architectural elements of their product code.  Managers will oversee several Scrum Teams and will be focused on larger concepts such as technical roadmaps, cross organizational planning and staff development.

So the Ugly Truth is that you don’t need to spend hundreds of thousands of dollars on coaches who will tell your teams what to do.  Instead hire people who have been on the ground, who can help your teams learn the skills they need such as TDD, BDD, User Story Mapping and Estimating.

Start small and grow the process into the organization.  You need to get your processes working in a small-scale and then move outwards into the organization.  The people who are successful in your pilot groups become your sales people for the next groups.  Trust me, I listen to the people who actually do the work more than I ever do people who try to provide me basic information that I can find anywhere online.

 

The Myth of Rockstars – Teams Deliver Not Individuals

Over the years I’ve had the opportunity to work with numerous software development teams in both large and small organizations many of whom have a singular focus of hiring technical rock stars.

Though it sounds like something we should all aspire to, I think that we often lose sight of how great a team is when everyone works together over the individual rock star mentality

Point in case the 2014 Chicago Bulls.  They yet again lost their rock star (Derrick Rose) due to injury and the team could have just thrown in the towel, but they didn’t.  They weren’t considered ‘rock stars’ by how people in the NBA view talent , but they had something that rock stars don’t always have, commitment to the team.  A selfless focus on what they had to do to win a game in the NBA and for several months they had the best record in the NBA from January through March.

Another Bulls analogy would be the championship Bulls of the 1990’s.  Until Coach Jackson was able to get Michael Jordan to work as a team member and not be the rock star that he obviously was, the Bulls didn’t win a championship.  Only when Michael began to work within the team structure were the Bulls able to win six championships.

The point here is that rock stars often can’t/don’t deliver for a team, rather they achieve greatness within the context of themselves.  Many technical rock stars are singularly focused on achieving technical dominance based upon this mythical rock star status we give to them.  This leads all to often to these individuals climbing into leadership roles for which they have no real skills to succeed with.

Teams that deliver understand that we all must sacrifice something of ourselves for the betterment of the team.  Rock stars don’t see it that way and those that never see it that way are destined to be winners to themselves and not to a greater good.

Sports unfortunately is often the best analogy to teams and in Agile our Scrum concepts come from a team formation from Rugby, so perhaps the sports analogies are appropriate in most cases when talking about technical teams that deliver.

Agile is all about delivery, and fast.  But with speed comes the need for discipline, multiple disciplines in a Scrum team.  Everyone, and I mean everyone, has to pitch in.  If you are a Java developer and you have legacy C++ code, guess what??  You need to learn from C++ so that when time is tight and the delivery important you can step in and help out.  That’s teamwork, no rock star needed, just a willingness to put yourself out for your team.  If you are a developer and there needs to be testing in order to close out the Sprint, guess what?? You need to test.  It’s amazing what you will learn when you test your own code from another perspective.

When I’m hiring I look for people who have had broad level of experiences.  People who have done just one thing but are considered a Rock Star for it, aren’t really my interest.  Great products and high quality come from those with well rounded backgrounds, who see the edges as places that we need to explore.

Great teams have that, broad level’s of experience.

So when you are thinking about building a team remember that an organization, any part of it, is a sum of the total parts, not just the rock stars.

 

 

 

Security and Compliance in Agile

It’s been awhile since I last posted anything but that’s because I’ve been knee-deep in helping a Security and Compliance team move from a traditional waterfall approach to an Agile approach to developing and managing their work.

Talk to any development group in a large organization, especially one that is Agile, and say the words Governance or Compliance (or worse Audit) and you will get a collective shudder and the sound of everyone running for the door.

The truth is this: Every organization has to protect their customers and their brand.  We do this in many different ways, but one thing that we don’t do well typically is ensuring that our Product Development teams actually know what these groups do and the value that they can bring in knowledge to developers.  We often view developers as people who know everything, hell they can code to make features come to life so they must also know about secure coding practices and the like (guess what, they don’t)

Security Governance and Compliance shouldn’t be considered necessary evils, rather they should be considered insurance policies that we take out every time we push code to production.  You wouldn’t think about driving a car without insurance because if you do hit someone and injure them, it will cost you a boat load of money.  Yes the odds may be small but taking that risk is one that most people won’t take.  We should look at our Security and Governance in the same light, in fact an even brighter light because very few drivers are out there actually trying to hit you to hurt you, whereas hackers are doing just that to your site day in and day out.

With the speed at which hackers and attackers are figuring out new ways to breach our security protocols Security and Compliance teams need to work in a much faster pace.  Think incremental improvements over gold plating a solution.

In Agile teams are focused on delivering value to the business every two weeks and as they go through their process of iterative design and development we also focus on technical excellence.  Product Development teams need to be able to get fast feedback on security questions so that they can incorporate Security changes before development begins.  As a Product Owner you have to understand that the velocity of your team to deliver value MUST include time for them to refactor code and implement ever better secure code, it’s simply not up for negotiation if you want a high quality secure product that your customers engage in.

The teams that I’m working with have been willing to engage and somewhat leery in moving to Agile and we are tackling how to handle a team that doesn’t deliver code and whose work doesn’t necessarily fit into a strict 2 week timeframe.  Additionally these teams aren’t used to thinking in short increments so breaking down their work doesn’t come naturally, but they are finding ways to do it and they seem to like the visibility and transparency Agile provides.

For Story estimation I developed a quick and easy way for them to estimate their work, taking into account that we don’t have true Scrum teams but loosely aligned people working on distinctly disparate tracks of work, it looks like this:

  1. 13 points – One person working one Story for the entire sprint.  This would typically fall into the same category as a Research spike.
  2. 8 points – One person working one story for half of the sprint
  3. 1-5 points – One person working one story for 1, 2, 3 or 4 days.

Some of the team has started to use this and we are starting to see what our potential team velocity might be.  This will help us next year when we do our Roadmap planning.

Additionally since we are using Rally, I’ve created six-week release windows that teams can align their work to and then assign the iteration that they will complete the work.  With both the Release and Iteration views I can see what the team is planning and what they are executing.

Perhaps the biggest challenge that we need to address next is how do we build a plan for work that may span 1-3 years in duration.  The team has commented several times that moving to Agile is hard since they are used to doing everything in a Waterfall method yet the reality is that whether we are doing Agile or Waterfall as a Program Manager I would expect them to identify high level milestones, understand and describe how the project will unfold.  None of this can’t be done in Agile.

The notion that Agile isn’t about planning has always amused me since I know that in Agile if you are doing it right it is all about PLANNING.  One thing that differentiates Agile from Waterfall from a Project Management perspective is that we don’t have a change management process.  I think we have believed that laying out a project in a Gantt chart with tons of up front planning that really results in Change Requests was effective.  Instead identify the key milestones that make up the project, commit to those and then understand that we’ll change along the way but the key business value that we are delivering shouldn’t.  If it does then you aren’t working on the right stuff.

To those Information Security Management groups who think that they can’t be ‘Agile’, think again.

 

 

 

 

 

Agile Metrics

One of the hardest things that we struggle with in Agile is with is how to report how we are doing with respect to our projects.  Agile or otherwise we still primarily think of our software development in a project orientation.

All of our historical metrics from our waterfall days talk about headcount, resourcing, man hours (days), project milestones, etc…..

In Agile we don’t produce the type of metrics that management has historically used as progress tools.

You hear quite often that in Agile velocity is king (kind of like cash) and in a real sense that is true for me.

Velocity represents the amount of value that a single Scrum team can deliver.  We know that each Scrum team has a specific set fixed cost and the value that they ‘consistently’ deliver reflects a key metric for management to key on.

  • Velocity – What can you glean from a single number?
    • Effective Estimation – A Scrum team that doesn’t deliver accurate or confident estimates they will see their velocity roller coaster from sprint to sprint.
      • What to look for? –
        • If a team consistently has to move a story to the next sprint because it couldn’t be completed that is an indicator for estimation improvement.  This has the effect of reducing the points for that sprint as they can’t accept an incomplete story.
        • If a team consistently has large stories entering their sprint, ie larger than 8 points, this is an indication that they haven’t decomposed their stories enough.  Lack of story breakdown results in more unknowns leading to inaccurate sprint commitments.
  • Planned vs Actual – Another flavor of identifying estimation issues.
    • What to track? – Once a team develops a consistent velocity (usually takes 3-5 sprints) then that is what we should expect each iteration, providing people aren’t on vacation, etc…A team that commits to varying levels of story points each sprint typically hasn’t performed enough story development or they lack a well-groomed backlog.
    • Burndown – I specifically like this as it reflects the above issues very nicely.
      • What to look for? – Pretty simple really.  A team that has performed effective story decomposition, planning and estimation should expect to see their work being delivered and completed throughout the entire sprint.
      • What to look for? – Teams who have a cliff dive burndown where all of the story points in development hit QA at the end of the sprint.  (Solution? – QA needs to set up a working agreement so that they will only test ‘x’ number of points in any given day.  If the team delivers 12 points on Friday and the agreement is 8 points for testing, then 4 story points will not get accepted, regardless if ‘development is done’ (cold uh?).
    • Sprint changes – Though harder to track with most of the tools out there (Rally has an app but I’m not sure it provides information that is easily derived regarding which stories actually left the sprint and which ones came in).  Regardless of how you track it, this one metric will tell you a lot about an organization and it’s overall planning effectiveness.  Remember – Agile accepts that change is inevitable, but in the real world you can’t have unmanaged change, teams can’t work effectively or productively in that manner.
    • Sprint Plan vs Actual – As teams improve in their planning and estimation skills, they should also get better at providing accurate estimates as to the number of Sprints it will take to complete a feature (aka project).  One of the things I had a VP say to me recently is that he wasn’t sure Agile was the way because at the end of the day he still had to make commitments to Sr. Mgt.  The notion that Agile can’t provide plans that have accuracy is wrong, however you  but need to focus on Discovery, Planning and Estimation efforts.  And we need to understand that there are places and times where we need to take time to do this work.  Thinking that we should always just be coding is a sure way to build something, but probably not the right something.

Succeeding to Fail in Agile

In an Agile environment everyone needs to understand that ‘being Agile’ is about taking the framework of tools, processes and methodologies and applying them to your specific organizational needs.

At Disney where I was part of a group that moved into Agile, our first foray into Agile Discovery and Planning was, well an epic fail. We we didn’t produce any work product that would allow us to enter a Sprint.  We developed a lot of ‘stuff’ but as it was at is it turned out, not the ‘right’ stuff.

But from that initial failure we evaluated what did work (group Discovery) and what didn’t (PO not taking ownership of story development).  From there we were able to begin to understand the ‘what’ that was important and over the course of around six months we developed a pretty solid delivery model that allowed the organization to feel confident in our delivery abilities.

Now, what worked for us at Disney more than likely won’t work for your organization.  The only way to find out is to try something, anything and see how it goes.

If you want to be a virtuosos at any musical instrument you need to practice, hour upon hour of repetition.  Along this long road of preparation you will find some efforts easy, others hard.  You will fail at mastering some part of a technique that just seems so difficult, you feel you will never get past that point and then you will try something else and BAM, you got it.

From our failures comes learning, the type of learning that becomes part of your DNA. Your mind takes over and can put thinking in the background as you play your instrument.

To be great in Agile, your organization has to change its very DNA.  If you want to succeed, you have to fail and you have to fail often and quickly.

You hear that a lot in Agile when we are talking about our test automation, let it fail early and often. We know that fixing issues while still in development is significantly cheaper than fixing them in Production.

Make sure you instill in your organization, management support for succeeding to fail.

Baseball players get into the Hall of Fame for succeeding to fail at a 70% clip, if you bat over .300 for your career you have a good chance of making it to Cooperstown.

As your teams embark on their Agile journey, keep these concepts in mind to ensure you are driving towards success:

  • Self-organization – We want this to happen and for teams to be empowered, because they know more than anyone, what the real pain points are to their daily work.  Let them try something to see if it works no matter how crazy you may think it sounds.
  • Retrospectives – Make sure that a team that is self organized and is empowered to try new approaches is having regular post Sprint retrospectives.  This continuous inspect and adapt cycle ensures that we are always evaluating what works and what doesn’t.

In the end if things aren’t working – TRY SOMETHING. What do you have to lose?

Investment Portfolio Management Applied to Product Management

I’ve been telling myself that I needed to attempt to apply formal investment portfolio management techniques to how we value, prioritize and manage our portfolio of product development efforts, so here goes (definitely still a work in progress)
Back in 1950’s Dr. Harry Markowitz created an investment model called the ‘Efficient Portfolio’.  Markowitz stipulated with his theory that an “Efficient Portfolio is one where no added diversification can lower the portfolio’s risk for a given return expectation (alternately, no additional expected return can be gained without increasing the risk of the portfolio)”.
The Markowitz Efficient Frontier is the set of all portfolios that will give the highest expected return for each given level of risk.
 
This model set the framework for how current money managers build a portfolio of securities that provide range of investment returns to meet each investors risk profile.  Providing investors with a broad range of risk/return choices allows individual investors to build an investment portfolio that meets their specific risk threshold with respect to a given rate of return.
An efficient portfolio looks like this:
 Efficient Frontier
When you are younger and have time to take chances your risk/return profile might be higher on the frontier, whereas at retirement you will slide down that scale as you are more interested in protecting your total investment.  One thing to note is that if your risk/return data falls above the efficient frontier, then you are accepting a level of risk that is not in line with the rate of return you might receive.  Pushing past the efficient frontier can open you up to unexpectedly high returns but conversely you can also expect very high negative returns due to the risk you are taking on.
Product Development organizations can utilize the Frontier as well, for example, a young startup will have a much higher appetite for risk as they understand that to take market share from competitors they need to take risks with speed to market.  However there are specific elements of risk that need to be considered as you speed your product to market.  Ensuring that Usability has been considered, Prototypes have been developed, code quality is considered and test automation all need to play part when you are building your risk/return Efficient Frontier for your Product Portfolio.
If we were to apply the Efficient Frontier to how we manage our Software Development investments we could build a risk/return profile that is easy to understand and align with the organizations risk/return profile.   There are many software projects that at inception are known to be risky, however a lack of empirical data often means that the projects will get the green light and then fail miserably.  The organizations inability to accurately asses risk/return at any time with their software development investments is a huge blind spot and keeps us from consistently delivering the value that the organization needs to stay competitive.
Agile addresses the value (return) part of what the Efficient Frontier speaks to however it talks nothing of Risk overtly.  Risk is more implied with the notion that we manage it by delivering in short increments and focus on shipping value consistently.  However Risk is more quantifiable as I mentioned earlier.
Building an Efficient Frontier in the investment world is a data intensive effort, which our current product/software development processes doesn’t easily support.  However I believe that we can use the formula that Markowitz created to generate an Efficient Frontier for Product and Software Development organizations.
For this effort we will make some assumptions with respect to the Frontier model and changes to Markowitz’s formula so that it works with our limited data set:
  1. Portfolio = Product Development
    1. An organization can have several Products in their Portfolio –
      1. Consumer Facing
      2. Internal Facing
      3. Infrastructure
      4. Research and Development
  2. A security is equivalent to a Scrum Team.
    1. These would equate to the individual securities that Markowitz speaks to in his model.  Where an investment portfolio consists of many securities, each with their own risk/return profiles so to does an organizations product development portfolio consist of the same.  Each team is a security that can on its own provide return that comes with an associated risk.
    2. Though we don’t think of investment securities as having dependencies (as software development teams have) in fact a diversified investment portfolio consists of a range of investments that will perform a certain way based upon the dependency that business has to the market that they operate in, so in this case the notion of a portfolio still holds as a viable means to build a Product Development Efficient Frontier.
  3. Potential Risk Parameters:
    1. Development Lifecycle – Waterfall, Agile, RUP, XP, Blended (use at macro level). You could equate this potentially to Bonds, Stocks or other investment instruments.
    2. Experience of Team
    3. Number of Scrum Teams
    4. % Test Automation
    5. Code Complexity
    6. Speed to Market
    7. Roadmap volatility

In my next post I’ll provide some supporting ways we can ‘build’ an efficient frontier for Agile Product Portfolio analysis that both Product and Program Managers can utilize to assess priorities for the entire organizational backlog.

Monetizing Agile Projects

Coming from a finance background my education and experience is grounded in ensuring that money invested returns ROI.  We think of things such as ROI, IRR Cash Flow, etc…

For example when a manufacturing company decides to make a hardware purchase for machinery (any kind) to produce some type of ‘goods/product’ they perform financial analysis regarding the cost of the equipment, the rate of return that it will generate, how quickly they can amortize it and ultimately what is the net profit that the machine is expected to generate over its anticipated lifespan.

For most of my career we start on projects that have a goal in mind, potentially new revenue or cost savings, some have gone so far as to try to determine the ROI, but in general I haven’t seen the type of due diligence that manufacturing type companies perform, applied universally to software development.

This may be an underlying cause of many of the software development projects never seeing the light of day or failing to deliver the expected outcome because we never performed effective financial plans that would establish the scope and speed necessary to deliver a software product so that it warrants the investment.

In Agile I think we have ways of providing some level of financial analysis that can provide us with an understanding if an idea is worth pursuing.

  1. Using the following as a guideline we can begin to estimate costs:
    1. Team Size – 5
    2. Blended Rate – $125 an hour
    3. Hourly team rate – $625
    4. Cost of Sprint(2 weeks) – $50,000
  2. Let’s assume that the feature that we want the team to work on has come back with an estimate of 5 Sprints.
    1. Estimated Development Cost – $250,000
  3. Let’s now assume that this investment is expected to yield an additional $1,000,000 in annual revenue.
  4. Expected ROI in the first year – 300%

Before we engage our teams we need to be sure that the $250k investment will return an appropriate level of return.   In this simple example its a no brainer.

But our world of software development isn’t always so clear-cut, we often don’t know what the expected outcome will be until we release the product into the wild.  There is often additional cost for refactoring before the product hits the mark with your consumers.

But using some of these simple ways of working through anticipated costs we can easily modify the example to reflect additional Sprints for refactoring once the product is released.

In this example let’s assume that the team requires an additional 8 sprints to move from MVP to final product.

The final cost of the product would climb to $625k and our return would drop significantly to 54%.  Still not bad but not the eye-popping number we initially thought it would be.

Factor in sustainment costs into this over the life of the product and you begin to see that your investments in your software absolutely need to go through the same level of analysis as other types of large infrastructure purchases that non-software organizations go through.

I’m currently working on creating a lightweight model based upon Markowitz’s Efficient Frontier investment model that money managers use today to ensure that your risk and return threshold is aligned.  I’ll be posting my initial thoughts and approach in a coming blog.