Managers

Software Engineering is Not Engineering (More Like Art)

Recently I had an interesting conversation regarding whether or not software developers were engineers or not. The most compelling argument that they could provide for why Software Engineering was truly an Engineering discipline centered around the fact that Universities and Colleges have their Computer Science group in the Engineering department.  That is not sufficient in my mind to by default call Software Engineers well, Engineers. Why? Consider the following

Engineers who build skyscrapers, airplanes, bridges and products that require low tolerance for defects are often operating under the laws of science and as such require that they account for many scientific elements when building their products.

They perform a great amount of 'testing' both in theory and with product before something is released for use.  Their engineering practices are based upon scientific principles that go through extensive peer review and have been building in knowledge for hundreds if not thousands of years.

Now lets look at the definition of Engineering:

  1. The branch of science and technology concerned with the design, building, and use of engines, machines, and structures.
  2. The work done by, or the occupation of, an engineer.
  3. (Business / Professions) the profession of applying scientific principles to the design, construction, and maintenance of engines, cars, machines, etc. (mechanical engineering), buildings, bridges, roads, etc. (civil engineering), electrical machines and communication systems (electrical engineering), chemical plant and machinery (chemical engineering), or aircraft (aeronautical engineering)

Though of this I think can be applied to the science of computers, ie the hardware = Machines I don't however see much if any of this being applied to Software Development.

A paper in 1996 (William Aspray, Reinhard Keil-Slawik, David L. Parnas) talks about some of the reasons that writing code isn't associated with Engineering:

"Computer science is often characterized as an engineering discipline with the systematic study and development of software as its principal subject matter. Software Engineering, however,although combining both key words, has not become a central discipline in most computer science departments.  In many respects, this discipline embodies the same ideosyncracies that can be observed within computer science as a whole such as:

• Highly innovative and rapidly changing field with no broadly recognised core of material that every practitioner must know; • Few results are supported by empirical or comparative studies; • Work within the field older than 3–4 years is rarely acknowledged or referenced; • Old problems are given new names and old solutions overlooked; • Evolution of the discipline is tightly coupled to economic and societal demands; • There is a need for interdisciplinary work comprising e.g. mathematics, psychology, business or management science, … ; • Continuing debate about whether there should be a discipline called software engineering, and if so, whether this should be treated as another discipline among the set of traditional engineering disciplines."

As you read through this I think we can start to see a clear delineation between the disciplines required of 'Engineering' and the disciplines that Software Development requires (Note - This is not an article to criticize people who write software, but rather a way for us to better understand how to manage them and how to help them deliver even better software)

Go to almost any software development team and you will find a mix of people who have ended up going down the path of writing software, they can be people who were Computer Science majors all the way over to people with a Liberal Arts degrees.

The languages that allow us to write software today have matured over the years and as such have become easier to learn and work with.   This mix of skills and backgrounds is not bad, but it deviates from what an Engineering team building an airplane might look like.  Full transparency here, I have never worked in an organization that does true Engineering, but I know many people who do, and I suspect that you would not find people who are designing/engineering bridges or airplanes who don't have a strong educational background in Engineering, but you will find these people in your software development teams.

So why is 'Software Engineering' more art than science?

I think that Software Development is more art than science because we must create software that we interact with on a more human basis.  Creativity abounds in web development as we continue to strive to bring intuitive and easy to use interfaces to the people who are our 'users'.  The people who write software must be able to synthesize multiple layers of code, from Front End to Back end and everything in between. This talent is probably more aligned to writing a symphony in that you have to deal with an entire array of players, each with different instruments (API, Legacy, etc..) and when you get it right you get beautiful music, when you get it wrong you get noise.

When writing software we have the option to try multiple paths to an end, whereas when building a bridge our ability to deviate from proven engineering practices success is much more limiting and catastrophic when we do when we don't.

One can also argue that the newer languages being created are designed to make writing software more accessible to more and more people who have not be through a computer science programs, we are striving to get our languages to ever more English like syntax.

Consider also that people who write software are not held to specific engineering practices.  Software, as it has evolved, has provided people who 'develop' software product with endless ways to implement it the underlying code.  The fact that there are not any set standards that every Software Engineer must use I think clearly shows that software is not a true engineering discipline, but rather a creative one.  Artists embellish, they innovate, the push boundaries into new ways of performing....I think this better describes many of our software developers more than engineering.

The beauty of current software development is that you have the ability to envision new ways of implementing code that delivers functionality and in doing so create new coding processes that others can use.  Software languages provide many different ways to write code to deliver a product functionality, that might be 3 lines of code or it could be 100, it depends on the background and experience of the individual writing the code.

I think when we start thinking about how to manage and support our Product Development teams in our organizations we would benefit from providing them the framework that encourages high quality through innovation, failing fast, and supports both the individual and groups ability to determine their success.  That is not to say that Sr. Leadership doesn't play a part here, it certainly does, by providing clear vision of what is wanted (think of a symphony conductor, who must bring out greatness while providing their interpretation of a particular piece), an ability to accept failure as part of success and a fearless ability to have confidence that when you provide the other two components people will bring greatness as a by-product.

Artists operate in a very Agile manner, we listen and feed off of each other and in the process discover new ways to approach our art. Listen to jazz music and you see the elements of Agile and creativity evident as the small group (Scrum team) takes a song and through listening to each other can create new music, sounds or approaches to an old song.

As with Artists, Engineers don't like to be told something can't be done, that quest to make something happen that has never existed before is what an artist strives for, there is an emotional rush that comes with artistic creation and I think the same is true of software product creation.

Engineers who build skyscrapers, bridges and planes are no less artistic or innovative in their designs however they must take a much more rigorous approach to what they deliver, they simply don't have the ability to go back and redo it.  If they get it wrong people die, though there is of course software that is also mission critical, but for the vast majority of us who develop software for a living, we are able to take a less rigorous approach when we create our products and we expect in Agile to move to a continuous delivery model so our work is never 'done'.

Rear View Mirrors aka Mater Vision (Retrospective Habits for Agile Teams)

250px-MaterCars One of my favorite movies is Cars and specifically the character Mater, rusty and crusty but full of humorous and unintended sage advice.

During one point in the movie, Mater starts driving in reverse, on the road, off the road and Lightening McQueen is very impressed and asks Mater how he drives so well and his answer was, Rear View Mirrors to which he adds I don't need to know where I'm going if I already know where I been.

I started thinking about how this applies to high functioning Agile teams and their ability to provide predictability to their organizations.

More specifically I'm talking about how Retrospectives act as our rear view mirrors providing us with key visibility to where we were just at and an opportunity to reflect on where we want to go.

As an Agile team we constantly reflect (via Retrospectives) on where we were just at so that where we are going is incrementally improved, much like Mater and his rear view mirrors.  He's constantly reflecting on where he's been such that he's already where he needs to be before he gets there...

For organizations that are starting to adopt Agile ensuring that your teams utilize the Retrospective process is key to seeing the types of productivity improvements that Agile can and should deliver.

Retrospectives need to be both a no holds barred conversation about what did and did not work, but the team needs to ensure that it's also a safe place in which to talk about the issues that keep us from being really successful.  As a manager or management team you need to back away from the team and give them room to organize themselves.  As I've said before if a team feels empowered and is clear on the vision of the organization they are capable of solving both team and organizational issues that you didn't think were easily solved.

Traits and habits of good Retrospectives are actually very simple:

  1. Have it consistently after each iteration.
  2. Hold the Retrospective right after your demo while all of the iteration context and issues are fresh in your mind.
  3. Each Retrospective should start with review of any action items from the previous sprint.
    1. Scrum Master - Ensure that this happens as this is the key to having the team feel like the issues that are causing problems are actually being addressed.
    2. Scrum Team - You need to commit to addressing the action items identified in a Sprint to ensure that you are applying continuous improvement to your teams processes.
  4. Working Agreements - The team needs to have a set of agreements by which their Retrospective will be run, follow these agreements, which should include:
    1. Respect your team members - Allow them to call you out if necessary if you were in fact a blocker (I have been and no it's not nice to hear that, but your teammates are relying on you to make sure that they are successful.)
    2. Attack the problem not the person - This is key, you all want to be successful, don't take things personally, ask questions and come up with solutions that might work.
    3. Understand that not every idea you have will make things better, be prepared to fail.
  5. Keep the Retrospective to the members of the Team that are responsible for delivering the work (aka Individual Contributors).  Managers and Stakeholders should not attend.  As a Scrum Master you may have to tactfully address this if these individuals want to attend.
  6. Relax, we aren't trying to solve world peace.

Continuous Improvement leads to predictable velocity providing you with the ability to be like Mater.