Scrum Fundamentals – Iteration, Sprints, and Feedback

I said in the previous post that one of the biggest problems brought about by a waterfall approach is the lack of feedback.  The future users of the software get to see the software only when it is finished, when the opportunity to provide valuable corrections and suggestions are practically over.

Not only is it too  late (or too expensive) to modify the program, the software is often actually unusable.  Scrum directly addresses these problems by mandating that software must be developed iteratively, in iterations, and the at the end of each iteration, the software must be demonstrated to the stakeholders to get their feedback. 

Scrum calls its iterations Sprints.  Scrum recommends that Sprints be between 2 to 4 weeks.  Whatever duration you choose, all the Sprint must be always the same duration.  If you go for 2 weeks, then all the Sprints must each be 2 weeks. 

This is in the ideal case of course, in practice a Scrum team that is just beginning to learn the ropes can choose a tentative duration to try out (say 2 weeks).  If the team finds that 2 weeks is too short, they are free to adjust it (and all future Sprints).  If they find that it suits them, they stick to it.

The key thing is to establish Sprints as a regular occurrence, like a heartbeat.  Sprints of different lengths are like arrhythmia, with all the unpleasantness it brings.

The completion of each Sprint produces a finished subset of the product.  The ideal is to deliver a working subset, preferably something that can be shipped.  In practice this is not so easy to achieve, for some functions rely on other functions which may not fit in the current Sprint. 

For example, an online store may include such functions as allowing the customer to search for products, order products, apply discounts, and have them shipped to an address of their choosing.   Very often all of these functions cannot be built within a single Sprint. So which ones to build in order to product a shippable subset?

The Scrum team (together with the Product Owner) decides which functions to build in order to deliver working software.  One such option could be to simply include only the search for products function.  So at the end of the Sprint you end up with software that searches for products, and does nothing else.  That is an acceptable choice.

It is far better than deciding to build all the user interfaces for all the above functions, but not have them work.  If you do this, what you get at the end of the Sprint is not working software, rather, you end up with a glorified series of screenshots.

Back to feedback.  At the end of each Sprint is a Demo of what was built.  The Scrum teams demonstrates the current product so that the Product Owner (and other stakeholders) can review it and give their feedback.   Is this what was envisioned?   If it was, was it what you wanted?  Very often, once we see the the incarnation of what we said we wanted, we come to the realisation that that is not really what we wanted.

The purpose of the Demo is NOT to show the Product Owner that the Scrum Team is working hard.  No.  The purpose is only feedback regarding the product.  (There is a time when evaluation of the Scrum Team occurs, but this is not that time, and I will talk about it later).

If there is one thing that feedback causes, it is change.  You can be sure that the feedback will result in changes, changes to what the software will do, and changes to what it is currently doing.

So what do we do with the changes?  I’ll talk about that in the next post.

Scrum Fundamentals – Iterations, Sprints, and Feedback

I said in the previous post that one of the biggest problems brought about by a waterfall approach is the lack of feedback.  The future users of the software get to see the software only when it is finished, when the opportunity to provide valuable corrections and suggestions are practically over.

Not only is it too  late (or too expensive) to modify the program by then, the resulting software is sometimes not only bad, but even unusable. 

Scrum directly addresses these problems by mandating that software must be developed iteratively, in iterations, and the at the end of each iteration, the software must be demonstrated to the stakeholders to get their feedback. 

Scrum calls its iterations Sprints.  Scrum recommends that Sprints be between 2 to 4 weeks.  Whatever duration you choose, all the Sprint must be always the same duration.  If you go for 2 weeks, then all the Sprints must each be 2 weeks. 

This is in the ideal case of course. In practice a Scrum team that is just beginning to learn the ropes can choose a tentative duration to try out (say 2 weeks).  If the team finds that 2 weeks is too short, they are free to adjust it (and all future Sprints).  If they find that it suits them, they stick to it.

The key thing is to establish Sprints as a regular occurrence, like a heartbeat.  Sprints of different lengths are like arrhythmia, with all the unpleasantness it brings.

The completion of each Sprint produces a finished subset of the product.  The ideal is to deliver a working subset, preferably something that can be shipped.  In practice this is not so easy to achieve, for some functions rely on other functions which may not fit in the current Sprint. 

For example, an online store may include such functions as allowing the customer to search for products, order products, apply discounts, and have them shipped to an address of their choosing.   Very often all of these functions cannot be built within a single Sprint. So which ones to build in order to product a shippable subset?

The Scrum team (together with the Product Owner) decides which functions to build in order to deliver working software.  One such option could be to simply include only the search for products function.  So at the end of the Sprint you end up with software that searches for products, and does nothing else.  That is an acceptable choice.

It is far better than deciding to build all the user interfaces for all the above functions, but not have them work.  If you do this, what you get at the end of the Sprint is not working software, rather, you end up with a glorified series of screenshots.

Back to feedback.  At the end of each Sprint is a Demo of what was built.  The Scrum teams demonstrates the current product so that the Product Owner (and other stakeholders) can review it and give their feedback.   Is this what was envisioned?   If it was, was it what you wanted?  Very often, once we see the the incarnation of what we said we wanted, we come to the realisation that that is not really what we wanted.

The purpose of the Demo is absolutely NOT to show the Product Owner that the Scrum Team is working hard.  No.  The purpose is only feedback regarding the product.  (There is a time when evaluation of the Scrum Team occurs, but this is not that time, and I will talk about it later).

If there is one thing that feedback causes, it is change.  You can be sure that the feedback will result in changes, changes to what the software will do, and changes to what it is currently doing.

So what do we do with the changes?  I’ll talk about that in the next post.

Scrum Fundamentals

Stripped to its essence, Scrum is an approach to delivering software.  Nothing more, nothing less.

There are many approaches to developing software.   At one end, you can have a very formal ‘waterfall’ style approach where the development lifecycle consists of major phases that follow one another.  These phases include the Requirements Phase, where the project analyses and documents what the stakeholders need.  The next phase would be the design phase, where designers design the system to be built, which once built will deliver the requirements identified in the Requirements Phase.

After the Design Phase comes the Build Phase.  This is where the actual product get built, based on the specifications produced from the Design Phase.  Once built, the product gets tested in the Test Phase, in order to ensure the quality of the built product.

The phases in the waterfall approach are very logically  structured.  Each phase produces exactly what the next phase requires.  From arguments based on reason alone, it seems to be the perfect approach. 

In reality, however, the above approach is riddled with inefficiencies and problems, not due to the phases themselves, but because they are performed by fallible human beings, using fallible communication methods.  

Perhaps the most commonly cited problem is that very often, the product that comes out at the end meets the specifications, which in turn meets the requirements, but somehow ends up not being what the end users wanted or expected.

The lack of feedback between the stakeholders and the product being built – as it’s being built – contributes in no small part, to the problem.

Agile, and in particular, Scrum, proposes to fix the above problem by increasing the visibility of the product being built, while it’s being built.

We can illustrate Scrum at a very high level in the IDEF0 format as shown.

The output of a Scrum process is working software.  The inputs are a vision of the product, and of course time and money.

The process is executed by the Scrum Team, who relies heavily on the Product Owner, who represents the business stakeholders and from whom comes the Vision of the product.

Scrum comes with a set of techniques and procedures, and guided by the principles espoused in the Agile Manifesto.  You execute Scrum using these techniques.  In theory, you get working software, and the software that is guaranteed to be what the customer wants.

How does Scrum do that?  We’ll start exploring beginning with the next post on this topic.

Investor Directed Portfolio Services

What is an IDPS?   An IDPS is a service offered by a service provider (called an  operator) to an investor, wherein the operator arranges execution of buy and sell orders from the investor, custody of assets held, and provides reporting services to the investor.

The IDPS provides a selection of investment options to the investor.   The key distinguishing feature of this service is that the investor is responsible for making investment decisions.  The IDPS provider does not provide investment advise. 

An example of an IDPS is www.schwab.com where individuals can buy and trade shares and are charged a commission on trades.

Member discretionary master funds and wrap accounts are also examples of IDPSs.

In Australia, only public companies who hold a securities dealers licence may operate an IDPS.

The IDPS is a vehicle for allowing investors access to investment opportunities.   An IDPS operator must ensure that investors using their IDPS have access to the same information available to those investing directly in an investment.  For example, if a public company or a fund issues notices to shareholders,  the operator must ensure that any investor in their database who have invested in that company or fund through that IDPS receive the same information.

Due to this requirement, an IDPS operator must have an arrangement with securities issuers to ensure they (the operator) receive the disclosures.

An IDPS operator must also provide their customer with quarterly reports of transactions and holdings of those customers.  Besides the quarterly reports, customer must also have access to such information on a continuing basis.

Investments made through an IDPS will be put in custody by a party other than the investor.  The operator of the IDPS does not have to be the same entity keeping the securities under custody.

Because an IDPS offers savings to the investor through economies of scale, it is considered an investment scheme.  The economies of scale include pooling of funds from other investors for transaction netting (this is a mechanism by the volume and size of transactions are reduced by identifying transactions which ‘cancel’ out each other)

Schedule driven projects

How many times have you been involved in programs where the most pressing concern of everyone is meeting the schedule? 

Quality (fitness for purpose in this case) is the first to go.  To hell with quality, we need to get this deliverable --- a document, a piece of software, testing, etc. – finished.

Project status reports often show green if the everything is ‘on schedule’.  There is little status reporting on whether things are being done right.

I was involved in a large multi-million dollar program a few of years ago.  Immediately upon joining the program, I saw how badly the system being built will turn out to be.  For example, data input is being designed without consideration of what output is required (how would we know what input is required if we don’t know what output is required?).

I raised and itemised my concerns, but they were brushed off.  The schedule pressure was of paramount importance.

The result was predictable.  It was a very successful program:

• The program came in on schedule (the nth update of the schedule)
• The program came in under budget (the nth adjustment of the budget of course)

Except for the minor fact that:

• The delivered information system was unusable and unused.
• A second remediation program had to be launched two years later to fix the problems of the successful program.

This second program did give long employment to a second set of consultants and employees.

Project Failure

Below is a list of some reasons why projects fail, taken from David Cleland’s chapter ‘Strategic Management’ in “The Field Guide to Project Management” (highlights mine)

Check the list to make sure you are not experiencing any of these in your current project.  If you are, then it’s time to take action to amend the situation!

• Inadequate senior management oversight
• Ineffective planning
• Inappropriate organizational design
• Lack of well-defined and delegated authority and responsibility
• Inefficient system for monitoring, evaluating, and controlling the use
  of resources on the project
• Ineffective contingency planning
• Limited team member participation in the making and execution of decisions on the project
• Unrealistic cost and schedule objectives
• Lack of customer commitment to project
• Limited customer oversight
• Inadequate management information system

Politics and nerds

Skill in organizational politics is a function of social skills. The more you have of one, the more you have of the other. Computer nerds are reputed to have a distaste for organizational politics. The word 'nerd' is often used as if the word itself meant 'someone who lacks social skills'.

It takes, however, an exceptionally analytical and logical mind to become a good programmer. In other words, you have to be rather smart to be a good programmer. So how come that smartness seem to flounder in the face of organizational politics?

Some may say that the smartness that nerds possess is of a different sort from the smartness required to engage in politics.  Perhaps.  There are indeed different kinds of smarts.  But I'm more convinced that their smartness is just not focused on politics.

Nerds simply detest politics. Such an attitude comes out of the pureness of heart mixed with  a little naiveté.  They see politics as an unnecessary evil played by people who need to cover up for their incompetence, or by people who see it as a tool to feed their greed. One of the early definitions of politics that roamed the internet was a play on the word itself ("poli" - many, "tics" - blood sucking parasites).

Besides the attitude toward politics, nerds are also unprepared to manage things that fight back. They deal extremely well with computers and software, which respond with a deterministic response to stimulus.  In other word, computers act the same way each time.  Any difference in reaction triggers deeper analysis and debugging to understand why the unexpected variation happened, and what needs to be done to remove that variation.

Politics is about managing people. And people are deterministically unpredictable. The same request to the same person will receive a different response each time. People tend to not want to be managed and tend to want to manage. The nerd is afraid that the engaged person may attempt a coup and end up managing the nerd. Such do not happen in a nerd cum machine interaction.

The pervasive view that politics is only for the incompetent is unfortunate because it gives the viewer an unnecessary disability.

Politics is a part of organizational life. It is an essential part which cannot be removed. Anytime two or more people get together, there is going to be politics. Even if these two people happen to be the most considerate of sweethearts so madly in love with each other. She will do what she needs to do to make sure his eyes do not flit to another. In other words, she will attempt to control his behaviour. She will attempt to manage.

Politics is so pervasive that it is just unavoidable. It's like air. No matter where you sit in the totem pole you are within its reach. The lowliest messenger has to conform to politics. If you're not in the totem pole, you are not in any organization. You are either unemployed, or a Warren Buffet.

While politics exist at the bottom and the middle, at the top it is even more acute, and the stakes higher.  The politics at the top is even more pervasive. Even the highest officer in the land has to contend with politics. The US president may have the most powerful military on earth, but his power over it is limited. Even he has to play games of give and take with senators and congressmen. He also has to engage the same games with countries ranging from the big ones like Russia and China, to small ones like Venezuela, and even with close allies such as Japan, Australia, and the UK.

Politics is as unavoidable and as necessary as grooming. Unfortunately, many nerds also tend to dislike grooming -- some cats have better grooming skills.

If you cannot avoid it, join it.

The first step is to embrace a new attitude toward politics. Politics is like the Marvel comics character ‘Galactus’. It is a "force of nature." Politics is neither immoral nor moral. It is to be likened to grooming, manners, eating from plates instead of cans, and common courtesy -- society expects them and woe to those who do not conform.

There is no free lunch.  There is no getting ahead without politics.