CMSI 543 / SYEG 557: Welcome to Week 02

This Week's Class Agenda

• Introduction
• Can't talk about Agile without talking about Waterfall
• Can't talk about Agile without talking about Software Engineering
• Software Development and Engineering Methodology
• Agile versus Waterfall ["Water-fail"?]
• Cayman Design
• Organizational Cultures

OK — Let's start at the beginning!…

Back in the good ol' days… In the early days of software, it used to be written by hand, and without much of an idea to issues of maintenance, deployment, bug fixes, organization, or anything else, really, except getting the software to work. If you read through the 1968 work by Don Knuth, in a series of books called The Art of Computer Programming, people started applying the idea of discipline to the whole software development process. Later, Fred Brooks' book The Mythical Man Month [Amazon link] gave us a documented experience of one of the first major, large-scale software engineering projects. It rapidly became apparent that just applying discipline was not enough; that process needed to include concepts of engineering like system architecture, formalized design processes, testing before release, and building for maintenance.

Just what IS Agile, anywho!?

All this engineering was modeled on the traditional approach of building hardware, which is done in an exceedingly linear fashion. By linear, I mean that there were distinct sets of steps that were followed from conception to delivery and beyond. The following diagram gives you an idea of what I mean by linear:

What this ended up causing is some provably difficult problems with the entire engineering process, particularly for large-scale development projects:

• Steps were intended to be discrete in nature, and were executed in a step-wise fashion
• Estimating the total cost was hard to do
• Estimating the total schedule was hard to do
• The customer had very little visibility into the entire process
• Planning was done one time and the resulting plan was rarely re-visited or updated
• Management got locked in on the idea that this is the proper way to organize projects
• There is a significant focus on documentation, ostensibly to facilitate communication
• Quick Quiz: Can you think of other things that are wrong with this?

Another way to look at this process [straightening out the Vee] is as a waterfall, in which the flow of the activities moves consistently downward from the inception to the conclusion, like this:

Reasons why the waterfall model may fail:

• Real projects don't seem to follow the sequential flow; changes can cause confusion
• Often difficult for the customer to state all requirements explicitly at the start of the project; the process has difficulty adapting to later requirements changes which are due to this natural uncertainty
• The customer must be patient since a working version of the program won't be delivered until near the end of the process
• A major design blunder can be disastrous; it will be expensive to fix, and can cause extensive delays.

So…… to help resolve some of these issues

With the arrival of rapidly changing software for things like the Internet and the World Wide Web, the tradional Waterfall Model didn't really work so well. Businesses needed more fleixibility to be able to respond to markets, and they needed the agility to be able to adapt and be flexible in the face of changes that might occur during product development. They also found that the heavyweight documentation required by Waterfall meant that teams of people had to spend lots of time and energy [and money of course] keeping all that documentation up to date so that it accurately reflected the actual product.

AGILE TO THE RESCUE!

Agile is an overarching term that include iterative approaches to software development the 'Manifesto for Agile Software Development'[Ashmore p.2]. Agile is designed to focus more on customer needs and providing business value than on documentation. It uses the concept of short iteration and producing working software with each pass through the process, as well as constant customer feedback and involvement. Thus, Agile helps organizations to focus on efficiency, collaboration, quality, and the creation of customer value. [Ashmore, p. 3]

What is the basic theory that it started with?

The basic idea is to take the best parts of the Waterfall process, namely the parts that EVERY software development project MUST perform, and shorten them up, performing the resulting cycle over and over again. With each iteration [sometimes called a sprint] the team adds more functionality to the product, and maintains a working software application through the entire development effort.

Notice that in this spiral model there are five main activities that happen again and again, within each iteration. That means with every pass through the process the team will be planning, designing, building, testing, and deploying a working software application. Notice also that these five activities are the CORE of the software development process, regardless of what methodology [or life cycle model, in the parlance of our times] is being used. These actions can be formally described as follows:

• Requirements Analysis often encompasses a very top-level design phase called conceptual design. This may include trade studies of which technologies to use, risk assessments, software tools, and so forth.
• Design frequently becomes conceptual design, preliminary design and critical or detailed design; the preliminary design may require scenarios or storyboards which can be done easily using UML [and other types of] diagrams
• Design Phases usually end with a design review. Design reviews are major milestone meetings which are attended by representatives from all interested parties, who are sometimes referred to as stakeholders. For a large project, it may be a week's worth of all-day meetings. They are kind of cool, since the companies usually go all out and have special conference rooms, computer hookups, presentation technology, and of course free food. There may be two or three of these during the design phase(s) of the project:
  • CoDR: Conceptual Design Review — this is the point at which the customer can review the result of trade studies, technology reviews, preliminary experimentation and prototyping, and so on, so the solution provider can demonstrate that they know what they are doing and can provide a product that solves the problem
  • PDR: Preliminary Design Review — this review is the point at which the solution provider wants to show the customer that there is a good design in progress, that addresses the requirements in the requirements document, with traceability to each requirement, and with a verification plan that shows what tests verify what requirement(s). If changes are required, it's probably not too big a deal if it is caught at this time.
  • CDR: Critical Design Review — this is the point at which the design had better be complete. It is the last chance for the solution provider to show the customer that what they are building fills the bill. If there are changes, they can still be made, but it is likely going to cause schedule slips (moving to the right) which is not a good thing. Schedule slips cost $$$$$, and also may cause the customer to view the solution provider in a somewhat less than favorable light. They may not want to work with you again…
• Construction/Code is sometimes broken into coding and debugging (which frequently incorporates preliminary unit testing)
• Test is usually unit testing and integration testing, and may also include user (alpha and/or beta) testing
• Deployment/Maintenance is the phase at which the software is released and controlled, with versions and builds, hot patches, service packs, and so on. There is usually only a skeleton crew assigned to the project by this time, which tracks the changes, makes and tests modifications, and builds and releases new versions. If this is a product for sale on the open market, there is usually some sort of production associated with this phase. There may be an exit strategy, or maybe not, depending on if the company wants to stay in that business (similar to Microsoft with Office or Windows).

Some interesting things to be aware of:

• 20-80 rule: 20% of development time is needed to design the basic system, and 80% of the development time is needed to ensure data integrity, security, and robustness.
• 90 percent syndrome: The first 90% of development takes the first 90% of the development time, and the last 10% takes the other 90% of the development time.
• Frederic Brooks, author of The Mythical Man-Month, says:

  The hardest single part of building a software system is deciding precisely what to build. No other part of the conceptual work is as difficult as establishing the detailed technical requirements, including all the interfaces to people, to machines, and to other software systems. No other part of the work so cripples the resulting system if done wrong. No other part is more difficult to rectify later.

To help you keep it straight……

Here is a table that shows the main features of the two software approaches, taken from [Ashmore p. 3]:

Waterfall	Agile
Prescriptive	Abstract
Extensive documentation	Minimal documentation
Sequential	Continuous
Formal	Informal
Process focus	Communication focus
Gradual change	Rapid change

The Nuts and the Bolts…

So what do those table entries actually mean? From the Ashmore book [Pilone p. 6]:

• Individuals and interactions over processes and tools: working together face-to-face helps reduce communication problems. Communication is a KEY element in software development, since so much is riding on the ability of team members to precisely understand each other. Trying to make a team stick to a process can wind up in form over function situations which can stifle productivity and often don't foster the trust needed to build and maintain a cohesive team.
• Working software over comprehensive documentation: Documentation is static and often only presents a single view of the project. Further, with heavyweight documentation, time and money must be spent to keep it up to date, so that it is descriptive and not just prescriptive. By using tools like TDD and Continuous Integration, you can present your customer with working software that they can try out, and that will allow them to provide you with constructive feedback on a regular short-term basis, that can be used to insure your team is producing what the customer determines is providing them business value.
• Customer collaboration over contract negotiation: One of the worst parts of a software development project is trying to estimate how long the project will take and how much money it will cost. This is difficult enough to do if nothing changes; however, during the development the market is rarely static, so the customer will want to change or add things while the software is being built. This means the team will need the ability to shift, change, even pivot at a moment's notice. This situation is NOT a bad thing, unless the team is unable to move in an agile way. A formal written contract can act as an inhibitor to that flexibility, and can add delays that might even make the product obsolete before it is even completed, if the contract changes take too long.
• Responding to change over following a plan: Since these changes are a GOOD thing, it is important to be able to respond to them without worrying about whether they are a part of some development plan or not. If the customer needs to change something, there is usually a good reason. If everyone is locked to a plan, there is much less ability to adapt to changes, even if they are good changes. This doesn't mean you DON'T have a plan, it simply means the implementation of the plan must be flexible enough so that the team can adapt easily and readily.
• A possible addition to the four goals: Software should be developed in a way that is ethical. There are quite a few remifications to this statement, and we will explore them in more detail as the semester rolls on. However, in a nutshell, we need to be able to say to each other and to our customers, I have an ethical problem with this idea or this application, and trust that the team and/or the customer will at least have the courage and reciprocal trust to hear out the rationale before taking any action.

Products

There are a number of applications [quite a large number, actually] that can help you manage an Agile project. No matter which flavor of Agile you choose, there are several very good tools that you can choose from. The ones listed below have the added advantage of being web-based and best of all they are FREE. The following list is a small selection; if you have a favorite that is not listed here, feel free to tell me about it and I will add it:

Scrum Tools

• Scrumblr
• PlanIT Poker
• Trello

Kanban Tools
• ClickUp
• Kanban Flow
• Pipefy

Other Generic Tools
• ZenHub
• Taiga

The Cayman Design Case Study

Your Ashmore textbook uses a running case study to provide examples of all the concepts discussed in presenting the material. As can be seen from the explanation on page 8 of the text, the Cayman Design Company is a made-up entity. However, there are a number of good examples that are made using this fiction.

The company is a 10-year-old web design company that is headquartered in Austin, TX, with a branch development team in India. They develop for both computer and mobile applications. Their management team has decided to start using Agile as part of their expansion into mobile applications. Their development teams are a mix of new hires and experienced professionals. To facilitate the move to Agile, Cayman hired a consultant to train the staff, implementing Scrum, Kanban boards, and XP. They also use video conferencing, e-mail, and instant messaging for communicating between and among teams. There are currently an average of six to eight Scrum teams.

Here are some highlights with page numbers from the text so you can read about them:

• Page 31: how testing was handled for the number of teams that needed it
• Page 34: how to keep managers from interfering in the process
• Page 66: Cayman's tool implementation and techniques
• Page 75: Cayman's initial weather application Minimum Viable Product [MVP]
• Page 116: Handling new requests for additional functionality
• Page 122: Some sample user stories for their weather application
• Page 126: Some example 'personas' for user stories
• Page 130: Managing the initial features for the product launch
• Page 133: Design decisions, data storage, and the effects on report creation
• Page 138: Concrete illustration of the MVP selection decision
• Page 154: Applying 'MoSCoW' to the Cayman weather application
• Page 183: Test Driven Development [TDD] and Refactoring
• Page 213: Application of Kanban to include new functions on existing product
• Page 242: Using surveys to gather customer feedback about potential new features
• Page 249: Managing features in a product release with continuous prduct enhancements

Coming up…

Next week we'll look into things a bit more deeply, starting with the different ways of looking at an organization, to see some of the different ways that the people IN that organization may view the processes and activities of software development in general, and Agile in particular. This will lead us [in future weeks] to the different roles that are required for an Agile development effort.