How Changing our Estimation Process Took our Project Endgame from WTF? to FTW!

Recently I had the pleasure of presenting a session at Midcamp Chicago entitled “How Changing our Estimation Process Took our Project Endgame from WTF? to FTW!”.  Perhaps you weren’t able to attend Midcamp, or maybe you were in attendance but decided to go to a different session (why would you do that?!), whatever the reason, here’s what you missed:

Wouldn’t it be nice if the majority of clients didn’t need to worry about the overall project budget and would just let your team work their development magic and build whatever they need? It would be great, but it certainly isn’t realistic. The reality is that most clients are working with a budget for their project and therefore you are too. This often translates to fixed bid projects that are awarded after an RFP process brings in proposals from a variety of potential vendors.  

When you receive an RFP, it frequently includes a laundry list of features that the client would like to have within the allotted budget.  So, you review the objectives in the RFP, you meet with the client to discuss and ask any questions you may have, and then convene internally to develop a game plan as to how you will tackle the project.  You write up your recommendations in your proposal and of course, you include a budget. But at this point, what is the budget really besides a total and complete wild guess? It is a beautiful budget, and you’ve worked very hard at putting it together for your potential client, but this early in the project it is based entirely on assumptions.  

What the client wants to see here is that you can meet their stated objectives within their defined budget, so they’re looking at the grand total and not always at the budget details.  So this means that the estimate needs to be updated as your project evolves.

We didn’t always revise the estimate as a project evolved, and we learned the hard way that not doing so gives clients unrealistic expectations of what they will be getting within their fixed budget.  The problem is that there is a rather chasmic difference between a bullet list of features in an RFP and complete specifications with interaction requirements and wireframes.

The same feature listed quickly in an RFP can require anywhere from 10-100 hours to build, depending on how it is spec’d out.  When the client writes their RFP and when you prepare your initial budget, the actual specifications for the features are unknown.  The actual scope of the features only becomes clear once you have conducted a discovery process and put together wireframes and interaction requirements.  We realized that in order to better manage client expectations and project budget on fixed bids, we needed to do two things:

• Integrate two new steps in our process in the form of estimate revisions at the end of the discovery and design phase as project milestones.
• Be transparent with our clients, telling them early and often that their budget will only cover so many features and it is their responsibility to prioritize and determine which features should be part of the Minimal Viable Product (or the basic, necessary features your client needs to launch their site). 

Our redefined process involves several steps:

Kick off Meeting 

It’s important to begin managing client expectations early on. These meetings are typically pretty informal discussions where we ask the client to describe their objectives in their own words.  We then review our process, the different project milestones, the budget review and make sure the client is aware that decisions made in terms of prioritization in the discovery phase will effect where their budget is spent when it comes time to build.

UX Sketches

After the kick off meeting, our UX designer meets with the client to get a handle on their priorities and vision for the site. After this meeting, he works to develop some preliminary sketches and we present these to the client.  The sketches are then refined based on client feedback in a rapid iterative design process.  We try to iterate quickly on the sketches and get multiple rounds of feedback until we come to a point where the client is satisfied with what we’ve put on paper.  With approved sketches in hand, we are ready for our next milestone.  

Early Technical Planning 

Our Early Tech Planning meetings typically involve the tech lead, UX designer and PM.  The UX designer reviews the sketches with the tech lead and the PM adds tickets to our project management system that cover the big pieces of work to be done.  The Tech lead determines a rough estimate for each ticket, at this point the features aren’t defined enough to truly decide how any specific feature will built.  The end goal is to determine an overall budget with a 40% accuracy rate. The idea here is to see where we stand in terms of development hours. We want to identify any key bits of functionality or features that may be taking a disproportionate amount of budget and see if that aligns with client priorities.  If not, we still have time to modify the sketches and upcoming wires accordingly and we haven’t lost any dev time building the features so there is no harm done.

Wireframes 

After our Early Tech Planning client review, we continue refining the UX sketches with the client until we come to a point where we have complete and approved wireframes and interaction requirements.  This is really the key to being able to complete the next step in our process - the Final Tech Planning.

Final Technical Planning

The Final Tech Planning is a complete review of the wires and interaction requirements in which we determine the information architecture, take implementation notes and estimate all the features with the goal of achieving a 10% accuracy rate.  We plan several meetings over the course of a week, each meeting is about 2 hours.  

The total number of meetings needed depends on the project complexity.  We include the tech lead, two developers, a themer, the UX designer and the PM.  We start with the list of issues that we added in our Early Tech Planning, we select a site section and the UX designer reviews the wireframes and interaction requirements with the team. We do not share the previous estimates with the dev team. Lots of discussion ensues as the developers decide upon how they will build what they see in the wires.

These decisions are written down in the form of implementation notes to be added to the individual tickets after the meetings.  As the developers define the best way to build what’s in front of them, they come to a point where they are ready to estimate the work at hand.  Each developer gives their number and we generally take the high end of average and this estimate is added to the ticket by the PM.

At the end of these long, intense, meetings, we tally up the estimates and see where we stand.  The PM then prepares a full feature list and hours breakdown that we present to the client.  If the total estimate is above the available budget, we also prepare our recommendations of  features that could potentially be de-scoped without sacrificing the project integrity.

There are several options that can come out of a situation in which the Final Tech Planning estimate is higher than the available hours:

1. We de-scope certain features/issues and save them for a later project phase: sometimes when confronted with budget restrictions a client will accept to wait to build certain features until they have further funding. 
2. Perhaps the client has an internal dev team that can take on some of the dev work and collaborate with our team: in this situation the development work becomes a co-build.  We determine which team will work on which issues and have weekly dev scrums to follow progress.  This allows the client to get more functionality without having to pay for additional development hours from our team. 
3. Perhaps there is additional budget to cover the hours: if the client doesn’t have an internal team to take on some of the work and they can not push any of the features back to a later phase of work, then perhaps they can find additional funding within their organization to cover the projected overage.

So, we share our recommendations of features that can potentially be de-scoped and give them other options of how the potential budget overage can be dealt with, this allows them to make the final decision.  The project scope and budget is in their hands. The client then takes some time to review and think through their options and feature list and then come back to us with a game plan.         

From Building to Wrapping the Project

Once the client has approved the feature list and everyone is in agreement as to the final scope, you can begin building.  This part of the project should go quite smoothly since you have done so much tech planning work already.  The dev team has everything they need to get started in the tickets, and the tech lead and PM can easily plan sprints since the estimates and dependencies are all laid out.  As you wrap the project you should see that an overall estimate accuracy - +/-10% holds true.  

With this process and the integrated client expectation readjustments, you should have a happy client who got what they wanted, within their budget and was informed throughout the process.  You have consulted with your client multiple times throughout the project, getting them to decide where they want their budget to be spent.  In our experience, clients truly appreciate this empowerment, and this transparency.  It makes them active project members, they haven’t just given us some money and a list of things and said build this, they are making decisions and molding the project as it progresses.