How I’d Fix the RFP Process for Buying Software
24 Aug 2010
The other day Ron Schmeltzer tweeted how broken the RFP process is for buying software and his opinion of the process (he dislikes it). [Correction: Ron was talking about consulting RFP’s, so the post below doesn’t really matter. But, it did get me off my bum to write the post below, something I’ve had in my head for quite some time.] I quickly responded, asking him how he’d do it differently if he were a buyer. I have some thoughts on this, but was curious what he thought. Brett Miller responded with a blog post he wrote back in July as to the pros and cons of the RFP process.
In general, my opinion about Brett’s points are that the cons outweigh the pros in most every case, and that there are alternative ways to achieve the pros without invoking the cons.
I feel quite strongly that the RFP process for software purchasing is totally broken, and have an idea to replace it based on the work that some early founders of WebLayers did when I was selling Actional to them at Credit Suisse.
First let me explain why I think it’s broken. Then I’ll share my recommended fix.
1. RFP’s are biased. Typically, RFP’s are issued by companies after they’ve done some due diligence. That due diligence is “biased” based on who they spoke to, that bias finds its way into the form and function of the RFP. If all vendors have been involved prior to the RFP issue, that’s fine. But, if not… then the RFP is weighted towards those who have participated. And, that’s not always good for the consumer.
2. RPF’s only provide a partial view into what’s important. RFP’s often have hundreds of questions, some requiring complex answers. They’re meant to (1) get a complete comparison of relevant information, and (2) standardize the answers. Well, by the time a purchase is made and an implementation happens, the state of the various features will change, so knowing the current state isn’t necessarily helpful. I realize it provides a baseline, but that assumes that none of the vendors stretches the truth. Also, a simple question like “Do you support WS-Security” doesn’t have a simple answer, like “yes”. Usually, the answer is something between “no” and “sort of”… there are interoperability issues, minimum platform issues, which pieces of the standard are supported, and how the support is implemented. Second, standardized answers are not useful for a large portion of the questions… and in my opinion those are the important questions. What RPF writers really should want to understand are what makes each vendor unique, and how their philosophy around the solution aligns with the needs of the organization.
3. RPF responses are difficult to write, and even more difficult to evaluate. Finally, companies usually have a very short time to respond to an RFP making the responses less than the quality documents anyone would really like. I know it’s surprising to buyers, but the product information you would expect to be cut-and-paste is not often available. Even a prior RFP response that’s 3 months earlier is probably out of date. And, even the best “cut-and-paster” out there (I think I’m up there) is hard pressed to weave multiple cut-and-paste sources back together into a professional looking and consistent document. What about the review process? It’s time consuming and similarly biased. A grading system would certainly be unable to evaluate things like strategic alignment and uniques… and anything less is subject to the preferences of the reader… and what they happen to pick up when reading the responses. Try this. After all the responses have been read, put a simple 10 question list of features/capabilities in front of the readers… and ask them to match them to the vendors that wrote the answers. Do you think they’d remember which vendor wrote which answer?
One final point. The time/effort it takes for the teams to write the responses and the team to evaluate them all… isn’t there a better way to spend our collective time to get better technology out there faster and solve problems sooner?
So, what do I recommend?
Keep in mind that I’ve been almost exclusively at vendors/integrators in my career so admittedly I’m probably leaving some administrative/purchasing requirements out. However, I think the following makes for a great place to start.
1. Use analysts only to get a view of the landscape and make sure you know all the relevant vendors out there. Analysts don’t have the time to do much hands-on evaluation to validate what the vendors tell them. And, analysts have their own biases which may or may not align with your own. Save the analysts for when you have specific questions about relative vendor comparisons and market trends.
2. Along with a non-subjective checklist of standards and IT requirements (such as interoperability with existing systems/platforms, support in particular countries, number of SI’s trained in a product suite, etc.) deliver a set of use cases for how the product would be used. The use cases should include some long-term (and therefore less specific items) and some short term cases. The short term ones should really address the driving need for the evaluation. At least one use case should test performance and scalability in order to prove out the scaling model and help drive to a final configuration (and therefore a final project cost). Other use cases should include interoperability testing for integration to existing systems, and how the product gets migrated between development and production.
Slight aside. By checklist I mean there should be nowhere on this list to explain anything. Answers should be unambiguous lists, yes/no, dates, numbers, etc. This keeps it black-and-white. If it needs an explanation, it best to have the vendor answer in the context of the use case.
3. Ask vendor participants to fill in the checklist, give “essay” answers to the use cases, and then provide 10 items that they believe you should evaluate as part of the evaluation. These 10 items will be all you need to understand how each believes they compete with the other vendors, the vendors’ philosophical alignment to the problem space, and their unique value propositions. By the way, these 10 items should include use cases on how to test them, and an explanation of why they are important to the proposed solution. Of course, these 10 items might be non-technical… like they might be about standards support, or SI relationships, or whatever the vendor thinks is important.
I believe if we moved in this direction, we’d have a process that got customers what they need, faster, with higher quality results. And, the efforts used in the decision process (by implementing the use cases in a POC) would be directly relevant to deploying the solution, so once the process is complete, you’ve done more than selected a vendor, you’ve begun your implementation.
Aug 24, 2010 @ 17:01:11
Dave,
I’m actually a huge fan of RFPs. Of course, I’m predominately looking at it from the client perspective, not the vendor side. Let me give you my reasons why I love RFPs.
1. They force the client stakeholders to think about, write out, order and agree what the goals are for the project/software/etc. Companies will often have very vague reasons for wanting to do something and even less understanding of how to evaluate if its successful. Forcing executives to agree to and state the purpose of the exercise is critical. Believe it or not, RFPs are a great way of doing this, if for no other reason than the fact that stakeholders usually have to go to the CFO or Board to get approval to spend that kind of money. (Note: The longer I’ve worked in software, the more I’ve come to love CFOs.)
2. It provides a checklist that stakeholders have to agree to and then use to justify why they want to go with a particular solution. Many times on the client side, one can only get limited amounts of time from critical stakeholders. Whatever they profess, key stakeholders can rarely devote the type of time a major project needs.
First having the meeting to agree to the checklist and discuss the importance of different items is key from an education perspective. Then when it comes to vendor evaluation, which is often months later, the checklist provides structure from which to have discussions and make evaluations/decisions.
Lacking the structure of a checklist with which to evaluate multiple solutions, stakeholders will often latch onto the first solution they see that seems to address their needs. **please see note below.
3. Addressing asymmetries in understanding and product expertise. Vendors know infinitely more about the market space than prospective clients. Hearing different vendors answer the same questions is extremely educational for different stakeholders. For example, often times a stakeholder won’t understand some answers being given, but if three vendors answer a question in one way, while one vendor provides a completely different answer; one gets an opportunity to have real discussion about why.
4. Pricing. While price is rarely the deciding factor, having multiple bids is critical for getting a good price. Especially if the selection period covers the vendor’s year-end or quarter-end. Often times one vendor will drop their price to a ridiculous level because that salesperson needs the sale for some other reason.
There’s a great line from Warren Buffett: “You can’t be any smarter than the dumbest competitor you’re competing against.” When there are equivalent products and one vendor is willing to drop their price by 80%, even if you’d never select that vendor, every other vendor has to drop their price to an equivalent level.
Price has nothing to do with selecting who you’re going to go with, but having multiple bids has everything to do with how much you pay for the product you’ve selected.
I hope this helps you understand why I love RFPs so much.
**Note: I don’t think people working for vendors fully understand the importance of presenting first. Most often in client firms, there’s really only one way discussed to do something. So the first person who can suggest a viable way to do it gets to do it that way. So managers/executives are “trained” to go with whatever way they see that works first and move forward rapidly with it. There is a huge advantage to being the first vendor who presents.
Aug 24, 2010 @ 22:35:40
Andy,
Thanks for the detailed response. I certainly appreciate your perspective from a customer point of view.
However… I’m not advocating not putting something out to bid. Therefore, your comment about pricing doesn’t apply. I still think you’d have multiple bids, so you’d have the same effect of “keeping the vendors honest, and having a strong negotiating position”.
With respect to your other points… I like your point about #3, where if one vendor answers differently it’s a good place to probe. That’s kinda like how the Israelis do airline security and it works quite well (compared to our way). However, I’m not sure why all three of your points can’t be addressed by the use cases I mentioned. In fact, a use-case driven RFP would align much better with agile development methods (which I think can be used to deliver any IT project in the enterprise “better”). You’d have a practical way to measure ROI, you’d have your checklist for stakeholder agreement, and you’d still have a way to compare vendors… though it would be about how they’d solve the use cases (or what they’d add/delete from them), rather than if they support pieces of technology the RFP writer read about somewhere and thinks is relevant.
In fact, since customers understand less about technology and more about their own use cases, by focusing on documenting their use cases, they can properly learn from the vendors to make an educated decision.
What I find is that customers ask “technology” questions on RFP’s that “don’t make sense” or “don’t make sense without more context” and so the answers become somewhat non-sensical. I thinks vendors, by definition, understand technology better than customers, and so let the vendors talk technology, let customers corral the vendors into focusing on the use cases they’ve defined.
My turn for an aside… I in now way think customers don’t know anything about technology. Many are extremely knowledgeable, and in many ways more knowledgeable than vendors because they have real-world experience. Most presales folks don’t get so see how the software is actually used. However, in a broad sense, vendors will have a better understanding of the relevant technology issues and pros and cons. Each vendor will have their own, and the job of the evaluator is to aggregate all the different philosophies, and pick one that aligns with their own.
I hope I’m making sense.
In summary. I think the use cases I suggest meet your requirements better than a simple list of technology questions do. They hold vendors accountable to results, rather than just technology because they are committed to delivering use cases, not just product features. And, the work done in evaluation, when done towards use cases can be implemented right away (within protocol of migration to production) rather than the POC/evaluation being an abstract exercise after which the real work gets started.
I also think that asking the vendors what they think is important is a good way to learn about the space, the relative strengths and weaknesses of the products, and so on.
By the way, I like the idea of CFO’s too… it’s just that there’s one or two I know that have been real assholes.
And, while there are some good reasons to present first, presenting last also has some advantages. First vendor to do the POC is usually the one who helps the customer setup their POC environment, so it looks like they’ve had a harder time. Also, having more time to prepare by not going first can make you look polished. And, presenting last gives you the final word, and an opportunity to answer all the remaining objections because your competitors have fired all their ammo… and you’re left able to leave uncertainty about those vendors in your wake without those vendors always having an opportunity to respond (or to respond in the intimate way you do when doing a week long POC).
Aug 31, 2010 @ 05:47:33
David,
very interesting post — I myself created a new process working with Siemens years ago when working on some stuff for the BBC — having suffered the hugely drawn out RFP process that left the customer often more confused — we decided to take an approach we called a Request for Engagement (RFE).
What should be in an RFE?:
Limited page number response
Keep it tight and relevant — recommendation is for less than 40 pages
But if a vendor calls and asks for more, it’s usually for good reason and you should be flexible (within reason)
Force the vendor them to cut the marketing material — you can even state that inclusion of marketing material will be marked down
Give specific use cases and ask vendors to respond to these cases
Include specific failure use cases
Make your document modular and service based
Help the vendor to answer and allows you to share the document review with others
Place your scenarios/questions into a modular structure
Make sure that your structure is conducive to being split into sections for answer by the vendor
Divide the document into sections — example sections may include:
Introductions
Your Business Drivers
Your Technical Drivers
The Vendors Financial Information and Company details
Example Scenarios (suggested you include 3 minimum)
Value Added Services and Software
Support and Training
Example Pricing and your estimated time-lines
Risk Management
As this question:
What are your Unique selling points and why are they valuable to me?
Vendors often know more about the competitive landscape than you.
Move things forward with a Pilot Project, not a Proof of concept.
What every we all come up with, the RFP and associated process is old, out of date with modern approaches and very expensive for everyone. The BBC project in question? They had 44 vendors respond… They had to hire an SI just to read and collate the responses! The smallest response was 6 volumes in size…
Sep 11, 2010 @ 14:03:04
That example of the BBC is hilarious, and not surprising. I think that the cost most companies put into RFP’s should be measured against their ROI… then they’d think about a different way to do it. And, you make a key point of doing a pilot, not a POC. That certainly would improve ROI and time to market.
Thought Piece on Open Source Software
Apr 07, 2011 @ 17:20:02
[…] against each other in a proof-of-concept. A lot of time and money is wasted shamelessly. I wrote more about this elsewhere. [↩] My favorite analogy is that of the musician. You can teach someone to play the piano. […]