Tutorial 1: UNDERSTAND THE PROPOSAL EVALUATION CRITERIA

Back to Tutorials

Before writing a proposal you must first understand how it will be evaluated as the proposal that you submit, must be responsive to these criteria. All solicitations contain a section where the evaluation criteria are discussed - usually placed somewhere towards the end of the solicitation document.




One criterion that is ever present in all SBIR/STTR solicitations is “innovation”. For example, DoD’s first evaluation criterion is “the soundness, technical merit and innovation of the proposed approach…” HHS lists Innovation as an evaluation criterion and elaborates by asking “Does the application challenge and seek to shift current research or clinical practice paradigms by utilizing novel theoretical concepts, approaches or methodologies? The Department of Energy evaluates the Strength of the Scientific/Technical Approach as evidenced by the innovativeness of the idea and the approach, and the significance of the scientific or technical challenge. NSF does not use the word “innovation” but instead talks about Intellectual Merit which encompasses the potential to advance knowledge. NASA talks about plans for developing and verifying the innovation which must demonstrate a clear understanding of the problem and the current state of the art.


Given this criterion, in your proposal one item that you must address is innovation. But how will you know if your work is innovative? How can you assure that you are responsive to this criterion? Guidance is actually provided in the way innovation is discussed in the various solicitations. In order to demonstrate an innovation, you must show an understanding of the current state of the art. You must review the literature and contrast your approach with that of conventional wisdom.


For some applicants reviewing the literature and contrasting your approach with conventional wisdom is commonplace. However for others who have been practicing their art for a number of years and who are not from an academic environment this may be somewhat foreign. However, what is required is consistently mentioned in the solicitations - you must demonstrate a knowledge of the state of the art and contrast what you are proposing with that as the baseline. You cannot assume the reviewer’s knowledge of the literature – it is your responsibility to demonstrate your knowledge of the state of the art and clearly call out what is innovative about your approach.




Another evaluation criterion that is common across all SBIR/STTR programs is Experience, Qualifications and Facilities. DOE evaluates the “Ability to carry out the project in an efficient manner as evidenced by the qualifications of the Principal Investigator or PI, other key staff, subcontractors and consultants”. The National Science Foundation looks at how well qualified the individual, team or organization is and if adequate resources are available to the PI to carry out the proposed activities. DoD evaluates the qualifications of the proposed principal/key investigators, supporting staff and consultants. The Department of Health and Human Services evaluates if the Program Directors, also called PDs, the PIs, collaborators and other researchers are well suited to the project; while NASA evaluates the technical capabilities and experience of the PI, project manager, key personnel, staff, consultants and subcontractors, if any and evaluates for consistency with the research effort and their degree of commitment and availability.


This evaluation criterion means that you must place considerable emphasis on putting your team in place. It’s not enough that you believe that you have the capabilities to do X, you must be able to demonstrate that you CAN do X. The government is looking to minimize its risk and will look for teams that have a track record for delivering comparable services.



Another criterion that all agencies include relates to commercialization. In the HHS guidelines, you will find commercialization mentioned as part of the “Significance” criterion where it asks “Does the proposed project have commercial potential to lead to a marketable product, process or service? Does the Commercialization Plan demonstrate a high probability of commercialization? The Department of Defense evaluates “The potential for commercial applications (that is, Government or private sector) and the benefits expected to accrue from this commercialization.” With the Department of Energy you will find this under “Impact”. DOE evaluates the evidence of impact by looking at the likelihood that the proposed work could lead to a marketable product or process and the likelihood that the project could attract further development funding after the SBIR or STTR project ends. The fourth factor that NASA evaluates is called “Commercial Potential and Feasibility” stating that “The offeror’s experience and record in technology commercialization, co-funding commitments from private or non-SBIR/non-STTR funding sources, existing and projected commitments for Phase III funding…will be considered along with the initial commercialization strategy for the innovation.” NSF looks at the broader impact which encompasses the potential of the proposed project to benefit society and contribute to the achievement of specific, desired societal outcomes. “




By reading the evaluation criteria before you begin proposal preparation, you will understand those items that you must address thoroughly within your proposal. What has been mentioned here are only criteria that all participating programs evaluate – specifically the innovation of what you propose; your team; and attention to commercialization in a manner that is consistent with that agency’s mission. These are not all of the evaluation criteria, so be sure to find and review that section of the Funding Opportunity Announcement or solicitation which states the criteria against which your proposal will be evaluated.


What successful companies do, once they have identified the evaluation criteria is assess “win themes”. In other words, they evaluate how they stack up against these evaluation criteria. This will surface gaps and will help in the elimination of potential topics to which they might respond. It is not uncommon that within a few days of a solicitation’s release potential proposers gather with colleagues or management to review those topics that seem relevant. The evaluation criteria play an important role in the down select process of topics to which a company decides to respond. It will also surface areas where more resources need to be garnered. For example, if you can readily demonstrate that your technology is innovative, but you need to strengthen your team, an immediate task becomes addressing the weaknesses in proposed personnel. In other words, you won’t eliminate all potential topics because of gaps – but you will focus on those where you will be more competitive.


The concept of “win themes” is not only useful for surfacing gaps and deciding which topics to address, but is also important to consider as you write your proposal. You must make it easy for reviewers to verify that your proposal is responsive to the evaluation criteria. This is accomplished by including language within your proposal that clearly and subtly calls out the evaluation criteria and emphasizes your strengths relative to them. Sometimes a proposer may choose to call out areas where they have a perceived weakness and clarify what they have done to mitigate this risk.


Understanding the evaluation criteria that will be applied to a proposal effects everything – from your internal down select process, to the early surfacing of gaps that you will address and the actual writing of your proposal. Those that develop winning proposal always allow sufficient time before submitting their proposals to have another party independently review the draft and evaluate how the proposal stacks-up against the evaluation criteria. Make sure that you allow sufficient time so that you can address the feedback that they provide.


Tutorial 1
Understand the proposal evaluation criteria

Results:


(1) True/False? In order to demonstrate that your proposed work is innovative it is recommended that you contrast your approach with the current state of the art.






(2) True/False? The most important role in a proposal team is the subcontractor.






(3) True/False? The word “commercialization” refers to bringing a product to market.






(4) True/False? Understanding the evaluation criteria that will be used to judge a proposal should be considered before beginning the proposal preparation process.






(5) True/False? It is best to have those individuals that worked on proposal development, be the final judge of responsiveness to the evaluation criteria.




Agency Micro-sites

US Flag An Official Website of the United States Government