Assessing the quality of ideas from prolific, early-stage product ideation

The goal of many idea generation techniques, such as brainstorming, is to generate a large quantity of ideas with the hopes of having a few outstanding novel ideas that are worth pursuing. The output of such sessions is a large number of rough concept sketches, which require a rapid means of screening to select a manageable set of promising ideas. In this study we develop and test metrics for evaluating large quantities of early-stage product idea sketches. In total, 1767 ideas for three different product themes were used as a test bed. With our findings, we suggest three independent qualities that fully describe an innovative product idea: creative (as a subjective judgment), useful (as defined as having practical applications), and feasible (as determined by experts). Reviewers' subjective ratings of idea creativity had a strong correlation with ratings of idea novelty (r²=.80), but negligible correlation with idea usefulness (r²=.16). The clarity of sketch positively influenced ratings of idea creativity. Another interesting finding is that the quantity of ideas generated by the individual subjects had a strong correlation with that subject's overall creativity scores (r²=.82) and novelty scores (r²=.85), but had weak correlations with that subject's usefulness scores (r²=.38).