Glantz and Popova raise a methodological issue regarding our study, focusing on the use of a single item to assess product interest. While multi-item scales can often be useful, or superior, it is not always so (e.g., West and Ussher, 2010). Indeed, as we note in the paper itself, single items have been used to assess interest in smoking and in e-cigarettes, and have shown themselves to be sensitive to variations due to many factors (e.g., Pepper et al, 2014). We also note that it was not our objective to assess a range of factors like social influences on e-cigarette adoption, but specifically and simply to focus on the effect of flavors, per se, and that it was important to keep the assessment brief, given that we were asking participants to respond to 24 stimuli.

Glantz and Popova also assert (without offering any basis) that the scale was subject to floor or ceiling effects. Since the teen nonsmokers in our sample already rated their interest in e-cigarettes at an average of 0.41 on a 0-10 scale, this seems to imply that the teens may have actually wanted to give e-cigarettes even lower – negative! – ratings. This certainly does not match Glantz and Popova's expressed concern about uptake of e-cigarettes among teens.

Glantz and Popova also failed to note that the paper reports data on participants' interest in other consumer products, using the same single item, and those data clearly demonstrate sensitivity to variations by population, product, and flavor.

Finally, Glantz and Popova attack our ethics by claiming (a) that consent had not been obtained, (b) that we had failed to provide some kind of required corrective education to overcome some expected adverse effect of taking our survey, and (c) that the study must be biased because it was sponsored by NJOY, a maker of e-cigarettes. We take each of these up in turn.

As the paper notes, there were multiple levels of consent. For the teen participants, parents had to consent for the teen to be enrolled in the panel and be recruited to surveys. The teen also had to consent to participate in the panel. The teen then had to consent to do the particular survey, and the teens (and adults) essentially consented to complete each item, as they were free to skip any item or bail on the entire survey.

We do not believe there was any need for corrective education after study participation. The study survey did not provide positive (or negative) information (true or deceptive) about smoking or e-cigarettes, or promote them in any way, as illustrated by the fact that interest in e-cigarettes was near zero among the teens. Participants were simply asked to rate their interest, as survey participants are asked to do in thousands of surveys every day. We note that surveys, including those fielded and sponsored by the government, routinely ask participants about smoking and their interest in smoking without any implication they have misled or damaged participants, and without any requirement for corrective education. (Dr. Popova has been criticized for conducting research in which participants were deceived about the basis for participation and deceived about product risks without any debriefing on either count [Reader Comments on Popova and Ling, 2014].)

The study reported was indeed sponsored by NJOY, a maker of e-cigarettes. We openly disclosed NJOY's (limited) role, as expected by research standards and the journal's policies. Calling out the sponsorship as a basis for claiming bias is weak. If Glantz and Popova believe the study was biased due to industry sponsorship, they should provide specific evidence of such bias. Better yet, they or others should conduct a study of the same issue using what they believe are unbiased methods. Data are the best corrective. On the issue of sponsorship and bias, analysts increasingly point out that many other author 'interests' need to be considered (Smith et al., 2009), and declared, and, indeed, others have wondered whether Glantz and Popova's interests might keep them from evaluating evidence even-handedly (Ledwith, 2015).

In short, we believe the paper stands on its own, and informs our understanding of the appeal of e-cigarettes and flavors to nonsmoking teens and smoking adults. To inform a debate that has been rich on loudly-expressed opinion and starved for data, we welcome further research on the topic.


Since conducting and publishing this research, PinneyAssociates has begun providing consulting services on tobacco harm minimization (including nicotine replacement therapy and digital vapor products) to Niconovum USA, RJ Reynolds Vapor Company, and RAI Services Company, all subsidiaries of Reynolds American Inc. In the past three years, PinneyAssociates has consulted to GlaxoSmithKline Consumer Healthcare on smoking cessation and NJOY on electronic cigarettes. SS and JGG also own an interest in intellectual property for a novel nicotine medication, an option for which has been sold to Niconovum USA.

NB: Readers who are interested in an open discussion on these issues can view and comment beneath this blog.


Ledwith S. INSIGHT-The smoke around e-cig science. Published online January 22, 2015. Accessed on February 24, 2015 and available at:

Pepper JK, Emery SL, Ribisl KM, Southwell BG, Brewer NT. Effects of advertisement on smokers’ interest in trying e-cigarettes: the roles of product comparison and visual cues. Tob Control 2014 Jul;23 Suppl 3:ii31-36.

Smith R, Feachem R, Feachem NS, Koehlmoos TP, and Kinlaw H. The fallacy of impartiality: competing interest bias in academic publications. J R Soc Med 2009;102:44-45.

West R, Ussher M. Is the ten-item Questionnaire of Smoking Urges (QSU-brief) more sensitive to abstinence than shorter craving measures? Psychopharm 2010;208:427-432.


Saul Shiffman, PhD, Mark. A. Sembower, MA, Janine L. Pillitteri, PhD, Karen K. Gerlach, PhD, MPH, and Joe Gitchell, PinneyAssociates, Inc.