{"id":40802255008,"date":"2023-05-25T15:37:48","date_gmt":"2023-05-25T19:37:48","guid":{"rendered":"https:\/\/www.monmouth.edu\/polling-institute\/?p=40802255008"},"modified":"2023-05-25T15:46:08","modified_gmt":"2023-05-25T19:46:08","slug":"different-poll-results-on-the-debt-ceiling-look-to-the-question-wording","status":"publish","type":"post","link":"https:\/\/www.monmouth.edu\/polling-institute\/2023\/05\/25\/different-poll-results-on-the-debt-ceiling-look-to-the-question-wording\/","title":{"rendered":"Different poll results on the debt ceiling? Look to the question wording."},"content":{"rendered":"\n<p>by Patrick Murray<\/p>\n\n\n\n<p>There have been some head-scratching over the seeming differences between results of &nbsp;recent polls from Monmouth, CNN, and Marist-NPR on the debt ceiling.&nbsp; I posted a <a href=\"https:\/\/twitter.com\/PollsterPatrick\/status\/1661408543344410627\" target=\"_blank\" rel=\"noreferrer noopener\">Twitter thread<\/a> about this, but here it is unrolled and with some additional context.<\/p>\n\n\n\n<p>If reputable pollsters have very different results on the same topic, first ask yourself this question:&nbsp; Is it an issue that regularly gets talked about where people have formed long-standing clear opinions (e.g. abortion), or is it an issue that \u2013 while it may be of crucial importance to the body politic and has been covered by the media \u2013 is one where public awareness and sense of salience is actually low (e.g. debt ceilings)?&nbsp; If the latter, then the likely cause of varying poll results is question wording.&nbsp;<\/p>\n\n\n\n<p>Yes, question wording really does matter when public knowledge and engagement is low.&nbsp; And that\u2019s the story behind the different findings on opinion about raising the debt ceiling.&nbsp;<\/p>\n\n\n\n<p><strong>QUESTION: Should raising the debt ceiling be a &#8220;clean&#8221; process?<\/strong> Here Monmouth &amp; Marist seem to agree, but CNN shows much lower support.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>CNN<\/strong>: 24% \u201cCongress should raise the debt ceiling no matter what.\u201d<\/li>\n\n\n\n<li><strong>Marist\/NPR<\/strong>:&nbsp; 52% \u201cCongress should increase the debt ceiling first to avoid a default on federal debt, and discuss spending cuts separately.\u201d<\/li>\n\n\n\n<li><strong>Monmouth<\/strong>: 51% \u201cThese two issues [<em>debt ceiling and negotiations over spending on federal programs<\/em>] should be dealt with separately.\u201d<\/li>\n<\/ul>\n\n\n\n<p><strong>Or should raising the debt limit be dependent on federal spending cuts?<\/strong> Here, all three polls appear to differ from one another &#8211; by a lot!<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>CNN<\/strong>: 60% \u201cCongress should only raise the debt ceiling if it cuts spending at the same time.\u201d<\/li>\n\n\n\n<li><strong>Marist\/NPR<\/strong>: 42% \u201cCongress should only increase the debt ceiling if it makes significant spending cuts at the same time, even if that means the U.S. defaults on its debt.\u201d<\/li>\n\n\n\n<li><strong>Monmouth<\/strong>: 25% \u201cRaising the debt ceiling should be tied to negotiations over spending on federal programs.\u201d<\/li>\n<\/ul>\n\n\n\n<p>This appears to be a case of the old adage: &#8220;Question wording matters most when people know the least.&#8221; Some issues are so salient that poll results are impervious to wording variations (e.g. abortion).&nbsp; Others are not \u2013 like the debt.&nbsp; Half the public have been paying close attention to the current debate (CNN &amp; Monmouth) and half feel they really understand the consequences of default (Monmouth). This means the true level of \u201chigh\u201d knowledge is likely lower. In other words, most Americans are not giving a lot of thought to what they think about the debt ceiling and have not really talked about it until a pollster asks them a question.<\/p>\n\n\n\n<p>Let&#8217;s take a look at each poll\u2019s question wording. First, the <a href=\"https:\/\/s3.documentcloud.org\/documents\/23822858\/cnn-poll-debt-ceiling-biden-border-security.pdf\" target=\"_blank\" rel=\"noreferrer noopener\">CNN poll<\/a>. Note they ask an awareness question with some context information on the issue before asking the debt handling question. Also, the debt question has a third response option (\u201callow default\u201d).<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"234\" src=\"https:\/\/www.monmouth.edu\/polling-institute\/wp-content\/uploads\/sites\/22\/2023\/05\/image-2-1024x234.png\" alt=\"CNN. Q23. As you may know, there is a limit to the amount of money the federal government can owe that is sometimes called the &quot;debt ceiling.&quot; The Secretary of the Treasury estimates that the government will not have enough money to pay all of its debts and keep all existing government programs running unless Congress raises the debt ceiling by\nearly June. How closely have you been following the discussions between Joe Biden and Congressional leaders on\nthe debt ceiling? [RESPONSES ROTATED IN ORDER FOR HALF\/IN REVERSE ORDER FOR HALF].\nMay 17-20, 2023. \nNET Closely 50%, \nVery closely 14%, \nSomewhat closely 36%,\nNET Not closely 50%,\nNot too closely 28%,\nNot closely at all 22%,\nNo opinion 0%.\n\n\" class=\"wp-image-40802255019\" srcset=\"https:\/\/www.monmouth.edu\/polling-institute\/wp-content\/uploads\/sites\/22\/2023\/05\/image-2-1024x234.png 1024w, https:\/\/www.monmouth.edu\/polling-institute\/wp-content\/uploads\/sites\/22\/2023\/05\/image-2-300x69.png 300w, https:\/\/www.monmouth.edu\/polling-institute\/wp-content\/uploads\/sites\/22\/2023\/05\/image-2-768x176.png 768w, https:\/\/www.monmouth.edu\/polling-institute\/wp-content\/uploads\/sites\/22\/2023\/05\/image-2-150x34.png 150w, https:\/\/www.monmouth.edu\/polling-institute\/wp-content\/uploads\/sites\/22\/2023\/05\/image-2.png 1377w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"239\" src=\"https:\/\/www.monmouth.edu\/polling-institute\/wp-content\/uploads\/sites\/22\/2023\/05\/image-1-1024x239.png\" alt=\"CNN. Q24. Which comes closest to your view? [RESPONSES ROTATED IN ORDER FOR HALF\/IN REVERSE ORDER FOR\nHALF].\nMay 17-20, 2023.\nCongress should raise the debt ceiling no matter what 24%.\nCongress should only raise the debt ceiling if it cuts spending at the same time 60%. \nCongress should not raise the debt\nceiling, and allow the US to default on its debts 15%.\nNo opinion *\n\n\" class=\"wp-image-40802255017\" srcset=\"https:\/\/www.monmouth.edu\/polling-institute\/wp-content\/uploads\/sites\/22\/2023\/05\/image-1-1024x239.png 1024w, https:\/\/www.monmouth.edu\/polling-institute\/wp-content\/uploads\/sites\/22\/2023\/05\/image-1-300x70.png 300w, https:\/\/www.monmouth.edu\/polling-institute\/wp-content\/uploads\/sites\/22\/2023\/05\/image-1-768x179.png 768w, https:\/\/www.monmouth.edu\/polling-institute\/wp-content\/uploads\/sites\/22\/2023\/05\/image-1-150x35.png 150w, https:\/\/www.monmouth.edu\/polling-institute\/wp-content\/uploads\/sites\/22\/2023\/05\/image-1.png 1370w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p>Next is <a href=\"https:\/\/maristpoll.marist.edu\/polls\/raising-the-u-s-debt-ceiling\/\" target=\"_blank\" rel=\"noreferrer noopener\">Marist<\/a>. They did not ask an opening awareness question nor did they provide a lot of context information.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"504\" src=\"https:\/\/www.monmouth.edu\/polling-institute\/wp-content\/uploads\/sites\/22\/2023\/05\/image-3-1024x504.png\" alt=\"Image.\nDCSCRS1. NPR\/PBS NewsHour\/Marist Poll National Tables May 15th through May 18th, 2023. National Adults. \n\nAs you may know, there is a debate in Washington about raising the federal debt ceiling, which is the amount of money that the federal government can borrow to pay its bills.  Do you think: \nCongress should increase the debt ceiling first to avoid a default on federal debt, and discuss spending cuts separately.\nNational Adults 52%. National Registered Voters 51%, Democrat 75%, Republican 30%, Independent  44%.\nCongress should only increase the debt ceiling if it makes significant spending cuts at the same time, even if that means the U.S. defaults on its debt.\nNational Adults 42%, National Registered Voters 43%,  Democrat 21%, Republican 65%, Independent 48%.\nVol. Unsure.\nNational Adults 6%, National Registered Voters 6%,  Democrat 4%, Republican 5%, Independent 7%.\n\" class=\"wp-image-40802255022\" srcset=\"https:\/\/www.monmouth.edu\/polling-institute\/wp-content\/uploads\/sites\/22\/2023\/05\/image-3-1024x504.png 1024w, https:\/\/www.monmouth.edu\/polling-institute\/wp-content\/uploads\/sites\/22\/2023\/05\/image-3-300x148.png 300w, https:\/\/www.monmouth.edu\/polling-institute\/wp-content\/uploads\/sites\/22\/2023\/05\/image-3-768x378.png 768w, https:\/\/www.monmouth.edu\/polling-institute\/wp-content\/uploads\/sites\/22\/2023\/05\/image-3-150x74.png 150w, https:\/\/www.monmouth.edu\/polling-institute\/wp-content\/uploads\/sites\/22\/2023\/05\/image-3.png 1153w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p>Finally, here&#8217;s <a href=\"https:\/\/www.monmouth.edu\/polling-institute\/reports\/monmouthpoll_us_052423\/\" target=\"_blank\" rel=\"noreferrer noopener\">Monmouth<\/a>, which asks a number of questions before the debt handling one. There is some potential framing for the respondent in those intervening questions, although the content is different from CNN&#8217;s intro. Also, Monmouth\u2019s is the only poll that provides an explicit &#8220;no opinion&#8221; option.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1017\" height=\"748\" src=\"https:\/\/www.monmouth.edu\/polling-institute\/wp-content\/uploads\/sites\/22\/2023\/05\/image-4.png\" alt=\"Image. Monmouth University.\n5.\tHow much have you read or heard about the recent debate over raising the federal debt ceiling \u2013 a lot, a little, or nothing at all?\n\tMay 2023\nA lot  45%,  A little 40%, Nothing at all  15%, (n)\t(981).\n6.\tHow much do you feel you understand what raising, or not raising, the debt ceiling would mean for the U.S. economy? Would you say you understand this a lot, a little, or not at all? \n\n7.\tSome people say the country will suffer significant economic problems if the debt ceiling is not raised. Do you think these claims are accurate or are they exaggerated, or do you have no opinion on this?\n\n8.\tDo you approve or disapprove of the way each of the following is handling the debt ceiling issue? [ITEMS WERE ROTATED]\n\tA. President Biden\n\tB. The Republicans in Congress\n\tC. The Democrats in Congress\n\n9.\tDo you think that raising the debt ceiling should be tied to negotiations over spending on federal programs, or should these two issues be dealt with separately, or do you have no opinion on this?\nMay 2023.\nNegotiations should be tied 25%, \nDealt with separately\t51%,\nNo opinion\t24%,\n(n)\t(981).\n\" class=\"wp-image-40802255023\" srcset=\"https:\/\/www.monmouth.edu\/polling-institute\/wp-content\/uploads\/sites\/22\/2023\/05\/image-4.png 1017w, https:\/\/www.monmouth.edu\/polling-institute\/wp-content\/uploads\/sites\/22\/2023\/05\/image-4-300x221.png 300w, https:\/\/www.monmouth.edu\/polling-institute\/wp-content\/uploads\/sites\/22\/2023\/05\/image-4-768x565.png 768w, https:\/\/www.monmouth.edu\/polling-institute\/wp-content\/uploads\/sites\/22\/2023\/05\/image-4-150x110.png 150w\" sizes=\"auto, (max-width: 1017px) 100vw, 1017px\" \/><\/figure>\n\n\n\n<p>There&#8217;s a lot going on here. The big difference between Monmouth and Marist results appears to be the former&#8217;s inclusion of the explicit &#8220;no opinion&#8221; in the question as read. This seems to lessen the number of respondents who select the &#8220;tied to spending\/cuts&#8221; response for Monmouth compared with Marist, which only records \u201cdon\u2019t know\u201d as an option if the respondent volunteers they cannot choose between the two stated responses. This also suggests, that the \u201cclean\u201d debt bill option has somewhat more deliberative support than the one that ties it to spending cuts.&nbsp; However, that\u2019s just one possible read of the variance.<\/p>\n\n\n\n<p>The difference between Monmouth\/Marist and CNN, though, is a little harder to pin down. Marist&#8217;s question specifies that tying the debt to spending cuts may lead to default, which may make that option less attractive than in CNN\u2019s question. CNN includes a separate, third response about allowing default, thus making the consequences of spending cuts less alarming in their question less alarming than in Marist\u2019s.&nbsp; It may also introduce some double-barreled potential (i.e., tapping opinion on more than one issue in the same question).&nbsp; Monmouth\u2019s question does not reference a possible default at all, which means that some respondents may think about that outcome in considering whether to choose the \u201ctie it to spending\u201d option and others may not.<\/p>\n\n\n\n<p>It&#8217;s also possible that the big difference with CNN compared to the other two polls is a &#8220;3-response&#8221; conundrum, where in the absence of fully formed opinion respondents gravitate to the middle option.&nbsp; One thing to note about the CNN question is that polling generally shows majority support for specific federal spending programs. The results to this debt question would seem to contradict those findings.&nbsp; I think the &#8220;no matter what&#8221; wording in CNN&#8217;s clean bill response option and the &#8220;damn the torpedoes, let\u2019 go into default\u201d option at the other end of the spectrum may make the middle response option that much more &#8220;reasonable&#8221; for people regardless of its actual content.<\/p>\n\n\n\n<p>So, what&#8217;s my take on the polls? When polling low awareness\/salience issues, there are no perfect questions.&nbsp; We end up tapping into somewhat ephemeral public opinion where there are few right or wrongs in question wording.&nbsp; It\u2019s the interpretation of the results we need to be careful about. You cannot assume that poll participants are responding literally to how a question is worded. &nbsp;Therefore, the more questions pollsters ask on these types of issues the better able we are to get a handle on the conditions under which public opinion is malleable.<\/p>\n\n\n\n<p>That also requires folks in the media to actually READ the questions. It&#8217;s the first thing I say to journalists when I\u2019m asked about any variations between polls. Many of you may shake your heads and say \u201cNah, that can\u2019t be it.\u201d But it often is. If only we could take a sample of public opinion like we extract a biological specimen. But we can\u2019t. Measuring how human beings form, express, and act on their attitudes is an inexact science because the very nature of opinion is always in flux.<\/p>\n\n\n\n<p>As to my takeaway on the debt issue? Most Americans don&#8217;t have a strong opinion on the best way to raise the ceiling or what that even means in broader terms. And they won&#8217;t have one unless there&#8217;s a default that impacts them directly. What we do know is that the public sees a partisan debate over the issue. Which means the most likely outcome is that partisan identity will drive who gets blamed for any unhappiness that comes out of a deal.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>by Patrick Murray There have been some head-scratching over the seeming differences between results of &nbsp;recent polls from Monmouth, CNN, and Marist-NPR on the debt ceiling.&nbsp; I posted a Twitter thread about this, but here it is unrolled and with some additional context. If reputable pollsters have very different results on the same topic, first [&hellip;]<\/p>\n","protected":false},"author":24,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-40802255008","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/www.monmouth.edu\/polling-institute\/wp-json\/wp\/v2\/posts\/40802255008","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.monmouth.edu\/polling-institute\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.monmouth.edu\/polling-institute\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.monmouth.edu\/polling-institute\/wp-json\/wp\/v2\/users\/24"}],"replies":[{"embeddable":true,"href":"https:\/\/www.monmouth.edu\/polling-institute\/wp-json\/wp\/v2\/comments?post=40802255008"}],"version-history":[{"count":6,"href":"https:\/\/www.monmouth.edu\/polling-institute\/wp-json\/wp\/v2\/posts\/40802255008\/revisions"}],"predecessor-version":[{"id":40802255040,"href":"https:\/\/www.monmouth.edu\/polling-institute\/wp-json\/wp\/v2\/posts\/40802255008\/revisions\/40802255040"}],"wp:attachment":[{"href":"https:\/\/www.monmouth.edu\/polling-institute\/wp-json\/wp\/v2\/media?parent=40802255008"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.monmouth.edu\/polling-institute\/wp-json\/wp\/v2\/categories?post=40802255008"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.monmouth.edu\/polling-institute\/wp-json\/wp\/v2\/tags?post=40802255008"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}