I wasn’t sure whether to call this entry “Taking a Poll on Tolls” or “Taking a Toll on Polls”.
A Monmouth University/Gannett New Jersey Poll on the toll road plan proposed by Governor Corzine was released today.
Our findings paint a slightly different picture from the poll released by the Bergen Record on Monday. This follows presidential primary polls where our polls differed as well. Herb Jackson did a pretty good job summing up the differences in methodology on his election blog, so I’ll just focus on the toll road polls here.
The Record poll, which is conducted by Research 2000 in Maryland, found 42% in favor of the plan, 52% opposed, and 6% with no opinion. The Monmouth/Gannett poll found only 15% in favor, 56% opposed, and 29% with no opinion. So, what’s up with that?
One of the key differences in assessing public reaction to the toll road plan is the way the questions were worded in the two polls. The Record’s poll described the plan as “reining in New Jersey’s public debt load by imposing a series of 50 percent toll hikes…Proceeds of bonds backed by the future revenue increases would be used to retire debt and fund new road improvements.” Our poll described it as a “plan to raise tolls about 50 percent every four years over the next 14 years in order to reduce state debt and fund transportation projects.”
The Record wording started off by emphasizing “reining in debt” while ours started with the cost issue. We also spelled out a time frame for the toll hikes. The Record’s poll asked respondents whether they “strongly favor, favor, oppose, or strongly oppose.” Our poll asked “do you favor or oppose this plan, or do you have no opinion?”
This may all seem a bit esoteric for the casual poll watcher, but in the short time frame after the toll plan was released (although it has been spoken about for months), question wording and response option choices can matter. Each poll started interviewing the day after Corzine’s State of the State address, but the Record completed interviewing in two days, while the Monmouth/Gannett poll interviewed for five days.
Differences in question wording, as in this case, are valid choices made by pollsters as ways to tap what the public is really thinking and it is incumbent upon us not to “create” opinion by phrasing questions that are far removed from the experiences and discussions of the typical resident.
And we have to be fair in the way we word the questions. Most “favor or oppose” questions are just that. The pollster will only record a “no opinion” response if the respondent insists. On this issues, I heard from a number of plan supporters that they believed many members of the public wouldn’t care about this plan since they don’t drive the toll roads. So, it made sense to explicitly include the “no opinion” option in the question we used.
But that alone doesn’t explain the differences between the two polls’ results. There are some interesting demographic differences in the poll breakdowns. In the Record poll, Democrats support the plan by a 62%-32% margin while Republicans reject it by 81%-17%. Independents reject it by a slimmer 55%-39% margin. In our poll, residents of all partisan stripes rejected the plan, including Democrats (48%-19%), Republicans (68%-9%) and independents (57%-16%).
There appears to be some serious difference in the Democrats interviewed by the Record and those interviewed by us. The Record poll’s sample was comprised of 600 New Jerseyans who reported they generally vote in state elections and are likely to vote this November. Our poll was composed of 804 New Jersey adult residents. [Side note: As a matter of standard procedure, the Monmouth/Gannett poll prefers full population samples when polling about issues that affect all residents, whether they vote or not.]
Does this mean that Governor Corzine does better among people who will go out to vote when (and if) he runs for re-election? Well, not necessarily. When we drilled down our sample to the most likely group of voters, we found that opinion of the plan stood at 16% favor, 58% oppose, and 26% no opinion – nearly identical to our results for the general population.
Aside from the type of sample used by each poll (likely voter vs. adult population), there are some key differences in how the surveys are weighted. We use weighting techniques make sure our surveys are representative of the population on region of state, gender, age, education and race.
One thing we don’t use in our weighting is party identification. Party ID is an attitude that is subject to change, unlike hard demographic data (unless of course you’re planning on a sex change operation). Most media pollsters ask something along the lines of: “In politics today, do you consider yourself a Democrat, Republican or independent?”.
A major difference between the two polls is our party breakdowns. Our full population sample in this poll identified itself as 37% Dem, 22% Rep – we are pretty a blue state – and 41% independent. The Bergen Record poll’s party-self identification is 28% Dem, 14% Rep, and 58% independent.
While New Jersey’s electorate is pretty fickle, it’s not that independent. Interestingly, the Bergen Poll party numbers roughly correspond to the party registration figures on the state’s official election rolls. However, as anyone of who has run a campaign in New Jersey knows, a good number of those “unaffiliated” voters consistently vote either Dem or Rep in general elections. You unaffiliateds who are party-line voters know who you are! They are only unaffiliated because they haven’t bothered to vote in one of New Jersey’s typically non-competitive primaries (making turnout projections for this year’s presidential primary that much more interesting).
The problem is if you weight the party preference question (“what do you consider yourself today?”) to the party split in the voter registration books for New Jersey – and I’m not saying this is what Research 2000 did – you’re mixing apples and oranges.
I have some more thoughts (and data) on weighting poll results by party ID. But it’s been a long week, so I’ll leave that for another post.