A new Pew Research Center report examining how political polling has changed over the past decade points directly to New York-based Siena University, whose polling operation has become one of the organizations using increasingly sophisticated weighting methods to compensate for who does — and does not — answer surveys.
The Pew Research Center analysis found nearly 70% of national election polls in 2024 weighted their results using political party affiliation or respondents’ reported vote in a previous election. Pew concluded those adjustments have generally made political polls modestly more accurate.
The shift matters in New York because the Siena Research Institute, based in Loudonville, conducts some of the state’s most closely watched political polling and also partners with The New York Times on national and state election surveys.
Pew specifically identifies New York Times/Siena polling among organizations that have used past-vote weighting and voter-file information to improve the political balance of survey samples.
Why pollsters changed their methods
Polling has always involved weighting. If a survey reaches too many people from one age group, educational level or other demographic category, researchers can adjust responses so the final sample better reflects the population they are attempting to measure.
The newer challenge is political.
Pew found growing evidence that Republicans have become somewhat less likely than Democrats to participate in political surveys, particularly during presidential elections. That creates what researchers call partisan nonresponse bias: even a poll that accurately reflects the population by age, race, gender and education can still end up politically skewed if the Republicans who answer surveys differ systematically from Republicans who do not.
That problem became especially visible beginning with the 2016 election and persisted in 2020 and 2024. Pew said political polling during all three presidential cycles tended to underestimate support for Donald Trump, although the magnitude of the errors varied.
The organization found that applying political weighting to its own postelection polling in 2020 and 2024 improved accuracy by roughly 1 to 2 percentage points for each candidate, translating into about a 2- to 3-point improvement in the margin between candidates.
Pew stressed that weighting does not eliminate polling error. It can improve estimates on average, but results can still be wrong if the people reached within a political group are themselves unrepresentative.
Siena is using the same tools in New York
The methodology is already visible in Siena’s New York polling.
A Siena poll of New York voters conducted in August used voter-file information and weighted its results using factors including age, region, race and ethnicity, education, gender, party identification and measures of respondents’ likelihood of voting.
Other Siena statewide surveys have been weighted to match New York State Board of Elections party-registration patterns in addition to Census-based demographic benchmarks.
That approach illustrates one of the methods Pew examined: using voter files and official registration information to prevent a survey sample from becoming disproportionately Democratic or Republican simply because members of one party were more willing to answer the poll.
For state and local polling, Pew noted that voter-file information can be especially useful because reliable benchmarks for party affiliation are harder to establish below the national level.
One statistic shows how difficult polling has become
The Pew report also highlights a striking example involving New York Times/Siena polling: a telephone survey conducted in 2022 reported a response rate of just 0.4%.
That does not mean the poll was necessarily inaccurate. Modern pollsters use weighting, voter files, multiple contact methods and other statistical tools to compensate for low participation.
But the number illustrates the central problem confronting the industry: most people contacted for surveys simply do not participate.
Pew said its own telephone survey response rates fell from 36% in 1997 to the low single digits in recent years. Online surveys face similar difficulties once researchers account for recruiting people into panels and their willingness to participate in individual surveys.
When response rates fall that far, the question is no longer simply whether enough people were interviewed. Researchers must determine whether the people who agreed to participate differ in meaningful ways from those who did not.
Past voting is increasingly part of the calculation
Another method used by New York Times/Siena and other major pollsters involves adjusting survey results based on how respondents say they voted in the previous election.
Under that approach, a pollster compares the composition of the survey with actual election results and adjusts the sample if previous Democratic or Republican voters are disproportionately represented.
Pew said election results provide a stable benchmark, but the method has weaknesses. People can misremember their previous vote, decline to disclose it or claim to have voted when they did not.
Pollsters have therefore developed different ways of handling people who were not eligible in the previous election or who will not disclose their prior vote. Pew said research suggests carefully modeling those respondents can produce better results than simply excluding them from the adjustment.
The Pew report identifies New York Times/Siena as one of several polling operations that have used both past-vote information and political data drawn from voter files.
What it means when New Yorkers read a poll
The findings offer an important distinction for readers interpreting political polling.
Weighting is not a pollster changing responses after the fact to produce a desired political result. It is a statistical effort to correct known imbalances in who participated in the survey.
That correction can matter significantly in a closely divided election. Pew found that modern presidential polling errors are often smaller than 5 percentage points on the margin between candidates, but even an error of two or three points can substantially change how a competitive race appears.
At the same time, Pew cautioned that political weighting is not universally appropriate and can introduce its own errors. Using an outdated estimate of the partisan makeup of an electorate, for example, could force a poll toward a political balance that no longer exists.
The organization also said not every survey needs political weighting. It is most useful when political affiliation is connected both to the likelihood that someone participates and to the subject being measured.
For New Yorkers, that means the party-registration and voter-file adjustments listed deep inside the methodology of a Siena poll are more than technical footnotes. They are part of a broader transformation in American polling aimed at answering one increasingly difficult question: how do researchers accurately measure public opinion when the people willing to answer a poll are no longer representative of everyone else?
FingerLakes1.com is the region’s leading all-digital news publication. The company was founded in 1998 and has been keeping residents informed for more than two decades. Have a lead? Send it to news@fingerlakes1.com.