Voting is not a privilege. It’s a right. But one group of citizens has been long denied that right in parts of the country.

In half the states, including California, people convicted of felonies who have served their time in prison re-enter their communities with the right to vote automatically and immediately restored. In Vermont, Maine and the District of Columbia, people retain their right to vote even when incarcerated.

But the other 25 states have at least some temporary voting restrictions on people formerly incarcerated on felony convictions. In 10 of those states, people convicted of certain crimes, such as murder and rape, are prohibited from voting permanently, unless in most cases the governor pardons them.

Forbidding people who finished prison sentences to vote is profoundly unjust – and unwise. It’s difficult enough for people convicted of a serious crime to make their way back into their communities, find jobs and reestablish relationships with family and friends. Disenfranchising them only stifles their efforts to re-engage not just in society but in our democracy. It’s even more outrageous when you consider that people returning from prison are counted in the U.S. census for purposes of representation in Congress and for the electoral college. (Even in prison, they get counted.)

So, it’s encouraging that Sen. Laphonza Butler (D-Calif.) has introduced a bill, the Next Step Home Act, that would restore voting rights in elections for federal offices, such as members of Congress, to some who were convicted of a felony and sentenced to federal prison when they return to their communities and finish out their parole (or probation in some cases). It would not apply to state elections.

However, Butler’s bill does not go far enough and would shut out too many people, benefiting mainly those who were released under the First Step Act. This bipartisan criminal reform bill was signed into law by former President Trump in 2018, and allows certain people to participate in rehabilitative programs that earn them credits toward shortening their time in prison or take advantage of sentencing reforms.

Butler’s bill also includes elderly offenders who are considered less likely to commit a crime again and reinstates their voting rights even if they are still under home detention, completing their parole.

But the nearly 30,000 people released under First Step is a fraction of the 3.5 million Americans no longer incarcerated but still disenfranchised from federal elections, according to the Brennan Center for Justice, which supports Butler’s bill.

So why would she seek to help only this particular segment of the former prison felon population? Because, it seems, that is what has a chance of passing both houses of Congress. This would need 60 votes to overcome a filibuster in the Senate, and according to a spokesperson for Butler, anchoring it to the First Step Act could help it garner the bipartisan goodwill of that previous bill.

In fact, there are at least three other Senate bills – the Democracy Restoration Act by Sen. Benjamin L. Cardin (D-Md.), the Freedom to Vote Act by Sen. Amy Klobuchar (D-Minn.) and the Inclusive Democracy Act by Sen. Peter Welch (D-Vt.) – that would restore the right to vote in federal elections to all formerly convicted felons. Butler is a co-sponsor of all three.

The bigger question is why is there such resistance to restoring the rights of people who were convicted of a felony. There is no valid reason or public safety concern that would argue against it. Giving back voting rights to a person released from prison is not the same as giving them a right to own a gun.

More troubling is that the laws used to stop people who were imprisoned from voting are rooted in racism that stretches back to before the adoption of the 15th Amendment to the Constitution in 1870 guaranteed the right to vote cannot be denied because of race. One way to stop Black people from exercising that right was to disenfranchise everyone convicted of a crime. Today, Black people are almost four times more likely than non-Black adults to be disenfranchised because of a past conviction. Of course, we hope that Butler’s bill passes – even if it just chips away at the disenfranchisement of one group of people who were formerly convicted. But if Congress were really acting fairly and smartly, it would pass one of the bills that restores federal voting rights to all people convicted of felonies who have served their time. To ignore them is a transgression of justice and a violation of their constitutional rights.