Oregon Sen. Ron Wyden, despite enormous pushback even from his fellow Democrats, still has high hopes for his pet idea to institute a wealth tax on the very wealthiest Americans.

But here are a lot of obstacles standing in the way of such a tax. There are the political hurdles, of course – who wants to be on the side of taxing paper gains on assets that could go bust tomorrow? And with an election coming?

But even more serious are the thorny legal issues surrounding a wealth tax:

The Constitution mandates that a direct tax has to be evenly apportioned among the states, which is hard to do when some states are richer than others. (This is why the income tax needed its own amendment.)

Now, most scholars think that a mark-to-market system would be less at risk in the judiciary than a Warren-style wealth tax. But whether the high court would bless a yearly tax on unrealized gains is a much less certain thing, even if they think perhaps such a tax is justified. (Take these tweets from Daniel Hemel from the University of Chicago’s law school, for instance.)

For instance: Joseph Bishop-Henchman of the conservative National Taxpayers Union cited a century-old case that found that unrealized gains weren’t income, and thus couldn’t be taxed under the Sixteenth Amendment. Interestingly enough, Chief Justice John Roberts cited that case, Eisner v. Macomber, while upholding Obamacare.

“Now Eisner is an old case and some academics think it’s outdated but it’s never been overruled,” Bishop-Henchman wrote in an email, later adding: “A direct tax on an individual that isn’t an income tax is unconstitutional.”

The plain language of court precedents and trivial things like constitutions mean little to those determined to get at a pile of someone else’s money. But this is a matter worth following closely. Remember: the original income tax was only supposed to affect the richest of the rich. Now, it’s a swamp that has swallowed us all.