I'm sorry, but I just can't agree with you that the current balance in our society is to "believe all victims": in fact, as an empirical, factual matter, I have a hard time imagining a single statement that is less true or more problematic, in that is nevertheless an idea that far too many people are willing to embrace right now, even if (or rather especially if) they do not work in a field where the reality of that presumption is rigorously tested. The fact is that the overwhelming default for the larger majority of people is to doubt (if not immediately attack) the person claiming to be a victim, and rush to the defense of the accused.
That's not to say that baseless accusations do not occur and that the consequences of it cannot be all-consuming for the accused, particularly as a professional matter--nor that a certain degree of mob mentality does not exist (on social media in particular). But if you want to talk about the reality of the world we live in, the fact is that, cancel culture or not, most victims face a gauntlet of doubt, harassment, and significant repercussions for speaking up. And up until recently that tended to include a significant risk of extreme isolation and ostricization. And all of this is true even in the context of the average accused--nevermind if you are talking about a celebrity, just about any one of whom benefits from a phalanx of zealous defenders who will launch themselves into the breach instantly to try to tear down (or often as not these days, directly terrorize) an accuser.
Now Tres has already more than aptly identified the right end of the stick here: the truth is, in a similar fashion to the rest of the polarization which has come to define our culture of late, people increasingly sort into two separate camps: those who embrace a confirmation bias in support of the accused and those who rally behind the accuser. Both tendencies rest in poorly predicated reasoning, and in that respect, I can support your assertions at least as far as the main thrust that dogmatic and impulsive alignment with either presumption can lead to injustices. However, as a statistical matter, we know some things about the rate of false accusations: for the most serious accusations (sexual assaults and rapes) research has long suggested that the rate of false accusation is somewhere in the low single digits as a percentile matter. Furthermore, it is thought that less than one in every two hundred assaulting parties ever pays for their crime in a criminal justice context, and the vast majority of assaults are not even reported at all. Rates of false report are less well studied for sexual harassment without a physical component, but given the lower level of stigma and that victims of these 'lesser' offenses have historically been told not to make waves over them, it's probable that the rates of false report are similar or even lower.
And if you want to talk about the consequences of rushing to conclusions, I would argue that they are far more substantial in the aggregate when we are more inclined to give the accused the benefit of the doubt than the accuser: careers and fortunes are as readily broken by the former tendency as the latter, and when you add in the psychological weight for the victim as well as the systemic effects for our society as whole, the costs of rushing to defend the accused vs. rushing to believe the accuser are pretty clearly not equal. Mind you, I am not saying that either is by any means ideal. But one scenario is clearly being unrealistically inflated in our collective consciousness and the other consistently diminished, in direct opposition to all good evidence we have as to prevalence of each.
Now part of our tendencies in that regard can be explained as an unfortunate consequence of something good in our societies: the presumption of innocence in the criminal context of our legal traditions. In much of the world, a presumption of innocence goes tot he accused which must be overcome before the government may deprive someone of their liberty. And that is as it should be. However, what is laudable when it comes to restraining policing powers of the government is not necessarily ideal in other contexts: the same reasoning for giving one party the benefit of the doubt over the other fails when we are talking about a non-criminal context, because we are no longer looking to restraint the influence of a third and much more powerful entity. And no, this does not mean that I think we should automatically leap to believe the potential victim either, but certainly we should not instead be embracing a moral panic about a supposed epidemic of false accusations--it all feeds into a culture of denialism of actual victims and the enabling of real abuses that without question occur at a much higher rate than false accusations.
So I agree with you, the number of false accusations is not zero and the consequences can be significant. However, we can accept that and simmultansouly recognize that fearing for one's reputation even if one comports themselves well with regard to others is largely an overblown (or as I put it with respect to Sytus, histrionic) fear, when compared against the reality that countless victims have labored under historically and into contemporary circumstances. That's the world we live in, my good friend.