There are some characters that, with time, became an iconic mark of the franchise, and if you pull them out well, it's simply not the same game.
You can do a SoulCalibur without any of them, yes, just two swords and a bunch of new people looking for them, but it will no longer be a "SoulCalibur" as we all are used to see it, it would either be a spin-off or a reboot
Which is kinda what happened with SCV in some ways
Is like saying that, I don't know, you can do a whole new dragonball story with just the seven dragon balls as an object to seek, but with completely new characters, how would you call that?
A series is not only made by its core-lore, but also by its characters. Without them it would simply become a whole different thing
Plus, we're talking about a fighting game, in which the roster is basically the first thing that comes in mind, even before the story