Personally, III remains my favorite game in the series, and I sort of think of it as the missing piece from IV and V. I still hold you could make a near-perfect game if you combined the good qualities of one of the recent Soul Caliburs with III's virtues while canceling out the flaws in both.
I think II and V are probably the best games in the series mechanically (II was where I joined the series, so I sadly can't comment on anything that came before) but I viewed III as a straight upgrade of II when I was younger and still wish for its content depth to be revisited with future games.
I thought that Tales of Souls is really how "story mode" should work in a Soul Calibur game; it took advantage of the narrative depth of the series to create a long, branching narrative unique to each character that explained where they were, and what led them to that fight and why. You were making choices and felt like you were really advancing the story, while II and IV's story mode felt more like "Character X curb-stomps five random people in various locations with no particular rhyme or reason, chats with an additional person before curb-stomping them, and fights the boss." III's quick time events could get a little repetitive, but between actually knowing why the hell I went from scrapping with Cervantes in the middle of the ocean to duking it out with Cassandra in a greek temple and Zas/Abyss/Night Terror's boss fights blowing Algol out of the water, I felt like beating Tales of Souls was a much more engaging and rewarding experience than its equivalents in the other games, including V's overly narrow story mode.
I liked both Weapon Master Mode and Chronicles of the Sword from II and III, and was rather disappointed neither of those alternate gameplay modes have been used since then. I liked the specialized fights and RPG elements, and I think a mode like that gives created characters more of a reason to be there.
Character creation was obviously new and untested in III, but I thought it had better layering of items than future Soul Caliburs, and to be honest, I miss the alignment/mood system giving you a wider variety of "personalities" for your character and the "original" movesets.
I know, I know, the original movesets were part of the reason people hated III, but I really think it gives the character more of an identity if you have access to a couple of styles that make them more than a reskin of the character you're best with.
I mean, I'd be really interested to see what would happen if they made a Soul Calibur game that had a more streamlined engine and online play like V but then added in several alternate gameplay modes like chronicles, re-introduced Tales of Souls as the default story mode alongside Arcade, and added say, five or six original weapon styles to character creation. Lots of original weapons results in bad balance, but I think it'd be really cool to see completed, balanced versions of disciplines like Grieve Edge, Katana & Shuriken, Hwang's Chinese Blade, a massively reworked version of the Dagger, and maybe some kind of weapon the series hasn't really explored yet, like a one-handed European arming sword or meteor hammer...
I think III's gameplay and AI could've used work, but its dedication to story and single-player content are something I think should be merged with the more mechanically excellent but content-shallow games that followed it.