SW 6, SW 8B/CL 8gB, & WP 3A, are probably the 3 biggest problems for Talim. (WP 3A is just good in general...). Also, once you fight Ivy's who can seemlessly transition into stances from the ground, while attacking/being attacked, during blockstun, and during throw animations, it becomes really hard to distinguish what stance she's in and when.
Talim can win against Ivy, but you have to be extremely precise. Even when at advantage, refrain from attacking too much (unless you know the other players habits) because CL 214B is her escape move. You want to catch her using it at disadvantage so you can block it and punish with 236B/4BB. Just watch for setups much like with countering X's 44B.
Same goes for CS and SS. Watch for setups. There's always a brief pause at some point before the throws come out. The best Ivy players are the ones who can mask this buffering period almost completely, but remember she can't chain the throw immediately after attacking.
Rolling on the ground, Using CL A+B and cancelling, Using a single B in SW, or after landing a shakeable stun, are a few examples of intelligently using buffering times. SS is harder to see than CS, because of CS's unique grab animation and the JF flash if they do the JF. But conversely, it's easier to tell when Ivy's in WP stance than in CL/SW transitions. Talim has a lot of TJs, anyway, and her 2A is nice, so you may even avoid these moves unintentionally at times.
If the match turns into a poking contest, (which with higher level players this matchup usually does...) Ivy wins out due to range and damage. That's when you have to be smart about picking your spots. Don't get greedy with +frames, BUT take advantage of every major damage opportunity when you see it.
I personally think Talim has harder matches than Ivy, it's just hard to play Talim like you would against others, because of Ivy's well roundedness.