Mentioning Washington and Jefferson in the same breath as Lee when speaking about monuments to honor each of them isn't a conflation. It is just a natural extension of the logic that says we should tear down the Lee monuments.
Washington, Jefferson, and Lee were all slaveholders. They all thought the same way about slaves/slavery and advocated for the perpetuation of slavery for various reasons.
What separates Washington and Jefferson from Lee is their time in history. Jefferson in particular made statements that the debate over slavery was probably going to tear the country apart eventually, but not during his day.
If Washington and Jefferson were around when the Civil War began, they almost surely would have been members of the Confederacy. Both were slaveholders from Virginia with states' rights sympathies.
But at least I have discovered something in this thread. It isn't necessarily the racism of the Confederacy that offends those who want the statues removed, but the fact that they were traitors (which is a somewhat debatable point). As long as you weren't a traitor, or as long as being a slave owner or racist isn't your "claim to fame," then having monuments and memorials to virulently racist slave owners is OK. That isn't what I expected.