Trump had ~100 secret service agents assigned to him as a former president as part of his personal security detail.
I have no idea what their shifts look like, but for the sake of argument, let's assume 1/3-1/2 would have been involved in his direct security, and as you said, another 20 were dispatched by Secret Service for the event. That's 50-70 agents for the event, plus the state and local law enforcement that was providing additional security.
View attachment 752593
Crooks was outside of any perimeter that was maintained by SS directly even if there had been more agents on site,
as local law enforcement were literally inside the building that Crooks shot from, and were charged with securing it as well as the entire outer perimeter.