Should be able to select modifier keys + click, instead of just click + modifiers. Common action I use is CTL+ALT, held, then Left Click, held to navigate multi directional scrolling without affecting the normal click functions. This does not seem to be possible with the way modifiers are currently applied.