Do you truly have Caseta Switches or do you have Caseta Dimmers? If you have dimmers, i would definitely recommend removing them and simply hard-wiring the hot and line wires together. You can then use any button controller you want in the switch panel.
If you have a switch there, is there any particular reason you want to leave the Caseta switch in place and not remove it? Removing it increases your options greatly.