SmartThings is in the middle of a huge transition: new hub, new app, new platform. Once that is complete, you can program in anything you want, from C to Python, and communicate with SmartThings through a REST API.
There is also a new “Rules API” intended to eventually replace the community built WebCore, which is essentially a scripting language for SmartThings. But both of these are still in early development.
https://smartthings.developer.samsung.com/
For now, your best bet is WebCORE:
FAQ: What is WebCoRE? (And what was CoRE?)
But all of that said, it’s critical to understand that SmartThings will continue to largely be a cloud-based platform. Your code, whether on the new platform or the old, will not run on your hub. It will run either in your own separate server or in the SmartThings cloud. All custom code runs in the cloud. Even the new Rules API still runs in the cloud. And the SmartThings hub doesn’t even support telnet: everything has to run up through the ST cloud, even their own mobile app.
If you are leaving Insteon and looking for something new, my suggestion would be Homeseer. Yes, it’s more expensive than SmartThings, but it’s also a much more robust local environment.