@dotMorten - You can think of ZigBee (at least in the HA context) as three distinct layers - the MAC/PHY layer which is defined by the IEEE 802.15.4 specification, the transport layer known as ZigBee Pro, and an application layer framework known as the ZigBee Cluster Library that defines methods, behaviors, and messages. The last two are defined and maintained by the ZigBee Alliance.
Thread is also based on the IEEE 802.15.4 specification for MAC/PHY, so you can use common radio silicon for Thread and ZigBee. Instead of ZigBee Pro for transport, the Thread specification defines extensions to 6LoWPAN (IPv6 over 802.15.4). That’s where it stops - Thread is purposefully application layer agnostic.
This is where the ZigBee Alliance and the Thread group are collaborating - the intent is for the ZigBee Alliance to make the ZigBee Cluster Library available to run over Thread. This is significant because it will then be relatively straightforward for existing ZigBee device manufacturers to convert their devices to Thread with an OTA update since they’re using the same silicon and firmware development will be familiar. It’s likely the the ZigBee Alliance will continue to develop the ZigBee Pro transport layer, as there are still contexts in which IP transport may not be the right answer.
I don’t think Thread and AllSeen have made any official announcements as to what AllJoyn could look like over Thread, but with Qualcom joining the board of the Thread Group I’m sure you can read between the lines. It’ll be interesting to see what happens though, since AllJoyn is not designed for constrained networks - the messages are rather large and the protocol is pretty chatty, which is never good for wireless devices using batteries. The ZigBee Cluster Library is already designed for those conditions. There are also new protocols like the one being defined by the Open Interconnect Consortium, which is also designed from the ground up to work well over networks made up of constrained devices (it’s based on IPv6 with COAP).
As for the ability to decide what ZigBee or Z-Wave devices you use - as @Ben suggested, you can easily write your own device type handlers for any ZigBee or Z-Wave device and install it into your own account without any intervention from SmartThings. We’ve open sourced (mostly Apache 2.0) the vast majority of our own device type handlers and example SmartApps, and we publish documentation (which, thanks to our Documentation Development team is getting better and better every day). But yes, you’re using our APIs, as you’re using the platform and infrastructure that we’ve built.
There are definitely other frameworks that you can use to roll your own - Windows 10 IoT, OpenHAB, etc - but they’re far more complex to build working systems from the ground up, and they don’t come with the back-end, user experience, or community that SmartThings does. Those tradeoffs will be worth it for some, and for others they won’t.