SmartTiles can only support video feeds that are viewable natively in the browser. RTSP is not, but you can run through a transcoding service, perhaps (VLC, Blue Iris, and some web fee based services).
In general all SmartTiles Video discussions are in this Topic: