Device Abstraction/Virtualization


#1

So, I’ve been using SmartThings a long time and have hundreds of devices. Almost weekly, I have to go around and fix bad devices (dropped off network, etc)… After getting the devices reconnected, often times as a new device, the biggest PITA is getting all of the SmartApps re-associated… both very specific apps & routines as well as all of the general ones like Alexa, IFTTT, Tiles, etc… then to get those linkages fixed… huge PITA! So, my smarthome feels 10% broken all of the time…

It seems to me that one of the biggest deficiencies of ST is the hard links between the devices and the apps. It’s very constraining and prevents all kinds of useful functionality (grouping, role based devices, easy replacement/changing of devices, etc). So, I’d like to get opinions about creating some kind of universal device abstraction functionality.

There are many solutions out there for virtual devices that are great, but they are mostly designed to solve point problems. The Universal Device Type Handler (uDTH) is probably at the basis of a solution like this. I think there needs to be a SmartApp to manage the creation of new devices, mapping between virtual and physical devices (including groupings), attribute/capability translation and sync. I suspect that an implementation like this could get complex fairly fast, so I’d image that you’d need a webCoRE-like design and web based interface (outside of ST app) (perhaps this could be built into webCoRE). ActionTiles could do a lite version of this at the presentation layer…

I’d imagine that the physical devices would have some kind of prefix like zzz (to keep them grouped and out of the way)… they would have only 1 SmartApp, the virtual device manager (VDM)… all of their events would be sent to the VDM, translated and sent to the virtual device(s). The Virtual Devices would subscribe to all of the unique SmartApps like webCoRE, Alexa, IFTTT, etc… When those apps trigger an action on the virtual device, the VDM would translate that accordingly back the the physical device…

Use cases:

  • back up your ST installation… you could backup the virtual device configurations (i.e. webCoRE)… if you need to replace a hub, it would be much easier to only have to repair the devices (yeah, still a huge PITA, but maybe 50% better)
  • group lights (3 smart bulbs for uplighting) to keep in sync
  • translate capabilities/attributes (switch to alarm or motion), just like the uDTH
  • aggregate functionality of multiple devices (smart lock+contact sensor=door: opened/closed/locked) (garage door open/close/position) (I realize I’m mixing states and actions, but you know what I mean)
  • replace a smart bulb with a smart switch, etc (zigbee, zwave, wifi, etc)
  • halloween or holidays when we want to temporarily repurpose devices, you can keep the original functionality programmed (so you don’t have to figure it out when you put things back) (i.e. I repurpose some smart outlets between hallway lamps and Christmas lights and you want it to do different things in each case… I have to figure out what it was doing before when I move it back… very time consuming and error prone) I’d label this role-based functionality… you can plug and unplug the virtual devices and the physical devices from each other…
  • once you configure your SmartTiles or Alexa or IFTTT, etc, with the virtual device, there isn’t a need to reconfigure/rediscover when you change the underlying infrastructure… (BTW, I’m an IT Infrastructure guys and the best thing that ever happened to us was virtualization, now cloud… all layers of abstraction that solved real world problems/limitations)
  • could be used to assign alternate names to the devices (similar to groups is Alexa, but would apply universally)
  • … I’m sure there are many more real work uses cases, but these are off the top of my head …

I feel like I might be missing something fundamental here… some smart apps talk in specific ways to devices (i.e. lock manager and locks) What are the other technical challenges? Are they show stoppers or do they only reduce 100% universal adoption? Perhaps a fair amount of overhead on the ST platform? Cloud only, I’m sure.

I would love to hear that something like this already exists… I’ve only seen it in many parts through the great work of the community… please advise.

I am no match to develop a solution like this, but this community has some great minds and skills. I’d be interested in your thoughts… seems like this should be core to the ST platform, but so does a rules engine, a dashboard, etc… perhaps the next big functionality for CoRE?

@JDRoberts @ady624 @erocm1231 @Mike_Maxwell


#2

I’m the wrong person to bring in on this kind of discussion since as an engineer I’m in general opposed to exactly this kind of abstraction. It burdens the platform and creates confusion about what specific devices can do. And I just have a philosophical sense that it shouldn’t be necessary.

What you need are Device Type handlers that accurately expose the true features and configurable parameters of a device, and a good back up and restore utility, and a rules engine UI that doesn’t depend on, as you say, those hard linkages.

But none of that should require device abstraction. It’s really all about the UI design.

If you can’t copy a rule and then change the devices associated to it, you run into these kinds of frustrations. But that’s not because of the DTH. It’s because of the rules engine. Or the lack of one.

JMO :wink:

BTW, @tgauchat loves these kind of abstraction concepts, so I’m tagging him on the discussion.


#3

For an example of the kind of approach I’m talking about, look at the Zipato rules creator. I’m not saying this is a perfect platform, I just like their UI philosophy. Note that they are also working with zwave and zigbee devices just like SmartThings, so it’s not the devices that are the issue. It’s the UI for the rules creation. :sunglasses:

https://community.zipato.com/knowledge-base/article/how-to-create-a-rule

https://community.zipato.com/knowledge-base/article/how-to-copy-rules-in-rule-creator


(ActionTiles.com co-founder Terry @ActionTiles; GitHub: @cosmicpuppy) #4

I’m no CoRE / WebCoRE expert my any means; but my feeling is YES … if the majority of your “automations” are (or can be) done via CoRE (and, well, except for Smart Lighting and Smart Home Monitor; you might as well use WebCoRE for everything else), then that’s an excellent place for an additional layer of redirection / abstraction to be done.


Should this be a fundamental part of SmartThings? I ought to avoid that question because I’m biased, have a conflict of interest (ActionTiles) and I’m realistic: There are an infinite amount of features that SmartThings could be incorporating in the product which would improve it to some folks and make it hopelessly complicated to others. There is no reason to be optimistic that they will implement any particular feature we’d like to see and the way we think it should be implemented.

The typical and effective industry answer?: Let a “platform” be a “platform”! :smiley: i.e., let 3rd party developers fill in the blanks. WebCoRE does this wonderfully; and, I’m not too proud to say, ActionTiles also fills a high-demand need that our customers love. :heart_eyes:

Should ActionTiles offer a redirection / abstraction layer to help insulate against device ID reassignment (Location rebuilds, etc.) … I think it is a good idea, but we get an infinite number of good ideas. Feel welcome to visit and post on our down Feedback Forum: https://ActionTiles.com/feedback


#5

I think most home automation systems have some kind of rule creation. So I do think that’s a basic platform function. The question is how you implement that. :sunglasses:

Here again are the Zipato rule creator steps for copying an existing rule:

I don’t see any reason why that would come from a third-party add on, it seems like pretty basic functionality. Again, JMO. :sunglasses:


(ActionTiles.com co-founder Terry @ActionTiles; GitHub: @cosmicpuppy) #6

With due respect, the Zipato UI you depicted looks extremely intimidating, even to me (though, obviously I’d figure it out pretty quickly).

There’s no way that SmartThings’s marketing/product management would approve of anything approach this level of complexity in appearance; it’s not appropriate for their target market.

The entire department would rather quit … JMO :sunglasses:.


(Glen King) #7

With due respect :wink:, that method actually looks simpler than ST once you get it down. Especially since there is no ‘official’ rules engine available.


(ActionTiles.com co-founder Terry @ActionTiles; GitHub: @cosmicpuppy) #8

“We’re” not the bulk of SmartThings’s target market; otherwise there would have been a rules-builder in the product at the time of launch (or in the years following). And a backup, restore / migration tool. And… an entry-exit grace delay in Smart Home Monitor. And… you name it.

I can only infer their strategy, but from day #1 it has been clear that SmartThings is (trying to be) different, rather than swallowing some pride and taking cues from what has worked well in dozens of existing HA systems.

Blazing a new trail takes guts and focus though. Some folks think this was the key to Apple’s success (or just a myth).

Anyhow… Drifting off Topic; but the theme is the usual: head-scratching about the choices of SmartThings’s designs and priorities. They work in mysterious ways!


#9

It’s only intimidating because you haven’t used it. :sunglasses: You would pick this up in two or three minutes because you start by building a very simple rule and then adding more conditions.

It’s really very simple similar to SmartRules, for example.

http://smartrulesapp.com

Which, by the way, does have a back up and restore, although I’m not sure if they have a copy feature for an individual rule.

Can I backup my rules? Can I dump my rules to see them all in one place?
Yes. On the settings screen, you can tap the “Export” button. This will generate an email with all of your rules written out in a format that is easy for you to read. There will also be a file attached that contains your rules in a format that is easy for the app to read. Send this email to yourself and keep it as a backup.
.
To import your rules from the backup, long press on that file and select “Copy to SmartRules”. SmartRules will open and the rules will be added. If any of the devices from a saved rule are not available, that rule will not be imported.

@obycode


(ActionTiles.com co-founder Terry @ActionTiles; GitHub: @cosmicpuppy) #10

Indeed, you’re reiterating my point.

Of course I’m not intimidated by a rules builder. Heck, I’d prefer it 1000x over what SmartThings actually offers. Prior to SmartThings I used x10, which had a graphical rule flow builder. It’s common sense; especially to a programmer who had to hand draw flow charts in COBOL and ForTran in secondary school quizzes!

But SmartThings has explicitly decided not to offer this. I personally believe it is because they had a vision that use-case based “SmartApps” would be simpler; and, I believe that at first glance to a novice consumer, the use-case based SmartApp approach looks simpler and is easier to market (not necessarily easier to use).

Only extensive research (focus group testing, etc.) of the actual target market can say whether my assumption is right or wrong; but my point… I can see what SmartThings was/is “aiming at”.


#11

If only…

Look at what’s missing from SmartThings:

Easy and obvious way to group lights, including dimmers, together

Easy and obvious way to trigger a motion sensor to turn on a light

Easy and obvious way to see what sensors are open.

Easy and obvious way to set up an exit/entrance delay for smart home monitor

Those are all mass-market use cases.

SmartThings UI has always seemed to be driven by proof of concept keynote speech slides, not consumer-driven use case design.

Just sayin’… :wink:


(ActionTiles.com co-founder Terry @ActionTiles; GitHub: @cosmicpuppy) #12

Both of those (well… Not group dimming) were in Dashboard Solution Modules introduced in App mid-V1, and dropped with App V2 (at release of Hub V2).

And sorely missed by those who used them.

Sigh. :confounded:

“Smart Lighting” is pretty easy (I think) and accomplishes simple triggers like you mention, and runs locally on Hub V2 (with supported Things, etc.). But … Since it had to be squeezed into the SmartApp paradigm, ST had limited flexibility in making it “easy”.


(Mario Fuchs) #13

I am totally with you on this one. When I started out with HA I looked at the available options and was surprised that all systems are, in lack of a better word, more or less monolithic. I ended up with ST due to it being the least closed in nature (albeit not fully open).

I work in the movie industry, more specifically with visual effects. Since years this industry has focused on workflow/pipeline optimisation and asset management systems. A general mindset of “let the computers do the heavy lifting” leaving time for creative creation is widely in place due to tax breaks wreaking havoc in a previously more lucrative market.

That’s all fine and dandy, but the control systems have to be put in place, and are still very much reliant on hardware, software and their respective file formats.

This will be a bit long winded, and is my take on the direction we’re hopefully headed. What does the VFX industry have in common with home automation you might ask? Quite a bit it might turn out IMHO.

//Before embarking on this journey: full disclaimer - English is not my native language, so please give me some oversight in this department.//

I’ll spare you the boring backend details about the bandwidth requirements that are the opposite of normal database users. A relative few generate wast amounts of large data, quite the opposite of normal database requirements and handling.

What’s interesting is what is going on software-wise. We are now at a point where the the asset management systems are encompassed by abstraction layers. I’ll get to this in a moment, but first you need a bit of background.

Historically it goes something like this: an operator (artist) of a specific software (texturer, rigger, modeller, animator, lighting, rendering, compositing) is assigned an asset to work with. This asset is then waterfalled throughout the production pipeline in the different departements that finally are rendered and put together to end up on the screen for all of you to enjoy.

These asset goes through several iterations and internal/external sign-off stages before handed over to the next department. All of this is handled via the asset management system (ASM for short). Much like handling code and branches, but now with huge mounts of data - terabytes close to petabytes aren’t uncommon for a feature film.

You check out the assignment, do the job, and check it back in basically. There are of course iterations and a constant version revision checking going on. Imagine being an animator starting out breathing life into crude box-like low-poly characters, at the same time the modeller and texture artist are working on the overall look.

The ASM keeps track of, and makes sure that assignment version checked out contains the latest revisions of all accompanying data up stream.

That’s all worked out and has been working quite well over the years. Kind of… until new software versions were introduced and new file formats with them, together with the crux of recalling old projects produced on old versions of software and operating systems.

(we are now a few years back in time) Welcome abstraction layers! What if you invent new, generic file formats that can be parsed/translated to any software (new and old)? and what if you could call up and inject an exact copy of a specific state of a workstation almost instantly?

This is happening on a larger scale as we speak. Need more rendering capacity? fire off a bunch if instances in the cloud.

All of this is handled by what I like to refer as
Event stream publisher (the ASM), that in turn is programmed with
Listeners who react on specific changes, who in turn instruct
subroutines to act on those

It goes something like this:
The modeller is done with his asset and checks it in and marks it done. For argument’s sake we’ll leave the approval process out of it, and just assume it is finalised. The act of checking it in and marking it “final” fires off multiple processes that snapshots the current state and informs anyone tagged downstream that they can pick it up for the next task. Environment variables are collected (software versions, plug-ins used, system version, hardware configuration) for possible later recall. At the same time the assets are made generic/abstracted for anyone that has access rights to parse into their specific environment.

  • “Oh, man you made me read all this and not a word about HA?”

Well, if you apply large parts of this and think of the abstraction layer as a kind of fabric/logic that encompasses not only functions like Smart Lighting or WebCore, but extend it to generic placeholders for SmartApps, Device handlers and even actual hardware?

Imagine having a system blanket that allows you to keep (backup and recall) all logic, and if you exchange a bulb for a better/newer version that has not only the same features, but allows for the injection of additional features without requiring a complete replacement of the DH and all the logic you’ve set up with it? A generic DH-container that can be made to enable future updates not imagined today.

To take it one step further: as long as the event stream publisher is made to talk/listen to the platform, you could in essence swap ST for the next big thing down the line. Don’t like it and want to go back? recall and fall back.

Why did I choose ST you might ask? because my experience is that any vendor that has embraced open formats and cross-platform exchange has survived. The others are either bankrupt or operate on super-specific tasks targeting such a small market-shares that they slip by.

There’s no real conclusion here, only a hope that my thoughts might spark some food for thought and discussion. Don’t forget that “the cloud” is nothing more than a data-centre somewhere in the ground. Subcription services aren’t all they are made up to be and a backlash is coming. In the words of Public Enemy: Don’t believe the hype.

Best regards,
/Mario


(John C) #14

Only once in my entire career have I seen use testing by impartial, outside people. It was for the UI of the configuration part of a touchscreen driver. Very successful, too.

What usually happens (after a company is no longer a startup!) is some marketing guy says, “the customer wants this!” It might be what the end user wants, but often isn’t. Then, under time pressure, some engineer is tasked with doing it and implements what (s)he thinks the marketing dude wants.

Eventually, the customer gets it and… You guessed it! “This isn’t what I wanted!”

A research group that includes that customer would have come up with something closer to what customers want. The engineer would own a piece of it, having participated in the group exercise, and the marketing dude would send out for pizza — and otherwise stay out of the room!

(Written tongue-in-cheek, but those of you in the trenches will recognize a lot of the participants. :yum:)


#15

Does this resemble what you are talking about?

http://www.monolithic.org/blogs/presidents-sphere/what-the-customer-really-wants


(John C) #16

Exactly! (Have always liked that cartoon!)