Professional / Mainstream Media Reviews?

Continuing the discussion from IFTTT not working (OAuth issues for UK servers):

I hope this Topic doesn’t look like I’m trying to “stir the pot / rock the boat … or trolling for negativity”. The intent of this Topic is to discuss Community perceptions of professional reviews (or lack thereof) of SmartThings. Occasionally we’ve had Topics that focus on one or the other article, but this one is generalized for now…


So… @Sean_Quorn_McNuzzhou: Why do you think SmartThings has not received, as far as I am aware, any significantly negative reviews from professional / mainstream media; even in the USA where the product has over 100,000 Customers? Heck, I haven’t even found any SmartThings-specific negative blogs or vlogs.

As a faithful, though critical, Customer, I’m glad to see lots of positive reviews. But I am always looking for other perspectives, since I know I’m not an “average consumer”. Amazon and the app stores do reflect a fair amount of negative sentiment; but the number of reviewers is very small compared to the installed base, and I’m trying to focus this Topic on deep-dive, professional media examples.

Discuss?

The IoT ecosystem is so immature that there is simply no solution capable of ticking all the boxes. It supports Hue, but it doesn’t support Z Wave. It supports Z Wave but it doesn’t support Apple HomeKit.

SmartThings has made huge strides in this area, with the aim of homogenising a disparate ecosystem. Compared to almost any other automation technology, the ethos has been an open one, embracing a range of protocols, partnering with the right people, making the API freely available and getting stuck in to IFTTT to address shortcomings.

Therefore, from a professional standpoint, it is almost impossible to give SmartThings a bad review. It’s a similar argument people make for politicians - it’s the best of a bad bunch.

From a personal perspective, there is plenty to complain about. The shocking native app, or lack thereof if you own an iPad, the slow roll outs for updates, and now a botched global expansion. SmartThings’ headline USP is based on its ability to integrate with an ecosystem beyond that of its own walls. To not be able interface with any third party applications or services is more than a little bump in the road. It’s a game changer. Of course, it will eventually be resolved. I just don’t understand how you can go to market with such a major problem going unresolved.

While I’m sure there are a great many advantages to being owned by Samsung, I’m certain there are also pressures - and I wouldn’t be surprised if the parent company had a role to play in this rushed launch.

I’m thinking about reviewing SmartThings as well as a range of other hubs for a mainstream publication in the near future. I’ll be sure to add the link and then you will have at least one negative review on your scorecard.

I understand your sentiment, but I disagree with the absolutenss of this statement.
Why can’t a review just honestly list the pros and cons of SmartThings, and, if necessary, discuss the implications of the negative points to the average consumer with an appropriate and accurate conclusion?

Further to the above – Why do you need to include “a range of other hubs” in a preliminary review? Serious question: From a professional journalist viewpoint: Is it necessary to have other products to compare to, in order to write a good accurate review of one specific product or brand? Why?

BTW: There have been many (net positive) reviews of SmartThings that do not compare it to other products (or barely mention other products).

I suppose that would depend on the type of publication one works for. If you are a gadget publication, it is enirely possible to write a review for SmartThings without touching on other hubs. But what if your focus is on Apple products, or audio equipment, or business?

It is precisely because it is such an immature market that it is important to use comparisons. There are Kickstarter campaigns starting every minute, nipping at the heals of ST. New partnerships are being forged every second. People want to know that the technology they are investing in will not be superseded by another vendor in a few months time. The vast majority of people are looking at automation not as a hobby, like us techies, but as a genuine tool to help simplify their lives.

They want to make sure they are choosing the best tool for the job.

2 Likes

I agree … and, in fact, there are many such reviews out there. But these all have “net positive” overall conclusions. So, not meaning to be argumentative, but I’ll toss back:

Is it possible to write an accurately negative review for SmartThings without touching on other hubs, and if so, why aren’t there any in publication?

I’ll redact something I said earlier. I probably wouldn’t give it negative review. The net, as you say, would probably be positive.

However, that is only because of experience with other hubs

And if I were to write a review before this current issue is resolved, it would have to be very short.

"At the time of writing, SmartThings does not currently fulfil the promises it makes. Want IFTTT? You can’t have it. Hate the app and want to use a third party GUI? Forget it.

SmartThings’ success in the US is nothing to be sniffed at and one has to assume that this is based on the quality of the platform, rather than the size of the marketing budget. Unfortunately, we can’t attest to the accuracy of this assumption, because - as of right now - it doesn’t do very much other than take up much needed space on the desk and tell us when the door is open.

For UK customers, the SmartThings hub is not smart, nor does it control very many things. It’s more like an expensive placemat, but without the advanced functionality of being able to support your coffee. Wait until Samsung pulls its finger out before you consider investing."

2 Likes

How do you define “success”?

  • By net positive reviews professional reviewer: Success.

  • By Amazon ratings: Mediocre (NB: 1-Star is the lowest possible review (rather than zero), so the ratings are technically all skewed by +1; therefore, the current average review of Hub V2 is 3-Stars which really means 2 … i.e., 2 out of 4, roughly implies that half the people don’t like it). Hub V1 had 3.8-Stars; so it either benefitted from more time, optimism, or the Hub V2 is a disappointment.

  • By Android and Apple app reviews? Also Mediocre to worse.

  • By sales? Around 100,000 Hubs after 2-3 years on the market. This is definitely a number that is hard to assign meaning to. I think there are about 2x to 3x as many Wink Hubs, but many of those were free with lightbulb purchases. And to compare apples to oranges (half literally…), Apple sold 6 million iPhones in the first year (2007?), but they had a lot of name recognition from PCs. Then again, Samsung is a world leader in phone, TV, and appliance sales, so that is a similar advantage to 2007’s Apple.

  • By name recognition? Undecided. I read a lot of smart home news… I don’t consider SmartThings’s level of media penetration to be very high yet. Certainly not when compared to Nest. Even specialized or vaporware gets similar or more coverage (Control4, HomeKit, alarm companies). And every new Kickstarter or IndieGogo related to smarthome get’s a wave of publicity, most of it not mentioning SmartThings as an existing and viable alternative. Blargh.

I disagree. A zero rating would be meaningless. Why not allow the consumer to make a negative rating? Some products are so bad they need -5 stars, right? This is all beside point since Amazon does not do a straight average of the star ratings. It’s more complex than that. Each review also has a rating sub-system of helpful or not helpful. Other things are taken into account. The actual method of calculation is not public knowledge but these are known factors:

  1. Star rating: A product’s overall star rating will now consider factors including the age of a review, helpful votes by customers, and whether the reviews are from verified purchasers.

  2. Review ranking: Similar machine-learned factors will help determine a review’s ranking in the list of reviews.

TLDR; Read the reviews as they are more meaningful than a simple star rating.

Hmmm… I’ll try not to divert too much on a mathematical or rating perception tangent… but my nerd side can’t resist. First – Yes …the problem would be best solved by allowing “negative ratings”, as each negative sentiment would effectively cancel an equally strong positive sentiment. Let’s start a petition. :smile:

But with no negative allowed, well, The Babylonians invented ZERO for good reason:

Let’s take a hypothetical with an equal number of “Hate It” (lowest rating) and “Love It” (highest rating).

  • With a scale of 1 Star to 5 Stars, that results in a net average rating of exactly 3 Stars: (1+5+1+5)/4 = 12/4 = 3

  • With a scale of 0 Stars to 4 Stars, the same sentiment results in a net average rating of exactly 2 Stars: (0+4+0+4)/4 = 8/4 = 2

So what’s the difference?:
I think there is a huge difference in what the reader perceives about the summary rating:

  • An average rating of 3/5 Stars = “Grade of 60%”. i.e., more than half-way to 5 Stars, so it is biased towards a net “favorable” rating.

  • An average rating of 2/4 Stars = “Grade of 50%”. i.e., it has no bias negative or positive. QED: Allowing zero star ratings could more accurately reflect the exact balance of negative and positive reviews per the defined hypothetical equally weighted sample scenario with non-adjusted averaging.

It is simple for an informed observer to adjust from one scale to the other, of course: Just subtract 1 from the scale and result. So a SmartThings Hub V2 rating of 3.1 Stars should be perceived as 2.1/4 or a Grade of 52.5%; barely a pass. Probably an F in most schools.

With Amazon’s secret fudge factors, all this is moot, of course, as you pointed out. :stuck_out_tongue_winking_eye:

It truly is ashame as the platform was really starting to become stable pre-V2 release but I felt like it still needed to be polished before that happened and they went and pulled the trigger which sunk all existing customers right down into the whirlpool of all of the new customer problems. I think by this weekend I’ll have my V2 system in “predictable” operating condition, meaning things work as I expect them to.

That being said I am having some sunset failures this weekend so we’ll see how it goes by this weekend. All that is left is re-creating my SmartTiles dashboard which I’m excited to hear gained a new partner. :wink: Congrats and good luck. Now Back on topic.

2 Likes

Well… The positive Professional Reviews continue… 4.5 filled circles from PC Mag

http://www.pcmag.com/article2/0,2817,2494589,00.asp

So… CNET picks SmartThings (free to $5/month) over SimpliSafe ($25/month?) for Home Security. Gee… that was a really tough call :unamused:.

But they acknowledge SmartThings App glitches more than I’ve seen in mainstream media…

The biggest problem with SmartThings is that the app is very confusing to navigate and occasionally glitchy. (The team has introduced at least two app updates since my initial review and it’s only slightly better than before.)

By far the hardest part is adding devices to the app and trying to sort them by room – a seemingly simple prospect, but the app often crashes when you try to add or edit a device.

It has some other strange quirks, too. For instance, you have to create a room for a device before you can edit that device. And, you can only edit a device if it’s the “featured” device in a room, meaning it’s the first one in the list of connected devices. I can’t tell you how many times I had to delete entire rooms or switch out the “featured” device just so that I could make changes to the settings.

1 Like

Well… here’s a negative review, but “just a blog”;


2 Likes

And another “blog entry”…

I’m a SmartThings user even though I believe the product has not been that great lately. Since the buyout from Samsung, the product has been sub-par.

Well… now there’s this:

Full discussion is in the Topic above; but for the record, my summary comment is:

I have very mixed feelings. Indeed, it’s “just a blog”, but it’s apparently a relatively high-profile blog on the website of a respected industry journal (Network World). So it probably has more impact than an average blog, and it could perhaps have been written more professionally, … on the other hand, it is what it is. There are no comments on the article, I haven’t searched to see if it was reblogged, etc…

So is this really a significant thing? Dunno; but I’ll repeat: I’d rather see an impartial (but in-depth) review of the product and platform, than a exposé of problems in the Community.

Only just noticed this thread. I’ve written a fair bit about ST in my column in PC Pro magazine, here in the UK. (I believe it also gets syndicated to other magazines around the world).

2 Likes

That’s great, Paul. Can you share some links?

We were told in another thread today that the Community is definitely not representative of the “average” SmartThings Customer (who has… 15 or fewer devices). We are early-adopters, super-users, geeks, hobbyists, industry folk, etc…

So viewpoints from the media help give just another set of perspectives…

1 Like

I don’t think the columns have gone on line - only in print. But I can post some snippets of copy here! Bear in mind these were written for a UK audience.

OK, this was a small intro in issue 258

Let’s talk for a moment about home automation kit. I mentioned Energenie a few months ago, which has a well priced off-the-shelf collection of bits and pieces, both for controlling mains powered devices and also radiators. It’s still slightly limited, as right now there’s no light dimming facilities, but I’m sure those are coming. Another well thought out system appears to be SmartThings from Samsung – I’ve not had much of a play with this yet, but the products appear to be well thought out, and it has the bonus what as well as working with Samsung’s own controller and sensors, it’ll also talk to many 3rd party Z-Wave, ZigBee and LAN connected kit.

And where Samsung has SmartThings, Apple has HomeKit. It’s a slightly different approach to that offered by Samsung and indeed most other vendors. There’s no central hub device – it relies on an iOS device for that. And HomeKit will only work over Bluetooth or Wi-Fi. If you want to use something like Z-Wave then you need an additional bridge. But the best thing about HomeKit is that it’s tied into Siri, so you can say things like “Siri, open the garage door” – that’s pretty neat.

In fact, if you have an iDevice and want to use this Siri integration but extend it beyond Apple’s HomeKit ecosystem there’s a neat thing called HomeBridge with you can download from GitHub - homebridge/homebridge: HomeKit support for the impatient.. It’s essentially a HomeKit emulator, and it’s dead simple to install it on something like a Raspberry Pi. Once you’ve done that you can connect the Pi to your SmartThings hub and use Siri to control your various devices. I’m really not sure whether it’s Apple or Samsung that will be more annoyed by this.

And then this was a follow up a month later:

Continuing the automation theme, last moth I made a brief reference to Samsung’s SmartThings, but I thought it was probably worth giving a few more details, as in many ways SmartThings is quite different from anything else on the market.

A good place to start is probably the Starter Kit (funny, that!). You can pick this up for around £200, or £150 if there happens to be one of the fairly regular special offers on at the time – keep your eyes peeled. You should probably think of it as an experimenter’s box, which will allow you to get a feel for what kit like this can actually do. What comes in the box won’t be enough to fully automate your home (or small business), although the sensors and controls will make a good starting point.

So, what do you get for your money? Well, the starting point is a hub. This requires a wired connection to your broadband router, although I checked and it works fine over a homeplug link if you need to install it in another part of the building. It’s a shame there’s no Wi-Fi option, but I’m guessing that might have caused interference with the other onboard radios. These comprise ZigBee (which runs at 2.4GHz), and Z-Wave, which here in Europe uses the 868MHz band. The hub also contains a Bluetooth radio, but this is currently disabled.

The Z-Wave frequency is really important, because it’s different in the US and the EU. This means that you need to be very careful when buying things like sensors on the Internet, especially as imports, as they may not work over here. ZigBee is better as it’s the same standard worldwide. Another thing that I should mention while we’re talking radios is that for most of the kit I’m about to mention, anything which is permanently powered (i.e. not running on batteries) will act as a repeater, so proving a meshed network. Which is neat.

Alongside the hub, the box contains four ‘Things’. The first is the only one of them which is controllable (the rest are all sensors), and it’s a plug-in power outlet. There’s no dimmer – it’s a simple on/off switch, but on top of that what you do get is the ability to measure the current draw, so it’s really a sensor too. And like all of the Samsung kit in the box it uses ZigBee for the comms.

The next Thing is what’s called a Multi-sensor. It’s a device that combines a temperature sensor, an accelerometer (which registers orientation, vibration, and movement), and an open/closed contact sensor which you can use for doors, windows, etc. Also in the box is a motion sensor – essentially a battery powered PIR movement detector, except it also measures temperature, and finally a Presence Sensor, which triggers an event when it comes in and out of range of the hub. It’s quite small, and Samsung suggests uses such as attaching it to your keying or leaving it in your car, so that the system knows when you are home (useful for people without geolocatable smartphones), or attaching to your pets’ collars.

All of these are tied together with very rich App, but I’ll cover that in more depth in a future column.

4 Likes

And I just know that @JDRoberts is going to pick me up on that bit :wink:

2 Likes

Zigbee is better for some use cases, zwave is better for others. Both are good low energy mesh protocols. :sunglasses: See:

http://thingsthataresmart.wiki/index.php?title=Z-wave_versus_Zigbee