[RELEASED] Beta testers needed for my SmartThings + Augmented Reality (AR) app - Smart AR Home

I am seeing the same issue as @joelw135 on my iPhone 7 Plus running the latest beta version of IOS.

When I open the app, it says: “looking for synchronization point”.
I point it to the picture on the way that I originally used, and I see the app logo superimposed over the picture. The instruction across the bottom say:“Reposition Sync Point”.

It’s a picture on a wall, what is there to reposition?

I see a list of switches in the ST AR smartapp.

I see that I need to re-think the order of the windows and available buttons :slight_smile:

Once you get “app logo superimposed over the picture” you should add devices to you home. Please go to settings → Edit devices → Add device. On the list you should see all devices that you selected when connecting with SmartThings AR smart app. Click “Add device”, choose one of them from the list and then click on the screen to place it in your home. From this point, the device will be there every time your restart the app and scan synchronization point.

Hopefully it’s also visible on the video I shared earlier

Thanks.
I have it working now.
I can see how this app can be helpful in some situations.
I have some thoughts on improvements to the interface.

1 Like

Same as others, my iPhone 6S just freezes on looking for synchronisation point.

Very cool. Got it up and running without any problems on Android Oreo / Samsung Note 8.

It does make a great demo of a smart home


Is there any chance that there could be more than one synchronization point allowed in the app?
It could be very inconvenient if you were somewhere away from the Sync point when you open the app.

EX: The sync point is in my living room. If I am in the garage or the basement, or the bedroom upstairs when I want to use the app, it appears as though I need to go to the living room and point to the sync point before I can control any devices.
If that is the case, being able to set up a few sync points around the home would make the app much more convenient to use.

1 Like

OK installed the new version, same problem. Now let me get something clear.

  1. Center object
  2. Click the button
  3. Wait to see the yellow dots.
  4. Wait until it is complete.
    Now I centered the object, and got the yellow dots, then I tried my best to hold it while it searched. After five minutes nothing happened.

I have a suggestion. Allow us in the app to take a photo with the iPhone camera, then you create a button to choose the photo from the photo roll. Then if it takes an hour then it wouldn’t matter, that we can’t hold the phone on the object. Does that make sense?

Thank you for the great feedback! I replied to your email with other suggestions.

Oh, that’s not good. I will try to get iPhone 6S ASAP and try it myself locally then. One comment. Once you get yellow dots, instead of just holding it, it’s better to move it a little, to the left and right as it finds more dots (feature points) this way and should localize the synchronization point sooner. But of course, if it hangs, then it won’t solve the problem.

You mentioned before “as soon as I move the phone I get a camera view”, so to understand correctly it only hangs when you are pointing the synchronization point/picture, right? If you look away, it’s fine? It’s important as it would mean it’s only related to image processing/feature selections (finding synchronization point).

Regarding taking a photo. It won’t work as even a small hand move will invalidate your relative rotation and position, which means the devices will be completely not aligned with your home. At the beginning I was doing something similar, first finding the synchronization point and then turning on AR Kit tracking. That fraction of a second between these two steps was enough to destroy experience.

–
Thanks again everyone for sending the feedback and suggestions! It’s very useful. I am saving all on my TODO list and I will continue working on it. :slight_smile:

]I have had some progress, but need clarification. I was able to \create a point. I choose some switches. But as soon as I exit the app on loading it says looking for sync again, and if the light is slightly different it can’t find it. What am I doing wrong?

Hi BB,

I have not read much past your initial post. Apologies.

I don’t really need AR as much as just a pseudo-3D I can swivel and turn. If needed, I could use my PC to assign 3D points, then use my 2D phone app to swivel and turn this.

To over simplify, all I personally want is a way to see all the temperatures inside and outside my house, when I have about two dozen devices (and 100 measures), on a house with 4 levels (basement to attic) and also outside, to four directions of compass.

It would also be cool for quick keys to focus on:

  • Basement / 1st / 2nd / attic floor etc
  • North / South / East / West alignment

From a particular viewpoint.

I am no game programmer, but this is your very basic 3D rotatable visualization using a 2D (phone or web) interface.

This would be great, and doesn’t need AR goggles.

For the record, there are so many things I wish “smart house” ecosystems had. Things like, comparing energy usage over time versus local weather and cloud cover - really complex things that not a single one has come close to covering.

But if you could make a simple 3D rotatable system using some readily available system like maybe TinkerCAD or SketchUp, that would be great.

I think many of us loss leaders here have found smart home tech to be much more trouble than pay off. But maybe you can help make it be cool.

Thanks!

I think you are doing it correctly. It’s just iPhone generation. I need to get 6S and debug the code myself to know for sure.

I think I have one idea that might be what you are looking for. I started working on it and hopefully soon it will be part of one of the next updates. You will get some kind of a 3D view on your house as an option to AR and walking in the house. It should improve practical part of the app. Stay tuned! :slight_smile:

OK standing by.

@BinaryBanana

My setup always stuck at Looking for device,Please wait .
iPhone X + iOS 11.3

It usually means that the app couldn’t connect with Smart Things. It might be caused by a typo in the client ID/secret value. I should probably add some error message for those cases.

Please check the client ID/secret values and try again. I hope it will solve the issue. I will add to my TODO list some error messaging. Thanks for reporting the problem!

@BinaryBanana
But this is after I type client ID/secret values and have authorized wit SmartThings and also select few device, I can also see the SmartApp in SmartThing been generated been selected with the device I have chose.
I have another iPhone 7 with same iOS 11.3. same failure


You are right. If you got through device selection, your Client ID value is correct. Your Client Secret is used later though, at the page you got stuck, so that’s still can be a potential root cause. Otherwise it would mean I have a nasty bug in the code, the one that doesn’t happen for most people.

If you don’t mind sharing, are you localized in US? There is a list of endpoints to call during authorization handshake, maybe I am using incorrect one if people are localized outside of US.

I definitely need better error handling/messages so people will know what’s going on.

@BinaryBanana

I finally make it work by create everything from Scratch, and It work fine, and I have some question here
.
1.Every time I want to use this AR to turn on or off of the SmartThings device, I have to re-sync synchronization point before I find all the objects I have create ?, Is that true ?

2.If the re-sync is not perfect, and all the object we create may not at its location we have create for them,Is any way we could let system remember the relative location from each device to the Sync point rather than re-sync everything

3.It is interesting to use this AR, and I need to spend some time to think if any feature could add to make this solution more powerful, so if any bugs, I will let you know