How do I run integration tests?

Hello,

I would like to run an existing integration test code to understand how it works and write a new test code, but I am unsure of how to do so. When I attempted to run the test code, I received error messages, as shown below.

C:\Users\x\Documents\GitHub\SmartThingsEdgeDrivers\drivers\SmartThings\zigbee-window-treatment\src>lua .\test\test_zigbee_window_treatment.lua
Running test “State transnsition from opening to partially open” (1 of 9)

Received unexpected error: module ‘init’ not found:
no field package.preload[‘init’]
no file ‘C:\lua\lua_libs-api_v3\init.lua’
no file ‘C:\lua\lua_libs-api_v3\init\init.lua’
no file ‘..lua’
no file ‘.\init\init.lua’
no file ‘’
no file ‘C:\lua\bin\init.dll’
no file ‘C:\lua\bin..\lib\lua\5.4\init.dll’
no file ‘C:\lua\bin\loadall.dll’
no file ‘.\init.dll’
stack traceback:
[C]: in function ‘require’
[C]: in function ‘xpcall’
C:\lua\lua_libs-api_v3\integration_test\init.lua:226: in upvalue ‘run_configured_test’
C:\lua\lua_libs-api_v3\integration_test\init.lua:342: in function ‘integration_test.run_registered_tests’
.\test\test_zigbee_window_treatment.lua:283: in main chunk
[C]: in ?
FAILED

Passed 0 of 9 tests

Hi, @Dalgona13
Did you set up the environment for this? Here are the instructions: Set Up Your Development Environment | Developer Documentation | SmartThings

Integration tests run the code locally using the ST Lua libraries you download from here: Releases · SmartThingsCommunity/SmartThingsEdgeDrivers · GitHub

And, you need to set up the LUA_PATH correctly.

Also, just to confirm that you’re on the right path (which based on the logs you shared it seems correct):

  1. Test files have to be executed from the driver src directory. This is because when executing on the Hub, that is the environment it is in, and the driver is required/assumed to start from an init.lua in the current directory.
  2. So for the above example you would need to :

cd \original\SmartThingsEdgeDrivers\drivers\SmartThings\ zigbee-window-treatment\src
lua test/test_zigbee_window_treatment.lua

@nayelyz , could you please ask devs to include a return to integration_test.run_registered_tests() ?
Something like so:

Modified version

--- Run all tests that have been registered printing logs and PASS/FAIL status of the test
function integration_test.run_registered_tests()
  local passed_tests = 0
  local failed_tests = 0 -- ADDED
  for i, test_config in ipairs(registered_tests) do
    local test_run_line = string.format("Running test \"%s\" (%d of %d)", test_config.test_name, i, #registered_tests)
    print(test_run_line)
    print(string.rep("-", #test_run_line))
    if test_config.test_init ~= nil then
      test_config.test_init()
    elseif integration_test.test_init_func ~= nil then
      integration_test.test_init_func()
    end
    if run_configured_test(test_config.test_func, test_config.expected_error) then
      passed_tests = passed_tests + 1
      print("PASSED")
    else
      failed_tests = failed_tests + 1 -- ADDED
      print("FAILED")
    end
    integration_test.reset_tests()
    print("\n")
  end
  print("Passed " .. passed_tests .. " of " .. #registered_tests .. " tests")

  return { -- ADDED THIS BLOCK
    passed = passed_tests,
    failed = failed_tests,
    total = #registered_tests,
  }
end

Original

--- Run all tests that have been registered printing logs and PASS/FAIL status of the test
function integration_test.run_registered_tests()
  local passed_tests = 0
  for i, test_config in ipairs(registered_tests) do
    local test_run_line = string.format("Running test \"%s\" (%d of %d)", test_config.test_name, i, #registered_tests)
    print(test_run_line)
    print(string.rep("-", #test_run_line))
    if test_config.test_init ~= nil then
      test_config.test_init()
    elseif integration_test.test_init_func ~= nil then
      integration_test.test_init_func()
    end
    if run_configured_test(test_config.test_func, test_config.expected_error) then
      passed_tests = passed_tests + 1
      print("PASSED")
    else
      print("FAILED")
    end
    integration_test.reset_tests()
    print("\n")
  end
  print("Passed " .. passed_tests .. " of " .. #registered_tests .. " tests")
end

So I don’t need to parse the output.

I have this workflow that may help you to understand all steps…

In my case, it loads capabilities on demand, so it requires ST CLI installed.
It executes unit tests in different versions. (46 BETA, 46, 47 BETA)
Also, I use NodeJS to generate some codes automatically.
So you may remove all unwanted steps for your case just to understand the flow.

Basically:

# Step Status
1 Have lua 5.3 installed :question:
2 Have ST API downloaded :question:
3 Have LUA_PATH defined properly :question:
4 Go to <driver>/src :green_circle:
5 Execute lua test/<filename>.lua :green_circle:

It seems you are trying to load module called “init” which doesn’t exist.
Usually, it is the folder name or file name.
For example:

local test = require "integration_test" -- folder name
local utils = require "integration_test.utils" -- file name

Thanks for your help!

I have solved it, but got new error messages as below. Could you help me again.

\SmartThingsEdgeDrivers\drivers\SmartThings\zigbee-window-treatment\src>lua .\test\test_zigbee_window_treatment.lua
Running test "State transnsition from opening to partially open" (1 of 9)
-------------------------------------------------------------------------
WARN  || Unexpected filesystem lookup for capability switch
Received unexpected error: C:\lua\lua_libs-api_v3\dkjson.lua:397: bad argument #3 to 'strfind' (number expected, got string)
stack traceback:
        [C]: in function 'string.find'
        C:\lua\lua_libs-api_v3\dkjson.lua:397: in upvalue 'scanwhite'
        C:\lua\lua_libs-api_v3\dkjson.lua:553: in function <C:\lua\lua_libs-api_v3\dkjson.lua:551>
        (...tail calls...)
        C:\lua\lua_libs-api_v3\st\capabilities\init.lua:42: in upvalue 'inline_type_schema'
        C:\lua\lua_libs-api_v3\st\capabilities\init.lua:47: in upvalue 'inline_type_schema'
        C:\lua\lua_libs-api_v3\st\capabilities\init.lua:47: in upvalue 'inline_type_schema'
        C:\lua\lua_libs-api_v3\st\capabilities\init.lua:61: in function 'st.capabilities.build_cap_from_json_string'
        C:\lua\lua_libs-api_v3\st\capabilities\init.lua:117: in metamethod 'index'
        C:\lua\lua_libs-api_v3\st\zigbee\defaults\init.lua:28: in main chunk
        [C]: in function 'require'
        .\init.lua:17: in main chunk
        [C]: in function 'require'
        [C]: in function 'xpcall'
        C:\lua\lua_libs-api_v3\integration_test\init.lua:226: in upvalue 'run_configured_test'
        C:\lua\lua_libs-api_v3\integration_test\init.lua:342: in function 'integration_test.run_registered_tests'
        .\test\test_zigbee_window_treatment.lua:283: in main chunk
        [C]: in ?
FAILED

You probably are using a custom capability.

It would help if you could provide the unit test code and the capability.

Indeed, @Dalgona13. I don’t see a reference to any switch capability in the original test file, so, as @w35l3y, it would be useful to see the portion of your code that is causing this issue.

BTW, @w35l3y, I shared your request with the team to see if they can apply the change.

1 Like

Thanks for your comments,
However, I have run an existing test code(test_zigbee_window_treatment.lua) that is included in SmartThings drivers and I know that it doesn’t use custom capabilities.

I have not made any modifications to the test_zigbee_window_treatment.lua file. It is the original file.

Ok.
Understood.
Good luck.

Make sure you are using Lua 5.3 There were changes in 5.4 that are not compatible with the SmartThings Lua libraries.

Thank you so much!
You are right, It has been solved by installing Lua 5.3.

I just saw v6_49 implemented my request to make integration_test.run_registered_tests() to return summary object with the result of the tests.

Please, let the team know I am very grateful for this.

This is the first test that worked without having to override lua_libs-api_v6/integration_test/init.lua

2 Likes