If you are directly constructing the capabilty from JSON, you also need to add it to the capability lookup table. The build command isn’t really intended to be a public interface, and was just used as a workaround before we had the correct definition syncing in place. Unfortunately there was a case that was missed with that syncing which is why a hub reboot is required after a device is joined to get that working. This is obviously a bug and work is ongoing on a fix. In the meantime if you use the build command directly, you also need the line:
local capabilities = require "st.capabilities"
capabilities["namespace.customCapability"] = capabilities.build_cap_from_json_string(custom_cap)
otherwise when the command is received with the ID of your custom capability it is going to try to look it up in the directory.