HAL Testing and Fibers next migration

[rslater-cs]

What type of PR is this? (check all applicable)

• 🍕 Feature
• 🐛 Bug Fix
• 📝 Documentation Update
• 🎨 Style
• 🧑‍💻 Code Refactor
• 🔥 Performance Improvements
• ✅ Test
• 🤖 Build
• 🔁 CI
• 📦 Chore (Release)
• ⏩ Revert

Description

This PR is far from done but good to look at as I am making some significant design choices that I would like guidance on

This PR currently has working:

• Core HAL tests:

• • Manager creation and deletion
• • Device creation and deletion
• • Capability creation and deletion
• • Capability endpoint calling
• • Invalid argument tests

• Modem tests:

• • Manager detection of Modem add and remove events
• • Half a test (no assertions but proof by logging) of modem sim warm swap working as expected

Spoofing modems without significant changes to production code has taken a while but I have come up with a good way to have mmcli and qmicli events fed into the modem and modem manager space without the driver needing to know that it is in a test or prod environment.

• The mmcli and qmicli modules now accept an alternative backend as part of their interface.
• We insert a backend that builds fake command objects which have the same interface as the real ones
• Each call for a command first finds a modem object in a fake modem registery and attaches a function onto the command which is called in specific senarios
• The commands also have channels which the tests can write into to emulate streaming data such as modem monitor and sim monitor
• This logic is all interacted with via the modem object in the test scripts

example of how a modem can be added, recieve a sim and then be removed:

local dummy_modem = require 'tests.hal.harness.devices.modem'
local modem = dummy_modem.new(context.with_cancel(ctx))
modem:set_address_index("0")
modem:set_mmcli_information{
  modem = {
      generic = {
          device = "/fake/port0",
          ["equipment-identifier"] = "123456789",
      }
  }
}

-- Make the modem appear on the bus so the manager and driver
-- start up and begin logging.
local wr_err = modem:appear()

sleep.sleep(0) -- yield

local sim_detect_sub = conn:request(new_msg_fn(
      { 'hal', 'capability', 'modem', '123456789', 'control', 'sim_detect' },
      {}
  ))
local _ = sim_detect_sub:next_msg_op():perform()

local sim = dummy_modem.new_sim()
sim:set_imsi("001010123456789")
sim:set_operator("00101", "Test Operator")
modem:insert_sim(sim)

sleep.sleep(0) -- yield

modem:disappear()

This should enable me to make HAL tests that can be interacted with only via the bus and dummy modems rather than the current approach you can see in test_core.lua where I have to slice the service into pieces. Even though this is a draft please let me know if this is a good route or if there is a better alternative you can see.

There are also some modem research files, I don't know where to back them up so they're in here for now, they show the modem output from mmcli and qmicli in many states and through many state machine transitions. You can completely disregard them as I will likely move these to the google drive once done (files under sprint_docs).

The new HAL tests are all undert tests/hal/. The hal tests in tests/ are out of date by about 1 year.

The new HAL tests cannot run from the main tests.lua file so please run them directly.

cd tests/hal
luajit test_core.lua
luajit test_manager_modemcard.lua
luajit test_modem_driver.lua

Related Issues, Tickets & Documents

Screenshots/Recordings

Manual test

• 👍 yes
• 🙅 no

Manual test description

Added tests?

• 👍 yes
• 🙅 no, because they aren't needed
• 🙋 no, because I need help

Added to documentation?

• 📜 README.md
• 🙅 no documentation needed

[optional] Are there any post-deployment tasks we need to perform?

[cpunt]

This looks good overall. Small naming nits to reduce ambiguity (same for qmicli):

• Rename use_backend() -> set_backend() (or override_backend() if you want to emphasize it’s a swap).
• Rename backend -> to something like mmcli_interface

[cpunt]

This isn't working for me:

luajit test_core.lua
FFFF.FFFF.F....FFFFFFFF.FF.F.F.
Failed tests:
-------------
1) TestHalCapabilityControl.test_capability_control_endpoints
test_core.lua:624: Expected to receive HAL capability event message for capability1 add
expected: nil, actual: "timeout"
stack traceback:
        test_core.lua:624: in function 'expect_capability_event'
        test_core.lua:850: in function 'TestHalCapabilityControl.test_capability_control_endpoints'

2) TestHalCapabilityControl.test_invalid_capability_control_endpoints
test_core.lua:624: Expected to receive HAL capability event message for capability1 add
expected: nil, actual: "timeout"
stack traceback:
        test_core.lua:624: in function 'expect_capability_event'
        test_core.lua:904: in function 'TestHalCapabilityControl.test_invalid_capability_control_endpoints'

3) TestHalCapabilityControl.test_no_endpoint_on_removal
test_core.lua:624: Expected to receive HAL capability event message for capability1 add
expected: nil, actual: "timeout"
stack traceback:
        test_core.lua:624: in function 'expect_capability_event'
        test_core.lua:951: in function 'TestHalCapabilityControl.test_no_endpoint_on_removal'

4) TestHalCapabilityControl.test_publish_control
test_core.lua:624: Expected to receive HAL capability event message for capability1 add
expected: nil, actual: "timeout"
stack traceback:
        test_core.lua:624: in function 'expect_capability_event'
        test_core.lua:992: in function 'TestHalCapabilityControl.test_publish_control'

5) TestHalCapabilityInfo.test_info_with_subtopic
test_core.lua:1095: Expected to receive capability info message
expected: nil, actual: "timeout"
stack traceback:
        test_core.lua:1095: in function 'TestHalCapabilityInfo.test_info_with_subtopic'

6) TestHalCapabilityInfo.test_simple_info
test_core.lua:1035: Expected to receive capability info message
expected: nil, actual: "timeout"
stack traceback:
        test_core.lua:1035: in function 'TestHalCapabilityInfo.test_simple_info'

7) TestHalCapabilityInfo.test_tabled_info
test_core.lua:1062: Expected to receive capability info message
expected: nil, actual: "timeout"
stack traceback:
        test_core.lua:1062: in function 'TestHalCapabilityInfo.test_tabled_info'

8) TestHalCapabilityInfo.test_tabled_info_publish_multiple
test_core.lua:1121: Expected to receive capability info message
expected: nil, actual: "timeout"
stack traceback:
        test_core.lua:1121: in function 'TestHalCapabilityInfo.test_tabled_info_publish_multiple'

9) TestHalConfig.test_invalid_config_does_not_affect_managers
test_core.lua:307: Expected to receive dummy manager status message
expected: nil, actual: "timeout"
stack traceback:
        test_core.lua:307: in function 'TestHalConfig.test_invalid_config_does_not_affect_managers'

10) TestHalConfig.test_partial_manager_removal
test_core.lua:368: Expected to receive dummy manager status message
expected: nil, actual: "timeout"
stack traceback:
        test_core.lua:368: in function 'TestHalConfig.test_partial_manager_removal'

11) TestHalConfig.test_reconfigure_manager
test_core.lua:252: Expected to receive dummy manager status message
expected: nil, actual: "timeout"
stack traceback:
        test_core.lua:252: in function 'TestHalConfig.test_reconfigure_manager'

12) TestHalConfig.test_remove_manager
test_core.lua:192: Expected to receive dummy manager status message: timeout
expected: nil, actual: "timeout"
stack traceback:
        test_core.lua:192: in function 'TestHalConfig.test_remove_manager'

13) TestHalConfig.test_simple_config
test_core.lua:67: Expected to receive dummy manager status message
expected: nil, actual: "timeout"
stack traceback:
        test_core.lua:67: in function 'TestHalConfig.test_simple_config'

14) TestHalDeviceCapabilityEvent.test_add_remove_add_capability_event
test_core.lua:624: Expected to receive HAL capability event message for capability1 add
expected: nil, actual: "timeout"
stack traceback:
        test_core.lua:624: in function 'expect_capability_event'
        test_core.lua:797: in function 'TestHalDeviceCapabilityEvent.test_add_remove_add_capability_event'

15) TestHalDeviceCapabilityEvent.test_device_capability_add_event
test_core.lua:624: Expected to receive HAL capability event message for capability1
expected: nil, actual: "timeout"
stack traceback:
        test_core.lua:624: in function 'expect_capability_event'
        test_core.lua:657: in function 'TestHalDeviceCapabilityEvent.test_device_capability_add_event'

16) TestHalDeviceCapabilityEvent.test_device_capability_duplicate_id_event
test_core.lua:624: Expected to receive HAL capability event message for capability1 first add
expected: nil, actual: "timeout"
stack traceback:
        test_core.lua:624: in function 'expect_capability_event'
        test_core.lua:775: in function 'TestHalDeviceCapabilityEvent.test_device_capability_duplicate_id_event'

17) TestHalDeviceCapabilityEvent.test_device_capability_invalid_event
test_core.lua:624: Expected to receive HAL capability event message for valid capability1
expected: nil, actual: "timeout"
stack traceback:
        test_core.lua:624: in function 'expect_capability_event'
        test_core.lua:749: in function 'TestHalDeviceCapabilityEvent.test_device_capability_invalid_event'

18) TestHalDeviceCapabilityEvent.test_device_capability_remove_event
test_core.lua:624: Expected to receive HAL capability event message for capability1 add
expected: nil, actual: "timeout"
stack traceback:
        test_core.lua:624: in function 'expect_capability_event'
        test_core.lua:714: in function 'TestHalDeviceCapabilityEvent.test_device_capability_remove_event'

19) TestHalDeviceCapabilityEvent.test_device_multi_capability_event
test_core.lua:624: Expected to receive HAL capability event message for capability1
expected: nil, actual: "timeout"
stack traceback:
        test_core.lua:624: in function 'expect_capability_event'
        test_core.lua:694: in function 'TestHalDeviceCapabilityEvent.test_device_multi_capability_event'

20) TestHalDeviceEvent.test_device_add_event
test_core.lua:456: Expected to receive HAL device event message
expected: nil, actual: "timeout"
stack traceback:
        test_core.lua:456: in function 'TestHalDeviceEvent.test_device_add_event'

21) TestHalDeviceEvent.test_device_remove_event
test_core.lua:480: Expected to receive HAL device event message
expected: nil, actual: "timeout"
stack traceback:
        test_core.lua:480: in function 'TestHalDeviceEvent.test_device_remove_event'

Ran 31 tests in 0.037 seconds, 10 successes, 21 failures

[rslater-cs]

I made a local change to fibers that allowed sleep(0) to be used as a yield. As you don't have the change it has broken the yielding mechanic and caused timeouts. I have updated so if you remake your env hopefully the tests will now pass

[cpunt]

This seems to fail:

vscode@3a3b31092cf6:/workspaces/devicecode-lua/tests/hal$ luajit test_manager_modemcard.lua
[TRACE 13:55:41] ../../src/services/hal/managers/modemcard.lua:36: Modem Detector: starting...
[DEBUG 13:55:41] ../../src/services/hal/managers/modemcard.lua:69: hal - Modem Detector: line could not be parsed
[TRACE 13:55:41] ../../src/services/hal/managers/modemcard.lua:92: Modem Manager: starting
E
Tests with errors:
------------------
1) TestHalModemcardManager.test_modem_monitor_events
test_manager_modemcard.lua:73: attempt to index local 'result' (a nil value)
stack traceback:
        test_manager_modemcard.lua:73: in function 'TestHalModemcardManager.test_modem_monitor_events'

Ran 1 tests in 0.005 seconds, 0 successes, 1 error