webdriverio: command --watch not working as expected

Discussed in https://github.com/webdriverio/webdriverio/discussions/12024

<div type='discussions-op-text'>

Originally posted by troyyer January 16, 2024 I am having an issue trying to use the ‘–watch’ feature (https://webdriver.io/docs/watcher) i saw this bug item https://github.com/webdriverio/webdriverio/issues/9612 but was unsure if it has been resolved as yet

Currently when i run a website test script using the Test Runner and with the watch command enabled, once the test has completed, the browser window will stay open (as expected), then when I try to update the test script file and after saving the file i get an error in the Terminal

2024-01-15T23:01:14.917Z INFO @wdio/local-runner: Start worker 0-0 with arg: run,./wdio.conf.js,--watch
[0-0] 2024-01-15T23:01:16.512Z INFO @wdio/local-runner: Run worker command: run
[0-0] 2024-01-15T23:01:17.665Z ERROR @wdio/runner: Error: You are trying to attach to a protocol stub, this should never occur, please file an issue.
[0-0]     at ProtocolStub.attachToSession (file:///C:/code/sample/node_modules/webdriverio/build/protocol-stub.js:31:19)
[0-0]     at attach (file:///C:/code/sample/node_modules/webdriverio/build/index.js:70:27)        
[0-0]     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
[0-0] RUNNING in chrome - file:///C:/code/sample/test/specs/test.e2e.js
[0-0] 2024-01-15T23:01:17.985Z INFO webdriver: COMMAND navigateTo("https://the-internet.herokuapp.com/dynamic_loading")
[0-0] 2024-01-15T23:01:17.986Z INFO webdriver: [POST] http://0.0.0.0:52056/session/f8d10edf46f9c1a82a570da029789b4a/url
[0-0] 2024-01-15T23:01:17.986Z INFO webdriver: DATA { url: 'https://the-internet.herokuapp.com/dynamic_loading' }
[0-0] FAILED in chrome - file:///C:/code/sample/test/specs/test.e2e.js
2024-01-15T23:01:18.164Z INFO @wdio/cli:launcher: Run onWorkerEnd hook

the reproducable sample is a 2 step process actually. for browser based testing my repo will do which i updated the package json to have an additional script https://github.com/troyyer/wdiosampleforwaituntil after installation

npm run watch

once the first test passed edit the test script in ‘/test/specs/test.e2e.js’ by making line 13 to read

it.only("tests dynamic loading", async () => { after saving, watch will rerun the file and the terminal will show the error and will not re-execute the test

rerun with wdio watch flag webtest

I also ran it for component testing i ran the cli installer with component testing

 npm init wdio . 

with the configurations below

===============================
🤖 WDIO Configuration Wizard 🧙
===============================

? What type of testing would you like to do? Component or Unit Testing - in the browser
    > https://webdriver.io/docs/component-testing
? Which framework do you use for building components? Lit (https://lit.dev/)
? Which framework do you want to use? Mocha (https://mochajs.org/)
? Do you want to use a compiler? No!
? Do you want WebdriverIO to autogenerate some test files? Yes
? What should be the location of your spec files? C:\code\wdio-component\src\**\*.test.js
? Which reporter do you want to use? spec
? Do you want to add a plugin to your test setup?
? Do you want to add a service to your test setup?

dependencies picked up by the cli installation

    "@wdio/browser-runner": "^8.27.2",
    "@wdio/cli": "^8.27.2",
    "@wdio/mocha-framework": "^8.27.2",
    "@wdio/spec-reporter": "^8.27.2",
    "lit": "^3.1.1"

after install run wdio with watch on

npm run wdio -- -- --watch

after the initial run has completed, go into file “{ProjectRoot}\src\Component.test.js” edit line 20 to now read await expect(button).not.toHaveText('count is 2') after saving the file i get an error in the Terminal rerun with wdio watch flag

About this issue

  • Original URL
  • State: open
  • Created 3 years ago
  • Reactions: 3
  • Comments: 18 (14 by maintainers)

Most upvoted comments

Hi @tech-dm-klymenko! Thank you for your reply and for sharing the possible timelines! We will be working forward to the fix

Meanwhile, I have one observation I wanted to share, just in case it could be of any help

Once received that failure, I went into the code from the error stack trace and debugged it. I’ve noticed a particular place in the attach function

So in this function, there is a call to the getProtocolDriver, which returns a driver for the protocol stub. Subsequently, the Driver.attachToSession function is called, which fails with the failure described in the beginning of this discussion

Whatsoever, if I change the code locally by assigning SupportedAutomationProtocols.webdriver as the automationProtocol parameter in the end of the params list (thus overwriting its value from the attachOptions), the failure does not appear:

const params: Capabilities.WebdriverIOConfig & { requestedCapabilities: Capabilities.RequestedStandaloneCapabilities } = {
        ...attachOptions,
        ...detectBackend(attachOptions.options),
        capabilities: attachOptions.capabilities || {},
        requestedCapabilities: attachOptions.requestedCapabilities || {},
        automationProtocol: SupportedAutomationProtocols.webdriver
    }

If I do it like this, the webdriverio Protocol driver is returned by the getProtocolDriver function later on, which then makes the spec run in the watch mode without a failure

However, having read the conversation above, I understand that the fix is likely to be more complicated than this. Anyway, I hope this observation can be of any use for you when implementing the fix 😃

Hi @vlad9719 Thanks for asking! I’ve already started working on the fix. However, I have a heavy workload until November 5th, so I won’t be able to work on the task frequently before then. After the 5th, I’ll have more time to focus on it in detail. Considering the number of nuances that need to be addressed, probably the fix won’t be finished very quickly. A rough estimate would be early November in the best case, and the second half of November as the most realistic timeframe.

I continue to keep that task in mind, but I am currently working on the Windows platform. On this platform, the project periodically builds with errors. Once the project starts building consistently, I will continue working on the task.

Yes, locally I have already started working on this task, approaching it in a similar manner to your description. I’ve tried to preserve the existing structure as much as possible. But if you say that we can extract the logic from packages/wdio-utils/src/node into a separate package, then I will do that. And I also agree that using sockets would be more convenient than an IPC channel.

So I’m ready to take on this task and try to implement it.

Sorry for the delayed response.

I like the concept of establishing a driver manager. This will be a more comprehensive and convenient solution in the long run.

I totally agree. A @wdio/driver package could be initiated by the @wdio/cli launcher, would run in a separate process, would expose a socket connection that CLI and worker can connect to and would allow to start driver based on certain capabilities. We could connect a certain driver with a cid so we would know rerunning the test which driver should be picked. We would move all the logic from packages/wdio-utils/src/node into that package and implement a minimal socket interface to communicate with the driver instance.

These are just some initial ideas, wonder what your thoughts are! Would you like to take a stab at this kind of effort?