vscode-extension-tester: [Bug] `npm run ui-test` starts a shared VSCode instance, not running test files
Describe the bug
- I’ve setup vscode-extension-tester[v5.8] in my repo
npm run ui-test
- VSCode opens inside a framed window
- Nothing happens
Console output
➤ npm run ui-test
> env-switcher@0.4.1 ui-test
> extest setup-and-run './out/ui-test/*.test.js' --code_version 1.71.0 --mocha_config './src/ui-test/setup/.mocharc.json' --extensions_dir '.vscode-test/extensions' --storage '.vscode-test/resources' --code_settings ./src/ui-test/setup/settings.json
WARNING: You are using the outdated VSCode version '1.71.0'. The latest stable version is '1.80.2'.
Downloading VSCode: 1.71.0 / stable
VS Code exists in local cache, skipping download
ChromeDriver 102.0.5005.61 exists in local cache, skipping download
Executing prepublish script 'npm run vscode:prepublish'...
> env-switcher@0.4.1 vscode:prepublish
> npm run compile
> env-switcher@0.4.1 compile
> tsc -p ./
DONE Packaged: /[redacted]/vscode-env-switcher/env-switcher-0.4.1.vsix (28 files, 387.21KB)
Installing extensions...
(node:5079) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
(Use `code --trace-deprecation ...` to show where the warning was created)
Extension 'env-switcher-0.4.1.vsix' was successfully installed.
Loading mocha configuration from ./src/ui-test/setup/.mocharc.json
LOADED TESTS
Detected user defined code settings
Writing code settings to /[redacted]/vscode-env-switcher/.vscode-test/resources/settings/User/settings.json
Launching browser...
Selenium Manager binary found at /[redacted]/vscode-env-switcher/node_modules/selenium-webdriver/bin/linux/selenium-manager
Note the LOADED TESTS
, it’s a console.log I’ve added to the testing file, which is based on statusBar-test.ts
in example repo:
import { expect } from 'chai';
import { StatusBar, VSBrowser } from 'vscode-extension-tester';
console.log(`LOADED TESTS`);
describe('Example status bar tests', () => {
let statusBar: StatusBar;
before(async () => {
console.log(`STARTING TESTS`);
// ...
});
it('Generic items', async () => {
console.log(`STARTING TESTS`);
// ...
});
});
When running npm run ui-test
, none of the other console logs are logged. The window is interactable, and I’ve opened a folder in it as that’s what the before
was supposed to do. When closing the window and rerunning it again with npm run ui-test
it will open the same folder and all changes to settings will persist.
I’ve tried clearing anything related to out
/test-resources
/test-extensions
/node_modules
/npm cache clean --force
.
I’ve tried with the latest VSCode version (1.80.2) and I’ve tried downgrading to the vscode-extension-tester@5.7.1
so it will be the same as in the sample project, separately and together.
I’ve tried to align it as much as I can to the sample project, including the resources and output directories.
When running tests in the sample project, VSCode window will be opened without a frame and the logs are different:
Detected user defined code settings
Writing code settings to /[redacted]/vscode-extension-tester/sample-projects/helloworld-sample/test-resources/settings/User/settings.json
Launching browser...
Browser ready in 4552 ms
Launching tests...
Expected behavior
- To run the tests
- To be able to debug the tests
Environment information
- OS:
- Windows - WSL2 with WSLg[Ubuntu 20.04]
- Linux
- macOS
- VSCode version: host[1.75.1], test[1.71.0 and 1.80.2]
- ExTester version: 5.8 and 5.7.1
- Node (npm) version: 16.13.0
About this issue
- Original URL
- State: closed
- Created a year ago
- Comments: 15 (5 by maintainers)
@EcksDy there is not manual mapping of vscode <=> chromedriver
There is mechanism which finds appropriate chromium/chromedriver version based on manifest provided by MIcrosoft for each release of VS Code.
We are supporting last 5 versions from another reason, which is more about complexity of vscode-extension-tester and resource we have to maintain this tool. So the support for last 5 releases means we are aiming to have vscode-extension-tester APi stable and working always with 5 last stable VS Code releases, we do not have resources to ensure better backwards compatibility at this moment.
Hopefully I have answered your question 🙂
Hello, this issue is an side effect of dependency updates in currect vscode-extension-tester release. I will provide fix and next release ASAP.
As workaround you can locally rever changes in
package-lock.json
file which are made when installing latest vscode-extension-tester. The changes whereselenium-webdriver
has newer version then 4.10.x needs to be reverted.@sergiuilie works for me! thanks