playwright-testing-library: Expect assertions can't be used due to returning ElementHandles instead of Locators

Hey, @ScubaDaniel thanks for creating an issue. The reason this library uses ElementHandles is that Locators did not exist when it was originally created. We’ve been working on a new API that returns Locators and leverages more @playwright/test features, there are just a few things I still need to sort out before we release it officially.

yarn add -D @playwright-testing-library/test@4.3.0-beta.1

I went ahead and released a beta version for you here. The readme hasn’t been updated there yet, though, so here’s a snippet from one of my commits with a basic usage example:

This will likely replace the fixtures that provided ElementHandle-based queries in a future major release, but for now the Locator queries are exported as locatorFixtures:

import { test as baseTest } from '@playwright/test'
import {
  locatorFixtures as fixtures,
  LocatorFixtures as TestingLibraryFixtures,
  within
} from '@playwright-testing-library/test/fixture';

const test = baseTest.extend<TestingLibraryFixtures>(fixtures);

const {expect} = test;

test('my form', async ({queries: {getByTestId}}) => {
  // Queries now return `Locator`
  const formLocator = getByTestId('my-form');

  // Locator-based `within` support
  const {getByLabelText} = within(formLocator);

  const emailInputLocator = getByLabelText('Email');

  // Interact via `Locator` API 🥳
  await emailInputLocator.fill('email@playwright.dev');

  // Assert via `Locator` APIs 🎉
  await expect(emailInputLocator).toHaveValue('email@playwright.dev');
})

You can also reference the tests for more examples.

Let me know if you have any issues with that release, and feel free to provide any feedback in the PR → https://github.com/testing-library/playwright-testing-library/pull/403

Ah, one other thing is that the configure API is not yet implemented, but I have it mostly there, so let me know if you need that.

@gajus just tried it out and this is what I’m seeing:

It looks good to me…? Let me know if I’m missing something.

Nice, thanks for the update and the snippet @gajus. I think there’s still value to this library in order to provide Testing Library parity in Playwright, but I’ll probably add something to the readme pointing out the role selector as an alternative.

I like to be pretty selective when considering Proxy, but this is a clever/convenient use case. Do you use TypeScript? Were you able to reliably augment/extend the Page type with your custom methods?

@ScubaDaniel heh, you had me worried for a second there as the *ByRole queries are the most important part of Testing Library imo — the issue you’re coming up against is that I don’t think the findBy* queries really make sense with Locator’s and the underlying selector engine API that we use to implement the Locator-based queries doesn’t support asynchronous calls.

You can see my TODO here… 😬 https://github.com/testing-library/playwright-testing-library/blob/beta/test/fixture/locators.test.ts#L151

I think there’s two action items here:

1. Figure out the asynchronous story for @playwright/test + our locator queries
2. Clearly document this so it’s not confusing that findBy* doesn’t exist

In looking back at your previous comment, it seems like you ended up looking for the findBy* methods after running into this problem trying to do it the Playwright way.

I’ll try to take a look at this issue, as I was planning on supporting the findBy* cases using @playwright/test’s built-in asynchronous support. Hopefully, once this is resolved, you should have no need for the findBy* queries.

@crazyvan25 I suspect your issue is actually related to this, I’ll let you know when I have a chance to dig into this stuff.

@sebinsua bleh, a dependency upgrade busted the second publish step, but I thought it went through on alpha. I manually published the Plawright Test package for beta.3, but I’ve now reverted the offending upgrade so it should be good going forward.

@jrolfs Strangely the alpha seemed to be published under the package name playwright-testing-library only. Any ideas why? I’ll switch to the beta now anyway.

I’m using it. It’s working great and I don’t have any additional feedback right now.

Also, I’m happy to help out with PRs and things if you’d like that.

In case y’all missed it, I published the findBy*/findAllBy* stuff on v4.4.0-alpha.1 a few days ago — I finally sorted the Semantic Release stuff, so it’s also out on v4.4.0-beta.3. I’ll probably delete the alpha release to prevent confusion.

@ScubaDaniel I know we co-opted your issue with all of this discussion, so I apologize for all the noise. If you’re still interested, you can try out the find queries with the aforementioned beta release.

@sebinsua let me know when you get a chance to try this stuff out. I’ll probably get the screen idea implemented as the last feature on the beta release channel. Unless you have additional feedback, I just have one last little thing I want to address before releasing this stuff officially on 4.x (I want to improve the error message for the findBy case that fails when an element is hidden. Right now the hard-coded 100ms timeout error will be confusing).

After that, I’m planning on putting together a 5.0 release that consolidates both the @playwright/test and playwright use cases onto the new Locator stuff. All of the different APIs are pretty confusing right now, and as a bonus it will clean up the code considerably. I’ll probably throw together a quick codemod for preserving ElementHandle behavior via locator.elementHandle() to ease migration.

Alright, findBy* queries are most of the way there in #488 if you wanna take a look. I stuck with Playwright’s waitFor and got our getBy* error idea working — I think using Playwright’s asynchronous utilities is ideal, if possible (as opposed to the waitFor from Testing Library).

One problematic thing I’ve found is that locator.waitFor() errors if the Locator has matched multiple elements. This causes a strict-mode violation error (see the strictness section in the docs).

Shit, I forgot about this. This kinda makes me want to try another avenue I was considering… essentially implementing the findBy* the same way that @testing-library/dom does with the waitFor from @testing-library/dom. The other reason I was considering this was to surface the Testing Library error instead of the Playwright locator.waitFor() timeout message (~I was also considering catching the timeout error and then running a final getBy* to produce the error~ lol, oops I totally missed you mentioned this exact thing).

Sound reasonable. Unless we just make it use the Playwright waitFor options? Do you know if there is a way to default it from Playwright’s own timeout configuration values?

Hrm, so you did get me thinking that perhaps the Playwright defaults would make the most sense here, but unfortunately, I don’t see any way of reading from the Playwright Test configuration in the documentation. Also, I do still want to keep supporting “vanilla” (sans @playwright/test) Playwright in some capacity going forward. I think I’ll probably stick with an independent asyncUtilTimeout value.

Unrelated to this, but because we don’t have a waitForElementToBeRemoved function, I’ve been implementing this in Playwright with the following pattern await expect(queryAllBy*(…)).toHaveCount(0). This works however I ran into a confusing problem in which I was using within(parentLocator).queryAllByLabelText(‘Label’) and waiting for the count to reach 0 but it wasn’t working as parentLocator had been created with a getBy* and therefore threw an error once there were 0 elements. I had to fix this by making sure that all parent locators are created with queryBy. (We don’t need to fix this right now but this is an example of a semantic mismatch between Playwright and Testing Library that makes the DX confusing.)

Do you think implementing waitForElementToBeRemoved would sidestep this sufficiently? I’ll have to sit with this a bit, but let me know if you think there’s something you think we should change with the within() implementation/behavior.

One other thing – is there anyway that we could allow users to configure getElementError? I’m finding the length of the error messages incredibly long as it prints out a bunch of HTML and CSS and I have to scroll up a long way to find the actual error message.

Okay, so this is something I’ve wanted to address since I created this library. I’d be curious if you have an idea of what a better default implementation/output would be. I do think we can add support for this option, but I really think we also need a better default because the e2e use case just always produces so much output. I also want to figure out how to preserve the ANSI formatting in the Testing Library errors.

---

Thank you so much for all of this feedback!

The main project I use this library on is still on the ElementHandle version (and jest-playwright). We do have one other project at my company that’s using @playwright/test and the more modern version of this library, but it’s just smoke tests and hasn’t seen a lot of action in a bit.

Note that this library is mostly redundant now that Playwright added ByRole selectors.

https://playwright.dev/docs/selectors#role-selector

If helpful, this is how we implemented it:

export const test = base.extend<{ page: Page }>({
  page: async ({ page: playwrightPage }, use, testInfo) => {
    const revocable = Proxy.revocable(page, {
      get(target, property, receiver) {
        if (property === 'findByRole') {
          return (role: string, name: RegExp | string) => {
            return target.locator(
              `role=${role}[name=${
                typeof name === 'string' ? '"' + name + '"' : String(name)
              }]`
            );
          };
        }

        return Reflect.get(target, property, receiver);
      },
    });

    await use(revocable.proxy);

    revocable.revoke();
  },
});

and then use it like any other selector page.findByRole('button', 'Sign Up'), etc.

@jrolfs Sorry to bug you but how far away from an official release are you, and what version of the package contains all of these changes?

I need to write some E2E tests soon, and I’m trying to weigh up whether I produce a greater technical debt by using a beta package, or by continuing to build upon the custom selector engine I originally created – if I continue to use the latter, I’ll need to rewrite all of my tests once this is released.