195 lines
8.3 KiB
Markdown
195 lines
8.3 KiB
Markdown
# End-to-end tests
|
|
|
|
Grafana Labs uses a minimal [homegrown solution](../../e2e/utils/index.ts) built on top of [Cypress](https://cypress.io) for its end-to-end (E2E) tests.
|
|
|
|
Important notes:
|
|
|
|
- We generally store all element identifiers ([CSS selectors](https://mdn.io/docs/Web/CSS/CSS_Selectors)) within the framework for reuse and maintainability.
|
|
- We generally do not use stubs or mocks as to fully simulate a real user.
|
|
- Cypress' promises [don't behave as you might expect](https://docs.cypress.io/guides/core-concepts/introduction-to-cypress.html#Mixing-Async-and-Sync-code).
|
|
- [Testing core Grafana](e2e-core.md) is different than [testing plugins](e2e-plugins.md)—core Grafana uses Cypress whereas plugins use [Playwright test](https://playwright.dev/).
|
|
|
|
## Framework structure
|
|
|
|
Our framework structure is inspired by [Martin Fowler's Page Object](https://martinfowler.com/bliki/PageObject.html).
|
|
|
|
- **`Selector`**: A unique identifier that is used from the E2E framework to retrieve an element from the browser
|
|
- **`Page`**: An abstraction for an object that contains one or more `Selector` identifiers with the `visit` function to go to the page.
|
|
- **`Component`**: An abstraction for an object that contains one or more `Selector` identifiers but without the `visit` function
|
|
- **`Flow`**: An abstraction that contains a sequence of actions on one or more `Page` abstractions that can be reused and shared between tests
|
|
|
|
## Basic example
|
|
|
|
Let's start with a simple [JSX](https://reactjs.org/docs/introducing-jsx.html) example containing a single input field that we want to populate during our E2E test:
|
|
|
|
```jsx
|
|
<input className="gf-form-input login-form-input" type="text" />
|
|
```
|
|
|
|
It is possible to target the field with a CSS selector like `.gf-form-input.login-form-input`. However, doing so is a brittle solution because style changes occur frequently.
|
|
|
|
Furthermore, there is nothing that signals to future developers that this input is part of an E2E test. At Grafana, we use `data-testid` attributes as our preferred way of defining selectors. See [Aria-Labels vs data-testid](#aria-labels-vs-data-testid) for more details.
|
|
|
|
```jsx
|
|
<input data-testid="Username input field" className="gf-form-input login-form-input" type="text" />
|
|
```
|
|
|
|
The next step is to create a `Page` representation in our E2E framework. Doing so glues the test with the real implementation using the `pageFactory` function. For that function we can supply a `url` and selector like in the following example:
|
|
|
|
```typescript
|
|
export const Login = {
|
|
// Called via `Login.visit()`
|
|
url: '/login',
|
|
// Called via `Login.username()`
|
|
username: 'data-testid Username input field',
|
|
};
|
|
```
|
|
|
|
In this example, the selector is prefixed with `data-testid`. The prefix is a signal to the framework to look for the selector in the `data-testid` attribute.
|
|
|
|
The next step is to add the `Login` page to the `Pages` export within [_\<repo-root>/packages/grafana-e2e-selectors/src/selectors/pages.ts_](../../packages/grafana-e2e-selectors/src/selectors/pages.ts) so that it appears when we type `e2e.pages` in your IDE.
|
|
|
|
```typescript
|
|
export const Pages = {
|
|
Login,
|
|
…,
|
|
…,
|
|
…,
|
|
};
|
|
```
|
|
|
|
Now that we have a page called `Login` in our `Pages` const, use it to add a selector in our HTML as shown in the following example. This page really signals to future developers that it is part of an E2E test.
|
|
|
|
Example:
|
|
|
|
```jsx
|
|
import { selectors } from '@grafana/e2e-selectors';
|
|
|
|
<input data-testid={selectors.pages.Login.username} className="gf-form-input login-form-input" type="text" />;
|
|
```
|
|
|
|
The last step in our example is to use our `Login` page as part of a test.
|
|
|
|
- Use the `url` property whenever you call the `visit` function. It is equivalent to the [`cy.visit()`](https://docs.cypress.io/api/commands/visit.html#Syntax) in Cypress.
|
|
- Access any defined selector from the `Login` page by invoking it. This is equivalent to the result of the Cypress function [`cy.get(…)`](https://docs.cypress.io/api/commands/get.html#Syntax).
|
|
|
|
```typescript
|
|
describe('Login test', () => {
|
|
it('passes', () => {
|
|
e2e.pages.Login.visit();
|
|
// To prevent flaky tests, always do a `.should` on any selector that you expect to be in the DOM.
|
|
// Read more here: https://docs.cypress.io/guides/core-concepts/retry-ability.html#Commands-vs-assertions
|
|
e2e.pages.Login.username().should('be.visible').type('admin');
|
|
});
|
|
});
|
|
```
|
|
|
|
## Advanced example
|
|
|
|
Let's take a look at an example that uses the same selector for multiple items in a list for instance. In this example app, there's a list of data sources that we want to click on during an E2E test.
|
|
|
|
```jsx
|
|
<ul>
|
|
{dataSources.map(({ id, name }) => (
|
|
<li className="card-item-wrapper" key={id}>
|
|
<a className="card-item" href={`datasources/edit/${id}`}>
|
|
<div className="card-item-name">{name}</div>
|
|
</a>
|
|
</li>
|
|
))}
|
|
</ul>
|
|
```
|
|
|
|
Like in the basic example, start by creating a page abstraction using the `pageFactory` function:
|
|
|
|
```typescript
|
|
export const DataSources = {
|
|
url: '/datasources',
|
|
dataSources: (dataSourceName: string) => `data-testid Data source list item ${dataSourceName}`,
|
|
};
|
|
```
|
|
|
|
You might have noticed that instead of a simple string as the selector, there's a function that takes a string parameter as an argument and returns a formatted string using the argument.
|
|
|
|
Just as before, you need to add the `DataSources` page to the exported const `Pages` in `packages/grafana-e2e-selectors/src/selectors/pages.ts`.
|
|
|
|
The next step is to use the `dataSources` selector function as in the following example:
|
|
|
|
```jsx
|
|
<ul>
|
|
{dataSources.map(({ id, name }) => (
|
|
<li className="card-item-wrapper" key={id}>
|
|
<a className="card-item" href={`datasources/edit/${id}`}>
|
|
<div className="card-item-name" data-testid={selectors.pages.DataSources.dataSources(name)}>
|
|
{name}
|
|
</div>
|
|
</a>
|
|
</li>
|
|
))}
|
|
</ul>
|
|
```
|
|
|
|
When this list is rendered with the data sources with names `A`, `B` and `C` ,the resulting HTML looks like this:
|
|
|
|
```html
|
|
<div class="card-item-name" data-testid="data-testid Data source list item A">A</div>
|
|
<div class="card-item-name" data-testid="data-testid Data source list item B">B</div>
|
|
<div class="card-item-name" data-testid="data-testid Data source list item C">C</div>
|
|
```
|
|
|
|
Now we can write our test. The one thing that differs from the previous [basic example](#basic-example) is that we pass in which data source we want to click as an argument to the selector function:
|
|
|
|
```typescript
|
|
describe('List test', () => {
|
|
it('clicks on data source named B', () => {
|
|
e2e.pages.DataSources.visit();
|
|
// To prevent flaky tests, always do a .should on any selector that you expect to be in the DOM.
|
|
// Read more here: https://docs.cypress.io/guides/core-concepts/retry-ability.html#Commands-vs-assertions
|
|
e2e.pages.DataSources.dataSources('B').should('be.visible').click();
|
|
});
|
|
});
|
|
```
|
|
|
|
## aria-label versus data-testid
|
|
|
|
Our selectors are set up to work with both `aria-label` attributes and `data-testid` attributes. The `aria-label` attributes help assistive technologies such as screen readers identify interactive elements of a page for our users.
|
|
|
|
A good example of a time to use an aria-label might be if you have a button with an **X** to close:
|
|
|
|
```
|
|
<button aria-label="close">X<button>
|
|
```
|
|
|
|
It might be clear visually that the **X** closes the modal, but audibly it would not be clear, for example.
|
|
|
|
```
|
|
<button aria-label="close">Close<button>
|
|
```
|
|
|
|
The example might read aloud to a user as "Close, Close" or something similar.
|
|
|
|
However, adding an aria-label to an element that is already clearly labeled or not interactive can be confusing and redundant for users with assistive technologies.
|
|
|
|
In such cases, don't add an unnecessary aria-label to a component so as to make them selectable for testing. Instead, use a data attribute that will not be read aloud with an assistive technology. For example:
|
|
|
|
```
|
|
<button data-testid="modal-close-button">Close<button>
|
|
```
|
|
|
|
We have added support for data attributes in our selectors Prefix your selector string with `data-testid`:
|
|
|
|
```typescript
|
|
export const Components = {
|
|
Login: {
|
|
openButton: 'open-button', // this looks for an aria-label
|
|
closeButton: 'data-testid modal-close-button', // this looks for a data-testid
|
|
},
|
|
};
|
|
```
|
|
|
|
and in your component, import the selectors and add the `data-testid`:
|
|
|
|
```
|
|
<button data-testid={Selectors.Components.Login.closeButton}>
|
|
```
|