End-to-end tests

This document provides a technical overview of the browser-based end-to-end (E2E) testing strategy for the application. It covers the setup, structure, and execution of tests that simulate real user workflows.

E2E Testing Purpose

End-to-end tests are the final and most comprehensive layer of our automated testing strategy. Their primary purpose is to validate the entire application stack by simulating real user interactions in a genuine browser environment.

Unlike unit tests that verify individual components in isolation or integration tests that check interactions between services, E2E tests confirm that all integrated pieces of the application function together as expected from the user's perspective. This includes:

• Rendering of the Razor frontend.
• Client-side interactions.
• Backend API calls and logic execution.
• Data retrieval from the Contentful CMS.
• Correct application of internationalization (i18n) rules.

By automating these workflows, we can quickly catch regressions and ensure that new features or refactors do not break existing functionality across the system.

Selenium WebDriver

The project utilizes Selenium WebDriver as the core framework for browser automation. It provides a programming interface to control browser behavior, enabling us to script user journeys programmatically.

The TheExampleApp.E2eTests project includes the Selenium.WebDriver NuGet package (version 3.7.0), which provides the necessary APIs for interacting with web elements.

<!-- TheExampleApp.E2eTests/TheExampleApp.E2eTests.csproj -->
<ItemGroup>
  <PackageReference Include="Selenium.WebDriver" Version="3.7.0" />
  ...
</ItemGroup>

Within the test code, we use the ChromeDriver class, a specific implementation of the IWebDriver interface, to drive the Google Chrome browser. Common operations include navigating to URLs, finding elements by their CSS selectors, and asserting their state (e.g., visibility, text content).

// TheExampleApp.E2eTests/Tests.cs

// Navigate to the home page
_browser.Navigate().GoToUrl(_basehost);

// Find an element by its CSS selector and get its text
var elementText = _browser.FindElementByCssSelector("h2.module-highlighted-course__title").Text;

// Find an element and click it
_browser.FindElementByCssSelector("#about-this-modal-trigger").Click();

ChromeDriver

To communicate with the Google Chrome browser, Selenium requires a corresponding driver executable. This project uses ChromeDriver.

The chromedriver (for Linux/macOS) and chromedriver.exe (for Windows) executables are included directly within the TheExampleApp.E2eTests project. They are configured to be copied to the build output directory, ensuring they are available at runtime without requiring a system-wide installation.

This is configured in the project's .csproj file:

<!-- TheExampleApp.E2eTests/TheExampleApp.E2eTests.csproj -->
<ItemGroup>
  <None Update="chromedriver">
    <CopyToOutputDirectory>Always</CopyToOutputDirectory>
  </None>
  <None Update="chromedriver.exe">
    <CopyToOutputDirectory>Always</CopyToOutputDirectory>
  </None>
</ItemGroup>

The ChromeDriver instance is initialized by providing the path to the directory containing these executables. A helper method, GetProjectPath, dynamically locates this directory at runtime.

Test Structure

The E2E tests are organized within the TheExampleApp.E2eTests project using the xUnit.net testing framework. The structure follows standard xUnit and object-oriented design patterns for setup and teardown.

• Test Class: All tests are contained within the Tests class.
• Setup and Teardown: The class implements IDisposable.
  • The constructor (public Tests()) is executed before each test. It initializes a new ChromeDriver instance, ensuring each test runs in a clean, isolated browser session.
  • The Dispose() method is executed after each test. It calls _browser.Dispose() to gracefully close the browser window and terminate the driver process, preventing resource leaks.
• Test Cases: Each test is a public method decorated with the [Fact] attribute.
• AAA Pattern: The tests follow the Arrange-Act-Assert pattern for clarity.

// TheExampleApp.E2eTests/Tests.cs
public class Tests : IDisposable
{
    ChromeDriver _browser;
    string _basehost = "http://localhost:59990/";

    // Setup: Runs before each test
    public Tests()
    {
        var startupAssembly = typeof(Tests).GetTypeInfo().Assembly;
        var root = GetProjectPath("", startupAssembly);
        var chromeOptions = new ChromeOptions();
        chromeOptions.AddArguments(new List<string>() { "headless" });
        _browser = new ChromeDriver(root, chromeOptions);
    }

    [Fact]
    public void ClickingOnHelpShouldOpenModal()
    {
        // Arrange: Navigate and find initial elements
        _browser.Navigate().GoToUrl(_basehost);
        var modalElement = _browser.FindElementByCssSelector("#about-this-modal");
        Assert.False(modalElement.Displayed);

        // Act: Perform the user action
        _browser.FindElementByCssSelector("#about-this-modal-trigger").Click();

        // Assert: Verify the outcome
        Assert.True(modalElement.Displayed);
    }

    // Teardown: Runs after each test
    public void Dispose()
    {
        _browser.Dispose();
    }
}

Running E2E Tests

To execute the E2E tests locally, the main web application must be running.

Prerequisites:

1. The application must be running and accessible at http://localhost:59990/. The base URL is hardcoded in the _basehost field in Tests.cs.
2. The .NET Core SDK (version 2.1) must be installed.
3. Google Chrome browser must be installed.

Execution Steps:

1. Start the application:

   cd TheExampleApp
   dotnet run

2. Run the tests: In a separate terminal, navigate to the root directory and execute the tests using the dotnet test command, targeting the E2E test project.

   dotnet test TheExampleApp.E2eTests

   Alternatively, tests can be run directly from the Test Explorer in Visual Studio.

Test Scenarios

The test suite covers several critical user workflows and application features:

Test Method	Description
ClickingOnHelpShouldOpenModal	Verifies that clicking the "About this App" trigger correctly opens the associated modal dialog.
SwitchingToGermanShouldPersistOverRequests	Tests the internationalization feature by setting the locale to German (de-DE) via a URL parameter and confirming that the language persists across subsequent page navigations.
ClickingOnLocaleShouldShowDropdown	Validates that the language selector dropdown becomes visible when the user clicks its trigger.
ClickingOnLocaleShouldSwitchLocale	Confirms that selecting a language from the UI dropdown successfully changes the application's content to the chosen language.
SettingsPageShouldDisplayDefaultCredentials	Checks that the settings page correctly pre-fills the form fields with the default Contentful Space ID and API tokens.
SettingsPageShouldDisplaySuccessMessageOnPost	Verifies that submitting the settings form displays a success confirmation message to the user.

Headless Testing

By default, the E2E tests are configured to run in headless mode. This means the Chrome browser executes all operations in memory without rendering a visible UI window. This is configured in the test class constructor by adding the headless argument to ChromeOptions.

// TheExampleApp.E2eTests/Tests.cs
public Tests()
{
    // ...
    var chromeOptions = new ChromeOptions();
    // The "headless" argument enables running tests without a visible browser UI.
    chromeOptions.AddArguments(new List<string>() { "headless" });
    _browser = new ChromeDriver(root, chromeOptions);
}

Benefits of headless testing include:

• Speed: Tests generally run faster without the overhead of rendering graphics.
• CI/CD Compatibility: Allows tests to run on servers and in containers that do not have a graphical user interface.

For local debugging, you can comment out the chromeOptions.AddArguments line to watch the test execution in a real browser window.

CI/CD Integration

The continuous integration pipeline, defined in .circleci/config.yml, is responsible for automatically running tests on every code change.

Important Note: The current CI pipeline executes a separate E2E test suite written in JavaScript using Cypress, not the C# Selenium tests described in this document. The C# tests are intended for local development and validation.

The CI process for E2E testing involves the following steps:

1. Run App: The .NET application is started in the background.

   # .circleci/config.yml
   - run:
       name: Run app
       command: | 
         cd TheExampleApp
         dotnet run 
       background: true

2. Clone E2E Tests: A separate repository containing Cypress tests is cloned.

   # .circleci/config.yml
   - run:
       name: Clone e2e tests
       command: |
         git clone https://github.com/contentful/the-example-app-e2e-tests.git ./test/e2e
         cd test/e2e

3. Install Dependencies: npm is used to install the Cypress test suite's dependencies.
4. Run E2E Tests: The Cypress test runner is invoked to execute the tests against the running application.

   # .circleci/config.yml
   - run:
       name: Run e2e tests
       command: |
         cd test/e2e 
         node_modules/.bin/cypress run --project /root/project --env LANGUAGE=dotnet

For more details on the full pipeline, refer to the CI/CD documentation.

Debugging E2E Tests

Debugging E2E tests can be challenging due to their dependency on a live application and browser environment. Here are some recommended strategies:

• Run in Non-Headless Mode: For local debugging, comment out the headless argument in Tests.cs to visually inspect the browser's behavior during test execution. This is the most effective way to see what the test is doing.
• Use Breakpoints: Since the tests are written in C#, you can use the Visual Studio debugger. Place breakpoints within a test method to pause execution and inspect variables, such as the state of the _browser object or the content of retrieved elements.
• Handle Timing Issues: The most common cause of flaky tests is timing. An element may not be present or interactive when Selenium tries to access it. The current tests use implicit waits, but for more robust tests, consider using WebDriverWait to explicitly wait for a specific condition (e.g., an element to be visible) before proceeding.
• Take Screenshots on Failure: While not implemented in the current suite, a powerful debugging technique is to automatically capture a screenshot when a test fails. This can be done by wrapping test assertions in a try...catch block and using the ITakesScreenshot interface in the catch block to save an image of the browser's state at the moment of failure.