Skip to content

About

A comprehensive C# BDD automation framework using SpecFlow, Playwright for UI testing, and API automation. Includes GitHub Actions CI pipeline and Azure deployment steps.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

Playwright Test Automation Framework

A comprehensive, enterprise-grade test automation framework built with Playwright, NUnit, and C# (.NET 10). This framework supports both UI and REST API testing with robust reporting, CI/CD integration, and best practices implementation.

.NET Playwright NUnit License

Table of Contents


Features

  • Cross-Browser Testing: Support for Chromium, Firefox, and WebKit
  • UI Testing: Page Object Model with JSON-based locators
  • API Testing: REST API testing with RestSharp
  • Parallel Execution: Run tests in parallel for faster execution
  • Rich Reporting: ExtentReports HTML reports with screenshots
  • Screenshot on Failure: Automatic screenshot capture on test failures
  • Video Recording: Record test execution videos
  • Trace Capture: Playwright trace files for debugging
  • Environment Management: Multiple environment configurations
  • CI/CD Ready: GitHub Actions, Jenkins, and Azure Pipelines support
  • Logging: Comprehensive logging with Serilog
  • Data-Driven Testing: JSON-based test data management

Prerequisites

Required Software

Software Version Download Link
.NET SDK 10.0 or later Download
Node.js 18.x or later Download
Git Latest Download

Verify Installation

# Check .NET version
dotnet --version

# Check Node.js version
node --version

# Check Git version
git --version

IDE Recommendations

  • Visual Studio 2022 (Windows/Mac)
  • Visual Studio Code with C# extension
  • JetBrains Rider

Project Structure

PlaywrightTestFramework/
├── Config/                          # Configuration classes
│   ├── ConfigurationManager.cs      # Central configuration manager
│   └── TestSettings.cs              # Settings models
├── Core/                            # Framework core components
│   ├── Api/                         # API testing infrastructure
│   │   ├── ApiClient.cs             # REST API client
│   │   └── Models/                  # API request/response models
│   ├── Browser/                     # Browser management
│   │   └── BrowserFactory.cs        # Browser factory
│   ├── Locators/                    # Locator management
│   │   ├── LocatorManager.cs        # JSON locator loader
│   │   └── LocatorModel.cs          # Locator models
│   ├── Logging/                     # Logging setup
│   │   └── LoggerSetup.cs           # Serilog configuration
│   ├── Pages/                       # Page Object base
│   │   └── BasePage.cs              # Base page class
│   ├── Reporting/                   # Reporting infrastructure
│   │   └── ReportManager.cs         # ExtentReports manager
│   └── Utilities/                   # Helper utilities
│       ├── TestDataManager.cs       # Test data loader
│       └── WaitHelper.cs            # Custom wait utilities
├── Locators/                        # JSON locator files
│   ├── LoginPage.json               # Login page locators
│   └── TodoPage.json                # Todo page locators
├── Pages/                           # Page Objects
│   └── TodoPage.cs                  # Todo page implementation
├── TestData/                        # Test data files
│   └── testdata.json                # Test data
├── Tests/                           # Test classes
│   ├── Base/                        # Test base classes
│   │   ├── UITestBase.cs            # UI test base
│   │   └── ApiTestBase.cs           # API test base
│   ├── UI/                          # UI tests
│   │   └── TodoTests.cs             # Todo UI tests
│   └── API/                         # API tests
│       ├── UserApiTests.cs          # User API tests
│       └── PostApiTests.cs          # Post API tests
├── appsettings.json                 # Base configuration
├── appsettings.dev.json             # Dev environment config
├── appsettings.staging.json         # Staging environment config
├── appsettings.prod.json            # Production environment config
└── PlaywrightTestFramework.csproj   # Project file

Quick Start

1. Clone the Repository

git clone https://github.com/amitbad/CS-Playwright-SpecFlow.git
cd PlaywrightTestFramework

2. Restore Dependencies

dotnet restore

3. Build the Project

dotnet build

4. Install Playwright Browsers

# On macOS/Linux
pwsh PlaywrightTestFramework/bin/Debug/net10.0/playwright.ps1 install

# Or using PowerShell directly
pwsh -Command "& PlaywrightTestFramework/bin/Debug/net10.0/playwright.ps1 install"

5. Run Tests

# Run all tests
dotnet test

# Run with detailed output
dotnet test --logger "console;verbosity=detailed"

Configuration

Environment Configuration

The framework uses a layered configuration system:

  1. Base Configuration (appsettings.json): Default settings
  2. Environment-Specific (appsettings.{env}.json): Environment overrides
  3. Environment Variables: Runtime overrides

Configuration Files

appsettings.json (Base)

{
  "TestSettings": {
    "Environment": "dev",
    "Browser": "chromium",
    "Headless": true,
    "Timeout": 30000,
    "RetryCount": 2,
    "ScreenshotOnFailure": true
  },
  "BrowserOptions": {
    "ViewportWidth": 1920,
    "ViewportHeight": 1080
  }
}

Environment Variables

Variable Description Default
TEST_ENVIRONMENT Environment name (dev/staging/prod) dev
BROWSER Browser to use (chromium/firefox/webkit) chromium
HEADLESS Run in headless mode true
BASE_URL Application base URL -
API_BASE_URL API base URL -
TEST_USERNAME Test user username -
TEST_PASSWORD Test user password -

Setting Environment Variables

macOS/Linux

export TEST_ENVIRONMENT=staging
export BROWSER=firefox
export HEADLESS=false

Windows (PowerShell)

$env:TEST_ENVIRONMENT = "staging"
$env:BROWSER = "firefox"
$env:HEADLESS = "false"

Windows (Command Prompt)

set TEST_ENVIRONMENT=staging
set BROWSER=firefox
set HEADLESS=false

Writing Tests

UI Tests

1. Create a Locator JSON File

Create Locators/MyPage.json:

{
  "pageName": "MyPage",
  "description": "My page locators",
  "locators": {
    "submitButton": {
      "type": "css",
      "value": "button[type='submit']",
      "description": "Submit button"
    },
    "usernameInput": {
      "type": "css",
      "value": "#username",
      "description": "Username input field"
    },
    "welcomeMessage": {
      "type": "text",
      "value": "Welcome",
      "description": "Welcome message text"
    }
  }
}

Supported Locator Types:

  • css - CSS selector
  • xpath - XPath expression
  • text - Text content
  • role - ARIA role
  • testid - Test ID attribute
  • placeholder - Placeholder text
  • label - Label text
  • alttext - Alt text

2. Create a Page Object

Create Pages/MyPage.cs:

using Microsoft.Playwright;
using PlaywrightTestFramework.Core.Locators;
using PlaywrightTestFramework.Core.Pages;

namespace PlaywrightTestFramework.Pages;

public class MyPage : BasePage
{
    protected override string PageName => "MyPage";

    public MyPage(IPage page, LocatorManager locatorManager) 
        : base(page, locatorManager) { }

    public async Task EnterUsernameAsync(string username)
    {
        await FillAsync("usernameInput", username);
    }

    public async Task ClickSubmitAsync()
    {
        await ClickAsync("submitButton");
    }

    public async Task<bool> IsWelcomeMessageVisibleAsync()
    {
        return await IsVisibleAsync("welcomeMessage");
    }
}

3. Create a Test Class

Create Tests/UI/MyPageTests.cs:

using FluentAssertions;
using NUnit.Framework;
using PlaywrightTestFramework.Pages;
using PlaywrightTestFramework.Tests.Base;

namespace PlaywrightTestFramework.Tests.UI;

[TestFixture]
[Category("UI")]
[Category("MyPage")]
public class MyPageTests : UITestBase
{
    private MyPage _myPage = null!;

    public override async Task SetUp()
    {
        await base.SetUp();
        _myPage = new MyPage(Page, LocatorManager);
    }

    [Test]
    [Description("Verify user can submit form")]
    public async Task SubmitForm_ShouldShowWelcomeMessage()
    {
        // Arrange
        LogStep("Navigate to page");
        await _myPage.NavigateAsync("/form");

        // Act
        LogStep("Enter username and submit");
        await _myPage.EnterUsernameAsync("testuser");
        await _myPage.ClickSubmitAsync();

        // Assert
        LogStep("Verify welcome message");
        var isVisible = await _myPage.IsWelcomeMessageVisibleAsync();
        isVisible.Should().BeTrue();
    }
}

API Tests

using FluentAssertions;
using NUnit.Framework;
using PlaywrightTestFramework.Core.Api.Models;
using PlaywrightTestFramework.Tests.Base;

namespace PlaywrightTestFramework.Tests.API;

[TestFixture]
[Category("API")]
public class MyApiTests : ApiTestBase
{
    [Test]
    [Description("Verify GET endpoint returns data")]
    public async Task GetUsers_ShouldReturnUsersList()
    {
        // Arrange
        LogStep("Send GET request");
        LogRequest("GET", "/users");

        // Act
        var response = await ApiClient.GetAsync<List<User>>("/users");

        // Assert
        LogResponse(response);
        response.IsSuccessful.Should().BeTrue();
        response.StatusCode.Should().Be(200);
        response.Data.Should().NotBeEmpty();
    }

    [Test]
    [Description("Verify POST creates resource")]
    public async Task CreateUser_ShouldReturnCreatedUser()
    {
        // Arrange
        var newUser = new { name = "Test User", email = "test@example.com" };
        LogRequest("POST", "/users", newUser);

        // Act
        var response = await ApiClient.PostAsync<User>("/users", newUser);

        // Assert
        LogResponse(response);
        response.StatusCode.Should().Be(201);
    }
}

Running Tests

Basic Commands

# Run all tests
dotnet test

# Run with verbose output
dotnet test --logger "console;verbosity=detailed"

# Run specific test category
dotnet test --filter "Category=UI"
dotnet test --filter "Category=API"

# Run specific test class
dotnet test --filter "FullyQualifiedName~TodoTests"

# Run specific test
dotnet test --filter "Name=AddTodo_ShouldDisplayInList"

# Run with specific browser
BROWSER=firefox dotnet test --filter "Category=UI"

# Run in parallel
dotnet test -- NUnit.NumberOfTestWorkers=4

Generate Reports

# Run tests with TRX report
dotnet test --logger "trx;LogFileName=results.trx"

# Run tests with HTML report
dotnet test --logger "html;LogFileName=results.html"

# Run with multiple loggers
dotnet test --logger "trx" --logger "html" --results-directory TestResults

Run by Environment

# Development
TEST_ENVIRONMENT=dev dotnet test

# Staging
TEST_ENVIRONMENT=staging dotnet test

# Production (read-only tests)
TEST_ENVIRONMENT=prod dotnet test --filter "Category=Smoke"

Reporting

ExtentReports

After test execution, find the HTML report at:

PlaywrightTestFramework/TestResults/ExtentReport.html

The report includes:

  • Test execution summary
  • Pass/Fail/Skip statistics
  • Detailed test steps
  • Screenshots on failure
  • System information
  • Execution timeline

Allure Reports (Optional)

# Install Allure CLI
brew install allure  # macOS
# or
scoop install allure  # Windows

# Generate report
allure generate allure-results -o allure-report --clean

# Open report
allure open allure-report

Test Artifacts

Artifact Location Description
HTML Report TestResults/ExtentReport.html Main test report
Screenshots TestResults/Screenshots/ Failure screenshots
Videos TestResults/Videos/ Test execution videos
Traces TestResults/Traces/ Playwright traces
Logs TestResults/Logs/ Execution logs

CI/CD Integration

Note: All CI/CD workflows are configured for MANUAL TRIGGERING ONLY. They will NOT run automatically on push or pull request. This allows you to configure your own credentials and settings before running tests.

GitHub Actions

The framework includes a ready-to-use GitHub Actions workflow at .github/workflows/test.yml.

Trigger Mode: Manual Only

  • No automatic triggers - Tests will NOT run on push or PR
  • Manual trigger only - You control when tests run

How to Run Tests Manually

  1. Go to Actions tab in your GitHub repository
  2. Select Playwright Tests workflow from the left sidebar
  3. Click Run workflow button (dropdown on the right)
  4. Select options:
    • Environment: dev/staging/prod
    • Browser: chromium/firefox/webkit
    • Test filter: Optional (e.g., Category=API)
    • Cross-browser: Check to run on all browsers
  5. Click Run workflow (green button)

GitHub Actions Setup

  1. Push your code to GitHub - The workflow file is already included
  2. Go to Actions tab - You'll see the "Playwright Tests" workflow
  3. Run manually when you're ready to test

Enabling Automatic Triggers (Optional)

If you want tests to run automatically, edit .github/workflows/test.yml:

on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main]
  workflow_dispatch:  # Keep this for manual runs
    # ... inputs ...

Viewing Results

  1. Go to Actions tab
  2. Click on the workflow run
  3. View test summary in the Summary tab
  4. Download artifacts:
    • test-results-{run_number} - Test reports
    • playwright-traces-{run_number} - Failure traces (on failure)

Jenkins

Prerequisites

  • Jenkins with Pipeline plugin
  • .NET SDK installed on Jenkins agent
  • PowerShell installed (for Playwright)

Setup Steps

  1. Create a New Pipeline Job

    • Go to Jenkins Dashboard
    • Click "New Item"
    • Enter job name
    • Select "Pipeline"
    • Click "OK"
  2. Configure Pipeline

    • In Pipeline section, select "Pipeline script from SCM"
    • Select "Git" as SCM
    • Enter repository URL
    • Set branch to */main
    • Script Path: Jenkinsfile
  3. Configure Build Parameters (Optional)

    • Check "This project is parameterized"
    • Parameters are defined in Jenkinsfile
  4. Run the Job

    • Click "Build with Parameters"
    • Select environment and browser
    • Click "Build"

Required Jenkins Plugins

  • Pipeline
  • Git
  • MSTest
  • HTML Publisher
  • AnsiColor

Jenkins Agent Requirements

# Install .NET SDK
# macOS
brew install dotnet-sdk

# Ubuntu
sudo apt-get install -y dotnet-sdk-10.0

# Install PowerShell
# macOS
brew install powershell

# Ubuntu
sudo apt-get install -y powershell

Azure DevOps Pipelines

Setup Steps

  1. Create Azure DevOps Project

    • Go to Azure DevOps
    • Create new project or use existing
  2. Import Repository

    • Go to Repos
    • Import from GitHub or push directly
  3. Create Pipeline

    • Go to Pipelines
    • Click "New Pipeline"
    • Select "Azure Repos Git"
    • Select your repository
    • Select "Existing Azure Pipelines YAML file"
    • Select /azure-pipelines.yml
    • Click "Continue"
  4. Run Pipeline

    • Click "Run"
    • Select parameters:
      • Environment
      • Browser
      • Test filter
      • Cross-browser option

Pipeline Features

  • Manual triggers only - No automatic runs on push/PR
  • Parameterized runs - Select environment, browser, filters
  • Cross-browser testing (optional parallel execution)
  • Test result publishing
  • Artifact storage

Enabling Automatic Triggers (Optional)

If you want tests to run automatically, edit azure-pipelines.yml:

trigger:
  branches:
    include:
      - main
      - develop

pr:
  branches:
    include:
      - main

Viewing Results in Azure DevOps

  1. Go to Pipelines > Runs
  2. Select your run
  3. View "Tests" tab for results
  4. Download artifacts from "Summary" tab

Best Practices

Test Organization

  1. Use Categories

    [Category("UI")]
    [Category("Smoke")]
    [Category("Login")]
  2. Use Descriptive Names

    [Test]
    [Description("Verify user can login with valid credentials")]
    public async Task Login_WithValidCredentials_ShouldSucceed()
  3. Follow AAA Pattern

    // Arrange
    // Act
    // Assert

Locator Best Practices

  1. Prefer stable locators

    • Test IDs: [data-testid="submit-btn"]
    • Roles: button, textbox
    • Labels: Associated label text
  2. Avoid fragile locators

    • Complex XPath
    • Index-based selectors
    • CSS classes that change
  3. Keep locators in JSON files

    • Easy to maintain
    • Non-developers can update
    • Version controlled

Page Object Best Practices

  1. One page object per page/component
  2. Methods should represent user actions
  3. Return page objects for navigation
  4. Keep assertions in tests, not page objects

API Test Best Practices

  1. Validate response structure
  2. Check response times
  3. Test error scenarios
  4. Use meaningful assertions

Troubleshooting

Common Issues

1. Playwright Browsers Not Installed

Error: Browser executable not found

Solution:

pwsh PlaywrightTestFramework/bin/Debug/net10.0/playwright.ps1 install

2. Permission Denied on macOS

Error: Permission denied

Solution:

chmod +x PlaywrightTestFramework/bin/Debug/net10.0/playwright.ps1

3. Tests Timeout

Error: Timeout 30000ms exceeded

Solutions:

  • Increase timeout in appsettings.json
  • Check network connectivity
  • Verify element locators

4. Element Not Found

Error: Element not found

Solutions:

  • Verify locator in JSON file
  • Check if element is in iframe
  • Add explicit wait

5. CI/CD Failures

Solutions:

  • Ensure headless mode is enabled
  • Install browser dependencies
  • Check environment variables

Debug Mode

Run tests with debug output:

DEBUG=pw:api dotnet test

View Playwright Traces

# Open trace viewer
npx playwright show-trace TestResults/Traces/trace.zip

Contributing

  1. Fork the repository
  2. Create a feature branch (git checkout -b feature/amazing-feature)
  3. Commit changes (git commit -m 'Add amazing feature')
  4. Push to branch (git push origin feature/amazing-feature)
  5. Open a Pull Request

Code Style

  • Follow C# naming conventions
  • Use meaningful variable names
  • Add XML documentation for public methods
  • Keep methods small and focused

License

This project is licensed under the MIT License - see the LICENSE file for details.


Support

For issues and questions:

  • Create a GitHub Issue
  • Check existing issues for solutions
  • Review troubleshooting section

Changelog

v1.0.0 (Initial Release)

  • Playwright integration with NUnit
  • Page Object Model with JSON locators
  • REST API testing support
  • ExtentReports integration
  • GitHub Actions, Jenkins, Azure Pipelines support
  • Comprehensive documentation

About

A comprehensive C# BDD automation framework using SpecFlow, Playwright for UI testing, and API automation. Includes GitHub Actions CI pipeline and Azure deployment steps.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages