Skip to content

Results

This library uses test-results-parser and performance-results-parser to read test results from functional and non-functional test frameworks.

Supports

TIP

Most automation testing frameworks, such as Selenium, Cypress, Playwright, Webdriver.io, PyTest, can output test results in standardized formats like JUnit XML. This tool leverages these standardized formats to ensure compatibility across different testing frameworks. This means you can seamlessly integrate your existing testing infrastructure with the testbeats without the need for extensive modifications.

Functional Test Results

✅ Mocha (json & mochawesome)

✅ TestNG

✅ JUnit

✅ xUnit

✅ Cucumber (json)

Non Functional Test Results

✅ JMeter (aggregate.csv)

✅ JMeter JTL

The .jtl file should contain headers and all the requests in test plan should be wrapped around transaction controllers.

Syntax

Functional

json
{
  "type": "testng",
  "files": ["path/to/results-file.xml"]
}
  • type (string) - name of the test results file format.

    mocha, testng, junit, xunit, cucumber

  • files (string[]) - path to the test result files. (allows * wild card)

Non-Functional

json
{
  "type": "jmeter",
  "files": ["tests/data/jmeter/sample.csv"],
  "thresholds": [
    {
      "metric": "Duration",
      "checks": ["avg<5000"]
    }
  ]
}
  • type (string) - name of the test results file format.

    jmeter

  • files (string[]) - path to the test result files. (allows * wild card)

  • thresholds (object[]) - pass/fail criteria for test metrics.

    • metric (string) - a measure on how a system performs under test conditions.

      Samples, Duration, Errors, Data Sent, Data Received, Latency

    • checks (string[]) - true/false criteria.

      sum, rate, avg, min, med, max, p90, p95, p99

    • scope (string) - scope of the threshold to be applied on.

      OVERALL, TRANSACTION

    • transactions (string[]) - list of transactions to apply the current threshold.

Config

Functional

Sample config file for a TestNG result file.

json
{
  "targets": [
    {
      "name": "teams",
      "inputs": {
        "url": "<incoming-webhook-url>"
      }
    }
  ],
  "results": [
    {
      "type": "testng",
      "files": ["path/to/testng-results.xml"]
    }
  ]
}

Wildcards

Merges multiple results into a single result.

Only * wildcard is supported by this tool.

json
{
  "targets": [
    {
      "name": "teams",
      "inputs": {
        "url": "<incoming-webhook-url>"
      }
    }
  ],
  "results": [
    {
      "type": "testng",
      "files": ["path/to/cypress-results-*.xml"]
    }
  ]
}

Non-Functional

Sample config file for a JMeter result file.

json
{
  "targets": [
    {
      "name": "teams",
      "inputs": {
        "url": "<incoming-webhook-url>"
      }
    }
  ],
  "results": [
    {
      "type": "jmeter",
      "files": ["path/to/result.jtl"]
    }
  ]
}

Using Thresholds

Sample config file for a JMeter result file using thresholds.

json
{
  "targets": [
    {
      "name": "teams",
      "inputs": {
        "url": "<incoming-webhook-url>"
      }
    }
  ],
  "results": [
    {
      "type": "jmeter",
      "files": ["path/to/aggregate.csv"],
      "thresholds": [
        {
          "metric": "Samples",
          "checks": ["sum>1"]
        },
        {
          "metric": "Duration",
          "checks": [ "p95<2000", "avg<2500" ],
          "scope": "TRANSACTION",
          "transactions": ["<transaction-name>"]
        }
      ]
    }
  ]
}

Released under the MIT License.