{"id":58695,"date":"2023-09-27T11:47:53","date_gmt":"2023-09-27T06:17:53","guid":{"rendered":"https:\/\/www.tothenew.com\/blog\/?p=58695"},"modified":"2023-09-27T11:47:53","modified_gmt":"2023-09-27T06:17:53","slug":"playwright-nodejs-custom-reports-a-deep-dive-into-test-pass-fail-analytics-for-test-outcomes","status":"publish","type":"post","link":"https:\/\/www.tothenew.com\/blog\/playwright-nodejs-custom-reports-a-deep-dive-into-test-pass-fail-analytics-for-test-outcomes\/","title":{"rendered":"Playwright (Nodejs) Custom Reports: A Deep Dive into Test Pass\/Fail Analytics for Test Outcomes"},"content":{"rendered":"

We’ll explore Playwright with Node.js Custom Test Reports and, with the help of sample code, demonstrate how these reports can be your key to unlocking valuable insights from test results, ultimately enhancing the quality of your testing outcomes.<\/p>\n

Why Test Pass\/Fail Analytics Matter
\n<\/strong><\/h2>\n

Before we dive into the practical aspects, let’s understand the significance of test pass\/fail analytics. In automated testing, a test’s outcome is categorized as either “pass” (successful execution) or “fail” (an issue that occurred during execution and can be timeout or skipped ). These verdicts offer vital insights that drive software quality improvement.<\/p>\n

Introduction to Playwright Custom Reports
\n<\/strong><\/h2>\n

Playwright, a versatile and developer-friendly tool, provides custom reporting capabilities that allow you to extract and analyze test data in a structured manner. These reports are much more than just summaries; they are your secret weapon for informed decision-making.<\/p>\n

Key Features of Playwright Custom Reports<\/strong><\/h2>\n

Detailed Test Logs:<\/em> <\/strong>Playwright custom reports offer exhaustive logs for every test run, pinpointing the exact step where a failure occurred.
\nCustom Metrics:<\/strong> <\/em>Define custom metrics relevant to your application to track specific key performance indicators (KPIs) during testing.<\/p>\n

Analysing Test Outcomes with Sample Code<\/strong><\/h2>\n

Let’s dive into the practical aspect of using Playwright Custom Reports with some sample code. We’ll outline a simple test scenario to demonstrate the power of these reports.<\/p>\n

\n
Create a js file with name - 'customerReportFilePath.js'\r\n<\/em>\r\n\"use strict\";<\/em>\r\nObject.defineProperty(exports, \"__esModule\", { value: true })<\/em>\r\nimport { writeFileSync } from \"fs\";<\/em>\r\nvar JSONSummaryReporter = \/** @class *\/ (function () {<\/em>\r\nclass JSONSummaryReporter {<\/em>\r\n\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0constructor () {<\/em>\r\n\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0this.executionDurationInMS = -1;<\/em>\r\n\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0this.testStartedAt = 0;<\/em>\r\n\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0this.passTests = [];<\/em>\r\n\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0this.skipTests = [];<\/em>\r\n\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0this.failTests = [];<\/em>\r\n\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0this.interruptedTests = [];<\/em>\r\n\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0this.timedOutTests = [];<\/em>\r\n\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0this.testStatus = 'unknown';<\/em>\r\n\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0this.warned = [];<\/em>\r\n}<\/em>\r\nonBegin() {<\/em>\r\n\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0this.testStartedAt = Date.now();<\/em>\r\n}<\/em>\r\nonTestEnd(test, result) {<\/em>\r\n\u00a0 \u00a0 \u00a0let testTitle = [];<\/em>\r\n\u00a0 \u00a0 \u00a0let testFileName = [];<\/em>\r\n\u00a0 \u00a0 \u00a0let clean = true;<\/em>\r\n\u00a0 \u00a0 \u00a0 \u00a0 \u00a0for (let index = 0, a = test.titlePath(); index < a.length; index++) {<\/em>\r\n\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 let fileName = a[index];<\/em>\r\n\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 if (fileName === '' && clean)<\/em>\r\n\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0continue;<\/em>\r\n\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 clean = false;<\/em>\r\n\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0testTitle.push(fileName);<\/em>\r\n\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 if (fileName.includes('spec.js')) {<\/em>\r\n\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 testFileName.push(fileName);<\/em>\r\n\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0}<\/em>\r\n\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0}<\/em>\r\n\/\/ This will publish the file name + test title + status + + line number test begins on and column<\/em>\r\n\u00a0 \u00a0 \u00a0 \u00a0let testDetails = \"\".concat(testFileName[0], \" : \").concat(test.title, \" : \", result.status, \" :\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0\").concat(test.location.line, \" : \").concat(test.location.column)<\/em>\r\n\/\/ Using the t variable in the push will push a full test test name + test description<\/em>\r\n\u00a0 \u00a0 \u00a0 let t = testTitle.join(' > ');<\/em>\r\n\u00a0 \u00a0 \u00a0 let status = !['passed', 'skipped', 'interrupted'].includes(result.status) && t.includes('@warn')<\/em>\r\n\u00a0 \u00a0 \u00a0 ? 'warned'<\/em>\r\n\u00a0 \u00a0 \u00a0 : result.status;<\/em>\r\n\u00a0 \u00a0 \u00a0 this[status].push(testDetails);<\/em>\r\n\u00a0 }<\/em>\r\nonEnd(result) {<\/em>\r\n\u00a0 \u00a0 var _this = this;<\/em>\r\n\u00a0 \u00a0 this.executionDurationInMS = Date.now() - this.testStartedAt;<\/em>\r\n\u00a0 \u00a0 this.testStatus = result.status;<\/em>\r\n\u00a0 \u00a0 \/\/ removing duplicate tests from pass tests array<\/em>\r\n\u00a0 \u00a0 this.passTests = this.passTests.filter(function (element, index) {<\/em>\r\n\u00a0 \u00a0 return _this.passTests.indexOf(element) === index;<\/em>\r\n});<\/em>\r\n\/\/ removing duplicate and flakey (pass on a retry) tests from the fail tests array<\/em>\r\n\u00a0 \u00a0this.failTests = this.failTests.filter(function (element, index) {<\/em>\r\n\u00a0 \u00a0 \u00a0if (!_this.passTests.includes(element))<\/em>\r\n\u00a0 \u00a0 \u00a0 \u00a0 return _this.failTests.indexOf(element) === index;<\/em>\r\n\u00a0 \u00a0 \u00a0});<\/em>\r\n\u00a0 \u00a0 \u00a0writeFileSync(`.\/summary.json`, JSON.stringify(this, null, ' '));<\/em>\r\n\u00a0 \u00a0 }<\/em>\r\n\u00a0 }<\/em>\r\n\u00a0 \u00a0return JSONSummaryReporter;<\/em>\r\n}());<\/em>\r\nconst _default = JSONSummaryReporter;<\/em>\r\nexport { _default as default };<\/em><\/pre>\n<\/blockquote>\n
In playright.config file add below line of code\u00a0 :<\/strong><\/p>\n
\u00a0 \u00a0 \u00a0 \u00a0 \u00a0reporter :['.\/customerReportFilepath.js'] \/\/ js file path\u00a0<\/em><\/pre>\n
Once the test execution ends, it will generate a json file \u2018summary.json\u2019<\/em><\/strong> with the execution results, test pass\/fail, and execution duration analytics. Similarly, we can get more test details based on the application requirements.<\/p>\n
\n
Result in 'summary.json' file :\r\n<\/strong>\r\n{<\/em>\r\n\u00a0 \u00a0 \u00a0\"executionDurationInMS\": 4157,<\/em>\r\n\u00a0 \u00a0 \u00a0 \"passTests\": [<\/em>\r\n\u00a0 \u00a0 \u00a0 \"DemoPassTest.spec.js : Demo test : passed : 20 : 9\"<\/em>\r\n\u00a0 \u00a0 \u00a0 \u00a0],<\/em>\r\n\u00a0 \u00a0 \u00a0 \"skipTests\": [<\/em>\r\n\u00a0 \u00a0 \u00a0 \u00a0],<\/em>\r\n\u00a0 \u00a0 \u00a0 \u00a0\"failTests\": [<\/em>\r\n\u00a0 \u00a0 \u00a0 \u00a0 \"DemoFailTest.spec.js : Demo Fail test : failed : 10 : 5\"<\/em>\r\n\u00a0 \u00a0 \u00a0 \u00a0 ],<\/em>\r\n\u00a0 \u00a0 \u00a0 \u00a0\"interruptedTests\": [<\/em>\r\n\u00a0 \u00a0 \u00a0 \u00a0 ],<\/em>\r\n\u00a0 \u00a0 \u00a0 \u00a0 \"warned\": [],<\/em>\r\n\u00a0 \u00a0 \u00a0 \u00a0 \"timedOutTests\": [],<\/em>\r\n\u00a0 \u00a0 \u00a0 \u00a0 \"testStatus\": \"failed\",<\/em>\r\n\u00a0 \u00a0 \u00a0 \u00a0 \"testStartedAt\": 169467477831<\/em>\r\n\u00a0 \u00a0 \u00a0}<\/em><\/pre>\n<\/blockquote>\n
Conclusion
\n<\/strong><\/h2>\n
Playwright Custom Reports provides test outcomes of automated testing. They illuminate the path to test pass\/fail insights, helping you navigate toward enhanced testing results. By grasping the importance of test execution analytics and harnessing the capabilities of Playwright Custom Reports, you can elevate your testing strategy.<\/p>\n
<\/div>","protected":false},"excerpt":{"rendered":"
We’ll explore Playwright with Node.js Custom Test Reports and, with the help of sample code, demonstrate how these reports can be your key to unlocking valuable insights from test results, ultimately enhancing the quality of your testing outcomes. Why Test Pass\/Fail Analytics Matter Before we dive into the practical aspects, let’s understand the significance of […]<\/p>\n","protected":false},"author":1645,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"iawp_total_views":69},"categories":[1818,1185],"tags":[4941,5464],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/posts\/58695"}],"collection":[{"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/users\/1645"}],"replies":[{"embeddable":true,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/comments?post=58695"}],"version-history":[{"count":2,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/posts\/58695\/revisions"}],"predecessor-version":[{"id":58760,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/posts\/58695\/revisions\/58760"}],"wp:attachment":[{"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/media?parent=58695"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/categories?post=58695"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/tags?post=58695"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}