Skip to content

A GitHub action to parse test results and post the summary as a PR comment

License

Notifications You must be signed in to change notification settings

bibipkins/dotnet-test-reporter

Use this GitHub action with your project
Add this Action to an existing workflow or create a new one
View on Marketplace

Repository files navigation

Dotnet Test Reporter

A GitHub action to parse test & coverage results and post them as a PR comment.

  • The action can process dotnet test results (a single or multiple .trx files), if there are any failing tests the action will fail. This allows integrating the action into your CI pipeline to short circuit and prevent further build/deploy operations as well as merging the code that caused tests to fail.
  • Optionally, test coverage can also be provided (a single opencover or cobertura .xml file) as well as a minimum coverage percentage threshold. If the threshold is provided and the coverage is not sufficient the action will fail.
  • The action also generates a workflow summary - a more detailed overview of processed tests and test coverage. For your convenience you can see the summary by following the link in the comment.
  • The action allows many configurations to suit your needs, please visit the Inputs and Examples sections.

Comment example

image

Summary example

Tests

✔️ Passed❌ Failed⚠️ Skipped⏱️ Time
29338.2s
✔️ eShopWeb.UnitTests.ApplicationCore.Entities.BasketTests.BasketAddItem - 6/6
ResultTest
✔️AddsBasketItemIfNotPresent
✔️CantAddItemWithNegativeQuantity
✔️CantModifyQuantityToNegativeNumber
✔️DefaultsToQuantityOfOne
✔️IncrementsQuantityOfItemIfPresent
✔️KeepsOriginalUnitPriceIfMoreItemsAdded
✔️ eShopWeb.UnitTests.ApplicationCore.Entities.BasketTests.BasketRemoveEmptyItems - 1/1
ResultTest
✔️RemovesEmptyBasketItems
✔️ eShopWeb.UnitTests.ApplicationCore.Entities.BasketTests.BasketTotalItems - 2/2
ResultTest
✔️ReturnsTotalQuantityWithMultipleItems
✔️ReturnsTotalQuantityWithOneItem
✔️ eShopWeb.UnitTests.ApplicationCore.Entities.OrderTests.OrderTotal - 1/3
ResultTestOutput
⚠️IsCorrectGiven1ItemSkipped test 1
⚠️IsCorrectGiven3ItemsSkipped test 2
✔️IsZeroForNewOrder
✔️ eShopWeb.UnitTests.ApplicationCore.Extensions.JsonExtensions - 3/3
ResultTest
✔️CorrectlyDeserializesJson(json: "{ \"id\": 3124, \"name\": \"Test Value 1\" }", expectedId: 3124, expectedName: "Test Value 1")
✔️CorrectlyDeserializesJson(json: "{ \"id\": 9, \"name\": \"Another test\" }", expectedId: 9, expectedName: "Another test")
✔️CorrectlySerializesAndDeserializesObject
❌ eShopWeb.UnitTests.ApplicationCore.Services.BasketServiceTests.AddItemToBasket - 1/2
ResultTestOutput
InvokesBasketRepositoryGetBySpecAsyncOnceError Message
Assert.Fail(): Failed test 1

Stack Trace
at eShopWeb.UnitTests.ApplicationCore.Services.BasketServiceTests.AddItemToBasket.InvokesBasketRepositoryGetBySpecAsyncOnce() in C:\Users\testuser\Files\Projects\eShopOnWeb\tests\UnitTests\ApplicationCore\Services\BasketServiceTests\AddItemToBasket.cs:line 20 --- End of stack trace from previous location ---
✔️InvokesBasketRepositoryUpdateAsyncOnce
✔️ eShopWeb.UnitTests.ApplicationCore.Services.BasketServiceTests.DeleteBasket - 1/1
ResultTest
✔️ShouldInvokeBasketRepositoryDeleteAsyncOnce
✔️ eShopWeb.UnitTests.ApplicationCore.Services.BasketServiceTests.TransferBasket - 4/4
ResultTest
✔️CreatesNewUserBasketIfNotExists
✔️InvokesBasketRepositoryFirstOrDefaultAsyncOnceIfAnonymousBasketNotExists
✔️RemovesAnonymousBasketAfterUpdatingUserBasket
✔️TransferAnonymousBasketItemsWhilePreservingExistingUserBasketItems
❌ eShopWeb.UnitTests.ApplicationCore.Specifications.BasketWithItems - 3/4
ResultTestOutput
MatchesBasketWithGivenBasketIdError Message
System.Exception : Test exception

Stack Trace
at eShopWeb.UnitTests.ApplicationCore.Specifications.BasketWithItems.MatchesBasketWithGivenBasketId() in C:\Users\testuser\Files\Projects\eShopOnWeb\tests\UnitTests\ApplicationCore\Specifications\BasketWithItemsSpecification.cs:line 24
✔️MatchesBasketWithGivenBuyerId
✔️MatchesNoBasketsIfBasketIdNotPresent
✔️MatchesNoBasketsIfBuyerIdNotPresent
❌ eShopWeb.UnitTests.ApplicationCore.Specifications.CatalogFilterPaginatedSpecification - 0/2
ResultTestOutput
⚠️Returns2CatalogItemsWithSameBrandAndTypeIdSkipped test 3
ReturnsAllCatalogItemsError Message
Assert.Fail(): Failed test 2

Stack Trace
at eShopWeb.UnitTests.ApplicationCore.Specifications.CatalogFilterPaginatedSpecification.ReturnsAllCatalogItems() in C:\Users\testuser\Files\Projects\eShopOnWeb\tests\UnitTests\ApplicationCore\Specifications\CatalogFilterPaginatedSpecification.cs:line 11
✔️ eShopWeb.UnitTests.ApplicationCore.Specifications.CatalogFilterSpecification - 7/7
ResultTest
✔️MatchesExpectedNumberOfItems(brandId: 1, typeId: 3, expectedCount: 1)
✔️MatchesExpectedNumberOfItems(brandId: 1, typeId: null, expectedCount: 3)
✔️MatchesExpectedNumberOfItems(brandId: 2, typeId: 3, expectedCount: 0)
✔️MatchesExpectedNumberOfItems(brandId: 2, typeId: null, expectedCount: 2)
✔️MatchesExpectedNumberOfItems(brandId: null, typeId: 1, expectedCount: 2)
✔️MatchesExpectedNumberOfItems(brandId: null, typeId: 3, expectedCount: 1)
✔️MatchesExpectedNumberOfItems(brandId: null, typeId: null, expectedCount: 5)

Coverage

📏 Line🌿 Branch
121 / 263 (46.01%)5 / 18 (27.78%)
✔️ ApplicationCore - 46.01%
FileTotalLineBranchLines to Cover
CatalogSettings.cs0 / 10%100%5
Entities\BaseEntity.cs1 / 1100%100%
Entities\BasketAggregate\Basket.cs18 / 1994.74%100%9
Entities\BasketAggregate\BasketItem.cs3 / 475%100%8
Entities\BuyerAggregate\Buyer.cs0 / 110%100%9, 11, 13, 15, 16, 18, 20-24
Entities\BuyerAggregate\PaymentMethod.cs0 / 30%100%5-7
Entities\CatalogBrand.cs0 / 10%100%7
Entities\CatalogItem.cs2 / 825%100%7-10, 12, 14
Entities\CatalogType.cs0 / 10%100%7
Entities\OrderAggregate\Address.cs13 / 1492.86%100%17
Entities\OrderAggregate\CatalogItemOrdered.cs12 / 1580%100%22, 23, 25
Entities\OrderAggregate\Order.cs21 / 2584%100%10, 11, 13, 40
Entities\OrderAggregate\OrderItem.cs9 / 1275%100%10, 11, 13
Exceptions\BasketNotFoundException.cs3 / 1225%100%11-13, 15-17, 19-21
Exceptions\GuardExtensions.cs3 / 475%50%12
Services\BasketService.cs17 / 5133.33%0%24, 25, 27, 29, 30, 39-46, 48-51, 58-68, 74-79
Services\OrderService.cs0 / 220%0%17-24, 27-38, 40, 41
Services\UriComposer.cs0 / 40%100%9, 12-14
Specifications\BaseSpecification.cs11 / 3333.33%100%17-19, 21, 22, 30-32, 34-38, 40-42, 44-46, 50-52
Specifications\BasketWithItemsSpecification.cs4 / 850%100%13-16
Specifications\CatalogFilterPaginatedSpecification.cs0 / 50%100%8-12
Specifications\CatalogFilterSpecification.cs4 / 4100%100%
Specifications\CustomerOrdersWithItemsSpecification.cs0 / 50%100%8-12

Note: despite this action was created specifically for dotnet you can use it with other languages and frameworks as long as you can generate .trx files. For example, your project can be a C#/dotnet backend with a TS/react frontend, in that case you would use the action twice. For react app you would need to:

  • install jest-trx-results-processor package to generate .trx test result files
  • add cobertura to coverageReporters array in case you would like to include a coverage

Inputs

github-token

Required - GitHub repository token.

results-path

Required - Path to the .trx file(s) containing test results. Supports glob patterns.
Examples: ./TestResults/result.trx, ./**/*.trx

coverage-path

Optional - Path to the file containing test coverage. Supports glob patterns.
Examples: ./TestResults/coverage.xml, ./**/coverage.xml

coverage-type

Optional - Coverage file type. Supported types are opencover and cobertura.
Default: opencover

coverage-threshold

Optional - Minimum allowed coverage. You can provide a coverage percentage ranging from 0.00 to 100.00.
Example: 80.42

comment-title

Optional - Pull Request comment title.
Example: My Custom Title
Default: Test Results

post-new-comment

Optional - Boolean flag. Set to true to post a new comment after each run. Set to false or leave blank to only update an existing comment.
Default: false

allow-failed-tests

Optional - Boolean flag. Set to true to prevent failed tests from failing the job. Set to false or leave blank to fail the job if there are any failed tests (recommended).
Default: false

show-failed-tests-only

Optional - Boolean flag. Set to true to show only the failed tests. This is useful if you have many tests and the results exceed the markdown comment limit in github Set to false or leave blank to show all the test results (recommended).
Default: false

show-test-output

Optional - Boolean flag. Set to true or leave blank to show the output of the tests. (recommended). Set to false if there is too much output leading to truncation on the summary
Default: true

Outputs

tests-total

Total number of tests

tests-passed

Number of tests passed

tests-failed

Number of tests failed

tests-skipped

Number of tests skipped

coverage-line

Line code coverage

coverage-lines-total

Total lines of code

coverage-lines-covered

Lines of code covered

coverage-branch

Branch code coverage

coverage-branches-total

Total branches

coverage-branches-covered

Branches covered

Examples

Note: please, always use the latest version

uses: bibipkins/[email protected]
with:
  github-token: ${{ secrets.GITHUB_TOKEN }}
  comment-title: 'Unit Test Results'
  results-path: ./TestResults/*.trx
  coverage-path: ./TestResults/coverage.xml
  coverage-threshold: 80