Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Custom Sharding iOS Firebase Error #2490

Open
micai23 opened this issue Apr 15, 2024 · 1 comment
Open

Custom Sharding iOS Firebase Error #2490

micai23 opened this issue Apr 15, 2024 · 1 comment

Comments

@micai23
Copy link

micai23 commented Apr 15, 2024 •
edited
Loading

Hi I'm trying to implement custom sharding on iOS. Without a custom sharding json file, i'm able to run tests using github actions. However with custom sharding the name of the xctestrun file changes and adds shard_0 to the original xctestrun file uploaded.

Error

2024-04-18_13-11-42.585010_OvOW_shard_0_TIDAL-UITests_UITestsPlan_iphoneos17.2-arm64_shard_0.xctestrun

┌─────────┬──────────────────────┬───────────────┬────────────────────────────────────────────────────────────────┬─────────────────┬───────────────────────┐
│ OUTCOME │      MATRIX ID       │   APP NAME    │                         TEST FILE NAME                         │ TEST AXIS VALUE │     TEST DETAILS      │
├─────────┼──────────────────────┼───────────────┼────────────────────────────────────────────────────────────────┼─────────────────┼───────────────────────┤
│ INVALID │ matrix-24jr1bniisa4d │ Tidal-app.zip │ TIDAL-UITests_UITestsPlan_iphoneos17.2-arm64_shard_0.xctestrun │                 │ MALFORMED_XC_TEST_ZIP │

Here are my config files

On github actions

    - name: Run Firebase UI Tests using Flank
        id: flank_run
        uses: Flank/flank@master
        with:
          version: v23.10.1
          service_account: ${{ secrets.GOOGLE_CREDENTIALS_DEBUG }}
          platform: ios
          flank_configuration_file: ./scripts/flank-firebase-tests.yml

The flank-firebase-tests.yml config file

gcloud:
  record-video: true
  timeout: 45m
  async: false
  test: ./Tidal-app.zip
  xctestrun-file: ./Tidal-UITests_UITestsPlan_iphoneos17.2-arm64.xctestrun
  device:
    - model: iphone14pro
      version: 16.6
      locale: en
      orientation: portrait
  type: xctest
flank:
  max-test-shards: 1
  shard-time: -1
  disable-sharding: false
  custom-sharding-json: ./scripts/firebase-custom-sharding.json

Sharding json file, firebase-custom-sharding.json

{
  "Configuration 1": [
    {
      "EmptyUser1Tests": [
        "TIDALUITestsEmptyUser",
        "TIDALUITestsNowPlayingSuggestions"
      ]
    }
  ]
}
@ypenn21
Copy link

ypenn21 commented Jul 31, 2024

The sharding feature is lacking currently. According to this blog.. https://sahil-insights.medium.com/real-device-testing-xcuitest-firebase-test-lab-github-actions-1cb5f07f4ba6 . However for android instrumentation it is available.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@ypenn21 @micai23