add GitHub action that use Testing Farm #4320

lilyLuLiu · 2024-08-19T07:31:51Z

openshift-ci · 2024-08-19T07:31:55Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

openshift-ci · 2024-08-19T07:32:02Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign evidolob for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

.github/workflows/linux-qe.yml

adrianriobo · 2024-09-18T08:29:11Z

.github/workflows/linux-qe.yml

+          path: |
+            **/*.xml
+            **/*.results
+            **/*.log


After reserve, there is no need for decommission the testing farm machine (the target one for arm I mean)

lilyLuLiu · 2024-09-26T07:36:05Z

I need to log in to the self-host runner to do some debug. So need to add my ssh public key into the github self-host runner from the testing farm. Waitting for https://gitlab.com/testing-farm/infrastructure/-/merge_requests/698 to be merged now.

lilyLuLiu · 2024-10-22T04:30:59Z

@adrianriobo Hi, I'm debugging run crc tests with machine from TestingFarm.

The issue is, we can only use the self-host runner(also from TestingFarm) to provision and connect to the arm64 machine from TestingFarm. That means we can't connect to the machine with a container(deliverset image).
When provision a TestingFarm machine, the ssh public key of proxy will be added into the runner, thus the runner can connect to the reserved machine through proxy.

So I think we can't use the container method to run crc tests with Testing Farm.
Do we move to other test way?

adrianriobo · 2024-10-30T11:14:00Z

.github/workflows/linux-artifacts.yml

+        with:
+          name: linux-binary
+          path: "out/linux-arm64/crc"
+  build-qe:


I would prefer not having this specific flow. Currently we have the [Build Windows artifacts] where windows bits are built and also the qe images, may we can rename it to just builder and add there the linux builds.

Take into account if Linux bits are already build / check in some other flow remove it from there. The goal should be build only once.... otherwise if in future the build process requires something else on any element we need to be aware they are being built in several places with several goals.

adrianriobo · 2024-10-30T11:14:44Z

.github/workflows/linux-qe.yml

+
+on:
+  workflow_run:
+    workflows: [Build Windows artifacts]


THis is aligned with previous comment if you create the unified builder flow this should be triggered for the new builder flow

adrianriobo · 2024-10-30T11:15:42Z

.github/workflows/linux-qe.yml

+
+jobs:
+  tests:
+    runs-on: [self-hosted, linux, testing-farm]


Similar to ci-definitions for crc-builder we want to ensure the previous flow has been completed and it is successful https://github.com/crc-org/ci-definitions/blob/7cb907719aef4898ef4eb0ec02d9124f5212d9d4/.github/workflows/crc-builder-pusher.yml#L12

adrianriobo · 2024-10-30T11:16:23Z

.github/workflows/linux-qe.yml

+        go: ['1.21']
+    steps:
+      - uses: actions/checkout@v4
+      - name: Set up Go


Why do we need go here? all components should be picked from as artifacts

adrianriobo · 2024-10-30T11:17:02Z

.github/workflows/linux-qe.yml

+          cd ${{ env.commit_sha }}
+
+          #timestamp=`date  %s`
+          #mkdir $timestamp


Can we clean comments (unused lines)

adrianriobo · 2024-10-30T11:19:07Z

.github/workflows/linux-qe.yml

+
+      - name: prepare env
+        run: |
+          sudo yum install podman openssh-server git make -y


Do we need git and make for any reason?

adrianriobo · 2024-10-30T11:24:11Z

.github/workflows/linux-qe.yml

+          echo $SSH_AGENT_PID > ssh_agent_pid
+          #kill $(cat ssh_agent_pid)
+          ssh-add id_rsa
+


We will need api invoke to add this as check for the PR

crc/.github/workflows/windows-qe-tpl.yml

Line 84 in 0342835

- name: Add status to the PR check

this previously need to correlate the execution https://github.com/crc-org/crc/blob/main/.github/workflows/windows-qe-tpl.yml#L61

adrianriobo · 2024-10-30T11:28:24Z

.github/workflows/linux-qe.yml

+          # run
+          cmd="crc-qe/run.sh -junitFilename crc-e2e-junit.xml -targetFolder crc-qe"
+          cmd_microshift="${cmd} -e2eTagExpression '@story_microshift'"
+          cmd_openshift="${cmd} -e2eTagExpression '~@minimal && ~@story_microshift'"


Here you created both microshift and openshift diffferent commands.... but you are only running the openshift .. do we have any restriction on the number of machines we can provision? otherwise may you could create a tpl similar to this https://github.com/crc-org/crc/blob/main/.github/workflows/windows-qe-tpl.yml which would be executed on the self hosted runner the template typically provision a machine install crc run e2e and communicate with gh api to add checks on the PR.

With this approach you can run both in parallel each of them on a machine

adrianriobo · 2024-10-30T11:29:00Z

.github/workflows/linux-qe.yml

+            **/*.xml
+            **/*.results
+            **/*.log
+


It is not required to decommission the provisioned machine? May not but I guess it would be a good practice to reduce cost on their end?

openshift-ci · 2024-11-04T06:14:53Z

@lilyLuLiu: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/integration-crc	`f6105d1`	link	true	`/test integration-crc`
ci/prow/e2e-crc	`f6105d1`	link	true	`/test e2e-crc`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

adrianriobo

In general LGTM although I would prefer to tackle the comments. Although I am also fine if you want to have it in ASAP and the create a follow up one to fix the comments.

adrianriobo · 2024-11-07T09:19:31Z

.github/workflows/windows-artifacts.yml

@@ -40,6  40,14 @@ jobs:
        with:
          name: windows-installer
          path: "./out/windows-amd64/crc-windows-installer.zip"
+      - name: Build Linux binary


Can you create a follow up issue on this one, about to renaming this action (as now it also includes linux build, it would be great to properly name it).

Also ensure linux build is only done in one place

adrianriobo · 2024-11-07T09:22:35Z

.github/workflows/linux-qe-test.yml

+      - completed
+
+jobs:
+  linux-e2e-ocp:


why linux-e2e-ocp? may better linux-qe

as it includes integration, e2e and microshift no?

adrianriobo · 2024-11-07T09:49:20Z

.github/workflows/linux-qe-template.yml

+            -e SSH_AUTH_SOCK=$(cat ssh_auth_sock) \
+            -v "$(cat ssh_auth_sock):$(cat ssh_auth_sock)" \
+            -v ${PWD}:/data:z \
+            quay.io/rhqp/crc-support:v0.5.1-linux return2testingfarm


What is this?

This is how the target is decommissioned? so you have to run it from inside the target 🤔

Tough if this is the reason Can you change the name of the step to make it clear? Or consider adding some comments on it to help others out to better understand the step.

adrianriobo · 2024-11-07T09:52:31Z

.github/workflows/linux-qe-template.yml

+          echo "commit_sha=${commit_sha}" >> "$GITHUB_ENV"
+          mkdir ${{ env.commit_sha }}-${{inputs.qe-type}}-${{inputs.preset}}
+          cd ${{ env.commit_sha }}-${{inputs.qe-type}}-${{inputs.preset}}
+


May consider add some comments in here to explain why we need the socket strategy on the target..i.e. the target can only be accessed through a bastion (which can only be accessed from self-hosted runner) as so we need to map the ssh-agent from the host to the containers used to access the target host

adrianriobo · 2024-11-07T09:55:48Z

.github/workflows/linux-qe-template.yml

+            -v "$(cat ssh_auth_sock):$(cat ssh_auth_sock)" \
+            -v ${PWD}:/data:z \
+            -v ${PWD}/crc:/opt/crc-support/crc:z \
+            quay.io/rhqp/crc-support:v0.5.1-linux crc-support/install.sh crc-support/crc


quay.io/crc-org/ci-crc-support:v1.0.0-linux Is now available. If possible change to this as from now on it should be the "official" one.

Let me do a check on using it directly with crc as a local asset. And will add here the snippet to make use of it here.

openshift-ci bot added the do-not-merge/work-in-progress label Aug 19, 2024

lilyLuLiu force-pushed the main branch 18 times, most recently from 44eb1cf to bd41140 Compare August 26, 2024 06:09

lilyLuLiu force-pushed the main branch from bd41140 to b7b20a0 Compare August 26, 2024 08:12

crc-org deleted a comment from cfergeau Aug 27, 2024

crc-org deleted a comment from lilyLuLiu Aug 27, 2024

crc-org deleted a comment from praveenkumar Aug 27, 2024

lilyLuLiu force-pushed the main branch from 16fb425 to 0d5b6b9 Compare September 5, 2024 03:49

lilyLuLiu removed the do-not-merge/work-in-progress label Sep 5, 2024

lilyLuLiu marked this pull request as ready for review September 5, 2024 03:51

openshift-ci bot requested a review from cfergeau September 5, 2024 03:51

openshift-ci bot requested a review from gbraad September 5, 2024 03:51

adrianriobo reviewed Sep 18, 2024

View reviewed changes

lilyLuLiu marked this pull request as draft September 27, 2024 05:57

openshift-ci bot added the do-not-merge/work-in-progress label Sep 27, 2024

lilyLuLiu force-pushed the main branch from 0d5b6b9 to 07649fa Compare October 29, 2024 06:57

lilyLuLiu marked this pull request as ready for review October 30, 2024 08:56

openshift-ci bot removed the do-not-merge/work-in-progress label Oct 30, 2024

openshift-ci bot requested a review from evidolob October 30, 2024 08:56

adrianriobo reviewed Oct 30, 2024

View reviewed changes

lilyLuLiu force-pushed the main branch from 07649fa to 5aa3387 Compare November 1, 2024 03:43

crc-org deleted a comment from adrianriobo Nov 1, 2024

add testing farm github action

f6105d1

lilyLuLiu force-pushed the main branch from 5aa3387 to f6105d1 Compare November 4, 2024 04:12

adrianriobo reviewed Nov 7, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add GitHub action that use Testing Farm #4320

add GitHub action that use Testing Farm #4320

lilyLuLiu commented Aug 19, 2024 •

edited

Loading

openshift-ci bot commented Aug 19, 2024

openshift-ci bot commented Aug 19, 2024

adrianriobo Sep 18, 2024

lilyLuLiu commented Sep 26, 2024

lilyLuLiu commented Oct 22, 2024 •

edited

Loading

adrianriobo Oct 30, 2024

adrianriobo Oct 30, 2024

adrianriobo Oct 30, 2024

adrianriobo Oct 30, 2024

adrianriobo Oct 30, 2024

adrianriobo Oct 30, 2024

adrianriobo Oct 30, 2024

adrianriobo Oct 30, 2024

adrianriobo Oct 30, 2024

openshift-ci bot commented Nov 4, 2024

adrianriobo left a comment

adrianriobo Nov 7, 2024

adrianriobo Nov 7, 2024

adrianriobo Nov 7, 2024

adrianriobo Nov 7, 2024

adrianriobo Nov 7, 2024

add GitHub action that use Testing Farm #4320

Are you sure you want to change the base?

add GitHub action that use Testing Farm #4320

Conversation

lilyLuLiu commented Aug 19, 2024 • edited Loading

openshift-ci bot commented Aug 19, 2024

openshift-ci bot commented Aug 19, 2024

Choose a reason for hiding this comment

lilyLuLiu commented Sep 26, 2024

lilyLuLiu commented Oct 22, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

openshift-ci bot commented Nov 4, 2024

adrianriobo left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lilyLuLiu commented Aug 19, 2024 •

edited

Loading

lilyLuLiu commented Oct 22, 2024 •

edited

Loading