Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Kata to use qemu-6.1 #2502

Merged
merged 3 commits into from
Sep 15, 2021
Merged

Update Kata to use qemu-6.1 #2502

merged 3 commits into from
Sep 15, 2021

Conversation

dgibson
Copy link
Contributor

@dgibson dgibson commented Aug 25, 2021

qemu-6.1 is now released. This includes a bunch of things we've been carrying around patches for, and also ACPI hotplug support for q35 which we need to fix some future problems.

Update our packaging to use the new qemu version.

@dgibson
Copy link
Contributor Author

dgibson commented Aug 25, 2021

/test
/test-vfio

Copy link
Contributor

@sameo sameo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice diff :)

@bergwolf
Copy link
Member

On qemu-experimental, does qemu-6.1 support virtio-fs dax?

@dgibson
Copy link
Contributor Author

dgibson commented Aug 31, 2021

On qemu-experimental, does qemu-6.1 support virtio-fs dax?

I think so, but I don't know the code well enough to be sure.

@bergwolf
Copy link
Member

@stefanha could you help to clarify qemu v6.1 status of virtio-fs dax support?

@stefanha
Copy link
Contributor

@stefanha could you help to clarify qemu v6.1 status of virtio-fs dax support?

DAX is not yet supported in QEMU 6.1.

@bergwolf
Copy link
Member

bergwolf commented Sep 1, 2021

Thanks @stefanha !

Hi @dgibson, since dax is not supported by qemu v6.1, we still need the qemu-experimental branch as its main goal is to provide a virtiofs dax-enabled version of qemu.

@dgibson
Copy link
Contributor Author

dgibson commented Sep 1, 2021

Hi @dgibson, since dax is not supported by qemu v6.1, we still need the qemu-experimental branch as its main goal is to provide a virtiofs dax-enabled version of qemu.

Yes, and more to the point I'll need to propagate and update the patches in the kata tree which add dax support; I incorrectly thought I could drop them. I'll update as soon as I get a chance.

@dgibson dgibson linked an issue Sep 2, 2021 that may be closed by this pull request
@dgibson
Copy link
Contributor Author

dgibson commented Sep 2, 2021

/test
/test-vfio

@dgibson
Copy link
Contributor Author

dgibson commented Sep 2, 2021

Ok, I've changed the series to keep the experimental branch, and move the virtiofs-dax patches forwards to it. Not entirely sure how to test that the patches still all apply properly on the experimental version - AFAICT the CI doesn't use it. I'm trying to do a manual attempt with Wainer's vagrant stuff, but I'm not really sure what I'm doing.

@dgibson
Copy link
Contributor Author

dgibson commented Sep 2, 2021

@stefanha the virtio fs dax patches that are in the Kata tree no longer apply clean to v6.1. I could probably rebase them, but it seems like it would be more sensible to grab a new snapshot of the virtio-fs-dax branch. Where would I find that?

@stefanha
Copy link
Contributor

stefanha commented Sep 2, 2021

@stefanha the virtio fs dax patches that are in the Kata tree no longer apply clean to v6.1. I could probably rebase them, but it seems like it would be more sensible to grab a new snapshot of the virtio-fs-dax branch. Where would I find that?

@dagrh can advise about the best git tree to use. He's working on upstreaming DAX in QEMU.

@dgibson
Copy link
Contributor Author

dgibson commented Sep 7, 2021

Ok, new version updates qemu-experimental to current tree that @dagrh pointed me towards.

@dgibson
Copy link
Contributor Author

dgibson commented Sep 7, 2021

/test
/test-vfio

@dgibson dgibson mentioned this pull request Sep 7, 2021
Copy link
Member

@jcvenegas jcvenegas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, seems that qemu 6.1.0 build in snap is broken, l will block this PR with requested-changes

@jcvenegas
Copy link
Member

/qemu-experimental

@jcvenegas
Copy link
Member

@dgibson I is green but there is a problem with s390x build

@GabyCT
Copy link
Contributor

GabyCT commented Sep 9, 2021

The error is the following

Applying patch: /home/jenkins/workspace/kata-containers-2.0-ubuntu-s390x-PR/go/src/github.com/kata-containers/kata-containers/tools/packaging/qemu/patches/6.1.x/*.patch
error: can't open patch '/home/jenkins/workspace/kata-containers-2.0-ubuntu-s390x-PR/go/src/github.com/kata-containers/kata-containers/tools/packaging/qemu/patches/6.1.x/*.patch': No such file or directory

/cc @Jakob-Naucke

If the script doesn't find a patches directory it expects, it gives an
error saying to create a dummy 'no_patches' file if you really don't want
any patches applied for that version.

But actual practice in the tree is to call the dummy file 'no_patches.txt'
rather than simply 'no_patches'.  Correct the message to match existing
practice.

Signed-off-by: David Gibson <[email protected]>
@dgibson
Copy link
Contributor Author

dgibson commented Sep 10, 2021

1

The error is the following

Applying patch: /home/jenkins/workspace/kata-containers-2.0-ubuntu-s390x-PR/go/src/github.com/kata-containers/kata-containers/tools/packaging/qemu/patches/6.1.x/*.patch
error: can't open patch '/home/jenkins/workspace/kata-containers-2.0-ubuntu-s390x-PR/go/src/github.com/kata-containers/kata-containers/tools/packaging/qemu/patches/6.1.x/*.patch': No such file or directory

/cc @Jakob-Naucke

Gah. Looks like the scripts in the tests repo break if the patches directory (not the tag_patches directory) has no patches in it, even with no_patches.txt. Why s390 is going through that path when the other builds don't, I have no idea, but for whatever reason it is.

@dgibson
Copy link
Contributor Author

dgibson commented Sep 10, 2021

Urgh. Ok, so the "static build" path used by most of the targets uses the patching scripts from within the kata-containers repo, which work. The other build path uses patching logic from the tests repo, which breaks in this case.

@dgibson
Copy link
Contributor Author

dgibson commented Sep 10, 2021

/test
/test-vfio
/qemu-experimental

@devimc
Copy link

devimc commented Sep 10, 2021

please fix your Depends-on -> Depends-on: github.com/kata-containers/tests#3958

@jcvenegas
Copy link
Member

/qemu-experimental

@dgibson
Copy link
Contributor Author

dgibson commented Sep 11, 2021

/test
/test-vfio

We need qemu-6.1 for ACPI PCI hotplug support for the q35 machine.  At the
moment qemu will use SHPC hotplug under the PCIe to PCI bridge on q35.
SHPC is too slow to use for our purposes (it requires a 5s delay).

Update the qemu version to v6.1.0.  This leaves the experimental version
*older* than the normal version, but we'll fix that up later.

We also need to tweak the snapcraft.yaml, since the location for configs
has changed in the new qemu version.

fixes kata-containers#1691

Signed-off-by: David Gibson <[email protected]>
This brings it back into line with the normal qemu version.  We refer to
v6.1.0 by full SHA in versions.yaml, rather than the tag, so that
apply_patches.sh sees it as different and applies the virtiofs DAX patches
which is what the experimental version is actually about having.

The virtiofs DAX patches themselves are updated to the version from
https://gitlab.com/virtio-fs/qemu, virtio-fs-dev branch as of commit
3620cb0a.

Signed-off-by: David Gibson <[email protected]>
@dgibson
Copy link
Contributor Author

dgibson commented Sep 11, 2021

/test
/test-vfio

@dgibson
Copy link
Contributor Author

dgibson commented Sep 11, 2021

/qemu-experimental

@dgibson
Copy link
Contributor Author

dgibson commented Sep 11, 2021

/retest-clh

@dgibson
Copy link
Contributor Author

dgibson commented Sep 12, 2021

/retest-ubuntu

2 similar comments
@dgibson
Copy link
Contributor Author

dgibson commented Sep 12, 2021

/retest-ubuntu

@dgibson
Copy link
Contributor Author

dgibson commented Sep 13, 2021

/retest-ubuntu

@Jakob-Naucke
Copy link
Member

See #2618 for the now-occurring s390x failure, which is unrelated to this PR.

@dgibson
Copy link
Contributor Author

dgibson commented Sep 14, 2021

See #2618 for the now-occurring s390x failure, which is unrelated to this PR.

Thanks for the information. I suspected that failure was unrelated, but it's good to have it confirmed.

I think we're ready to merge. Any objections?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Update qemu and qemu-experimental to 6.1.0
9 participants