Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for encryptedDevice OSDs on partitions #14533

Open
samcday opened this issue Aug 5, 2024 · 5 comments
Open

Support for encryptedDevice OSDs on partitions #14533

samcday opened this issue Aug 5, 2024 · 5 comments
Labels

Comments

@samcday
Copy link

samcday commented Aug 5, 2024

Is this a bug report or feature request?

  • Feature Request

What should the feature do:

Allow Ceph clusters deployed by Rook to provision and use encryptedDevice OSDs on disk partitions.

What is use case behind this feature:

I'm using CoreOS on a fleet of commodity machines, many of these machines have a root disk that is much larger than needed. I'd like to carve out a partition on these devices and give that to Ceph so I can make full use of the available disk capacity.

IIRC this (using partitions as OSDs) was once entirely unsupported in Rook/Ceph, right? Seems that you can do this nowadays, but not if you also try to enable the encryptedDevice config option.

Environment:

Rook v1.14.9
Ceph version: 18.2.4 reef (stable)

@samcday samcday added the feature label Aug 5, 2024
samcday added a commit to samcday/home-cluster that referenced this issue Aug 5, 2024
The issue is that Rook disallows creating encryptedDevice OSDs on
partitions. I tried to then provision the luks device myself from
Ignition, but Rook/Ceph rejects that too (for different reasons?).

I do wonder if such a thing could/should be supported in future somehow.
So I've opened rook/rook#14533
@travisn
Copy link
Member

travisn commented Aug 5, 2024

IIRC the encrypted OSDs currently require lvm mode, while Rook only supports partitions in raw mode. @satoru-takeuchi @guits Is that correct? Or what would it take to support this?

@satoru-takeuchi
Copy link
Member

@travisn

Correct. Although Ceph itself supports encryted raw mode osd since this commit, Rook only supports encrypted lvm mode osd.

In Ceph, encrypted lvm mode OSD only supports disk and encrypted raw mode OSD would support both disk and partition. Once I tried to support encryted raw mode OSD in Rook, I canceled this work the implementation would take a long time and I didn't have the rook for this work.

Since I still don't have enough time for this work for now, I expect someone else send a PR.

@samcday
Copy link
Author

samcday commented Aug 6, 2024

I haven't (yet!) made any code contributions to Rook. I'm not sure this would be a good place to start as it seems a little complex and delicate.

Since we've got confirmation now that such a thing is indeed feasible, may we keep this feature request open and hope that a kind (or commercially motivated) soul contributes the necessary patches?

I have short-term plans to grow my cluster further, and also longer term plans that, if successful, would mean a lot of further growth too. At some point I may find I have enough SATA/nvme GBs hiding between the couch cushions that I roll up my own sleeves here ;)

Copy link

github-actions bot commented Oct 7, 2024

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in a week if no further activity occurs. Thank you for your contributions.

@samcday
Copy link
Author

samcday commented Oct 8, 2024

Ping to convince the robot overlords this issue should (please) stay open

@github-actions github-actions bot removed the wontfix label Oct 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants