Multi-tenancy through catalogs in MariaDB Server

Let’s say you are a Cloud Service Provider, with many customers – each having many MariaDB Server users and databases. What if several such customers could share a single instance of MariaDB Server? That’s what we call the catalog feature, a feature that – if implemented – could potentially save lots of resources (and thus costs!) in a number of high-end use cases.

How the idea was born

At CloudFest 2023 near Frankfurt in March, we had in-depth meetings with a number of heavy MariaDB Server users – ones that one best would describe as Cloud Service Providers (CSPs). In the context of MariaDB Server, they sell running instances of MariaDB Server. It turns out that many users – in particular those running WordPress blogs – have very light use of MariaDB, in which case it begs the question whether they need their own server instance in the first place. Couldn’t they share it, without being subject to the limitation of each customer having only one database schema?

Sharing MariaDB Server instances isn’t trivial

Sure, one MariaDB Server instance can have many users, and users can have different rights for various databases (also called schemas) which are being handled by the instance. That’s trivial. But if you think of the concept of CSP customer, then you would likely want to group the usage and billing of all the users of that customer into one bundle. And the customer would need a root user, handling all the user rights of that customer.

That’s already complicated. Without catalogs, users and databases are not being grouped in one single instance of MariaDB Server. Setting up access rights is notoriously difficult and error-prone; Customer A’s users having rights to access all databases would see all Customer B’s databases. Not nice. Defeats the purpose of separating customers.

Enter Catalogs: Bundling users and databases

The idea with catalogs is to introduce a new level of abstraction between the MariaDB Server instance and individual databases (including the system database that contains the users). This new catalog concept would in practice have a one-to-one relationship to CSP customers.

Let us say that mercedes and bmw are both customers of IONOS (a randomly chosen CSP that suddenly came to mind because they happen to be a sponsor of MariaDB Foundation). Then they can both have a database called cars. Upon connecting, Mercedes would provide mercedes as part of the API call connecting to the database, and BMW would provide bmw.

Also, system variables will generally be catalog specific. Individual customers are thus individually configurable and observable.

What is also cool is that if the CSP uses catalogs, the database schemas used by a client on-prem can map one-to-one onto a catalog in the cloud. No need to change schemas based on artificial constraints of “one schema per customer”, as one catalog can have many databases (which for MariaDB Server is synonymous with schema).

Why does this save resources?

MariaDB Server prides itself on being very configurable when it comes to memory settings and other parameters. Still, there is the InnoDB Buffer Pool and other memory areas, which do need to exist on a MariaDB Server instance level. In case each CSP customer gets their own MariaDB Server instance, that memory lies idling around between calls, and it takes resources to set up.

Saving those memory areas is why cost savings for large users of MariaDB Server may be in the seven figures per year, and at any rate much higher than the development cost of the feature.

What does “multi-tenant” refer to?

The first time I heard the term “multi-tenant”, I had to go to the dictionary. To save my fellow non-English natives a visit to the bookshelf or Google Translate, let me say that a “tenant” is the opposite of “land-lord”. A tenant rents a flat (or apartment) from a landlord. So “multi-tenant” in the cloud context refers to users sharing resources in some way, be it a computer, a virtual machine, or (in our case) a database server instance.

Are there drawbacks? Yes, but small ones

Once implemented, the catalog feature comes nearly without drawbacks. One is that it’s a new feature which needs to be implemented, in cPanel, in Plesk, in whatever surrounding software is used in giving CSP customers access to MariaDB Server.

The other one is that different customers still can slightly interfere with each other. In the unlikely event that Mercedes crashes their MariaDB Server instance, then it crashes also for BMW, as they share the same instance. Obvious. But server crashes aren’t that common, anyway.

But I heard “catalog” being used with a different meaning?

I have another word of warning, which is that the term “catalog” isn’t as much of an industry standard one as “table” or “view”. In the process of writing this blog entry, I googled for “catalogs”, and different database management systems use this term for different purposes.

So why did we pick “catalog”? Because we didn’t find any other more commonly used word. And because the word “catalog” was already used in the protocol for this very purpose, without having been implemented in the server. Finally, our usage of the word “catalog” doesn’t necessarily differ from that of Oracle Database or Postgres.

Be that as it may, readers of this blog entry know what we mean by catalogs. And I believe it’s a useful feature.

Call to action: Early adopters

If you agree it’s a useful feature and would like to be involved in the specification, development, and testing phase, do tell us. Catalogs are a feature which requires quite a bit of coding, for which MariaDB Foundation lacks the development resources. Hence, the development of the feature is subject to finding financing, or “feature sponsors”, if you will. And in order to get this to fit the bill of what the main users want, we expect to work more closely with the early adopters also from a specs, development, and testing perspective.

But rest assured. Obviously, catalogs will be a normal feature of MariaDB Server in due course. The more interest we see, the faster it will be developed.