Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE]: Expose Rebuild index via Alter API for users. #8738

Closed
MichelDiz opened this issue Mar 7, 2023 · 9 comments
Closed

[FEATURE]: Expose Rebuild index via Alter API for users. #8738

MichelDiz opened this issue Mar 7, 2023 · 9 comments
Labels
dgraph Issue or PR created by an internal Dgraph contributor. kind/feature Something completely new we should consider. Stale

Comments

@MichelDiz
Copy link
Contributor

MichelDiz commented Mar 7, 2023

Use case

The user can run this function to possibly get more performance.

(go buildIndexes(su, rebuild, closer))

There are reports that the simple fact of removing the index and putting it back. It resulted in perf changes. The goal here is to give Rebuild access directly. Without the user having to go all around.

Current state

Have to be done by manually removing the index.

Solution proposal

Give an operation for this.

e.g:

op.setRebuildAll(true);

OR

op.setRebuildattr("age");

Also (maybe? why not?)
op.setRebuildattr("dgraph.type");
@MichelDiz MichelDiz added kind/feature Something completely new we should consider. dgraph Issue or PR created by an internal Dgraph contributor. labels Mar 7, 2023
@MichelDiz MichelDiz changed the title [FEATURE]: Expose Rebuild (go buildIndexes(su, rebuild, closer)) index via Alter API for users. [FEATURE]: Expose Rebuild index via Alter API for users. Mar 7, 2023
@mangalaman93
Copy link
Contributor

@MichelDiz this is already available through the Alter API. You can drop indexes and then, add them back again and Dgraph will build the indexes again.

@MichelDiz
Copy link
Contributor Author

That is not true. When you drop the predicate it will DELETE the tablet. The objective here is to have a non-destructive rebuild.

Please, close an issue with proves. I couldn't find any "drop indexes only" or "rebuild indexes" in the Schema API.

There is only:
dropAll
dropData
dropAttr
dropNs(namespace)

PS. If you don't get the objective, ask to clarify. I can improve the text to be more clear if the case.

@MichelDiz MichelDiz reopened this Mar 8, 2023
@mangalaman93
Copy link
Contributor

How will that benefit? Internally the implementation will drop and recreate the indexes?

@MichelDiz
Copy link
Contributor Author

MichelDiz commented Mar 8, 2023

I don't believe there are any drops(for data). Dgraph has always done this, rebuilding the index on the fly with each new Schema change. It was after I requested for this to stop that we didn't do it anymore. I remember Martin doing that. Since then I noticed that users were complaining about indexing. So I suspected it was this change.

We didn't drop the data. We just deleted the index and then we recreate it.

@MichelDiz
Copy link
Contributor Author

MichelDiz commented Mar 8, 2023

Here the issue #2828

That was one of many changes the devs did back then. In the reindexing and schema code in general.

@mangalaman93
Copy link
Contributor

I still am not sure what is expected here. Maybe elaborate more. Currently we have enough of options for the user to delete attributes, any indexes (one or more) and recreate them.

@MichelDiz
Copy link
Contributor Author

MichelDiz commented Mar 8, 2023

Rebuild ≠ delete attributes

any indexes (one or more) and recreate them

How? I believe we are not talking about the same thing.


And there is a way to rebuild the index, but it's not obvious. And it's not professional either.
The way is, remove the index, wait and put it back again. This is not obvious.
Let's say the user really needs to rebuild indexing. We haven't documented anything about that. We do not recommend anything about this anywhere. They learn by trial and error. So it's better to have an explicit function than a "hidden way".


In Neo4J you can do

SHOW INDEXES YIELD name RETURN 'DROP INDEX '   name  ';'

And this will drop only the index table, not the data or the schema. They also have other functions like "verify".

Rebuild indexes is a common thing in any Database. Not sure why you are so confused with the concept.
You can throw on google "rebuild index" and you gonna see thousands of examples in SQL and others.

The point is that, if this option exists in the market of databases, why we don't? Is there any specific reason?
Rebuilding indexes are considered a maintenance function due fragmentation.


quote:

The script below allows you to rebuild indexes for all databases and all tables within a database. This could be further tweaked
to handle only indexes that need maintenance based on fragmentation levels as well as then doing either an index reorg or an index rebuild.

The script uses two cursors one for the databases and another for the tables within the database.

Rebuild All Indexes Script for SQL Server 2005 and Later

https://www.mssqltips.com/sqlservertip/1367/sql-server-script-to-rebuild-all-indexes-for-all-tables-and-all-databases/

ALTER INDEX oldindex REBUILD PARAMETERS('tablespace=TBS_3');

https://docs.oracle.com/database/121/SPATL/alter-index-rebuild.htm#SPATL1017

Optimize index maintenance to improve query performance and reduce resource consumption

https://learn.microsoft.com/en-us/sql/relational-databases/indexes/reorganize-and-rebuild-indexes?view=sql-server-ver16

@mangalaman93
Copy link
Contributor

I think I see your point now. You are right that we don't have an explicit API for rebuilding indexes.

Copy link

This issue has been stale for 60 days and will be closed automatically in 7 days. Comment to keep it open.

@github-actions github-actions bot added the Stale label Jul 24, 2024
@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Aug 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dgraph Issue or PR created by an internal Dgraph contributor. kind/feature Something completely new we should consider. Stale
Development

No branches or pull requests

2 participants