Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Progressively migrate from knex to kysely. #630

Open
alnorris opened this issue Aug 3, 2023 · 12 comments
Open

Progressively migrate from knex to kysely. #630

alnorris opened this issue Aug 3, 2023 · 12 comments
Labels
documentation Improvements or additions to documentation migrations Related to migrations question Further information is requested

Comments

@alnorris
Copy link

alnorris commented Aug 3, 2023

Has anyone done this before? Just wondering if there is anything I should be aware of that might cause issues? I thought about keeping my knex schema migrations until all queries are transitioned over while using kysely-codegen for types.

Perhaps a knex -> kysely migration guide in the docs a codemod would be a good idea?

@igalklebanov igalklebanov added documentation Improvements or additions to documentation question Further information is requested migrations Related to migrations labels Aug 3, 2023
@koskimas
Copy link
Member

koskimas commented Aug 3, 2023

You should be able to use knex and Kysely side by side in the same project without any issues. The only thing I can think of, off the top of my head is that, depending on the setup, you might end up with two connection pools. That might not be an issue, but something to keep in mind.

Knex uses it's own connection pool, so there's no way to share that with Kysely currently. We could in theory create a KnexDialect that takes a knex instance and uses it as the driver 🤔

@korichdaniel
Copy link
Contributor

korichdaniel commented Sep 10, 2023

I did a small patch to work with same connection until I move completely to kysely.

class KnexDatabaseConnection implements DatabaseConnection {
	private trx: Knex.Transaction<unknown, unknown>;

	public async startTransaction(settings: TransactionSettings) {
		this.trx = await knex.transaction(null, settings);
	}

	public async commit() {
		await this.trx?.commit();
	}

	public async rollback() {
		await this.trx.rollback();
	}

	public async executeQuery<R>(compiledQuery: CompiledQuery<unknown>): Promise<QueryResult<R>> {
		return await (this.trx || knex).raw(adaptQueryToKnex(compiledQuery.sql), [
			...compiledQuery.parameters
		] as any[]);
	}

	public async *streamQuery<R>(
		_compiledQuery: CompiledQuery<unknown>,
		_chunkSize?: number
	): AsyncIterableIterator<QueryResult<R>> {
		yield;
	}
}

PostgresDriver.prototype.acquireConnection = async function (): Promise<DatabaseConnection> {
	return new KnexDatabaseConnection();
};

PostgresDriver.prototype.releaseConnection = function (): Promise<void> {
	return;
};

PostgresDriver.prototype.beginTransaction = async function (
	connection: DatabaseConnection,
	settings: TransactionSettings
): Promise<void> {
	await (connection as KnexDatabaseConnection).startTransaction(settings);
};

PostgresDriver.prototype.commitTransaction = async function (connection: DatabaseConnection): Promise<void> {
	await (connection as KnexDatabaseConnection).commit();
};

PostgresDriver.prototype.rollbackTransaction = async function (connection: DatabaseConnection): Promise<void> {
	await (connection as KnexDatabaseConnection).rollback();
};

const kysely = new Kysely<DB>({
	// @ts-expect-error PostgresDialect requires a connection to the DB, we want to avoid it as
	// we are only using kysely as a pure query builder and performing the actual operation via knex
	dialect: new PostgresDialect({})
});

function adaptQueryToKnex(kyselyQuery: string) {
	return kyselyQuery.replace(/\$[0-9] /g, '?');
}

@lghartmann
Copy link

At my company we started migrating from knex to Kysely last week, everything was going fine until today, the file with the generated types is pretty long and the autocomplete stopped working (it was working fine last week), my senior is testing different TS versions and some other stuff

@igalklebanov
Copy link
Member

At my company we started migrating from knex to Kysely last week, everything was going fine until today, the file with the generated types is pretty long and the autocomplete stopped working (it was working fine last week), my senior is testing different TS versions and some other stuff

How many tables?

@lghartmann
Copy link

At my company we started migrating from knex to Kysely last week, everything was going fine until today, the file with the generated types is pretty long and the autocomplete stopped working (it was working fine last week), my senior is testing different TS versions and some other stuff

How many tables?

Currently 629 tables, the generated types file is approximately 9.5k lines long, we're currently using typescript 5.3.3.

@igalklebanov
Copy link
Member

igalklebanov commented Mar 18, 2024

At my company we started migrating from knex to Kysely last week, everything was going fine until today, the file with the generated types is pretty long and the autocomplete stopped working (it was working fine last week), my senior is testing different TS versions and some other stuff

How many tables?

Currently 629 tables, the generated types file is approximately 9.5k lines long, we're currently using typescript 5.3.3.

Try splitting - e.g. kysely-codegen has a filter argument. There's no healthy way a single service should know all tables.

@lghartmann
Copy link

At my company we started migrating from knex to Kysely last week, everything was going fine until today, the file with the generated types is pretty long and the autocomplete stopped working (it was working fine last week), my senior is testing different TS versions and some other stuff

How many tables?

Currently 629 tables, the generated types file is approximately 9.5k lines long, we're currently using typescript 5.3.3.

Try splitting - e.g. kysely-codegen has a filter argument. There's no healthy way a single service should know all tables.

We spent the afternoon investigating this issue. Whenever i started a fresh file autocomplete worked just fine (even with 629 tables generated), as soon as finished and saved, closed the files, reloaded window (VSCode) and reopened the TS file the autocomplete stayed as "loading..." Then we applied your mentioned filter, generated tables we're reduced to 49, i recreated the same file, tested it again, working fine, sometimes it gets a little slow to load the autocomplete but its pretty ok, it looks like as the file gets bigger the autocomplete gets slower (already felt it slower with 2 simple select queries). Thanks igalklebanov

@igalklebanov
Copy link
Member

We've published https://github.com/kysely-org/kysely-knex, give it a try. Let us know how it went!

@the21st
Copy link

the21st commented Apr 11, 2024

@koskimas @igalklebanov How would you suggest we migrate existing database migrations managed by knex if we want to migrate to kysely?

Edit: to be more specific, how would you suggest we handle the transition from our existing knex_migrations table in our production DB to kysely_migrations ?

@igalklebanov
Copy link
Member

The easy way - just delete the old migrations and start anew.
The hard way - I'll investigate.

@igalklebanov
Copy link
Member

igalklebanov commented Apr 14, 2024

v0.2 released with a custom migration source that allows using Kysely in Knex-managed migrations, side-by-side with Knex.

@the21st
Copy link

the21st commented Apr 15, 2024

@igalklebanov wow, thanks so much! Will check it out with our team soon 🙂

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation migrations Related to migrations question Further information is requested
Projects
None yet
Development

No branches or pull requests

6 participants