Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(@effect/sql): SqlQuery module #3309

Closed
wants to merge 8 commits into from

Conversation

dilame
Copy link
Contributor

@dilame dilame commented Jul 20, 2024

Add SqlQuery module with findAll, void, findOne, and single dual APIs for running SQL queries with schema validation.

SqlQuery.findAll

Runs an SQL query and validates the results against a provided schema, returning all results as an array.

import { Schema } from "@effect/schema"
import { SqlClient, SqlQuery } from "@effect/sql"
import { Effect } from "effect"

const userSchema = Schema.struct({
  id: Schema.number,
  name: Schema.string,
})

const findAllUsers = Effect.gen(function* () {
  const sql = yield* SqlClient.SqlClient

  return yield* SqlQuery.findAll(userSchema)(
    sql`SELECT id, name FROM users`
  )
})

SqlQuery.void

Runs an SQL query and discards the result.

import { SqlClient, SqlQuery } from "@effect/sql"
import { Effect } from "effect"

const updateUser = Effect.gen(function* () {
  const sql = yield* SqlClient.SqlClient

  return yield* SqlQuery.void(
    sql`UPDATE users SET name = "Alice" WHERE id = 1`
  )
})

SqlQuery.findOne

Runs an SQL query and validates the results against a provided schema, returning the first result as an Option.

import { Schema } from "@effect/schema"
import { SqlClient, SqlQuery } from "@effect/sql"
import { Effect, Option } from "effect"

const userSchema = Schema.struct({
  id: Schema.number,
  name: Schema.string,
})

const findUser = Effect.gen(function* () {
  const sql = yield* SqlClient.SqlClient

  const result = yield* SqlQuery.findOne(userSchema)(
    sql`SELECT id, name FROM users WHERE id = 1`
  )
  if(Option.isSome(result)) {
    console.log(result.value)
  } else {
    console.log("User not found")
  }
})

SqlQuery.single

Runs an SQL query and validates the results against a provided schema, returning the first result.
If no result is found, it throws a NoSuchElementException.

import { Schema } from "@effect/schema"
import { SqlClient, SqlQuery } from "@effect/sql"
import { Effect, Console } from "effect"
import * as Cause from "effect/Cause"

const userSchema = Schema.struct({
  id: Schema.number,
  name: Schema.string,
})

const findSingleUser = Effect.gen(function* () {
  const sql = yield* SqlClient.SqlClient

  return yield* SqlQuery.single(userSchema)(
    sql`SELECT id, name FROM users WHERE id = 1`
  ).pipe(
    Effect.catchTag("NoSuchElementException", () => Console.log("User not found"))
  )
})

@dilame dilame requested a review from tim-smart as a code owner July 20, 2024 03:12
Copy link

changeset-bot bot commented Jul 20, 2024

🦋 Changeset detected

Latest commit: bd25c9c

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 12 packages
Name Type
@effect/sql Minor
@effect/cluster Major
@effect/sql-d1 Major
@effect/sql-drizzle Major
@effect/sql-kysely Major
@effect/sql-mssql Major
@effect/sql-mysql2 Major
@effect/sql-pg Major
@effect/sql-sqlite-bun Major
@effect/sql-sqlite-node Major
@effect/sql-sqlite-react-native Major
@effect/sql-sqlite-wasm Major

Not sure what this means? Click here to learn what changesets are.

Click here if you"re a maintainer who wants to add another changeset to this PR

@dilame
Copy link
Contributor Author

dilame commented Jul 20, 2024

Not sure if we really need single because it"s too simple, just Effect.flatten

@tim-smart
Copy link
Contributor

There is also these schema APIs https://effect-ts.github.io/effect/sql/SqlSchema.ts.html#findall

@dilame
Copy link
Contributor Author

dilame commented Jul 20, 2024

There is also these schema APIs https://effect-ts.github.io/effect/sql/SqlSchema.ts.html#findall

Of course i"ve seen it:) And implemented this PR because SqlQuery doesn"t fit my needs. I don"t need to parse input, because it"s already parsed. I would say that it"s a very rare case when you pass input directly to SQL, because usually SQL is not the input edge of the application.

Also, i use "ports and adapters" or "hexagonal" (call as you wish) architecture, where my persistence layer is abstracted, so my application should not know about the underlying driver. It could be SQL DB, redis, or filesystem. SqlSchema module emits SqlError, my application is not allowed to know about it. So in order to map error type i anyway do

export const FetchBotLaunchConfigPostgres = Layer.effect(
  PersistenceFetchBotLaunchConfigImpl,
  Effect.gen(function* () {
    const sql = yield* SqlClient.SqlClient

    return {
      fetchLaunchConfig: (instanceId: string) => {
        return SqlSchema.single({
          Request: Schema.String,
          Result: PersistenceFetchBotLaunchConfigRes,
          execute: (instanceId) =>
            sql`SELECT *
                FROM fxngine.instance_launch_config
                WHERE instance_id = ${instanceId}::bigint`
        })(instanceId).pipe(
          Effect.catchTags({
            SqlError: (cause) =>
              new PersistenceError({ source: "fetchBotLaunchConfig", cause }),
            NoSuchElementException: () =>
              new PersistenceFetchBotLaunchConfigNotFoundError({ instanceId })
          })
        )
      }
    }
  })
)

Which makes no sense

@github-actions github-actions bot requested a review from mikearnaldi as a code owner July 20, 2024 11:41
@github-actions github-actions bot force-pushed the next-minor branch 5 times, most recently from f9b6732 to ca34a85 Compare July 22, 2024 07:44
@datner
Copy link
Member

datner commented Jul 22, 2024

So in order to map error type i anyway do [...] Which makes no sense

Why not? This makes perfect sense. "ports and adapters" thats the adapting part.

export const FetchBotLaunchConfigPostgres = Layer.effect(
  PersistenceFetchBotLaunchConfigImpl,
  Effect.gen(function* () {
    const sql = yield* SqlClient.SqlClient
    const byId = SqlSchema.single({
      Request: Schema.String,
      Result: PersistenceFetchBotLaunchConfigRes,
      execute: (instanceId) =>
      sql`SELECT *
      FROM fxngine.instance_launch_config
      WHERE instance_id = ${instanceId}::bigint`
    })

    return {
      fetchLaunchConfig: (instanceId: string) => byId(instanceId).pipe(
        Effect.catchTags({
          SqlError: (cause) =>
            new PersistenceError({ source: "fetchBotLaunchConfig", cause }),
          NoSuchElementException: () =>
            new PersistenceFetchBotLaunchConfigNotFoundError({ instanceId })
        })
      )
    }
  })
)

doesn"t fit my needs. I don"t need to parse input, because it"s already parsed. I would say that it"s a very rare case when you pass input directly to SQL, because usually SQL is not the input edge of the application.

this is just a misunderstanding of the role of the Request there. It"s not to parse, its to encode. For example your api wants to force that fetching Bar must be from a known Foo. it can do

SqlSchema.single({
  Request: Foo,
  ...,
  execute: (foo) => sql`...${foo.id}`
})

Now lets say that id is a complex expression of multiple values like the shard its in + id of service that created it + it"s own unique id, so its parsed into a nice { shard: string, origin: string, uid: string }. In this case it would encode it back to just the db-represented raw string. You having Schema.String there is not because we expect the user to pass a number, it"s because thats the arbitrary input you decided for this exposed function, it"s still working as expected.

There is no question about if "it"s a very rare case when you pass input directly to SQL", as this is an implicit assumption that input that reached this level would be valid. Remember, schema also does validation, but it"s much more than that.

You should bring your ideas up to debate more often, I promise effect is very well thought out and I feel really bad when you put in this much effort..

@github-actions github-actions bot force-pushed the next-minor branch 9 times, most recently from 95019bb to 263c2b6 Compare July 24, 2024 12:18
@tim-smart
Copy link
Contributor

Going to close this, I don"t think it adds enough value to be an addition to /sql.

@tim-smart tim-smart closed this Aug 20, 2024
@dilame
Copy link
Contributor Author

dilame commented Oct 6, 2024

I understand that the schema is also about encoding, not just validation. However, in my case, encoding is unnecessary for SQL queries. I deal with dozens of SQL queries, and i am forced to use S.Any as a placeholder. The example of encoding an ID is an edge case that applies only in specific, rare situations. I respectfully ask for reconsideration of the PR, as it provides a solution that is suitable for the majority of users who don’t need this kind of encoding for SQL.

Let"s view at this PR from another perspective – how would you suggest to make an SQL that does not requires input encoding?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

6 participants