Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support to output declaration files or .dts on build or service.transform #95

Closed
aelbore opened this issue May 9, 2020 · 28 comments
Closed

Comments

@aelbore
Copy link

aelbore commented May 9, 2020

in typescript you can add on tsconfig.json "declaration": true, it can output or produce dts files. hope you can support on this :)
by the way nice work.

@evanw
Copy link
Owner

evanw commented May 9, 2020

This is not something that I'm planning to support. The parser in esbuild skips over type annotations as if they were whitespace, so the AST doesn't contain any type annotations. This means esbuild doesn't have the information necessary to generate a .d.ts file.

@aelbore
Copy link
Author

aelbore commented May 9, 2020

@evanw thanks for the reply, we can close this or reopen if soon its in your roadmap.
nice work.

@aelbore aelbore closed this as completed May 9, 2020
@jaredpalmer
Copy link

jaredpalmer commented Jan 15, 2021

@evanw I was just speaking with client at [redacted] (2nd or 3rd largest user of TS in the world). They believe you could potentially solve this by parsing files and stripping away implementation, in a second pass, but only keeping annotations and signatures. This isn't how the actual TS compiler does it, it's much more complex. However, according to [redacted], the above solution would work and be extremely fast.

Another reason you might want to take up this issue is that esbuild would become THE go to way of compiling all things TS, not just apps. Food for thought.

@wgebczyk
Copy link

wgebczyk commented Jan 21, 2021

@evanw I would as well like to get support to generate .d.ts - I'm not Xnd largest user of TS, but still like to have that option. Maybe you could point into place where such support might be placed and I would try myself to add it?

Of course if your vision for this tool is that it will not be supported, then of course will stop complaining about that.

@a-b-r-o-w-n
Copy link

Also consider library authors that want to use esbuild to compile their TS library but still include *.d.ts files for users. Right now, there is an additional burden on the package author to complicate their build process by requiring multiple steps (esbuild tsc), and reduces the overall value of esbuild by increasing the time to compile.

I have experimented with creating a plugin that emits declaration files, but the typescript api is not well suited for parsing a single file at a time and the plugin api doesn't offer a way to do work after all files have been processed -- thus creating a bottleneck.

@evanw Do you have any thoughts for library maintainers on this? Is your guidance to use a mutli-stage build? If I'm still forced to use the typescript compiler, is there still a compelling reason to use esbuild?

@ycjcl868
Copy link
Contributor

1

@ycjcl868
Copy link
Contributor

ycjcl868 commented Feb 19, 2021

Also consider library authors that want to use esbuild to compile their TS library but still include *.d.ts files for users. Right now, there is an additional burden on the package author to complicate their build process by requiring multiple steps (esbuild tsc), and reduces the overall value of esbuild by increasing the time to compile.

I have experimented with creating a plugin that emits declaration files, but the typescript api is not well suited for parsing a single file at a time and the plugin api doesn't offer a way to do work after all files have been processed -- thus creating a bottleneck.

@evanw Do you have any thoughts for library maintainers on this? Is your guidance to use a mutli-stage build? If I'm still forced to use the typescript compiler, is there still a compelling reason to use esbuild?

I think a reason is the bundled speed faster than other tools.

esbuild for bundled and tsc --emitDeclarationOnly for declaration might be the current solution, for example: esbuild src/index.ts --bundle --platform=node --target=node10 --outdir=dist && tsc --emitDeclarationOnly --outDir dist

image

@mattrossman
Copy link

@ycjcl868 Have you found a way to speed up the declarations step? It seems silly that esbuild can finish the bundle in a fraction of a second, while tsc takes multiple seconds just to extract types.

@jalal246
Copy link

jalal246 commented Mar 5, 2021

@mattrossman This is really a good point and a real struggle. You bundle the whole project in less than a second then you have to wait to emit types. Feels like I am doing something wrong. Another point when bundling external dependency, the code is already bundled successfully but I have to find a way to include the external types.

@prantlf
Copy link

prantlf commented Mar 6, 2021

Another fast TypeScript compiler (swc) lacks this feature too, but according to their main developer it should be possible to implement.

@splashsky
Copy link

I agree on this Issue - Webpack has a neat plugin to bundle declaration files neatly into a single-file declaration. I want to advocate esbuild as much as possible, and this seems like a really common sense feature to focus on.

@markbrocato
Copy link

I really want ESBuild to output typings. I'd be able to ditch tsc completely and save several seconds on every build while watching in dev mode.

@Floffah
Copy link

Floffah commented May 10, 2021

I made a small plugin for doing this - esbuild-plugin-d.ts
It uses the typescript compiler api rather than just calling tsc so it will still add time onto the build.

@kdy1
Copy link

kdy1 commented Jul 10, 2021

I found this while closing outdated issues. I'm writing a comment because I made a mistake.

Another fast TypeScript compiler (swc) lacks this feature too, but according to their main developer it should be possible to implement.

It's not possible without implementing a full typescript type system.
I thought it to be possible, but it was not and I'm working on a new typescript type checker.


Btw, this is sort of extremely hard stuff and I'm not sure if it's a viable option for an open-source project. I used an enormous amount of time, but still not done.

@eric-burel
Copy link

Hi guys, I am ok with this feature not being supported out of the box, but could it please be somehow documented, even just 1 sentence leading to the right documentation?

As far as I understand this thread, I am supposed to fallback to tsc to generate the definitions. That's perfectly acceptable in my context: I write a library of packages, so the d.ts files are only needed when publishing the full library to get correct autocompletion. This step is not mandatory for each build.
But I'd love to see a line on this in the TS documentation of Esbuild.

@evanw
Copy link
Owner

evanw commented Nov 16, 2021

Good point. I added this to the documentation: https://esbuild.github.io/content-types/#no-type-system.

@eric-burel
Copy link

eric-burel commented Jan 7, 2022

For the record I've done this eventually:

    "build": "yarn run build:common && yarn run build:types",
    "build:common": "esbuild index.ts --outdir=dist --sourcemap --bundle --minify",
    "build:types": "tsc --emitDeclarationOnly --declaration --project tsconfig.build.json"

I am just discovering Esbuild, but it seems to work ok so far!

Building the types is taking most of the build time. But since the Esbuild time is negligible, it still means a 40% cutdown on the build time, eg from 15 seconds with webpack to 0.24 seconds with Esbuild 8 seconds of building the types.

Edit: actually I had to dig a bit more for a neutral build : "build:common": "esbuild index.ts --outdir=dist --sourcemap --bundle --minify --platform=neutral --main-fields=module,main,browser --external:tty"

But that's unrelated to .d.ts, using  tsc configured to only output definition file works perfect.

@eric-burel
Copy link

I have created an independent repository to demo building full-stack, typed packages, with Esbuild and other tools: https://github.com/VulcanJS/npm-the-right-way

I've used tsc directly for the typings with success, however at the moment the Esbuild version doesn't work in Next.js, somehow the imports are not reckognized.

I'd be glad to get some help 🙏 , so we can figure a setup that works and use it as canonical example

@kettanaito
Copy link

kettanaito commented Mar 4, 2022

A couple of suggestions from my side:

Use the pre and post npm script prefixes

{
  "scripts": {
    "build": "esbuild ...",
    "postbuild": "tsc ..."
  }
}

This will run postbuild any time build completes.

Use a simple esbuild plugin

// esbuild.js
import { execSync } from 'child_process'
import { build } from 'esbuild'

build({
  ...options,
  plugins: [
    {
      name: 'TypeScriptDeclarationsPlugin',
      setup(build) {
        build.onEnd((result) => {
          if (result.errors.length > 0) return
          execSync('tsc')
        })
      }
    } 
  ]
})

@blutorange
Copy link

So I suppose rewriting the TS compiler is out of the question. Somebody on the swc thread mentioned assuming all types are explicit to make this easier. They also said

All of ours are anyway, because we've enabled typescript-eslint/explicit-module-boundary-types.

Personally, we're using the same constraint on our code as well, and I think it would be quite helpful even with that restriction.

Unfortunately, that restriction is not enough. The ESLint rule only applied to exported symbols. It doesn't require explicit types for internal constants and functions. TS has a typeof type, so the following doesn't violate the ESRule, but still requires a full type checker:

function bar<T>(arr: T[]) {
    for (const a of arr) { if (typeof a === 'string') return a }
    return x;
} 
export function foo(x: string): ReturnType<typeof bar> {
    return undefined as any;
}

So we'd either have to ban the typeof type as well, or resolve the referenced symbols and require explicit type annotations for those as well. And perhaps I'm missing some other features of TS that require further restrictions?

Would you even consider implementing a limited feature that imposes some restriction on the TS input files?

@benjamincburns
Copy link

Looks like the creator of SWC (@kdy1) is actively working on a fast type checker called STC. It's apparently not ready for primetime yet, but I'm mentioning it here for tracking purposes, and in hopes that some people reading this thread might want to pitch in and help get it over the finish line, as it would likely help esbuild offer this feature as well as other more full-featured TS support.

@ehaynes99
Copy link

ehaynes99 commented Feb 22, 2023

So we'd either have to ban the typeof type as well, or resolve the referenced symbols and require explicit type annotations for those as well. And perhaps I'm missing some other features of TS that require further restrictions?

Would you even consider implementing a limited feature that imposes some restriction on the TS input files?

It's not possible, even with explicitly specified with "normal" types, because you still have to track down where those types came from and determine if they're valid. A tool that is capable of doing that would effectively be a complete type checker.

type BarStuff = ReturnType<typeof bar>

type OtherType = { foo: string; bar: BarStuff }

type ExplicitType = OtherType['bar']

export function foo(x: string): ExplicitType {
    // ...
}

It's really important to note, however, that even if esbuild could emit type definitions, you still have to run tsc to verify the validity of the code. This is true even with apps where you don't need the type definitions.

esbuild will happily compile:

export const value: number = 'oh no!'

into

"use strict";
export const value = "oh no!";

And if it could emit type definitions like suggested above, it would then happily emit:

export declare const value: number; // WRONG!

There's no point in using TypeScript if nothing between you and production checks the types.

tsc has 3 jobs:

  • check that you've used types correctly throughout the entire source tree
  • strip the typescript out and emit javascript (<--- esbuild does this)
  • strip the javascript out and emit type definitions

The first one is the time consuming part. Emitting the files is relatively quick:

$ time ./node_modules/.bin/tsc -p tsconfig.build.json --emitDeclarationOnly

real    0m8.901s
user    0m14.255s
sys     0m0.590s

$ time ./node_modules/.bin/tsc -p tsconfig.build.json --noEmit

real    0m8.723s
user    0m13.855s
sys     0m0.637s

So you wouldn't be saving 8.9s, only a maximum of 0.178s.

Not trying to downplay the significance of esbuild or swc. They're fantastic at what they do, but they're also doing a lot less work than tsc. You need to build an app during development many hundreds of times, and you don't really need to wait for a full type-check on every keystroke, but you have to do it eventually before releasing something, so the definitions can be built then.

@aleclarson
Copy link

I believe esbuild could support declaration bundles without too much hassle by stripping non-exported statements, exported function bodies, and exported variable initializers. If an exported function/variable isn't explicitly typed, assume unknown. I think this would be a fair compromise, since public APIs should have explicit contracts anyway (to avoid regressions).

@evanw
Copy link
Owner

evanw commented May 23, 2023

@aleclarson You'll be excited to learn that there's an effort to officially support exactly this within the TypeScript project itself: microsoft/TypeScript#47947. The proposal is a flag called --isolatedDeclarations and is similar to the existing --isolatedModules flag. If you enable this proposed flag, the TypeScript compiler would then restrict you to only using types in a way that an external tool without a type system (e.g. esbuild) could process them into a .d.ts file without compromises. Work on this proposal is progressing actively: microsoft/TypeScript#53463. There's also some more context in this thread: microsoft/TypeScript#53500.

@ehaynes99
Copy link

ehaynes99 commented May 24, 2023

@aleclarson What's the point? It still doesn't absolve you of having to run tsc for type checking, yet inevitably, people will start skipping it because they'll be under the illusion that this is actually doing what they want. But it wouldn't. The exported type definitions would never be verified as correct, which is effectively just bypassing the entire type system. Literally every type in the entire project could be wrong, and it would still export the nonsense type definition.

  • The build step would pass as long as esbuild could emit JavaScript
  • Tests using esbuild as the transformer would still pass as long as the compiled JavaScript worked
  • eslint doesn't report typescript errors
  • you might think the red squiggles in an editor would save you, but that does nothing when you change an exported value that breaks other files

Other projects would import those types. If those use tsc, without skipLibCheck, the types would finally be checked by the consuming app and fail to build. With skipLibCheck, the incorrect types would surface in those applications at runtime.

The flag @evanw mentions would at least enforce the new "rules", but... it would be done by using tsc, so still no escape from it. esbuild would essentially need to replicate all of the functionality of that flag to be able to emit correct types, and even then, you'll need TypeScript support to get feedback in the editor about the new requirements.

@blutorange
Copy link

I can't speak for others, but for us, this could be useful when we have multiple projects where one project depends on another and we want to do quick dev / snapshot builds. Although I agree the risk of generating invalid type declarations that break things down the pipleline is high and hard to debug for other people without intimate knowledge of the build process, so this might not be worth it. I guess we really want a faster tsc command ; )

@aleclarson
Copy link

@ehaynes99 I run both tsc --noEmit and my test runner in watch mode while developing. The test runner compiles TypeScript from source files. Only bundle the source files and emit type declarations after development cycles.

@bschlenk
Copy link

I think esbuild generating .d.ts files would be great for monorepos, where:

  • there are lots of dependencies on other packages within the repo
  • you need to build those dependencies locally to generate the .js / .d.ts files (unless you want to check in all your output)
  • you already know types are correct because that was run in CI

In this case you can still be confident without real type checking, but still have the types locally in the consumers of other monorepo packages.

mandarini pushed a commit to nrwl/nx that referenced this issue May 16, 2024
<!-- Please make sure you have read the submission guidelines before
posting an PR -->
<!--
https://github.com/nrwl/nx/blob/master/CONTRIBUTING.md#-submitting-a-pr
-->

<!-- Please make sure that your commit message follows our format -->
<!-- Example: `fix(nx): must begin with lowercase` -->

## Current Behavior
esbuild doesn't support the creation of declaration files (*.d.ts) and
probably never will (see evanw/esbuild#95).
Since declaration files are essential for published libraries,
it would be great if @nx/esbuild:esbuild would provide an option to
output them.

## Expected Behavior

- Introduced a new boolean valued `declaration` option for the `esbuild`
executor
- If `declaration` or the tsconfig option
[declaration](https://www.typescriptlang.org/tsconfig#declaration) is
true,
then the TypeScript compiler is used before esbuild to generate the
declarations
- The output directory structure of the declarations can be influenced
by setting the TypeScript rootDir via the `declarationRootDir` option

## Related Issue(s)
#20688

### Additional Comment
Please note that the generated declaration files directory structure is
affected by the tsconfig `rootDir` property.

For a library that doesn't reference other libraries inside the
monorepo, the `rootDir` property can be changed freely.
If a library inside the monorepo is referenced the `rootDir` property
must be set to the workspace root.

The `tsc` executor has a sophisticated check that automatically sets the
`rootDir` to the workspace root if a library is referenced.

https://github.com/nrwl/nx/blob/master/packages/js/src/executors/tsc/tsc.impl.ts#L104

This check is quite complex and specific to the `tsc` executor options.
Therefore, it hasn't been included inside the esbuild implementation.

The current implementation leaves it to the user to solve the edge case
by removing the `declarationRootDir` option or by setting the
`declarationRootDir` to `.`.

In the future, it might make sense to generalize and use the `tsc`
executor check.
FrozenPandaz pushed a commit to nrwl/nx that referenced this issue May 21, 2024
<!-- Please make sure you have read the submission guidelines before
posting an PR -->
<!--
https://github.com/nrwl/nx/blob/master/CONTRIBUTING.md#-submitting-a-pr
-->

<!-- Please make sure that your commit message follows our format -->
<!-- Example: `fix(nx): must begin with lowercase` -->

## Current Behavior
esbuild doesn't support the creation of declaration files (*.d.ts) and
probably never will (see evanw/esbuild#95).
Since declaration files are essential for published libraries,
it would be great if @nx/esbuild:esbuild would provide an option to
output them.

## Expected Behavior

- Introduced a new boolean valued `declaration` option for the `esbuild`
executor
- If `declaration` or the tsconfig option
[declaration](https://www.typescriptlang.org/tsconfig#declaration) is
true,
then the TypeScript compiler is used before esbuild to generate the
declarations
- The output directory structure of the declarations can be influenced
by setting the TypeScript rootDir via the `declarationRootDir` option

## Related Issue(s)
#20688

### Additional Comment
Please note that the generated declaration files directory structure is
affected by the tsconfig `rootDir` property.

For a library that doesn't reference other libraries inside the
monorepo, the `rootDir` property can be changed freely.
If a library inside the monorepo is referenced the `rootDir` property
must be set to the workspace root.

The `tsc` executor has a sophisticated check that automatically sets the
`rootDir` to the workspace root if a library is referenced.

https://github.com/nrwl/nx/blob/master/packages/js/src/executors/tsc/tsc.impl.ts#L104

This check is quite complex and specific to the `tsc` executor options.
Therefore, it hasn't been included inside the esbuild implementation.

The current implementation leaves it to the user to solve the edge case
by removing the `declarationRootDir` option or by setting the
`declarationRootDir` to `.`.

In the future, it might make sense to generalize and use the `tsc`
executor check.

(cherry picked from commit 7f32d86)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests