Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add bigint type #525

Merged
merged 10 commits into from
Nov 5, 2020
Merged

Add bigint type #525

merged 10 commits into from
Nov 5, 2020

Conversation

littledan
Copy link
Collaborator

@littledan littledan commented Feb 22, 2018

This patch adds a bigint type to WebIDL with the following properties:

  • bigint corresponds directly to BigInt
  • The conversion from JS is based on ToBigInt, namely it throws on Number.
  • bigint is not grouped as a "numeric" type, as this category is often
    used to refer to types which have a JavaScript representation of Number.
  • bigint is included in the overloading resolution algorithm, and it's
    possible to overload a function for BigInt and numeric arguments.

Although it may be on the early side for such a change, the lack of
BigInt WebIDL integration is already coming up in some designs for integrating
BigInt with the Web Platform, such as w3c/IndexedDB#231


Preview | Diff

@bzbarsky
Copy link
Collaborator

It's a little weird that this setup would allow passing "1" as a bigint but not allow passing 1.

I assume bigint should not in fact be a "JSON type" because it can't be represented in JSON? Looks to me like ToJSON throws on BigInts in the BigInt proposal.

@domenic
Copy link
Member

domenic commented Feb 22, 2018

In particular this situation is interesting:

interface X {
  void foo(bigint x);
  void bar(unsigned long long y);
}

What does calling someX.foo("1") do? What should it do?

@bzbarsky
Copy link
Collaborator

What does calling someX.foo("1") do?

The way the PR is written right now, it would call the unsigned long long overload, if you meant to name both your operations the same. If you really did mean different names, then it would convert the string to bigint and call the one thing named "foo".

What should it do?

Good question. ;)

Here's a worse case:

interface X {
  void foo(bigint x);
  void bar(bigint x);
  void bar(Node x);
}

Here foo("1") would convert "1" to bigint, but bar("1") would throw, given the bits in the PR at the moment, because there is no case in https://heycam.github.io/webidl/#dfn-overload-resolution-algorithm step 12 that supports a declared type of bigint with an arg that does not have Type(arg) == BigInt. So the exact changes we want to the overload resolution algorithm here require some thought. Adding a new overload should generally not cause previously-valid calls to throw.

Also, we need corresponding changes to https://heycam.github.io/webidl/#es-to-union; those aren't in the PR afaict.

@littledan
Copy link
Collaborator Author

Also, we need corresponding changes to https://heycam.github.io/webidl/#es-to-union; those aren't in the PR afaict.

Added union support. I also added union support for Symbol (since I didn't see it added yet, and there are some funny cases where it could intersect).

@bzbarsky
Copy link
Collaborator

Please don't do anything with Symbol here. It's not in the algorithm there on purpose, because you can never have a union accepting a Symbol anyway. The only way to pass Symbols in is via any. See discussion in #377

@littledan
Copy link
Collaborator Author

@bzbarsky Reverted symbol changes and added support in the overload algorithm for falling back to a bigint overload as the second-to-last resort (before an any overload).

index.bs Outdated Show resolved Hide resolved
@littledan
Copy link
Collaborator Author

What does calling someX.foo("1") do? What should it do?

I'd say, call the numeric overload over the bigint one. Doesn't this situation already exist in cases like the following, which would also call the numeric overload over the boolean one given a string argument?

interface X {
  void foo(boolean x);
  void foo(unsigned long long y);
}

@bzbarsky
Copy link
Collaborator

I'd say, call the numeric overload over the bigint one

Why? The bigint conversion is the less lossy one in this case, I would think.

which would also call the numeric overload over the boolean one given a string argument?

It would, yes. Arguably conversion to numeric is less lossy than conversion to boolean.

More generally, it seems we're going out of our way to provide seamless conversions from strings (but not other primitives!) to bigints. This is in contrast to numeric and boolean types, which can be converted from all primitives. So in some sense, there is some sort of privileged relationship between bigint and strings that is not shared by other primitive values in JS or other primitive-like idl types...

@bzbarsky
Copy link
Collaborator

bzbarsky commented Feb 27, 2018

Oh, and unlike a lot of overload scenarios, the "overload numeric and bigint" is not completely insane, because if you want to have an API that takes "numbers of arbitrary size" your options are:

  1. Have it just take bigint. Callers can pass BigInt(0) or "0" but not 0.
  2. Have it take (bigint or some-numeric-type). Callers can pass 0 sanely.

Option 1 is slightly harder to use as an API, and the reasoning behind "0" being OK but 0 not being OK is subtle and likely to make anyone trying to just pass a small numeric constant be annoyed.

Option 2 involves an overload of bigint and numeric, and dataloss if you pass a long-enough string and it gets converted to the numeric.

@littledan
Copy link
Collaborator Author

@bzbarsky I see; maybe we should then prohibit that particular overload and force an explicit string overload as well? There could still be some more obscure data loss cases, though.

@bzbarsky
Copy link
Collaborator

There really isn't a mechanism so far for prohibiting an overload but allowing it of more types are added. We could make bigint and numeric not distinguishable, forcing option 1. Or we could add some sort of mechanism along these lines. Or we could make es-to-idl conversion for bigint more like https://tc39.github.io/proposal-bigint/#sec-bigint-constructor-number-value than like https://tc39.github.io/proposal-bigint/#sec-to-bigint I suppose....

@littledan
Copy link
Collaborator Author

Oh, and unlike a lot of overload scenarios, the "overload numeric and bigint" is not completely insane, because if you want to have an API that takes "numbers of arbitrary size"

I'm not sure if that's in line with the goals of the BigInt proposal, to have new APIs which take both out of the box by design. I think most APIs fall into the category of "logically making sense for arbitrary-size integers" or "logically making sense for floating point values"; there's definitely stuff in the intersection ("logically making sense for only small integers"), and for that, Number works well. I was imagining overloading would mostly make sense when trying to upgrade an API from a Number version to a BigInt version, when we initially thought that it made sense only on small integers, but it turned out it was also important on large integers.

Option 1 is slightly harder to use as an API, and the reasoning behind "0" being OK but 0 not being OK is subtle and likely to make anyone trying to just pass a small numeric constant be annoyed.

The hypothesis of the BigInt proposal design is that programmers will keep track of whether something is a BigInt operation or a Number operation, and the types of scalar values, and not do things generically. All the JS programmers I've heard from seem basically on board with this design. Part of this here is, even for small numeric constants, you put an n at the end of the literal. I don't want to encourage a proliferation of Web APIs which

Or we could make es-to-idl conversion for bigint more like https://tc39.github.io/proposal-bigint/#sec-bigint-constructor-number-value than like https://tc39.github.io/proposal-bigint/#sec-to-bigint I suppose....

If we did that, and privileged the bigint path, we'd get surprising exceptions for things like f("3.2"), wouldn't we?

I don't see a great answer here which makes everything beautiful, intuitive and not throw errors. Maybe we should start by making bigint and number indistinguishable, and make sure that unions throw in the ambiguous case. I uploaded a patch which does that.

@bzbarsky
Copy link
Collaborator

bzbarsky commented Mar 2, 2018

Part of this here is, even for small numeric constants, you put an n at the end of the literal.

Ah, if there are simple bigint literals that helps a lot.

Maybe we should start by making bigint and number indistinguishable

Yes, please.

@littledan
Copy link
Collaborator Author

littledan commented Mar 2, 2018

Ah, if there are simple bigint literals that helps a lot.

Yes, BigInt literals in both JavaScript and WebIDL look like 1n.

Yes, please.

How does this patch look?

@inexorabletash
Copy link
Member

Preface: this is all excellent discussion and @littledan's proactive approach to getting bigint integrated into the platform is awesome.

Although it may be on the early side for such a change...

On that note: are there concrete examples of API that plan to consume/produce bigints, either via WebIDL or ES? (Indexed DB is unfortunately not a good example, since it requires custom integration)

I'm slightly nervous that since it's too early to have good examples of bigint usage we may be missing something. @littledan describes "the hypothesis of the BigInt proposal design" which makes sense to me wanting to keep mentally separate from the rest of the platform.... but what will eventual idiomatic use of bigints look like?

@littledan
Copy link
Collaborator Author

WebIDL has a number of cases where it switches among all of the JS primitive types. If BigInt is a type in JavaScript, maybe we should have some description or other of what the behavior is when a BigInt ends up floating into certain operations, such as converting an ES value to any, even if we want to wait until we have more information to decide how to define what it means to overload with a bigint type.

One place BigInt will come up is the WebAssembly-JS API. If the proposed BigInt/i64 integration is combined with the proposed WebAssembly.Global interface, then I think we'd want to say that the type of the WebAssembly.Global.prototype.valueOf method is bigint or unrestricted double, for example. But maybe we could define something special in prose in that particular document, without updating WebIDL so generally.

Besides that, I don't have a particular other web platform feature that where I want to propose BigInt usage. All the examples I can think of (constant time 64-bit arithmetic operations, a date format representing time in nanoseconds since forever ago, etc) might make more sense in the JavaScript standard itself than as WebIDL-based APIs. Like the IndexedDB integration, I'm fine waiting until we have more motivation for this change.

As much as it's appropriate to wait for more applications to get input for the design of the BigInt WebIDL binding, I wanted to document a suggestion for best practices here, at least from the perspective of the way BigInt was designed in JavaScript, in particular, avoiding implicit conversions that lose information, and continuing to use Number exclusively everywhere it is currently used in WebIDL.

@bzbarsky
Copy link
Collaborator

bzbarsky commented Mar 2, 2018

The only way to pass Symbols in is via any

This may not be true. I am checking with Domenic.

any definitely needs to handle all JS types, so we need something there even if we don't add a syntactic way to declare bigints.

@littledan
Copy link
Collaborator Author

Another way to handle both the union and overloading semantics could be to do something similar to what the JS spec does: Call ToNumeric in the ambiguous cases, and make the judgment based on whether a Number or BigInt falls out. This is what we do for most overloaded operators such as -. ToNumeric converts strings to Numbers--this is required, for compatibility. Presumably a similar compatibility risk (but perhaps not as serious, depending how much the method is used) would exist if there's an existing method which takes a numeric type and wants to add a bigint overload, or a bigint member to a union.

@littledan
Copy link
Collaborator Author

Any thoughts on whether we should support overloading between BigInt and Number or not? Note that one specification proposal where this arises is the integration of BigInt and Intl.NumberFormat (not implemented anywhere AFAIK): tc39/ecma402#236 . That PR follows the ToNumeric algorithm as in #525 (comment) .

littledan added a commit to littledan/design-principles that referenced this pull request Dec 4, 2018
Only use BigInt where it is really needed! Number is good for existing
use cases, and existing APIs shouldn't "upgrade" to BigInt unless there
is a very strong reason.

Probably not ready to merge yet, as we will need more practical use cases
to emerge in the web platform to see how this comes up. For the same reason,
WebIDL support is not yet landed, and some open questions remain about how
flexible overloading/unions should be, see
whatwg/webidl#525 (comment)
@annevk
Copy link
Member

annevk commented Dec 5, 2018

I think it actually makes sense to not allow overloading/union BigInt with any other IDL numeric type as that allows for (maybe even encourages) the kind of APIs TC39 doesn't want. Where the API silently does BigInt(arg) if you pass it a non-BigInt after crossing the IDL boundary.

I'm also not sure why we'd support coercing string to BigInt, I think it makes sense to throw there as well.

index.bs Show resolved Hide resolved
index.bs Outdated Show resolved Hide resolved
index.bs Outdated Show resolved Hide resolved
@Ms2ger
Copy link
Member

Ms2ger commented Dec 2, 2019

I think I addressed the remaining comments. @bzbarsky, would you mind reviewing?

@bzbarsky
Copy link
Collaborator

bzbarsky commented Dec 3, 2019

I will try to get to this, but there's a lot to page in and I am pretty swamped with other things at the moment. I can't promise any timeline other than "not in the next two days"...

@annevk
Copy link
Member

annevk commented Apr 30, 2020

So IDB doesn't depend on this as far as I can tell and it's not clear that's moving forward anyway. Are there other places that might use this through IDL? I'd rather not add it until we have a concrete consumer I think.

@ExE-Boss
Copy link
Contributor

ExE-Boss commented Apr 30, 2020

Well, WebAssembly intends to use bigint for i64 handling in JavaScript code: WebAssembly/spec#707.

@Ms2ger
Copy link
Member

Ms2ger commented Apr 30, 2020

I guess the way WebAssembly uses any is problematic wrt the formal definition of any as a discriminated union, and fixing this would require a bigint type in IDL. Note future changes in the reference-types proposal will extend WebAssembly.Global with a bunch of non-integer types.

@annevk
Copy link
Member

annevk commented Apr 30, 2020

Ah, so that would be a problem for postMessage() and such too then? That does seem like something we ought to fix. (The weird thing with WebAssembly is that browsers don't seem to use IDL to implement it.)

@littledan
Copy link
Collaborator Author

Right, if we don't want to permit bigint as an actual IDL type, we should make certain changes to WebIDL to let it "pass through". I've been hoping that this patch would land, so I didn't formulate another one for that subset. (I think WebIDL is just sort of incoherent right now, with BigInt existing as a type in JS.)

I've heard from a few people in the TAG that they are interested in this patch landing. Comments here about why would be appreciated. @cynthia @kenchris

If we conclude that this patch won't land, then let's formulate that more minimal patch and land that instead.

@rtoy
Copy link

rtoy commented Nov 3, 2020

It would make the API for a WebAudio noise generator a bit nicer if BigInt were supported. We currently using an array of two 32-bit unsigned ints to specify the 64-bit seed we need.

@domenic
Copy link
Member

domenic commented Nov 3, 2020

@rtoy the only thing this PR was waiting on was an actual consumer, so if you're providing one, then I think we should go ahead and merge this. @annevk?

@annevk
Copy link
Member

annevk commented Nov 4, 2020

That seems reasonable. And it doesn't seem to want to use overloading so we don't have to address that for now.

@rtoy
Copy link

rtoy commented Nov 4, 2020

Just to follow up, see https://github.com/WebAudio/web-audio-api-v2/issues/8#issuecomment-721439274 and https://github.com/WebAudio/web-audio-api-v2/issues/8#issuecomment-721803254. Using BigInt makes the API a bit nicer, but an array of 2 32-bit unsigned integers works fine too.

Copy link
Member

@domenic domenic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just did a re-review and this looks good to me. @TimothyGu, would you be able to give it a double-check and/or merge?

@TimothyGu TimothyGu removed the do not merge yet Pull request must not be merged per rationale in comment label Nov 5, 2020
@TimothyGu TimothyGu removed the request for review from bzbarsky November 5, 2020 17:04
@TimothyGu
Copy link
Member

CI seems to be failing due to infrastructural reasons.

@TimothyGu TimothyGu merged commit ccef980 into whatwg:master Nov 5, 2020
foolip added a commit to w3c/webref that referenced this pull request Mar 23, 2021
Added to Web IDL in whatwg/webidl#525 but now
used (with wrong case) in w3c/webrtc-encoded-transform#77.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.