Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Eliminate fixed-content tokens from serialized AST data 🚀 #10755

Open
Tracked by #10752
kazcw opened this issue Aug 5, 2024 · 0 comments
Open
Tracked by #10752

Eliminate fixed-content tokens from serialized AST data 🚀 #10755

kazcw opened this issue Aug 5, 2024 · 0 comments

Comments

@kazcw
Copy link
Contributor

kazcw commented Aug 5, 2024

Many AST fields always have the same corresponding text, e.g. the import keyword in an import statement, or the = operator in an assignment. We should not serialize these fields, and in turn not create Token objects for them in Java and JS.

In JS we will need to codegen type-specific printing logic to insert the known strings when concretizing. We will also need spacing information for the elided tokens. We can represent this with a SMI-sized bitfield. The bitfield too can be elided as long as its value is the default for its tree type. In case the user does something unusual like "x = 23", we can fall back to a full representation.

This change will probably not affect backend code at all, as the backend has no reason to look at these fields, and uses span info for any printing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: New
Development

No branches or pull requests

2 participants