Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: npm workspace and better Deno workspace support #24334

Merged
merged 101 commits into from
Jul 4, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
101 commits
Select commit Hold shift click to select a range
3660270
Start on workspace refactor.
dsherret Jun 17, 2024
2c591ef
Some work on tasks
dsherret Jun 18, 2024
81dc4a8
Switching more to workspace
dsherret Jun 18, 2024
271df70
More.
dsherret Jun 18, 2024
b155d9a
Remove cli/args/package_json.rs
dsherret Jun 18, 2024
c1d52f8
some work on deno compile
dsherret Jun 18, 2024
c8dbbb1
tasks
dsherret Jun 19, 2024
04fd3c6
update publishing
dsherret Jun 19, 2024
03b30f4
Adding fmting
dsherret Jun 19, 2024
1e8b196
Merge branch 'main' into workspace_start
dsherret Jun 19, 2024
cb6b545
some work on linting
dsherret Jun 19, 2024
5e9fd3f
finish lint
dsherret Jun 20, 2024
4fe5441
Some work on lsp and installing from all the packages
dsherret Jun 20, 2024
19ad5c4
separate out test options from workspace test options
dsherret Jun 20, 2024
6eed364
finish up test
dsherret Jun 20, 2024
c04afc7
Add bench
dsherret Jun 20, 2024
49b1a23
compiling
dsherret Jun 20, 2024
76dcc18
handle base properly
dsherret Jun 20, 2024
4205667
fix slow types linting
dsherret Jun 20, 2024
2033d89
Use SharedLocal and fix some vendor tests.
dsherret Jun 21, 2024
297d191
Start making import map optional
dsherret Jun 21, 2024
8583651
Start investigating @types issue
dsherret Jun 21, 2024
df4e20e
Fix. Wasn't thinking.
dsherret Jun 21, 2024
a7b5857
fix info
dsherret Jun 21, 2024
c5df354
fix watcher tests
dsherret Jun 21, 2024
77c0977
working npm tests
dsherret Jun 21, 2024
d38c216
some import specifier fixes
dsherret Jun 22, 2024
ffa9bb1
Fix some lsp issues.
dsherret Jun 24, 2024
e887d93
Fix test
dsherret Jun 24, 2024
64c20ac
We understand the npm workspace now
dsherret Jun 24, 2024
171bed6
Make deno compile work with relative specifiers
dsherret Jun 24, 2024
5c330c1
Fix in import_map/deno_emit
dsherret Jun 25, 2024
2b4247c
Compile fixes
dsherret Jun 25, 2024
3c9fec7
Fix some byonm stuff
dsherret Jun 25, 2024
e691125
Merge branch 'main' into workspace_start
dsherret Jun 25, 2024
4d3ac9e
Update to actual deps
dsherret Jun 26, 2024
e844ee1
refactor: move `PackageJson` to deno_config
dsherret Jun 26, 2024
5770c6b
Update
dsherret Jun 26, 2024
1ca7b82
fix failing tests
dsherret Jun 26, 2024
6b65431
Update deno_config
dsherret Jun 26, 2024
25439b0
fix failing test by clearing package json cache
dsherret Jun 26, 2024
c72410e
fix formatting
dsherret Jun 26, 2024
f41a348
Merge branch 'main' into workspace_start
dsherret Jun 26, 2024
b1e4645
Merge branch 'package_json_deno_config' into workspace_start
dsherret Jun 26, 2024
11eb6ff
Fixes for latest changes
dsherret Jun 26, 2024
1e28b4e
clippy
dsherret Jun 26, 2024
f626f10
Resolve todo
dsherret Jun 26, 2024
3fe1853
update from what will be main
dsherret Jun 26, 2024
c4b55bf
Resolve some todos
dsherret Jun 26, 2024
698d9aa
Merge branch 'main' into workspace_start
dsherret Jun 26, 2024
1281f76
Merge branch 'main' into workspace_start
dsherret Jun 26, 2024
217a217
fix failing tests
dsherret Jun 26, 2024
163bef4
npm workspace resolution
dsherret Jun 27, 2024
05ce93e
handle going to npm workspace deps for npm specifiers too
dsherret Jun 27, 2024
de1b298
Add test for workspace
dsherret Jun 27, 2024
2fd7ffa
add an npm specifier referencing a workspace package
dsherret Jun 27, 2024
0a9e1c3
move this down to deno_config
dsherret Jun 27, 2024
33ab587
updates for latest
dsherret Jun 27, 2024
2c35049
update to new Workspace::discover signature
dsherret Jun 28, 2024
6e9c118
Support `@workspace` tag in an npm specifier
dsherret Jun 28, 2024
a063f9a
Support pkg json "workspace:" specifiers
dsherret Jun 28, 2024
91c3c1b
Move test
dsherret Jun 28, 2024
8851529
Initialize node_modules dir with used workspace members
dsherret Jun 28, 2024
8dd40a9
Fix some tests and update var names
dsherret Jun 29, 2024
66c863b
Update workspace basic test
dsherret Jun 29, 2024
5df7730
Merge branch 'main' into workspace_start
dsherret Jun 29, 2024
44c189e
Merge branch 'main' into workspace_start
dsherret Jun 29, 2024
07a57b4
Update for failing compile
dsherret Jun 29, 2024
ebfe959
Merge branch 'main' of https://github.com/denoland/deno into workspac…
dsherret Jul 1, 2024
9a1e921
improve resolving root dir for deno compile
dsherret Jul 1, 2024
497c22d
Resolve more todos about errors in sub path resolution
dsherret Jul 1, 2024
b0fc65d
Add tests for and fix formatting
dsherret Jul 1, 2024
914be64
Fix some issues and add tests for linting
dsherret Jul 1, 2024
bb1988e
Add no slow types workspace lint test
dsherret Jul 1, 2024
36c1ca0
Merge branch 'main' into workspace_start
dsherret Jul 1, 2024
7fee5f6
Format and ignore some files from formatting
dsherret Jul 1, 2024
ea41334
Add tests for exports not found in npm workspace
dsherret Jul 1, 2024
59fb3db
Merge branch 'main' into workspace_start
dsherret Jul 1, 2024
f8cf42a
diagnostics
dsherret Jul 1, 2024
57fdbde
Add tests for lockfile going to the workspace root
dsherret Jul 1, 2024
8a97d8c
Add test for check
dsherret Jul 2, 2024
9ce4caa
Update deno_config
dsherret Jul 2, 2024
803b097
Add bench tests
dsherret Jul 2, 2024
754b308
add test
dsherret Jul 2, 2024
9d38ecf
Merge branch 'main' into workspace_start
dsherret Jul 2, 2024
6aaac91
fix specs
dsherret Jul 2, 2024
1448564
was using a backslash in a path
dsherret Jul 2, 2024
3f7569f
Bump ci cache
dsherret Jul 2, 2024
7e6644f
Switch back to resolving based on multiple args.
dsherret Jul 3, 2024
db95f58
update some tests
dsherret Jul 3, 2024
43f1f5f
Merge branch 'main' into workspace_start
dsherret Jul 3, 2024
372ceed
update tests
dsherret Jul 3, 2024
0c98628
fix copyright
dsherret Jul 3, 2024
51bed91
add tests for vendor
dsherret Jul 3, 2024
8ac81df
update test for pkg with no deps in lockfile in workspace
dsherret Jul 3, 2024
2a957e0
Merge branch 'main' into workspace_start
dsherret Jul 3, 2024
650a43d
- Fix private registries not working with auto-install in deno compile
dsherret Jul 3, 2024
ae7ebec
maybe fix windows tests
dsherret Jul 3, 2024
4d0d459
Updates from review
dsherret Jul 3, 2024
21673a3
Merge branch 'main' into workspace_start
dsherret Jul 3, 2024
b282f75
Upgrade deno_unsync. Remove fatal diagnostic test because it's in den…
dsherret Jul 4, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Fix some issues and add tests for linting
  • Loading branch information
dsherret committed Jul 1, 2024
commit 914be648c388a01375faa53ca231e661e96305eb
12 changes: 6 additions & 6 deletions cli/tools/fmt.rs
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 129,12 @@ pub async fn format(flags: Flags, fmt_flags: FmtFlags) -> Result<(), AnyError> {
Ok(())
}

struct PathsWithOptions {
base: PathBuf,
paths: Vec<PathBuf>,
options: FmtOptions,
}

fn resolve_paths_with_options_batches(
cli_options: &CliOptions,
fmt_flags: &FmtFlags,
Expand All @@ -154,12 160,6 @@ fn resolve_paths_with_options_batches(
Ok(paths_with_options_batches)
}

struct PathsWithOptions {
base: PathBuf,
paths: Vec<PathBuf>,
options: FmtOptions,
}

async fn format_files(
caches: &Arc<Caches>,
fmt_flags: &FmtFlags,
Expand Down
103 changes: 63 additions & 40 deletions cli/tools/lint/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -91,28 91,15 @@ pub async fn lint(flags: Flags, lint_flags: LintFlags) -> Result<(), AnyError> {
Ok(async move {
let factory = CliFactory::from_flags(flags)?;
let cli_options = factory.cli_options();
let mut linter = WorkspaceLinter::new(
factory.caches()?.clone(),
factory.module_graph_creator().await?.clone(),
cli_options.workspace.clone(),
&cli_options.resolve_workspace_lint_options(&lint_flags)?,
);
let lint_config = cli_options.resolve_lint_config()?;
for (ctx, lint_options) in
cli_options.resolve_lint_options_for_members(&lint_flags)?
{
let files =
collect_lint_files(cli_options, lint_options.files.clone())
.and_then(|files| {
if files.is_empty() {
Err(generic_error("No target files found."))
} else {
Ok(files)
}
})?;
_ = watcher_communicator.watch_paths(files.clone());

let lint_paths = if let Some(paths) = &changed_paths {
let mut paths_with_options_batches =
resolve_paths_with_options_batches(cli_options, &lint_flags)?;
for paths_with_options in &mut paths_with_options_batches {
_ = watcher_communicator
.watch_paths(paths_with_options.paths.clone());

let files = std::mem::take(&mut paths_with_options.paths);
paths_with_options.paths = if let Some(paths) = &changed_paths {
// lint all files on any changed (https://github.com/denoland/deno/issues/12446)
files
.iter()
Expand All @@ -126,13 113,25 @@ pub async fn lint(flags: Flags, lint_flags: LintFlags) -> Result<(), AnyError> {
} else {
files
};
}

let mut linter = WorkspaceLinter::new(
factory.caches()?.clone(),
factory.module_graph_creator().await?.clone(),
cli_options.workspace.clone(),
&cli_options.resolve_workspace_lint_options(&lint_flags)?,
);
for paths_with_options in paths_with_options_batches {
linter
.lint_files(lint_options, lint_config.clone(), ctx, lint_paths)
.lint_files(
paths_with_options.options,
lint_config.clone(),
paths_with_options.ctx,
paths_with_options.paths,
)
.await?;
}
if linter.file_count == 0 {
return Err(generic_error("No target files found."));
}

linter.finish();

Ok(())
Expand Down Expand Up @@ -174,18 173,18 @@ pub async fn lint(flags: Flags, lint_flags: LintFlags) -> Result<(), AnyError> {
cli_options.workspace.clone(),
&workspace_lint_options,
);
for (ctx, lint_options) in
cli_options.resolve_lint_options_for_members(&lint_flags)?
{
let files = &lint_options.files;
let target_files = collect_lint_files(cli_options, files.clone())?;
let paths_with_options_batches =
resolve_paths_with_options_batches(cli_options, &lint_flags)?;
for paths_with_options in paths_with_options_batches {
linter
.lint_files(lint_options, lint_config.clone(), ctx, target_files)
.lint_files(
paths_with_options.options,
lint_config.clone(),
paths_with_options.ctx,
paths_with_options.paths,
)
.await?;
}
if linter.file_count == 0 {
return Err(generic_error("No target files found."));
}
linter.finish()
};
if !success {
Expand All @@ -196,6 195,36 @@ pub async fn lint(flags: Flags, lint_flags: LintFlags) -> Result<(), AnyError> {
Ok(())
}

struct PathsWithOptions {
ctx: WorkspaceMemberContext,
paths: Vec<PathBuf>,
options: LintOptions,
}

fn resolve_paths_with_options_batches(
cli_options: &CliOptions,
lint_flags: &LintFlags,
) -> Result<Vec<PathsWithOptions>, AnyError> {
let members_lint_options =
cli_options.resolve_lint_options_for_members(lint_flags)?;
let mut paths_with_options_batches =
Vec::with_capacity(members_lint_options.len());
for (ctx, lint_options) in members_lint_options {
let files = collect_lint_files(cli_options, lint_options.files.clone())?;
if !files.is_empty() {
paths_with_options_batches.push(PathsWithOptions {
ctx,
paths: files,
options: lint_options,
});
}
}
if paths_with_options_batches.is_empty() {
return Err(generic_error("No target files found."));
}
Ok(paths_with_options_batches)
}

type WorkspaceModuleGraphFuture =
SharedLocal<LocalBoxFuture<'static, Result<Rc<ModuleGraph>, Rc<AnyError>>>>;

Expand Down Expand Up @@ -277,12 306,6 @@ impl WorkspaceLinter {
.collect::<HashSet<_>>();
futures.push(
async move {
// let graph = module_graph_creator
// .create_and_validate_publish_graph(
// &vec![publish_config.clone()],
// true,
// )
// .await?;
let graph = workspace_module_graph_future
.await
.map_err(|err| anyhow!("{:#}", err))?;
Expand Down
15 changes: 15 additions & 0 deletions tests/specs/lint/workspace/__test__.jsonc
Original file line number Diff line number Diff line change
@@ -0,0 1,15 @@
{
"tests": {
"root_check": {
"args": "lint",
"exitCode": 1,
"output": "root.out"
},
"subdir_check": {
"args": "lint",
"cwd": "a",
"exitCode": 1,
"output": "a.out"
}
}
}
32 changes: 32 additions & 0 deletions tests/specs/lint/workspace/a.out
Original file line number Diff line number Diff line change
@@ -0,0 1,32 @@
error[no-eval]: `eval` call is not allowed
--> [WILDLINE]a.ts:1:1
|
1 | eval("");
| ^^^^^^^^
= hint: Remove the use of `eval`

docs: https://lint.deno.land/rules/no-eval


error[no-await-in-loop]: Unexpected `await` inside a loop.
--> [WILDLINE]a.ts:4:3
|
4 | await Deno.open("test");
| ^^^^^^^^^^^^^^^^^^^^^^^
= hint: Remove `await` in loop body, store all promises generated and then `await Promise.all(storedPromises)` after the loop

docs: https://lint.deno.land/rules/no-await-in-loop


error[no-explicit-any]: `any` type is not allowed
--> [WILDLINE]a.ts:9:25
|
9 | export function test(): any {
| ^^^
= hint: Use a specific type other than `any`

docs: https://lint.deno.land/rules/no-explicit-any


Found 3 problems
Checked 1 file
11 changes: 11 additions & 0 deletions tests/specs/lint/workspace/a/a.ts
Original file line number Diff line number Diff line change
@@ -0,0 1,11 @@
eval("");

for (let i = 0; i < 10; i ) {
await Deno.open("test");
}

const unused = 1;

export function test(): any {
return {};
}
12 changes: 12 additions & 0 deletions tests/specs/lint/workspace/a/deno.json
Original file line number Diff line number Diff line change
@@ -0,0 1,12 @@
{
"lint": {
"rules": {
"include": [
"no-await-in-loop"
],
"exclude": [
"no-unused-vars"
]
}
}
}
11 changes: 11 additions & 0 deletions tests/specs/lint/workspace/b/b.ts
Original file line number Diff line number Diff line change
@@ -0,0 1,11 @@
eval("");

for (let i = 0; i < 10; i ) {
await Deno.open("test");
}

const unused = 1;

export function test(): any {
return {};
}
9 changes: 9 additions & 0 deletions tests/specs/lint/workspace/b/deno.json
Original file line number Diff line number Diff line change
@@ -0,0 1,9 @@
{
"lint": {
"rules": {
"exclude": [
"no-explicit-any"
]
}
}
}
11 changes: 11 additions & 0 deletions tests/specs/lint/workspace/deno.json
Original file line number Diff line number Diff line change
@@ -0,0 1,11 @@
{
"workspace": [
"./a",
"./b"
],
"lint": {
"rules": {
"include": ["no-eval"]
}
}
}
82 changes: 82 additions & 0 deletions tests/specs/lint/workspace/root.out
Original file line number Diff line number Diff line change
@@ -0,0 1,82 @@
error[no-eval]: `eval` call is not allowed
--> [WILDLINE]root.ts:1:1
|
1 | eval("");
| ^^^^^^^^
= hint: Remove the use of `eval`

docs: https://lint.deno.land/rules/no-eval


error[no-unused-vars]: `unused` is never used
--> [WILDLINE]root.ts:7:7
|
7 | const unused = 1;
| ^^^^^^
= hint: If this is intentional, prefix it with an underscore like `_unused`

docs: https://lint.deno.land/rules/no-unused-vars


error[no-explicit-any]: `any` type is not allowed
--> [WILDLINE]root.ts:9:25
|
9 | export function test(): any {
| ^^^
= hint: Use a specific type other than `any`

docs: https://lint.deno.land/rules/no-explicit-any


error[no-eval]: `eval` call is not allowed
--> [WILDLINE]a\a.ts:1:1
|
1 | eval("");
| ^^^^^^^^
= hint: Remove the use of `eval`

docs: https://lint.deno.land/rules/no-eval


error[no-await-in-loop]: Unexpected `await` inside a loop.
--> [WILDLINE]a\a.ts:4:3
|
4 | await Deno.open("test");
| ^^^^^^^^^^^^^^^^^^^^^^^
= hint: Remove `await` in loop body, store all promises generated and then `await Promise.all(storedPromises)` after the loop

docs: https://lint.deno.land/rules/no-await-in-loop


error[no-explicit-any]: `any` type is not allowed
--> [WILDLINE]a.ts:9:25
|
9 | export function test(): any {
| ^^^
= hint: Use a specific type other than `any`

docs: https://lint.deno.land/rules/no-explicit-any


error[no-eval]: `eval` call is not allowed
--> [WILDLINE]b.ts:1:1
|
1 | eval("");
| ^^^^^^^^
= hint: Remove the use of `eval`

docs: https://lint.deno.land/rules/no-eval


error[no-unused-vars]: `unused` is never used
--> [WILDLINE]b.ts:7:7
|
7 | const unused = 1;
| ^^^^^^
= hint: If this is intentional, prefix it with an underscore like `_unused`

docs: https://lint.deno.land/rules/no-unused-vars


Found 8 problems
Checked 3 files
11 changes: 11 additions & 0 deletions tests/specs/lint/workspace/root.ts
Original file line number Diff line number Diff line change
@@ -0,0 1,11 @@
eval("");

for (let i = 0; i < 10; i ) {
await Deno.open("test");
}

const unused = 1;

export function test(): any {
return {};
}