Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: npm workspace and better Deno workspace support #24334

Merged
merged 101 commits into from
Jul 4, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
101 commits
Select commit Hold shift click to select a range
3660270
Start on workspace refactor.
dsherret Jun 17, 2024
2c591ef
Some work on tasks
dsherret Jun 18, 2024
81dc4a8
Switching more to workspace
dsherret Jun 18, 2024
271df70
More.
dsherret Jun 18, 2024
b155d9a
Remove cli/args/package_json.rs
dsherret Jun 18, 2024
c1d52f8
some work on deno compile
dsherret Jun 18, 2024
c8dbbb1
tasks
dsherret Jun 19, 2024
04fd3c6
update publishing
dsherret Jun 19, 2024
03b30f4
Adding fmting
dsherret Jun 19, 2024
1e8b196
Merge branch 'main' into workspace_start
dsherret Jun 19, 2024
cb6b545
some work on linting
dsherret Jun 19, 2024
5e9fd3f
finish lint
dsherret Jun 20, 2024
4fe5441
Some work on lsp and installing from all the packages
dsherret Jun 20, 2024
19ad5c4
separate out test options from workspace test options
dsherret Jun 20, 2024
6eed364
finish up test
dsherret Jun 20, 2024
c04afc7
Add bench
dsherret Jun 20, 2024
49b1a23
compiling
dsherret Jun 20, 2024
76dcc18
handle base properly
dsherret Jun 20, 2024
4205667
fix slow types linting
dsherret Jun 20, 2024
2033d89
Use SharedLocal and fix some vendor tests.
dsherret Jun 21, 2024
297d191
Start making import map optional
dsherret Jun 21, 2024
8583651
Start investigating @types issue
dsherret Jun 21, 2024
df4e20e
Fix. Wasn't thinking.
dsherret Jun 21, 2024
a7b5857
fix info
dsherret Jun 21, 2024
c5df354
fix watcher tests
dsherret Jun 21, 2024
77c0977
working npm tests
dsherret Jun 21, 2024
d38c216
some import specifier fixes
dsherret Jun 22, 2024
ffa9bb1
Fix some lsp issues.
dsherret Jun 24, 2024
e887d93
Fix test
dsherret Jun 24, 2024
64c20ac
We understand the npm workspace now
dsherret Jun 24, 2024
171bed6
Make deno compile work with relative specifiers
dsherret Jun 24, 2024
5c330c1
Fix in import_map/deno_emit
dsherret Jun 25, 2024
2b4247c
Compile fixes
dsherret Jun 25, 2024
3c9fec7
Fix some byonm stuff
dsherret Jun 25, 2024
e691125
Merge branch 'main' into workspace_start
dsherret Jun 25, 2024
4d3ac9e
Update to actual deps
dsherret Jun 26, 2024
e844ee1
refactor: move `PackageJson` to deno_config
dsherret Jun 26, 2024
5770c6b
Update
dsherret Jun 26, 2024
1ca7b82
fix failing tests
dsherret Jun 26, 2024
6b65431
Update deno_config
dsherret Jun 26, 2024
25439b0
fix failing test by clearing package json cache
dsherret Jun 26, 2024
c72410e
fix formatting
dsherret Jun 26, 2024
f41a348
Merge branch 'main' into workspace_start
dsherret Jun 26, 2024
b1e4645
Merge branch 'package_json_deno_config' into workspace_start
dsherret Jun 26, 2024
11eb6ff
Fixes for latest changes
dsherret Jun 26, 2024
1e28b4e
clippy
dsherret Jun 26, 2024
f626f10
Resolve todo
dsherret Jun 26, 2024
3fe1853
update from what will be main
dsherret Jun 26, 2024
c4b55bf
Resolve some todos
dsherret Jun 26, 2024
698d9aa
Merge branch 'main' into workspace_start
dsherret Jun 26, 2024
1281f76
Merge branch 'main' into workspace_start
dsherret Jun 26, 2024
217a217
fix failing tests
dsherret Jun 26, 2024
163bef4
npm workspace resolution
dsherret Jun 27, 2024
05ce93e
handle going to npm workspace deps for npm specifiers too
dsherret Jun 27, 2024
de1b298
Add test for workspace
dsherret Jun 27, 2024
2fd7ffa
add an npm specifier referencing a workspace package
dsherret Jun 27, 2024
0a9e1c3
move this down to deno_config
dsherret Jun 27, 2024
33ab587
updates for latest
dsherret Jun 27, 2024
2c35049
update to new Workspace::discover signature
dsherret Jun 28, 2024
6e9c118
Support `@workspace` tag in an npm specifier
dsherret Jun 28, 2024
a063f9a
Support pkg json "workspace:" specifiers
dsherret Jun 28, 2024
91c3c1b
Move test
dsherret Jun 28, 2024
8851529
Initialize node_modules dir with used workspace members
dsherret Jun 28, 2024
8dd40a9
Fix some tests and update var names
dsherret Jun 29, 2024
66c863b
Update workspace basic test
dsherret Jun 29, 2024
5df7730
Merge branch 'main' into workspace_start
dsherret Jun 29, 2024
44c189e
Merge branch 'main' into workspace_start
dsherret Jun 29, 2024
07a57b4
Update for failing compile
dsherret Jun 29, 2024
ebfe959
Merge branch 'main' of https://github.com/denoland/deno into workspac…
dsherret Jul 1, 2024
9a1e921
improve resolving root dir for deno compile
dsherret Jul 1, 2024
497c22d
Resolve more todos about errors in sub path resolution
dsherret Jul 1, 2024
b0fc65d
Add tests for and fix formatting
dsherret Jul 1, 2024
914be64
Fix some issues and add tests for linting
dsherret Jul 1, 2024
bb1988e
Add no slow types workspace lint test
dsherret Jul 1, 2024
36c1ca0
Merge branch 'main' into workspace_start
dsherret Jul 1, 2024
7fee5f6
Format and ignore some files from formatting
dsherret Jul 1, 2024
ea41334
Add tests for exports not found in npm workspace
dsherret Jul 1, 2024
59fb3db
Merge branch 'main' into workspace_start
dsherret Jul 1, 2024
f8cf42a
diagnostics
dsherret Jul 1, 2024
57fdbde
Add tests for lockfile going to the workspace root
dsherret Jul 1, 2024
8a97d8c
Add test for check
dsherret Jul 2, 2024
9ce4caa
Update deno_config
dsherret Jul 2, 2024
803b097
Add bench tests
dsherret Jul 2, 2024
754b308
add test
dsherret Jul 2, 2024
9d38ecf
Merge branch 'main' into workspace_start
dsherret Jul 2, 2024
6aaac91
fix specs
dsherret Jul 2, 2024
1448564
was using a backslash in a path
dsherret Jul 2, 2024
3f7569f
Bump ci cache
dsherret Jul 2, 2024
7e6644f
Switch back to resolving based on multiple args.
dsherret Jul 3, 2024
db95f58
update some tests
dsherret Jul 3, 2024
43f1f5f
Merge branch 'main' into workspace_start
dsherret Jul 3, 2024
372ceed
update tests
dsherret Jul 3, 2024
0c98628
fix copyright
dsherret Jul 3, 2024
51bed91
add tests for vendor
dsherret Jul 3, 2024
8ac81df
update test for pkg with no deps in lockfile in workspace
dsherret Jul 3, 2024
2a957e0
Merge branch 'main' into workspace_start
dsherret Jul 3, 2024
650a43d
- Fix private registries not working with auto-install in deno compile
dsherret Jul 3, 2024
ae7ebec
maybe fix windows tests
dsherret Jul 3, 2024
4d0d459
Updates from review
dsherret Jul 3, 2024
21673a3
Merge branch 'main' into workspace_start
dsherret Jul 3, 2024
b282f75
Upgrade deno_unsync. Remove fatal diagnostic test because it's in den…
dsherret Jul 4, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
More.
  • Loading branch information
dsherret committed Jun 18, 2024
commit 271df70c30a95b7f40a1b0064e671461cfcb48f0
78 changes: 20 additions & 58 deletions cli/args/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1381,21 1381,6 @@ impl CliOptions {
&self.npmrc
}

pub fn maybe_package_json_deps(&self) -> Option<PackageJsonDeps> {
if matches!(
self.flags.subcommand,
DenoSubcommand::Task(TaskFlags { task: None, .. })
) {
// don't have any package json dependencies for deno task with no args
None
} else {
self
.maybe_package_json()
.as_ref()
.map(package_json::get_local_package_json_version_reqs)
}
}

pub fn resolve_fmt_options(
&self,
fmt_flags: FmtFlags,
Expand Down Expand Up @@ -1441,16 1426,6 @@ impl CliOptions {
})
}

pub fn resolve_config_excludes(&self) -> Result<PathOrPatternSet, AnyError> {
let maybe_config_files = if let Some(config_file) = &self.maybe_config_file
{
Some(config_file.to_files_config()?)
} else {
None
};
Ok(maybe_config_files.map(|f| f.exclude).unwrap_or_default())
}

pub fn resolve_test_options(
&self,
test_flags: TestFlags,
Expand Down Expand Up @@ -1555,11 1530,7 @@ impl CliOptions {
}

pub fn check_js(&self) -> bool {
self
.maybe_config_file
.as_ref()
.map(|cf| cf.get_check_js())
.unwrap_or(false)
self.workspace.check_js()
}

pub fn coverage_dir(&self) -> Option<String> {
Expand Down Expand Up @@ -1706,11 1677,7 @@ impl CliOptions {

pub fn unstable_bare_node_builtins(&self) -> bool {
self.flags.unstable_config.bare_node_builtins
|| self
.maybe_config_file()
.as_ref()
.map(|c| c.has_unstable("bare-node-builtins"))
.unwrap_or(false)
|| self.workspace.has_unstable("bare-node-builtins")
}

pub fn use_byonm(&self) -> bool {
Expand All @@ -1727,28 1694,16 @@ impl CliOptions {
.as_ref()
.map(|s| matches!(s.kind, NpmProcessStateKind::Byonm))
.unwrap_or(false)
|| self
.maybe_config_file()
.as_ref()
.map(|c| c.has_unstable("byonm"))
.unwrap_or(false)
|| self.workspace.has_unstable("byonm")
}

pub fn unstable_sloppy_imports(&self) -> bool {
self.flags.unstable_config.sloppy_imports
|| self
.maybe_config_file()
.as_ref()
.map(|c| c.has_unstable("sloppy-imports"))
.unwrap_or(false)
|| self.workspace.has_unstable("sloppy-imports")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I must have missed it, but how are unstable features enabled in the workspace setting? I hope it's only at the top level?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, it's only at the top level. Otherwise it surfaces diagnostics in deno_config.

}

pub fn unstable_features(&self) -> Vec<String> {
let mut from_config_file = self
.maybe_config_file()
.as_ref()
.map(|c| c.json.unstable.clone())
.unwrap_or_default();
let mut from_config_file = self.workspace.unstable_features().to_vec();

from_config_file.extend_from_slice(&self.flags.unstable_config.features);

Expand Down Expand Up @@ -1787,12 1742,18 @@ impl CliOptions {
{
full_paths.push(import_map_path);
}
if let Some(specifier) = self.maybe_config_file_specifier() {
if specifier.scheme() == "file" {
if let Ok(path) = specifier.to_file_path() {
full_paths.push(path);

for (_, folder) in self.workspace.config_folders() {
if let Some(deno_json) = &folder.deno_json {
if deno_json.specifier.scheme() == "file" {
if let Ok(path) = deno_json.specifier.to_file_path() {
full_paths.push(path);
}
}
}
if let Some(pkg_json) = &folder.pkg_json {
full_paths.push(pkg_json.path.clone());
}
}
full_paths
}
Expand Down Expand Up @@ -1901,8 1862,9 @@ impl StorageKeyResolver {
// otherwise we will use the path to the config file or None to
// fall back to using the main module's path
options
.maybe_config_file
.as_ref()
.workspace
.resolve_start_ctx()
.deno_json()
.map(|config_file| Some(config_file.specifier.to_string()))
})
}
Expand Down Expand Up @@ -1990,7 1952,7 @@ mod test {
let config_file = ConfigFile::new(
config_text,
config_specifier,
&deno_config::ParseOptions::default(),
&deno_config::ConfigParseOptions::default(),
)
.unwrap();
let actual = resolve_import_map_specifier(
Expand All @@ -2014,7 1976,7 @@ mod test {
let config_file = ConfigFile::new(
config_text,
config_specifier,
&deno_config::ParseOptions::default(),
&deno_config::ConfigParseOptions::default(),
)
.unwrap();
let actual = resolve_import_map_specifier(
Expand Down
63 changes: 36 additions & 27 deletions cli/factory.rs
Original file line number Diff line number Diff line change
@@ -1,6 1,5 @@
// Copyright 2018-2024 the Deno authors. All rights reserved. MIT license.

use crate::args::deno_json::deno_json_deps;
use crate::args::CliOptions;
use crate::args::DenoSubcommand;
use crate::args::Flags;
Expand Down Expand Up @@ -57,6 56,7 @@ use std::path::PathBuf;

use deno_config::workspace::Workspace;
use deno_config::workspace::WorkspaceResolver;
use deno_config::ConfigFile;
use deno_core::error::AnyError;
use deno_core::futures::FutureExt;
use deno_core::parking_lot::Mutex;
Expand Down Expand Up @@ -321,44 321,53 @@ impl CliFactory {
.collect()
}

fn deno_json_deps(
maybe_deno_json: Option<&ConfigFile>,
) -> BTreeSet<String> {
maybe_deno_json
.map(|c| {
crate::args::deno_json::deno_json_deps(c)
.into_iter()
.map(|req| req.to_string())
.collect()
})
.unwrap_or_default()
}

self.services.lockfile.get_or_init(|| {
let maybe_lockfile = self.options.maybe_lockfile();

// initialize the lockfile with the workspace's configuration
if let Some(lockfile) = &maybe_lockfile {
let package_json_deps = (!self.options.no_npm())
.then(|| {
self
.package_json_deps_provider()
.reqs()
.map(|reqs| {
reqs.into_iter().map(|s| format!("npm:{}", s)).collect()
})
.unwrap_or_default()
})
.unwrap_or_default();
let mut lockfile = lockfile.lock();
let root_folder = self.options.workspace.root_folder().1;
let (root_url, root_folder) = self.options.workspace.root_folder();
let config = deno_lockfile::WorkspaceConfig {
root: WorkspaceMemberConfig {
package_json_deps: pkg_json_deps(root_folder.pkg_json.as_ref()),
dependencies: deno_json_deps(root_folder.deno_json.as_ref())
.into_iter()
.map(|req| req.to_string())
.collect(),
package_json_deps: pkg_json_deps(root_folder.pkg_json.as_deref()),
dependencies: deno_json_deps(root_folder.deno_json.as_deref()),
},
members: workspace_config
.members
members: self
.options
.workspace
.config_folders()
.iter()
.map(|member| {
.enumerate()
.filter(|(_, (folder_url, _))| *folder_url != root_url)
.map(|(i, (folder_url, folder))| {
(
member.package_name.clone(),
folder
.deno_json
.as_ref()
.and_then(|d| d.json.name.clone())
.or_else(|| {
folder.pkg_json.as_ref().and_then(|d| d.name.clone())
})
.or_else(|| root_url.make_relative(folder_url))
// todo(THIS PR): what would be better here?
.unwrap_or_else(|| format!("member_{}", i)),
WorkspaceMemberConfig {
package_json_deps: Default::default(),
dependencies: deno_json_deps(&member.config_file)
.into_iter()
.map(|req| req.to_string())
.collect(),
package_json_deps: pkg_json_deps(folder.pkg_json.as_deref()),
dependencies: deno_json_deps(folder.deno_json.as_deref()),
},
)
})
Expand Down
2 changes: 1 addition & 1 deletion cli/graph_container.rs
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 98,7 @@ impl MainModuleGraphContainer {
&self,
files: &[String],
) -> Result<Vec<ModuleSpecifier>, AnyError> {
let excludes = self.cli_options.resolve_config_excludes()?;
let excludes = self.cli_options.workspace.resolve_config_excludes()?;
Ok(
files
.iter()
Expand Down