Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Spurious recompilation in multiproject builds with external subprojects. #35

Closed
nuttycom opened this issue Jun 5, 2011 · 5 comments
Closed

Spurious recompilation in multiproject builds with external subprojects. #35

nuttycom opened this issue Jun 5, 2011 · 5 comments
Labels
Bug

Comments

@nuttycom
Copy link

nuttycom commented Jun 5, 2011

I'm finding some strange behavior with the new external multiproject support. I have two projects, A and B which both dependOn external project C using RootProject("../c").

If I go to project A and compile it, C is also compiled, and all is well. However, if I then go to project B and compile it, this also causes C to be entirely recompiled. This of course is awful, because C takes a couple of minutes to compile and this happens whenever I switch between A and B.

Here is an example of build output related to the attached projects:

knuttycombe@floorshow sbt (master)]$ ls
a b c
[knuttycombe@floorshow sbt (master)]$ cd c/
[knuttycombe@floorshow c (master)]$ sbt compile
Getting net.java.dev.jna jna 3.2.3 ...
:: retrieving :: org.scala-tools.sbt#boot-app
confs: [default]
1 artifacts copied, 0 already retrieved (838kB/80ms)
Getting Scala 2.8.1 (for sbt)...
:: retrieving :: org.scala-tools.sbt#boot-scala
confs: [default]
4 artifacts copied, 0 already retrieved (15296kB/873ms)
Getting org.scala-tools.sbt sbt_2.8.1 0.10.0 ...
:: retrieving :: org.scala-tools.sbt#boot-app
confs: [default]
34 artifacts copied, 0 already retrieved (6012kB/794ms)
[info] Set current project to default (in build file:/Users/knuttycombe/tmp/sbt/c/)
[info] Updating...
[info] Done updating.
[info] Compiling 1 Scala source to /Users/knuttycombe/tmp/sbt/c/target/scala-2.8.1.final/classes...
[success] Total time: 3 s, completed Jun 4, 2011 9:46:53 PM
[knuttycombe@floorshow c (master)]$ cd ../b
[knuttycombe@floorshow b (master)]$ sbt compile
Getting net.java.dev.jna jna 3.2.3 ...
:: retrieving :: org.scala-tools.sbt#boot-app
confs: [default]
1 artifacts copied, 0 already retrieved (838kB/19ms)
Getting Scala 2.8.1 (for sbt)...
:: retrieving :: org.scala-tools.sbt#boot-scala
confs: [default]
4 artifacts copied, 0 already retrieved (15296kB/248ms)
Getting org.scala-tools.sbt sbt_2.8.1 0.10.0 ...
:: retrieving :: org.scala-tools.sbt#boot-app
confs: [default]
34 artifacts copied, 0 already retrieved (6012kB/63ms)
[info] Compiling 1 Scala source to /Users/knuttycombe/tmp/sbt/b/project/target/scala_2.8.1/classes...
[info] Set current project to b (in build file:/Users/knuttycombe/tmp/sbt/b/)
[info] Updating...
[info] Compiling 1 Scala source to /Users/knuttycombe/tmp/sbt/c/target/scala-2.8.1.final/classes...
[info] Done updating.
[info] Compiling 1 Scala source to /Users/knuttycombe/tmp/sbt/b/target/scala-2.8.1.final/classes...
[success] Total time: 3 s, completed Jun 4, 2011 9:47:16 PM
[knuttycombe@floorshow b (master)]$ cd ../a
[knuttycombe@floorshow a (master)]$ sbt compile
Getting net.java.dev.jna jna 3.2.3 ...
:: retrieving :: org.scala-tools.sbt#boot-app
confs: [default]
1 artifacts copied, 0 already retrieved (838kB/17ms)
Getting Scala 2.8.1 (for sbt)...
:: retrieving :: org.scala-tools.sbt#boot-scala
confs: [default]
4 artifacts copied, 0 already retrieved (15296kB/236ms)
Getting org.scala-tools.sbt sbt_2.8.1 0.10.0 ...
:: retrieving :: org.scala-tools.sbt#boot-app
confs: [default]
34 artifacts copied, 0 already retrieved (6012kB/65ms)
[info] Compiling 1 Scala source to /Users/knuttycombe/tmp/sbt/a/project/target/scala_2.8.1/classes...
[info] Set current project to a (in build file:/Users/knuttycombe/tmp/sbt/a/)
[info] Updating...
[info] Compiling 1 Scala source to /Users/knuttycombe/tmp/sbt/c/target/scala-2.8.1.final/classes...
[info] Done updating.
[info] Compiling 1 Scala source to /Users/knuttycombe/tmp/sbt/a/target/scala-2.8.1.final/classes...
[success] Total time: 3 s, completed Jun 4, 2011 9:47:41 PM
[knuttycombe@floorshow a (master)]$ cd ../b/
[knuttycombe@floorshow b (master)]$ sbt compile
[info] Set current project to b (in build file:/Users/knuttycombe/tmp/sbt/b/)
[info] Compiling 1 Scala source to /Users/knuttycombe/tmp/sbt/c/target/scala-2.8.1.final/classes...
[success] Total time: 6 s, completed Jun 4, 2011 9:47:58 PM
[knuttycombe@floorshow b (master)]$

Something interesting to note: although c is recompiled when I switch back to b, the classes in b that depend on c are not!
@nuttycom
Copy link
Author

nuttycom commented Jun 5, 2011

Haven't been able to figure out how to attach a file for this ticket, so here's the source organization:

./a/project/Build.scala
import sbt._
import Keys._

object ABuild extends Build {
lazy val a = Project("a", file(".")) dependsOn (c)
lazy val c = RootProject(file("../c"))
}

./a/src/main/scala/a/A.scala
package a
import c.C

class A(c: C)

./b/project/Build.scala
import sbt._
import Keys._

object BBuild extends Build {
lazy val b = Project("b", file(".")) dependsOn (c)
lazy val c = RootProject(file("../c"))
}

./b/src/main/scala/b/B.scala
package b
import c.C

case class B(c: C)

./c/build.sbt
name := "c"

./c/src/main/scala/c/C.scala
package c

case class C(value: String)

@nuttycom nuttycom closed this as completed Jun 5, 2011
@nuttycom nuttycom reopened this Jun 5, 2011
@harrah
Copy link
Member

harrah commented Jun 5, 2011

Apparently, you are supposed to paste in a gist and link to it, which is somewhat unfortunate.

The cause of the problem is that by default, sbt uses the project/boot directory in the project you start sbt in and in particular, the Scala files there. So, when you move between projects, sbt sees a different version of scala-library.jar for c (the one in a/project/boot and then in b/project/boot) and thinks the source needs recompiling.

There are two solutions I can think of. First, you can have a global boot directory that is shared by all projects on your machine. You can do this by setting the sbt.boot.directory system property in your launcher script. It can also be passed directly to the sbt script to just try it out at first. For example:

sbt -Dsbt.boot.directory=/home/user/.sbt/boot/

The other option is to have another project that refers to the three projects and then always run sbt on that project. This other project doesn't have to contain the subdirectories.

@nuttycom
Copy link
Author

nuttycom commented Jun 5, 2011

Having a common SBT boot directory sounds like it might be generally good
practice in any case, because it would minimize startup times for new
projects as well. I'll give that a try, thanks!
On Jun 5, 2011 12:47 AM, "harrah" <
[email protected]>
wrote:

Apparently, you are supposed to paste in a gist and link to it, which is
somewhat unfortunate.

The cause of the problem is that by default, sbt uses the project/boot
directory in the project you start sbt in and in particular, the Scala files
there. So, when you move between projects, sbt sees a different version of
scala-library.jar for c (the one in a/project/boot and then in
b/project/boot) and thinks the source needs recompiling.

There are two solutions I can think of. First, you can have a global boot
directory that is shared by all projects on your machine. You can do this by
setting the sbt.boot.directory system property in your launcher script. It
can also be passed directly to the sbt script to just try it out at first.
For example:

sbt -Dsbt.boot.directory=/home/user/.sbt/boot/

The other option is to have another project that refers to the three
projects and then always run sbt on that project. This other project doesn't
have to contain the subdirectories.

Reply to this email directly or view it on GitHub:
harrah#35 (comment)

@nuttycom
Copy link
Author

nuttycom commented Jun 5, 2011

The global sbt boot directory looks like it does the trick. I still have a bunch more questions, but I'll bring them up on the list. Thanks!

@nuttycom nuttycom closed this as completed Jun 5, 2011
@harrah
Copy link
Member

harrah commented Jun 6, 2011

To close this out, I added comments about this on Setup and Full Configuration.

eed3si9n pushed a commit to eed3si9n/sbt that referenced this issue Apr 22, 2017
@jrudolph
Merge pull request sbt#35 from jroper/1.0-support
9491a79 
Added support for cross building against sbt 1.0
dwijnand pushed a commit to dwijnand/sbt that referenced this issue Jun 26, 2017
@eed3si9n @dwijnand
Merge pull request sbt#35 from eed3si9n/wip/warning
202931a 
Remove some warnings
dwijnand pushed a commit to dwijnand/sbt that referenced this issue Jun 30, 2017
@eed3si9n @dwijnand
Merge pull request sbt#35 from eed3si9n/wip/warning
5ab3b41 
Remove some warnings
dwijnand pushed a commit to dwijnand/sbt that referenced this issue Jul 7, 2017
@eed3si9n @dwijnand
Merge pull request sbt#35 from eed3si9n/wip/warning
ceca932 
Remove some warnings
eed3si9n pushed a commit to eed3si9n/sbt that referenced this issue Sep 22, 2020
@jrudolph
add support for sbt 0.13, fixes sbt#35
f7c66f5
eed3si9n pushed a commit to eed3si9n/sbt that referenced this issue Apr 20, 2021
@jsuereth
Merge pull request sbt#35 from sbt/feature/windows-sbt-config
f5a22c9 
Feature/windows sbt config
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@nuttycom @harrah