Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Java DaCapo benchmark sample provide fixes #646

Open
wants to merge 22 commits into
base: oe_port
Choose a base branch
from
Open

Add Java DaCapo benchmark sample provide fixes #646

Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
22 commits
Select commit Hold shift click to select a range
28ece2d
Fix dependency in Makefile for Java sample
prp Jul 19, 2020
2c6ee45
Output thread struct address in backtraces
prp Jul 19, 2020
adb6119
Move dumping of mount table to correct location
prp Jul 19, 2020
1d3b408
Add Java DaCapo benchmark sample
prp Jul 19, 2020
6b4ce95
Run first DaCapo benchmark by default
prp Jul 19, 2020
f5f5d04
Debugging DaCapo benchmark h2
prp Jul 19, 2020
7368de3
Fix bug to correctly report segfault address in signal handler
prp Jul 22, 2020
d5d745e
Improve mmap logging
prp Jul 22, 2020
fbaa03c
Pass through SGXLKL_TRACE_SIGNAL correctly
prp Jul 22, 2020
18d3f1c
Update Java DaCapo benchmark sample
prp Jul 22, 2020
dbb7ed9
Add Java DaCapo benchmark test to CI to check correct signal handling
prp Jul 22, 2020
30d52f4
Fix spurious line in Makefile
prp Jul 23, 2020
30a553e
Revert OE changes
prp Jul 27, 2020
c6af984
Document limitations of in-enclave signal handling
prp Jul 27, 2020
7518776
Add enclave_config option "unsafe_host_signals"
prp Jul 27, 2020
a65be49
Update signal handling to return dummy fault address in hw mode
prp Jul 27, 2020
09b5a1f
Update Java DaCapo benchmark sample
prp Jul 27, 2020
1c9e089
Update Java DaCapo test
prp Jul 27, 2020
9dd5458
Point to previous OE branch
prp Jul 27, 2020
fdb1381
Set default for unsafe_host_signals to true
prp Jul 27, 2020
4a2993a
Update enclave config ref file in test
prp Jul 27, 2020
066e154
Address review comments
prp Jul 27, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .gitmodules
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,5 11,5 @@
branch = oe_port
[submodule "openenclave"]
path = openenclave
url = https://github.com/openenclave/openenclave.git
url = https://github.com/lsds/openenclave.git
prp marked this conversation as resolved.
Show resolved Hide resolved
branch = feature/sgx-lkl-support
2 changes: 1 addition & 1 deletion openenclave
Open in desktop
Submodule openenclave updated 5 files
4 −2 enclave/core/sgx/exception.c
3 −1 host/sgx/exception.c
1 −0 host/sgx/exception.h
1 −0 host/sgx/linux/exception.c
3 −1 include/openenclave/internal/sgx/td.h
7 changes: 7 additions & 0 deletions samples/languages/java/dacapo-benchmark/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 1,7 @@
FROM alpine:3.10

RUN apk add --no-cache \
wget fontconfig ttf-dejavu openjdk8-jre

RUN mkdir /dacapo \
&& wget "https://sourceforge.net/projects/dacapobench/files/latest/download" -O /dacapo/dacapo-9.12-MR1-bach.jar
prp marked this conversation as resolved.
Show resolved Hide resolved
46 changes: 46 additions & 0 deletions samples/languages/java/dacapo-benchmark/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 1,46 @@
Running the Java DaCapo benchmark suite with SGX-LKL
====================================================

**Note that this is work-in-progress.**

1. Build a Docker container with the DaCapo benchmark jar:
```
$ docker build -t java-dacapo .
```

2. Convert the container to an SGX-LKL root file system image:
```
$ sgx-lkl-disk create --docker=java-dacapo --size=500M java-dacapo.img
```

3. Edit `java-dacapo-enclave_config.json` to choose which benchmark to run. The full list of benchmarks is:
```
avrora batik eclipse fop h2 jython luindex lusearch lusearch-fix pmd sunflow tomcat tradebeans tradesoap xalan
```

By default, the benchmark `h2` is run. The enclave size is 2 GiB.

4. Run the benchmark with SGX-LKL:
```
$ sgx-lkl-run-oe --hw-debug --host-config=java-dacapo-host_config.json --enclave-config=java-dacapo-enclave_config.json
```

Notes
-----

- Currently the follwing benchmarks are passing/failing:

- PASSING: avrora fop h2 luindex pmd sunflow xalan
- FAILING: batik (doesn't work with OpenJDK) eclipse (doesn't work with OpenJDK) jython (illegal instruction) lusearch-fix (segfault) tomcat (networking issue) tradebeans (networking issue) tradesoap (networking issue)

- When running multiple benchmarks in sequence, ensure that the root file system image has not been corrupted after a failed benchmark run.
prp marked this conversation as resolved.
Show resolved Hide resolved

- You can add `-verbose:gc` to the java parameters to output GC activity. Other verbose JVM Hotspot options are:
```
"-verbose:class",
"-verbose:jni",
"-verbose:gc",
"-XX: PrintCompilation",
"-XX: PrintGCDetails",
"-Xcomp"
```
31 changes: 31 additions & 0 deletions samples/languages/java/dacapo-benchmark/java-dacapo-enclave_config.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 1,31 @@
{

"args": [
"/usr/bin/java",
"-XX:InitialCodeCacheSize=80000k",
"-XX:ReservedCodeCacheSize=80000K",
"-Xms50000k",
"-Xmx700m",
"-XX:CompressedClassSpaceSize=20000K",
"-XX: PerfDisableSharedMem",
"-XX: UseMembar",
"-Dsun.zip.disableMemoryMapping=true",
"-jar",
"/dacapo/dacapo-9.12-MR1-bach.jar",
"h2"
],

"env": [
"LD_LIBRARY_PATH=/usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64/server:/usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64:/usr/lib/jvm/java-1.8-openjdk/jre/../lib/amd64:/usr/lib/jvm/java-1.8-openjdk/lib/amd64/jli:/usr/lib:/lib",
],

"ethreads": "8",

"image_sizes": {
"num_heap_pages": 524288
},

"mmap_files": "shared",

"verbose": true
}
9 changes: 9 additions & 0 deletions samples/languages/java/dacapo-benchmark/java-dacapo-host_config.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 1,9 @@
{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not just host-config.json?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like making them specific to the application, so from looking at the filename, you have a better understanding what's inside.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I find that weird and I don't think we should advocate/recommend that. A folder should be the logical unit that groups things and inside that we have a standard layout. Don't you think?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, I would say that there are two options:

  1. We assume that the enclave/host configs should have well-known fixed filenames, i.e. enclave_config.json and host_config.json. This would be similar to the Dockerfile or Makefile convention. In that case though, the --encave-config and --host-config parameters should be optional and only needed if a user (for whatever reason) wants to override the filenames. This would be similar to how docker or make work -- otherwise we are forcing users to provide redundant options with each invocation.

  2. We don't assume that there is a single approved filename and instead require users to be explicit (as it is now).

As you can tell, I'm in favour of (2), as I'm not a big fan of "magic" filenames that people must remember. That said, if people prefer (1), I'm also ok with that.

"verbose": true,

"root": {
"image_path": "java-dacapo.img"
},

"tap_device": "sgxlkl_tap0"
}
4 changes: 2 additions & 2 deletions samples/languages/java/Makefile → samples/languages/java/helloworld/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,10 34,10 @@ $(DISK_IMAGE): $(PROG)

run: run-hw

run-hw: $(ROOT_FS)
run-hw: $(DISK_IMG)
@echo "sgx-lkl-java --hw-debug ${DISK_IMAGE} HelloWorld"
@${SGXLKL_ENV} ${SGXLKL_JAVA_RUN} --hw-debug ${DISK_IMAGE} HelloWorld

run-sw: $(ROOT_FS)
run-sw: $(DISK_IMG)
@echo "sgx-lkl-java --sw-debug ${DISK_IMAGE} HelloWorld"
@${SGXLKL_ENV} ${SGXLKL_JAVA_RUN} --sw-debug ${DISK_IMAGE} HelloWorld
0 samples/languages/java/README.md → samples/languages/java/helloworld/README.md
View file
Open in desktop
File renamed without changes.
0 samples/languages/java/app/HelloWorld.java → ...uages/java/helloworld/app/HelloWorld.java
View file
Open in desktop
File renamed without changes.
9 changes: 6 additions & 3 deletions src/enclave/enclave_mem.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -415,13 415,15 @@ void* enclave_mmap(
char* rv = ret == MAP_FAILED ? " (FAILED)" : "";
SGXLKL_TRACE_MMAP(
"mmap stats: TOTAL: %8zuKB, USED: %8zuKB, MAX USED: %8zuKB, FREE: "
"%8zuKB, ALLOCATED: %6zuKB (addr = %p, ret = %p) %s%s\n",
"%8zuKB, ALLOCATED: %6zuKB (addr=%p len=%lu prot=%i ret=%p) %s%s\n",
total / 1024,
used / 1024,
mmap_max_allocated / 1024,
free / 1024,
requested / 1024,
addr,
length,
prot,
ret,
mfixed,
rv);
Expand Down Expand Up @@ -468,13 470,14 @@ int enclave_munmap(void* addr, size_t length)
size_t used = total - free;
SGXLKL_TRACE_MMAP(
"munmap stats: TOTAL: %8zuKB, USED: %8zuKB, MAX USED: %8zuKB, "
"FREE: %8zuKB, FREED: %6zuKB (addr = %p)\n",
"FREE: %8zuKB, FREED: %6zuKB (addr=%p len=%lu)\n",
total / 1024,
used / 1024,
mmap_max_allocated / 1024,
free / 1024,
requested / 1024,
addr);
addr,
length);
}
#endif

Expand Down
23 changes: 19 additions & 4 deletions src/enclave/enclave_signal.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -130,9 130,10 @@ static uint64_t sgxlkl_enclave_signal_handler(
{
SGXLKL_TRACE_SIGNAL(
"Exception SIGILL (illegal instruction) received (code=%d "
"address=0x%lx opcode=0x%x)\n",
"address=0x%lx rip=0x%lx opcode=0x%x)\n",
exception_record->code,
exception_record->address,
exception_record->context->rip,
opcode);

_sgxlkl_illegal_instr_hook(opcode, exception_record->context);
Expand All @@ -144,10 145,12 @@ static uint64_t sgxlkl_enclave_signal_handler(
if (ret != -1)
{
SGXLKL_TRACE_SIGNAL(
"Exception %s received (code=%d address=0x%lx opcode=0x%x)\n",
"Exception %s received (code=%d address=0x%lx rip=0x%lx "
"opcode=0x%x)\n",
trap_info.description,
exception_record->code,
exception_record->address,
exception_record->context->rip,
opcode);

#ifdef DEBUG
Expand Down Expand Up @@ -180,13 183,14 @@ static uint64_t sgxlkl_enclave_signal_handler(
"Exception %s received before LKL initialisation/after LKL "
"shutdown (lt->tid=%i [%s] "
"code=%i "
"addr=0x%lx opcode=0x%x "
"addr=0x%lx rip=0x%lx opcode=0x%x "
"ret=%i)\n",
trap_info.description,
lt ? lt->tid : -1,
lt ? lt->funcname : "(?)",
exception_record->code,
(void*)exception_record->address,
exception_record->context->rip,
opcode,
ret);
}
Expand All @@ -196,7 200,18 @@ static uint64_t sgxlkl_enclave_signal_handler(

info.si_errno = 0;
info.si_code = exception_record->code;
info.si_addr = (void*)exception_record->address;

// Return faulting address for segfaults. This code should be
// generalised to handle all types of exceptions.
if (exception_record->code == OE_EXCEPTION_PAGE_FAULT)
{
info.si_addr = (void*)exception_record->address;
}
else
{
info.si_addr = (void*)exception_record->context->rip;
}

info.si_signo = trap_info.signo;

/**
Expand Down
2 changes: 1 addition & 1 deletion src/include/host/sgxlkl_params.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 51,7 @@
#ifndef SGXLKL_RELEASE
/* These environment variables do not have config settings, they are
* automatically passed through and imported in the enclave */
extern const char* sgxlkl_auto_passthrough[11];
extern const char* sgxlkl_auto_passthrough[12];
#endif

#endif /* SGXLKL_PARAMS_H */
8 changes: 4 additions & 4 deletions src/lkl/setup.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1320,10 1320,6 @@ static void* lkl_termination_thread(void* args)
}
}

#ifdef DEBUG
display_mount_table();
#endif

/* Unmount root.
* We are calling umount with the MNT_DETACH flag for the root
* file system, otherwise the call fails to unmount the file
Expand All @@ -1336,6 1332,10 @@ static void* lkl_termination_thread(void* args)
if (res < 0)
sgxlkl_warn("Could not unmount root disk, %s\n", lkl_strerror(res));

#ifdef DEBUG
display_mount_table();
#endif

SGXLKL_VERBOSE("calling lkl_virtio_netdev_remove()\n");
lkl_virtio_netdev_remove();

Expand Down
5 changes: 3 additions & 2 deletions src/main-oe/sgxlkl_params.c
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 1,6 @@
#include "host/sgxlkl_params.h"

const char* sgxlkl_auto_passthrough[11] = {"SGXLKL_DEBUGMOUNT",
const char* sgxlkl_auto_passthrough[12] = {"SGXLKL_DEBUGMOUNT",
"SGXLKL_PRINT_APP_RUNTIME",
"SGXLKL_TRACE_HOST_SYSCALL",
"SGXLKL_TRACE_INTERNAL_SYSCALL",
Expand All @@ -10,4 10,5 @@ const char* sgxlkl_auto_passthrough[11] = {"SGXLKL_DEBUGMOUNT",
"SGXLKL_TRACE_REDIRECT_SYSCALL",
"SGXLKL_TRACE_MMAP",
"SGXLKL_TRACE_SYSCALL",
"SGXLKL_TRACE_THREAD"};
"SGXLKL_TRACE_THREAD",
"SGXLKL_TRACE_SIGNAL"};
2 changes: 1 addition & 1 deletion src/sched/lthread.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1102,7 1102,7 @@ void lthread_dump_all_threads(void)
int tid = lt->tid;
char* funcname = lt->funcname;
sgxlkl_info("-------------------------------------------------------------\n");
sgxlkl_info("%s%i: tid=%i [%s]\n", lt == lthread_self() ? "*" : "", i, tid, funcname);
sgxlkl_info("%s%i: tid=%i (%p) [%s]\n", lt == lthread_self() ? "*" : "", i, tid, lt, funcname);
sgxlkl_print_backtrace(lt == lthread_self() ? __builtin_frame_address(0) : lt->ctx.ebp);

lt_queue = lt_queue->next;
Expand Down
7 changes: 7 additions & 0 deletions tests/languages/java/dacapo-benchmark/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 1,7 @@
FROM alpine:3.10

RUN apk add --no-cache \
wget fontconfig ttf-dejavu openjdk8-jre

RUN mkdir /dacapo \
&& wget "https://sourceforge.net/projects/dacapobench/files/latest/download" -O /dacapo/dacapo-9.12-MR1-bach.jar
prp marked this conversation as resolved.
Show resolved Hide resolved
28 changes: 28 additions & 0 deletions tests/languages/java/dacapo-benchmark/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 1,28 @@
include ../../../common.mk

DISK_IMAGE=java-dacapo.img
IMAGE_SIZE=750M

ifeq ($(SGXLKL_VERBOSE),)
SGXLKL_ENV=\
SGXLKL_TAP=sgxlkl_tap0 SGXLKL_VERBOSE=1 SGXLKL_KERNEL_VERBOSE=1 SGXLKL_TRACE_LKL_SYSCALL=0 SGXLKL_TRACE_SIGNAL=0
else
SGXLKL_ENV=
endif

.DELETE_ON_ERROR:
.PHONY: run run-hw run-sw clean

$(DISK_IMAGE): $(PROG_SRC)
${SGXLKL_DISK_TOOL} create --size=${IMAGE_SIZE} --docker=Dockerfile ${DISK_IMAGE}

run: run-hw run-sw

run-hw: $(DISK_IMAGE)
${SGXLKL_ENV} ${SGXLKL_STARTER} --hw-debug --host-config=java-dacapo-host_config.json --enclave-config=java-dacapo-enclave_config.json

run-sw: $(DISK_IMAGE)
${SGXLKL_ENV} ${SGXLKL_STARTER} --sw-debug --host-config=java-dacapo-host_config.json --enclave-config=java-dacapo-enclave_config.json

clean:
rm -f $(DISK_IMAGE)
31 changes: 31 additions & 0 deletions tests/languages/java/dacapo-benchmark/java-dacapo-enclave_config.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 1,31 @@
{

"args": [
"/usr/bin/java",
"-XX:InitialCodeCacheSize=80000k",
"-XX:ReservedCodeCacheSize=80000K",
"-Xms50000k",
"-Xmx700m",
"-XX:CompressedClassSpaceSize=20000K",
"-XX: PerfDisableSharedMem",
"-XX: UseMembar",
"-Dsun.zip.disableMemoryMapping=true",
"-jar",
"/dacapo/dacapo-9.12-MR1-bach.jar",
"h2"
],

"env": [
"LD_LIBRARY_PATH=/usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64/server:/usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64:/usr/lib/jvm/java-1.8-openjdk/jre/../lib/amd64:/usr/lib/jvm/java-1.8-openjdk/lib/amd64/jli:/usr/lib:/lib",
],

"ethreads": "8",

"image_sizes": {
"num_heap_pages": 524288
},

"mmap_files": "shared",

"verbose": true
}
9 changes: 9 additions & 0 deletions tests/languages/java/dacapo-benchmark/java-dacapo-host_config.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 1,9 @@
{
"verbose": true,

"root": {
"image_path": "java-dacapo.img"
},

"tap_device": "sgxlkl_tap0"
}