Hello! Great to hear that you have it working; getting SILE working (and packaged) on Windows is an important goal. I want to see this happen as easily as possible, but I don't have much experience with mingw32 myself. I have just downloaded MXE (http://mxe.cc/) and have been playing with it. This contains packages for harfbuzz, fontconfig, etc. and so should help to get these statically built.

I have committed a patch to libtexpdf to make building on mingw32 work better. Do you remember what you needed to do to get justenoughharfbuzz.so and justenoughlibtexpdf.so built?

I am also looking into using luabuild (https://github.com/stevedonovan/luabuild) to package up SILE into a Windows executable. We'll use this bug to track progress, and then hopefully make a Wiki page when we know how to do it properly.

I am now repeating the process of building SILE.

I don't yet understand what is significant for getting SILE working on my MinGW32/Windows8.1 though, I will report on my process of building SILE. I omit the process of installing Lua and LuaRocks as I can't remind it well, however, I just followed instruction found by searching internet with keywords "lua mingw luarocks".

I am not familiarized with autotools thus I got various troubles in creating configure script. I spent much time to find out that I need to pass option "-I/usr/local/share/aclocal" for pkg-config specific macros to aclocal.

For executing configure script, I simply tried

./cofigure --prefix=/usr/local

but it could not find harfbuzz library installed under /usr/local. Then I must do

env CFLAGS=-I/usr/local/include LIBS=-L/usr/local/lib ./configure --prefix=/usr/local

to create Makefile.

Next, building libtexpdf failed due to some missing declaration and I needed to fix it as mentioned in the first post. The problem still remains in creating shared library; libtool complained as

libtool: link: warning: undefined symbols not allowed in i686-pc-mingw32 shared libraries

and failed to creating libtexpdf shared library. (actually there are undefined symbols)

I defined various functions which is indeed undefined:

fseeko and ftello in mfileio.h (simply #define to fseeko64 and ftello64)
fsyscp_fopen in mfileio.h (simply replacing with fopen might be OK)
win32_vfprintf in error.h (just using vfprintf might be OK)
bzero in pdfdoc.c  (use memset instead)
IS_KANJI in dpxfile.c (for a moment simply #define to 0)
concat3 in dpxfile.c (it is just a function that concatenate three strings)
get_wstring_from_fsyscp in dpxfile.c (for a moment replaced with a function returning NULL)

but libtool still complained about undefined symbols and won't build shared library. Then, I added

-no-undefined

option to "libtexpdf_la_LDFLAGS" in libtexpdf/Makefile.am and recreated Makefile.(I am not sure if it is proper solution)

To build justenough*.dll, I needed to add "-no-undefined" flag to "justenough*_la_LDFLAGS" and " -llua" to "justenough*_la_LIBADD" in src/Makefile.am. I also changed ".so" suffix to ".dll" as suited for Windows paltform in src/Makefile.am.

Finally, I must modify "sile" Lua script. Modifications I made is changing ".so" to ".dll" and replacing "/usr" to "/mingw/msys/1.0" in all search path string. Actually, "/mingw/msys/1.0" is mounted to "/usr" but Lua interpreter doesn't understand it.

Now just

make install

There might be something missing but SILE is working at least on my system.

Here are some notes from my experiments so far:

• I have now fixed the undefined functions in libtexpdf.
• I've added Lua libraries to src/Makefile.am.
• I've added -no-undefined to the LDFLAGS in both SILE and libtexpdf.
• I downloaded MXE, added the shared target in settings.mk and then ran make gcc libpng harfbuzz freetype fontconfig lua to set up a mingw32 cross-compilation environment.
• Now in libtexpdf: ./configure --host=i686-w64-mingw32.shared; make successfully compiles libtexpdf-0.dll.
• In SILE, ./configure --host=i686-w64-mingw32.shared; make works right up to the point of copying the .libs/*.so into core/, but I do not have a cross-platform way of determining the shared library extension.
• Adding -shrext .so to LDFLAGS in src/Makefile.am did not seem to help.

Anyway, Hirata-san, if you try updating your SILE and libtexpdf repositories, you should find it a little bit easier to build now.

Thank you for updating.

However, I got another problem; stringapiset.h included from dpxfile.c is not found in my mingw32 system. According to Microsoft's site:

http://msdn.microsoft.com/en-us/library/dd319072.aspx

MultiByteToWideChar and CP_UTF8 seems to be declared in stringapiset.h in Windows8 but it was declared in winnls.h before Windows8. Maybe my mingw32 environment emulates older versions of Windows. Including windows.h instead may work but another problem such as redefinition of ERROR arises. I will consult with the person who implemented that part in kpathsea library.

And ftello and fseeko is still not defined. kpathsea library has definition

#define ftello ftello64
#define fseeko fseeko64

in mingw32.h so it must be OK to have the same definition.

For determining the shared library extension, I tried adding

AC_CANONICAL_HOST
case $host_os in
  cygwin* | mingw* )
      SHARED_LIB_EXT="dll"
      ;;
  *)
      SHARED_LIB_EXT="so"
     ;;
esac

and

AC_SUBST([SHARED_LIB_EXT])

to configure.ac and replacing "so" with "@SHARED_LIB_EXT@" in sile.in and src/Makefile.am. I made some research on the internet. Maybe the above solution is only reasonable one.

Some more experiments that I need to just leave here for now; this time, on Windows itself:

Install msys2
pacman -S devel-base
pacman -S mingw-w64-x86_64-toolchain
pacman -S mingw64/mingw-w64-x86_64-harfbuzz
(Restart with ming64_shell)
git clone https://www.github.com/simoncozens/sile.git
cd sile
git submodule update --init --recursive --remote
cd libtexpdf
git checkout master
./configure
make # No problems!
cd ..
autoreconf
Install luaforwindows, add to path
./configure LUA=lua.exe LUA_INCLUDE=-I/c/Program.../Lua/5.1/include
make

libxtexpdf builds fine, but configuring SILE fails because of incorrectly escaping/passing the spaces in the Luawindows path names when testing for include files.

Simon, are you missing ./bootstrap.sh call in your instructions?

For some reason I'm still running into a problem with libtoolize and autoconf, even though I followed almost exactly the same process (and my msys installation is only one day old). I'll try again tomorrow on a clean instance of msys2. Here are the errors I'm getting:

$ ./bootstrap.sh
libtoolize: putting auxiliary files in AC_CONFIG_AUX_DIR, 'build-aux'.
libtoolize: copying file 'build-aux/ltmain.sh'
libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'.
libtoolize: copying file 'm4/ltversion.m4'
libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am.
configure.ac:57: error: possibly undefined macro: AC_MSG_FAILURE
      If this token and others are legitimate, please use m4_pattern_allow.
      See the Autoconf documentation.
autoreconf: /usr/bin/autoconf failed with exit status: 1
configure.ac:57: error: possibly undefined macro: AC_MSG_FAILURE
      If this token and others are legitimate, please use m4_pattern_allow.
      See the Autoconf documentation.
autoreconf: /usr/bin/autoconf failed with exit status: 1
sed: can't read configure: No such file or directory

Running the same command the second time:

$ ./bootstrap.sh
libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am.
libtoolize: putting auxiliary files in '.'.
libtoolize: copying file './ltmain.sh'
libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'.
libtoolize: copying file 'm4/libtool.m4'
libtoolize: copying file 'm4/ltoptions.m4'
libtoolize: copying file 'm4/ltsugar.m4'
libtoolize: copying file 'm4/ltversion.m4'
libtoolize: copying file 'm4/lt~obsolete.m4'
configure.ac:29: error: possibly undefined macro: AC_MSG_FAILURE
      If this token and others are legitimate, please use m4_pattern_allow.
      See the Autoconf documentation.
configure.ac:52: error: possibly undefined macro: AC_DEFINE
autoreconf: /usr/bin/autoconf failed with exit status: 1

Running configure:

$ ./configure
configure: error: cannot find install-sh, install.sh, or shtool in "." "./.." "./../.."

The only difference I can think of is that I'm building with MSYSTEM=mingw32, but I'll try with the default as well tomorrow.

pkulchenko,

Maybe a development file of pkg-config is not installed?

Simon,

Now I got

mfileio.c:112:7: error: conflicting types for 'xfile_size'
 off_t xfile_size (FILE *file, const char *name)

At first sight, there are no conflicting type, but off_t is actually defined differently in different palces. I think

#define off_t off64_t

should be placed in mfileio.h.

I am using Lua installed via mingw-get and LuaRocks installed in /C/Lua/5.2/LuaRocks/. Find no problems.

@shirat74, thank you for taking a look at this; I did have pkg-config installed, but it didn't help.

@simoncozens, I got this resolved with running automake --force-missing --add-missing command; you may want to add it to the bootstrap.sh before ./configure.

I can run configure now, but it fails on harfbuzz token, even though I ran pacman -S mingw64/mingw-w64-x86_64-harfbuzz:

./configure: line 13015: syntax error near unexpected token `HARFBUZZ,'
./configure: line 13015: `PKG_CHECK_MODULES(HARFBUZZ, harfbuzz,'

@simoncozens, here is updated bootstrap.sh file that generates configure for me in one run:

#!/bin/sh
git submodule update --init --recursive --remote
autoreconf --install
libtoolize
aclocal
automake --force-missing --add-missing
autoreconf
(cd libtexpdf; autoreconf -I m4)
sed 's/rm -f core/rm -f/' configure > config.cache
mv config.cache configure
chmod  x configure

I added the four lines after autoreconf --install based on this suggestion. It's for OSX, but it worked for me on Windows as well.

I still have the configure failure on HARFBUZZ though...

OK. I think the reason of failure is that aclocal can' t find pkg.m4 file; PKG_CHECK_MODULES is defined in that. I got exactly same error before installing pkg.m4 (which is part of pkg-config dev package) and adding -I option to point the location of it.

ok, I managed to build it on windows and even got some PDF files generated (although they seem to be a bit different from the expected results, but at least I got the output).

Here is the list of my instructions for those who want to follow:

• Set up your environment; I'm using msys2, so here is the list of commands you need for the empty/base environment (I'm using 32bit files, so you may need to adjust the names for 64bits):

  pacman -S mingw-w64-i686-gcc
  pacman -S autoconf automake libtool pkg-config
  pacman -S mingw-w64-i686-pango mingw-w64-i686-harfbuzz
  pacman -S mingw-w64-i686-lua51

• Adjust the environment; lua5.1.exe was not detected for some reason, so I copied mingw32/bin/lua5.1.exe to mingw/bin/lua.exe. I also copied Lua 5.1 headers to mingw32/include (lua.h, luaconf.h, lauxlib.h, and lualib.h).
• Get lpeg and lxp libraries. I compiled them myself and copied to sile folder.

All the following runs from mingw32_shell.bat after cloning sile to sile folder.

• Run ./bootstrap:

#!/bin/sh
git submodule update --init --recursive --remote
autoreconf --install
libtoolize
aclocal
automake --force-missing --add-missing
autoreconf
(cd libtexpdf; autoreconf -I m4)
sed 's/rm -f core/rm -f/' configure > config.cache
mv config.cache configure
chmod  x configure

• make; make install
• Copy libtexpdf/.libs/libtexpdf-0.dll to libtexpdf-0.dll (this is the main sile folder where lpeg.dll and lxp.dll are copied as well).
• Run ./sile examples/test.sil.

This generates test.pdf, but the fonts look significantly different from the original file. This is running on the stock Windows 8.1.

SILE defaults to the Gentium font, which you can pick up from here: http://scripts.sil.org/cms/scripts/page.php?item_id=Gentium

It turned out I installed Gentium Plus instead of the basic one. Looks good now, except Irene's letter in test.pdf.

Simon, just want to mention one more thing; I was building this with a mix of 32bit and 64bit environment (even though I expected to only use 32bit-one) and didn't run into the compilation issue @shirat74 mentioned. I did run into some other issues and decided to rebuild from scratch with a cleaner environment and that's what I got the same compilation error, which was fixed by adding #define off_t off64_t to mfileio.h:

diff --git a/mfileio.h b/mfileio.h
index 1a795c7..c37d84b 100644
--- a/mfileio.h
    b/mfileio.h
@@ -52,6  52,7 @@ extern long tell_position (FILE *file);

 extern long file_size (FILE *file);

 #define off_t off64_t
 #define xseek_absolute(file, pos, name) fseeko (file, (off_t)(pos), SEEK_SET)
 #define xseek_relative(file, pos, name) fseeko (file, (off_t)(pos), SEEK_CUR)
 #define xseek_end(file, name) fseeko (file, (off_t)0, SEEK_END)

You may want to incorporate that change.

The rest of the process worked find with my changes to bootstrap.h.

I've been stalling on this because that off_t define scares me. Are you sure it does the right thing on 32-bit environments? I'm also not sure if it will have any implications on non-Windows.

@simoncozens, I actually built and ran it in a 32-bit environment and it works as expected for me.

I'm not sure about the implications, but the error is clear and one of the places has to change as the signature is shown as off_t in one place and off64_t in the other. It probably compiles fine in 64bit environment as off_t and off64_t are likely the same there.

Is there a easy and clear way to get a windows binary of sile ?

I just tried to follow the instructions here without success.
I don't have much experience with msys2, but more detail would be much appreciated.

Is there any specific step where it's failing for you? I've done this a while ago, so not sure if the current process is any different from the one I followed last year. I may have some time to try in few days if you still have issues...

• Fresh install of msys2 64 bit
• pacman -S base-devel
• pacman -S mingw-w64-x86_64-toolchain
• pacman -S mingw64/mingw-w64-x86_64-harfbuzz
  • updating font cache... C:/msys2-64/mingw64/bin/fc-cache.exe: error while loading shared libraries: libfontconfig-1.dll: cannot open shared object file: No such file or directory
• restart msys2 shell
• pacman -S git
• git clone https://www.github.com/simoncozens/sile.git
• cd sile
• git submodule update --init --recursive --remote
• cd libtexpdf
• git checkout master
• cd ..
• ./bootstrap.sh (per pkulchenko below)
• ./configure
  • configure: error: no acceptable C compiler found in $PATH
    See `config.log' for more details

@Queueless, I think you are missing running bootstrap.sh as it will generate the configure script.

@pkulchenko k, added that step- hit next road block; hoping to put all the steps in one post so that it will be there for others

Just to be clear, I'm blocked; the configure script doesn't see a compiler

Any one else willing to lend a hand? I'm still stuck, as per my previous comment

the configure script doesn't see a compiler

sorry if my question may sound stupid. Are you using the mingw32_shell.bat (or the mingw64_shell.bat for 64-bit) script that sets up the toolchain environment? Because if you simply run the msys2_shell.bat, you don't get gcc, etc. in the $PATH...

@anthrotype, no I wasn't using the mingw64 shell- thanks for pointing that out, that got me past the missing compiler.

Now I'm stuck with the configure script looking the Lua and not finding it. I'v run both:

• pacman -S mingw64/mingw-w64-x86_64-lua
• pacman -S mingw64/mingw-w64-x86_64-lua51

and restarted mingw64.exe, but the configure script still doesn't see the a lua instance installed.

@Queueless, I ran into the same issue; look at the related step I described earlier in this thread: #82 (comment).

Thanks. I will spend more time investigating how to pack everything on Windows into binary for a better distribution. There are a few things that need to tune inside Lua building script and the vendoring things. I would try to write a single script to automate all this.

I would eventually like the Windows deliverable to be a sile.exe, a small program embedding the SILE code and the Lua interpreter, rather than calling lua sile.lua from a batch file.

@simoncozens, @harrysummer, I think it's a good goal, but it's difficult to achieve and it's definitely not going to be small, given that just ICU libraries are 30Mb. Given the list of libraries I have (listed in #410 (comment) and #410 (comment)), the total size is going to be close to 38Mb (and these are stripped 32bit versions; 64bit versions are going to be a bit larger).

I've made a bash script to build SILE under MSYS2 MinGW, include building lua and lua packages. ICU has not been included yet. Some dependency libraries such as mingw-w64-x86_64-zlib, mingw-w64-x86_64-expat, mingw-w64-x86_64-libtool, mingw-w64-x86_64-icu need to be installed prior to the building: https://gist.github.com/harrysummer/b9b779a5ef0c7e575191c560b1808bcc

Some comments regarding the current script:

1. The building process starts with compiling lua 5.3, luarocks and lua packages, which are downloaded from the official website, unpacked, possibly modified by sed, and compiled, the vendor folder of SILE's repository is untouched.
2. $PREFIX contains the result of the compilation, Currently, $PREFIX folder location cannot be changed because the path is set to be fixed in sile.exe compiled from Lua.
3. Current it still has problems running outside of MinGW because it depends on icu and harfbuzz which are part of MinGW.

I would eventually like the Windows deliverable to be a sile.exe, a small program embedding the SILE code and the Lua interpreter, rather than calling lua sile.lua from a batch file.

@simoncozens, @harrysummer, I think it's a good goal, but it's difficult to achieve and it's definitely not going to be small, given that just ICU libraries are 30Mb. Given the list of libraries I have (listed in #410 (comment) and #410 (comment)), the total size is going to be close to 38Mb (and these are stripped 32bit versions; 64bit versions are going to be a bit larger).

Maybe we could directly copy those dependency's DLL files from /mingw64/bin into the install directory of SILE?

Could someone please provide me with a tarball/zip with a reasonably working, reasonably recent¹ build of SILE for Windows? (I.e. an already compiled .exe.) It can totally be a one-off archive, though ideally with all dependencies (DLLs, fonts, whatever), or at least mention of what mingw libs (or other files, like fonts, etc., maybe git checkout of SILE repo at some commit) are required. This would significantly help me to continue my effort to finish the math support feature in SILE (#468, #220).

¹ doesn't actually need to be very recent I believe; what I need specifically is that it should successfully build PDFs (on my PC...) and allow me to access the justenoughlibtexpdf API from Lua.

I'd be very grateful for help!

@harrysummer, could you help?

Obviously a pre-compiled .exe with dependencies would be more convenient, but if you end up waiting too long for that I wanted to point out that I've had very good results running SILE in the Bash for Windows (Windows Subsystem for Linux) environment that comes with Windows 10. You basically need to find a webpage that shows you how to activate that feature of Windows 10, and then just follow the SILE build instructions for Linux (specifically Ubuntu) systems.

I should be able to provide an archive with the (relatively recent) executable and all its dependencies, but only in 5 days or so, as I'm traveling without access to the machine with the binaries.

@pkulchenko I'd be very grateful! TIA :) if noone jumps in earlier, then I'll go find myself some other leisure activity over Christmas ;)

@mnjames Thanks for the note. By the way, speaking of contraptions, I did part of the work in a NixOS VM hosted in HyperV on Windows 10, and Nix was actually helpful at that stage ;) but now what's left to be done is pure Lua, and I just want to reduce friction, can't push myself to jump through the hoops anymore. In fact, I just realized 2 days ago, that this is the biggest mental obstacle for me now, preventing me to push this forward little by little in stolen bits of free time; but then I remembered about this thread and thought, hey, maybe someone here has something lying around and could help me without much trouble?

Ok, there were some steps I could still do without the binary (prototyping the rendering in @pkulchenko's ZeroBrane Studio love2d — thanks Paul!); as of today I finished those steps and got fully successful prototype renders in love2d, so if anyone waited to jump in with some link to a nicely archived precompiled Windows binary of SILE, now's a really good time to shed your shyness and go for it! ;) TIA

@akavel, great to hear about the progress! I'm working on the binary package, but ran into a couple of issues with recompiling justenoughlibtexpdf (to include colorpush_rgb changes). I'll update when these are resolved.

(@pkulchenko As a side note, if those colorpush_rgb changes are not very old, a binary without them should be probably ok for me if it's compiling ok for you. That said, I probably won't need it for at least 10h since now.)

That's what I used previously, but I wanted to make sure that the binaries work on the latest SILE code. Already resolved the issue. Will upload shortly.

Ok, give this archive a try.

Instructions:

1. Download the archive.
2. Extract the files from the archive to the SILE folder (after ./bootsrap.sh and ./configure are executed). You are not required to execute ./bootsrap.sh and ./configure, but if you don't run it, you'll need to copy sile.in to sile and replace @SILE_PATH@ with ., @SHARED_LIB_EXT@ with dll and @SILE_LIB_PATH@ with . in it.
3. Update paths in simple-pdf-lua.bat to point to the location of fonts.conf (included in the archive) and the lua/luajit executable you are going to use.
4. Execute simple-pdf-lua.bat; you should now get examples/simple.pdf if everything goes well.

Do you mean I still need to call bootstrap.sh and configure? So I need to install MSYS anyway? :/

Also, by "SILE folder" do you mean clone of the git archive? (if yes, what commit?) or just an empty folder?

Do you mean I still need to call bootstrap.sh and configure? So I need to install MSYS anyway?

You can skip the steps, but you need to generate the sile file, which is generated from sile.in based on configured paths. You can skip bootstrap and configure steps if you update the file manually by replacing @SILE_PATH@ with ., @SHARED_LIB_EXT@ with dll and @SILE_LIB_PATH@ with ..

If you are using the latest SILE master branch, the first several lines of the sile file look like this:

package.path = '?.lua;./?.lua;./lua-libraries/?.lua;./lua-libraries/?/init.lua;lua-libraries/?.lua;lua-libraries/?/init.lua;' .. package.path
local pathvar = os.getenv("SILE_PATH")
if pathvar then
	for path in string.gmatch(pathvar, "[^;] ") do
		package.path =  path .. "/?.lua;" .. package.path
	end
end
package.cpath = package.cpath .. ";core/?.dll;./?.dll;"

Also, by "SILE folder" do you mean clone of the git archive? (if yes, what commit?) or just an empty folder?

Yes, the clone of the git archive; I've tested on the latest commit, but any recent (from Sep 2017) should work.

Ok, it worked, thanks! :) For the record, I edited the .bat a bit (had lua in path, so also simplified the last line):

@echo off
set FONTCONFIG_FILE=%~dp0\fonts.conf
call lua sile -e "require('core/libtexpdf-output')" -d fonts examples\simple.sil

and got output like below:

C:\dnload\lua-etc\sile-bin2>simple-pdf-lua.bat
This is SILE 0.9.5-unreleased
<examples\simple.sil>
[fonts] Looking for Gentium Plus;10;400;;normal;;LTR
[fonts] Resolved font family Gentium Plus -> C:/Windows/Fonts/mingliub.ttc

! Font 'Gentium Plus' not available, falling back to 'MingLiU-ExtB'
[1]

Probably have to install the Gentium Plus font from somewhere, but still got some reasonably looking PDF in examples/simple.pdf. Probably will call it a day here, and come back tomorrow if I'll have time (hopefully!).

Big thanks @pkulchenko !

For the record, I edited the .bat a bit (had lua in path, so also simplified the last line):

Looks good; I updated the file in the archive and re-uploaded (in case someone else wants to use it)

Probably have to install the Gentium Plus font from somewhere, but still got some reasonably looking PDF in examples/simple.pdf

Great! Yes, you'll need Gentium Plus; check the readme for the download link.

Ok, FTR I confirm I installed the Gentium Plus EXE via the link; didn't change the default directory from "Program Files", but it anyway properly copied the TTFs into c:\windows\fonts, so SILE found them, and rendered the example pdf with them. Cool, thanks again!

(For the record, I managed to implement the final step of the prototype pipeline for asciimath rendering in SILE — thanks again to @pkulchenko for help with the binary! A screenshot of the first results is available in #468. A lot of integration work and polishing remains, but I just wanted to post a quick update and thank you here, and to notify that I'll stop hijacking this thread now and will continue posting updates in #220 in case anybody's interested.)

@akavel, looks great! Thank you for the update; I'm definitely interested!

Update on this issue. I managed to build windows binary and setup a CI for it. All dependencies are put together with SILE. Here is the CI link wher eyou can download the latest artifact. https://xiarui.visualstudio.com/personal/_build?definitionId=1

This is all kinds of amazing. Is it ready for a pull request?

@simoncozens I think I will be ready for that in several days after I clean some dirty changes in libtexpdf. Will create a PR once it's done.

Anybody with an interest in compiling SILE under MinGW ... please see my comments on the Windows binary distribution issue, it seems to me this is obsolete but if I'm mistaken somebody should probably tell me so before I rip it out.

@alerque, for the record, I do use SILE on windows with MinGW binaries. In fact, I recently recompiled them and tested with the current SILE master branch without issues.