Memcached Proxy build in ARM64 #1048
Comments
|
Thanks for your report! I was able to reproduce this on an ARM32 at least (ARM64 I can try a little later). This appears to be a packaging problem with the tarball. In the meantime building from the git source does work; I've been doing tests on AWS graviton and such. You can build from there with a few more steps: Going to leave this issue open until I figure out the tarball issue. It's not obvious at a glance. |
|
Note for self: Running |
|
yup, the tarball process is pulling in compiled objects under vendor/ .. sigh. |
|
@dormando thanks for your time and fixing it! One small problem though, the following command is still providing version 1.6.19 wget http://memcached.org/latestMaybe some CD needs to be run to update it? |
|
I think they have been failing for a while: https://build.memcached.org/#/builders/9 |
|
No the link from the homepage works fine: https://memcached.org/ - I moved the release script to a new machine and it was failing to restart ngninx so the |
|
The builder isn't related to the tarball release; it's an "aspirational" builder for pre-testing the tarball so I stop breaking it like this :) Haven't had time to go make the test work after setting it up. |
|
Thanks, I was able to download the tar now, but while building a couple of files were missing: What I did was # download extract http://memcached.org/latest
./configure --prefix=/usr --enable-proxy
make install |
|
can you gist or attach the full build log? I just tried the .21 tarball on amd64 and arm32 and both build fine. I pre-tested them before release as well. Try separating the commands? in case some middle step is failing? |
|
on raspberry pi I got the same error by running make install directly. Please run edit: reopening this issue since I'm pretty sure |
|
I'm trying, but I get stuck with the sh-4.2# ./configure --prefix=/usr --enable-proxy
sh-4.2# make
make all-recursive
make[1]: Entering directory `/memcached-1.6.21'
Making all in doc
make[2]: Entering directory `/memcached-1.6.21/doc'
make all-am
make[3]: Entering directory `/memcached-1.6.21/doc'
make[3]: Nothing to be done for `all-am'.
make[3]: Leaving directory `/memcached-1.6.21/doc'
make[2]: Leaving directory `/memcached-1.6.21/doc'
Making all in vendor
make[2]: Entering directory `/memcached-1.6.21/vendor'
cd lua && make all CFLAGS="-g" && cd ..
make[3]: Entering directory `/memcached-1.6.21/vendor/lua'
make[4]: Entering directory `/memcached-1.6.21/vendor/lua/src'
Guessing Linux
make[5]: Entering directory `/memcached-1.6.21/vendor/lua/src'
make all SYSCFLAGS="-DLUA_USE_LINUX" SYSLIBS="-Wl,-E -ldl"
make[6]: Entering directory `/memcached-1.6.21/vendor/lua/src'
gcc -std=gnu99 -g -c -o lapi.o lapi.c
gcc -std=gnu99 -g -c lcode.c
gcc -std=gnu99 -g -c -o lctype.o lctype.c
gcc -std=gnu99 -g -c -o ldebug.o ldebug.c
gcc -std=gnu99 -g -c -o ldo.o ldo.c
gcc -std=gnu99 -g -c -o ldump.o ldump.c
gcc -std=gnu99 -g -c -o lfunc.o lfunc.c
gcc -std=gnu99 -g -c -o lgc.o lgc.c
gcc -std=gnu99 -g -c llex.c
gcc -std=gnu99 -g -c -o lmem.o lmem.c
gcc -std=gnu99 -g -c -o lobject.o lobject.c
gcc -std=gnu99 -g -c -o lopcodes.o lopcodes.c
gcc -std=gnu99 -g -c lparser.c
gcc -std=gnu99 -g -c -o lstate.o lstate.c
gcc -std=gnu99 -g -c -o lstring.o lstring.c
gcc -std=gnu99 -g -c -o ltable.o ltable.c
gcc -std=gnu99 -g -c -o ltm.o ltm.c
gcc -std=gnu99 -g -c -o lundump.o lundump.c
gcc -std=gnu99 -g -c -o lvm.o lvm.c
gcc -std=gnu99 -g -c -o lzio.o lzio.c
gcc -std=gnu99 -g -c -o lauxlib.o lauxlib.c
gcc -std=gnu99 -g -c -o lbaselib.o lbaselib.c
gcc -std=gnu99 -g -c -o lcorolib.o lcorolib.c
gcc -std=gnu99 -g -c -o ldblib.o ldblib.c
gcc -std=gnu99 -g -c -o liolib.o liolib.c
gcc -std=gnu99 -g -c -o lmathlib.o lmathlib.c
gcc -std=gnu99 -g -c -o loadlib.o loadlib.c
gcc -std=gnu99 -g -c -o loslib.o loslib.c
gcc -std=gnu99 -g -c -o lstrlib.o lstrlib.c
gcc -std=gnu99 -g -c -o ltablib.o ltablib.c
gcc -std=gnu99 -g -c -o lutf8lib.o lutf8lib.c
gcc -std=gnu99 -g -c -o linit.o linit.c
ar rcu liblua.a lapi.o lcode.o lctype.o ldebug.o ldo.o ldump.o lfunc.o lgc.o llex.o lmem.o lobject.o lopcodes.o lparser.o lstate.o lstring.o ltable.o ltm.o lundump.o lvm.o lzio.o lauxlib.o lbaselib.o lcorolib.o ldblib.o liolib.o lmathlib.o loadlib.o loslib.o lstrlib.o ltablib.o lutf8lib.o linit.o
ranlib liblua.a
gcc -std=gnu99 -g -c -o lua.o lua.c
gcc -std=gnu99 -o lua lua.o liblua.a -lm -Wl,-E -ldl
liblua.a(loslib.o): In function `os_tmpname':
/memcached-1.6.21/vendor/lua/src/loslib.c:172: warning: the use of `tmpnam' is dangerous, better use `mkstemp'
gcc -std=gnu99 -g -c -o luac.o luac.c
gcc -std=gnu99 -o luac luac.o liblua.a -lm -Wl,-E -ldl
make[6]: Leaving directory `/memcached-1.6.21/vendor/lua/src'
make[5]: Leaving directory `/memcached-1.6.21/vendor/lua/src'
make[4]: Leaving directory `/memcached-1.6.21/vendor/lua/src'
make[3]: Leaving directory `/memcached-1.6.21/vendor/lua'
cd mcmc && make all && cd ..
make[3]: Entering directory `/memcached-1.6.21/vendor/mcmc'
gcc -g -O2 -Wall -Werror -pedantic -o example example.c mcmc.c
In file included from example.c:12:0:
mcmc.h:11:27: error: C style comments are not allowed in ISO C90 [-Werror]
#define MCMC_CONNECTING 3 // nonblock mode.
^
mcmc.h:11:27: error: (this will be reported only once per input file) [-Werror]
mcmc.h:66:10: error: ISO C90 doesn't support unnamed structs/unions [-Werror=pedantic]
};
^
mcmc.h:74:10: error: ISO C90 doesn't support unnamed structs/unions [-Werror=pedantic]
};
^
mcmc.h:81:10: error: ISO C90 doesn't support unnamed structs/unions [-Werror=pedantic]
};
^
mcmc.h:61:5: error: union has no named members [-Werror=pedantic]
union {
^
mcmc.h:82:6: error: ISO C90 doesn't support unnamed structs/unions [-Werror=pedantic]
};
^
example.c:14:1: error: C style comments are not allowed in ISO C90 [-Werror]
// in this form the socket syscalls are handled externally to the client, so
^
example.c:14:1: error: (this will be reported only once per input file) [-Werror]
example.c: In function 'show_response_buffer':
example.c:43:9: error: ISO C90 forbids mixed declarations and code [-Werror=pedantic]
size_t toread = resp.vlen - resp.vlen_read;
^
example.c: In function 'buffer_mode':
example.c:158:5: error: 'for' loop initial declarations are only allowed in C99 mode
for (int x = 0; strlen(requests[x]) != 0; x ) {
^
example.c:158:5: note: use option -std=c99 or -std=gnu99 to compile your code
example.c: In function 'main':
example.c:174:32: error: '__STDC_VERSION__' undeclared (first use in this function)
printf("C version: %ld\n", __STDC_VERSION__);
^
example.c:174:32: note: each undeclared identifier is reported only once for each function it appears in
cc1: all warnings being treated as errors
mcmc.c:11:1: error: C style comments are not allowed in ISO C90 [-Werror]
// TODO: move these structs into mcmc.h, but only expose them if
^
mcmc.c:11:1: error: (this will be reported only once per input file) [-Werror]
In file included from mcmc.c:13:0:
mcmc.h:11:27: error: C style comments are not allowed in ISO C90 [-Werror]
#define MCMC_CONNECTING 3 // nonblock mode.
^
mcmc.h:11:27: error: (this will be reported only once per input file) [-Werror]
mcmc.h:66:10: error: ISO C90 doesn't support unnamed structs/unions [-Werror=pedantic]
};
^
mcmc.h:74:10: error: ISO C90 doesn't support unnamed structs/unions [-Werror=pedantic]
};
^
mcmc.h:81:10: error: ISO C90 doesn't support unnamed structs/unions [-Werror=pedantic]
};
^
mcmc.h:61:5: error: union has no named members [-Werror=pedantic]
union {
^
mcmc.h:82:6: error: ISO C90 doesn't support unnamed structs/unions [-Werror=pedantic]
};
^
mcmc.c: In function '_mcmc_parse_value_line':
mcmc.c:71:5: error: ISO C90 forbids mixed declarations and code [-Werror=pedantic]
char *n = NULL;
^
mcmc.c:73:5: error: ISO C90 forbids mixed declarations and code [-Werror=pedantic]
uint32_t flags = strtoul(p, &n, 10);
^
mcmc.c:80:5: error: ISO C90 forbids mixed declarations and code [-Werror=pedantic]
uint32_t bytes = strtoul(p, &n, 10);
^
mcmc.c:87:5: error: ISO C90 forbids mixed declarations and code [-Werror=pedantic]
uint64_t cas = 0;
^
mcmc.c:99:5: error: ISO C90 forbids mixed declarations and code [-Werror=pedantic]
int buffer_remain = ctx->buffer_used - (r->value - ctx->buffer_head);
^
mcmc.c: In function '_mcmc_parse_stat_line':
mcmc.c:130:5: error: ISO C90 forbids mixed declarations and code [-Werror=pedantic]
int snamelen = p - sname;
^
mcmc.c:134:5: error: ISO C90 forbids mixed declarations and code [-Werror=pedantic]
char *stat = p;
^
mcmc.c: In function '_mcmc_parse_response':
mcmc.c:183:5: error: ISO C90 forbids mixed declarations and code [-Werror=pedantic]
int code = MCMC_ERR;
^
mcmc.c:240:25: error: ISO C90 forbids mixed declarations and code [-Werror=pedantic]
char *n = NULL;
^
mcmc.c:248:29: error: ISO C90 forbids mixed declarations and code [-Werror=pedantic]
int buffer_remain = ctx->buffer_used - (r->value - ctx->buffer_head);
^
mcmc.c: In function 'mcmc_request_writev':
mcmc.c:566:5: error: 'for' loop initial declarations are only allowed in C99 mode
for (int i = 0; i < iovcnt; i ) {
^
mcmc.c:566:5: note: use option -std=c99 or -std=gnu99 to compile your code
cc1: all warnings being treated as errors
make[3]: *** [all] Error 1
make[3]: Leaving directory `/memcached-1.6.21/vendor/mcmc'
make[2]: *** [all] Error 2
make[2]: Leaving directory `/memcached-1.6.21/vendor'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/memcached-1.6.21'
make: *** [all] Error 2Checking the sh-4.2# ./configure CFLAGS="-std=c99" --prefix=/usr --enable-proxy
sh-4.2# make
# but exact same error, same stackAnd if I run sh-4.2# ./configure CFLAGS="-std=c99" --prefix=/usr --enable-proxy
sh-4.2# make install
make install-recursive
make[1]: Entering directory `/memcached-1.6.21'
Making install in doc
make[2]: Entering directory `/memcached-1.6.21/doc'
make install-am
make[3]: Entering directory `/memcached-1.6.21/doc'
make[4]: Entering directory `/memcached-1.6.21/doc'
make[4]: Nothing to be done for `install-exec-am'.
/usr/bin/mkdir -p '/usr/share/man/man1'
/usr/bin/install -c -m 644 memcached.1 '/usr/share/man/man1'
make[4]: Leaving directory `/memcached-1.6.21/doc'
make[3]: Leaving directory `/memcached-1.6.21/doc'
make[2]: Leaving directory `/memcached-1.6.21/doc'
Making install in vendor
make[2]: Entering directory `/memcached-1.6.21/vendor'
make[2]: `install' is up to date.
make[2]: Leaving directory `/memcached-1.6.21/vendor'
make[2]: Entering directory `/memcached-1.6.21'
gcc -DHAVE_CONFIG_H -I. -DNDEBUG -Ivendor/lua/src -Ivendor/liburing/src/include -std=c99 -pthread -pthread -Wall -pedantic -Wmissing-prototypes -Wmissing-declarations -Wredundant-decls -MT memcached-memcached.o -MD -MP -MF .deps/memcached-memcached.Tpo -c -o memcached-memcached.o `test -f 'memcached.c' || echo './'`memcached.c
mv -f .deps/memcached-memcached.Tpo .deps/memcached-memcached.Po
gcc -DHAVE_CONFIG_H -I. -DNDEBUG -Ivendor/lua/src -Ivendor/liburing/src/include -std=c99 -pthread -pthread -Wall -pedantic -Wmissing-prototypes -Wmissing-declarations -Wredundant-decls -MT memcached-hash.o -MD -MP -MF .deps/memcached-hash.Tpo -c -o memcached-hash.o `test -f 'hash.c' || echo './'`hash.c
mv -f .deps/memcached-hash.Tpo .deps/memcached-hash.Po
gcc -DHAVE_CONFIG_H -I. -DNDEBUG -Ivendor/lua/src -Ivendor/liburing/src/include -std=c99 -pthread -pthread -Wall -pedantic -Wmissing-prototypes -Wmissing-declarations -Wredundant-decls -MT memcached-jenkins_hash.o -MD -MP -MF .deps/memcached-jenkins_hash.Tpo -c -o memcached-jenkins_hash.o `test -f 'jenkins_hash.c' || echo './'`jenkins_hash.c
mv -f .deps/memcached-jenkins_hash.Tpo .deps/memcached-jenkins_hash.Po
gcc -DHAVE_CONFIG_H -I. -DNDEBUG -Ivendor/lua/src -Ivendor/liburing/src/include -std=c99 -pthread -pthread -Wall -pedantic -Wmissing-prototypes -Wmissing-declarations -Wredundant-decls -MT memcached-murmur3_hash.o -MD -MP -MF .deps/memcached-murmur3_hash.Tpo -c -o memcached-murmur3_hash.o `test -f 'murmur3_hash.c' || echo './'`murmur3_hash.c
mv -f .deps/memcached-murmur3_hash.Tpo .deps/memcached-murmur3_hash.Po
gcc -DHAVE_CONFIG_H -I. -DNDEBUG -Ivendor/lua/src -Ivendor/liburing/src/include -std=c99 -pthread -pthread -Wall -pedantic -Wmissing-prototypes -Wmissing-declarations -Wredundant-decls -MT memcached-slabs.o -MD -MP -MF .deps/memcached-slabs.Tpo -c -o memcached-slabs.o `test -f 'slabs.c' || echo './'`slabs.c
mv -f .deps/memcached-slabs.Tpo .deps/memcached-slabs.Po
gcc -DHAVE_CONFIG_H -I. -DNDEBUG -Ivendor/lua/src -Ivendor/liburing/src/include -std=c99 -pthread -pthread -Wall -pedantic -Wmissing-prototypes -Wmissing-declarations -Wredundant-decls -MT memcached-items.o -MD -MP -MF .deps/memcached-items.Tpo -c -o memcached-items.o `test -f 'items.c' || echo './'`items.c
mv -f .deps/memcached-items.Tpo .deps/memcached-items.Po
gcc -DHAVE_CONFIG_H -I. -DNDEBUG -Ivendor/lua/src -Ivendor/liburing/src/include -std=c99 -pthread -pthread -Wall -pedantic -Wmissing-prototypes -Wmissing-declarations -Wredundant-decls -MT memcached-assoc.o -MD -MP -MF .deps/memcached-assoc.Tpo -c -o memcached-assoc.o `test -f 'assoc.c' || echo './'`assoc.c
mv -f .deps/memcached-assoc.Tpo .deps/memcached-assoc.Po
gcc -DHAVE_CONFIG_H -I. -DNDEBUG -Ivendor/lua/src -Ivendor/liburing/src/include -std=c99 -pthread -pthread -Wall -pedantic -Wmissing-prototypes -Wmissing-declarations -Wredundant-decls -MT memcached-thread.o -MD -MP -MF .deps/memcached-thread.Tpo -c -o memcached-thread.o `test -f 'thread.c' || echo './'`thread.c
mv -f .deps/memcached-thread.Tpo .deps/memcached-thread.Po
gcc -DHAVE_CONFIG_H -I. -DNDEBUG -Ivendor/lua/src -Ivendor/liburing/src/include -std=c99 -pthread -pthread -Wall -pedantic -Wmissing-prototypes -Wmissing-declarations -Wredundant-decls -MT memcached-daemon.o -MD -MP -MF .deps/memcached-daemon.Tpo -c -o memcached-daemon.o `test -f 'daemon.c' || echo './'`daemon.c
mv -f .deps/memcached-daemon.Tpo .deps/memcached-daemon.Po
gcc -DHAVE_CONFIG_H -I. -DNDEBUG -Ivendor/lua/src -Ivendor/liburing/src/include -std=c99 -pthread -pthread -Wall -pedantic -Wmissing-prototypes -Wmissing-declarations -Wredundant-decls -MT memcached-stats_prefix.o -MD -MP -MF .deps/memcached-stats_prefix.Tpo -c -o memcached-stats_prefix.o `test -f 'stats_prefix.c' || echo './'`stats_prefix.c
mv -f .deps/memcached-stats_prefix.Tpo .deps/memcached-stats_prefix.Po
gcc -DHAVE_CONFIG_H -I. -DNDEBUG -Ivendor/lua/src -Ivendor/liburing/src/include -std=c99 -pthread -pthread -Wall -pedantic -Wmissing-prototypes -Wmissing-declarations -Wredundant-decls -MT memcached-util.o -MD -MP -MF .deps/memcached-util.Tpo -c -o memcached-util.o `test -f 'util.c' || echo './'`util.c
mv -f .deps/memcached-util.Tpo .deps/memcached-util.Po
gcc -DHAVE_CONFIG_H -I. -DNDEBUG -Ivendor/lua/src -Ivendor/liburing/src/include -std=c99 -pthread -pthread -Wall -pedantic -Wmissing-prototypes -Wmissing-declarations -Wredundant-decls -MT memcached-cache.o -MD -MP -MF .deps/memcached-cache.Tpo -c -o memcached-cache.o `test -f 'cache.c' || echo './'`cache.c
cache.c: In function 'cache_create':
cache.c:20:5: warning: implicit declaration of function 'strdup' [-Wimplicit-function-declaration]
char* nm = strdup(name);
^
cache.c:20:16: warning: initialization makes pointer from integer without a cast [enabled by default]
char* nm = strdup(name);
^
mv -f .deps/memcached-cache.Tpo .deps/memcached-cache.Po
gcc -DHAVE_CONFIG_H -I. -DNDEBUG -Ivendor/lua/src -Ivendor/liburing/src/include -std=c99 -pthread -pthread -Wall -pedantic -Wmissing-prototypes -Wmissing-declarations -Wredundant-decls -MT memcached-bipbuffer.o -MD -MP -MF .deps/memcached-bipbuffer.Tpo -c -o memcached-bipbuffer.o `test -f 'bipbuffer.c' || echo './'`bipbuffer.c
mv -f .deps/memcached-bipbuffer.Tpo .deps/memcached-bipbuffer.Po
gcc -DHAVE_CONFIG_H -I. -DNDEBUG -Ivendor/lua/src -Ivendor/liburing/src/include -std=c99 -pthread -pthread -Wall -pedantic -Wmissing-prototypes -Wmissing-declarations -Wredundant-decls -MT memcached-base64.o -MD -MP -MF .deps/memcached-base64.Tpo -c -o memcached-base64.o `test -f 'base64.c' || echo './'`base64.c
mv -f .deps/memcached-base64.Tpo .deps/memcached-base64.Po
gcc -DHAVE_CONFIG_H -I. -DNDEBUG -Ivendor/lua/src -Ivendor/liburing/src/include -std=c99 -pthread -pthread -Wall -pedantic -Wmissing-prototypes -Wmissing-declarations -Wredundant-decls -MT memcached-logger.o -MD -MP -MF .deps/memcached-logger.Tpo -c -o memcached-logger.o `test -f 'logger.c' || echo './'`logger.c
logger.c: In function 'logger_thread':
logger.c:863:5: error: unknown type name 'useconds_t'
useconds_t to_sleep = MIN_LOGGER_SLEEP;
^
logger.c:875:13: warning: implicit declaration of function 'usleep' [-Wimplicit-function-declaration]
usleep(to_sleep);
^
make[2]: *** [memcached-logger.o] Error 1
make[2]: Leaving directory `/memcached-1.6.21'
make[1]: *** [install-recursive] Error 1
make[1]: Leaving directory `/memcached-1.6.21'
make: *** [install] Error 2I also tried with gcc version: sh-4.2# gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-redhat-linux/4.8.5/lto-wrapper
Target: x86_64-redhat-linux
Configured with: ../configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-bootstrap --enable-shared --enable-threads=posix --enable-checking=release --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-linker-hash-style=gnu --enable-languages=c,c ,objc,obj-c ,java,fortran,ada,go,lto --enable-plugin --enable-initfini-array --disable-libgcj --with-isl=/builddir/build/BUILD/gcc-4.8.5-20150702/obj-x86_64-redhat-linux/isl-install --with-cloog=/builddir/build/BUILD/gcc-4.8.5-20150702/obj-x86_64-redhat-linux/cloog-install --enable-gnu-indirect-function --with-tune=generic --with-arch_32=x86-64 --build=x86_64-redhat-linux
Thread model: posix
gcc version 4.8.5 20150623 (Red Hat 4.8.5-44.0.3) (GCC) |
|
I'm trying to make it easy to reproduce, I hope something like this helps? (it got stuck on configuring libevent for me though) FROM oraclelinux:7-slim
ARG MEMCACHED_VERSION="1.6.21"
RUN yum install -y make gcc wget gzip
RUN wget https://github.com/libevent/libevent/releases/download/release-2.1.12-stable/libevent-2.1.12-stable.tar.gz && tar -xzf libevent-2.1.12-stable.tar.gz
RUN cd libevent-2.1.12-stable && ./configure && make && make install
RUN wget http://memcached.org/latest \
-O memcached-${MEMCACHED_VERSION}.tar.gz \
&& tar xzf memcached-${MEMCACHED_VERSION}.tar.gz
WORKDIR memcached-${MEMCACHED_VERSION}
RUN ./configure --prefix=/usr --enable-proxy
RUN make
RUN make install |
|
Sorry, I wasn't expecting an 8 year old version of gcc for the proxy. I don't have any buildbots that old either. Can you try with CFLAGS="-std=c11"? - the mcmc client uses c11 style code. Please make sure you run If it does I need to circle back with extra configure tests when proxy is enabled. |
|
Come on, using an up-to-date compiler is easy, right? 😆 |
|
I'll keep it open because I need to fix bare May I ask why you were using such an old version of linux? (the oldest one even offered by oracle, apparently). I was worried you were stuck there but you seem to be able to use something newer. |
|
Can't say specifically why but, company policy. Because I'm starting to explore the Memcached Proxy as a mcrouter replacement, I have time to be allowed to use OL8, which has a newer gcc. |
|
Thanks, that's good to know. Are you on an old version of mcrouter? The recent stuff only builds with very new GCC so far as I know. The memcached proxy is in a stage where I'm grinding out polish. The documentation and example configurations (https://github.com/memcached/memcached-proxylibs) are in need of work. If you're having trouble figuring out how to translate a mcrouter configuration into mc proxy please let me know. I'll happily write a configuration for you, or ideally update the standard libraries to make what you need to do easy. I don't really intend long term that most people write "route libraries", but it's abstracted this way so bigger users can fully customize things. |
|
That'd be awesome! We were probably on an old version, we had to drop it because of ARM building issues. Our topology would be something like: Our config was 1 pool, all sync. This was our helm chart snippet, defining the mcrouter config: {
"pools": {
"pool-1": {
"servers": [
{{- $global := . }}
{{- range $i, $e := until (.Values.memcached.replicas | int) }}
"memcached-{{ $i }}.memcached.svc.cluster.local:{{ $global.Values.memcached.port }}",
{{- end }}
]
}
},
"named_handles": [],
"route": {
"type": "PrefixSelectorRoute",
"policies": {
"pr_": {
"type": "OperationSelectorRoute",
"default_policy": "AllSyncRoute|Pool|Process",
"operation_policies": {
"get": {
"type": "FailoverRoute",
"children": [{
"type": "PoolRoute",
"pool": "pool-1"
}]
}
}
}
},
"wildcard": "NullRoute"
}
}Of course, after this we'd like to improve it, but this is the most basic setup. I don't know how this can work with the Lua scripts. Passing env variables with Helm is an option I guess. |
|
Thanks! I'll get back to you in a day with maybe a full example. My plan is that people don't write lua code directly, you use one of the provided libraries (probaly simple.lua). It looks from your configuration like you have a single memcached node that is replicated 3 times (for I'm guessing redundancy?). You would use a configuration like: https://github.com/memcached/memcached-proxylibs/blob/main/examples/basic-replicate.lua I'll follow up tomorrow with a more complete example that uses the prefix routing (you seem to be routing 'pr_*" and nulling the rest). You'd probably use the same type of helm chart to generate the lua file. Just need to fill in the server list at the bottom there. I haven't decided how to expand simple.lua's features yet, as I kind of need to hear from actual users like yourself. In this case it should do mostly what you want, but it doesn't spread gets across all three servers unless the first one fails, so I'll add some options for that. |
|
@dormando, I've been trying with different variations and I can't make it work. What I need:
My config: local s = require("simple")
verbose(true)
router {
router_type = "keyprefix",
default_pool = "default",
match_prefix = "pr_(%a )#",
}
pool {
name = "default",
-- choose 1 backend with builtin hash function
backends = {
"memcached-0.memcached.cache.svc.cluster.local:11211",
"memcached-1.memcached.cache.svc.cluster.local:11211",
"memcached-2.memcached.cache.svc.cluster.local:11211"
},
}
pool {
-- example key = "pr_mySvc#whatever"
name = "mySvc",
-- replicate in all backends
backends = {
"memcached-0.memcached.cache.svc.cluster.local:11211",
"memcached-1.memcached.cache.svc.cluster.local:11211",
"memcached-2.memcached.cache.svc.cluster.local:11211"
},
}addkey pr_dmn#hello 0 60 3
CLIENT_ERROR parsing request
get some
SERVER_ERROR lua failure
set mykey 0 60 3
CLIENT_ERROR parsing request
set pr_dmn#hello 0 60 3
CLIENT_ERROR parsing requestI don't quite get the zones concept. But I tried with this config: router {
router_type = "keyprefix",
default_pool = "default",
match_prefix = "pr_(%a )#",
}
pool {
name = "default",
-- choose 1 backend with builtin hash function
backends = {
"memcached-0.memcached.cache.svc.cluster.local:11211",
"memcached-1.memcached.cache.svc.cluster.local:11211",
"memcached-2.memcached.cache.svc.cluster.local:11211"
},
}
pool {
name = "dmn",
-- replicate in all zones
backends = { -- what should I put in backends? I'm forced by an error
"memcached-0.memcached.cache.svc.cluster.local:11211",
"memcached-1.memcached.cache.svc.cluster.local:11211",
"memcached-2.memcached.cache.svc.cluster.local:11211"
},
zones = {
z1 = {
"memcached-0.memcached.cache.svc.cluster.local:11211",
},
z2 = {
"memcached-1.memcached.cache.svc.cluster.local:11211",
},
z3 = {
"memcached-2.memcached.cache.svc.cluster.local:11211"
},
},
}Same errors with zones. router {
router_type = "flat",
}
pool {
name = "default",
backends = {
"memcached-0.memcached.cache.svc.cluster.local:11211",
"memcached-1.memcached.cache.svc.cluster.local:11211",
"memcached-2.memcached.cache.svc.cluster.local:11211"
},
} |
|
@tomasAlabes sorry! I was out sick. That's unfortunate you're seeing some errors there, and also what you're trying to do won't work with Would you be open to a short chat on discord? (see link at https://memcached.org) - I'd like to either write a short config that does what you need to figure out how to adjust simple.lua. If not we can continue to discuss in this issue I'd just like to take less of your time. It's currently possible to do everything you need via the raw API, but not via any abstraction like I'm still pushing for the polishing stage of this project. My current task is doing an adjustment to the raw API before creating a better route library. This is to avoid having to deal with back compat issues, and is probably a few weeks of work. |
|
Started the conversation in Discord :) |
|
@tomasAlabes I don't see any conversation in the discord. Did you join this one? https://discord.gg/4c4GDKk |
|
The build fix is now released in 1.6.22 |
|
Completely forgot why I was keeping this open... Looks like I was intending to add a C11 support checker to autoconf when proxy is enabled. For passers-by, if you want to try out the proxy you should start with routelib: https://github.com/memcached/memcached-proxylibs/tree/main/lib/routelib - the simple.lua library used here has been removed. |
Describe the bug
ARM64 build instructions fail because of AMD Lua sources
To Reproduce
I don't have a clear snippet to provide, also because I'm not familiar with C compiling and Lua. If I find one I will add it here.
It should be as easy as running
makein ARM64.System Information
Details
The text was updated successfully, but these errors were encountered: