Settings for log rotation (max_days, max_size) #42

z9r5 · 2018-02-16T09:24:25Z

Old logs should be cleaned from time to time. 3 options will be introduced for that purpose:

max_days N — keep records not older than N days;
max_size M — store not more than M GBs of records;
max_days N max_size M — if both options are in use, both limits (in days and GBs) should be always applied.

z9r5 · 2018-02-27T05:12:41Z

You can use the bash script below as a temporal solution.
It takes an oldest table and delete it if the value of TARGET_PERCENT more than current percent of use /var/lib/clickhouse directory.

#!/bin/bash 

TARGET_PERCENT=95
POD=$(/usr/local/bin/kubectl get po -n loghouse |awk '/clickhouse-server/{ print $1}')
CURRENT_PERCENT=$(/usr/local/bin/kubectl -n loghouse exec -ti $POD -- /bin/df -h |awk '/\/var\/lib\/clickhouse/{print $5}')
CURRENT_PERCENT=${CURRENT_PERCENT//%/}
if [ "$CURRENT_PERCENT" -gt "$TARGET_PERCENT" ]; then
  CL_PASS=$(/usr/local/bin/kubectl -n loghouse get po $POD -o yaml |grep CLICKHOUSE_PASS -A1 | awk '/value/{print $2}')
  echo "CURRENT_PERCENT $CURRENT_PERCENT -gt TARGET_PERCENT $TARGET_PERCENT"
  TBL=$(/usr/local/bin/kubectl -n loghouse exec -ti $POD -- clickhouse-client -h 127.0.0.1 -d logs --password $CL_PASS -q "show tables" |grep -E "logs[0-9]{10}" |sort |head -1)
  echo "drop table $TBL"
  /usr/local/bin/kubectl -n loghouse exec -ti $POD -- clickhouse-client -h 127.0.0.1 -d logs --password $CL_PASS -q "drop table $TBL"
fi

You can also use it in crontab.
Example:

*/15 * * * * root /opt/scripts/loghouse.cleaner.sh >> /var/log/loghouse.cleaner.log 2>&1

broken-ufa · 2018-10-12T15:24:09Z

@zimmermann-russia you are right, but this patch makes you script better:

--- loghouse.cleaner.sh 2018-10-12 18:21:34.046754060  0300
    loghouse.cleaner_new.sh     2018-10-12 18:22:27.719205599  0300
@@ -4,10  4,12 @@
 POD=$(/usr/local/bin/kubectl get po -n loghouse |awk '/clickhouse-server/{ print $1}')
 CURRENT_PERCENT=$(/usr/local/bin/kubectl -n loghouse exec -ti $POD -- /bin/df -h |awk '/\/var\/lib\/clickhouse/{print $5}')
 CURRENT_PERCENT=${CURRENT_PERCENT//%/}
-if [ "$CURRENT_PERCENT" -gt "$TARGET_PERCENT" ]; then
 while [ "$CURRENT_PERCENT" -gt "$TARGET_PERCENT" ]; do
   CL_PASS=$(/usr/local/bin/kubectl -n loghouse get po $POD -o yaml |grep CLICKHOUSE_PASS -A1 | awk '/value/{print $2}')
   echo "CURRENT_PERCENT $CURRENT_PERCENT -gt TARGET_PERCENT $TARGET_PERCENT"
   TBL=$(/usr/local/bin/kubectl -n loghouse exec -ti $POD -- clickhouse-client -h 127.0.0.1 -d logs --password $CL_PASS -q "show tables" |grep -E "logs[0-9]{10}" |sort |head -1)
   echo "drop table $TBL"
   /usr/local/bin/kubectl -n loghouse exec -ti $POD -- clickhouse-client -h 127.0.0.1 -d logs --password $CL_PASS -q "drop table $TBL"
-fi
   CURRENT_PERCENT=$(/usr/local/bin/kubectl -n loghouse exec -ti $POD -- /bin/df -h |awk '/\/var\/lib\/clickhouse/{print $5}')
   CURRENT_PERCENT=${CURRENT_PERCENT//%/}
 done

z9r5 · 2018-10-12T15:54:54Z

@broken-ufa Thanks!

z9r5 · 2022-02-04T04:36:07Z

Closed because of loghouse is no longer being actively developed.

z9r5 pushed a commit that referenced this issue Feb 27, 2018

Add link to #42 issue in readme.md

9018ec0

z9r5 closed this as completed Feb 4, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Settings for log rotation (max_days, max_size) #42

Settings for log rotation (max_days, max_size) #42

z9r5 commented Feb 16, 2018

z9r5 commented Feb 27, 2018

broken-ufa commented Oct 12, 2018 •

edited

Loading

z9r5 commented Oct 12, 2018

z9r5 commented Feb 4, 2022

Settings for log rotation (max_days, max_size) #42

Settings for log rotation (max_days, max_size) #42

Comments

z9r5 commented Feb 16, 2018

z9r5 commented Feb 27, 2018

broken-ufa commented Oct 12, 2018 • edited Loading

z9r5 commented Oct 12, 2018

z9r5 commented Feb 4, 2022

broken-ufa commented Oct 12, 2018 •

edited

Loading