Thursday, November 1, 2018

Bash function to nicely output the elapsed time




The bash function:

### Function: take integer UNIXTIMESTAMP start calc the diff against the current "date +'%s'". Displays elapses timestamp as "HH:MM:SS (NNN sec)"
function printelapsed() {
    local let s=$((`date +'%s'`-$1))
    ((all=s, sec=s%60, s/=60, min=s%60, hrs=s/60))
    printf "%d:%02d:%02d (%d sec)" $hrs $min $sec $all
### end func

The code usage:

let STARTSTEP=$(date +'%s')   ### Start the timer
#### DO SOME STUFF HERE ###
echo "[`date +"%Y-%m-%d %H:%M:%S"`] Elapsed: `printelapsed $STARTSTEP`." >> $LOGFILE

The output:

[2018-11-01 02:51:40] Total elapsed: 0:06:39 (399 sec).

How to keep logs compressed, organized into monthly bundles, and deleted after N days



find "$DIRLOG" -type f -mtime +365 -print | xargs rm -f > /dev/null 2>&1                                                                           ### Delete all log files older than N days.
find "$DIRLOG" -type f -mtime +14 -name "$(date +%Y%m -d '1 month ago')*.zip" -print | xargs zip -jm0 "${DIRLOG}/$(date +"%Y%m" -d '1 month ago')_$(hostname)_${PROGRAM/%.sh/}.zip" /dev/null 2>&1 ### Archives into monthly zip bundle
find "$DIRLOG" -type f -mtime +14 -name "$(date +%Y%m -d '1 month ago')*.log" -print | xargs zip -jm9 "${DIRLOG}/$(date +"%Y%m" -d '1 month ago')_$(hostname)_${PROGRAM/%.sh/}.zip" /dev/null 2>&1 ### Old logs into monthly zip bundle


Result:
[nz@server logs]$ ls -l
total 6292
-rw-r--r-- 1 nz nz 505376 Nov 15  2017 201710_server_backup_n_groom.zip
-rw-r--r-- 1 nz nz 481965 Dec 16  2017 201711_server_backup_n_groom.zip
-rw-r--r-- 1 nz nz 497859 Jan 15  2018 201712_server_backup_n_groom.zip
-rw-r--r-- 1 nz nz 506941 Feb 16  2018 201801_server_backup_n_groom.zip
-rw-r--r-- 1 nz nz 454156 Mar 16  2018 201802_server_backup_n_groom.zip
-rw-r--r-- 1 nz nz 465012 Apr 13  2018 201803_server_backup_n_groom.zip
-rw-r--r-- 1 nz nz 472104 May 16 20:50 201804_server_backup_n_groom.zip
-rw-r--r-- 1 nz nz 483660 Jun 16 20:50 201805_server_backup_n_groom.zip
-rw-r--r-- 1 nz nz 468871 Jul 16 20:50 201806_server_backup_n_groom.zip
-rw-r--r-- 1 nz nz 491261 Aug 16 20:50 201807_server_backup_n_groom.zip
-rw-r--r-- 1 nz nz 485345 Sep 16 20:50 201808_server_backup_n_groom.zip
-rw-r--r-- 1 nz nz 470833 Oct 15 23:50 201809_server_backup_n_groom.zip
-rw-r--r-- 1 nz nz   4491 Oct 16 23:17 20181016225001_server_ODM_DW_PROD_DIFF_backup.zip
-rw-r--r-- 1 nz nz   3407 Oct 17 20:54 20181017205001_server_HISTDB_DIFF_backup.zip
-rw-r--r-- 1 nz nz   2622 Oct 17 22:40 20181017224001_server_GLOBALS_FULL_backup.zip
-rw-r--r-- 1 nz nz   3530 Oct 17 22:45 20181017224501_server_NZDBA_DIFF_backup.zip
-rw-r--r-- 1 nz nz   4506 Oct 17 23:17 20181017225001_server_ODM_DW_PROD_DIFF_backup.zip
-rw-r--r-- 1 nz nz   3402 Oct 18 20:53 20181018205001_server_HISTDB_DIFF_backup.zip
-rw-r--r-- 1 nz nz   2614 Oct 18 22:40 20181018224001_server_GLOBALS_FULL_backup.zip
....
-rw-r--r-- 1 nz nz   3396 Oct 30 20:54 20181030205001_server_HISTDB_DIFF_backup.zip
-rw-r--r-- 1 nz nz   2604 Oct 30 22:40 20181030224001_server_GLOBALS_FULL_backup.zip
-rw-r--r-- 1 nz nz   3523 Oct 30 22:45 20181030224501_server_NZDBA_DIFF_backup.zip
-rw-r--r-- 1 nz nz   4495 Oct 30 23:17 20181030225001_server_ODM_DW_PROD_DIFF_backup.zip
-rw-r--r-- 1 nz nz   3405 Oct 31 20:53 20181031205001_server_HISTDB_DIFF_backup.zip
-rw-r--r-- 1 nz nz   2595 Oct 31 22:40 20181031224001_server_GLOBALS_FULL_backup.zip
-rw-r--r-- 1 nz nz   3578 Oct 31 22:45 20181031224501_server_NZDBA_DIFF_backup.zip
-rw-r--r-- 1 nz nz   4502 Oct 31 23:17 20181031225001_server_ODM_DW_PROD_DIFF_backup.zip
-rw-r--r-- 1 nz nz 251955 Oct 31 22:50 201810_server_backup_n_groom.zip

[nz@s2dw1a server]$ unzip -l 201808_s2dw1a_backup_n_groom.zip
Archive:  201808_nz_backup_n_groom.zip
  Length     Date   Time    Name
 --------    ----   ----    ----
     2607  08-09-18 22:40   20180809224001_server_GLOBALS_FULL_backup.zip
     3540  08-05-18 22:45   20180805224501_server_NZDBA_DIFF_backup.zip
     2620  08-04-18 22:40   20180804224001_server_GLOBALS_FULL_backup.zip
     4496  08-05-18 23:18   20180805225001_server_ODM_DW_PROD_DIFF_backup.zip
     2612  08-06-18 22:40   20180806224001_server_GLOBALS_FULL_backup.zip
     4630  08-06-18 23:36   20180806225001_server_ODM_DW_PROD_FULL_backup.zip
     2614  08-11-18 22:40   20180811224001_server_GLOBALS_FULL_backup.zip
....
     4490  08-30-18 23:18   20180830225001_server_ODM_DW_PROD_DIFF_backup.zip
     3402  08-31-18 20:55   20180831205001_server_HISTDB_DIFF_backup.zip
     2618  08-31-18 22:40   20180831224001_server_GLOBALS_FULL_backup.zip
     3565  08-31-18 22:45   20180831224501_server_NZDBA_DIFF_backup.zip
     4490  08-31-18 23:18   20180831225001_server_ODM_DW_PROD_DIFF_backup.zip
 --------                   -------
   459621                   128 files
[nz@s2dw1a logs]$