PyExpireBackups

Wolfgang Fahl

OsProject

OsProject
edit
id	PyExpireBackups
state	active
owner	Wolfgang Fahl
title	Backup expiration based on rules (yearly,monthly,weekly,daily ...)
url	https://github.com/WolfgangFahl/PyExpireBackups
version	0.0.5
description	Expires backup files so save diskspace for unneeded backups - with a pattern of days,weeks,months and years a limit on the total number of backup files is set
date	2022-04-02
since	2022-03-31
until

Installation

pip install PyExpireBackups
# alternatively if your pip is not a python3 pip
pip3 install PyExpireBackups 
# local install from source directory of PyExpireBackups 
pip install .

upgrade

pip install PyExpireBackups  -U
# alternatively if your pip is not a python3 pip
pip3 install PyExpireBackups -U

usage

expireBackups -h
usage: expireBackups [-h] [-d] [--days DAYS] [--weeks WEEKS] [--months MONTHS] [--years YEARS]
                     [--minFileSize MINFILESIZE] [--rootPath ROOTPATH] [--baseName BASENAME] [--ext EXT]
                     [--createTestFiles CREATETESTFILES] [-f] [-V]

Backup expiration based on rules (yearly,monthly,weekly,daily ...)

  Created by Wolfgang Fahl on 2022-04-01.
  Copyright 2008-2022 Wolfgang Fahl. All rights reserved.

  Licensed under the Apache License 2.0
  http://www.apache.org/licenses/LICENSE-2.0

  Distributed on an "AS IS" basis without warranties
  or conditions of any kind, either express or implied.

USAGE

optional arguments:
  -h, --help            show this help message and exit
  -d, --debug           show debug info
  --days DAYS           number of consecutive days to keep a daily backup (default: 7)
  --weeks WEEKS         number of consecutive weeks to keep a weekly backup (default: 6)
  --months MONTHS       number of consecutive month to keep a monthly backup (default: 8)
  --years YEARS         number of consecutive years to keep a yearly backup (default: 4)
  --minFileSize MINFILESIZE
                        minimum File size in bytes to filter for (default: 1)
  --rootPath ROOTPATH
  --baseName BASENAME   the basename to filter for (default: None)
  --ext EXT             the extension to filter for (default: None)
  --createTestFiles CREATETESTFILES
                        create the given number of temporary test files (default: None)
  -f, --force
  -V, --version         show program's version number and exit

Example

OsProject[edit]

OsProject
edit
id	PyExpireBackups
state	active
owner	Wolfgang Fahl
title	Backup expiration based on rules (yearly,monthly,weekly,daily ...)
url	https://github.com/WolfgangFahl/PyExpireBackups
version	0.0.5
description	Expires backup files so save diskspace for unneeded backups - with a pattern of days,weeks,months and years a limit on the total number of backup files is set
date	2022-04-02
since	2022-03-31
until

Installation[edit]

pip install PyExpireBackups
# alternatively if your pip is not a python3 pip
pip3 install PyExpireBackups 
# local install from source directory of PyExpireBackups 
pip install .

upgrade[edit]

pip install PyExpireBackups  -U
# alternatively if your pip is not a python3 pip
pip3 install PyExpireBackups -U

OsProject
edit
id	PyExpireBackups
state	active
owner	Wolfgang Fahl
title	Backup expiration based on rules (yearly,monthly,weekly,daily ...)
url	https://github.com/WolfgangFahl/PyExpireBackups
version	0.0.5
description	Expires backup files so save diskspace for unneeded backups - with a pattern of days,weeks,months and years a limit on the total number of backup files is set
date	2022-04-02
since	2022-03-31
until

Dry Run[edit]

expireBackups --ext .tgz
keeping 7 files for dayly backup
keeping 6 files for weekly backup
keeping 8 files for monthly backup
keeping 4 files for yearly backup
expiring 269 files dry run
#   1✅:   0.0 days(    5 GB/    5 GB)→./sql_backup.2022-04-02.tgz
#   2✅:   3.0 days(    5 GB/    9 GB)→./sql_backup.2022-03-30.tgz
#   3✅:   4.0 days(    5 GB/   14 GB)→./sql_backup.2022-03-29.tgz
#   4✅:   5.0 days(    5 GB/   18 GB)→./sql_backup.2022-03-28.tgz
#   5✅:   7.0 days(    5 GB/   23 GB)→./sql_backup.2022-03-26.tgz
#   6✅:   9.0 days(    5 GB/   27 GB)→./sql_backup.2022-03-24.tgz
#   7✅:  11.0 days(    5 GB/   32 GB)→./sql_backup.2022-03-22.tgz
#   8❌:  15.0 days(    5 GB/   37 GB)→./sql_backup.2022-03-18.tgz
#   9❌:  17.0 days(    5 GB/   41 GB)→./sql_backup.2022-03-16.tgz
#  10✅:  18.0 days(    5 GB/   46 GB)→./sql_backup.2022-03-15.tgz
#  11❌:  19.0 days(    5 GB/   50 GB)→./sql_backup.2022-03-14.tgz
#  12❌:  20.0 days(    5 GB/   55 GB)→./sql_backup.2022-03-13.tgz
#  13❌:  22.0 days(    5 GB/   59 GB)→./sql_backup.2022-03-11.tgz
#  14❌:  23.0 days(    5 GB/   64 GB)→./sql_backup.2022-03-10.tgz
#  15✅:  35.0 days(    4 GB/   68 GB)→./sql_backup.2022-02-26.tgz
#  16❌:  37.0 days(    4 GB/   73 GB)→./sql_backup.2022-02-24.tgz
#  17❌:  39.0 days(    4 GB/   77 GB)→./sql_backup.2022-02-22.tgz
#  18❌:  40.0 days(    5 GB/   82 GB)→./sql_backup.2022-02-21.tgz
#  19✅:  43.0 days(    4 GB/   86 GB)→./sql_backup.2022-02-18.tgz
#  20❌:  44.0 days(    4 GB/   91 GB)→./sql_backup.2022-02-17.tgz
#  21❌:  46.0 days(    4 GB/   95 GB)→./sql_backup.2022-02-15.tgz
#  22❌:  47.0 days(    5 GB/  100 GB)→./sql_backup.2022-02-14.tgz
...
# 264✅:1426.0 days(    4 GB/    1 TB)→./sql_backup.2018-05-07.tgz
# 265❌:1433.0 days(    4 GB/    1 TB)→./sql_backup.2018-04-30.tgz
# 266❌:1447.0 days(    4 GB/    1 TB)→./sql_backup.2018-04-16.tgz
# 267❌:1482.0 days(    4 GB/    1 TB)→./sql_backup.2018-03-12.tgz
# 268❌:1489.0 days(    4 GB/    1 TB)→./sql_backup.2018-03-05.tgz
# 269❌:1509.0 days(    4 GB/    1 TB)→./sql_backup.2018-02-12.tgz
kept 24 files   105 GB

actual run with -f option[edit]

expireBackups --ext .tgz -f
keeping 7 files for dayly backup
keeping 6 files for weekly backup
keeping 8 files for monthly backup
keeping 4 files for yearly backup
expiring 24 files dry run
#   1✅:   0.0 days(    5 GB/    5 GB)→./sql_backup.2022-04-02.tgz
#   2✅:   3.0 days(    5 GB/    9 GB)→./sql_backup.2022-03-30.tgz
#   3✅:   4.0 days(    5 GB/   14 GB)→./sql_backup.2022-03-29.tgz
#   4✅:   5.0 days(    5 GB/   18 GB)→./sql_backup.2022-03-28.tgz
#   5✅:   7.0 days(    5 GB/   23 GB)→./sql_backup.2022-03-26.tgz
#   6✅:   9.0 days(    5 GB/   27 GB)→./sql_backup.2022-03-24.tgz
#   7✅:  11.0 days(    5 GB/   32 GB)→./sql_backup.2022-03-22.tgz
#   8✅:  18.0 days(    5 GB/   37 GB)→./sql_backup.2022-03-15.tgz
#   9✅:  35.0 days(    4 GB/   41 GB)→./sql_backup.2022-02-26.tgz
#  10✅:  43.0 days(    4 GB/   46 GB)→./sql_backup.2022-02-18.tgz
#  11✅:  50.0 days(    4 GB/   50 GB)→./sql_backup.2022-02-11.tgz
#  12✅:  58.0 days(    4 GB/   54 GB)→./sql_backup.2022-02-03.tgz
#  13✅:  65.0 days(    4 GB/   59 GB)→./sql_backup.2022-01-27.tgz
#  14✅:  94.0 days(    4 GB/   63 GB)→./sql_backup.2021-12-29.tgz
#  15✅: 148.0 days(    4 GB/   67 GB)→./sql_backup.2021-11-05.tgz
#  16✅: 177.0 days(    4 GB/   71 GB)→./sql_backup.2021-10-07.tgz
#  17✅: 205.0 days(    4 GB/   75 GB)→./sql_backup.2021-09-09.tgz
#  18✅: 236.0 days(    4 GB/   80 GB)→./sql_backup.2021-08-09.tgz
#  19✅: 267.0 days(    4 GB/   84 GB)→./sql_backup.2021-07-09.tgz
#  20✅: 295.0 days(    4 GB/   87 GB)→./sql_backup.2021-06-11.tgz
#  21✅: 324.0 days(    4 GB/   91 GB)→./sql_backup.2021-05-13.tgz
#  22✅: 691.0 days(    6 GB/   97 GB)→./sql_backup.2020-05-11.tgz
#  23✅:1062.0 days(    5 GB/  102 GB)→./sql_backup.2019-05-06.tgz
#  24✅:1426.0 days(    4 GB/  105 GB)→./sql_backup.2018-05-07.tgz
kept 24 files   105 GB

debug dry run[edit]

expireBackups --ext .tgz -d
keeping 7 files for dayly backup
Δ 0(-1.0) days for dayly(1.0) 1/7   0.0 days ✅(    5 GB):./sql_backup.2022-04-02.tgz
Δ 3(2.0) days for dayly(1.0) 2/7   3.0 days ✅(    5 GB):./sql_backup.2022-03-30.tgz
Δ 1(0.0) days for dayly(1.0) 3/7   4.0 days ✅(    5 GB):./sql_backup.2022-03-29.tgz
Δ 1(0.0) days for dayly(1.0) 4/7   5.0 days ✅(    5 GB):./sql_backup.2022-03-28.tgz
Δ 2(1.0) days for dayly(1.0) 5/7   7.0 days ✅(    5 GB):./sql_backup.2022-03-26.tgz
Δ 2(1.0) days for dayly(1.0) 6/7   9.0 days ✅(    5 GB):./sql_backup.2022-03-24.tgz
Δ 2(1.0) days for dayly(1.0) 7/7  11.0 days ✅(    5 GB):./sql_backup.2022-03-22.tgz
keeping 6 files for weekly backup
Δ 7(0.0) days for weekly(7.0) 1/6  18.0 days ✅(    5 GB):./sql_backup.2022-03-15.tgz
Δ 17(10.0) days for weekly(7.0) 2/6  35.0 days ✅(    4 GB):./sql_backup.2022-02-26.tgz
Δ 8(1.0) days for weekly(7.0) 3/6  43.0 days ✅(    4 GB):./sql_backup.2022-02-18.tgz
Δ 7(0.0) days for weekly(7.0) 4/6  50.0 days ✅(    4 GB):./sql_backup.2022-02-11.tgz
Δ 8(1.0) days for weekly(7.0) 5/6  58.0 days ✅(    4 GB):./sql_backup.2022-02-03.tgz
Δ 7(0.0) days for weekly(7.0) 6/6  65.0 days ✅(    4 GB):./sql_backup.2022-01-27.tgz
keeping 8 files for monthly backup
Δ 29(1.0) days for monthly(28.0) 1/8  94.0 days ✅(    4 GB):./sql_backup.2021-12-29.tgz
Δ 54(26.0) days for monthly(28.0) 2/8 148.0 days ✅(    4 GB):./sql_backup.2021-11-05.tgz
Δ 29(1.0) days for monthly(28.0) 3/8 177.0 days ✅(    4 GB):./sql_backup.2021-10-07.tgz
Δ 28(0.0) days for monthly(28.0) 4/8 205.0 days ✅(    4 GB):./sql_backup.2021-09-09.tgz
Δ 31(3.0) days for monthly(28.0) 5/8 236.0 days ✅(    4 GB):./sql_backup.2021-08-09.tgz
Δ 31(3.0) days for monthly(28.0) 6/8 267.0 days ✅(    4 GB):./sql_backup.2021-07-09.tgz
Δ 28(0.0) days for monthly(28.0) 7/8 295.0 days ✅(    4 GB):./sql_backup.2021-06-11.tgz
Δ 29(1.0) days for monthly(28.0) 8/8 324.0 days ✅(    4 GB):./sql_backup.2021-05-13.tgz
keeping 4 files for yearly backup
Δ 367(3.0) days for yearly(364.0) 1/4 691.0 days ✅(    6 GB):./sql_backup.2020-05-11.tgz
Δ 371(7.0) days for yearly(364.0) 2/41062.0 days ✅(    5 GB):./sql_backup.2019-05-06.tgz
Δ 364(0.0) days for yearly(364.0) 3/41426.0 days ✅(    4 GB):./sql_backup.2018-05-07.tgz

create TestFiles for Trying it out in your environment[edit]

expireBackups --createTestFiles 30
created 30 test files with extension '.tst' in /tmp
Please try out 
expireBackups --rootPath /tmp --baseName expireBackup --ext .tst --minFileSize 0
then try appending the -f option to the command that will actually delete files (which are in a temporary directory
and run the command another time with that option to see that no files are deleted any more on second run
expireBackups --rootPath /tmp --baseName expireBackup --ext .tst --minFileSize 0
keeping 7 files for dayly backup
keeping 6 files for weekly backup
expiring 30 files dry run
#   1✅:   1.0 days(    0  B/    0  B)→/tmp/expireBackupTest-1daysOld-ss9tl1g_.tst
#   2✅:   2.0 days(    0  B/    0  B)→/tmp/expireBackupTest-2daysOld-j4m0jk00.tst
#   3✅:   3.0 days(    0  B/    0  B)→/tmp/expireBackupTest-3daysOld-q7b8de0g.tst
#   4✅:   4.0 days(    0  B/    0  B)→/tmp/expireBackupTest-4daysOld-xqa8z0k8.tst
#   5✅:   5.0 days(    0  B/    0  B)→/tmp/expireBackupTest-5daysOld-mrqx5raw.tst
#   6✅:   6.0 days(    0  B/    0  B)→/tmp/expireBackupTest-6daysOld-y0gunu9t.tst
#   7✅:   7.0 days(    0  B/    0  B)→/tmp/expireBackupTest-7daysOld-2par7sek.tst
#   8❌:   8.0 days(    0  B/    0  B)→/tmp/expireBackupTest-8daysOld-jg9_fpin.tst
#   9❌:   9.0 days(    0  B/    0  B)→/tmp/expireBackupTest-9daysOld-9peeb1yi.tst
#  10❌:  10.0 days(    0  B/    0  B)→/tmp/expireBackupTest-10daysOld-ded_f1fj.tst
#  11❌:  11.0 days(    0  B/    0  B)→/tmp/expireBackupTest-11daysOld-dcn2an98.tst
#  12❌:  12.0 days(    0  B/    0  B)→/tmp/expireBackupTest-12daysOld-d1f_l5tl.tst
#  13❌:  13.0 days(    0  B/    0  B)→/tmp/expireBackupTest-13daysOld-dy8jg11j.tst
#  14✅:  14.0 days(    0  B/    0  B)→/tmp/expireBackupTest-14daysOld-x4mpow_c.tst
#  15❌:  15.0 days(    0  B/    0  B)→/tmp/expireBackupTest-15daysOld-ln2fkkht.tst
#  16❌:  16.0 days(    0  B/    0  B)→/tmp/expireBackupTest-16daysOld-ycrryytq.tst
#  17❌:  17.0 days(    0  B/    0  B)→/tmp/expireBackupTest-17daysOld-jor9sj4i.tst
#  18❌:  18.0 days(    0  B/    0  B)→/tmp/expireBackupTest-18daysOld-ywogrc32.tst
#  19❌:  19.0 days(    0  B/    0  B)→/tmp/expireBackupTest-19daysOld-zdjk76vh.tst
#  20❌:  20.0 days(    0  B/    0  B)→/tmp/expireBackupTest-20daysOld-u7t9thta.tst
#  21✅:  21.0 days(    0  B/    0  B)→/tmp/expireBackupTest-21daysOld-g_mq3yke.tst
#  22❌:  22.0 days(    0  B/    0  B)→/tmp/expireBackupTest-22daysOld-x3yvl10i.tst
#  23❌:  23.0 days(    0  B/    0  B)→/tmp/expireBackupTest-23daysOld-7tyjl3f7.tst
#  24❌:  24.0 days(    0  B/    0  B)→/tmp/expireBackupTest-24daysOld-u0cacvjs.tst
#  25❌:  25.0 days(    0  B/    0  B)→/tmp/expireBackupTest-25daysOld-jgm7h9d1.tst
#  26❌:  26.0 days(    0  B/    0  B)→/tmp/expireBackupTest-26daysOld-kia0ja07.tst
#  27❌:  27.0 days(    0  B/    0  B)→/tmp/expireBackupTest-27daysOld-6470pec4.tst
#  28✅:  28.0 days(    0  B/    0  B)→/tmp/expireBackupTest-28daysOld-fk4c5m9q.tst
#  29❌:  29.0 days(    0  B/    0  B)→/tmp/expireBackupTest-29daysOld-nwmyvgf8.tst
#  30❌:  30.0 days(    0  B/    0  B)→/tmp/expireBackupTest-30daysOld-5ncjuw56.tst
kept 10 files     0  B

tickets[edit]

PyExpireBackups Wolfgang Fahl

PyExpireBackups

Contents

OsProject

Installation

upgrade

usage

Example

OsProject[edit]

Installation[edit]

upgrade[edit]

OsProject[edit]

Installation[edit]

usage[edit]

usage[edit]

Contents

Example[edit]

Dry Run[edit]

actual run with -f option[edit]

debug dry run[edit]

create TestFiles for Trying it out in your environment[edit]

Dry Run[edit]

actual run with -f option[edit]

debug dry run[edit]

create TestFiles for Trying it out in your environment[edit]

tickets[edit]