Difference between revisions of "PyExpireBackups"
Jump to navigation
Jump to search
| Line 127: | Line 127: | ||
</source> | </source> | ||
== debug dry run == | == debug dry run == | ||
| − | <source lang='bash'> | + | <source lang='bash' highlight='1'> |
expireBackups --ext .tgz -d | expireBackups --ext .tgz -d | ||
keeping 7 files for dayly backup | keeping 7 files for dayly backup | ||
| Line 158: | Line 158: | ||
Δ 364(0.0) days for yearly(364.0) 3/41426.0 days ✅( 4 GB):./sql_backup.2018-05-07.tgz | Δ 364(0.0) days for yearly(364.0) 3/41426.0 days ✅( 4 GB):./sql_backup.2018-05-07.tgz | ||
</source> | </source> | ||
| + | |||
== create TestFiles for Trying it out in your environment == | == create TestFiles for Trying it out in your environment == | ||
<source lang='bash' highlight='1,7'> | <source lang='bash' highlight='1,7'> | ||
Revision as of 14:13, 2 April 2022
OsProject
| OsProject | |
|---|---|
 edit
| |
| id | PyExpireBackups |
| state | active |
| owner | Wolfgang Fahl |
| title | Backup expiration based on rules (yearly,monthly,weekly,daily ...) |
| url | https://github.com/WolfgangFahl/PyExpireBackups |
| version | 0.0.5 |
| description | Expires backup files so save diskspace for unneeded backups - with a pattern of days,weeks,months and years a limit on the total number of backup files is set |
| date | 2022-04-02 |
| since | 2022-03-31 |
| until | |
Installation
pip install PyExpireBackups
# alternatively if your pip is not a python3 pip
pip3 install PyExpireBackups
# local install from source directory of PyExpireBackups
pip install .
upgrade
pip install PyExpireBackups -U
# alternatively if your pip is not a python3 pip
pip3 install PyExpireBackups -U
usage
expireBackups -h
usage: expireBackups [-h] [-d] [--days DAYS] [--weeks WEEKS] [--months MONTHS] [--years YEARS]
[--minFileSize MINFILESIZE] [--rootPath ROOTPATH] [--baseName BASENAME] [--ext EXT]
[--createTestFiles CREATETESTFILES] [-f] [-V]
Backup expiration based on rules (yearly,monthly,weekly,daily ...)
Created by Wolfgang Fahl on 2022-04-01.
Copyright 2008-2022 Wolfgang Fahl. All rights reserved.
Licensed under the Apache License 2.0
http://www.apache.org/licenses/LICENSE-2.0
Distributed on an "AS IS" basis without warranties
or conditions of any kind, either express or implied.
USAGE
optional arguments:
-h, --help show this help message and exit
-d, --debug show debug info
--days DAYS number of consecutive days to keep a daily backup (default: 7)
--weeks WEEKS number of consecutive weeks to keep a weekly backup (default: 6)
--months MONTHS number of consecutive month to keep a monthly backup (default: 8)
--years YEARS number of consecutive years to keep a yearly backup (default: 4)
--minFileSize MINFILESIZE
minimum File size in bytes to filter for (default: 1)
--rootPath ROOTPATH
--baseName BASENAME the basename to filter for (default: None)
--ext EXT the extension to filter for (default: None)
--createTestFiles CREATETESTFILES
create the given number of temporary test files (default: None)
-f, --force
-V, --version show program's version number and exit
Example
Dry Run
expireBackups --ext .tgz
keeping 7 files for dayly backup
keeping 6 files for weekly backup
keeping 8 files for monthly backup
keeping 4 files for yearly backup
expiring 269 files dry run
# 1✅: 0.0 days( 5 GB/ 5 GB)→./sql_backup.2022-04-02.tgz
# 2✅: 3.0 days( 5 GB/ 9 GB)→./sql_backup.2022-03-30.tgz
# 3✅: 4.0 days( 5 GB/ 14 GB)→./sql_backup.2022-03-29.tgz
# 4✅: 5.0 days( 5 GB/ 18 GB)→./sql_backup.2022-03-28.tgz
# 5✅: 7.0 days( 5 GB/ 23 GB)→./sql_backup.2022-03-26.tgz
# 6✅: 9.0 days( 5 GB/ 27 GB)→./sql_backup.2022-03-24.tgz
# 7✅: 11.0 days( 5 GB/ 32 GB)→./sql_backup.2022-03-22.tgz
# 8❌: 15.0 days( 5 GB/ 37 GB)→./sql_backup.2022-03-18.tgz
# 9❌: 17.0 days( 5 GB/ 41 GB)→./sql_backup.2022-03-16.tgz
# 10✅: 18.0 days( 5 GB/ 46 GB)→./sql_backup.2022-03-15.tgz
# 11❌: 19.0 days( 5 GB/ 50 GB)→./sql_backup.2022-03-14.tgz
# 12❌: 20.0 days( 5 GB/ 55 GB)→./sql_backup.2022-03-13.tgz
# 13❌: 22.0 days( 5 GB/ 59 GB)→./sql_backup.2022-03-11.tgz
# 14❌: 23.0 days( 5 GB/ 64 GB)→./sql_backup.2022-03-10.tgz
# 15✅: 35.0 days( 4 GB/ 68 GB)→./sql_backup.2022-02-26.tgz
# 16❌: 37.0 days( 4 GB/ 73 GB)→./sql_backup.2022-02-24.tgz
# 17❌: 39.0 days( 4 GB/ 77 GB)→./sql_backup.2022-02-22.tgz
# 18❌: 40.0 days( 5 GB/ 82 GB)→./sql_backup.2022-02-21.tgz
# 19✅: 43.0 days( 4 GB/ 86 GB)→./sql_backup.2022-02-18.tgz
# 20❌: 44.0 days( 4 GB/ 91 GB)→./sql_backup.2022-02-17.tgz
# 21❌: 46.0 days( 4 GB/ 95 GB)→./sql_backup.2022-02-15.tgz
# 22❌: 47.0 days( 5 GB/ 100 GB)→./sql_backup.2022-02-14.tgz
...
# 264✅:1426.0 days( 4 GB/ 1 TB)→./sql_backup.2018-05-07.tgz
# 265❌:1433.0 days( 4 GB/ 1 TB)→./sql_backup.2018-04-30.tgz
# 266❌:1447.0 days( 4 GB/ 1 TB)→./sql_backup.2018-04-16.tgz
# 267❌:1482.0 days( 4 GB/ 1 TB)→./sql_backup.2018-03-12.tgz
# 268❌:1489.0 days( 4 GB/ 1 TB)→./sql_backup.2018-03-05.tgz
# 269❌:1509.0 days( 4 GB/ 1 TB)→./sql_backup.2018-02-12.tgz
kept 24 files 105 GB
actual run with -f option
expireBackups --ext .tgz -f
keeping 7 files for dayly backup
keeping 6 files for weekly backup
keeping 8 files for monthly backup
keeping 4 files for yearly backup
expiring 24 files dry run
# 1✅: 0.0 days( 5 GB/ 5 GB)→./sql_backup.2022-04-02.tgz
# 2✅: 3.0 days( 5 GB/ 9 GB)→./sql_backup.2022-03-30.tgz
# 3✅: 4.0 days( 5 GB/ 14 GB)→./sql_backup.2022-03-29.tgz
# 4✅: 5.0 days( 5 GB/ 18 GB)→./sql_backup.2022-03-28.tgz
# 5✅: 7.0 days( 5 GB/ 23 GB)→./sql_backup.2022-03-26.tgz
# 6✅: 9.0 days( 5 GB/ 27 GB)→./sql_backup.2022-03-24.tgz
# 7✅: 11.0 days( 5 GB/ 32 GB)→./sql_backup.2022-03-22.tgz
# 8✅: 18.0 days( 5 GB/ 37 GB)→./sql_backup.2022-03-15.tgz
# 9✅: 35.0 days( 4 GB/ 41 GB)→./sql_backup.2022-02-26.tgz
# 10✅: 43.0 days( 4 GB/ 46 GB)→./sql_backup.2022-02-18.tgz
# 11✅: 50.0 days( 4 GB/ 50 GB)→./sql_backup.2022-02-11.tgz
# 12✅: 58.0 days( 4 GB/ 54 GB)→./sql_backup.2022-02-03.tgz
# 13✅: 65.0 days( 4 GB/ 59 GB)→./sql_backup.2022-01-27.tgz
# 14✅: 94.0 days( 4 GB/ 63 GB)→./sql_backup.2021-12-29.tgz
# 15✅: 148.0 days( 4 GB/ 67 GB)→./sql_backup.2021-11-05.tgz
# 16✅: 177.0 days( 4 GB/ 71 GB)→./sql_backup.2021-10-07.tgz
# 17✅: 205.0 days( 4 GB/ 75 GB)→./sql_backup.2021-09-09.tgz
# 18✅: 236.0 days( 4 GB/ 80 GB)→./sql_backup.2021-08-09.tgz
# 19✅: 267.0 days( 4 GB/ 84 GB)→./sql_backup.2021-07-09.tgz
# 20✅: 295.0 days( 4 GB/ 87 GB)→./sql_backup.2021-06-11.tgz
# 21✅: 324.0 days( 4 GB/ 91 GB)→./sql_backup.2021-05-13.tgz
# 22✅: 691.0 days( 6 GB/ 97 GB)→./sql_backup.2020-05-11.tgz
# 23✅:1062.0 days( 5 GB/ 102 GB)→./sql_backup.2019-05-06.tgz
# 24✅:1426.0 days( 4 GB/ 105 GB)→./sql_backup.2018-05-07.tgz
kept 24 files 105 GB
debug dry run
expireBackups --ext .tgz -d
keeping 7 files for dayly backup
Δ 0(-1.0) days for dayly(1.0) 1/7 0.0 days ✅( 5 GB):./sql_backup.2022-04-02.tgz
Δ 3(2.0) days for dayly(1.0) 2/7 3.0 days ✅( 5 GB):./sql_backup.2022-03-30.tgz
Δ 1(0.0) days for dayly(1.0) 3/7 4.0 days ✅( 5 GB):./sql_backup.2022-03-29.tgz
Δ 1(0.0) days for dayly(1.0) 4/7 5.0 days ✅( 5 GB):./sql_backup.2022-03-28.tgz
Δ 2(1.0) days for dayly(1.0) 5/7 7.0 days ✅( 5 GB):./sql_backup.2022-03-26.tgz
Δ 2(1.0) days for dayly(1.0) 6/7 9.0 days ✅( 5 GB):./sql_backup.2022-03-24.tgz
Δ 2(1.0) days for dayly(1.0) 7/7 11.0 days ✅( 5 GB):./sql_backup.2022-03-22.tgz
keeping 6 files for weekly backup
Δ 7(0.0) days for weekly(7.0) 1/6 18.0 days ✅( 5 GB):./sql_backup.2022-03-15.tgz
Δ 17(10.0) days for weekly(7.0) 2/6 35.0 days ✅( 4 GB):./sql_backup.2022-02-26.tgz
Δ 8(1.0) days for weekly(7.0) 3/6 43.0 days ✅( 4 GB):./sql_backup.2022-02-18.tgz
Δ 7(0.0) days for weekly(7.0) 4/6 50.0 days ✅( 4 GB):./sql_backup.2022-02-11.tgz
Δ 8(1.0) days for weekly(7.0) 5/6 58.0 days ✅( 4 GB):./sql_backup.2022-02-03.tgz
Δ 7(0.0) days for weekly(7.0) 6/6 65.0 days ✅( 4 GB):./sql_backup.2022-01-27.tgz
keeping 8 files for monthly backup
Δ 29(1.0) days for monthly(28.0) 1/8 94.0 days ✅( 4 GB):./sql_backup.2021-12-29.tgz
Δ 54(26.0) days for monthly(28.0) 2/8 148.0 days ✅( 4 GB):./sql_backup.2021-11-05.tgz
Δ 29(1.0) days for monthly(28.0) 3/8 177.0 days ✅( 4 GB):./sql_backup.2021-10-07.tgz
Δ 28(0.0) days for monthly(28.0) 4/8 205.0 days ✅( 4 GB):./sql_backup.2021-09-09.tgz
Δ 31(3.0) days for monthly(28.0) 5/8 236.0 days ✅( 4 GB):./sql_backup.2021-08-09.tgz
Δ 31(3.0) days for monthly(28.0) 6/8 267.0 days ✅( 4 GB):./sql_backup.2021-07-09.tgz
Δ 28(0.0) days for monthly(28.0) 7/8 295.0 days ✅( 4 GB):./sql_backup.2021-06-11.tgz
Δ 29(1.0) days for monthly(28.0) 8/8 324.0 days ✅( 4 GB):./sql_backup.2021-05-13.tgz
keeping 4 files for yearly backup
Δ 367(3.0) days for yearly(364.0) 1/4 691.0 days ✅( 6 GB):./sql_backup.2020-05-11.tgz
Δ 371(7.0) days for yearly(364.0) 2/41062.0 days ✅( 5 GB):./sql_backup.2019-05-06.tgz
Δ 364(0.0) days for yearly(364.0) 3/41426.0 days ✅( 4 GB):./sql_backup.2018-05-07.tgz
create TestFiles for Trying it out in your environment
expireBackups --createTestFiles 30
created 30 test files with extension '.tst' in /tmp
Please try out
expireBackups --rootPath /tmp --baseName expireBackup --ext .tst --minFileSize 0
then try appending the -f option to the command that will actually delete files (which are in a temporary directory
and run the command another time with that option to see that no files are deleted any more on second run
expireBackups --rootPath /tmp --baseName expireBackup --ext .tst --minFileSize 0
keeping 7 files for dayly backup
keeping 6 files for weekly backup
expiring 30 files dry run
# 1✅: 1.0 days( 0 B/ 0 B)→/tmp/expireBackupTest-1daysOld-ss9tl1g_.tst
# 2✅: 2.0 days( 0 B/ 0 B)→/tmp/expireBackupTest-2daysOld-j4m0jk00.tst
# 3✅: 3.0 days( 0 B/ 0 B)→/tmp/expireBackupTest-3daysOld-q7b8de0g.tst
# 4✅: 4.0 days( 0 B/ 0 B)→/tmp/expireBackupTest-4daysOld-xqa8z0k8.tst
# 5✅: 5.0 days( 0 B/ 0 B)→/tmp/expireBackupTest-5daysOld-mrqx5raw.tst
# 6✅: 6.0 days( 0 B/ 0 B)→/tmp/expireBackupTest-6daysOld-y0gunu9t.tst
# 7✅: 7.0 days( 0 B/ 0 B)→/tmp/expireBackupTest-7daysOld-2par7sek.tst
# 8❌: 8.0 days( 0 B/ 0 B)→/tmp/expireBackupTest-8daysOld-jg9_fpin.tst
# 9❌: 9.0 days( 0 B/ 0 B)→/tmp/expireBackupTest-9daysOld-9peeb1yi.tst
# 10❌: 10.0 days( 0 B/ 0 B)→/tmp/expireBackupTest-10daysOld-ded_f1fj.tst
# 11❌: 11.0 days( 0 B/ 0 B)→/tmp/expireBackupTest-11daysOld-dcn2an98.tst
# 12❌: 12.0 days( 0 B/ 0 B)→/tmp/expireBackupTest-12daysOld-d1f_l5tl.tst
# 13❌: 13.0 days( 0 B/ 0 B)→/tmp/expireBackupTest-13daysOld-dy8jg11j.tst
# 14✅: 14.0 days( 0 B/ 0 B)→/tmp/expireBackupTest-14daysOld-x4mpow_c.tst
# 15❌: 15.0 days( 0 B/ 0 B)→/tmp/expireBackupTest-15daysOld-ln2fkkht.tst
# 16❌: 16.0 days( 0 B/ 0 B)→/tmp/expireBackupTest-16daysOld-ycrryytq.tst
# 17❌: 17.0 days( 0 B/ 0 B)→/tmp/expireBackupTest-17daysOld-jor9sj4i.tst
# 18❌: 18.0 days( 0 B/ 0 B)→/tmp/expireBackupTest-18daysOld-ywogrc32.tst
# 19❌: 19.0 days( 0 B/ 0 B)→/tmp/expireBackupTest-19daysOld-zdjk76vh.tst
# 20❌: 20.0 days( 0 B/ 0 B)→/tmp/expireBackupTest-20daysOld-u7t9thta.tst
# 21✅: 21.0 days( 0 B/ 0 B)→/tmp/expireBackupTest-21daysOld-g_mq3yke.tst
# 22❌: 22.0 days( 0 B/ 0 B)→/tmp/expireBackupTest-22daysOld-x3yvl10i.tst
# 23❌: 23.0 days( 0 B/ 0 B)→/tmp/expireBackupTest-23daysOld-7tyjl3f7.tst
# 24❌: 24.0 days( 0 B/ 0 B)→/tmp/expireBackupTest-24daysOld-u0cacvjs.tst
# 25❌: 25.0 days( 0 B/ 0 B)→/tmp/expireBackupTest-25daysOld-jgm7h9d1.tst
# 26❌: 26.0 days( 0 B/ 0 B)→/tmp/expireBackupTest-26daysOld-kia0ja07.tst
# 27❌: 27.0 days( 0 B/ 0 B)→/tmp/expireBackupTest-27daysOld-6470pec4.tst
# 28✅: 28.0 days( 0 B/ 0 B)→/tmp/expireBackupTest-28daysOld-fk4c5m9q.tst
# 29❌: 29.0 days( 0 B/ 0 B)→/tmp/expireBackupTest-29daysOld-nwmyvgf8.tst
# 30❌: 30.0 days( 0 B/ 0 B)→/tmp/expireBackupTest-30daysOld-5ncjuw56.tst
kept 10 files 0 B
tickets
- Issue 8 - alway keep the most recent file✓
- Issue 7 - show summary about kept files✓
- Issue 6 - show debug information✓
- Issue 5 - allow dry run✓
- Issue 4 - allow to specify minimum size of files✓
- Issue 3 - show # age and size of file and total (sum) of file sizes✓
- Issue 2 - allow to create TestFiles via commandLine✓
- Issue 1 - refactor com.bitplan.backup Java solution✓