Check for expiration dates on files currently on disk

This commit is contained in:
Bruno Bord 2019-09-20 15:40:55 +02:00
parent 4cf36c1d80
commit dc8ed7ec1e
No known key found for this signature in database
GPG Key ID: 9499EA6788BF80A1
3 changed files with 40 additions and 8 deletions

View File

@ -2,7 +2,7 @@
## master (unreleased)
Nothing here yet.
* Added a ``--check-only`` argument to ``download.py`` to display the expiration dates of the files currently on disk.
## 0.0.2 (2019-08-23)

View File

@ -1,5 +1,7 @@
# Data files for Skyfield
## Rationale
[Skyfield](https://rhodesmill.org/skyfield/) is a Python library for astronomical computations. It depends on various data files to accurately compute moon phases, planet positions, etc.
Several issues are raised by these data files:
@ -9,6 +11,17 @@ Several issues are raised by these data files:
* In some countries, or behind some filtering proxies, the USNO is considered as a military website, and thus is **blocked**.
* These files have **an expiration date** (in a more or less distant future). As a consequence, even if the files are already downloaded in the right path, at each runtime you could possibly have to download one or more files before making any computation using them.
## Currently known expiration dates
| File | Date |
|:---------------:|:-------------:|
| deltat.data | 2020-06-01 |
| Leap_Second.dat | 2020-07-28 |
| deltat.preds | 2021-01-01 |
| de421.bsp | *unknown*(\*) |
(\*) Even though its expiration date has not been extracted through a convenient script, we can state that it should expire in the year 2053.
## Goal for this project
* Providing at least the most common of these assets in Python Package.
@ -43,7 +56,6 @@ If you want to make sure that the data files would **never** be downloaded, you
load = Loader(get_skyfield_data_path(), expire=False)
```
## Developers
We're providing a ``Makefile`` with basic targets to play around with the toolkit. use ``make help`` to get more details.

View File

@ -99,6 +99,16 @@ def download(url, target):
shutil.copyfileobj(response, fd)
def get_expiration_date(target, params):
expiration_date = None
if exists(target):
expiration_func = params.get("expiration_func")
if expiration_func:
with open(target, 'rb') as fd:
expiration_date = expiration_func(fd)
return expiration_date
def check_should_i_download(target, params):
"""
Check if I should download the target file or not.
@ -108,10 +118,8 @@ def check_should_i_download(target, params):
if exists(target):
# Search if the file has expired
expiration_func = params.get("expiration_func")
if expiration_func:
with open(target, 'rb') as fd:
expiration_date = expiration_func(fd)
expiration_date = get_expiration_date(target, params)
if expiration_date:
if date.today() <= expiration_date:
should_i_download = False
reason = 'file will expire at {}'.format(expiration_date)
@ -119,7 +127,7 @@ def check_should_i_download(target, params):
reason = "expiration date: {}".format(expiration_date)
else:
should_i_download = False
reason = "file already exists, no expiration function"
reason = "file already exists, no expiration function/date"
else:
reason = "file not here"
@ -148,7 +156,15 @@ def main(args):
url = "{}/{}".format(server, filename)
target = join(get_skyfield_data_path(), filename)
if args.force:
if args.check_only:
expiration_date = get_expiration_date(target, params)
print(
"File: {} => expiration: {}".format(
filename, expiration_date or "`unknown`"
)
)
continue
elif args.force:
should_i_download, reason = True, "Forced download"
else:
should_i_download, reason = check_should_i_download(
@ -169,5 +185,9 @@ if __name__ == '__main__':
'--force', action="store_true", default=False,
help="Force download, ignore expiration date or presence on disk."
)
parser.add_argument(
"--check-only", action="store_true", default=False,
help="Check only the expiration dates of the files on disk."
)
args = parser.parse_args()
main(args)