add tools/setup_base_environments.py development script
Installs packages into multiple Python environments so they can be used for testing this project. This automates all the previously manually done work on setting up those environments. The script contains a multitude of open TODO comments and should still be considered 'work in progress', but successfully gets the work done at least on one Windows 7 x64 SP1 machine with 17 different parallel Python installations. Updated relevant HACKING.rst docs. Relevant configuration added to the main project Python configuration file 'setup.cfg'. All Python modules indended for use in different project development utility scripts have been placed under the suds_devel package folder located under the tools project folder. The new scripts: - are included in the project's source distribution, - are not installed with the project - do not need py2to3 processing - do not have any tests of their own yet tools/__* folders get created and used as local caches by the new tools/setup_base_environments.py development script and so must not included in the project's source distribution.
This commit is contained in:
Jurko Gospodnetić
parent
c6b64a1849
commit
ea02fb96ad
152
HACKING.rst
152
HACKING.rst
|
|
@ -54,7 +54,8 @@ TOP-LEVEL PROJECT FILES & FOLDERS
|
|||
|
||||
| tools/
|
||||
|
||||
* Project development utility scripts.
|
||||
* Project development utility scripts. Any internal Python modules are located
|
||||
under its ``suds_devel/`` package folder.
|
||||
|
||||
| MANIFEST.in
|
||||
|
||||
|
|
@ -277,11 +278,16 @@ In all command-line examples below pyX, pyXY & pyXYZ represent a Python
|
|||
interpreter executable for a specific Python version X, X.Y & X.Y.Z
|
||||
respectively.
|
||||
|
||||
Notes in this section should hold for all Python releases except some older ones
|
||||
explicitly listed at the end of this section.
|
||||
Setting up the development & testing environment
|
||||
------------------------------------------------
|
||||
|
||||
Testing
|
||||
-------
|
||||
``tools/setup_base_environments.py`` script should be used for setting up your
|
||||
basic Python environments so they support testing our project. The script can
|
||||
be configured configured from the main project Python configuration file
|
||||
``setup.cfg``. It implements all the backward compatibility tweaks that would
|
||||
otherwise need to be done manually in order to be able to test our project in
|
||||
those environments. These tweaks are no longer documented elsewhere so anyone
|
||||
interested in the details should consult the script's sources.
|
||||
|
||||
Project's test suite requires the ``pytest`` testing framework to run. The test
|
||||
code base is compatible with pytest 2.4.0+ (prior versions do not support
|
||||
|
|
@ -295,6 +301,30 @@ The testing environment is generally set up as follows:
|
|||
#. Install ``pip`` using ``setuptools`` (optional).
|
||||
#. Install ``pytest`` using ``pip`` or ``setuptools``.
|
||||
|
||||
Some older Python environments may have slight issues caused by varying support
|
||||
levels in different used Python packages, but the basic testing functionality
|
||||
has been tested to make sure it works on as wide array of supported platforms as
|
||||
possible.
|
||||
|
||||
Examples of such issues:
|
||||
|
||||
* Colors not getting displayed on a Windows console terminal, and possibly
|
||||
ANSI color code escape sequences getting displayed instead.
|
||||
* ``pip`` utility not being runnable from the command-line using the ``py -m
|
||||
pip`` syntax for some older versions.
|
||||
* Some specific older Python versions having no SSL support and so must reuse
|
||||
installations downloaded by other Python versions.
|
||||
|
||||
|
||||
Running the project tests
|
||||
-------------------------
|
||||
|
||||
``tools/run_all_tests.cmd`` script is a basic *poor man's tox* development
|
||||
script that can be used for running the full project test suite using multiple
|
||||
Python interpreter versions on a Windows development machine. It is intended to
|
||||
be replaced by a more portable ``tox`` based or similar automated testing
|
||||
solution some time in the future.
|
||||
|
||||
To run all of the project unit tests with a specific interpreter without
|
||||
additional configuration options run the project's ``setup.py`` script with the
|
||||
'test' parameter and an appropriate Python interpreter. E.g. run any of the
|
||||
|
|
@ -334,12 +364,6 @@ options. Some interesting ones:
|
|||
-x stop on first failure
|
||||
--pdb enter Python debugger on failure
|
||||
|
||||
``tools/run_all_tests.cmd`` script is a basic *poor man's tox* development
|
||||
script that can be used for running the full project test suite using multiple
|
||||
Python interpreter versions on a Windows development machine. It is intended to
|
||||
be replaced by a more portable ``tox`` based or similar automated testing
|
||||
solution some time in the future.
|
||||
|
||||
Setting up multiple parallel Python interpreter versions on Windows
|
||||
-------------------------------------------------------------------
|
||||
|
||||
|
|
@ -369,112 +393,6 @@ to determine where to install its package data. In that case you can set those
|
|||
entries manually, e.g. by using a script similar to the one found at
|
||||
`<http://nedbatchelder.com/blog/201007/installing_python_packages_from_windows_installers_into.html>`_.
|
||||
|
||||
Setting up specific Python versions
|
||||
-----------------------------------
|
||||
|
||||
Installing setuptools on Python 2.4.x & 2.5.x
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
* ``setuptools``
|
||||
|
||||
* 1.4.2 - last version supporting Python 2.4 & 2.5.
|
||||
|
||||
* Install using the ``ez_setup.py`` script from the ``setuptools`` 1.4.2
|
||||
release::
|
||||
|
||||
py24 ez_setup_1.4.2.py
|
||||
|
||||
Python 2.4.x
|
||||
~~~~~~~~~~~~
|
||||
|
||||
* ``pip``
|
||||
|
||||
* 1.1 - last version supporting Python 2.4.
|
||||
|
||||
* Install using::
|
||||
|
||||
py244 -m easy_install pip==1.1
|
||||
|
||||
* Can not be run using ``python.exe -m pip``.
|
||||
|
||||
* Workaround is to use one of the ``pip`` startup scripts found in the
|
||||
Python installation's ``Scripts`` folder or the following invocation::
|
||||
|
||||
py244 -c "import pip;pip.main()" <regular-pip-options>
|
||||
|
||||
* ``pytest``
|
||||
|
||||
* 2.4.1 - last version supporting Python 2.4.
|
||||
|
||||
* Install::
|
||||
|
||||
py244 -c "import pip;pip.main()" install pytest==2.4.1 py==1.4.15
|
||||
|
||||
* ``pytest`` marked as depending on ``py`` package version >= 1.4.16 which
|
||||
is not Python 2.4 compatible (tested up to and including 1.4.18), so
|
||||
``py`` package version 1.4.15 is used instead.
|
||||
|
||||
* With the described configuration ``pytest``'s startup scripts will not
|
||||
work (as they explicitly check ``pytest``'s package dependencies), but
|
||||
``pytest`` can still be run using::
|
||||
|
||||
py244 -m pytest <regular-pytest-options>
|
||||
|
||||
* When running project tests on Windows using this Python version, the output
|
||||
will contain lots of terminal escape sequences instead of being colored, but
|
||||
otherwise the tests should run without a glitch.
|
||||
|
||||
Python 2.4.3
|
||||
~~~~~~~~~~~~
|
||||
|
||||
* First see more general Python 2.4.x related notes above - list of compatible
|
||||
required package versions, general caveats, etc.
|
||||
* Does not work with HTTPS links so you can not use the Python package index
|
||||
directly, since it, at some point, switched to using HTTPS links only.
|
||||
|
||||
* You could potentially work around this problem by somehow mapping its https:
|
||||
links to http: ones or download its link page manually, locally modify it to
|
||||
contain http: links and then use that download link page instead of the
|
||||
default downloaded one.
|
||||
* An alternative and tested solution is to download the required installation
|
||||
packages locally using Python 2.4.4 and then install them locally into the
|
||||
Python 2.4.3 environment.
|
||||
|
||||
* In the example code below, we name the local installation package storage
|
||||
folder ``target_folder`` for illustration purposes only, with
|
||||
``full_target_folder_path`` representing its full path.
|
||||
|
||||
* First install ``setuptools`` as described under `Installing setuptools on
|
||||
Python 2.4.x & 2.5.x`_.
|
||||
* Then use Python 2.4.4 to download pip & pytest related installation
|
||||
packages::
|
||||
|
||||
py244 -m easy_install --zip-ok --multi-version --always-copy --exclude-scripts --install-dir "target_folder" pip==1.1
|
||||
py244 -c "import pip;pip.main()" install pytest==2.4.1 py==1.4.15 -d "target_folder" --exists-action=i
|
||||
|
||||
* Install ``pip`` from its local installation package (``target_folder``
|
||||
name used in this command must not contain any whitespace characters or
|
||||
may be given as a local ``file:///`` URL consisting of an absolute path,
|
||||
ending with a trailing ``/`` character and with any embedded spaces
|
||||
encoded as ``%20``)::
|
||||
|
||||
py243 -m easy_install -f "target_folder" --allow-hosts=None pip==1.1
|
||||
|
||||
* Install ``pytest`` from its local installation packages (``target_folder``
|
||||
name used in this command must be specified as a local ``file:///`` URL
|
||||
consisting of an absolute path, but without a trailing ``/`` character or
|
||||
any embedded character encoding)::
|
||||
|
||||
py243 -c "import pip;pip.main()" install pytest==2.4.1 py==1.4.15 -f "file:///full_target_folder_path" --no-index
|
||||
|
||||
Python 3.1.x
|
||||
~~~~~~~~~~~~
|
||||
|
||||
* Unfortunately pytest prior to release 2.6 does not support Python versions
|
||||
3.1.x out of the box, but can be patched to do so. See pytest pull request
|
||||
#168 ('https://bitbucket.org/hpk42/pytest/pull-request/168') which has been
|
||||
successfully applied to a pytest 2.5.2 release.
|
||||
|
||||
|
||||
EXTERNAL DOCUMENTATION
|
||||
=================================================
|
||||
|
|
|
|||
|
|
@ -16,5 +16,8 @@ include notes/*.rst
|
|||
include notes/*.txt
|
||||
|
||||
# Tools.
|
||||
prune tools/__* # local cache folders
|
||||
include tools/*.cmd
|
||||
include tools/*.py
|
||||
include tools/*.txt
|
||||
recursive-include tools/suds_devel *.py
|
||||
|
|
|
|||
|
|
@ -285,6 +285,9 @@ version 0.7 (development)
|
|||
taking at least one input parameter no longer causes suds to report an invalid
|
||||
extra argument error.
|
||||
* Improved internal project ``HACKING.rst`` documentation.
|
||||
* Added a script for automatically setting up required development Python
|
||||
environments for this project, hopefully supporting the full range of
|
||||
supported Python versions out of the box.
|
||||
* ``setup.py`` improvements.
|
||||
|
||||
* Python 3.0.x releases explicitly marked as not supported.
|
||||
|
|
|
|||
111
setup.cfg
111
setup.cfg
|
|
@ -1,30 +1,119 @@
|
|||
# ------------------------------------------
|
||||
# --- Build system related configuration ---
|
||||
# ------------------------------------------
|
||||
# ------------------------------------------------------------------------------
|
||||
#
|
||||
# Main project configuration.
|
||||
#
|
||||
# ------------------------------------------------------------------------------
|
||||
# Specific scripts using this configuration may support interpolating values
|
||||
# from other options inside the same section using the '%(option)s' syntax. The
|
||||
# same syntax may also be used to interpolate additional values provided by the
|
||||
# calling script instead of being read from the configuration file itself, e.g.
|
||||
# value '%(__name__)s' always refers to the name of the current section.
|
||||
#
|
||||
# For example, if interpolation is enabled, configuration
|
||||
#
|
||||
# [demo-section]
|
||||
# basic = Foo
|
||||
# derived = %(basic value)s-Bar
|
||||
#
|
||||
# generates the following options located in the section 'demo-section':
|
||||
#
|
||||
# 'basic' with the value 'Foo'
|
||||
# 'derived' with the value 'Foo-Bar'
|
||||
|
||||
|
||||
# --------------------
|
||||
# --- Build system ---
|
||||
# --------------------
|
||||
|
||||
[install]
|
||||
# Added in the original project as a fix (trunk SVN revision 7) for a problem
|
||||
# Added in the original project as a fix (trunk SVN revision 7) for a problem
|
||||
# with missing .pyo files when building rpms with Python 2.4.
|
||||
optimize = 1
|
||||
|
||||
[sdist]
|
||||
# '.tar.bz2' source distribution format takes the least space while '.zip' is
|
||||
# '.tar.bz2' source distribution format takes the least space while '.zip' is
|
||||
# the only format supported out-of-the-box on Windows.
|
||||
formats = bztar,zip
|
||||
|
||||
|
||||
# ----------------------------------
|
||||
# --- Test related configuration ---
|
||||
# ----------------------------------
|
||||
# --------------------------------------
|
||||
# --- Test & development environment ---
|
||||
# --------------------------------------
|
||||
|
||||
# Target Python environments.
|
||||
[env:2.4.4 x86]
|
||||
command = py244
|
||||
[env:2.4.3 x86]
|
||||
command = py243
|
||||
[env:2.5.4 x64]
|
||||
command = py254
|
||||
[env:2.5.4 x86]
|
||||
command = py254_x86
|
||||
[env:2.6.6 x64]
|
||||
command = py266
|
||||
[env:2.6.6 x86]
|
||||
command = py266_x86
|
||||
[env:2.7.6 x64]
|
||||
command = py276
|
||||
[env:2.7.6 x86]
|
||||
command = py276_x86
|
||||
[env:3.1.3 x64]
|
||||
command = py313
|
||||
[env:3.2.5 x64]
|
||||
command = py325
|
||||
[env:3.2.5 x86]
|
||||
command = py325_x86
|
||||
[env:3.3.3 x64]
|
||||
command = py333
|
||||
[env:3.3.3 x86]
|
||||
command = py333_x86
|
||||
[env:3.3.5 x64]
|
||||
command = py335
|
||||
[env:3.3.5 x86]
|
||||
command = py335_x86
|
||||
[env:3.4.0 x64]
|
||||
command = py340
|
||||
[env:3.4.0 x86]
|
||||
command = py340_x86
|
||||
|
||||
[pytest]
|
||||
# Folders 'pytest' unit testing framework should avoid when collecting test
|
||||
# Folders 'pytest' unit testing framework should avoid when collecting test
|
||||
# cases to run, e.g. internal build & version control system folders.
|
||||
norecursedirs = .git .hg .svn build dist
|
||||
|
||||
# Regular test modules have names matching 'test_*.py'. Modules whose names
|
||||
# end with '__pytest_assert_rewrite_needed.py' contain testing utility
|
||||
# Regular test modules have names matching 'test_*.py'. Modules whose names end
|
||||
# with '__pytest_assert_rewrite_needed.py' contain testing utility
|
||||
# implementations that need to be considered as test modules so pytest would
|
||||
# apply its assertion rewriting on them or else they would not work correctly
|
||||
# when tests are run with disabled Python assertions.
|
||||
python_files = test_*.py *__pytest_assert_rewrite_needed.py
|
||||
|
||||
[setup base environments - actions]
|
||||
# Regular actions (True/False).
|
||||
report environment configuration = False
|
||||
report raw environment scan results = False
|
||||
|
||||
# Setup actions (Yes/No/IfNeeded).
|
||||
setup setuptools = IfNeeded
|
||||
download installations = IfNeeded
|
||||
install environments = Yes
|
||||
|
||||
[setup base environments - folders]
|
||||
# Regular relative paths are interpreted relative to the folder containing this
|
||||
# configuration. An alternative base folder may be explicitly specified using
|
||||
# the following syntax:
|
||||
# <BASE-FOLDER>//<path>
|
||||
# Where <path> stands for the specified relative path, while <BASE-FOLDER>
|
||||
# stands for one of the following literals
|
||||
# PROJECT-FOLDER - base project folder
|
||||
# SCRIPT-FOLDER - folder containing the script reading this configuration
|
||||
# INI-FOLDER - folder containing this configuration (default)
|
||||
installation cache = SCRIPT-FOLDER//__down load__
|
||||
pip download cache = %(installation cache)s/__pip download cache__
|
||||
ez_setup folder = PROJECT-FOLDER//.
|
||||
|
||||
[setup base environments - reuse pre-installed setuptools]
|
||||
# Reusing pre-installed setuptools distributions (True/False).
|
||||
old = False
|
||||
best = True
|
||||
future = True
|
||||
|
|
|
|||
|
|
@ -1,6 +1,7 @@
|
|||
Tools & utilities used during suds development.
|
||||
|
||||
Only stand-alone scripts should be located in this folder.
|
||||
Only stand-alone scripts should be located in this folder. Any imported
|
||||
additional modules should be located under the suds_devel package folder.
|
||||
|
||||
All Python scripts under this folder should be prepared so their sources are
|
||||
directly compatible with both Python 2.x & 3.x without the need for any py2to3
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load Diff
|
|
@ -0,0 +1,102 @@
|
|||
# -*- coding: utf-8 -*-
|
||||
|
||||
# This program is free software; you can redistribute it and/or modify it under
|
||||
# the terms of the (LGPL) GNU Lesser General Public License as published by the
|
||||
# Free Software Foundation; either version 3 of the License, or (at your
|
||||
# option) any later version.
|
||||
#
|
||||
# This program is distributed in the hope that it will be useful, but WITHOUT
|
||||
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
|
||||
# FOR A PARTICULAR PURPOSE. See the GNU Library Lesser General Public License
|
||||
# for more details at ( http://www.gnu.org/licenses/lgpl.html ).
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with this program; if not, write to the Free Software Foundation, Inc.,
|
||||
# 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
|
||||
# written by: Jurko Gospodnetić ( jurko.gospodnetic@pke.hr )
|
||||
|
||||
"""
|
||||
Basic configuration support shared in different development utility scripts.
|
||||
|
||||
"""
|
||||
|
||||
import os.path
|
||||
import sys
|
||||
|
||||
if sys.version_info < (3,):
|
||||
import ConfigParser as configparser
|
||||
else:
|
||||
import configparser
|
||||
|
||||
|
||||
class BadConfiguration(Exception):
|
||||
def __init__(self, message):
|
||||
Exception.__init__(self, message)
|
||||
|
||||
|
||||
class Config(object):
|
||||
|
||||
# Typed option values.
|
||||
BOOLEAN_TRUE = ('1', 'yes', 'true', 'on', '+')
|
||||
BOOLEAN_FALSE = ('0', 'no', 'false', 'off', '-')
|
||||
IF_NEEDED = ('maybe', '?',
|
||||
'ifneeded', 'if needed',
|
||||
'asneeded', 'as needed',
|
||||
'ondemand', 'on demand')
|
||||
|
||||
class TriBool:
|
||||
Yes = object()
|
||||
IfNeeded = object()
|
||||
No = object()
|
||||
|
||||
def __init__(self, ini_file):
|
||||
self.__init_reader(ini_file)
|
||||
|
||||
def _get_bool(self, section, option):
|
||||
x = self._reader.get(section, option).lower()
|
||||
if x in self.BOOLEAN_TRUE:
|
||||
return True
|
||||
if x in self.BOOLEAN_FALSE:
|
||||
return False
|
||||
raise BadConfiguration("Option '%s.%s' must be a boolean value." % (
|
||||
section, option))
|
||||
|
||||
def _get_tribool(self, section, option):
|
||||
x = self._reader.get(section, option).lower()
|
||||
if x in self.BOOLEAN_TRUE:
|
||||
return Config.TriBool.Yes
|
||||
if x in self.BOOLEAN_FALSE:
|
||||
return Config.TriBool.No
|
||||
if x in self.IF_NEEDED:
|
||||
return Config.TriBool.IfNeeded
|
||||
raise BadConfiguration("Option '%s.%s' must be Yes, No or IfNeeded." %
|
||||
(section, option))
|
||||
|
||||
def _read_environment_configuration(self):
|
||||
self.python_environments = []
|
||||
for x in self._reader.sections():
|
||||
if x.lower().startswith("env:"):
|
||||
command = self._reader.get(x, "command")
|
||||
if not command:
|
||||
raise BadConfiguration("'%s.%s' environment command "
|
||||
"configuration option must not be empty." % (section,
|
||||
option))
|
||||
self.python_environments.append(command)
|
||||
|
||||
def __init_reader(self, ini_file):
|
||||
if not os.path.isfile(ini_file):
|
||||
raise BadConfiguration("Missing configuration file '%s'." % (
|
||||
ini_file))
|
||||
try:
|
||||
f = open(ini_file, "r")
|
||||
except (KeyboardInterrupt, SystemExit):
|
||||
raise
|
||||
except Exception:
|
||||
raise BadConfiguration("Can not access configuration file '%s' - "
|
||||
"%s." % (ini_file, sys.exc_info()[1]))
|
||||
try:
|
||||
print("Reading configuration file '%s'..." % (ini_file,))
|
||||
self._reader = configparser.ConfigParser()
|
||||
self._reader.readfp(f)
|
||||
finally:
|
||||
f.close()
|
||||
|
|
@ -0,0 +1,175 @@
|
|||
# -*- coding: utf-8 -*-
|
||||
|
||||
# This program is free software; you can redistribute it and/or modify it under
|
||||
# the terms of the (LGPL) GNU Lesser General Public License as published by the
|
||||
# Free Software Foundation; either version 3 of the License, or (at your
|
||||
# option) any later version.
|
||||
#
|
||||
# This program is distributed in the hope that it will be useful, but WITHOUT
|
||||
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
|
||||
# FOR A PARTICULAR PURPOSE. See the GNU Library Lesser General Public License
|
||||
# for more details at ( http://www.gnu.org/licenses/lgpl.html ).
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with this program; if not, write to the Free Software Foundation, Inc.,
|
||||
# 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
|
||||
# written by: Jurko Gospodnetić ( jurko.gospodnetic@pke.hr )
|
||||
|
||||
"""
|
||||
Manipulating egg distributions.
|
||||
|
||||
"""
|
||||
|
||||
import os
|
||||
import os.path
|
||||
|
||||
from suds_devel.zip import zip_folder_content
|
||||
|
||||
|
||||
def zip_eggs_in_folder(folder):
|
||||
"""
|
||||
Make all egg distributions in the given folder be stored as egg files.
|
||||
|
||||
In case setuptools downloads one of its target packages as an unzipped egg
|
||||
folder (e.g. if installed from an already installed unzipped egg), we need
|
||||
to zip it ourselves. This is because there seems to be no way to make
|
||||
setuptools perform an installation from a local unzipped egg folder.
|
||||
Specifying either that folder or its parent folder as a setuptools
|
||||
find-links URL just makes the folder be treated as a regular non-egg
|
||||
folder.
|
||||
|
||||
"""
|
||||
eggs = _detect_eggs_in_folder(folder)
|
||||
for egg in eggs:
|
||||
egg.normalize()
|
||||
|
||||
|
||||
# Egg distribution related file & folder name extensions.
|
||||
_egg_ext = os.extsep + "egg"
|
||||
_zip_ext = _egg_ext + os.extsep + "zip"
|
||||
|
||||
|
||||
class _Egg:
|
||||
"""
|
||||
Represents a single egg distribution.
|
||||
|
||||
Helps track & manage formats the distribution is stored in:
|
||||
- zipped egg file with .egg extension
|
||||
- zipped egg file with .egg.zip extension
|
||||
- unzipped egg folder
|
||||
|
||||
"""
|
||||
|
||||
# Indicators whether the egg distribution has a '.egg' file or folder.
|
||||
NONE = object()
|
||||
FILE = object()
|
||||
FOLDER = object()
|
||||
|
||||
def __init__(self, path, egg, zip):
|
||||
assert egg in (_Egg.NONE, _Egg.FILE, _Egg.FOLDER)
|
||||
assert zip.__class__ is bool
|
||||
assert zip or egg is not _Egg.NONE
|
||||
self.__path = path
|
||||
self.__egg = egg
|
||||
self.__zip = zip
|
||||
|
||||
def has_egg_file(self):
|
||||
return self.__egg is _Egg.FILE
|
||||
|
||||
def has_egg_folder(self):
|
||||
return self.__egg is _Egg.FOLDER
|
||||
|
||||
def has_zip(self):
|
||||
return self.__zip
|
||||
|
||||
def normalize(self):
|
||||
"""
|
||||
Makes sure this egg distribution is stored only as an egg file.
|
||||
|
||||
The egg file will be created from another existing distribution format
|
||||
if needed.
|
||||
|
||||
"""
|
||||
if self.has_egg_file():
|
||||
if self.has_zip():
|
||||
self.__remove_zip()
|
||||
else:
|
||||
if self.has_egg_folder():
|
||||
if not self.has_zip():
|
||||
self.__zip_egg_folder()
|
||||
self.__remove_egg_folder()
|
||||
self.__rename_zip_to_egg()
|
||||
|
||||
def set_egg(self, egg):
|
||||
assert egg in (_Egg.FILE, _Egg.FOLDER)
|
||||
assert self.__egg is _Egg.NONE
|
||||
self.__egg = egg
|
||||
|
||||
def set_zip(self):
|
||||
assert not self.__zip
|
||||
self.__zip = True
|
||||
|
||||
def __path_egg(self):
|
||||
return self.__path + _egg_ext
|
||||
|
||||
def __path_zip(self):
|
||||
return self.__path + _zip_ext
|
||||
|
||||
def __remove_egg_folder(self):
|
||||
assert self.has_egg_folder()
|
||||
import shutil
|
||||
shutil.rmtree(self.__path_egg())
|
||||
self.__egg = _Egg.NONE
|
||||
|
||||
def __remove_zip(self):
|
||||
assert self.has_zip()
|
||||
os.remove(self.__path_zip())
|
||||
self.__zip = False
|
||||
|
||||
def __rename_zip_to_egg(self):
|
||||
assert self.has_zip()
|
||||
assert not self.has_egg_file()
|
||||
assert not self.has_egg_folder()
|
||||
os.rename(self.__path_zip(), self.__path_egg())
|
||||
self.__egg = _Egg.FILE
|
||||
self.__zip = False
|
||||
|
||||
def __zip_egg_folder(self):
|
||||
assert self.has_egg_folder()
|
||||
assert not self.has_zip()
|
||||
zip_folder_content(self.__path_egg(), self.__path_zip())
|
||||
self.__zip = True
|
||||
|
||||
|
||||
def _detect_eggs_in_folder(folder):
|
||||
"""
|
||||
Detect egg distributions located in the given folder.
|
||||
|
||||
Only direct folder content is considered and subfolders are not searched
|
||||
recursively.
|
||||
|
||||
"""
|
||||
eggs = {}
|
||||
for x in os.listdir(folder):
|
||||
zip = x.endswith(_zip_ext)
|
||||
if zip:
|
||||
root = x[:-len(_zip_ext)]
|
||||
egg = _Egg.NONE
|
||||
elif x.endswith(_egg_ext):
|
||||
root = x[:-len(_egg_ext)]
|
||||
if os.path.isdir(os.path.join(folder, x)):
|
||||
egg = _Egg.FOLDER
|
||||
else:
|
||||
egg = _Egg.FILE
|
||||
else:
|
||||
continue
|
||||
try:
|
||||
info = eggs[root]
|
||||
except KeyError:
|
||||
eggs[root] = _Egg(os.path.join(folder, root), egg, zip)
|
||||
else:
|
||||
if egg is not _Egg.NONE:
|
||||
info.set_egg(egg)
|
||||
if zip:
|
||||
info.set_zip()
|
||||
return eggs.values()
|
||||
|
|
@ -0,0 +1,429 @@
|
|||
# -*- coding: utf-8 -*-
|
||||
|
||||
# This program is free software; you can redistribute it and/or modify it under
|
||||
# the terms of the (LGPL) GNU Lesser General Public License as published by the
|
||||
# Free Software Foundation; either version 3 of the License, or (at your
|
||||
# option) any later version.
|
||||
#
|
||||
# This program is distributed in the hope that it will be useful, but WITHOUT
|
||||
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
|
||||
# FOR A PARTICULAR PURPOSE. See the GNU Library Lesser General Public License
|
||||
# for more details at ( http://www.gnu.org/licenses/lgpl.html ).
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with this program; if not, write to the Free Software Foundation, Inc.,
|
||||
# 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
|
||||
# written by: Jurko Gospodnetić ( jurko.gospodnetic@pke.hr )
|
||||
|
||||
"""
|
||||
Class representing a single Python environment.
|
||||
|
||||
Includes support for:
|
||||
- fetching information about a specific Python environment
|
||||
- executing a command in a specific Python environment
|
||||
|
||||
"""
|
||||
|
||||
import sys
|
||||
if sys.version_info < (3, 3):
|
||||
import os
|
||||
import subprocess
|
||||
|
||||
|
||||
class BadEnvironment(Exception):
|
||||
"""
|
||||
Problem occurred while scanning a Python environment.
|
||||
|
||||
The problem may be either a technical one in the scanning process itself,
|
||||
or it may be a problem with something detected about the Python environment
|
||||
in question, e.g. it might be using an incompatible Python version.
|
||||
|
||||
Specifying the environment scan result information when constructing this
|
||||
exception is optional. This information may be added to an exception later
|
||||
on using the set() method.
|
||||
|
||||
"""
|
||||
|
||||
def __init__(self, message, out=None, err=None, exit_code=None):
|
||||
Exception.__init__(self, message)
|
||||
self.message = message
|
||||
self.out = out
|
||||
self.err = err
|
||||
self.exit_code = exit_code
|
||||
|
||||
def raw_initial_scan_results(self):
|
||||
return self.out, self.err, self.exit_code
|
||||
|
||||
def set(self, out, err, exit_code):
|
||||
assert self.out is None
|
||||
assert self.err is None
|
||||
assert self.exit_code is None
|
||||
self.out = out
|
||||
self.err = err
|
||||
self.exit_code = exit_code
|
||||
|
||||
|
||||
class _UndefinedParameter:
|
||||
"""Internal class used to indicate undefined parameter values."""
|
||||
pass
|
||||
|
||||
|
||||
class Environment:
|
||||
"""
|
||||
Represents a single Python environment.
|
||||
|
||||
Automatically scans the given Python environment and provides information
|
||||
about it.
|
||||
|
||||
Allows running commands using the environment's Python interpreter.
|
||||
|
||||
"""
|
||||
|
||||
def __init__(self, name):
|
||||
self.__name = name
|
||||
self.__command = "%s.cmd" % (name,)
|
||||
self.__raw_initial_scan_results = self.__initial_scan()
|
||||
|
||||
def command(self):
|
||||
return self.__command
|
||||
|
||||
def description(self):
|
||||
return "%s on %s" % (self.sys_version, self.sys_platform)
|
||||
|
||||
def execute(self, args=[], input=None, capture_output=False,
|
||||
cwd=_UndefinedParameter):
|
||||
stdin = subprocess.PIPE
|
||||
close_stdin = False
|
||||
if input is None:
|
||||
if sys.version_info >= (3, 3):
|
||||
stdin = subprocess.DEVNULL
|
||||
else:
|
||||
stdin = open(os.devnull, "r")
|
||||
close_stdin = True
|
||||
try:
|
||||
kwargs = {}
|
||||
kwargs["stdin"] = stdin
|
||||
if capture_output:
|
||||
kwargs["stdout"] = subprocess.PIPE
|
||||
kwargs["stderr"] = subprocess.PIPE
|
||||
if cwd is not _UndefinedParameter:
|
||||
kwargs["cwd"] = cwd
|
||||
kwargs["universal_newlines"] = True
|
||||
popen = subprocess.Popen([self.command()] + args, **kwargs)
|
||||
out, err = popen.communicate(input)
|
||||
finally:
|
||||
if close_stdin:
|
||||
stdin.close()
|
||||
return out, err, popen.returncode
|
||||
|
||||
def name(self):
|
||||
return self.__name
|
||||
|
||||
def raw_initial_scan_results(self):
|
||||
return self.__raw_initial_scan_results
|
||||
|
||||
def __check_python_version(self):
|
||||
if self.sys_version_info < (2, 4):
|
||||
raise BadEnvironment("Unsupported Python version (%s, %s-bit)"
|
||||
% (self.python_version, self.pointer_size_in_bits))
|
||||
|
||||
def __collect_scanned_values(self, values):
|
||||
v = values
|
||||
# System.
|
||||
self.platform_architecture = v["platform.architecture"]
|
||||
self.pointer_size_in_bits = v["pointer size in bits"]
|
||||
self.sys_path = v["sys.path"]
|
||||
self.sys_version = v["sys.version"]
|
||||
self.sys_version_info_formatted = v["sys.version_info (formatted)"]
|
||||
self.sys_version_info_raw = v["sys.version_info (raw)"]
|
||||
self.sys_platform = v["sys.platform"]
|
||||
self.sys_executable = v["sys.executable"]
|
||||
# Packages.
|
||||
self.ctypes_version = v["ctypes version"]
|
||||
self.pip_version = v["pip version"]
|
||||
self.pytest_version = v["pytest version"]
|
||||
self.setuptools_version = v["setuptools version"]
|
||||
self.virtualenv_version = v["virtualenv version"]
|
||||
# Extra package info.
|
||||
self.setuptools_zipped_egg = v["setuptools zipped egg"]
|
||||
|
||||
def __construct_python_version(self):
|
||||
"""
|
||||
Construct a setuptools compatible Python version string.
|
||||
|
||||
Constructed based on the environment's reported sys.version_info.
|
||||
|
||||
"""
|
||||
major, minor, micro, release_level, serial = self.sys_version_info
|
||||
assert release_level in ("alfa", "beta", "candidate", "final")
|
||||
assert release_level != "final" or serial == 0
|
||||
parts = [str(major), ".", str(minor), ".", str(micro)]
|
||||
if release_level != "final":
|
||||
parts.append(release_level[0])
|
||||
parts.append(str(serial))
|
||||
self.python_version = "".join(parts)
|
||||
|
||||
def __initial_scan(self):
|
||||
scanner = self.__initial_scanner()
|
||||
try:
|
||||
scan_results = scanner.scan(self)
|
||||
self.__collect_scanned_values(scan_results)
|
||||
self.__parse_scanned_version_info()
|
||||
self.__check_python_version()
|
||||
except BadEnvironment:
|
||||
sys.exc_info()[1].set(*scanner.raw_scan_results())
|
||||
raise
|
||||
return scanner.raw_scan_results()
|
||||
|
||||
@staticmethod
|
||||
def __initial_scanner():
|
||||
s = EnvironmentScanner()
|
||||
|
||||
s.add_import("os.path")
|
||||
s.add_import("platform")
|
||||
s.add_import("struct")
|
||||
s.add_import("sys")
|
||||
|
||||
s.add_function("""\
|
||||
def version_info_string():
|
||||
major, minor, micro, release_level, serial = sys.version_info
|
||||
if not (isinstance(major, int) and isinstance(minor, int) and
|
||||
isinstance(micro, int) and isinstance(release_level, str) and
|
||||
isinstance(serial, int)) or "," in release_level:
|
||||
return ""
|
||||
# At least Python versions 2.7.6 & 3.1.3 require an explicit tuple() cast.
|
||||
return "%d,%d,%d,%s,%d" % tuple(sys.version_info)
|
||||
""")
|
||||
s.add_function("""\
|
||||
def setuptools_zipped_egg():
|
||||
try:
|
||||
from pkg_resources import (
|
||||
DistributionNotFound, EGG_DIST, get_distribution)
|
||||
except ImportError:
|
||||
return Skip
|
||||
try:
|
||||
d = get_distribution("setuptools")
|
||||
except DistributionNotFound:
|
||||
return Skip
|
||||
if d.precedence != EGG_DIST or not os.path.isfile(d.location):
|
||||
return Skip # file = zipped egg; folder = unzipped egg
|
||||
return d.location
|
||||
""")
|
||||
|
||||
s.add_field("platform.architecture", "platform.architecture()")
|
||||
s.add_field("pointer size in bits", '8 * struct.calcsize("P")')
|
||||
s.add_field("sys.executable")
|
||||
s.add_field("sys.path")
|
||||
s.add_field("sys.platform")
|
||||
s.add_field("sys.version")
|
||||
s.add_field("sys.version_info (formatted)", "version_info_string()")
|
||||
s.add_field("sys.version_info (raw)", "sys.version_info")
|
||||
|
||||
s.add_package_version_field("ctypes", default=None)
|
||||
s.add_package_version_field("pip", default=None)
|
||||
s.add_package_version_field("pytest", default=None)
|
||||
s.add_package_version_field("setuptools", default=None)
|
||||
s.add_package_version_field("virtualenv", default=None)
|
||||
|
||||
s.add_field("setuptools zipped egg", "setuptools_zipped_egg()", None)
|
||||
return s
|
||||
|
||||
def __parse_scanned_version_info(self):
|
||||
"""Parses the environment's formatted version info string."""
|
||||
string = self.sys_version_info_formatted
|
||||
try:
|
||||
major, minor, micro, release_level, serial = string.split(",")
|
||||
if (release_level in ("alfa", "beta", "candidate", "final") and
|
||||
(release_level != "final" or serial == "0") and
|
||||
major.isdigit() and # --- --- --- --- --- --- --- --- ---
|
||||
minor.isdigit() and # Explicit isdigit() checks to detect
|
||||
micro.isdigit() and # leading/trailing whitespace.
|
||||
serial.isdigit()): # --- --- --- --- --- --- --- --- ---
|
||||
self.sys_version_info = (int(major), int(minor), int(micro),
|
||||
release_level, int(serial))
|
||||
self.__construct_python_version()
|
||||
return
|
||||
except (KeyboardInterrupt, SystemExit):
|
||||
raise
|
||||
except Exception:
|
||||
pass
|
||||
raise BadEnvironment("Unsupported Python version (%s)" % (string,))
|
||||
|
||||
|
||||
class EnvironmentScanner:
|
||||
"""
|
||||
Allows scanning a given Python environment for specific information.
|
||||
|
||||
Runs a scanner script in the given Python environment, crafted based on the
|
||||
information we wish to find out about the environment, and then collects
|
||||
and returns the requested information.
|
||||
|
||||
"""
|
||||
|
||||
# Explicitly specified scan output start & finish markers allow us to
|
||||
# safely ignore extra output that might be added by the environment's
|
||||
# Python interpreter startup. This can be useful, for instance, if you want
|
||||
# to debug this script by having the Python interpreter startup script
|
||||
# output the exact calls made to the Python interpreter.
|
||||
SCAN_START_MARKER = "--- SCAN START ---"
|
||||
SCAN_FINISH_MARKER = "--- SCAN FINISH ---"
|
||||
|
||||
__scanner_script__startup = """\
|
||||
class Skip:
|
||||
pass
|
||||
|
||||
def print_field(id, value):
|
||||
if value is not Skip:
|
||||
print("%s: %s" % (id, value))
|
||||
"""
|
||||
|
||||
# If available, setuptools can give us more detailed version information
|
||||
# read from the package's meta-data than simply reading it from its
|
||||
# __version__ attribute. For example, in case of setuptools it can tell us
|
||||
# that we are dealing with version '3.7dev' while __version__ would tell us
|
||||
# just '3.7'. Also, some older packages, such as pip 1.1, may not have a
|
||||
# __version__ attribute set in their main module at all.
|
||||
__scanner_script__package_version_scanner = """\
|
||||
def package_version(package_name):
|
||||
try:
|
||||
package = __import__(package_name, {}, {}, ("__version__",))
|
||||
except (KeyboardInterrupt, SystemExit):
|
||||
raise
|
||||
except Exception:
|
||||
return Skip # No package - no version.
|
||||
try:
|
||||
version = package.__version__
|
||||
except AttributeError:
|
||||
version = "unknown"
|
||||
try:
|
||||
from pkg_resources import (DistributionNotFound, get_distribution)
|
||||
except ImportError:
|
||||
return version
|
||||
try:
|
||||
return get_distribution(package_name).version
|
||||
except DistributionNotFound:
|
||||
return version
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
self.__out = None
|
||||
self.__err = None
|
||||
self.__exit_code = None
|
||||
self.__scan_for_version_info = False
|
||||
self.__extra_script_functions = []
|
||||
self.__extra_script_imports = []
|
||||
self.__fields = []
|
||||
|
||||
def add_field(self, id, getter=None, default=_UndefinedParameter):
|
||||
if getter is None:
|
||||
getter = id
|
||||
self.__add_field(id, "print_field(%r, %s)" % (id, getter), default)
|
||||
|
||||
def add_function(self, code):
|
||||
self.__extra_script_functions.append(code)
|
||||
|
||||
def add_import(self, module_name):
|
||||
self.__extra_script_imports.append(module_name)
|
||||
|
||||
def add_package_version_field(self, package_name, default=_UndefinedParameter):
|
||||
self.__scan_for_version_info = True
|
||||
field_id = "%s version" % (package_name,)
|
||||
field_getter = "print_field(%r, package_version(%r))" % (
|
||||
field_id, package_name)
|
||||
self.__add_field(field_id, field_getter, default)
|
||||
|
||||
def raw_scan_results(self):
|
||||
return self.__out, self.__err, self.__exit_code
|
||||
|
||||
def scan(self, environment):
|
||||
self.__scan(environment)
|
||||
raw_data = self.__parse_raw_scanner_output()
|
||||
return self.__map_raw_data_to_expected_fields(raw_data)
|
||||
|
||||
def __add_field(self, id, getter, default):
|
||||
assert id not in [x[0] for x in self.__fields]
|
||||
self.__fields.append((id, getter, default))
|
||||
|
||||
@staticmethod
|
||||
def __extract_value(raw_data, id, default):
|
||||
if default is not _UndefinedParameter:
|
||||
return raw_data.pop(id, default)
|
||||
try:
|
||||
return raw_data.pop(id)
|
||||
except KeyError:
|
||||
raise BadEnvironment("Missing scan output record (%s)" % (
|
||||
sys.exc_info()[1],))
|
||||
|
||||
def __map_raw_data_to_expected_fields(self, raw_data):
|
||||
scanner_results = {}
|
||||
for id, getter, default in self.__fields:
|
||||
assert id not in scanner_results
|
||||
scanner_results[id] = self.__extract_value(raw_data, id, default)
|
||||
if raw_data:
|
||||
raise BadEnvironment("Extra scan output records (%s)" % (
|
||||
",".join(raw_data.keys()),))
|
||||
return scanner_results
|
||||
|
||||
def __parse_raw_scanner_output(self):
|
||||
assert self.__scanned()
|
||||
result = {}
|
||||
in_scanner_output = False
|
||||
for line in self.__out.split("\n"):
|
||||
if not in_scanner_output:
|
||||
in_scanner_output = line.startswith(self.SCAN_START_MARKER)
|
||||
continue
|
||||
if line.startswith(self.SCAN_FINISH_MARKER):
|
||||
return result
|
||||
split_result = line.split(": ", 1)
|
||||
if len(split_result) != 2:
|
||||
raise BadEnvironment("Error parsing scan output record")
|
||||
key, value = split_result
|
||||
if key in result:
|
||||
raise BadEnvironment("Duplicate scan output record (%s)" %
|
||||
(key,))
|
||||
result[key] = value
|
||||
if not in_scanner_output:
|
||||
raise BadEnvironment("No valid scan output detected")
|
||||
raise BadEnvironment("Scan output truncated")
|
||||
|
||||
def __scan(self, environment):
|
||||
assert not self.__scanned()
|
||||
try:
|
||||
self.__out, self.__err, self.__exit_code = environment.execute(
|
||||
input=self.__scanner_script(), capture_output=True)
|
||||
except (KeyboardInterrupt, SystemExit):
|
||||
raise
|
||||
except Exception:
|
||||
e_type, e = sys.exc_info()[:2]
|
||||
try:
|
||||
raise BadEnvironment("%s: %s" % (e_type.__name__, e))
|
||||
finally:
|
||||
del e # break Python3 e->frame->..->frame->e reference cycle
|
||||
if self.__exit_code != 0:
|
||||
raise BadEnvironment("Scan failed (exit code %d)" %
|
||||
(self.__exit_code,))
|
||||
if self.__err:
|
||||
raise BadEnvironment("Scan failed (error output detected)")
|
||||
if not self.__out:
|
||||
raise BadEnvironment("Scan failed (no output)")
|
||||
|
||||
def __scanned(self):
|
||||
return self.__exit_code is not None
|
||||
|
||||
def __scanner_script(self):
|
||||
script_lines = []
|
||||
if self.__extra_script_imports:
|
||||
for x in self.__extra_script_imports:
|
||||
script_lines.append("import %s" % (x,))
|
||||
script_lines.append("")
|
||||
script_lines.append(self.__scanner_script__startup)
|
||||
script_lines.append("")
|
||||
if self.__scan_for_version_info:
|
||||
script_lines.append(self.__scanner_script__package_version_scanner)
|
||||
script_lines.extend(self.__extra_script_functions)
|
||||
script_lines.append("")
|
||||
script_lines.append('print("%s")' % (self.SCAN_START_MARKER,))
|
||||
script_lines.extend(getter for id, getter, default in self.__fields)
|
||||
script_lines.append('print("%s")' % (self.SCAN_FINISH_MARKER,))
|
||||
script_lines.append("")
|
||||
return "\n".join(script_lines)
|
||||
|
|
@ -0,0 +1,32 @@
|
|||
# -*- coding: utf-8 -*-
|
||||
|
||||
# This program is free software; you can redistribute it and/or modify it under
|
||||
# the terms of the (LGPL) GNU Lesser General Public License as published by the
|
||||
# Free Software Foundation; either version 3 of the License, or (at your
|
||||
# option) any later version.
|
||||
#
|
||||
# This program is distributed in the hope that it will be useful, but WITHOUT
|
||||
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
|
||||
# FOR A PARTICULAR PURPOSE. See the GNU Library Lesser General Public License
|
||||
# for more details at ( http://www.gnu.org/licenses/lgpl.html ).
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with this program; if not, write to the Free Software Foundation, Inc.,
|
||||
# 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
|
||||
# written by: Jurko Gospodnetić ( jurko.gospodnetic@pke.hr )
|
||||
|
||||
"""
|
||||
Generic exception classes shared in different development utility modules.
|
||||
|
||||
"""
|
||||
|
||||
class EnvironmentSetupError(Exception):
|
||||
"""
|
||||
Signal to move onto setting up the next environment.
|
||||
|
||||
Raised after the current environment's setup fails. Error message stored in
|
||||
the exception is expected to be reported to the user.
|
||||
|
||||
"""
|
||||
def __init__(self, message):
|
||||
Exception.__init__(self, message)
|
||||
|
|
@ -0,0 +1,92 @@
|
|||
# -*- coding: utf-8 -*-
|
||||
|
||||
# This program is free software; you can redistribute it and/or modify it under
|
||||
# the terms of the (LGPL) GNU Lesser General Public License as published by the
|
||||
# Free Software Foundation; either version 3 of the License, or (at your
|
||||
# option) any later version.
|
||||
#
|
||||
# This program is distributed in the hope that it will be useful, but WITHOUT
|
||||
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
|
||||
# FOR A PARTICULAR PURPOSE. See the GNU Library Lesser General Public License
|
||||
# for more details at ( http://www.gnu.org/licenses/lgpl.html ).
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with this program; if not, write to the Free Software Foundation, Inc.,
|
||||
# 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
|
||||
# written by: Jurko Gospodnetić ( jurko.gospodnetic@pke.hr )
|
||||
|
||||
"""
|
||||
Utility for converting version strings into a directly comparable tuple.
|
||||
|
||||
Shamelessly stolen from setuptools 3.7 (development version) where this code is
|
||||
located in its pkg_resources.py module. The code has not been modified in any
|
||||
way, including comments and coding style, so that it would be as easy as
|
||||
possible to upgrade it to a newer version if that becomes needed.
|
||||
|
||||
Having this functionality in our project allows our tool scripts to run even
|
||||
without locally installed setuptools installed, e.g. the script for setting up
|
||||
the base development environments.
|
||||
|
||||
"""
|
||||
|
||||
import re
|
||||
|
||||
component_re = re.compile(r'(\d+ | [a-z]+ | \.| -)', re.VERBOSE)
|
||||
replace = {'pre':'c', 'preview':'c','-':'final-','rc':'c','dev':'@'}.get
|
||||
|
||||
def _parse_version_parts(s):
|
||||
for part in component_re.split(s):
|
||||
part = replace(part, part)
|
||||
if not part or part=='.':
|
||||
continue
|
||||
if part[:1] in '0123456789':
|
||||
# pad for numeric comparison
|
||||
yield part.zfill(8)
|
||||
else:
|
||||
yield '*'+part
|
||||
|
||||
# ensure that alpha/beta/candidate are before final
|
||||
yield '*final'
|
||||
|
||||
def parse_version(s):
|
||||
"""Convert a version string to a chronologically-sortable key
|
||||
|
||||
This is a rough cross between distutils' StrictVersion and LooseVersion;
|
||||
if you give it versions that would work with StrictVersion, then it behaves
|
||||
the same; otherwise it acts like a slightly-smarter LooseVersion. It is
|
||||
*possible* to create pathological version coding schemes that will fool
|
||||
this parser, but they should be very rare in practice.
|
||||
|
||||
The returned value will be a tuple of strings. Numeric portions of the
|
||||
version are padded to 8 digits so they will compare numerically, but
|
||||
without relying on how numbers compare relative to strings. Dots are
|
||||
dropped, but dashes are retained. Trailing zeros between alpha segments
|
||||
or dashes are suppressed, so that e.g. "2.4.0" is considered the same as
|
||||
"2.4". Alphanumeric parts are lower-cased.
|
||||
|
||||
The algorithm assumes that strings like "-" and any alpha string that
|
||||
alphabetically follows "final" represents a "patch level". So, "2.4-1"
|
||||
is assumed to be a branch or patch of "2.4", and therefore "2.4.1" is
|
||||
considered newer than "2.4-1", which in turn is newer than "2.4".
|
||||
|
||||
Strings like "a", "b", "c", "alpha", "beta", "candidate" and so on (that
|
||||
come before "final" alphabetically) are assumed to be pre-release versions,
|
||||
so that the version "2.4" is considered newer than "2.4a1".
|
||||
|
||||
Finally, to handle miscellaneous cases, the strings "pre", "preview", and
|
||||
"rc" are treated as if they were "c", i.e. as though they were release
|
||||
candidates, and therefore are not as new as a version string that does not
|
||||
contain them, and "dev" is replaced with an '@' so that it sorts lower than
|
||||
than any other pre-release tag.
|
||||
"""
|
||||
parts = []
|
||||
for part in _parse_version_parts(s.lower()):
|
||||
if part.startswith('*'):
|
||||
# remove '-' before a prerelease tag
|
||||
if part<'*final':
|
||||
while parts and parts[-1]=='*final-': parts.pop()
|
||||
# remove trailing zeros from each series of numeric parts
|
||||
while parts and parts[-1]=='00000000':
|
||||
parts.pop()
|
||||
parts.append(part)
|
||||
return tuple(parts)
|
||||
|
|
@ -0,0 +1,143 @@
|
|||
# -*- coding: utf-8 -*-
|
||||
|
||||
# This program is free software; you can redistribute it and/or modify it under
|
||||
# the terms of the (LGPL) GNU Lesser General Public License as published by the
|
||||
# Free Software Foundation; either version 3 of the License, or (at your
|
||||
# option) any later version.
|
||||
#
|
||||
# This program is distributed in the hope that it will be useful, but WITHOUT
|
||||
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
|
||||
# FOR A PARTICULAR PURPOSE. See the GNU Library Lesser General Public License
|
||||
# for more details at ( http://www.gnu.org/licenses/lgpl.html ).
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with this program; if not, write to the Free Software Foundation, Inc.,
|
||||
# 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
|
||||
# written by: Jurko Gospodnetić ( jurko.gospodnetic@pke.hr )
|
||||
|
||||
"""
|
||||
Support for patching an existing pytest installation to make it work correctly
|
||||
in a Python 3.1 environment.
|
||||
|
||||
pytest versions prior to its 2.6 release do not support Python 3.1 out of the
|
||||
box, but can be patched to do so. Basic gist of this patch is to replace a
|
||||
single call to the builtin function 'callable()' in pytest's _pytest/runner.py
|
||||
module with a call to 'py.builtin.callable()'. The patch has been tested to
|
||||
make pytest 2.5.2 work correctly with Python 3.1 and is based on code found in
|
||||
pytest pull request #168 & commit 04c4997da865344f2ebb8569c73c51c57cd4ba05.
|
||||
|
||||
"""
|
||||
|
||||
import os
|
||||
import os.path
|
||||
import re
|
||||
import sys
|
||||
|
||||
from suds_devel.environment import EnvironmentScanner
|
||||
from suds_devel.exception import EnvironmentSetupError
|
||||
from suds_devel.parse_version import parse_version
|
||||
|
||||
|
||||
def patch(env):
|
||||
assert env.sys_version_info[:2] == (3, 1)
|
||||
pytest_location, pytest_version = _scan(env)
|
||||
if parse_version(pytest_version) >= parse_version("2.6.0"):
|
||||
return
|
||||
print("Patching installed pytest package...")
|
||||
file_path = os.path.join(pytest_location, "_pytest", "runner.py")
|
||||
file_temp_path = _file_temp_path(file_path)
|
||||
try:
|
||||
try:
|
||||
prepatched_count, patched_count = _patch(file_path, file_temp_path,
|
||||
unpatched_regex="(.*\s)callable([(].*)$",
|
||||
patched_regex="(.*\s)py[.]builtin[.]callable([(].*)",
|
||||
patch_pattern="%spy.builtin.callable%s")
|
||||
if (prepatched_count, patched_count) not in ((1, 0), (0, 1)):
|
||||
_error(file_path, "file content not recognized")
|
||||
if prepatched_count:
|
||||
print("WARNING: pytest already patched")
|
||||
else:
|
||||
os.remove(file_path)
|
||||
os.rename(file_temp_path, file_path)
|
||||
finally:
|
||||
_try_remove_file(file_temp_path)
|
||||
except (EnvironmentSetupError, KeyboardInterrupt, SystemExit):
|
||||
raise
|
||||
except Exception:
|
||||
_error(file_path, str(sys.exc_info()[1]))
|
||||
|
||||
|
||||
def _error(file_path, message):
|
||||
raise EnvironmentSetupError("can not patch pytest module '%s' - %s" % (
|
||||
file_path, message))
|
||||
|
||||
|
||||
def _file_temp_path(file_path):
|
||||
for i in range(100):
|
||||
temp_path = "%s.patching.%d.tmp" % (file_path, i)
|
||||
if not os.path.exists(temp_path):
|
||||
return temp_path
|
||||
_error(file_path, "can not find available temp file name")
|
||||
|
||||
|
||||
def _patch(source, dest, unpatched_regex, patched_regex, patch_pattern):
|
||||
"""
|
||||
Patch given source file into the given destination file.
|
||||
|
||||
Patching is done line by line. Given un-patched and patched line detection
|
||||
regular expressions are expected never to match the same line.
|
||||
|
||||
Does not modify the source file in any way.
|
||||
|
||||
Returns the number of detected pre-patched lines, and a number of newly
|
||||
patched lines.
|
||||
|
||||
"""
|
||||
prepatched_count = 0
|
||||
patched_count = 0
|
||||
unpatched_matcher = re.compile(unpatched_regex, re.DOTALL)
|
||||
patched_matcher = re.compile(patched_regex)
|
||||
f_in = None
|
||||
f_out = None
|
||||
try:
|
||||
f_in = open(source, "r")
|
||||
f_out = open(dest, mode="w")
|
||||
for line in f_in:
|
||||
match = unpatched_matcher.match(line)
|
||||
if match:
|
||||
assert not patched_matcher.match(line)
|
||||
patched_count += 1
|
||||
line = patch_pattern % match.groups()
|
||||
assert patched_matcher.match(line)
|
||||
elif patched_matcher.match(line):
|
||||
prepatched_count += 1
|
||||
f_out.write(line)
|
||||
finally:
|
||||
if f_in:
|
||||
f_in.close()
|
||||
if f_out:
|
||||
f_out.close()
|
||||
return prepatched_count, patched_count
|
||||
|
||||
|
||||
def _scan(env):
|
||||
"""Scan the given Python environment's pytest installation."""
|
||||
s = EnvironmentScanner()
|
||||
s.add_function("""\
|
||||
def get_pytest_location():
|
||||
from pkg_resources import get_distribution
|
||||
return get_distribution("pytest").location
|
||||
""")
|
||||
s.add_field("pytest location", "get_pytest_location()")
|
||||
s.add_package_version_field("pytest", default="")
|
||||
scan_results = s.scan(env)
|
||||
return scan_results["pytest location"], scan_results["pytest version"]
|
||||
|
||||
|
||||
def _try_remove_file(path):
|
||||
try:
|
||||
os.remove(path)
|
||||
except (KeyboardInterrupt, SystemExit):
|
||||
raise
|
||||
except Exception:
|
||||
pass
|
||||
|
|
@ -0,0 +1,104 @@
|
|||
# -*- coding: utf-8 -*-
|
||||
|
||||
# This program is free software; you can redistribute it and/or modify it under
|
||||
# the terms of the (LGPL) GNU Lesser General Public License as published by the
|
||||
# Free Software Foundation; either version 3 of the License, or (at your
|
||||
# option) any later version.
|
||||
#
|
||||
# This program is distributed in the hope that it will be useful, but WITHOUT
|
||||
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
|
||||
# FOR A PARTICULAR PURPOSE. See the GNU Library Lesser General Public License
|
||||
# for more details at ( http://www.gnu.org/licenses/lgpl.html ).
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with this program; if not, write to the Free Software Foundation, Inc.,
|
||||
# 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
|
||||
# written by: Jurko Gospodnetić ( jurko.gospodnetic@pke.hr )
|
||||
|
||||
"""
|
||||
Generic functionality shared in different development utility modules.
|
||||
|
||||
"""
|
||||
|
||||
import os
|
||||
import os.path
|
||||
import sys
|
||||
if sys.version_info < (3, 0):
|
||||
from urllib import quote as url_quote
|
||||
else:
|
||||
from urllib.parse import quote as url_quote
|
||||
|
||||
|
||||
def any_contains_any(strings, candidates):
|
||||
"""Whether any of the strings contains any of the candidates."""
|
||||
for string in strings:
|
||||
for c in candidates:
|
||||
if c in string:
|
||||
return True
|
||||
|
||||
|
||||
class FileJanitor:
|
||||
"""Janitor class for removing a specific file."""
|
||||
|
||||
def __init__(self, path):
|
||||
self.__path = path
|
||||
|
||||
def clean(self):
|
||||
try:
|
||||
os.remove(self.__path)
|
||||
except (KeyboardInterrupt, SystemExit):
|
||||
raise
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
def path_to_URL(path, escape=True):
|
||||
"""Convert a local file path to a absolute path file protocol URL."""
|
||||
# We do not use urllib's builtin pathname2url() function since:
|
||||
# - it has been commented with 'not recommended for general use'
|
||||
# - it does not seem to work the same on Windows and non-Windows platforms
|
||||
# (result starts with /// on Windows but does not on others)
|
||||
# - urllib implementation prior to Python 2.5 used to quote ':' characters
|
||||
# as '|' which would confuse pip on Windows.
|
||||
url = os.path.abspath(path)
|
||||
for sep in (os.sep, os.altsep):
|
||||
if sep and sep != "/":
|
||||
url = url.replace(sep, "/")
|
||||
if escape:
|
||||
# Must not escape ':' or '/' or Python will not recognize those URLs
|
||||
# correctly. Detected on Windows 7 SP1 x64 with Python 3.4.0, but doing
|
||||
# this always does not hurt since both are valid ASCII characters.
|
||||
no_protocol_URL = url_quote(url, safe=":/")
|
||||
else:
|
||||
no_protocol_URL = url
|
||||
return "file:///%s" % (no_protocol_URL,)
|
||||
|
||||
|
||||
def report_error(message):
|
||||
print("ERROR: %s" % (message,))
|
||||
|
||||
|
||||
def requirement_spec(package_name, *args):
|
||||
"""Identifier used when specifying a requirement to pip or setuptools."""
|
||||
if not args or args == (None,):
|
||||
return package_name
|
||||
version_specs = []
|
||||
for version_spec in args:
|
||||
if isinstance(version_spec, (list, tuple)):
|
||||
operator, version = version_spec
|
||||
else:
|
||||
assert isinstance(version_spec, str)
|
||||
operator = "=="
|
||||
version = version_spec
|
||||
version_specs.append("%s%s" % (operator, version))
|
||||
return "%s%s" % (package_name, ",".join(version_specs))
|
||||
|
||||
|
||||
def path_iter(path):
|
||||
"""Returns an iterator over all the file & folder names in the path."""
|
||||
parts = []
|
||||
while path:
|
||||
path, item = os.path.split(path)
|
||||
if item:
|
||||
parts.append(item)
|
||||
return reversed(parts)
|
||||
|
|
@ -0,0 +1,110 @@
|
|||
# -*- coding: utf-8 -*-
|
||||
|
||||
# This program is free software; you can redistribute it and/or modify it under
|
||||
# the terms of the (LGPL) GNU Lesser General Public License as published by the
|
||||
# Free Software Foundation; either version 3 of the License, or (at your
|
||||
# option) any later version.
|
||||
#
|
||||
# This program is distributed in the hope that it will be useful, but WITHOUT
|
||||
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
|
||||
# FOR A PARTICULAR PURPOSE. See the GNU Library Lesser General Public License
|
||||
# for more details at ( http://www.gnu.org/licenses/lgpl.html ).
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public License
|
||||
# along with this program; if not, write to the Free Software Foundation, Inc.,
|
||||
# 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
|
||||
# written by: Jurko Gospodnetić ( jurko.gospodnetic@pke.hr )
|
||||
|
||||
"""
|
||||
Zip compression related utilities.
|
||||
|
||||
"""
|
||||
|
||||
import os
|
||||
import os.path
|
||||
import sys
|
||||
import zipfile
|
||||
|
||||
from suds_devel.utility import path_iter
|
||||
|
||||
|
||||
def zip_folder_content(folder, zip_file):
|
||||
success = False
|
||||
zippy = zipfile.ZipFile(zip_file, "w", _zip_compression())
|
||||
try:
|
||||
archiver = _Archiver(zippy, folder)
|
||||
for root, folders, files in os.walk(folder):
|
||||
archiver.add_folder_with_files(root, files)
|
||||
success = True
|
||||
finally:
|
||||
zippy.close()
|
||||
if not success:
|
||||
os.remove(zip_file)
|
||||
|
||||
|
||||
class _Archiver:
|
||||
|
||||
def __init__(self, zip_file, folder):
|
||||
self.__zip_file = zip_file
|
||||
self.__base_folder_parts = list(path_iter(folder))
|
||||
|
||||
def add_folder_with_files(self, folder, files):
|
||||
path_prefix = self.__path_prefix(folder)
|
||||
for file in files:
|
||||
assert file
|
||||
file_path = os.path.join(folder, file)
|
||||
self.__zip_file.write(file_path, path_prefix + file)
|
||||
# If no files are present in this folder and this is not the base
|
||||
# folder then we need to add the folder itself as an explicit entry.
|
||||
if not files and path_prefix:
|
||||
# Old Python versions did not support using the ZipFile.write()
|
||||
# method on folders so we do it manually by adding a 0-size entry
|
||||
# using ZipFile.writestr(). Encountered using Python 2.4.3.
|
||||
# N.B. An archived folder name must include a trailing slash, which
|
||||
# is exactly what we have in our prepared path_prefix value.
|
||||
self.__zip_file.writestr(path_prefix, "")
|
||||
|
||||
def __path_prefix(self, folder):
|
||||
"""
|
||||
Path prefix to be used when archiving any items from the given folder.
|
||||
|
||||
Expects the folder to be located under the base folder path and the
|
||||
returned path prefix does not include the base folder information. This
|
||||
makes sure we include just the base folder's content in the archive,
|
||||
and not the base folder itself.
|
||||
|
||||
"""
|
||||
path_parts = path_iter(folder)
|
||||
_skip_expected(path_parts, self.__base_folder_parts)
|
||||
result = "/".join(path_parts)
|
||||
if result:
|
||||
result += "/"
|
||||
return result
|
||||
|
||||
|
||||
# Mimic the next() built-in introduced in Python 2.6. Does not need to be
|
||||
# perfect but only as good as we need it internally in this module.
|
||||
if sys.version_info < (2, 6):
|
||||
def _iter_next(iter):
|
||||
return iter.next()
|
||||
else:
|
||||
_iter_next = next
|
||||
|
||||
|
||||
def _skip_expected(iter, expected):
|
||||
for expected_value in expected:
|
||||
value = _iter_next(iter)
|
||||
assert value == expected_value
|
||||
|
||||
|
||||
_zip_compression_value = None
|
||||
|
||||
def _zip_compression():
|
||||
global _zip_compression_value
|
||||
if _zip_compression_value is None:
|
||||
try:
|
||||
import zlib
|
||||
_zip_compression_value = zipfile.ZIP_DEFLATED
|
||||
except ImportError:
|
||||
_zip_compression_value = zipfile.ZIP_STORED
|
||||
return _zip_compression_value
|
||||
Loading…
Reference in New Issue