|Summary:||easy_install can't install anything because it uses HTTP but pypi.python.org requires HTTPS|
|Product:||Red Hat Enterprise Linux 6||Reporter:||Radek Bíba <rbiba>|
|Component:||python-setuptools||Assignee:||Charalampos Stratakis <cstratak>|
|Status:||CLOSED CURRENTRELEASE||QA Contact:||Mirosław Zalewski <mzalewsk>|
|Version:||6.9||CC:||briang, cstratak, dkutalek, hhorak, jamills, jkejda, jonathan.allen, mhroncok, mkolaja, mzalewsk, nobody+bgollahe, pviktori, rbiba, toneata|
|Target Milestone:||rc||Keywords:||Regression, Reopened, ZStream|
|Fixed In Version:||python-setuptools-0.6.10-4.el6||Doc Type:||If docs needed, set a value|
Previously, the easy_install utility failed to download packages from the Python Package Index (PyPI) repository. This happened because PyPI has recently switched to using the HTTPS protocol, which easy_install did not support. With this update, easy_install is now able to connect to PyPI using HTTPS, and, as a result, the described problem no longer occurs.
|:||1519849 (view as bug list)||Environment:|
|Last Closed:||2018-08-21 16:17:57 UTC||Type:||Bug|
|oVirt Team:||---||RHEL 7.3 requirements from Atomic Host:|
|Cloudforms Team:||---||Target Upstream Version:|
|Bug Depends On:|
|Bug Blocks:||1494507, 1519849|
Description Radek Bíba 2017-11-07 12:24:10 UTC
Description of problem: The easy_install utility worked fine a few weeks ago but now it's broken. See, for example, here: # easy_install stitches Searching for stitches Reading http://pypi.python.org/simple/stitches/ Couldn't find index page for 'stitches' (maybe misspelled?) Scanning index of all packages (this may take a while) Reading http://pypi.python.org/simple/ No local packages or download links found for stitches error: Could not find suitable distribution for Requirement.parse('stitches') Now, stitches *is* an existing project. Open the first URL -- http://pypi.python.org/simple/stitches/ -- and you'll see it. But, the URL has become https... and easy_install didn't switch to that one or didn't use the HTTPS URL automatically. FWIW, it does the latter in RHEL 7. A potential workaround is using "-i https://pypi.python.org/simple". This works for some projects, but if the dependency chain contains something that wants to install stuff using easy_install again, the installation of that dependency fails. This is the case with stitches: # easy_install -i https://pypi.python.org/simple stitches Searching for stitches Reading https://pypi.python.org/simple/stitches/ Best match: stitches 0.11 Downloading https://pypi.python.org/packages/9d/73/681005f0f0fb9483f0d94547a35059c776144fc998780b9ef00167a50f3d/stitches-0.11.tar.gz#md5=0ac0b6e418767e6ec5cf684460d1972f Processing stitches-0.11.tar.gz Running stitches-0.11/setup.py -q bdist_egg --dist-dir /tmp/easy_install-ZMIU8w/stitches-0.11/egg-dist-tmp-82ctsZ zip_safe flag not set; analyzing archive contents... stitches.connection: module references __file__ Adding stitches 0.11 to easy-install.pth file Installed /usr/lib/python2.6/site-packages/stitches-0.11-py2.6.egg Processing dependencies for stitches ... ... Searching for pynacl>=1.0.1 Reading https://pypi.python.org/simple/pynacl/ Best match: PyNaCl 1.2.0 Downloading https://pypi.python.org/packages/fa/38/0f35ec4beb6562f1abfa07914db1cea978e93da409ba6293f810d9e677d6/PyNaCl-1.2.0.tar.gz#md5=7a31e95f5f457184b1660b71d6808b19 Processing PyNaCl-1.2.0.tar.gz Running PyNaCl-1.2.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-a0vNJS/PyNaCl-1.2.0/egg-dist-tmp-QYvAk9 Checking .pth file support in . /usr/bin/python -E -c pass Searching for cffi>=1.4.1 Reading http://pypi.python.org/simple/cffi/ Couldn't find index page for 'cffi' (maybe misspelled?) Scanning index of all packages (this may take a while) Reading http://pypi.python.org/simple/ No local packages or download links found for cffi>=1.4.1 error: Could not find suitable distribution for Requirement.parse('cffi>=1.4.1') So the only solution is hacking the source code: # sed -i s,http://pypi.python.org,https://pypi.python.org, /usr/lib/python2.6/site-packages/setuptools/command/easy_install.py Another (unrelated) issue appears later in the process (that's another story), but at least everything is downloaded using HTTPS. So please tweak the easy_install.py file. Version-Release number of selected component (if applicable): python-setuptools-0.6.10-3.el6.noarch
Comment 1 Charalampos Stratakis 2017-11-07 14:41:18 UTC
easy_install is not the recommended way of installing packages from pypi on RHEL 6 as unfortunately the version of setuptools that we ship is way too old. Combined with the fact that RHEL 6 is currently on production phase 3, this issue is not one that we can realistically fix. The recommended way to install packages from pypi on RHEL 6 is through pip that is shipped with the python software collection.
Comment 2 Radek Bíba 2017-11-07 15:28:16 UTC
I prefer pip myself, but it doesn't always help -- in some cases easy_install is called while installing a dependency. Here's a reproducer: # pip install pynacl DEPRECATION: Python 2.6 is no longer supported by the Python core team, please upgrade your Python. A future version of pip will drop support for Python 2.6 Collecting pynacl /usr/lib/python2.6/site-packages/pip-9.0.1-py2.6.egg/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning. SNIMissingWarning /usr/lib/python2.6/site-packages/pip-9.0.1-py2.6.egg/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning. InsecurePlatformWarning Downloading PyNaCl-1.2.0.tar.gz (3.3MB) 100% |████████████████████████████████| 3.3MB 369kB/s Complete output from command python setup.py egg_info: Couldn't find index page for 'cffi' (maybe misspelled?) No local packages or download links found for cffi>=1.4.1 Traceback (most recent call last): File "<string>", line 1, in <module> File "/tmp/pip-build-OCphNh/pynacl/setup.py", line 248, in <module> "Programming Language :: Python :: 3.6", File "/usr/lib64/python2.6/distutils/core.py", line 113, in setup _setup_distribution = dist = klass(attrs) File "/usr/lib/python2.6/site-packages/setuptools/dist.py", line 221, in __init__ self.fetch_build_eggs(attrs.pop('setup_requires')) File "/usr/lib/python2.6/site-packages/setuptools/dist.py", line 245, in fetch_build_eggs parse_requirements(requires), installer=self.fetch_build_egg File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 538, in resolve dist = best[req.key] = env.best_match(req, self, installer) File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 780, in best_match return self.obtain(req, installer) # try and download/install File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 792, in obtain return installer(requirement) File "/usr/lib/python2.6/site-packages/setuptools/dist.py", line 293, in fetch_build_egg return cmd.easy_install(req) File "/usr/lib/python2.6/site-packages/setuptools/command/easy_install.py", line 466, in easy_install raise DistutilsError(msg) distutils.errors.DistutilsError: Could not find suitable distribution for Requirement.parse('cffi>=1.4.1') ---------------------------------------- Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-OCphNh/pynacl/
Comment 3 Charalampos Stratakis 2017-11-07 16:07:03 UTC
Reopening the bugzilla for the time being. Will request an opinion from PM.
Comment 4 Charalampos Stratakis 2017-11-07 17:53:32 UTC
High-level overview of the issue: easy_install, a binary we ship with the python-setuptools package, no longer fulfills its primary purpose due to an API change in a third-party service. easy_install is used to download packages from PyPI, a repository of third-party Python libraries. PyPI recently switched to supporting https requests only . Due to that change, easy_install is not able to fetch packages from the default repository anymore, so use cases that required that functionality are currently broken on RHEL 6 systems. We did not receive customer reports about this yet, but since this affects intallation, we worry that they'd only notice when they need to re-install a system in an emergency. Thus requesting the opinion of PM in the matter. Please note that depending on the severity of the issue, this bugzilla can be considered a z-stream candidate.  https://mail.python.org/pipermail/distutils-sig/2017-October/031712.html
Comment 5 Honza Horak 2017-11-10 14:04:17 UTC
We talked about this with James yesterday, and it actually seems like something we should try to solve -- however, by solving we might either mean a fix, or maybe we can be fine with a work-wround/KCS article if there is something users can do to fix the issue. So, the question is -- is there any way to mitigate the issue, other than fixing the package?
Comment 6 Petr Viktorin 2017-11-10 14:29:56 UTC
The tool itself has an option for alternate index URL, but transitive dependencies will call it unconfigured (see comment 2). A possible workaround is to install transitive dependencies first, i.e. in the example above, install `cffi` with pip before installing `pynacl`. However, there's no good way to list all transitive dependencies, so users would need to try installing what they need, on error install the first missing dependency, and repeat.
Comment 7 Charalampos Stratakis 2017-11-10 14:45:58 UTC
(In reply to Honza Horak from comment #5) > We talked about this with James yesterday, and it actually seems like > something we should try to solve -- however, by solving we might either mean > a fix, or maybe we can be fine with a work-wround/KCS article if there is > something users can do to fix the issue. > > So, the question is -- is there any way to mitigate the issue, other than > fixing the package? Per comment 1 the fix to the package would be relatively simple: --- a/setuptools/command/easy_install.py +++ b/setuptools/command/easy_install.py @@ -181,7 +181,7 @@ class easy_install(Command): else: self.all_site_dirs.append(normalize_path(d)) if not self.editable: self.check_site_dir() - self.index_url = self.index_url or "http://pypi.python.org/simple" + self.index_url = self.index_url or "https://pypi.python.org/simple" self.shadow_path = self.all_site_dirs[:] for path_item in self.install_dir, normalize_path(self.script_dir): if path_item not in self.shadow_path: A workaround could be that the same patch can be applied on a running system to the /usr/lib/python2.6/site-packages/setuptools/command/easy_install.py (and then the respective .pyc and .pyo files have to be deleted as well). That will essentially have the same effect, however this could prove problematic due to meddling with protected directories here (e.g. SELinux will then always complain from that point when using easy_install, due to python trying to generate the .pyc and .pyo files). All in all, a fix to the package seems less problematic than the available workarounds.
Comment 9 Marcel Kolaja 2017-12-01 08:14:56 UTC
(In reply to Radek Bíba from comment #0) > Description of problem: > The easy_install utility worked fine a few weeks ago but now it's broken. Radek, does this mean that it is a Regression? Thanks!
Comment 10 Radek Bíba 2017-12-01 08:28:28 UTC
Kind of. There was no change on the setuptools side that would cause this breakage, but the change at pypi.python.org has a negative impact on easy_install. I guess we could call it an indirect regression.