[openstack-dev] psycopg2 wheel packaging issues

[Stephen Finucane]

I imagine most people have been seeing warnings like the one below
raised by various openstack packages recently:

.tox/py27/lib/python2.7/site-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2
wheel package will be renamed from release 2.8; in order to keep installing from binary
please use "pip install psycopg2-binary" instead. For details see:
<http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>.

Based on this warning, I had done what seemed to be the obvious thing
to do and proposed adding psycopg2-binary to the list of global
requirements [1]. This would allow us to replace all references to
psycopg2 with psycopg2-wheel in individual projects. However, upon
further investigation it seems this is not really an option since the
two packages exist in the same namespace and will clobber each other.
I've now abandoned this patch.

Does anyone with stronger Python packaging-fu than I have a better
solution for the psycopg2 folks? There's a detailed description of why
this was necessary on GitHub [2] along with some potential resolutions,
none of which seem to be acceptable. If nothing better is possible, it
seems we'll simply have to live with (or silence) these warnings in
psycopg2 2.7.x and start installing libpg again once 2.8 is released.

Cheers,
Stephen

[1] https://review.openstack.org/#/c/561924/
[2] https://github.com/psycopg/psycopg2/issues/674

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-***@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/c

[Doug Hellmann]

Post by Stephen Finucane
I imagine most people have been seeing warnings like the one below
.tox/py27/lib/python2.7/site-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2
wheel package will be renamed from release 2.8; in order to keep installing from binary
<http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>.
Based on this warning, I had done what seemed to be the obvious thing
to do and proposed adding psycopg2-binary to the list of global
requirements [1]. This would allow us to replace all references to
psycopg2 with psycopg2-wheel in individual projects. However, upon
further investigation it seems this is not really an option since the
two packages exist in the same namespace and will clobber each other.
I've now abandoned this patch.
Does anyone with stronger Python packaging-fu than I have a better
solution for the psycopg2 folks? There's a detailed description of why
this was necessary on GitHub [2] along with some potential resolutions,
none of which seem to be acceptable. If nothing better is possible, it
seems we'll simply have to live with (or silence) these warnings in
psycopg2 2.7.x and start installing libpg again once 2.8 is released.
Cheers,
Stephen
[1] https://review.openstack.org/#/c/561924/
[2] https://github.com/psycopg/psycopg2/issues/674

Bundling an SSL library seems like a particularly bad situation, but if
its ABI isn't stable it may be all they can do.

Perhaps some of the folks in the community who actually use Postgresql
can get involved with helping the upstream maintainers of psycopg and
libpg sort things out.

In the mean time, is there any reason we can't just continue to
install psycopg2 from source in our gate jobs after 2.8? If the
wheel packages for psycopg2 2.7.x are bad perhaps we can come up
with a way to pass --no-binary when installing it, but it's not
clear if we need to. Does the bug affect us?

Doug

Doug

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-***@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mai

[Stephen Finucane]

Post by Doug Hellmann

Post by Stephen Finucane
I imagine most people have been seeing warnings like the one below
.tox/py27/lib/python2.7/site-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2
wheel package will be renamed from release 2.8; in order to keep installing from binary
<http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>.
Based on this warning, I had done what seemed to be the obvious thing
to do and proposed adding psycopg2-binary to the list of global
requirements [1]. This would allow us to replace all references to
psycopg2 with psycopg2-wheel in individual projects. However, upon
further investigation it seems this is not really an option since the
two packages exist in the same namespace and will clobber each other.
I've now abandoned this patch.
Does anyone with stronger Python packaging-fu than I have a better
solution for the psycopg2 folks? There's a detailed description of why
this was necessary on GitHub [2] along with some potential resolutions,
none of which seem to be acceptable. If nothing better is possible, it
seems we'll simply have to live with (or silence) these warnings in
psycopg2 2.7.x and start installing libpg again once 2.8 is released.
Cheers,
Stephen
[1] https://review.openstack.org/#/c/561924/
[2] https://github.com/psycopg/psycopg2/issues/674

Bundling an SSL library seems like a particularly bad situation, but if
its ABI isn't stable it may be all they can do.
Perhaps some of the folks in the community who actually use Postgresql
can get involved with helping the upstream maintainers of psycopg and
libpg sort things out.

Yes, this would be my hope.

Post by Doug Hellmann
In the mean time, is there any reason we can't just continue to
install psycopg2 from source in our gate jobs after 2.8? If the
wheel packages for psycopg2 2.7.x are bad perhaps we can come up
with a way to pass --no-binary when installing it, but it's not
clear if we need to. Does the bug affect us?

The only reason we might have issues is the libpq dependency. This was
required in 2.6 and will be required once again in 2.8. If this hasn't
been dropped from the list of requirements then we won't see any
breakages. If we do, we know where the issue lies.

Stephen

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-***@lists.openstack.org?subject:unsubscribe
http://lists