[egenix-users] Pip + virtualenv problem installing egenix-mx-base

[M.-A. Lemburg]

Leonardo Santagada wrote:
> On Wed, May 11, 2011 at 7:04 PM, M.-A. Lemburg <mal at egenix.com> wrote:
>> Leonardo Santagada wrote:
>>> On Wed, May 11, 2011 at 1:45 PM, M.-A. Lemburg <mal at egenix.com> wrote:
>>>> It doesn't use setuptools, that's correct.
>>>>
>>>
>>> Any idea if you are going to use setuptools one day, or just skip to
>>> distutils2/packaging? Would a patch be accepted?
>>
>> We will likely never use setuptools to build our eggs (our bdist_egg
>> implementation in mxSetup.py works with pure distutils) ...
>>
>>> ps: Pip only works with source+setuptools and disutils2 will not have
>>> eggs, only source+setuptools or setup.cfg will be suported then.
>>
>> ... but switching to such a limited system isn't an option either :-)
> 
> why not support it? I don't know how mxSetup.py works, but there
> should be a way to support setuptools or at least
> --single-version-externally-managed

Well, let's put it this way: why doesn't pip support standard
distutils packages that do not use setuptools ?

It would not be a problem to add the above option as dummy
to distutils in mxSetup.py, but that would only solve the problem
for our packages, not the many other plain distutils packages
on PyPI.

pip should really just run "python setup.py install" in an external
process without adding setuptools to the Python runtime environment
upfront, since setuptools itself is not compatible with distutils.
It monkey patches distutils in ways that break its API, replacing
it with different logic, instead of doing things the right way by
registering new commands and using subclassing to add functionality.

Note that we did try to support setuptools at one point, but
the failures it caused in our package setups forced us to remove
that support again - certain functions in distutils that we
needed simply didn't work anymore with setuptools' monkey
patching in place.

>> I don't think pip/distutils2 will have a great future without support
>> for binary eggs: People without compiler and all the necessary
>> development dependencies installed won't be able to build
>> C extensions and distributions without source code won't work
>> either, making the system pretty much unusable for commercial
>> products.
> 
> well this a discussion for distutils-sig, I don't know their plans,
> but I would appreciate pure python distributions with pre compiled c
> extensions either as optional (in case you don't have a compiler) or
> mandatory (if there is no source for it in the src tarball). Python
> specific pure python eggs are an annoyance at least for me.

That won't work for packages that don't include source code,
since Python's byte code and marshal magic usually changes
with every release, so you still need separate downloads
per Python release.

It will also not work for packages that interface to other
libraries - those require a C extension.

Adding pure Python implementations of C extensions that don't
need external libraries is also a rather impractical approach:
the reason you write C extensions is to get the best possible
performance. Until Python ships with a really good JIT compiler,
a pure Python replacement would only serve as very poor
work-around.

We tried that with mxDateTime in the early days: there was
a pure Python implementation of it that got used in case the
C extension wasn't available. Support for it got dropped
rather soon, since no one was using it. In the few cases
where it did get used, this usually happened as result of
a compile error and was not intended by the users (who did
not notice the compile error).

IMHO, the only really useful feature in setuptools
was the egg format which has all batteries included. I don't
understand why they would want to drop the only useful feature
and keep all the rest ;-)

Anyway, this is a distutils-sig discussion. Just wanted to
explain some of the reasons why we don't support setuptools
and why pip currently doesn't work with egenix-mx-base.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, May 12 2011)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2011-06-20: EuroPython 2011, Florence, Italy               39 days to go

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/