Hi! I really hope that everyone reading this is still doing okay, and if that isn't the case, I wish you a good day!
pip 20.2 Released!
Last Wednesday, pip 20.2 was released, delivering
the 2020-resolver as well as many other improvements!
I was lucky to be able to get the fast-deps feature to be
included as part of the release. A brief description of this
experimental feature as well as testing instruction can be found on
Python Discuss.
The public exposure of the feature also remind me of some further
optimization to make on
the lazy wheel.
Hopefully without download parallelization it would not be too slow
to put off testing by concerned users of pip.
Preparation for Download Parallelization
As of this moment, we already have:
- Multithreading pool fallback working
- An opt-in to use lazy wheel to optain dependency information, and thus getting a list of wheels at the end of resolution ready to be downloaded together
What's left is only to interject a parallel download somewhere after the dependency resolution step. Still, this struggles me way more than I've ever imagined. I got so stuck that I had to give myself a day off in the middle of the week (and study some Rust), then I came up with something what was agreed upon as difficult to maintain.
Indeed, a large part of this is my fault, for not communicating the design
thoroughly with pip's maintainers and not carefully noting
stuff down during (verbal) discussions with my mentor. Thankfully
Chris Hunt came to
the rescue and did a refactoring that will make my future work much
easier and cleaner.