pydantic: Import slowdown after v1
After upgrading to v1, I’ve noticed that it takes almost double the time to import Pydantic. Observe:
$ python3 -m pip install -U 'pydantic<1'
[...]
Successfully installed pydantic-0.32.2
$ seq 2 | xargs -I -- python3 -X importtime -c '
import pydantic
print("pydantic version:", pydantic.VERSION)
' 2>&1 | begin
head -n 1
tail -n 2
end
import time: self [us] | cumulative | imported package
import time: 1058 | 70730 | pydantic
pydantic version: 0.32.2
$ python3 -m pip install -U pydantic
[...]
Successfully installed pydantic-1.3
$ seq 2 | xargs -I -- python3 -X importtime -c '
import pydantic
print("pydantic version:", pydantic.VERSION)
' 2>&1 | begin
head -n 1
tail -n 2
end
import time: self [us] | cumulative | imported package
import time: 802 | 128663 | pydantic
pydantic version: 1.3
v0.32.2 takes approximately 70ms to import on my MBP; v1.3 takes 130ms.
attrs for comparison takes 36ms:
$ seq 2 | xargs -I -- python3 -X importtime -c '
import attr
print("attrs version:", attr.__version__)
' 2>&1 | begin
head -n 1
tail -n 2
end
import time: self [us] | cumulative | imported package
import time: 882 | 35659 | attr
attrs version: 19.3.0
I can understand that this might not be something you’d want to spend time on, depending on what your direction for Pydantic is; but slow warmups have been plaguing Python CLIs for a very long time. If you’d like to investigate this further, tuna can help you visualise your dependency tree. Here’s what it’s showing me for tuna (python3 -X importtime -c 'import pydantic' 2>| psub)
:
There is the unfortunate practice in Python of importing the entire package’s contents in __init__
and (unfortunately) once you’ve done that, there’s no going back - not without a breaking release anyway.
As always, thanks for all your work on Pydantic.
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Comments: 16 (16 by maintainers)
Commits related to this issue
- improve pydantic import time, fix #1127, fix #1039 — committed to pydantic/pydantic by samuelcolvin 5 years ago
- improve pydantic import time, fix #1127, fix #1039 — committed to pydantic/pydantic by samuelcolvin 5 years ago
- improve pydantic import time, fix #1127, fix #1039 — committed to pydantic/pydantic by samuelcolvin 5 years ago
- improve pydantic import time (#1132) * improve pydantic import time, fix #1127, fix #1039 * more tweaks * tweak utils.py * defering inspect — committed to pydantic/pydantic by samuelcolvin 4 years ago
- improve pydantic import time (#1132) * improve pydantic import time, fix #1127, fix #1039 * more tweaks * tweak utils.py * defering inspect — committed to cuenca-mx/pydantic by samuelcolvin 4 years ago
- Sync 2.14 branch into main (#1127) Co-authored-by: sydney-runkle <sydneymarierunkle@gmail.com> Co-authored-by: Sydney Runkle <54324534+sydney-runkle@users.noreply.github.com> Co-authored-by: Samuel... — committed to alexdrydew/pydantic by davidhewitt 6 months ago
It appears to be caused, in no small part, by the pre-compilation of this particular regex with a range containing 262,057 characters:
I think the
re
module might’ve not been optimised for this particular use case 😉