...
Code Block |
---|
from guppy import hpy import code hp=hpy() ... # reset the heap counters hp.setrelheap() ... # just print the heap somewhere: h = hp.heap() log.debug(f"\nheapy: {h}") # or possibly interrupt the code execution and inspect the hp object: code.interact(local=dict(globals(), **locals())) |
A typical dump in pyFF's mainloop then looks like this
Partition of a set of 117936 objects. Total size = 75634733 bytes.
Index Count % Size % Cumulative % Kind (class / dict of class)
0 1771 2 59385607 79 59385607 79 bytes
1 25917 22 8466168 11 67851775 90 dict (no owner)
2 25388 22 2640352 3 70492127 93 dict of pyff.samlmd.EntitySet
3 23656 20 2232132 3 72724259 96 str
4 25388 22 1218624 2 73942883 98 pyff.samlmd.EntitySet
5 6795 6 380520 1 74323403 98 lxml.etree._Element
6 2501 2 199024 0 74522427 99 tuple
7 870 1 154828 0 74677255 99 types.CodeType
8 1024 1 139264 0 74816519 99 function
9 45 0 127872 0 74944391 99 dict of module
<196 more rows. Type e.g. '_.more' to view.>
Another way of profiling pyFF's memory usage is just following top or htop for a long-running pyFF process, that has a 60s refresh interval. I normally use this pipeline
Code Block |
---|
- when update:
- load:
- edugain.xml
- when request:
- select:
- pipe:
- when accept application/samlmetadata+xml application/xml:
- first
- finalize:
cacheDuration: PT12H
validUntil: P10D
- sign:
key: cert/sign.key
cert: cert/sign.crt
- emit application/samlmetadata+xml
- break
- when accept application/json:
- discojson
- emit application/json
- break |
to feed the edugain feed that has been dowloaded using
$ curl http://mds.edugain.org/ -o edugain.xml
Un/Pickling etree.ElementTree object
...