.tgt.heapykinds.Use

Name

Use

Synopsis

Constructor
guppy.hpy( [ht = Any+]) -> Use
Methods
findex( 0..*:kind: Kind+) -> (
Subkind of: EquivalenceRelation
callable: (index: notnegative+)
)
heap() -> IdentitySet
heapg() -> IdentitySet
heapu() -> Stat
idset(nodes: iterable+) -> IdentitySet
iso( 0..*:node: Any+) -> IdentitySet
load( alt:[fn: loadablefilenamestring+ or fn: loadableiterableofstrings+] [use_readline = boolean+]) -> Stat
monitor()
pb( [filename: profilefilename+])
setref()
setrelheap( [reference: UniSet+])
setrelheapg( [reference: UniSet+])
setrelheapu( [reference: UniSet+])
Attributes
Anything: Kind
Nothing: IdentitySet
Clodo:  (
EquivalenceRelation
callable: ( alt:[tc: typeexceptdict+ or dictof = typeoremptytuple+]) -> Kind
)
Id:  (
EquivalenceRelation
callable: (address: objectaddress+) -> Kind
)
Module:  (
EquivalenceRelation
callable: ( draw:[name = modulename+ , at = moduleaddress+]) -> Kind
)
Prod:  (
EquivalenceRelation
callable: () -> KindOfProdFamily
callable: (path_prefix: string+) -> Kind
callable: (scope: Any+) -> Kind
callable: (filename: string+, lineno: positive+) -> KindOfProdFamily
)
Rcs:  (
EquivalenceRelation
callable: ( 0..*: alt:[kind: ClodoKind+ or sok: SetOfClodoKind+]) -> KindOfRetClaSetFamily
)
Root: RootStateType
Size:  (
EquivalenceRelation
callable: (size: notnegative+) -> KindOfSizeFamily
)
Type:  (
EquivalenceRelation
callable: (type: type+) -> KindOfTypeFamily
)
Unity:  (
EquivalenceRelation
callable: () -> Kind
)
Via:  (
EquivalenceRelation
callable: ( 0..*:rel: relationname+) -> KindOfInViaFamily
)
doc:  anything

Methods

findex( 0..*:kind: Kind+) -> (
Subkind of: EquivalenceRelation
callable: (index: notnegative+)
Calling the returned equivalence relation creates an equivalence class.
Argument
index: notnegative+
The position of the matching kind in the sequence of kinds. The first one has index 0. Specifying the length of the sequence means that the equivalence class returned is the one where none of the kinds in the sequence matched.
)
Create an equivalence relation based on a sequence of kinds. The name is a combination of find and index. The classification of each objects is done as follows:

For each kind in the sequence, check whether the object is an element of that kind. If it is, the classification is the index of that kind in the sequence. If the end of the sequence is reached, the classification is the length of the sequence.

Argument
kind: Kind+
Each argument specifies the kind in that position in the sequence.
Bugs
Though the Kind objects representing the equivalence classes work with set operations such as intersection and union, the tests such as subset and equality do not generally give the expected result.
heap() -> IdentitySet
Traverse the heap from a root to find all reachable and visible objects.
The objects that belong to an heapy instance are normally made invisible. This may be modified in various ways.
See also
setref
See also
The ht argument to guppy.hpy
heapg() -> IdentitySet
Note
This method is depreciated. The heapu method is the recommended replacement.
Finds the objects in the heap that remain after garbage collection but are not reachable from the root except possibly via containers created by heapy.
This can be used to find objects in extension modules that remain in memory even though they are gc-collectable and not reachable.
See also
setrelheapg
Bug
Sometimes it shows objects in the reachable heap as well, if they are contained in sets created by a previous call to .heapg(). It was tricky to separate out those objects reachable only via the heapy containers.
heapu() -> Stat
Finds the objects in the heap that remain after garbage collection but are not reachable from the root.
This can be used to find objects in extension modules that remain in memory even though they are gc-collectable and not reachable.
Returns an object containing a statistical summary of the objects found - not the objects themselves. This is to avoid making the objects reachable.
See also
setref
idset(nodes: iterable+) -> IdentitySet
Create a set of objects based on identity.
Argument
nodes: iterable+
The argument must be an iterable and may yield any kind of objects.
Note
This method is the same as iso except for the argument.
iso( 0..*:node: Any+) -> IdentitySet
AKA is object or identity set of objects.
Create a set of objects based on identity.
Argument
node: Any+
Any kind of objects are valid arguments.
Example
>>> from guppy import hpy
>>> hpy().iso(1, 1, 1.0, [], [], {})
Partition of a set of 5 objects. Total size = 444 bytes.
 Index  Count   %     Size   % Cumulative  % Kind (class / dict of class)
     0      1  20      248  56       248  56 dict (no owner)
     1      2  40      144  32       392  88 list
     2      1  20       28   6       420  95 int
     3      1  20       24   5       444 100 float
>>>
Note
This method is the same as idset except for the argument.
load( alt:[fn: loadablefilenamestring+ or fn: loadableiterableofstrings+] [use_readline = boolean+]) -> Stat
Load heapy-related data from a serialized form. Currently it handles data generated by Stat.dump.
Arguments
fn: loadablefilenamestring+
A string argument is treated as a file name.
fn: loadableiterableofstrings+
An open file or an iterator will be iterated over enough to read one package of data, and another call to load will read the next package.
use_readline = boolean+
If true, the method will use .readline() instead of iteration, which may be necessary in case the input comes from a pipe since otherwise the Python runtime would try to read ahead a big block before returning the first package of data.
Note
Why it is not always using readline is that AFAIK iteration should normally be used since it is faster as well as more general.
Returns typically, one package of statistical data.
Maybe it may return other kinds of data (in the future) since there is a directive in the file format to choose an alternative loader.
monitor()
Start an interactive remote monitor.

This can be used to get information about the state, in particular the memory usage, of separately running Python processes. It can connect to such processes, and bring up an interactive console in each process in which Python can be used much as it is normally used, while the target process under observation continues to execute as usual, typically under user control in another window. By observing and comparing the content of the heap of the target process at different times it is possible to find problems such as memory leaks, and then investigate the reasons for the alleged misbehaviours. It is sometimes also possible to test a fix in the running target process before updating its source code and restarting it.

The target process needs to be enabled to accept remote monitoring. This is implemented via a thread that tries to connect to a particular socket. This thread can be started in the following way:

import guppy.heapy.RM

It is possible to put that command in the sitecustomize.py file of the installation. This may be practical so that all Python processes can be monitored, but there may be security issues to consider since any process with access to the monitoring socket can take over the processes that accept remote monitoring.

A process may disable remote monitoring of itself in the following way:

from guppy.heapy.Remote import off; off()

Example
nicosys [175] solitaire.py &
[1] 25928
nicosys [176] python -c "from guppy import hpy;hpy().monitor()"
<Monitor>
*** Connection 1 opened ***
<Monitor> h

Documented commands (type help <topic>):
========================================
exit  h  help  int  lc  q  sc

<Monitor> lc
CID PID   ARGV
  1 25928 ['/home/nilsson/bin/solitaire.py']
<Monitor> sc 1
Remote connection 1. To return to Monitor, type <Ctrl-C> or .<RETURN>
<Annex> h

Documented commands (type help <topic>):
========================================
close  h  help  int  isolatest  q  reset  stat

<Annex> stat
Target overview
------------------------------------
target.sys.executable   = /usr/local/bin/python
target.sys.argv         = ['/home/nilsson/bin/solitaire.py']
target.wd               = /var/tmp
target.pid              = 25928
<Annex> int
Interactive console. To return to Annex, type '-'.
>>> hp.heap()
Partition of a set of 21919 objects. Total size = 1558688 bytes.
 Index  Count   %     Size   % Cumulative  % Kind (class / dict of class)
     0  11467  52   739904  47    739904  47 str
...
     9     52   0    27040   2   1436516  92 dict of __main__.Card
<62 more rows. Type e.g. '_.more' to view.>
>>> .
<Monitor> exit
pb( [filename: profilefilename+])
Create a Profile Browser window.
Argument
filename: profilefilename+
The name of a file containing profile data. The data should be in the format as generated by the Stat .dump method. For example, you can generate the data file by doing the following from time to time:

>> from guppy import hpy
>> hpy().heap().stat.dump(filename)
The dump method appends a sample to the end of the file, containing a summary description of the current heap contents. The sequence of such samples can be browsed in the Profile Browser window.
See also
setref()
Set a reference point for heap usage measurement.
This applies to both the heap and heapu methods. The heap() method will only show the objects allocated after the time setref was called. The heapu() method, since it deals with summary data and not actual objects, will show the difference of sizes and counts compared to when setref was called.
setrelheap( [reference: UniSet+])
Note
This method may become depreciated or reserved for internal use; the setref method is the preferred public interface.
Set relative heap view.
With no argument, set a variable in self to the current set of objects in the heap. This set will be subtracted from the returned set in subsequent calls to heap . In this way, it will show the heap relative to its contents when setrelheap was called, so that only newly allocated objects are included.
Argument
reference: UniSet+
Pass an alternative set here to set the relative heap reference to that value instead of the current heap. For example, pass Nothing to make the heap view show all visible objects again.
setrelheapg( [reference: UniSet+])
Note
This method is depreciated. Use setref or setrelheapu instead.
Equivalent to setrelheap except it applies to heapg instead. I may consider combining these methods to one.
setrelheapu( [reference: UniSet+])
Note
This method may become depreciated or reserved for internal use; the setref method is the preferred public interface.
Equivalent to setrelheap except it applies to heapu instead. Both of these methods are combined in the setref method.

Attributes

Anything: Kind
A kind that represents all kinds of objects.
Nothing: IdentitySet
The empty set.
Clodo
Equivalence relation by type or dict owner.
It defines objects to be be equivalent when their classifications, as defined below, are equal.
  1. If the object is not a dictionary, the classification will be the type of the object.
  2. The object is a dictionary. The referrers of the object are searched to find one that 'owns' the dictionary. That is, the dict is a builtin __dict__ attribute of the owner. If no such owner is found, the object will be the classified as a dict with no owner. If the owner is found, it will be classified in its turn, by type as defined in Type . The object will be classified as a dict owned by such a kind of object.
Clodo:  (
EquivalenceRelation
callable: ( alt:[tc: typeexceptdict+ or dictof = typeoremptytuple+]) -> Kind
Creates an equivalence class of Clodo.
Arguments
tc: typeexceptdict+
If this argument is specified, it must be or represent a Python type object, except that it may not be the type dict, and the resulting Kind will be the same as that returned from Type(tc).
dictof = typeoremptytuple+
If this argument is specified, and is an empty tuple, the result will be a kind representing a dict that has no owner. Otherwise the owner kind will be created as by Type (dictof), and the result will be a kind representing a dict that is owned by that class.
)
Id
This equivalence relation defines objects to be equivalent only if they are identical, i.e. have the same address.
Id:  (
EquivalenceRelation
callable: (address: objectaddress+) -> Kind
Creates an equivalence class.
Argument
address: objectaddress+
The argument should be the address of an object.
)
Module
This equivalence relation defines objects to be equivalent if they are the same module, or if none of them is a module. Partitioning a set of objects using this equivalence relation will therefore result in one singleton set for each module and one set containing all other objects.
Module:  (
EquivalenceRelation
callable: ( draw:[name = modulename+ , at = moduleaddress+]) -> Kind
Create a Kind that matches objects to the given module.
Arguments
name = modulename+
The name may be specified as a string.
at = moduleaddress+
The address may also be specified, as an alternative or together with the name.
)
Prod
In this equivalence relation, objects are classified by the producer, the line in which the object was allocated.
Prod:  (
EquivalenceRelation
callable: () -> KindOfProdFamily
Create a Kind that matches objects whose producer is unknown.
callable: (path_prefix: string+) -> Kind
Create a Kind that matches objects whose producer has a path that starts with the given prefix.
Argument
path_prefix: string+
A path prefix filter on the file name
callable: (scope: Any+) -> Kind
Create a Kind that matches objects whose producer has is within the given scope
Argument
scope: Any+
Anything that can be used by inspect.getsourcelines()
callable: (filename: string+, lineno: positive+) -> KindOfProdFamily
Create a Kind that matches objects matches the given producer. Arguments may be None but will need alternative equivalence relations from alt() method to match.
Arguments
filename: string+
File name of the producer
lineno: positive+
Line number of the producer
)
Rcs
(Referrer classification set.)
In this equivalence relation, objects are classified by classifying their referrers.
The classification of the referrers is done using the classifier of the Clodo equivalence relation. The classifications of the referrers are collected in a set. This set represents the classification of the object.
Rcs:  (
EquivalenceRelation
callable: ( 0..*: alt:[kind: ClodoKind+ or sok: SetOfClodoKind+]) -> KindOfRetClaSetFamily
Create an equivalence class from user-specified classification arguments. The arguments should together specify a set of Kind objects, each of which must represent an equivalence class of Clodo.
Arguments
kind: ClodoKind+
This adds a single Kind to the set of Kinds of referrers.
sok: SetOfClodoKind+
This adds each Kind in the sok argument to the total set of Kinds of referrers.
)
Root: RootStateType
This attribute is a symbolic root containing attributes from which all reachable objects in the heap can be reached. It is the only value (a singleton) of its kind; see heapykinds.RootStateType for a description of its attributes.
Size
In this equivalence relation, objects are classified by memory size, so each equivalence class represents a particular size of object.
Size:  (
EquivalenceRelation
callable: (size: notnegative+) -> KindOfSizeFamily
Create a Kind representing a particular size.
)
Type
In this equivalence relation, objects are classified by type so each equivalence class represents objects of a particular type.
Type:  (
EquivalenceRelation
callable: (type: type+) -> KindOfTypeFamily
Create a Kind representing a particular type.
Argument
type: type+
A Python type object or a representation of it.
)
Unity
In this equivalence relation, all objects are considered equivalent. There is only one equivalence class.
Unity:  (
EquivalenceRelation
callable: () -> Kind
Returns the equivalence class, which is Anything .
)
Via
In this equivalence relation, objects are classified by how they are referred from their referrers, so each equivalence class represents objects that have a particular set of relations to their referrers.
Via:  (
EquivalenceRelation
callable: ( 0..*:rel: relationname+) -> KindOfInViaFamily
Create a Kind representing a particular set of referrer relations.
Argument
rel: relationname+
Each argument specifies one referrer relation. The arguments should be strings and can be of any of the following forms.

[expression] Indexing of a dict, list, tuple (etc). The expression must be a Python expression that can be evaluated in a local environment. The environment will contain the builtins and a name 'hp' that is bound to the current Use instance.
.attribute Getting an attribute from a builtin type or a slot of a slotted type. (I.E. not an attribute that is in a dict of an object.)
.f_locals["name"] A local variable of a frame.
.f_locals  ["name"] A variable in a CELL of a frame. Note the space between  f_locals and [ . This is to distinguish it from ordinary locals, and still use a syntax that could be used to access those variables directly from Python.
.keys()[integer] A key in a dictionary, at the indicated place in its keys().
)
doc:  anything
Overview documentation for top level Heapy object. Provides a listing of the available attributes. Accessing the attribute name on the doc objects gives further info, eg:
>>> hp.doc.heap
gives doc for the heap method when hp is the top level Heapy object. References may be embedded in the documentations. To access a reference, opening up a web browser with the doc for it one can do eg:
>>> hp.doc.heap[1]
The reference number 0 is special. If it is provided, it is the reference to the html doc for the described object itself. So to see in the web browser the doc for the heap method one can do:
>>> hp.doc.heap[0]

Generated by GSL-HTML 3.1.4.post1 on Tue Oct 24 11:59:30 2023