.tgt.heapykinds.Use

Use

Constructor

guppy.hpy( [ht = Any+]) -> Use

Methods

findex( ^0..*:kind: Kind+) -> (

Subkind of: EquivalenceRelation

callable: (index: notnegative+)

)

heap() -> IdentitySet

heapg() -> IdentitySet

heapu() -> Stat

idset(nodes: iterable+) -> IdentitySet

iso( ^0..*:node: Any+) -> IdentitySet

load( ^alt:[fn: loadablefilenamestring+ ^or fn: loadableiterableofstrings+] [use_readline = boolean+]) -> Stat

monitor()

pb( [filename: profilefilename+])

setref()

setrelheap( [reference: UniSet+])

setrelheapg( [reference: UniSet+])

setrelheapu( [reference: UniSet+])

Attributes

Anything: Kind

Nothing: IdentitySet

Clodo:  (

EquivalenceRelation

callable: ( ^alt:[tc: typeexceptdict+ ^or dictof = typeoremptytuple+]) -> Kind

)

Id:  (

EquivalenceRelation

callable: (address: objectaddress+) -> Kind

)

Module:  (

EquivalenceRelation

callable: ( ^draw:[name = modulename+ , at = moduleaddress+]) -> Kind

)

Prod:  (

EquivalenceRelation

callable: () -> KindOfProdFamily

callable: (path_prefix: string+) -> Kind

callable: (scope: Any+) -> Kind

callable: (filename: string+, lineno: positive+) -> KindOfProdFamily

)

Rcs:  (

EquivalenceRelation

callable: ( ^{0..*: alt:}[kind: ClodoKind+ ^or sok: SetOfClodoKind+]) -> KindOfRetClaSetFamily

)

Root: RootStateType

Size:  (

EquivalenceRelation

callable: (size: notnegative+) -> KindOfSizeFamily

)

Type:  (

EquivalenceRelation

callable: (type: type+) -> KindOfTypeFamily

)

Unity:  (

EquivalenceRelation

callable: () -> Kind

)

Via:  (

EquivalenceRelation

callable: ( ^0..*:rel: relationname+) -> KindOfInViaFamily

)

doc:  anything

Subkind of: EquivalenceRelation

callable: (index: notnegative+)

Calling the returned equivalence relation creates an equivalence class.

Argument

index: notnegative+

The position of the matching kind in the sequence of kinds. The first one has index 0. Specifying the length of the sequence means that the equivalence class returned is the one where none of the kinds in the sequence matched.

)

Create an equivalence relation based on a sequence of kinds. The name is a combination of find and index. The classification of each objects is done as follows:

For each kind in the sequence, check whether the object is an element of that kind. If it is, the classification is the index of that kind in the sequence. If the end of the sequence is reached, the classification is the length of the sequence.

Argument

kind: Kind+

Each argument specifies the kind in that position in the sequence.

Bugs

Though the Kind objects representing the equivalence classes work with set operations such as intersection and union, the tests such as subset and equality do not generally give the expected result.

Traverse the heap from a root to find all reachable and visible objects.

The objects that belong to an heapy instance are normally made invisible. This may be modified in various ways.

See also

setref

See also

The ht argument to guppy.hpy

Note

This method is depreciated. The heapu method is the recommended replacement.

Finds the objects in the heap that remain after garbage collection but are not reachable from the root except possibly via containers created by heapy.

This can be used to find objects in extension modules that remain in memory even though they are gc-collectable and not reachable.

See also

setrelheapg

Bug

Sometimes it shows objects in the reachable heap as well, if they are contained in sets created by a previous call to .heapg(). It was tricky to separate out those objects reachable only via the heapy containers.

Finds the objects in the heap that remain after garbage collection but are not reachable from the root.

This can be used to find objects in extension modules that remain in memory even though they are gc-collectable and not reachable.

Returns an object containing a statistical summary of the objects found - not the objects themselves. This is to avoid making the objects reachable.

See also

setref

Create a set of objects based on identity.

Argument

nodes: iterable+

The argument must be an iterable and may yield any kind of objects.

Note

This method is the same as iso except for the argument.

AKA is object or identity set of objects.

Create a set of objects based on identity.

Argument

node: Any+

Any kind of objects are valid arguments.

Example

>>> from guppy import hpy
>>> hpy().iso(1, 1, 1.0, [], [], {})
Partition of a set of 5 objects. Total size = 444 bytes.
 Index  Count   %     Size   % Cumulative  % Kind (class / dict of class)
     0      1  20      248  56       248  56 dict (no owner)
     1      2  40      144  32       392  88 list
     2      1  20       28   6       420  95 int
     3      1  20       24   5       444 100 float
>>>

Note

This method is the same as idset except for the argument.

Load heapy-related data from a serialized form. Currently it handles data generated by Stat.dump.

Arguments

fn: loadablefilenamestring+

A string argument is treated as a file name.

fn: loadableiterableofstrings+

An open file or an iterator will be iterated over enough to read one package of data, and another call to load will read the next package.

use_readline = boolean+

If true, the method will use .readline() instead of iteration, which may be necessary in case the input comes from a pipe since otherwise the Python runtime would try to read ahead a big block before returning the first package of data.

Note

Why it is not always using readline is that AFAIK iteration should normally be used since it is faster as well as more general.

Returns typically, one package of statistical data.

Maybe it may return other kinds of data (in the future) since there is a directive in the file format to choose an alternative loader.

Start an interactive remote monitor.

This can be used to get information about the state, in particular the memory usage, of separately running Python processes. It can connect to such processes, and bring up an interactive console in each process in which Python can be used much as it is normally used, while the target process under observation continues to execute as usual, typically under user control in another window. By observing and comparing the content of the heap of the target process at different times it is possible to find problems such as memory leaks, and then investigate the reasons for the alleged misbehaviours. It is sometimes also possible to test a fix in the running target process before updating its source code and restarting it.

The target process needs to be enabled to accept remote monitoring. This is implemented via a thread that tries to connect to a particular socket. This thread can be started in the following way:

import guppy.heapy.RM

It is possible to put that command in the sitecustomize.py file of the installation. This may be practical so that all Python processes can be monitored, but there may be security issues to consider since any process with access to the monitoring socket can take over the processes that accept remote monitoring.

A process may disable remote monitoring of itself in the following way:

from guppy.heapy.Remote import off; off()

Example

nicosys [175] solitaire.py &
[1] 25928
nicosys [176] python -c "from guppy import hpy;hpy().monitor()"
<Monitor>
*** Connection 1 opened ***
<Monitor> h

Documented commands (type help <topic>):
========================================
exit  h  help  int  lc  q  sc

<Monitor> lc
CID PID   ARGV
  1 25928 ['/home/nilsson/bin/solitaire.py']
<Monitor> sc 1
Remote connection 1. To return to Monitor, type <Ctrl-C> or .<RETURN>
<Annex> h

Documented commands (type help <topic>):
========================================
close  h  help  int  isolatest  q  reset  stat

<Annex> stat
Target overview
------------------------------------
target.sys.executable   = /usr/local/bin/python
target.sys.argv         = ['/home/nilsson/bin/solitaire.py']
target.wd               = /var/tmp
target.pid              = 25928
<Annex> int
Interactive console. To return to Annex, type '-'.
>>> hp.heap()
Partition of a set of 21919 objects. Total size = 1558688 bytes.
 Index  Count   %     Size   % Cumulative  % Kind (class / dict of class)
     0  11467  52   739904  47    739904  47 str
...
     9     52   0    27040   2   1436516  92 dict of __main__.Card
<62 more rows. Type e.g. '_.more' to view.>
>>> .
<Monitor> exit

Create a Profile Browser window.

Argument

filename: profilefilename+

The name of a file containing profile data. The data should be in the format as generated by the Stat .dump method. For example, you can generate the data file by doing the following from time to time:

>> from guppy import hpy
>> hpy().heap().stat.dump(filename)

The dump method appends a sample to the end of the file, containing a summary description of the current heap contents. The sequence of such samples can be browsed in the Profile Browser window.

See also

• Heapy Profile Browser
• Screenshot

Set a reference point for heap usage measurement.

This applies to both the heap and heapu methods. The heap() method will only show the objects allocated after the time setref was called. The heapu() method, since it deals with summary data and not actual objects, will show the difference of sizes and counts compared to when setref was called.

Note

This method may become depreciated or reserved for internal use; the setref method is the preferred public interface.

Set relative heap view.

With no argument, set a variable in self to the current set of objects in the heap. This set will be subtracted from the returned set in subsequent calls to heap . In this way, it will show the heap relative to its contents when setrelheap was called, so that only newly allocated objects are included.

Argument

reference: UniSet+

Pass an alternative set here to set the relative heap reference to that value instead of the current heap. For example, pass Nothing to make the heap view show all visible objects again.

Note

This method is depreciated. Use setref or setrelheapu instead.

Equivalent to setrelheap except it applies to heapg instead. I may consider combining these methods to one.

Note

This method may become depreciated or reserved for internal use; the setref method is the preferred public interface.

Equivalent to setrelheap except it applies to heapu instead. Both of these methods are combined in the setref method.

A kind that represents all kinds of objects.

The empty set.

Equivalence relation by type or dict owner.

It defines objects to be be equivalent when their classifications, as defined below, are equal.

1. If the object is not a dictionary, the classification will be the type of the object.
2. The object is a dictionary. The referrers of the object are searched to find one that 'owns' the dictionary. That is, the dict is a builtin __dict__ attribute of the owner. If no such owner is found, the object will be the classified as a dict with no owner. If the owner is found, it will be classified in its turn, by type as defined in Type . The object will be classified as a dict owned by such a kind of object.

EquivalenceRelation

callable: ( ^alt:[tc: typeexceptdict+ ^or dictof = typeoremptytuple+]) -> Kind

Creates an equivalence class of Clodo.

Arguments

tc: typeexceptdict+

If this argument is specified, it must be or represent a Python type object, except that it may not be the type dict, and the resulting Kind will be the same as that returned from Type(tc).

dictof = typeoremptytuple+

If this argument is specified, and is an empty tuple, the result will be a kind representing a dict that has no owner. Otherwise the owner kind will be created as by Type (dictof), and the result will be a kind representing a dict that is owned by that class.

)

This equivalence relation defines objects to be equivalent only if they are identical, i.e. have the same address.

EquivalenceRelation

callable: (address: objectaddress+) -> Kind

Creates an equivalence class.

Argument

address: objectaddress+

The argument should be the address of an object.

)

This equivalence relation defines objects to be equivalent if they are the same module, or if none of them is a module. Partitioning a set of objects using this equivalence relation will therefore result in one singleton set for each module and one set containing all other objects.

EquivalenceRelation

callable: ( ^draw:[name = modulename+ , at = moduleaddress+]) -> Kind

Create a Kind that matches objects to the given module.

Arguments

name = modulename+

The name may be specified as a string.

at = moduleaddress+

The address may also be specified, as an alternative or together with the name.

)

In this equivalence relation, objects are classified by the producer, the line in which the object was allocated.

EquivalenceRelation

callable: () -> KindOfProdFamily

Create a Kind that matches objects whose producer is unknown.

callable: (path_prefix: string+) -> Kind

Create a Kind that matches objects whose producer has a path that starts with the given prefix.

Argument

path_prefix: string+

A path prefix filter on the file name

callable: (scope: Any+) -> Kind

Create a Kind that matches objects whose producer has is within the given scope

Argument

scope: Any+

Anything that can be used by inspect.getsourcelines()

callable: (filename: string+, lineno: positive+) -> KindOfProdFamily

Create a Kind that matches objects matches the given producer. Arguments may be None but will need alternative equivalence relations from alt() method to match.

Arguments

filename: string+

File name of the producer

lineno: positive+

Line number of the producer

)

(Referrer classification set.)

In this equivalence relation, objects are classified by classifying their referrers.

The classification of the referrers is done using the classifier of the Clodo equivalence relation. The classifications of the referrers are collected in a set. This set represents the classification of the object.

EquivalenceRelation

callable: ( ^{0..*: alt:}[kind: ClodoKind+ ^or sok: SetOfClodoKind+]) -> KindOfRetClaSetFamily

Create an equivalence class from user-specified classification arguments. The arguments should together specify a set of Kind objects, each of which must represent an equivalence class of Clodo.

Arguments

kind: ClodoKind+

This adds a single Kind to the set of Kinds of referrers.

sok: SetOfClodoKind+

This adds each Kind in the sok argument to the total set of Kinds of referrers.

)

This attribute is a symbolic root containing attributes from which all reachable objects in the heap can be reached. It is the only value (a singleton) of its kind; see heapykinds.RootStateType for a description of its attributes.

In this equivalence relation, objects are classified by memory size, so each equivalence class represents a particular size of object.

EquivalenceRelation

callable: (size: notnegative+) -> KindOfSizeFamily

Create a Kind representing a particular size.

)

In this equivalence relation, objects are classified by type so each equivalence class represents objects of a particular type.

EquivalenceRelation

callable: (type: type+) -> KindOfTypeFamily

Create a Kind representing a particular type.

Argument

type: type+

A Python type object or a representation of it.

)

In this equivalence relation, all objects are considered equivalent. There is only one equivalence class.

EquivalenceRelation

callable: () -> Kind

Returns the equivalence class, which is Anything .

)

In this equivalence relation, objects are classified by how they are referred from their referrers, so each equivalence class represents objects that have a particular set of relations to their referrers.

EquivalenceRelation

callable: ( ^0..*:rel: relationname+) -> KindOfInViaFamily

Create a Kind representing a particular set of referrer relations.

Argument

rel: relationname+

Each argument specifies one referrer relation. The arguments should be strings and can be of any of the following forms.

[expression]	Indexing of a dict, list, tuple (etc). The expression must be a Python expression that can be evaluated in a local environment. The environment will contain the builtins and a name 'hp' that is bound to the current Use instance.
.attribute	Getting an attribute from a builtin type or a slot of a slotted type. (I.E. not an attribute that is in a dict of an object.)
.f_locals["name"]	A local variable of a frame.
.f_locals  ["name"]	A variable in a CELL of a frame. Note the space between  f_locals and [ . This is to distinguish it from ordinary locals, and still use a syntax that could be used to access those variables directly from Python.
.keys()[integer]	A key in a dictionary, at the indicated place in its keys().

)

Overview documentation for top level Heapy object. Provides a listing of the available attributes. Accessing the attribute name on the doc objects gives further info, eg:

>>> hp.doc.heap

gives doc for the heap method when hp is the top level Heapy object. References may be embedded in the documentations. To access a reference, opening up a web browser with the doc for it one can do eg:

>>> hp.doc.heap[1]

The reference number 0 is special. If it is provided, it is the reference to the html doc for the described object itself. So to see in the web browser the doc for the heap method one can do:

>>> hp.doc.heap[0]