You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.rst
+43-8Lines changed: 43 additions & 8 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -107,10 +107,10 @@ decorator function. Use as follows::
107
107
del b
108
108
return a
109
109
110
-
If a python script with decorator ``@profile`` is called using ``-m
110
+
If a python script with decorator ``@profile`` is called using ``-m
111
111
memory_profiler`` in the command line, the ``precision`` parameter is ignored.
112
112
113
-
Time-based memory usage
113
+
Time-based memory usage
114
114
==========================
115
115
Sometimes it is useful to have full memory usage reports as a function of
116
116
time (not line-by-line) of external processes (be it Python scripts or not).
@@ -131,14 +131,14 @@ e.g. `mprof run -h`.
131
131
In the case of a Python script, using the previous command does not
132
132
give you any information on which function is executed at a given
133
133
time. Depending on the case, it can be difficult to identify the part
134
-
of the code that is causing the highest memory usage.
134
+
of the code that is causing the highest memory usage.
135
135
136
136
Adding the `profile` decorator to a function and running the Python
137
-
script with
137
+
script with
138
138
139
139
mprof run <script>
140
140
141
-
will record timestamps when entering/leaving the profiled function. Runnning
141
+
will record timestamps when entering/leaving the profiled function. Running
142
142
143
143
mprof plot
144
144
@@ -152,16 +152,49 @@ A discussion of these capabilities can be found `here <http://fa.bianp.net/blog/
152
152
153
153
.. warning:: If your Python file imports the memory profiler `from memory_profiler import profile` these timestamps will not be recorded. Comment out the import, leave your functions decorated, and re-run.
154
154
155
-
The available commands for `mprof` are:
155
+
The available commands for `mprof` are:
156
156
157
-
- ``mprof run``: running an executable, recording memory usage
157
+
- ``mprof run``: running an executable, recording memory usage
158
158
- ``mprof plot``: plotting one the recorded memory usage (by default,
159
159
the last one)
160
160
- ``mprof list``: listing all recorded memory usage files in a
161
161
user-friendly way.
162
162
- ``mprof clean``: removing all recorded memory usage files.
163
163
- ``mprof rm``: removing specific recorded memory usage files
164
164
165
+
Tracking forked child processes
166
+
===============================
167
+
In a multiprocessing context the main process will spawn child processes whose
168
+
system resources are allocated separately from the parent process. This can
169
+
lead to an inaccurate report of memory usage since by default only the parent
170
+
process is being tracked. The ``mprof`` utility provides two mechanisms to
171
+
track the usage of child processes: sum the memory of all children to the
172
+
parent's usage and track each child individual.
173
+
174
+
To create a report that combines memory usage of all the children and the
175
+
parent, use the ``include_children`` flag in either the ``profile`` decorator or
176
+
ass a command line argument to ``mprof``::
177
+
178
+
mprof run --include-children <script>
179
+
180
+
The second method tracks each child independently of the main process,
181
+
serializing child rows by index to the output stream. Use the ``multiprocess``
182
+
flag and plot as follows::
183
+
184
+
mprof run --multiprocess <script>
185
+
mprof plot
186
+
187
+
This will create a plot using matplotlib similar to this:
You can combine both the ``include_children`` and ``multiprocess`` flags to show
194
+
the total memory of the program as well as each child individually.
195
+
196
+
.. warning:: currently the child tracking only works if a ``stream`` is provided to the ``profile`` (e.g. from the command line or in the decorator).
197
+
165
198
Setting debugger breakpoints
166
199
=============================
167
200
It is possible to set breakpoints depending on the amount of memory used.
@@ -260,7 +293,7 @@ LogFile of memory profiler module.
260
293
>>> import sys
261
294
>>> sys.stdout = LogFile('memory_profile_log')
262
295
263
-
``Customised reporting:``
296
+
``Customized reporting:``
264
297
265
298
Sending everything to the log file while running the memory_profiler
266
299
could be cumbersome and one can choose only entries with increments
@@ -412,6 +445,8 @@ cleanup.
412
445
413
446
`Dmitriy Novozhilov <https://github.com/demiurg906>`_ and `Sergei Lebedev <https://github.com/superbobry>`_ added support for `tracemalloc <https://docs.python.org/3/library/tracemalloc.html>`_.
414
447
448
+
`Benjamin Bengfort <https://github.com/bbengfort>`_ added support for tracking the usage of individual child processes and plotting them.
0 commit comments