Effective Python 3 Flashcards

(503 cards)

1
Q

“What wall might you hit in Python programs?”

A

“At some point in writing Python programs

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

“What might happen even after optimizing your code?”

A

“Even after optimizing your code

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

“What is a reasonable assumption for modern computers?”

A

“On modern computers that have an increasing number of CPU cores

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

“Does the GIL allow true parallelism in threads?”

A

“Unfortunately

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

“What is another common suggestion for performance?”

A

“Another common suggestion is to re-write your performance-critical code as an extension module

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

“What can C do for Python programs?”

A

“C gets you closer to the bare metal and can run faster than Python

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

“Can C extensions start native threads?”

A

“C extensions can also start native threads independent of the Python interpreter than run in parallel and utilize multiple CPU cores with no concern for the GIL.”

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

“Is Python’s C extension API well documented?”

A

“Python’s API for C extensions is well documented and a good choice for an escape hatch.”

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

“What tools can help with C extensions?”

A

“It’s also worth checking out tools like SWIG and CLIF.”

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

“What is the cost of rewriting in C?”

A

“However

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

“How does Python code compare to C code in terms of complexity?”

A

“Code that is short and understandable in Python can become verbose and complicated in C.”

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

“What does porting to C require?”

A

“Such a port requires extensive testing to ensure that the functionality is equivalent to the original Python code and that no bugs have been introduced.”

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

“Is it sometimes worth porting to C?”

A

“Sometimes it’s worth it

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

“What tools ease the transition to C?”

A

“There are even open-source tools such as Cython and Numba that ease the transition to C.”

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

“Is moving one piece to C sufficient?”

A

“The problem is that moving one piece of your program to C isn’t sufficient most of the time.”

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

“What do optimized Python programs usually have as sources of slowness?”

A

“Optimized Python programs usually don’t have one major source of slowness; rather

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

“What would you need to port to get C’s benefits?”

A

“To get the benefits of C’s bare metals and threads

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

“Is there a better way than porting to C?”

A

“There has to be a better way to preserve your investment in Python to solve difficult computation problems.”

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

“What module may be what you need for parallelism?”

A

“The multiprocessing module

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

“What does multiprocessing enable?”

A

“It enables Python to use multiple CPU cores in parallel by running additional interpreters as child processes.”

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

“Are child processes separate from the main interpreter?”

A

“These child processes are separate from the main interpreter

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
22
Q

“What can each child fully utilize?”

A

“Each child can fully utilize one CPU core.”

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
23
Q

“What does each child have?”

A

“Each child has a link to the main process where it receives instructions to do computation and return results.”

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
24
Q

“Can you change ThreadPoolExecutor to ProcessPoolExecutor?”

A

“In some cases

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
25
“What is step 1 of how ProcessPoolExecutor works?”
“It takes each item from the numbers input data to map.”
26
“What is step 2 of how ProcessPoolExecutor works?”
“It serializes items into binary data by using pickle.”
27
“What is step 3 of how ProcessPoolExecutor works?”
“It copies the serialized data from the main interpreter process to a child interpreter process over a local socket.”
28
“What is step 4 of how ProcessPoolExecutor works?”
“It deserializes the data back into Python objects
29
“What is step 5 of how ProcessPoolExecutor works?”
“It imports the Python module using the gcd function.”
30
“What is step 6 of how ProcessPoolExecutor works?”
“It runs the function on the input data in parallel with other child processes.”
31
“What is step 7 of how ProcessPoolExecutor works?”
“It serializes the result back into binary data.”
32
“What is step 8 of how ProcessPoolExecutor works?”
“It copies that binary data back through the socket.”
33
“What is step 9 of how ProcessPoolExecutor works?”
“It deserializes the binary data back into Python objects in the parent process.”
34
“What is step 10 of how ProcessPoolExecutor works?”
“It merges the results from multiple children into a single list to return.”
35
“Does multiprocessing look simple to programmers?”
“Though it looks simple to a programmer
36
“What do you need in most other languages to coordinate threads?”
“In most other languages
37
“Why is the overhead of multiprocessing high?”
“The overhead of using multiprocessing via ProcessPoolExecutor is high because of all the serialization and deserialization that must that must happen between parent and child processes.”
38
“What types of functions is multiprocessing well-suited for?”
“The above scheme is well-suited to certain types of isolated
39
“What does isolated mean?”
“Isolated means functions that don’t need to share state with other parts of the program.”
40
“What does high-leveraged mean?”
“High-leveraged means situations where only a small amount of data must be transmitted between the parent and child processes to enable a large amount of computation.”
41
“What is an example of an algorithm that works well?”
“The greatest common divisor algorithm is one example of this
42
“What happens if your computation doesn’t have these characteristics?”
“If your computation doesn’t have these characteristics
43
“What does multiprocessing provide when ProcessPoolExecutor isn’t enough?”
“When that happens
44
“Are these advanced features complex?”
“But all these features are very complex.”
45
“Why is it hard to reason about these tools?”
“It’s hard enough to reason about tools in the memory space of a single process shared between Python threads.”
46
“What makes this approach much more difficult?”
“Extending that complexity to other processes and involving sockets makes this approach much more difficult to understand.”
47
“What should you initially avoid?”
“You should initially avoid all parts of the multiprocessing module.”
48
“What can you start with?”
“You can start with the ThreadPoolExecutor class to run isolated
49
“What can you move towards later?”
“Later you can move towards theProcessPoolExecutor class to get a speedup.”
50
“When should you consider using multiprocessing directly?”
“Finally
51
“What wall might you hit in Python programs?”
“At some point in writing Python programs
52
“What might happen even after optimizing your code?”
“Even after optimizing your code
53
“What is a reasonable assumption for modern computers?”
“On modern computers that have an increasing number of CPU cores
54
“Does the GIL allow true parallelism in threads?”
“Unfortunately
55
“What is another common suggestion for performance?”
“Another common suggestion is to re-write your performance-critical code as an extension module
56
“What can C do for Python programs?”
“C gets you closer to the bare metal and can run faster than Python
57
“Can C extensions start native threads?”
“C extensions can also start native threads independent of the Python interpreter than run in parallel and utilize multiple CPU cores with no concern for the GIL.”
58
“Is Python’s C extension API well documented?”
“Python’s API for C extensions is well documented and a good choice for an escape hatch.”
59
“What tools can help with C extensions?”
“It’s also worth checking out tools like SWIG and CLIF.”
60
“What is the cost of rewriting in C?”
“However
61
“How does Python code compare to C code in terms of complexity?”
“Code that is short and understandable in Python can become verbose and complicated in C.”
62
“What does porting to C require?”
“Such a port requires extensive testing to ensure that the functionality is equivalent to the original Python code and that no bugs have been introduced.”
63
“Is it sometimes worth porting to C?”
“Sometimes it’s worth it
64
“What tools ease the transition to C?”
“There are even open-source tools such as Cython and Numba that ease the transition to C.”
65
“Is moving one piece to C sufficient?”
“The problem is that moving one piece of your program to C isn’t sufficient most of the time.”
66
“What do optimized Python programs usually have as sources of slowness?”
“Optimized Python programs usually don’t have one major source of slowness; rather
67
“What would you need to port to get C’s benefits?”
“To get the benefits of C’s bare metals and threads
68
“Is there a better way than porting to C?”
“There has to be a better way to preserve your investment in Python to solve difficult computation problems.”
69
“What module may be what you need for parallelism?”
“The multiprocessing module
70
“What does multiprocessing enable?”
“It enables Python to use multiple CPU cores in parallel by running additional interpreters as child processes.”
71
“Are child processes separate from the main interpreter?”
“These child processes are separate from the main interpreter
72
“What can each child fully utilize?”
“Each child can fully utilize one CPU core.”
73
“What does each child have?”
“Each child has a link to the main process where it receives instructions to do computation and return results.”
74
“Can you change ThreadPoolExecutor to ProcessPoolExecutor?”
“In some cases
75
“What is step 1 of how ProcessPoolExecutor works?”
“It takes each item from the numbers input data to map.”
76
“What is step 2 of how ProcessPoolExecutor works?”
“It serializes items into binary data by using pickle.”
77
“What is step 3 of how ProcessPoolExecutor works?”
“It copies the serialized data from the main interpreter process to a child interpreter process over a local socket.”
78
“What is step 4 of how ProcessPoolExecutor works?”
“It deserializes the data back into Python objects
79
“What is step 5 of how ProcessPoolExecutor works?”
“It imports the Python module using the gcd function.”
80
“What is step 6 of how ProcessPoolExecutor works?”
“It runs the function on the input data in parallel with other child processes.”
81
“What is step 7 of how ProcessPoolExecutor works?”
“It serializes the result back into binary data.”
82
“What is step 8 of how ProcessPoolExecutor works?”
“It copies that binary data back through the socket.”
83
“What is step 9 of how ProcessPoolExecutor works?”
“It deserializes the binary data back into Python objects in the parent process.”
84
“What is step 10 of how ProcessPoolExecutor works?”
“It merges the results from multiple children into a single list to return.”
85
“Does multiprocessing look simple to programmers?”
“Though it looks simple to a programmer
86
“What do you need in most other languages to coordinate threads?”
“In most other languages
87
“Why is the overhead of multiprocessing high?”
“The overhead of using multiprocessing via ProcessPoolExecutor is high because of all the serialization and deserialization that must that must happen between parent and child processes.”
88
“What types of functions is multiprocessing well-suited for?”
“The above scheme is well-suited to certain types of isolated
89
“What does isolated mean?”
“Isolated means functions that don’t need to share state with other parts of the program.”
90
“What does high-leveraged mean?”
“High-leveraged means situations where only a small amount of data must be transmitted between the parent and child processes to enable a large amount of computation.”
91
“What is an example of an algorithm that works well?”
“The greatest common divisor algorithm is one example of this
92
“What happens if your computation doesn’t have these characteristics?”
“If your computation doesn’t have these characteristics
93
“What does multiprocessing provide when ProcessPoolExecutor isn’t enough?”
“When that happens
94
“Are these advanced features complex?”
“But all these features are very complex.”
95
“Why is it hard to reason about these tools?”
“It’s hard enough to reason about tools in the memory space of a single process shared between Python threads.”
96
“What makes this approach much more difficult?”
“Extending that complexity to other processes and involving sockets makes this approach much more difficult to understand.”
97
“What should you initially avoid?”
“You should initially avoid all parts of the multiprocessing module.”
98
“What can you start with?”
“You can start with the ThreadPoolExecutor class to run isolated
99
“What can you move towards later?”
“Later you can move towards theProcessPoolExecutor class to get a speedup.”
100
“When should you consider using multiprocessing directly?”
“Finally
101
“What is the next step after writing a useful Python program?”
“Once you’ve written a useful Python program
102
“What is just as important as correct functionality?”
“Making programs dependable when they encounter unexpected circumstances is just as important as making programs with correct functionality.”
103
“What does Python have to help harden programs?”
“Python has built-in features and programs that help you harden your programs so they’re robust in a wide variety of situations.”
104
“What is one dimension of robustness?”
“One dimension of robustness is scalability and performance.”
105
“What will you often see when implementing programs that handle non-trivial data?”
“When you’re implementing Python programs that handle non-trivial amounts of data
106
“What does Python include for high performance?”
“Luckily
107
“How many distinct times might you want to take action during exception handling?”
“There are four distinct times when you might want to take action during exception handling.”
108
“What captures these four distinct times?”
“These are captured in the functionality of try
109
“What does each block serve?”
“Each block serves a unique purpose in the compound statement
110
“When should you use try/finally?”
“Use try/finally when you want exceptions to propagate up but also when you want to write cleanup code even when exceptions occur.”
111
“What is one common usage of try/finally?”
“One common usage of try/finally is for reliably closing file handles.”
112
“When should you call open with finally blocks?”
“If you want to use a finally block with file handles
113
“When should you use try/except/else blocks?”
“Use try/except/else blocks to make it clear which exceptions will be handled by your code and which exceptions propagate up.”
114
“When does the else block run?”
“When the try block doesn’t raise an exception
115
“What does the else block help you do?”
“The else block helps you minimize the amount of code in the try block
116
“How can you ensure exceptions propagate up to the caller?”
“If you want exceptions to propagate up to the caller
117
“When should you use try/except/else/finally?”
“Use try/except/else/finally when you want to do it all in one compound statement.”
118
“What is the with statement used for?”
“The with statement in Python is used to indicate when code is running in a special context.”
119
“Why is using with with try/finally a good strategy?”
“Using with with a try/finally block is a good strategy because it gets rid of the need to write repetitive code.”
120
“How can you make objects and functions work with with statements?”
“It’s easy to make your objects and functions work with with statements by using a the contextlib module.”
121
“What does the contextlib module contain?”
“It contains the contextmanager decorator which lets a simple function be used with with statements.”
122
“How does this compare to the standard way?”
“This is much easier than defining a new class with the special methods **enter** and **exit** (the standard way).”
123
“What happens in a @contextmanager?”
“In a @contextmanager
124
“What may a context manager passed to with return?”
“The context manager passed to a with statement may also return an object.”
125
“How is this object assigned?”
“This object is assigned to a local variable as part of the as compound statement.”
126
“What does this give the code in the with block?”
“This gives the code running within the with block the ability to directly interact with its context.”
127
“Why is using with…as… with open a good idea?”
“Using a with…as… statement with open is a good idea because it highlights a critical section
128
“How can you enable your functions to supply values for as targets?”
“To enable your own functions to supply values for as targets
129
“What is UTC?”
“Coordinated universal time (UTC) is the standard
130
“What does UTC work great for?”
“UTC works great for computers that represent time as seconds since the UNIX epoch.”
131
“Is UTC ideal for humans?”
“But UTC isn’t ideal for humans.”
132
“How do humans reference time?”
“Humans reference time relative to where they’re currently located.”
133
“What will you probably find yourself doing if your program handles time?”
“If your program handles time
134
“How many ways does Python have for time zone conversions?”
“Python has 2 ways of accomplishing time zone conversions.”
135
“What is the old way for time zone conversions?”
“The old way
136
“What is the new way for time zone conversions?”
“The new way
137
“What should you be familiar with regarding time?”
“You should be familiar with time and datetime to understand why datetime is the best choice and why time should be avoided.”
138
“What does the localtime function do?”
“The localtime function in the time module lets you convert a UNIX timestamp (seconds since the UNIX epoch in UTC) to a local time that matches the host computer’s time zone.”
139
“How can local time be printed in human-readable format?”
“The user’s local time can be printed in a human-readable format using the strftime function.”
140
“What will you often need to do besides converting from UTC?”
“You’ll often need to go the other way as well
141
“How can you convert from local time to UTC?”
“You can do this by using the strptime function to parse the time string
142
“Can you directly manipulate return values from time functions?”
“You might assume you can directly manipulate the return values from the time
143
“Is directly manipulating time function returns a good idea?”
“But this is a bad idea.”
144
“Why is managing time zones yourself problematic?”
“Time zones change all the time because of local laws. It’s too complicated to manage yourself
145
“What do operating systems have for time zones?”
“Many operating systems have configuration files that keep up with the time zone changes automatically.”
146
“Can Python use OS time zones?”
“Python lets you use these time zones through the time module supports it.”
147
“Is time zone functionality available on all platforms?”
“On other platforms like Windows
148
“What is the problem with the time module?”
“The problem with the time module is that it’s platform-dependent.”
149
“What determines the time module’s behavior?”
“Its behavior is determined by how the underlying C functions work with the host operating system.”
150
“What does this make the time module?”
“This makes the functionality of the time module unreliable.”
151
“How does time fail with multiple time zones?”
“It fails to to consistently work for multiple time zones.”
152
“When should you use the time module?”
“You should avoid using time
153
“What should you use for other time conversions?”
“For other conversions
154
“What is the second option for representing times?”
“The second option for representing times in Python is the datetime class from the datetime module.”
155
“Can datetime convert from UTC to local time?”
“Like time
156
“Can datetime convert local time back to UTC?”
“The datetime module can convert a local time back to a Unix timestamp in UTC.”
157
“Does datetime have facilities for converting between local times?”
“Unlike the time module
158
“What does datetime provide for time zone operations?”
“However
159
“What is missing from Python’s default installation?”
“The Python default installation is missing time zone definitions besides UTC.”
160
“How has the Python community addressed this gap?”
“The Python community has addressed this gap with the pytz module that’s available from the Python Package Index.”
161
“What does pytz contain?”
“pytz contains a full database of every time zone definition you might need.”
162
“How should you convert time zones effectively with pytz?”
“To convert pytz effectively
163
“What should you do with UTC values?”
“Perform any datetime operations you need on the UTC values.”
164
“What should you do as a final step?”
“Then
165
“Are datetime and pytz conversions consistent?”
“With datetime and pytz
166
“What can the pickle module do?”
“The pickle module can serialize Python objects into a stream of bytes and deserialize bytes back into objects.”
167
“Should pickled byte streams be used between untrusted parties?”
“Pickled byte streams shouldn’t be used to communicate between untrusted parties.”
168
“What is the purpose of pickle?”
“The purpose of pickle is to let you pass Python objects between programs that you control over binary channels.”
169
“Is pickle’s serialization format safe?”
“The pickle module’s serialization format is unsafe by design.”
170
“What does the serialized data contain?”
“The serialized data contains what is essentially a program that describes how to reconstruct the original Python object.”
171
“What could a malicious pickle payload do?”
“This means a malicious pickle payload could be used to compromise any part of a Python program that attempts to deserialize it.”
172
“Is the json module safe?”
“By contrast
173
“What does serialized JSON data contain?”
“Serialized JSON data contains a simple description of an object hierarchy.”
174
“Does deserializing JSON expose programs to risk?”
“Deserializing JSON data does not expose a Python program to additional risk.”
175
“When should JSON be used?”
“Formats like JSON should be used for communication between programs or people who don’t trust each other.”
176
“What can the dump function do?”
“The dump the function can write Python objects to files.”
177
“What can the load function do?”
“The load function can read files and restore Python objects.”
178
“What happens if you update a class used with pickle?”
“If you update a class
179
“Why does this behavior occur?”
“This behavior is a byproduct of the way the pickle module works.”
180
“What is pickle’s primary use case?”
“Its primary use case is making object serialization easy.”
181
“What happens as pickle usage moves beyond trivial?”
“As soon as your use of pickle moves beyond trivial usage
182
“How can you fix pickle problems?”
“Fixing these problems is straightforward using the copyreg module.”
183
“What does copyreg let you do?”
“This module lets you register function the functions responsible for serialization and deserialization Python objects
184
“How can you ensure pickled objects have all attributes?”
“In the simplest case
185
“When might you need versioning with pickle?”
“Sometimes you need to make backwards-incompatible changes to your Python objects by removing fields.”
186
“Does the default argument approach work with removed fields?”
“Doing so prevents the default argument approach from working.”
187
“How can you fix the removed fields problem?”
“You can fix the problem mentioned above by adding a version parameter to the functions supplied by copyreg.”
188
“What other issue might you run into with pickle?”
“One other issue you might run into with pickle is breakage from renaming the class.”
189
“What happens over a program’s life cycle?”
“Often over the life cycle of a program
190
“What does this do to pickle?”
“Unfortunately
191
“What is encoded in pickled data?”
“The import path of a serialized object’s class is encoded in the pickled data.”
192
“How can you solve the renaming problem?”
“The solution to the problem mentioned above is to use copyreg.”
193
“What can you specify with copyreg for unpickling?”
“You can specify a stable identifier for the function to use for unpickling an object.”
194
“What does this allow you to do?”
“This allows you to transition pickled data to different classes with different names when it’s deserialized.”
195
“What does this give you?”
“It gives you level of indirection.”
196
“What is the problem with using this strategy?”
“The only problem with using this strategy is you can’t change the path of the modules you reference.”
197
“What must remain available for deserialization?”
“Once data is serialized with a function it must remain available on the import path for deserialization in the future.”
198
“Is Python a good language for numerical data?”
“Python is a good language for writing code that interacts with numerical data.”
199
“What can Python’s integer type represent?”
“Python’s integer type can represent values of any practical size.”
200
“What standard does Python’s floating point comply with?”
“Its double-precision floating point type complies with the IEEE 754 standard.”
201
“What other type does Python provide?”
“The language also provides a standard complex number type for imaginary values.”
202
“Are these types enough for every situation?”
“However
203
“What is the solution for precision problems?”
“The solution is to use the Decimal class from the decimal module.”
204
“What does the Decimal class provide?”
“The Decimal class provides fixed point math of 28 places by default.”
205
“Can Decimal go higher than 28 places?”
“It can go even higher if you need to.”
206
“What does Decimal work around?”
“This works around the precision issues in IEEE 754 floating point numbers.”
207
“What else does Decimal give you?”
“The class also gives you more control over rounding behaviors.”
208
“How many ways can Decimal instances be given starting values?”
“Decimal instances can be given starting values in two different ways.”
209
“What is the first way to create a Decimal?”
“The first way is by passing a str containing the number to the Decimal constructor.”
210
“Why is the str approach good?”
“This ensures there is no loss of precision due to the inherit nature of Python floating point numbers.”
211
“What is the second way to create a Decimal?”
“The second way is by directly passing a float or an int instance to the constructor.”
212
“Which constructor should you use for exact answers?”
“If you care about exact answers
213
“What function does Decimal have for rounding?”
“The Decimal class has a function for rounding to exactly the decimal place needed with the desired rounding behavior.”
214
“Does Decimal have limitations?”
“While Decimal works great for fixed-point numbers
215
“What does Python’s dynamic nature cause?”
“The dynamic nature of Python causes surprising behaviors in its runtime performance.”
216
“What operations might you assume are slow?”
“Operations you might assume you would be slow are actually very fast (e.g. string manipulation
217
“What features might you assume are fast?”
“Language features you might assume would be fast are actually very slow (e.g. attribute access
218
“Can the source of slowness be obvious?”
“The true source of slowness in a Python program can be obscure.”
219
“What is the best approach to optimization?”
“The best approach is to ignore your intuition and directly measure the performance of a Python program before you try and optimize it.”
220
“What does Python provide for determining execution time?”
“Python provides a built-in profiler for determining which parts of a program are responsible for it’s execution time.”
221
“What does this mean for optimization efforts?”
“This means that you can focus your optimization efforts on the biggest sources of trouble and ignore parts of the problem that don’t impact speed (i.e. follow Amadahl’s law).”
222
“How many built-in profilers does Python provide?”
“Python provides 2 built-in profilers: one that is pure Python (profile) and another that is a C-extension module (cProfile).”
223
“Which profiler is better?”
“The cProfile module is better because of its minimal impact on the performance of your program while it’s being profiled.”
224
“What does the pure-Python profiler do?”
“The pure-Python alternative imposes a high overhead that skews the results.”
225
“What should you make sure when profiling?”
“When profiling a Python program
226
“What should you watch out for when profiling?”
“Watch out for functions that access the network or resources on disk.”
227
“Why might these functions seem to have large impact?”
“These might seem to have a large impact on a program’s execution time because of the slowness of the underlying systems.”
228
“What should you do if your program uses a cache?”
“If your program uses a cache to mask the latency of slow resources like these
229
“How can you extract statistics from cProfile?”
“When you run a test function in cProfile
230
“What do Stats methods let you do?”
“The Stats object’s various methods let you adjust how to sort and select the profiling information to only show the things that matter.”
231
“How is profiling output organized?”
“The output is a table of information organized by function.”
232
“When is the data sample taken?”
“The data sample is taken only from the time the profiler was active.”
233
“What does ncalls mean?”
“ncalls - The number of calls to the function during the profiling period.”
234
“What does tottime mean?”
“tottime - The number of seconds spent executing the function
235
“What does tottime percall mean?”
“tottime percall - The average number of seconds spent in the function each time it is called
236
“What does cumtime mean?”
“cumtime - The cumulative number of seconds spent executing the function
237
“What does cumtime percall mean?”
“cumtime percall - The average number of seconds spent in the function each time it is called
238
“What might you find when profiling a whole program?”
“Sometimes when you’re profiling a whole program
239
“Why does the default profiler option make this hard?”
“The default option from the profiler makes a situation like this hard to understand because it doesn’t show that the utility function is called by many parts of your program.”
240
“What method shows which callers contributed?”
“To deal with the problem mentioned above
241
“What does print_callers print?”
“print_callers prints a profile statistics table that shows the function called on the left and which function was responsible for making the call on the right.”
242
“What is a FIFO queue?”
“A common need for writing programs is a first-in
243
“When is a FIFO queue used?”
“A FIFO queue is used when one function gathers values to process and another function handles them in the order in which they were received.”
244
“What do Python programmers often use as a FIFO queue?”
“Python programmers often use the list module as a FIFO queue.”
245
“What is good about processing items from beginning to end?”
“One good thing about processing the items from the beginning to the end
246
“What do you often want to minimize with producer-consumer queues?”
“When you use producer-consumer queues
247
“What can the consumer then do?”
“The consumer can then process through the backlog of items at a consistent pace
248
“When does using a list work fine?”
“Using a list for a producer-consumer queue works fine up to a point
249
“How does append perform?”
“The append method takes roughly constant time for the list type
250
“Is there overhead for list capacity?”
“There is overhead for the list type to increase its capacity under the covers as new items are added
251
“How does pop(0) perform?”
“The total time for dequeing items from a list with pop(0) scales quadratically as the length of the queue increases.”
252
“Why does pop(0) scale quadratically?”
“This happens because pop(0) needs to move every item to the back of the index
253
“How many operations does consuming a queue require?”
“You need to call pop(0) for every item in the list
254
“Does this scale?”
“This doesn’t scale.”
255
“What does Python provide to solve this problem?”
“Python provides the deque class in the collections module to solve this problem.”
256
“What is deque?”
“deque is a double-ended queue implementation.”
257
“What operations does deque provide?”
“It provides constant time operations for inserting or removing items from its beginning or end.”
258
“What does this make deque ideal for?”
“This makes it ideal for FIFO queues.”
259
“How does popleft scale?”
“The popleft function scales linearly instead of scaling superlinearly like pop(0).”
260
“When is deque a great choice?”
“If you know that the performance of a critically depends on the speed of producer-consumer queues
261
“What if you’re not sure about performance?”
“If you’re not sure
262
“What is common to find yourself with?”
“It’s common to find yourself with a large amount of data in memory as a sorted list that you want to search.”
263
“How long does searching with index take?”
“Regardless of the data your specific program needs to process
264
“What might you want to search for?”
“If you’re not sure whether a specific value is in a list
265
“What is the simplest way to find closest index?”
“The simplest way to do this is to linearly scan the list and compare each item to your goal value.”
266
“What does the bisect module provide?”
“Python’s bisect module provides better ways to accomplish these kinds of searches through ordered lists.”
267
“What function can you use for efficient binary search?”
“You can use the bisect_left function to do any efficient binary search through any sequence of sorted items.”
268
“What index does bisect_left return?”
“The index it returns will either be where the item was already present in the list or where you’d want to insert the item in the list to keep it in sorted order.”
269
“What is the complexity of bisect’s algorithm?”
“The complexity of the binary search algorithm that the bisect module uses is logarithmic.”
270
“How does bisect compare to linear search?”
“That means that searching a list with a million items takes roughly the same amount of time with bisect as linearly searching a list with 20 items with list.index.”
271
“How can you perform speed tests?”
“You can perform speed tests with the timeit module.”
272
“Is bisect limited to list types?”
“The best part about bisect that it’s not limited to the list type: you can use it with any Python object that acts like a sequence.”
273
“Does bisect provide additional features?”
“The module provides additional features for more advanced situations.”
274
“What is one limitation of Python’s queue implementations?”
“One of the limitations of Python’s other queue implementations is that they are first-in
275
“What do you often need instead of FIFO?”
“Often
276
“What is the right tool for processing by importance?”
“To accomplish this
277
“What increases as you look through a list?”
“If you have to look through a list
278
“What does heapq solve?”
“Python’s heapq module solves list’s performance problems by implementing efficient priority queues.”
279
“What is a heap?”
“A heap is a data structure that allows for a list of items to be maintained where the computationally complexity (i.e. even better than linear scaling).”
280
“Do you need to understand heap implementation?”
“A good part of the module is that you don’t have to understand how heaps are implemented in order to use its functions.”
281
“What does heapq require?”
“The heapq module requires items in the priority queue to be comparable and have a natural sorting order.”
282
“How can you quickly give classes comparable behavior?”
“One way to quickly give some classes this behavior is by using the totat_ordering class decorator from the functools module and implementing the **lt** special method.”
283
“What is another way to create a heap?”
“Another way to create a heap is to create a list with the items in any order and then use the sort method.”
284
“What is yet another way to create a heap?”
“Another way to create a heap is to use heapq.heapify function to create a heap in linear time.”
285
“How do heap-based implementations scale?”
“Heap-based priority queue implementations scale better than list-based ones and don’t linearly degrade performance.”
286
“What is a downside of heap-based queues?”
“One of the downsides to a heap-based priority queue is that it might use significant memory even though it is fast because of heapq.”
287
“What should you plan for in robust systems?”
“If you’re trying to build a robust system
288
“What should you plan for with memory?”
“You should plan for your program using a huge memory cost and mitigated it with various constraints.”
289
“What else does heapq provide?”
“Beyond creating priority queues
290
“When is heapq a great choice?”
“The module is a great choice when its functionality matches the problem you’re facing (e.g. the queue.PriorityQueue class is another thread-safe option).”
291
“Can Python parallelize CPU-bound computation easily?”
“Although Python isn’t able to parallelize CPU-bound computation without extra effort
292
“Is it easy to use Python’s I/O tools wrong?”
“That said
293
“What is a good way to work with bytes?”
“A good way to work with bytes is to use Python’s memoryview type
294
“What is the buffer protocol?”
“The buffer protocol is a low-level C API that allows the Python runtime and C extensions to access the underlying data buffers that are behind objects like bytes instances.”
295
“What is the best part of memoryview instances?”
“The best part of memoryview instances is that slicing them results in another memoryview instance without copying the underlying data.”
296
“What can memoryview provide?”
“By enabling zero-copy operations
297
“What is a good way to write code with bytes?”
“A good way to write code is to use Python’s bytearray type on conjunction with memoryview.”
298
“What is one limitation with bytes instances?”
“One limitation with bytes instances is that they are read-only and don’t allow for individual indexes to be updated.”
299
“What is the bytearray type?”
“The bytearray type is like a mutable version of bytes that allows for arbitrary positions to be overwritten.”
300
“What does bytearray use for its values?”
“bytearray uses integers for its values instead of bytes.”
301
“Can memoryview wrap a bytearray?”
“A memoryview can also be used to wrap a bytearray.”
302
“What can you do when you slice a memoryview wrapping bytearray?”
“When you slice such a memoryview
303
“What does this eliminate?”
“This eliminates the copying costs from above that are required to splice bytes instances back together if they are separated.”
304
“Does Python have compile-time static type checking?”
“No
305
“Is there anything in the Python interpreter that will ensure your program will work correctly when you run it?”
“No
306
“Does Python support optional type annotations?”
“Yes
307
“What kind of language is Python fundamentally?”
“Python is still fundamentally a dynamic language
308
“With Python
do you know if the functions your program calls will be defined at runtime?”
309
“Is Python’s dynamic behavior both a blessing and a curse?”
“Yes
310
“What do a lot of Python programmers say about going without compile-time static type checking?”
“A lot of Python programmers say it’s worth going without compile-time static type checking because of the productivity gained from brevity and simplicity.”
311
“Do most people have at least one horror story about a Python program encountering a runtime error?”
“Yes
312
“Is compile-time static type safety everything?”
“No
313
“Should you always test your code regardless of what language it’s in?”
“Yes
314
“Is it more important to write tests in Python than in other languages?”
“Yes
315
“Do the same dynamic features that create risks also make testing easier?”
“Yes
316
“Can you use Python’s dynamic nature to implement tests?”
“Yes
317
“What should you think of tests as?”
“You should think of tests as an insurance policy on your code.”
318
“What do good tests give you?”
“Good tests give you confidence that your code is correct.”
319
“What makes it easy to identify what changed when you refactor or expand your code?”
“Behavior (not implementation) makes it easy to identify what changed.”
320
“Does having good tests make it easier or harder to modify Python code?”
“It seems counterintuitive
321
“When debugging a Python program
what will get you surprisingly far?”
322
“Are Python internals often easy to access?”
“Yes
323
“What do you need to do to see how the state of your program changes while it runs?”
“All you need to do is call print to see how the state of your program changes while it runs and understand where it goes wrong.”
324
“What does the print function output?”
“The print function outputs a human-readable string of whatever you supply to it.”
325
“What is calling print on something equivalent to (first option)?”
“Calling the str function before passing the value to print.”
326
“What is calling print on something equivalent to (second option)?”
“Using the %s format string with the % operator.”
327
“What is calling print on something equivalent to (third option)?”
“Default formatting of the value with a debt string.”
328
“What is calling print on something equivalent to (fourth option)?”
“Calling the format function.”
329
“What is calling print on something equivalent to (fifth option)?”
“Explicitly calling the **format** special method.”
330
“What is calling print on something equivalent to (sixth option)?”
“Explicitly calling the **str** special method.”
331
“What is the problem with printing a human-readable string for a value?”
“The problem with printing a human-readable string for a value doesn’t make it clear what the actual type and its specific composition are.”
332
“If you’re debugging a program with print
do type differences matter?”
333
“What do you almost always want while debugging?”
“What you almost always want while debugging is to see the repr function of an object.”
334
“What does the repr function return?”
“The repr function returns a printable representation of an object
335
“For most built-in types
is the string returned by repr a valid Python expression?”
336
“What should happen when you pass a value from repr to eval?”
“Passing a value from repr to eval should result in the same Python object you started with.”
337
“Should you use eval with caution?”
“Yes
338
“When you’re debugging with print
what should you do to ensure any difference in type is clear?”
339
“What is calling print(repr()) on something equivalent to?”
“Calling print(repr()) on something is equivalent to using the %r format string with the % operator or an f-string with the !r type conversion.”
340
“For instances of Python classes
is the default human-readable string value the same as the repr value?”
341
“Does this mean you need to explicitly call repr when passing an instance to print?”
“No
342
“Is the default implementation of repr for object subclasses helpful?”
“Unfortunately
343
“Why isn’t the default repr implementation helpful for object subclasses?”
“This is because you can’t pass it to eval
344
“How many solutions are there to the unhelpful default repr problem?”
“There are 2 solutions to the problem mentioned above.”
345
“What can you do if you have control of the class to fix repr?”
“If you have control of the class
346
“What can you do when you don’t have control over the class definition?”
“When you don’t have control over the class definition
347
“What is the canonical way to write tests in Python?”
“The canonical way to write tests in Python is to use the unittest module.”
348
“What should you create to define tests?”
“To define tests
349
“Is the test file naming scheme a style choice?”
“Yes
350
“What does the test file contain?”
“The file contains tests for each behavior that you expect.”
351
“How are tests organized?”
“Tests are organized into TestCase subclasses.”
352
“What is each test case?”
“Each test case is a method beginning with the word test.”
353
“When is a test considered to have passed successfully?”
“If a test method runs without any kind of Exception (including AssertionError from assert statements)
354
“What happens if one test fails in a TestCase subclass?”
“If one test fails
355
“Why does the TestCase subclass continue running other tests after one fails?”
“This lets you get a full picture of how all your tests are doing instead of stopping at the first sign of trouble.”
356
“How can you run only a specific test method?”
“If you want to iterate quickly and to fix or improve a specific test
357
“Can you invoke the debugger from within test methods?”
“Yes
358
“What does the TestCase class provide?”
“The TestCase class provides helper methods for making assertions in your tests.”
359
“What is assertEqual used for?”
“assertEqual is used for verifying equality.”
360
“What is assertTrue used for?”
“assertTrue is used for verifying Boolean expressions.”
361
“Why are TestCase helper methods better than built-in assert statements?”
“These are better than built-in assert statements because they print out all of the inputs and outputs to help you understand the exact reason the test is failing.”
362
“What is the assertRaises helper method for?”
“There’s also an assertRaises helper method for verifying exceptions.”
363
“Can assertRaises be used as a context manager?”
“Yes
364
“What does assertRaises make clear?”
“This appears similar to a try/except statement and makes it abundantly clear where an exception needs to be raised.”
365
“Can you define your own helper methods in TestCase subclasses?”
“Yes
366
“What should you ensure about your custom helper method names?”
“Just ensure that your method names don’t begin with the word test
367
“What do custom test helpers often use besides TestCase assertion methods?”
“In addition to calling TestCase assertion methods
368
“When might it be a good idea to define one TestCase subclass?”
“It might be a good idea to define one TestCase subclass for each set of related tests.”
369
“When might you need to have one TestCase subclass per function?”
“You also might need to have one TestCase subclass for each function that has a lot of edge cases.”
370
“Can a TestCase subclass span all functions in a single module?”
“Yes
371
“Can you make a TestCase subclass that defines an entire module?”
“Yes
372
“Can you create a TestCase subclass for each basic class and all of its methods?”
“Yes
373
“What does the subTest helper method do?”
“The TestCase class also provides a subTest helper method that lets you avoid boilerplate by defining multiple tests within a single test method.”
374
“When is subTest especially helpful?”
“This is especially helpful for writing data-driven tests.”
375
“What does subTest allow the test method to do?”
“It allows the test method to continue testing other cases even after one of them fails.”
376
“When can pytest be especially helpful?”
“Depending on your project’s complexity and testing requirements
377
“Do TestCase classes often need to have the test environment set up before test methods can be run?”
“Yes
378
“What is the test environment setup sometimes called?”
“This is sometimes called a test harness.”
379
“How do you set up the test environment?”
“To do this
380
“When are setUp and tearDown methods called?”
“These methods are called before and after each test method
381
“What does overriding setUp and tearDown let you ensure?”
“This lets you ensure that each test runs in isolation.”
382
“Is running tests in isolation an important best practice?”
“Yes
383
“When your program gets complicated
what kind of tests will you want?”
384
“What is the difference between unit tests and integration tests?”
“This is the difference between unit tests and integration tests.”
385
“Why is it important to write both types of tests in Python?”
“In Python
386
“What is one common problem with integration tests?”
“One common problem is that setting up your test environment for integration tests is computationally expensive and may require a lot of wall-clock time.”
387
“What does the unittest module support to handle expensive test setup?”
“To handle this situation
388
“What can you do with module-level test harness initialization?”
“You can configure expensive resources a single time.”
389
“What happens after you configure expensive resources once?”
“Then have all TestCase classes and their test methods run without repeating that initialization.”
390
“When can the test harness be torn down?”
“Later
391
“What is another common need when writing tests?”
“Another common need when writing tests is to use mocked functions and classes to simulate when it’s too difficult or slow to use the real thing.”
392
“Is setting up components for testing a lot of work?”
“Yes
393
“What will the long setup time do to the tests?”
“The long time it will take to set them up will slow down the tests and make them harder to maintain as well.”
394
“What is a good approach to solve the component setup problem?”
“A good approach to solve the problem mentioned above is to use a mock.”
395
“What does a mock let you do?”
“A mock lets you provide expected responses for dependent functions
396
“Should you confuse mocks with fakes?”
“No
397
“What would a fake provide?”
“A fake would provide most of the behavior of the component but with a simpler implementation.”
398
“What is an example of a fake?”
“Example of a fake: a basic in-memory
399
“What does the Mock class create?”
“The Mock class creates a mock function.”
400
“What is the return_value attribute of the mock?”
“The return_value attribute of the mock is the value to return when it’s called.”
401
“What does the spec argument indicate?”
“The spec argument indicates that a mock should act like the given object
402
“What can you do once you create a mock?”
“Once you create a mock
403
“How do you know if the code that called the mock provided the correct arguments?”
“To know if the code that called the mock provided the correct arguments
404
“What does assert_called_once_with verify?”
“assert_called_once_with verifies that a single call with exactly the given parameters was made.”
405
“What happens if the wrong parameters are supplied to assert_called_once_with?”
“If the wrong parameters are supplied
406
“What happens to any TestCase that uses the assertion with wrong parameters?”
“Any TestCase that uses the assertion fails if wrong parameters are supplied.”
407
“How can you indicate that any value is good for an argument?”
“If you don’t care about some of the individual parameters
408
“What method verifies that the mock’s most recent call matches your expectations?”
“You can also use Mock’s assert_called_with method to verify that the mock’s most recent call matches your expectations.”
409
“When is ANY useful in tests?”
“ANY is useful in tests when a parameter is not core to the behavior that’s being tested.”
410
“Should you use ANY more liberally or conservatively?”
“It’s often worth erring on the side of under-specifying tests by using ANY more liberally instead of over-specifying tests.”
411
“What does over-specifying tests mean?”
“Over-specifying tests means having to sift through the various test parameter expectations.”
412
“Does Mock make it easy to mock exceptions being raised?”
“Yes
413
“Are there many more features available in Mock?”
“Yes
414
“What should you look at for the full range of Mock options?”
“You should take a look at Mock’s documentation for the full range of options.”
415
“What is one approach to get functions to use mock dependent ones instead of real ones?”
“One approach is to inject everything as keyword-only arguments.”
416
“What do you need to do to use the keyword-only arguments strategy?”
“To use this strategy
417
“What can you use to verify that a function had made 2 calls?”
“You can use the unittest.mock.call helper and the assert_has_calls method to verify that a function had made 2 calls.”
418
“Does the strategy of using keyword-only arguments for injecting mocks work?”
“Yes
419
“What makes injecting mocks easier than keyword-only arguments?”
“The unittest.mock.patch family of functions makes injecting mocks easier.”
420
“What does patch do?”
“patch temporarily reassigns an attribute of a module or class.”
421
“What does patch work for?”
“patch works for many modules
422
“How can patch be used (three ways)?”
“It can be used with with statements as a function decorator
423
“Does patch work in all cases?”
“No
424
“What is an example of when patch doesn’t work?”
“For example
425
“What is one solution when patch doesn’t work?”
“One solution to the problem mentioned above
426
“What is another strategy when you can’t use patch for everything?”
“Another strategy is to use a keyword-only argument for the field you can’t mock to inject the field you can’t mock
427
“What function can you use to create many mocks and set their expectations?”
“You can use the patch.multiple function to create many mocks and set their expectations.”
428
“What do the keyword arguments that get passed to patch.multiple correspond to?”
“The keyword arguments that get passed to patch.multiple correspond to names in a module that you want to get overridden during the test.”
429
“What does the DEFAULT value indicate in patch.multiple?”
“The DEFAULT value that you set for the keyword arguments (kwarg=DEFAULT) indicates that you want a standard Mock instance to be created for each name.”
430
“Will the generated mocks adhere to the specification of the objects they simulate?”
“Yes
431
“Will the mocks work as expected with patch.multiple?”
“Yes
432
“Is it possible to further improve test readability when using patch.multiple?”
“Yes
433
“What does using unittest.mock module to write tests with complex dependencies result in?”
“Using the unittest.mock module to write tests that have complex dependencies results in a lot of boilerplate.”
434
“What could boilerplate make more difficult?”
“This could make it more difficult for new readers of your code to understand what the tests are trying to verify.”
435
“What is one way to improve tests with complex dependencies?”
“One way to improve tests like this is to use a wrapper object to encapsulate an object’s interface instead of passing an instance to one of your functions as an argument.”
436
“Why is it worth refactoring your code to use better abstractions?”
“It’s often worth refactoring your code to use better abstractions because it facilitates creating mocks and writing tests.”
437
“What does the Mock class return for any attribute name accessed?”
“The Mock class returns a mock object for any attribute name’s accessed.”
438
“Can those attributes be called like methods?”
“Yes
439
“What does this enable you to do?”
“This enables you to use them to set expectations and verify calls.”
440
“What does this make easy?”
“This makes it easy to mock out all the methods of a class.”
441
“When is using the spec parameter to Mock especially useful?”
“Using the spec parameter to Mock is especially useful when mocking classes.”
442
“What does using spec ensure?”
“It ensures that the code under test doesn’t call a misspelled method name by accident.”
443
“What common pitfall does this allow you to avoid?”
“This allows you to avoid a common pitfall where the same bug is present in both the code and the unit test.”
444
“What does this pitfall mask?”
“This pitfall masks a real error that will later reveal itself in production.”
445
“How can you inject mock instances in a program for end-to-end testing?”
“If you want to test a program end-to-end with a mid-level integration test
446
“Does everyone encounter bugs in code while developing programs?”
“Yes
447
“Can using the print function help track down issues?”
“Yes
448
“Is writing tests for specific cases that cause trouble a great way to isolate problems?”
“Yes
449
“Are the tools mentioned above enough to find every root cause?”
“No
450
“What should you try when you need something more powerful?”
“When you need something more powerful
451
“What does the debugger let you do (three things)?”
“The debugger lets you inspect program state
452
“How do you use a debugger in most other programming languages?”
“In most other programming languages
453
“What is the easiest way to use the debugger in Python?”
“In contrast
454
“Is there a difference between starting a Python program to run the debugger and starting it normally?”
“No
455
“What do you have to do to initiate the debugger?”
“To initiate the debugger
456
“What is calling the breakpoint function equivalent to?”
“This is equivalent to importing the pdb module and running the set_trace function.”
457
“What happens as soon as the breakpoint function runs?”
“As soon as the breakpoint function runs
458
“What happens to the terminal that started the program?”
“The terminal that started the program turns into an interactive Python shell.”
459
“What can you do at the (pdb) prompt?”
“At the (pdb) prompt
460
“What is another way to print variable values at the (pdb) prompt?”
“You can also use p to print variable values.”
461
“How can you see a list of all local variables?”
“You can see a list of all local variables by calling the locals function.”
462
“What else can you do at the (pdb) prompt (five things)?”
“You can import modules
463
“Does the debugger have special commands?”
“Yes
464
“How can you see the full list of debugger commands?”
“You can type help to see the full list of commands.”
465
“What does the ‘where’ command do?”
“where: Prints the current execution call stack; this lets you figure out where you are in your program and how you arrived at the breakpoint trigger.”
466
“What does the ‘up’ command do?”
“up: Move your scope up the execution call stack to the caller of the current function; this allows you to inspect local variables in higher levels of the program that led to the breakpoint.”
467
“What does the ‘down’ command do?”
“down: Move your scope back down the execution call stack one level.”
468
“What does the ‘step’ command do?”
“step: Run the program until the next line of execution in the program
469
“What does the ‘next’ command do?”
“next: Run the program until the next line of execution in the current function
470
“What does the ‘return’ command do?”
“return: Run the program until the current function returns
471
“What does the ‘continue’ command do?”
“continue: Continue running the program until the next breakpoint is hit (either through the breakpoint call or one added by the debugger command).”
472
“What does the ‘quit’ command do?”
“quit: Exit the debugger and end the program; run this command if you’ve found the problem
473
“Can the breakpoint function be called anywhere in the program?”
“Yes
474
“What can you do if you know the problem happens only under special circumstances?”
“If you know that the problem you’re trying to debug happens only under special circumstances
475
“What is another useful way to reach the debugger prompt?”
“Another useful way to reach the debugger prompt is to use the post-mortem debugger.”
476
“What does the post-mortem debugger enable you to do?”
“This enables you to debug a program after it’s already raised an exception and crashed.”
477
“When is the post-mortem debugger especially helpful?”
“This is especially helpful when you’re not quite sure where to put the breakpoint.”
478
“What command can you use to run the program under the control of the pdb module?”
“You can use the command python3 -m pdb -c continue to run the program under the control of the pdb module.”
479
“What does the continue command tell pdb to do?”
“The continue command tells pdb to get the program started immediately.”
480
“What happens once the program is running under pdb control?”
“Once it’s running
481
“What can you do at that point?”
“At which point you can inspect the program state.”
482
“How can you use post-mortem debugging after an uncaught exception in the interactive Python interpreter?”
“You can also use post-mortem debugging after hitting an uncaught exception in the interactive Python interpreter by calling the pm function of the pdb module.”
483
“How is this often done in a single line?”
“This is often done in a single line as import pdb; pdb.pm().”
484
“What does memory management in CPython use?”
“Memory management in the default implementation of Python (CPython) used reference counting.”
485
“What does reference counting ensure?”
“This ensures that as soon as all references to an object have expired
486
“What does clearing memory free up?”
“This frees up that space for other data.”
487
“Does CPython have a built-in cycle detector?”
“Yes
488
“In theory
do Python programmers have to worry about allocating or deallocating memory?”
489
“Who takes care of memory management automatically?”
“It’s taken care of automatically by the language and the CPython runtime.”
490
“In practice
what happens with memory?”
491
“Is figuring out where a Python program is using or leaking memory easy?”
“No
492
“What is the first way to debug memory usage?”
“The first way to debug memory usage is to ask the gc module to list every object currently known by the garbage collector.”
493
“Is gc.get_objects a precise tool?”
“No
494
“What is the problem with gc.get_objects?”
“The problem with gc.get_objects is that it doesn’t tell you how the objects were allocated.”
495
“In complicated programs
how could objects of a specific class be allocated?”
496
“What is more important than knowing the overall number of objects?”
“Knowing the overall number of objects isn’t nearly as important as identifying the code for responsible for allocating the objects that are leaking memory.”
497
“When was the tracemalloc module introduced?”
“Python 3.4 introduced a new tracemalloc module for solving this problem.”
498
“What does tracemalloc make possible?”
“tracemalloc makes it possible to connect an object back to where it was allocated.”
499
“How do you use tracemalloc?”
“You use it by taking before and after snapshots of memory usage and comparing them to see what’s changed.”
500
“What will the size and count labels make clear?”
“The size and count labels that get printed will make it immediately clear which objects are dominating your program’s memory usage and where in the source code they were allocated.”
501
“Can tracemalloc print out the full stack trace of each allocation?”
“Yes
502
“How do you specify the number of frames to track?”
“You can specify the number of frames to track by passing it to the tracemalloc.start function.”
503
“What is a stack trace most valuable for?”
“A stack trace is the most valuable for figuring out which particular usage of a common function or class is responsible for memory consumption in a program.”