# Serious Python  ## Metadata - Author: [[Julien Danjou]] - Full Title: Serious Python - Category: #python #programming-best-practices ## Highlights - consider your project structure, which should be fairly simple. Use packages and hierarchy wisely: a deep hierarchy can be a nightmare to navigate, while a flat hierarchy tends to become bloated. ([Location 273](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=273)) - avoid making the common mistake of storing unit tests outside the package directory. ([Location 274](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=274)) - The standard name for a Python installation script is setup.py. It comes with its companion setup.cfg, which should contain the installation script configuration details. When run, setup.py will install your package using the Python distribution utilities. ([Location 280](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=280)) - Finally, the docs directory should contain the package’s documentation in reStructuredText format, which will be consumed by Sphinx ([Location 283](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=283)) - The following top-level directories also frequently appear: etc for sample configuration files tools for shell scripts or related tools bin for binary scripts you’ve written that will be installed by setup.py ([Location 288](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=288)) - For example, they might create functions.py or exceptions.py files. This is a terrible approach and doesn’t help any developer when navigating the code. ([Location 293](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=293)) - Organize your code based on features, not on types. It is also a bad idea to create a module directory that contains only an __init__.py file, because it’s unnecessary nesting. ([Location 296](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=296)) - Python requires an __init__.py file to be present for the directory to be considered a submodule. ([Location 304](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=304)) - PEP 8: Style Guide for Python Code (https://www.python.org/dev/peps/pep-0008/). This document defines the standard style for writing Python code. The list of guidelines boils down to: Use four spaces per indentation level. Limit all lines to a maximum of 79 characters. Separate top-level function and class definitions with two blank lines. Encode files using ASCII or UTF-8. Use one module import per import statement and per line. Place import statements at the top of the file, after comments and docstrings, grouped first by standard, then by third party, and finally by local library imports. Do not use extraneous whitespaces between parentheses, square brackets, or braces or before commas. Write class names in camel case (e.g., CamelCase), suffix exceptions with Error (if applicable), and name functions in lowercase with words and underscores (e.g., separated_by_underscores). Use a leading underscore for _private attributes or methods. ([Location 344](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=344)) - First, you need to know that the import keyword is actually a wrapper around a function named __import__. ([Location 473](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=473)) - Don’t forget that modules, once imported, are essentially objects whose attributes (classes, functions, variables, and so on) are objects. ([Location 491](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=491)) - The sys.modules variable is a standard Python dictionary that contains all loaded modules. That means that calling sys.modules.keys(), for example, will return the complete list of the names of loaded modules. ([Location 501](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=501)) - When importing modules, Python relies on a list of paths to know where to look for the module. This list is stored in the sys.path variable. To check which paths your interpreter will search for modules, just enter sys.path. ([Location 506](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=506)) - It’s important to note that the list will be iterated over to find the requested module, so the order of the paths in sys.path is important. It’s useful to put the path most likely to contain the modules you are importing early in the list to speed up search time. Doing so also ensures that if two modules with the same name are available, the first match will be picked. ([Location 518](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=518)) - The meta path finder is an object that will allow you to load custom objects as well as standard .py files. A meta path finder object must expose a find_module(fullname, path=None) method that returns a loader object. ([Location 535](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=535)) - No matter how useful an external library might be, be wary of letting it get its hooks into your actual source code. Otherwise, if something goes wrong and you need to switch libraries, you might have to rewrite huge swaths of your program. A better idea is to write your own API—a wrapper that encapsulates your external libraries and keeps them out of your source code. ([Location 672](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=672)) - The egg-link file contains the path to add to sys.path to look for packages. ([Location 712](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=712)) - The main difference between frameworks and external libraries is that applications use frameworks by building on top of them: your code will extend the framework rather than vice versa. ([Location 735](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=735)) - I always recommend designing libraries and APIs from the top down, applying design criteria such as the Single Responsibility Principle (SRP) at each layer. Think about what the caller will want to do with the library and create an API that supports those features. ([Location 827](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=827)) - Think about what values can be stored in an instance and used by the methods versus what needs to be passed to each method every time. ([Location 828](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=828)) - I almost always suggest converting filtering loops like these into generator expressions, which are more efficient and easier to understand. ([Location 837](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=837)) - I recommend that your project documentation always include the following: The problem your project is intended to solve, in one or two sentences. The license your project is distributed under. If your software is open source, you should also include this information in a header in each code file; just because you’ve uploaded your code to the Internet doesn’t mean that people will know what they’re allowed to do with it. A small example of how your code works. Installation instructions. Links to community support, mailing list, IRC, forums, and so on. A link to your bug tracker system. A link to your source code so that developers can download and start delving into it right away. ([Location 876](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=876)) - Read the Docs (http://readthedocs.org/) allows you to build and publish your documentation online automatically. ([Location 887](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=887)) - You can get Sphinx from http://www.sphinx-doc.org/. There are installation instructions on the site, but the easiest method is to install with pip install sphinx. ([Location 892](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=892)) - Your documentation begins with the index.rst file, but it doesn’t have to end there: reST supports include directives to include reST files from other reST files, so there’s nothing stopping you from dividing your documentation into multiple files. ([Location 917](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=917)) - For example, sphinx.ext.autodoc extracts reST-formatted docstrings from your modules and generates .rst files for inclusion. ([Location 924](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=924)) - The autodoc extension gives you the power to include most of your documentation in your source code. ([Location 945](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=945)) - you’re writing a Python library, you’ll usually want to format your API documentation with a table of contents containing links to individual pages for each module. The sphinx.ext.autosummary module was created specifically to handle this common use case. ([Location 949](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=949)) - Another useful feature of Sphinx is the ability to run doctest on your examples automatically when you build your documentation. The standard Python doctest module searches your documentation for code snippets and tests whether they accurately reflect what your code does. ([Location 962](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=962)) - Every paragraph starting with the primary prompt >>> is treated as a code snippet to test. For example, if you wanted to document the standard print function from Python, you could write this documentation snippet and doctest would check the result:     To print something to the standard output, use the :py:func:`print` function: >>> print("foobar")     foobar ([Location 965](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=965)) - You can also use doctest for documentation-driven development (DDD): write your documentation and examples first and then write code to match your documentation. ([Location 973](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=973)) - Sphinx extension called sphinxcontrib-pecanwsme that analyzes docstrings and actual Python code to generate REST API documentation automatically. ([Location 993](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=993)) - For other HTTP frameworks, such as Flask, Bottle, and Tornado, you can use sphinxcontrib.httpdomain. ([Location 996](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=996)) - To distinguish between public and private APIs, the Python convention is to prefix the symbol for a private API with an underscore: foo is public, but _bar is private. You should use this convention both to recognize whether another API is public or private and to name your own APIs. In contrast to other languages, such as Java, Python does not enforce any restriction on accessing code marked as private or public. The naming conventions are just to facilitate understanding among programmers. ([Location 1035](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=1035)) - Your document should cover the following: New elements of the new interface Elements of the old interface that are deprecated Instructions on how to migrate to the new interface ([Location 1052](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=1052)) - The triple quotes here, """, indicate the start and end of the docstrings, which will be pulled into the documentation when the user enters help(Car.turn_left) into the terminal or extracts the documentation with an external tool such as Sphinx. ([Location 1070](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=1070)) - Though deprecated modules should be marked well enough in documentation that users will not attempt to call them, Python also provides the warnings module, which allows your code to issue various kinds of warnings when a deprecated function is called. ([Location 1083](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=1083)) - However, manually writing all those warnings, docstring updates, and so on can become tedious, so the debtcollector library has been created to help automate some of that. The debtcollector library provides a few decorators that you can use with your functions to make sure the correct warnings are emitted and the docstring is updated correctly. ([Location 1126](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=1126)) - Don’t make it too complicated. Keep it simple. Complicated APIs are hard to understand and hard to document. ([Location 1153](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=1153)) - Make the magic visible. When your API does things that your documentation doesn’t explain, your end users will want to crack open your code and see what’s going on under the hood. ([Location 1157](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=1157)) - Don’t forget use cases. When you’re so focused on writing code, it’s easy to forget to think about how your library will actually be used. ([Location 1160](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=1160)) - Write unit tests. TDD (test-driven development) is a very efficient way to write libraries, especially in Python, because it forces the developer to assume the role of the end user from the very beginning, which leads the developer to design for usability. ([Location 1162](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=1162)) - Document early and include your documentation build in continuous integration. ([Location 1184](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=1184)) - Use docstrings to document classes and functions in your API. ([Location 1187](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=1187)) - Give practical examples throughout. Have at least one “startup guide” that will show newcomers how to build a working example. ([Location 1189](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=1189)) - Document the evolution of your API in detail, version by version. ([Location 1192](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=1192)) - Make your documentation accessible and, if possible, comfortable to read. ([Location 1193](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=1193)) - Finally, choose a theme that is both efficient and attractive. ([Location 1196](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=1196)) - You also must be careful of making any kind of time zone conversion before storing your timestamps. ([Location 1214](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=1214)) - The datetime.datetime object can be either time zone aware, in which case it embeds time zone information, or time zone unaware, in which case it does not. ([Location 1221](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=1221)) - The datetime API always returns unaware datetime objects by default, and since there is no way for you to tell what the time zone is from the output, these objects are pretty useless. ([Location 1247](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=1247)) - When you are building datetime objects, I strongly recommend that you always make sure they are time zone aware. ([Location 1252](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=1252)) - Python developers rely on the dateutil project to obtain tzinfo classes. ([Location 1264](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=1264)) - You can install dateutil using pip with the command pip install python-dateutil. The dateutil API allows you to obtain a tzinfo object based on a time zone name, ([Location 1269](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=1269)) - With the setup.py file as the root of a project, all users have to do to build or install your software is run that file with the appropriate command as its argument. Even if your distribution includes C modules in addition to native Python ones, distutils can handle them automatically. ([Location 1418](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=1418)) - One of the notable successors is the packaging library known as setuptools, which offers more frequent updates and advanced features, such as automatic dependency handling, the Egg distribution format, and the easy_install command. ([Location 1422](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=1422)) - Eventually, development on setuptools slowed down too, but it wasn’t long before another group of developers forked it to create a new library called distribute, which offered several advantages over setuptools, including fewer bugs and Python 3 support. ([Location 1441](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=1441)) - distutils is part of the Python Standard Library and can handle simple package installations. setuptools, the standard for advanced package installations, was at first deprecated but is now back in active development and the de facto standard. distribute has been merged back into setuptools as of version 0.7; distutils2 (aka packaging) has been abandoned. distlib might replace distutils in the future. ([Location 1462](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=1462)) - One of the advantages of Wheel is that its naming conventions allow you to specify whether your distribution is intended for a specific architecture and/or Python implementation (CPython, PyPy, Jython, and so on). This is particularly useful if you need to distribute modules written in C. ([Location 1562](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=1562)) - The easiest way to visualize the entry points available in a package is to use a package called entry point inspector. You can install it by running pip install entry-point-inspector. ([Location 1674](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=1674)) - The epi tool is just a thin layer on top of the complete Python library pkg_resources. ([Location 1698](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=1698)) - Helping us circumvent these problems, setuptools offers the console_scripts feature. This entry point can be used to make setuptools install a tiny program in the system path that calls a specific function in one of your modules. ([Location 1710](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=1710)) - Using console scripts is a technique that removes the burden of writing portable scripts, while ensuring that your code stays in your Python package and can be imported (and tested) by other programs. ([Location 1752](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=1752)) - The ExtensionManager class of stevedore provides a simple way to load all extensions of an entry point group. ([Location 1815](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=1815)) - The setuptools library provides a complete solution to packaging, not only to transport your code in different formats and upload it to PyPI, but also to handle connection with other software and libraries via entry points. ([Location 1834](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=1834)) - First, you should store tests inside a tests submodule of the application or library they apply to. Doing so will allow you to ship the tests as part of your module so that they can be run or reused by anyone—even after your software is installed—without necessarily using the source package. ([Location 1875](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=1875)) - For small projects with simple usage, the pytest package comes to the rescue—once installed via pip, pytest provides the pytest command, which loads every file whose name starts with test_ and then executes all functions within that start with test_. ([Location 1887](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=1887)) - Pytest provides a dynamic marking system that allows you to mark tests with a keyword that can be used as a filter. ([Location 1970](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=1970)) - In unit testing, you’ll often need to execute a set of common instructions before and after running a test, and those instructions will use certain components. For example, you might need an object that represents the configuration state of your application, and you’ll likely want that object to be initialized before each test, then reset to its default values when the test is achieved. ([Location 2003](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=2003)) - These components, known as fixtures, are set up before a test and cleaned up after the test has finished. ([Location 2006](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=2006)) - Because we used the yield keyword and made database a generator, the code after the yield statement runs when the test is done. ([Location 2021](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=2021)) - In that case, you can pass the scope argument to the fixture decorator, specifying the scope of the fixture: ([Location 2025](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=2025)) - By specifying the scope="module" parameter, you initialize the fixture once for the whole module, and the same database connection will be passed to all test functions requesting a database connection. ([Location 2030](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=2030)) - Specifying the autouse=True keyword argument to the pytest.fixture() function will make sure the fixture is called before running any test in the module or class it is defined in, as in this example: ([Location 2034](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=2034)) - When unit testing, you may want to run the same error-handling test with several different objects that trigger that error, or you may want to run an entire test suite against different drivers. ([Location 2042](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=2042)) - An easy way to achieve this is by using parameterized fixtures, which will run all the tests that use them several times, once for each of the defined parameters. ([Location 2048](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=2048)) - Mock objects are simulated objects that mimic the behavior of real application objects, but in particular and controlled ways. These are especially useful in creating environments that describe precisely the conditions for which you would like to test code. ([Location 2061](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=2061)) - One use case is in writing an HTTP client, since it is likely impossible (or at least extremely complicated) to spawn the HTTP server and test it through all scenarios to return every possible value. HTTP clients are especially difficult to test for all failure scenarios. ([Location 2064](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=2064)) - The call_count attribute ➋ is a simple way of checking the number of times a method has been called. ([Location 2104](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=2104)) - Using mocking, we can simulate any problem, such as a web server returning a 404 error, an I/O error, or a network latency issue. We can make sure code returns the correct values or raises the correct exception in every case, ensuring our code always behaves as expected. ([Location 2180](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=2180)) - A great complement to unit testing, the coverage tool identifies whether any of your code has been missed during testing. It uses code analysis tools and tracing hooks to determine which lines of your code have been executed; when used during a unit test run, it can show you which parts of your codebase have been crossed over and which parts have not. ([Location 2183](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=2183)) - One of the central uses of virtual environments is to provide a clean environment for running unit tests. ([Location 2288](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=2288)) - The tox management tool aims to automate and standardize how tests are run in Python. To that end, it provides everything needed to run an entire test suite in a clean virtual environment, while also installing your application to check that the installation works. ([Location 2294](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=2294)) - Python’s decorators are a handy way to modify functions. ([Location 2442](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=2442)) - A decorator is a function that takes another function as an argument and replaces it with a new, modified function. ([Location 2450](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=2450)) - Class decorators work in the same way as function decorators, but they act on classes rather than functions. ([Location 2532](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=2532)) - Another possible use for class decorators is to wrap a function or class with classes. For example, class decorators are often used for wrapping a function that’s storing a state. ([Location 2541](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=2541)) - Fortunately, the functools module in the Python Standard Library solves this problem with the update_wrapper() function, which copies the attributes from the original function that were lost to the wrapper itself. ([Location 2574](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=2574)) - For this, Python has the inspect module, which allows us to retrieve a function’s signature and operate on it, ([Location 2626](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=2626)) - The function that does the heavy lifting here is inspect.getcallargs(), which returns a dictionary containing the names and values of the arguments as key-value pairs. In our example, this function returns {'username': 'admin','type': 'chocolate'}. ([Location 2636](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=2636)) - Using functools.wraps and the inspect module, you should be able to write any custom decorator that you would ever need. ([Location 2641](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=2641)) - Decorators are a terrific way to implement the Don’t Repeat Yourself mantra so cherished by developers. ([Location 2644](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=2644)) - A method is a function that is stored as a class attribute. ([Location 2648](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=2648)) - We are told that get_size() is a function—but why is that? The reason is that at this stage, get_size() is not tied to any particular object. Therefore, it is treated as a normal function. ([Location 2655](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=2655)) - Static methods belong to a class, rather than an instance of a class, so they don’t actually operate on or affect class instances. Instead, a static method operates on the parameters it takes. Static methods are generally used to create utility functions, because they do not depend on the state of the class or its objects. ([Location 2684](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=2684)) - Using the @staticmethod decorator gives us several things. The first is speed: Python does not have to instantiate a bound method for each Pizza object we create. Bound methods are objects, too, and creating them has a CPU and memory cost—even if it’s low. ([Location 2695](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=2695)) - Second, static methods improve the readability of the code. When we see @staticmethod, we know that the method does not depend on the state of the object. ([Location 2701](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=2701)) - Third, static methods can be overridden in subclasses. ([Location 2703](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=2703)) - Class methods are bound to a class rather than its instances. That means that those methods cannot access the state of the object but only the state and methods of the class. ([Location 2711](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=2711)) - Class methods are principally useful for creating factory methods, which instantiate objects using a different signature than __init__: ([Location 2724](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=2724)) - If we used a @staticmethod here instead of a @classmethod, we would have to hardcode the Pizza class name in our method, making any class inheriting from Pizza unable to use our factory for its own purposes. ([Location 2729](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=2729)) - An abstract method is defined in an abstract base class that may not itself provide any implementation. When a class has an abstract method, it cannot be instantiated. As a consequence, an abstract class (defined as a class that has at least one abstract method) must be used as a parent class by another class. ([Location 2738](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=2738)) - A mixin is a class that inherits from two or more other classes, combining their features. ([Location 2831](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=2831)) - As you should know by now, classes are objects in Python. The construct used to create a class is a special statement that you should be well familiar with: class classname(expression of inheritance). ([Location 2836](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=2836)) - The class method mro() returns the method resolution order used to resolve attributes—it defines how the next method to call is found via the tree of inheritance between classes. ([Location 2846](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=2846)) - We already saw that the canonical way to call a method in a parent class is to use the super() function, but what you probably don’t know is that super() is actually a constructor and you instantiate a super object each time you call it. ([Location 2850](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=2850)) - At first glance, it might seem like this unbound kind of super object is useless, but actually the way the super class implements the descriptor protocol __get__ makes unbound super objects useful as class attributes: ([Location 2887](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=2887)) - The best precaution is to use tricks such as having all your methods accept their arguments using *args, **kwargs. ([Location 2905](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=2905)) - Decorators are essential when it comes to code factorization, and proper use of the built-in decorators provided by Python can vastly improve the neatness of your Python code. ([Location 2914](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=2914)) - Abstract classes are especially useful when providing an API to other developers and services. ([Location 2915](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=2915)) - Many Python developers are unaware of the extent to which you can use functional programming in Python, which is a shame: with few exceptions, functional programming allows you to write more concise and efficient code. ([Location 2920](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=2920)) - When you write code using a functional style, your functions are designed to have no side effects: instead, they take an input and produce an output without keeping state or modifying anything not reflected in the return value. Functions that follow this ideal are referred to as purely functional. ([Location 2931](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=2931)) - The practical advantages of functional programming include the following: Modularity Writing with a functional style forces a certain degree of separation in solving your individual problems and makes sections of code easier to reuse in other contexts. Since the function does not depend on any external variable or state, calling it from a different piece of code is straightforward. Brevity Functional programming is often less verbose than other paradigms. Concurrency Purely functional functions are thread-safe and can run concurrently. Some functional languages do this automatically, which can be a big help if you ever need to scale your application, though this is not quite the case yet in Python. Testability Testing a functional program is incredibly easy: all you need is a set of inputs and an expected set of outputs. They are idempotent, meaning that calling the same function over and over with the same arguments will always return the same result. ([Location 2942](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=2942)) - A generator is an object that behaves like an iterator, in that it generates and returns a value on each call of its next() method until a StopIteration is raised. ([Location 2951](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=2951)) - When execution reaches the yield statement, the function returns a value as with a return statement, but with one notable difference: the interpreter will save a stack reference, and this will be used to resume the function’s execution when the next() function is called again. ([Location 2957](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=2957)) - When a function returns, it’s removed from the stack, and the value it returns is passed to the calling function. In the case of a generator, the function does not really return but yields instead. Python therefore saves the state of the function as a stack reference, resuming the execution of the generator at the point it saved when the next iteration of the generator is needed. ([Location 2961](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=2961)) - When it is iterated over, the range() class returns a generator that dynamically generates our list of integers. ([Location 2999](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=2999)) - By generating values on the fly, generators allow you to handle large data sets with minimal consumption of memory and processing cycles. ([Location 3002](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=3002)) - A yield statement also has a less commonly used feature: it can return a value in the same way as a function call. ([Location 3005](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=3005)) - List comprehension, or listcomp for short, allows you to define a list’s contents inline with its declaration. ([Location 3072](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=3072)) - Using a list comprehension presents two advantages: code written using listcomps is usually shorter and therefore compiles down to fewer operations for Python to perform. Rather than creating a list and calling append over and over, Python can just create the list of items and move them into a new list in a single operation. ([Location 3081](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=3081)) - Using list comprehension rather than for loops is a neat way to define lists quickly. ([Location 3093](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=3093)) - The map() function takes the form map(function, iterable) and applies function to each item in iterable to return a list in Python 2 or an iterable map object in Python 3, ([Location 3105](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=3105)) - The filter() function takes the form filter(function or None, iterable) and filters the items in iterable based on the result returned by function. ([Location 3119](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=3119)) - The enumerate() function takes the form enumerate(iterable[, start]) and returns an iterable object that provides a sequence of tuples, each consisting of an integer index (starting with start, if provided) and the corresponding item in iterable. ([Location 3132](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=3132)) - The sorted() function takes the form sorted(iterable, key=None, reverse=False) and returns a sorted version of iterable. The key argument allows you to provide a function that returns the value to sort on, as shown here: ([Location 3142](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=3142)) - The any(iterable) and all(iterable) functions return a Boolean depending on the values returned by iterable. ([Location 3150](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=3150)) - These functions are useful for checking whether any or all of the values in an iterable satisfy a given condition. ([Location 3158](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=3158)) - The difference here is that any() returns True when at least one element meets the condition, while all() returns True only if every element meets the condition. ([Location 3162](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=3162)) - The zip() function takes the form zip(iter1 [,iter2 [...]]). It takes multiple sequences and combines them into tuples. This is useful when you need to combine a list of keys and a list of values into a dict. ([Location 3167](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=3167)) - The functools.partial() method is typically useful in place of lambda and should be considered a superior alternative. The lambda function is something of an anomaly in the Python language, and dropping it altogether was considered for Python 3 due to the function’s limited body size of a single line. ([Location 3266](https://readwise.io/to_kindle?action=open&asin=B074S4G1L5&location=3266))