tox 4 - call to arms

what's coming soon (and how to help)

Lightning talk - PyCon US 2021 - 🌐 edition

by Bernat Gabor / @gjbernat / Bloomberg

https://bit.ly/tox-py-us-21

Who am I?

  • software engineer at Bloomberg
  • OSS contributor gaborbernat@github - https://bernat.tech
  • tox author and maintainer
  • 😎 parent to two Yorkshire Terriers (Silky & Junior) with my wonderful fiancΓ©e Lisa

    Photo by Bernat Gabor - All rights reserved

Testing a library (or app)

  1. create virtual environment 
    
              virtualenv venv
  2. build (in a PEP-517/PEP-518 isolated environment) and install the package 
    
              pip install .

              # or the longer form
              # pyproject-build --sdist .
              # .\venv\Scripts\pip install .\dist\pi_approx-1.0.0.tar.gz
  3. install test dependencies 
    
                .\venv\Scripts\pip.exe install 'pytest>=6'
  4. run test 
    
              .\venv\Scripts\pytest tests

Testing a library (or app)

  1. create a configuration file, tox.ini 
    
              [tox]
              isolated_build = true

              [testenv]
              deps = pytest>=6
              commands = pytest tests
  2. install tox (Python project - e.g., via pipx) and then invoke it as: 
    
              tox -e py39
              # bonus do this for any python version: tox -e py310,py38

tox - status quo

today 1992 commits - 4 million downloads per month - 14k lines
first Sat Jun 5 11:45:30 2010 +0200 - released 12th July 2010 


            ❯ tox --version
            3.23.1 imported from c:\users\gabor\.local\pipx\venvs\tox\lib\site-packages\tox\__init__.py

soon 208+ commits - complete rewrite - 12.5k lines
Sun Mar 29 15:35:05 2020 +0100 - released Q3/Q4 2021 


              ❯ tox --version
              4.0.0 from c:\users\gabor\.local\pipx\venvs\tox\lib\site-packages\tox\__init__.py

Why rewrite? - contributor/maintainer

Remove technical debts (drop Python 2 + add type hints)


          class TestenvConfig:
              # Python 3 only, as __getattribute__ is ignored for old-style types on Python 2
              def __getattribute__(self, name):
                  rv = object.__getattribute__(self, name)
                  if isinstance(rv, Exception):
                      raise rv
                  return rv

              if six.PY2:

                  def __getattr__(self, name):
                      if name in self._missing_subs:
                          raise self._missing_subs[name]
                      raise AttributeError(name)

              def get_envbindir(self):
                  ...

Why rewrite? - contributor/maintainer

Remove technical debts (drop Python 2 + add type hints)


            from typing import Callable, Dict, Iterator, List, Mapping, Optional, Tuple

            Replacer = Callable[[str, List[str]], str]
            
            class SetEnv:
                def __init__(self, raw: str) -> None:
                    self.replacer: Replacer = lambda s, c: s
                    self._later: List[str] = []
                    self._raw: Dict[str, str] = {}
                    ...

                def load(self, item: str, chain: Optional[List[str]] = None) -> str:
                    if chain is None:
                        chain = [f"env:{item}"]
                    if item in self._materialized:
                        return self._materialized[item]
                    raw = self._raw[item]
                    result = self.replacer(raw, chain)  # apply any replace options
                    self._materialized[item] = result
                    self._raw.pop(item, None)  # if the replace requires the env we may be called again, so allow pop to fail
                    return result
            
                def __contains__(self, item: object) -> bool:
                    return isinstance(item, str) and item in self.__iter__()
                ...

Why rewrite? - contributor/maintainer

Better abstractions


          ❯ lsd  --tree .\src\tox\config
          ο„• config
          β”œβ”€β”€ ξ˜† __init__.py
          β”œβ”€β”€ ξ˜† parallel.py
          └── ξ˜† reporter.py
          

            ❯ lsd  --tree .\src\tox\config
            ο„• config
            β”œβ”€β”€ ξ˜† __init__.py
            β”œβ”€β”€ ο„• cli
            β”‚  β”œβ”€β”€ ξ˜† __init__.py
            β”‚  β”œβ”€β”€ ξ˜† env_var.py
            β”‚  β”œβ”€β”€ ξ˜† ini.py
            β”‚  β”œβ”€β”€ ξ˜† parse.py
            β”‚  └── ξ˜† parser.py
            β”œβ”€β”€ ο„• loader
            β”‚  β”œβ”€β”€ ξ˜† __init__.py
            β”‚  β”œβ”€β”€ ξ˜† api.py
            β”‚  β”œβ”€β”€ ξ˜† convert.py
            β”‚  β”œβ”€β”€ ο„• ini
            β”‚  β”‚  β”œβ”€β”€ ξ˜† __init__.py
            β”‚  β”‚  β”œβ”€β”€ ξ˜† factor.py
            β”‚  β”‚  └── ξ˜† replace.py
            β”‚  β”œβ”€β”€ ξ˜† memory.py
            β”‚  β”œβ”€β”€ ξ˜† str_convert.py
            β”‚  └── ξ˜† stringify.py
            β”œβ”€β”€ ξ˜† main.py
            β”œβ”€β”€ ξ˜† of_type.py
            β”œβ”€β”€ ξ˜† set_env.py
            β”œβ”€β”€ ξ˜† sets.py
            β”œβ”€β”€ ο„• source
            β”‚  β”œβ”€β”€ ξ˜† __init__.py
            β”‚  β”œβ”€β”€ ξ˜† api.py
            β”‚  β”œβ”€β”€ ξ˜† discover.py
            β”‚  β”œβ”€β”€ ξ˜† ini.py
            β”‚  β”œβ”€β”€ ξ˜† legacy_toml.py
            β”‚  β”œβ”€β”€ ξ˜† setup_cfg.py
            β”‚  └── ξ˜† tox_ini.py
            └── ξ˜† types.py

Why rewrite? - user

Improve performance


            ❯ hyperfine 'tox4 -a' 'tox -a'
            Benchmark #1: tox4 -a
            Time (mean Β± Οƒ):     408.2 ms Β±  24.6 ms    [User: 0.0 ms, System: 10.0 ms]
            Range (min … max):   369.7 ms … 460.0 ms    10 runs

            Benchmark #2: tox -a
              Time (mean Β± Οƒ):      1.240 s Β±  0.037 s    [User: 2.8 ms, System: 7.3 ms]
              0  Range (min … max):    1.205 s …  1.334 s    10 runs

            Summary
              'tox4 -a' ran
                3.04 Β± 0.20 times faster than 'tox -a'

Why rewrite? - user

More colors

Why rewrite? - user

More colors

Why rewrite? - user

Wheels support (not just sdist)


          [testenv]
          package = wheel
          isolated_build_env = .pkg

Why rewrite? - user

Introduce new interfaces -> Future new features

  • Swap out virtual environments with conda environments 
    
              ❯ tox --runner conda
  • Swap out virtual environments with Docker containers/images 
    
                ❯ tox --runner docker --image python:3.9
                ❯ tox --runner docker --image ubuntu:latest
  • Remove built-in assumption of Python environment (support for other languages: e.g., JavaScript): 
    
              ❯ tox -e package_js

Why rewrite? - user

Better UI - separation of concerns

show default tox targets


          ❯ tox -l
          py39
          py38

show all tox targets


          ❯ tox -a
          py39
          py38
          extra

show configuration values for a tox target


          ❯ tox --showconfig -e py39
          [testenv:py39]
          envdir = C:\Users\gabor\git\tox-ini-fmt\.tox\py39
          setenv = SetenvDict: {'COVERAGE_FILE': '{toxworkdir}
          ...
          

          ❯ tox -a -l --showconfig -e py39
          ???

sub-commands to the rescue


              ❯ tox4 --help
    
              run (r)                   run environments
              run-parallel (p)          run environments in parallel
              depends (de)              visualize tox environment dependencies
              list (l)                  list environments
              devenv (d)                sets up a development environment at ENVDIR based on the tox configuration specified
              config (c)                show tox configuration
              quickstart (q)            Command line script to quickly create a tox config file for a Python project
              legacy (le)               legacy entry-point command

Why rewrite? - user

    Consistent configuration handling
  • Every default can be changed via tox.ini config file

            # config file: 'C:\\Users\\gabor\\AppData\\Local\\tox\\tox\\config.ini' missing (change via env var TOX_CONFIG_FILE)
            # [tox]
            # color = 0
            tox4 r -e test

Why rewrite? - user

    Consistent configuration handling
  • Every default can be changed via tox.ini config file
  • Every default can be set via TOX_ env var

            #
            #
            #
            env TOX_COLOR=0 tox4 r -e test

Why rewrite? - user

    Consistent configuration handling
  • Every default can be changed via tox.ini config file
  • Every default can be set via TOX_ env var
  • Every option can be overwritten via the -x flag

            #
            #
            #
            env TOX_COLOR=0 tox4 r -e test -x testenv.base_python=python3.9

How to help?

Help us now so we don't have to test in PROD (your CI environments)

Try our pre-releases


            ❯ virtualenv venv
            ❯ venv/bin/pip install tox --pre
            ❯ venv/bin/tox4

PS. we broke all the plugins; if you maintain a plugin, please reach out to us

πŸ™ thank you πŸ™