Contributing#

kikuchipy is a community maintained project. We welcome contributions in the form of bug reports, documentation, code, feature requests, and more. The source code is hosted on GitHub. These guidelines provide resources on how best to contribute.

Tip

This guide can look intimidating to people who want to contribute, but have limited experience with tools like git, pytest, and sphinx. The shortest route to start contributing is to create a GitHub account and explain what you want to do in an issue.

This project follows the all-contributors specification.

Questions, comments, and feedback#

Have a question, comment, suggestion for improvements, or any other inquiries regarding the project? Feel free to ask a question, open an issue or make a pull request in our GitHub repository. We also have a Gitter chat.

Code of Conduct#

kikuchipy has a Code of Conduct that should be honoured by everyone who participates in the kikuchipy community.

Setting up a development installation#

You need a fork of the repository in order to make changes to kikuchipy.

Make a local copy of your forked repository and change directories:

git clone https://github.com/your-username/kikuchipy.git
cd kikuchipy

Set the upstream remote to the main kikuchipy repository:

git remote add upstream https://github.com/pyxem/kikuchipy.git

We recommend installing in a conda environment with the Miniconda distribution:

conda create --name kp-dev
conda activate kp-dev

Then, install the required dependencies while making the development version available globally (in the conda environment):

pip install --editable .[dev]

This installs all necessary development dependencies, including those for running tests and building documentation.

Code style#

The code making up kikuchipy is formatted closely following the Style Guide for Python Code with The Black Code style. We use pre-commit to run black automatically prior to each local commit. Please install it in your environment:

pre-commit install

Next time you commit some code, your code will be formatted inplace according to black.

Note that black won’t format docstrings. We follow the numpydoc standard (with some exceptions), and the docstrings are checked against this standard when building the documentation.

Comment lines should preferably be limited to 72 characters.

Package imports should be structured into three blocks with blank lines between them (descending order): standard library (like os and typing), third party packages (like numpy and hyperspy) and finally kikuchipy imports.

We use type hints in the function definition without type duplication in the function docstring, for example:

def my_function(a: int, b: Optional[bool] = None) -> Tuple[float, np.ndarray]:
    """This is a new function.

    Parameters
    ----------
    a
        Explanation of ``a``.
    b
        Explanation of flag ``b``. Default is ``None``.

    Returns
    -------
    values
        Explanation of returned values.
    """

We import modules lazily using the specification in PEP 562.

Making changes#

If you want to add a new feature, branch off of the develop branch, and when you want to fix a bug, branch off of main instead.

To create a new feature branch that tracks the upstream development branch:

git checkout develop -b your-awesome-feature-name upstream/develop

When you’ve made some changes you can view them with:

git status

Add and commit your created, modified or deleted files:

git add my-file-or-directory
git commit -s -m "An explanatory commit message"

The -s makes sure that you sign your commit with your GitHub-registered email as the author. You can set this up following this GitHub guide.

Keeping your branch up-to-date#

If you are adding a new feature, make sure to merge develop into your feature branch. If you are fixing a bug, merge main into your bug fix branch instead.

To update a feature branch, switch to the develop branch:

git checkout develop

Fetch changes from the upstream branch and update develop:

git pull upstream develop --tags

Update your feature branch:

git checkout your-awesome-feature-name
git merge develop

Sharing your changes#

Update your remote branch:

git push -u origin your-awesome-feature-name

You can then make a pull request to kikuchipy’s develop branch for new features and main branch for bug fixes. Good job!

Building and writing documentation#

The documentation contains three categories of documents: examples, tutorials, and the reference. The documentation strategy is based on the Diátaxis Framework. New documents should fit into one of these categories.

We use Sphinx for documenting functionality. Install necessary dependencies to build the documentation:

pip install --editable .[doc]

Note

The tutorials and examples require some small datasets to be downloaded via the kikuchipy.data module upon building the documentation. See the section on the data module for more details.

Then, build the documentation from the doc directory:

cd doc
make html

The documentation’s HTML pages are built in the doc/build/html directory from files in the reStructuredText (reST) plaintext markup language. They should be accessible in the browser by typing file:///your/absolute/path/to/kikuchipy/doc/build/html/index.html in the address bar.

We use Sphinx-Gallery to build the Examples. The examples are located in the top source directory examples/, and a new directory doc/examples/ is created when the docs are built.

We use nbsphinx for converting notebooks into tutorials. The tutorials are located in the top source directory tutorials/, and links to these notebooks are added using nbsphinx-link.

Here are some tips for writing tutorial notebooks:

  • All notebooks should have a Markdown cell with this message at the top, “This notebook is part of the kikuchipy documentation https://kikuchipy.org. Links to the documentation won’t work from the notebook.”, and have "nbsphinx": "hidden" in the cell metadata so that the message is not visible when displayed in the documentation.

  • Use _ = ax[0].imshow(...) to silence matplotlib output if a matplotlib command is the last line in a cell.

  • Refer to our API reference with this general Markdown [fft_filter()](../reference/generated/kikuchipy.signals.EBSD.fft_filter.rst). Remember to add the parentheses () to functions and methods.

  • Reference sections in other tutorial notebooks using this general Markdown [image quality](feature_maps.ipynb#image-quality).

  • Reference external APIs via standard Markdown like [Signal2D](http://hyperspy.org/hyperspy-doc/current/api/hyperspy._signals.signal2d.html).

  • The Sphinx gallery thumbnail used for a notebook is set by adding the nbsphinx-thumbnail tag to a code cell with an image output. The notebook must be added to the gallery in the README.rst to be included in the documentation pages.

  • The furo Sphinx theme displays the documentation in a light or dark theme, depending on the browser/OS setting. It is important to make sure the documentation is readable with both themes. This means explicitly printing the signal axes manager, like print(s.axes_manager), and displaying all figures with a white background for axes labels and ticks and figure titles etc. to be readable.

  • Whenever the documentation is built (locally or on the Read the Docs server), nbsphinx only runs the notebooks without any cell output stored. It is recommended that notebooks are stored without cell output, so that functionality within them are run and tested to ensure continued compatibility with code changes. Cell output should only be stored in notebooks which are too computationally intensive for the Read the Docs server to handle, which has a limit of 15 minutes and 3 GB of memory per documentation build.

  • We also use black to format notebooks cells. To run the black formatter on your notebook(s) locally please specify the notebook(s), ie. black my_notebook.ipynb or black *.ipynb, as black . will not format .ipynb files without explicit consent. To prevent black from automatically formatting regions of your code, please wrap these code blocks with the following:

    # fmt: off
python_code_block = not_to_be_formatted
# fmt: on

    Please see the black documentation for more details.

  • Displaying interactive 3D plots with PyVista requires a Jupyter backend, and we use pythreejs. This can either be passed to the plotting function, or it can be set in a hidden (see point above) notebook cell at the top of the notebook via pyvista.set_jupyter_backend("pythreejs").

In general, we run all notebooks every time the documentation is built with Sphinx, to ensure that all notebooks are compatible with the current API at all times. This is important! For computationally expensive notebooks however, we store the cell outputs so the documentation doesn’t take too long to build, either by us locally or the Read The Docs GitHub action. To check that the notebooks with stored cell outputs are compatible with the current API, we run a scheduled GitHub Action every Monday morning which checks that the notebooks run OK and that they produce the same output now as when they were last executed. We use nbval for this.

The tutorial notebooks can be run interactively in the browser with the help of Binder. When creating a server from the kikuchipy source code, Binder installs the packages listed in the environment.yml configuration file, which must include all doc dependencies listed in setup.py necessary to run the notebooks.

Deprecations#

We attempt to adhere to semantic versioning as best we can. This means that as little, ideally no, functionality should break between minor releases. Deprecation warnings are raised whenever possible and feasible for functions/methods/properties/arguments, so that users get a heads-up one (minor) release before something is removed or changes, with a possible alternative to be used.

The decorator should be placed right above the object signature to be deprecated:

@deprecate(since=0.8, removal=0.9, alternative="bar")
def foo(self, n):
    return n + 1

@property
@deprecate(since=0.9, removal=0.10, alternative="another", object_type="property")
def this_property(self):
    return 2

Running and writing tests#

All functionality in kikuchipy is tested via the pytest framework. The tests reside in a test directory within each module. Tests are short methods that call functions in kikuchipy and compare resulting output values with known answers. Install necessary dependencies to run the tests:

pip install --editable .[tests]

Some useful fixtures, like a dummy scan and corresponding background pattern, are available in the conftest.py file.

Note

Some kikuchipy.data module tests check that data not part of the package distribution can be downloaded from the kikuchipy-data GitHub repository, thus downloading some datasets of ~15 MB to your local cache.

To run the tests:

pytest --cov --pyargs kikuchipy

The --cov flag makes coverage.py print a nice report in the terminal. For an even nicer presentation, you can use coverage.py directly:

coverage html

Then, you can open the created htmlcov/index.html in the browser and inspect the coverage in more detail.

To run only a specific test function or class, .e.g the TestEBSD class:

pytest -k TestEBSD

This is useful when you only want to run a specific test and not the full test suite, e.g. when you’re creating or updating a test. But remember to run the full test suite before pushing!

Docstring examples are tested with pytest as well. If you’re in the top directory you can run:

pytest --doctest-modules --ignore-glob=kikuchipy/*/tests kikuchipy/*.py

Tips for writing tests of Numba decorated functions:

  • A Numba decorated function numba_func() is only covered if it is called in the test as numba_func.py_func().

  • Always test a Numba decorated function calling numba_func() directly, in addition to numba_func.py_func(), because the machine code function might give different results on different OS with the same Python code. See this issue for a case where this happened.

Adding data to the data module#

Example datasets used in the documentation and tests are included in the kikuchipy.data module via the pooch Python library. These are listed in a file registry (kikuchipy.data._registry.py) with their file verification string (hash, SHA256, obtain with e.g. sha256sum <file>) and location, the latter potentially not within the package but from the kikuchipy-data repository or elsewhere, since some files are considered too large to include in the package.

If a required dataset isn’t in the package, but is in the registry, it can be downloaded from the repository when the user passes allow_download=True to e.g. nickel_ebsd_large(). The dataset is then downloaded to a local cache, in the location returned from pooch.os_cache("kikuchipy"). The location can be set with a global KIKUCHIPY_DATA_DIR variable locally, e.g. by setting export KIKUCHIPY_DATA_DIR=~/kikuchipy_data in ~/.bashrc. Pooch handles downloading, caching, version control, file verification (against hash) etc. of files not included in the package. If we have updated the file hash, pooch will re-download it. If the file is available in the cache, it can be loaded as the other files in the data module.

With every new version of kikuchipy, a new directory of datasets with the version name is added to the cache directory. Any old directories are not deleted automatically, and should then be deleted manually if desired.

Improving performance#

When we write code, it’s important that we (1) get the correct result, (2) don’t fill up memory, and (3) that the computation doesn’t take too long. To keep memory in check, we should use Dask wherever possible. To speed up computations, we should use Numba wherever possible.

Continuous integration (CI)#

We use GitHub Actions to ensure that kikuchipy can be installed on Windows, macOS and Linux (Ubuntu). After a successful installation of the package, the CI server runs the tests. After the tests return no errors, code coverage is reported to Coveralls. Add "[skip ci]" to a commit message to skip this workflow on any commit to a pull request.