Contributing#
kikuchipy is a community maintained project. We welcome contributions in the form of bug reports, documentation, code, feature requests, and more. The source code is hosted on GitHub. These guidelines provide resources on how best to contribute.
Tip
This guide can look intimidating to people who want to contribute, but have limited
experience with tools like git
, pytest
, and sphinx
. The shortest route
to start contributing is to create a GitHub account and explain what you want to do
in an issue.
This project follows the all-contributors specification.
Questions, comments, and feedback#
Have a question, comment, suggestion for improvements, or any other inquiries regarding the project? Feel free to ask a question, open an issue or make a pull request in our GitHub repository. We also have a Gitter chat.
Code of Conduct#
kikuchipy has a Code of Conduct that should be honoured by everyone who participates in the kikuchipy community.
Setting up a development installation#
You need a fork of the repository in order to make changes to kikuchipy.
Make a local copy of your forked repository and change directories:
git clone https://github.com/your-username/kikuchipy.git
cd kikuchipy
Set the upstream
remote to the main kikuchipy repository:
git remote add upstream https://github.com/pyxem/kikuchipy.git
We recommend installing in a conda environment with the Miniconda distribution:
conda create --name kp-dev
conda activate kp-dev
Then, install the required dependencies while making the development version available
globally (in the conda
environment):
pip install --editable .[dev]
This installs all necessary development dependencies, including those for running tests and building documentation.
Code style#
The code making up kikuchipy is formatted closely following the Style Guide for Python
Code with The Black Code style. We
use pre-commit to run black
automatically prior to each
local commit. Please install it in your environment:
pre-commit install
Next time you commit some code, your code will be formatted inplace according to
black
.
Note that black
won’t format docstrings. We follow the numpydoc standard
(with some exceptions), and the docstrings are checked against this standard when
building the documentation.
Comment lines should preferably be limited to 72 characters.
Package imports should be structured into three blocks with blank lines between them
(descending order): standard library (like os
and typing
), third party packages
(like numpy
and hyperspy
) and finally kikuchipy imports.
We use type hints in the function definition without type duplication in the function docstring, for example:
def my_function(a: int, b: Optional[bool] = None) -> Tuple[float, np.ndarray]:
"""This is a new function.
Parameters
----------
a
Explanation of ``a``.
b
Explanation of flag ``b``. Default is ``None``.
Returns
-------
values
Explanation of returned values.
"""
We import modules lazily using the specification in PEP 562.
Making changes#
If you want to add a new feature, branch off of the develop
branch, and when you
want to fix a bug, branch off of main
instead.
To create a new feature branch that tracks the upstream development branch:
git checkout develop -b your-awesome-feature-name upstream/develop
When you’ve made some changes you can view them with:
git status
Add and commit your created, modified or deleted files:
git add my-file-or-directory
git commit -s -m "An explanatory commit message"
The -s
makes sure that you sign your commit with your GitHub-registered email as the author. You can set this up following
this GitHub guide.
Keeping your branch up-to-date#
If you are adding a new feature, make sure to merge develop
into your feature
branch. If you are fixing a bug, merge main
into your bug fix branch instead.
To update a feature branch, switch to the develop
branch:
git checkout develop
Fetch changes from the upstream branch and update develop
:
git pull upstream develop --tags
Update your feature branch:
git checkout your-awesome-feature-name
git merge develop
Building and writing documentation#
The documentation contains three categories of documents: examples
, tutorials
,
and the reference
. The documentation strategy is based on the
Diátaxis Framework. New documents should fit into one of
these categories.
We use Sphinx for documenting functionality. Install necessary dependencies to build the documentation:
pip install --editable .[doc]
Note
The tutorials and examples require some small datasets to be downloaded via the
kikuchipy.data
module upon building the documentation. See the section on the
data module for more details.
Then, build the documentation from the doc
directory:
cd doc
make html
The documentation’s HTML pages are built in the doc/build/html
directory from files
in the reStructuredText (reST) plaintext
markup language. They should be accessible in the browser by typing
file:///your/absolute/path/to/kikuchipy/doc/build/html/index.html
in the address
bar.
We use Sphinx-Gallery to build
the Examples. The examples are located in the top source directory
examples/
, and a new directory doc/examples/
is created when the docs are built.
We use nbsphinx for converting
notebooks into tutorials. The tutorials are located in the top source directory
tutorials/
, and links to these notebooks are added using
nbsphinx-link.
Here are some tips for writing tutorial notebooks:
All notebooks should have a Markdown cell with this message at the top, “This notebook is part of the kikuchipy documentation https://kikuchipy.org. Links to the documentation won’t work from the notebook.”, and have
"nbsphinx": "hidden"
in the cell metadata so that the message is not visible when displayed in the documentation.Use
_ = ax[0].imshow(...)
to silencematplotlib
output if amatplotlib
command is the last line in a cell.Refer to our API reference with this general Markdown
[fft_filter()](../reference/generated/kikuchipy.signals.EBSD.fft_filter.rst)
. Remember to add the parentheses()
to functions and methods.Reference sections in other tutorial notebooks using this general Markdown
[image quality](feature_maps.ipynb#image-quality)
.Reference external APIs via standard Markdown like
[Signal2D](http://hyperspy.org/hyperspy-doc/current/api/hyperspy._signals.signal2d.html)
.The Sphinx gallery thumbnail used for a notebook is set by adding the
nbsphinx-thumbnail
tag to a code cell with an image output. The notebook must be added to the gallery in the README.rst to be included in the documentation pages.The
furo
Sphinx theme displays the documentation in a light or dark theme, depending on the browser/OS setting. It is important to make sure the documentation is readable with both themes. This means explicitly printing the signal axes manager, likeprint(s.axes_manager)
, and displaying all figures with a white background for axes labels and ticks and figure titles etc. to be readable.Whenever the documentation is built (locally or on the Read the Docs server),
nbsphinx
only runs the notebooks without any cell output stored. It is recommended that notebooks are stored without cell output, so that functionality within them are run and tested to ensure continued compatibility with code changes. Cell output should only be stored in notebooks which are too computationally intensive for the Read the Docs server to handle, which has a limit of 15 minutes and 3 GB of memory per documentation build.We also use
black
to format notebooks cells. To run theblack
formatter on your notebook(s) locally please specify the notebook(s), ie.black my_notebook.ipynb
orblack *.ipynb
, asblack .
will not format.ipynb
files without explicit consent. To preventblack
from automatically formatting regions of your code, please wrap these code blocks with the following:# fmt: off python_code_block = not_to_be_formatted # fmt: on
Please see the black documentation for more details.
Displaying interactive 3D plots with PyVista requires a Jupyter backend, and we use pythreejs. This can either be passed to the plotting function, or it can be set in a hidden (see point above) notebook cell at the top of the notebook via
pyvista.set_jupyter_backend("pythreejs")
.
In general, we run all notebooks every time the documentation is built with Sphinx, to ensure that all notebooks are compatible with the current API at all times. This is important! For computationally expensive notebooks however, we store the cell outputs so the documentation doesn’t take too long to build, either by us locally or the Read The Docs GitHub action. To check that the notebooks with stored cell outputs are compatible with the current API, we run a scheduled GitHub Action every Monday morning which checks that the notebooks run OK and that they produce the same output now as when they were last executed. We use nbval for this.
The tutorial notebooks can be run interactively in the browser with the help of Binder. When creating a server from the kikuchipy source code, Binder installs the packages listed in the environment.yml configuration file, which must include all doc dependencies listed in setup.py necessary to run the notebooks.
Deprecations#
We attempt to adhere to semantic versioning as best we can. This means that as little, ideally no, functionality should break between minor releases. Deprecation warnings are raised whenever possible and feasible for functions/methods/properties/arguments, so that users get a heads-up one (minor) release before something is removed or changes, with a possible alternative to be used.
The decorator should be placed right above the object signature to be deprecated:
@deprecate(since=0.8, removal=0.9, alternative="bar")
def foo(self, n):
return n + 1
@property
@deprecate(since=0.9, removal=0.10, alternative="another", object_type="property")
def this_property(self):
return 2
Running and writing tests#
All functionality in kikuchipy is tested via the pytest
framework. The tests reside in a test
directory within each module. Tests are short
methods that call functions in kikuchipy and compare resulting output values with known
answers. Install necessary dependencies to run the tests:
pip install --editable .[tests]
Some useful fixtures,
like a dummy scan and corresponding background pattern, are available in the
conftest.py
file.
Note
Some kikuchipy.data
module tests check that data not part of the package
distribution can be downloaded from the kikuchipy-data GitHub repository, thus downloading some datasets of ~15
MB to your local cache.
To run the tests:
pytest --cov --pyargs kikuchipy
The --cov
flag makes coverage.py
print a nice report in the terminal. For an even nicer presentation, you can use
coverage.py
directly:
coverage html
Then, you can open the created htmlcov/index.html
in the browser and inspect the
coverage in more detail.
To run only a specific test function or class, .e.g the TestEBSD
class:
pytest -k TestEBSD
This is useful when you only want to run a specific test and not the full test suite, e.g. when you’re creating or updating a test. But remember to run the full test suite before pushing!
Docstring examples are tested with pytest as well. If you’re in the top directory you can run:
pytest --doctest-modules --ignore-glob=kikuchipy/*/tests kikuchipy/*.py
Tips for writing tests of Numba decorated functions:
A Numba decorated function
numba_func()
is only covered if it is called in the test asnumba_func.py_func()
.Always test a Numba decorated function calling
numba_func()
directly, in addition tonumba_func.py_func()
, because the machine code function might give different results on different OS with the same Python code. See this issue for a case where this happened.
Adding data to the data module#
Example datasets used in the documentation and tests are included in the
kikuchipy.data
module via the pooch
Python library. These are listed in a file registry (kikuchipy.data._registry.py
)
with their file verification string (hash, SHA256, obtain with e.g.
sha256sum <file>
) and location, the latter potentially not within the package but
from the kikuchipy-data repository or
elsewhere, since some files are considered too large to include in the package.
If a required dataset isn’t in the package, but is in the registry, it can be downloaded
from the repository when the user passes allow_download=True
to e.g.
nickel_ebsd_large()
. The dataset is then downloaded to a local
cache, in the location returned from pooch.os_cache("kikuchipy")
. The location can
be set with a global KIKUCHIPY_DATA_DIR variable locally, e.g. by setting
export KIKUCHIPY_DATA_DIR=~/kikuchipy_data
in ~/.bashrc
. Pooch handles
downloading, caching, version control, file verification (against hash) etc. of files
not included in the package. If we have updated the file hash, pooch will re-download
it. If the file is available in the cache, it can be loaded as the other files in the
data module.
With every new version of kikuchipy, a new directory of datasets with the version name is added to the cache directory. Any old directories are not deleted automatically, and should then be deleted manually if desired.
Improving performance#
When we write code, it’s important that we (1) get the correct result, (2) don’t fill up memory, and (3) that the computation doesn’t take too long. To keep memory in check, we should use Dask wherever possible. To speed up computations, we should use Numba wherever possible.
Continuous integration (CI)#
We use GitHub Actions to ensure that
kikuchipy can be installed on Windows, macOS and Linux (Ubuntu). After a successful
installation of the package, the CI server runs the tests. After the tests return no
errors, code coverage is reported to Coveralls. Add "[skip ci]"
to a commit message to skip this workflow on any commit to a pull request.