{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"nbsphinx": "hidden"
},
"source": [
"This notebook is part of the `kikuchipy` documentation https://kikuchipy.org.\n",
"Links to the documentation won't work from the notebook."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Load and save data\n",
"\n",
"In this tutorial, we will load and save electron backscatter diffraction (EBSD) patterns, Kikuchi master patterns generated with *EMsoft* and virtual backscatter electron (VBSE) images from and to various formats supported by kikuchipy."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Load patterns\n",
"\n",
"### From a file\n",
"\n",
"kikuchipy can read and write experimental EBSD patterns and Kikuchi master patterns from or to many formats (see [supported formats](#Supported-file-formats)).\n",
"To load patterns from a file use, the [load()](../reference/generated/kikuchipy.load.rst) function.\n",
"Let's import the necessary libraries and read the nickel EBSD example dataset directly from file (not via [kikuchipy.data.nickel_ebsd_small()](../reference/generated/kikuchipy.data.nickel_ebsd_small.rst))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Exchange inline for notebook or qt5 (from pyqt) for interactive plotting\n",
"%matplotlib inline\n",
"\n",
"import os\n",
"from pathlib import Path\n",
"import tempfile\n",
"\n",
"import dask.array as da\n",
"import imageio.v3 as iio\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"\n",
"import hyperspy.api as hs\n",
"import kikuchipy as kp"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"datadir = Path(\"../../kikuchipy/data\")\n",
"nordif_ebsd = \"nordif/Pattern.dat\"\n",
"s = kp.load(datadir / nordif_ebsd)\n",
"s"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Or, load the stereographic projection of the upper hemisphere of an EBSD master pattern for a 20 keV beam energy from a modified version of *EMsoft*'s master pattern file, returned from their `EMEBSDmaster.f90` program"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"emsoft_master_pattern = (\n",
" \"emsoft_ebsd_master_pattern/ni_mc_mp_20kv_uint8_gzip_opts9.h5\"\n",
")\n",
"s_mp = kp.load(datadir / emsoft_master_pattern)\n",
"s_mp"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Both the stereographic and the square Lambert projections of this master pattern can be loaded with [kikuchipy.data.nickel_ebsd_master_pattern_small()](../reference/generated/kikuchipy.data.nickel_ebsd_master_pattern_small.rst).\n",
"\n",
"All file readers support accessing the data's metadata without loading it into memory (with the [Dask library](https://docs.dask.org/en/latest)), which can be useful when processing large data sets, one data chunk at a time, to avoid memory errors"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"s_lazy = kp.load(datadir / nordif_ebsd, lazy=True)\n",
"print(s_lazy)\n",
"s_lazy.data"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Parts or all of the data can be read into memory by calling [compute()](http://hyperspy.org/hyperspy-doc/current/api/hyperspy._signals.lazy.html#hyperspy._signals.lazy.LazySignal.compute)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"s_lazy_copy = s_lazy.inav[:2, :].deepcopy()\n",
"s_lazy_copy.compute()\n",
"s_lazy_copy"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"s_lazy.compute()\n",
"s_lazy"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"\n",
"Note\n",
"\n",
"When lazily loaded EBSD patterns are processed, they are processed chunk by chunk, which can lead to longer processing times, so processing lazy data sets should be done with some care.\n",
"See the relevant [HyperSpy user guide](http://hyperspy.org/hyperspy-doc/current/user_guide/big_data.html) for some tips.\n",
"\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Patterns can be visualized by navigating the navigation space"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"s.plot()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Upon loading, kikuchipy reads some scan information from the file and stores it in the `original_metadata` attribute"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# s.original_metadata # Long output"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Other information is stored in a standard location in the `metadata` attribute in fields specified in the [HyperSpy metadata structure](https://hyperspy.org/hyperspy-doc/current/user_guide/metadata_structure.html); these are not used by methods in kikuchipy"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"s.metadata"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The number of patterns in horizontal and vertical direction, pattern size in pixels, scan step size and detector pixel size are stored in the `axes_manager` attribute"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"print(s.axes_manager)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This information can be modified directly, and information in `metadata` and `axes_manager` can be modified by the [EBSD](../reference/generated/kikuchipy.signals.EBSD.rst) class methods [set_scan_calibration()](../reference/generated/kikuchipy.signals.EBSD.set_scan_calibration.rst) and [set_detector_calibration()](../reference/generated/kikuchipy.signals.EBSD.set_detector_calibration.rst).\n",
"\n",
"In addition to the `metadata`, `original_metadata` and `axes_manager` properties, kikuchipy tries to read the following from the file if available:\n",
"\n",
"- a [CrystalMap](https://orix.readthedocs.io/en/stable/reference/generated/orix.crystal_map.CrystalMap.html) with indexing results into the [xmap](../reference/generated/kikuchipy.signals.EBSD.xmap.rst) attribute\n",
"\n",
"- an [EBSDDetector](../reference/generated/kikuchipy.detectors.EBSDDetector.rst) describing the projection/pattern center (PC) values into the [detector](../reference/generated/kikuchipy.signals.EBSD.detector.rst) attribute\n",
"\n",
"- a static background into the [static_background](../reference/generated/kikuchipy.signals.EBSD.static_background.rst) attribute.\n",
"\n",
"The `xmap` attribute is `None` if it is not read, while some readers return an \"empty\" crystal map with points containing the identity rotation and the same phase ID of an undefined phase with no point group set."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"s.xmap"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"s.xmap.rotations"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"s.xmap.phase_id"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"s.detector"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"s.detector.pc"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"s.static_background"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"\n",
"Note\n",
"\n",
"*New in version 0.8.*\n",
"\n",
"The `EBSD` class inherits all methods from HyperSpy's [Signal2D](https://hyperspy.org/hyperspy-doc/current/api/hyperspy._signals.signal2d.html#hyperspy._signals.signal2d.Signal2D) class.\n",
"An effort is made to handle the mentioned attributes of `xmap`, `detector` and `static_background` when the navigation and/or signal dimensions are cropped, like when calling `EBSD.inav[]` or `EBSD.isig[]`.\n",
"This handling is new, and so needs time and more tests to mature.\n",
"It is considered experimental functionality.\n",
"\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### From a NumPy array\n",
"\n",
"An `EBSD`, [EBSDMasterPattern](../reference/generated/kikuchipy.signals.EBSDMasterPattern.rst) or [ECPMasterPattern](../reference/generated/kikuchipy.signals.ECPMasterPattern.rst) (electron channeling pattern) signal can be created directly from a NumPy array.\n",
"To create a data set of (60 x 60) pixel patterns in a (10 x 20) grid, i.e. 10 and 20 patterns in the horizontal and vertical scan directions respectively, of random intensities"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"s_np = kp.signals.EBSD(np.random.random((20, 10, 60, 60)))\n",
"s_np"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### From a Dask array\n",
"\n",
"When processing large data sets, it is useful to load data lazily with the Dask library.\n",
"This can be done upon reading patterns [from a file](#From-a-file) by setting `lazy=True` when using the `load()` function, or directly from a `dask.array.Array`"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"s_da = kp.signals.LazyEBSD(\n",
" da.random.random((20, 10, 60, 60), chunks=(2, 10, 60, 60))\n",
")\n",
"print(s_da)\n",
"s_da.data"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### From a HyperSpy signal\n",
"\n",
"HyperSpy provides the method [set_signal_type()](https://hyperspy.org/hyperspy-doc/current/api/hyperspy.signal.html#hyperspy.signal.BaseSignal.set_signal_type) to change between [BaseSignal](https://hyperspy.org/hyperspy-doc/current/api/hyperspy.signal.html#hyperspy.signal.BaseSignal) subclasses, of which `EBSD`, `EBSDMasterPattern`, `ECPMasterPattern` and [VirtualBSEImage](../reference/generated/kikuchipy.signals.VirtualBSEImage.rst) are four.\n",
"To get one of these signals from a [HyperSpy Signal2D](https://hyperspy.org/hyperspy-doc/current/api/hyperspy._signals.signal2d.html#hyperspy._signals.signal2d.Signal2D) signal"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"s_hs = hs.signals.Signal2D(np.random.random((20, 10, 60, 60)))\n",
"s_hs"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"for signal_type in [\n",
" \"EBSD\",\n",
" \"VirtualBSEImage\",\n",
" \"ECPMasterPattern\",\n",
" \"EBSDMasterPattern\",\n",
"]:\n",
" s_hs.set_signal_type(signal_type)\n",
" print(s_hs)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Save patterns\n",
"\n",
"To save signals to file, use the [save()](../reference/generated/kikuchipy.signals.EBSD.save.rst) method.\n",
"For example, to save an `EBSD` signal in an HDF5 file in our default [h5ebsd format](#kikuchipy-h5ebsd), with file name `\"patterns.h5\"`"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"temp_dir = Path(tempfile.mkdtemp())\n",
"s.save(temp_dir / \"patterns\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"\n",
"\n",
"Warning\n",
"\n",
"If we want to overwrite an existing file:\n",
"\n",
"```python\n",
"s.save(\"patterns.h5\", overwrite=True)\n",
"```\n",
"\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"If we want to save patterns in [NORDIF's binary format](#NORDIF-binary) instead"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"s.save(temp_dir / \"patterns.dat\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"To save an `EBSDMasterPattern` to an HDF5 file, we use the [save method inherited from HyperSpy](https://hyperspy.org/hyperspy-doc/current/api/hyperspy.signal.html#hyperspy.signal.BaseSignal.save) to write to their [HDF5 specification](https://hyperspy.org/hyperspy-doc/current/user_guide/io.html#hspy-hyperspy-s-hdf5-specification) or [zarr specification](https://hyperspy.org/hyperspy-doc/current/user_guide/io.html#zspy-hyperspy-s-zarr-specification)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"s_hs.save(temp_dir / \"master_pattern.hspy\")\n",
"s_hs"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"These master patterns can then be read into an `EBSDMasterPattern` signal again via HyperSpy's [load()](http://hyperspy.org/hyperspy-doc/current/api/hyperspy.io.html#hyperspy.io.load)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"s_mp2 = hs.load(\n",
" temp_dir / \"master_pattern.hspy\", signal_type=\"EBSDMasterPattern\"\n",
")\n",
"s_mp2"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"\n",
"Note\n",
"\n",
"To save results from statistical decomposition (machine learning) of patterns to file see the section [Saving and loading results](https://hyperspy.org/hyperspy-doc/current/user_guide/mva.html#saving-and-loading-results) in HyperSpy's user guide.\n",
"Note that the file extension `.hspy` or `.zspy` must be used upon saving, `s.save('patterns.hspy')`, as the default extension in kikuchipy, `.h5`, yields a kikuchipy h5ebsd file where the decomposition results aren't saved.\n",
"The saved patterns can then be reloaded using HyperSpy's `load()` function passing the `signal_type=\"EBSD\"` parameter [as explained above](#From-a-HyperSpy-signal).\n",
"\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Supported file formats\n",
"\n",
"Currently, kikuchipy has readers and writers for the following formats:\n",
"\n",
"| Format | Read | Write |\n",
"| ------------------------------------------------------------------------- | ---- | ----- |\n",
"| [EMsoft simulated EBSD HDF5](#EMsoft-simulated-EBSD-HDF5) | Yes | No |\n",
"| [EMsoft EBSD master pattern HDF5](#EMsoft-EBSD-master-pattern-HDF5) | Yes | No |\n",
"| [EMsoft ECP master pattern HDF5](#EMsoft-ECP-master-pattern-HDF5) | Yes | No |\n",
"| [EMsoft TKD master pattern HDF5](#EMsoft-TKD-master-pattern-HDF5) | Yes | No |\n",
"| [kikuchipy h5ebsd](#kikuchipy-h5ebsd) | Yes | Yes |\n",
"| [Bruker h5ebsd](#Bruker-h5ebsd) | Yes | No |\n",
"| [EDAX h5ebsd](#EDAX-h5ebsd) | Yes | No |\n",
"| [EDAX binary](#EDAX-binary) | Yes | No |\n",
"| [NORDIF binary](#NORDIF-binary) | Yes | Yes |\n",
"| [NORDIF calibration patterns](#NORDIF-calibration-patterns) | Yes | No |\n",
"| [Oxford Instruments binary](#Oxford-Instruments-binary) | Yes | No |\n",
"| [Oxford Instruments h5ebsd (H5OINA)](#Oxford-Instruments-h5ebsd-(H5OINA)) | Yes | No |\n",
"| [Directory of EBSD patterns](#Directory-of-EBSD-patterns) | Yes | No |\n",
"\n",
"\n",
"\n",
"Note\n",
"\n",
"If you want to process your patterns with kikuchipy, but use an unsupported EBSD vendor software, or if you want to write your processed patterns to a vendor format that does not support writing, please request this feature in our [issue tracker](https://github.com/pyxem/kikuchipy/issues).\n",
"\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### EMsoft simulated EBSD HDF5\n",
"\n",
"Dynamically simulated EBSD patterns returned by EMsoft's `EMEBSD.f90` program as HDF5 files can be read as an `EBSD` signal"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"emsoft_ebsd = \"emsoft_ebsd/simulated_ebsd.h5\" # Dummy data set\n",
"s_sim = kp.load(datadir / emsoft_ebsd)\n",
"s_sim"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Here, the EMsoft simulated EBSD [file_reader()](../reference/generated/kikuchipy.io.plugins.emsoft_ebsd.file_reader.rst) is called, which takes the optional argument `scan_size`.\n",
"Passing `scan_size=(2, 5)` will reshape the pattern data shape from `(10, 10, 10)` to `(2, 5, 10, 10)`"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"s_sim2 = kp.load(datadir / emsoft_ebsd, scan_size=(2, 5))\n",
"print(s_sim2)\n",
"print(s_sim2.data.shape)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Simulated EBSD patterns can be written to the [kikuchipy h5ebsd format](#kikuchipy-h5ebsd), the [NORDIF binary format](#NORDIF-binary), or to HDF5/zarr files using HyperSpy's HDF5/zarr specifications [as explained above](#Save-patterns)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### EMsoft EBSD master pattern HDF5\n",
"\n",
"Master patterns returned by EMsoft's `EMEBSDmaster.f90` program as HDF5 files can be read as an [EBSDMasterPattern](../reference/generated/kikuchipy.signals.EBSDMasterPattern.rst) signal"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"s_mp = kp.load(datadir / emsoft_master_pattern)\n",
"\n",
"print(s_mp)\n",
"print(s_mp.projection)\n",
"print(s_mp.hemisphere)\n",
"print(s_mp.phase)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Here, the EMsoft EBSD master pattern [file_reader()](../reference/generated/kikuchipy.io.plugins.emsoft_ebsd_master_pattern.file_reader.rst) is called, which takes the optional arguments `projection`, `hemisphere` and `energy`.\n",
"The stereographic projection is read by default.\n",
"Passing `projection=\"lambert\"` will read the square Lambert projection instead.\n",
"The upper hemisphere is read by default.\n",
"Passing `hemisphere=\"lower\"` or `hemisphere=\"both\"` will read the lower hemisphere projection or both, respectively.\n",
"Master patterns for all beam energies are read by default.\n",
"Passing `energy=(10, 20)` or `energy=15` will read the master pattern(s) with beam energies from 10 to 20 keV, or just 15 keV, respectively"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"s_mp = kp.load(\n",
" datadir / emsoft_master_pattern,\n",
" projection=\"lambert\",\n",
" hemisphere=\"both\",\n",
" energy=20,\n",
")\n",
"\n",
"print(s_mp)\n",
"print(s_mp.projection)\n",
"print(s_mp.hemisphere)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Master patterns can be written to HDF5/zarr files using HyperSpy's HDF5/zarr specification [as explained above](#Save-patterns).\n",
"\n",
"See Jackson et al. (2019) for a hands-on tutorial explaining how to simulate these patterns with EMsoft, and Callahan & De Graef (2013) for details of the underlying theory."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### EMsoft ECP master pattern HDF5\n",
"\n",
"Master patterns returned by EMsoft's `EMECPmaster.f90` program as HDF5 files can be read as an [ECPMasterPattern](../reference/generated/kikuchipy.signals.ECPMasterPattern.rst) signal"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Load a dummy pattern\n",
"s_mp = kp.load(datadir / \"emsoft_ecp_master_pattern/ecp_master_pattern.h5\")\n",
"\n",
"print(s_mp)\n",
"print(s_mp.projection)\n",
"print(s_mp.hemisphere)\n",
"print(s_mp.phase)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Here, the EMsoft ECP master pattern [file_reader()](../reference/generated/kikuchipy.io.plugins.emsoft_ecp_master_pattern.file_reader.rst) is called, which supports the same parameters as the [EMsoft EBSD master pattern reader](#EMsoft-EBSD-master-pattern-HDF5)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### EMsoft TKD master pattern HDF5\n",
"\n",
"Master patterns returned by EMsoft's `EMTKDmaster.f90` program as HDF5 files can be read as an [EBSDMasterPattern](../reference/generated/kikuchipy.signals.EBSDMasterPattern.rst) signal"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Load a dummy pattern\n",
"s_mp = kp.load(datadir / \"emsoft_tkd_master_pattern/tkd_master_pattern.h5\")\n",
"\n",
"print(s_mp)\n",
"print(s_mp.projection)\n",
"print(s_mp.hemisphere)\n",
"print(s_mp.phase)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Here, the EMsoft TKD master pattern [file_reader()](../reference/generated/kikuchipy.io.plugins.emsoft_tkd_master_pattern.file_reader.rst) is called, which supports the same parameters as the [EMsoft EBSD master pattern reader](#EMsoft-EBSD-master-pattern-HDF5)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### kikuchipy h5ebsd\n",
"\n",
"The h5ebsd format Jackson et al. (2014) is based on the [HDF5 open standard](https://www.hdfgroup.org/solutions/hdf5/) (Hierarchical Data Format version 5).\n",
"HDF5 files can be read and edited using e.g. the HDF Group's reader [HDFView](https://www.hdfgroup.org/downloads/hdfview) or the Python package [h5py](https://docs.h5py.org/en/stable).\n",
"Upon loading an HDF5 file with extension .h5, .hdf5, or .h5ebsd, the correct reader is determined from the file.\n",
"Other supported h5ebsd formats are listed in the [table above](#Supported-file-formats).\n",
"\n",
"If an h5ebsd file contains multiple scans, as many scans as desirable can be read from the file.\n",
"For example, if the file contains two scans with names `\"Scan 1\"` and `\"Scan 2\"`"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"kikuchipy_ebsd = \"kikuchipy_h5ebsd/patterns.h5\"\n",
"s1, s2 = kp.load(\n",
" datadir / kikuchipy_ebsd, scan_group_names=[\"Scan 1\", \"Scan 2\"]\n",
")\n",
"print(s1)\n",
"print(s2)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Here, the h5ebsd [file_reader()](../reference/generated/kikuchipy.io.plugins.kikuchipy_h5ebsd.file_reader.rst) is called.\n",
"If only `\"Scan 1\"` is to be read, `scan_group_names=\"Scan 1\"` can be passed"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"s1 = kp.load(datadir / kikuchipy_ebsd, scan_group_names=\"Scan 1\")\n",
"s1"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The `scan_group_names` parameter is unnecessary if only the first scan in the file is to be read, since reading only the first scan in the file is the default behaviour.\n",
"\n",
"So far, only [saving patterns](#Save-patterns) to kikuchipy's own h5ebsd format is supported.\n",
"It is possible to write a new scan with a scan name `\"Scan x\"`, where `x` is an integer, to an existing, but closed, h5ebsd file in the kikuchipy format, e.g. one containing only `Scan 1`, by passing"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"new_file = \"patterns_new.h5\"\n",
"s1.save(temp_dir / new_file, scan_number=1)\n",
"s2.save(temp_dir / new_file, add_scan=True, scan_number=2)\n",
"\n",
"s1_new, s2_new = kp.load(\n",
" temp_dir / new_file, scan_group_names=[\"Scan 1\", \"Scan 2\"]\n",
")\n",
"print(s1_new)\n",
"print(s2_new)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Here, the h5ebsd [file_writer()](../reference/generated/kikuchipy.io.plugins.kikuchipy_h5ebsd.file_writer.rst) is called.\n",
"The EBSD class attributes `xmap` (crystal map), `detector` (EBSD detector) and `static_background` are written to and read from file if available."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Bruker h5ebsd\n",
"\n",
"Bruker Nano's h5ebsd files with extension .h5, .hdf5, or .h5ebsd can be read.\n",
"Available parameters when calling `kikuchipy.load(\"bruker_patterns.h5\")` are described in the [file_reader()](../reference/generated/kikuchipy.io.plugins.bruker_h5ebsd.file_reader.rst).\n",
"As with the kikuchipy h5ebsd reader, multiple scans can be read at once.\n",
"\n",
"The projection center (PC) arrays are read into the `detector` attribute, and the static background pattern is read into the `static_background` attribute.\n",
"The orientation and phase data are so far not read, but note that this can be read using [orix](https://orix.readthedocs.io/en/stable/reference/generated/orix.io.plugins.bruker_h5ebsd.html)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### EDAX h5ebsd\n",
"\n",
"EDAX's h5ebsd files with extension .h5, .hdf5, or .h5ebsd can be read.\n",
"Available parameters when calling `kikuchipy.load(\"edax_patterns.h5\")` are described in the [file_reader()](../reference/generated/kikuchipy.io.plugins.edax_h5ebsd.file_reader.rst).\n",
"As with the kikuchipy h5ebsd reader, multiple scans can be read at once.\n",
"\n",
"The projection center (PC) arrays are read into the `detector` attribute.\n",
"The orientation and phase data are so far not read."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### EDAX binary\n",
"\n",
"Patterns stored in the EDAX TSL's binary UP1/2 file format, with intensities as 8-bit or 16-bit unsigned integer, can be read. File version 1 and >= 3 are supported."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"edax_binary_path = datadir / \"edax_binary/\" # Directory with dummy signals\n",
"s_edax = kp.load(edax_binary_path / \"edax_binary.up1\")\n",
"s_edax"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Here, the EDAX binary [file_reader()](../reference/generated/kikuchipy.io.plugins.edax_binary.file_reader.rst) is called.\n",
"\n",
"Files with version 1 has no information on the navigation (map) shape, so we can pass this to the reader in the `nav_shape` parameter"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"s_edax2 = kp.load(edax_binary_path / \"edax_binary.up1\", nav_shape=(3, 3))\n",
"s_edax2"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Patterns acquired in an hexagonal grid *can* be read, but the returned signal will have only one navigation dimension"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"s_edax3 = kp.load(edax_binary_path / \"edax_binary.up2\")\n",
"s_edax3"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### NORDIF binary\n",
"\n",
"Patterns acquired using NORDIF's acquisition software are stored in a binary file usually named \"Pattern.dat\".\n",
"Scan information is stored in a separate text file usually named \"Setting.txt\", and both files usually reside in the same directory.\n",
"If this is the case, the patterns can be loaded by passing the file name as the only parameter.\n",
"If this is not the case, the setting file can be passed upon loading"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"s_nordif = kp.load(\n",
" datadir / nordif_ebsd, setting_file=datadir / \"nordif/Setting.txt\"\n",
")\n",
"s_nordif"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Here, the NORDIF [file_reader()](../reference/generated/kikuchipy.io.plugins.nordif.file_reader.rst) is called.\n",
"If the scan information, i.e. scan and pattern size, in the setting file is incorrect or the setting file is not available, patterns can be loaded by passing"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"s_nordif = kp.load(\n",
" datadir / nordif_ebsd, scan_size=(1, 9), pattern_size=(60, 60)\n",
")\n",
"s_nordif"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"If a static background pattern named \\\"Background acquisition pattern.bmp\\\" is stored in the same directory as the pattern file, this is stored in `EBSD.static_background` upon loading"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"s_nordif.static_background"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Patterns can also be [saved to a NORDIF binary file](#Save-patterns), upon which the NORDIF [file_writer()](../reference/generated/kikuchipy.io.plugins.nordif.file_writer.rst) is called.\n",
"Note, however, that so far no new setting file, background pattern, or calibration patterns are created upon saving."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### NORDIF calibration patterns\n",
"\n",
"NORDIF calibration patterns in bitmap format named \"Calibration (x,y).bmp\", where \"x\" and \"y\" correspond to coordinates listed in the NORDIF setting file, usually named \"Setting.txt\", can be loaded"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"s_nordif_cal = kp.load(datadir / \"nordif/Setting.txt\")\n",
"s_nordif_cal"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Here, the NORDIF calibration patterns [file_reader()](../reference/generated/kikuchipy.io.plugins.nordif_calibration_patterns.file_reader.rst) is called.\n",
"Lazy loading is not supported for this reader, thus the `lazy` parameter is not used.\n",
"\n",
"If a static background pattern named \"Background calibration pattern.bmp\" is stored in the same directory as the pattern file, this is stored in `EBSD.static_background` upon loading.\n",
"\n",
"The pixel indices of the calibration patterns into the full electron image (named \"Area.bmp\" by the NORDIF software) are read into the `original_metadata`.\n",
"The shapes of this image and the region of interest (ROI) of the scanned area within this image are also attempted to be read.\n",
"All this information can be useful when obtaining a plane of projection centers (PCs) for all patterns in the full dataset based on these calibration patterns (see other user guide tutorials for this)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Oxford Instruments binary\n",
"\n",
"Uncompressed patterns stored in the Oxford Instruments binary .ebsp file format, with intensities as 8-bit or 16-bit unsigned integer, can be read"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"oxford_binary_path = datadir / \"oxford_binary\"\n",
"s_oxford = kp.load(oxford_binary_path / \"patterns.ebsp\")\n",
"s_oxford"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Here, the Oxford Instruments binary [file_reader()](../reference/generated/kikuchipy.io.plugins.oxford_binary.file_reader.rst) is called.\n",
"\n",
"Every pattern's flattened index into the 2D navigation map, as well as their entry in the file (map order isn't always the same as file order) can be retrieved from `s_oxford.original_metadata.map1d_id` and `s_oxford.original_metadata.file_order`, respectively.\n",
"If available in the file, every pattern's row and column beam position in microns can be retrieved from `s_oxford.original_metadata.beam_y` and `s_oxford.original_metadata.beam_x`, respectively.\n",
"All these are 1D arrays."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"s_oxford.original_metadata"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"If the beam positions aren't present in the file, the returned signal will have a single navigation dimension the size of the number of patterns.\n",
"\n",
"Files with only the non-indexed patterns can also be read. The returned signal will then have a single navigation dimension the size of the number of patterns.\n",
"The flattened index into the 2D navigation map mentioned above can be useful to determine the location of each non-indexed pattern."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Oxford Instruments h5ebsd (H5OINA)\n",
"\n",
"Oxford Instruments' h5ebsd files (H5OINA) with extension .h5oina can be read.\n",
"Available parameters when calling `kikuchipy.load(\"patterns.h5oina\")` are described in the [file_reader()](../reference/generated/kikuchipy.io.plugins.oxford_h5ebsd.file_reader.rst).\n",
"As with the kikuchipy h5ebsd reader, multiple scans can be read at once.\n",
"\n",
"The projection center (PC) arrays are read into the `detector` attribute and the static background pattern is read into the `static_background` attribute.\n",
"The orientation and phase data are so far not read."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Directory of EBSD patterns\n",
"\n",
"Many EBSD patterns in image files in a directory can be read as an `EBSD` signal, assuming all images with that file extension have the same shape and data type. Valid formats are those [supported by imageio](https://imageio.readthedocs.io/en/stable/formats/index.html).\n",
"\n",
"To demonstrate how the images can be read, we will first write nine patterns to a temporary with their filenames formatted a certain way"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"temp_dir2 = Path(tempfile.mkdtemp())\n",
"s = kp.data.nickel_ebsd_small()\n",
"y, x = np.indices(s.axes_manager.navigation_shape[::-1])\n",
"y = y.ravel()\n",
"x = x.ravel()\n",
"s.unfold_navigation_space()\n",
"for i in range(s.axes_manager.navigation_size):\n",
" iio.imwrite(temp_dir2 / f\"pattern_x{x[i]}y{y[i]}.tif\", s.data[i])\n",
"\n",
"# Print file contents\n",
"os.listdir(temp_dir2)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"If filenames are formatted like this `_x0y0.tif` or `-0-0.png`, we do not have to specify this file name pattern when reading the images"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"s1 = kp.load(temp_dir2 / \"*.tif\")\n",
"s1"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Here [file_reader()](../reference/generated/kikuchipy.io.plugins.ebsd_directory.file_reader.rst) of this plugin was called.\n",
"If filenames are formatted some other way, we have to pass this as a [regular expression](https://docs.python.org/3/library/re.html) to `xy_pattern`"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"s2 = kp.load(\n",
" temp_dir2 / \"*.tif\",\n",
" xy_pattern=r\"_x(\\d+)y(\\d+).tif\",\n",
" show_progressbar=True,\n",
")\n",
"s2"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Here we also printed a progressbar when reading the patterns from file, which can be useful when we want to read a large number of images."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## From kikuchipy into other software\n",
"\n",
"Patterns saved in the [h5ebsd format](#kikuchipy-h5ebsd) can be read by the dictionary indexing and related routines in [EMsoft](https://github.com/EMsoft-org/EMsoft) using the `EMEBSD` reader.\n",
"Those routines in EMsoft also have a `NORDIF` reader.\n",
"\n",
"Patterns saved in the [h5ebsd format](#kikuchipy-h5ebsd) can of course be read in Python like any other HDF5 data set"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"nbsphinx-thumbnail": {
"tooltip": "Loading and saving EBSD and EBSD master patterns from and to various file formats"
},
"tags": [
"nbsphinx-thumbnail"
]
},
"outputs": [],
"source": [
"import h5py\n",
"\n",
"with h5py.File(datadir / kikuchipy_ebsd, mode=\"r\") as f:\n",
" dset = f[\"Scan 2/EBSD/Data/patterns\"]\n",
" print(dset)\n",
" patterns = dset[()]\n",
" print(patterns.shape)\n",
" plt.figure()\n",
" plt.imshow(patterns[0], cmap=\"gray\")\n",
" plt.axis(\"off\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Load and save virtual BSE images\n",
"\n",
"One or more virtual backscatter electron (BSE) images in a [VirtualBSEImage](../reference/generated/kikuchipy.signals.VirtualBSEImage.rst) signal can be read and written to file using one of HyperSpy's many readers and writers.\n",
"If they are only to be used internally in HyperSpy, they can be written to and read back from HyperSpy's HDF5/zarr specification [as explained above for EBSD master patterns](#Save-patterns).\n",
"\n",
"If we want to write the images to image files, HyperSpy also provides a series of image readers/writers, as explained in their [IO user guide](https://hyperspy.org/hyperspy-doc/current/user_guide/io.html#images).\n",
"If we wanted to write them as a stack of TIFF images"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Get virtual image from image generator\n",
"vbse_imager = kp.imaging.VirtualBSEImager(s)\n",
"print(vbse_imager)\n",
"\n",
"print(vbse_imager.grid_shape)\n",
"vbse = vbse_imager.get_images_from_grid()\n",
"print(vbse)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"vbse.rescale_intensity()\n",
"vbse.unfold_navigation_space() # 1D navigation space required for TIFF\n",
"vbse"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"vbse_fname = \"vbse.tif\"\n",
"vbse.save(temp_dir / vbse_fname) # Easily read into e.g. ImageJ"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We can also write them to e.g. `png` or `bmp` files with `Matplotlib`"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"nav_size = vbse.axes_manager.navigation_size\n",
"for i in range(nav_size):\n",
" plt.imsave(temp_dir / f\"vbse{i}.png\", vbse.inav[i].data)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Read the TIFF stack back into a `VirtualBSEImage` signal"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"vbse2 = hs.load(temp_dir / vbse_fname, signal_type=\"VirtualBSEImage\")\n",
"vbse2"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"nbsphinx": "hidden"
},
"outputs": [],
"source": [
"# Remove files written to disk in this user guide\n",
"for file in os.listdir(temp_dir):\n",
" os.remove(temp_dir / file)\n",
"for file in os.listdir(temp_dir2):\n",
" os.remove(temp_dir2 / file)\n",
"os.rmdir(temp_dir)\n",
"os.rmdir(temp_dir2)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.6"
},
"widgets": {
"application/vnd.jupyter.widget-state+json": {
"state": {},
"version_major": 2,
"version_minor": 0
}
}
},
"nbformat": 4,
"nbformat_minor": 4
}