Compare commits
2 Commits
0d492c59d2
...
49857071d2
Author | SHA1 | Date | |
---|---|---|---|
49857071d2 | |||
7cd8d3556b |
|
@ -4,4 +4,4 @@ Contribution
|
||||||
Contributions are always welcome! Whether it's an enhancement, bug fix, or new usage examples, your input is valuable.
|
Contributions are always welcome! Whether it's an enhancement, bug fix, or new usage examples, your input is valuable.
|
||||||
If you're interested in contributing to RIA Toolkit OSS, please don't hesitate to reach out to the project maintainers.
|
If you're interested in contributing to RIA Toolkit OSS, please don't hesitate to reach out to the project maintainers.
|
||||||
|
|
||||||
The project guidelines are outlined in the project ``README.md`` file.
|
The project’s guidelines are outlined in its ``README.md`` file.
|
||||||
|
|
|
@ -1,4 +1,7 @@
|
||||||
Getting Started
|
Getting Started
|
||||||
===============
|
===============
|
||||||
|
|
||||||
RIA Toolkit OSS is a Python library.
|
.. todo::
|
||||||
|
|
||||||
|
Getting started instructions are coming soon! In the meantime, feel free
|
||||||
|
to explore the project documentation, where many components include usage examples.
|
||||||
|
|
|
@ -1,30 +1,69 @@
|
||||||
Installation
|
Installation
|
||||||
============
|
============
|
||||||
|
|
||||||
Installation from Conda Forge (recommended)
|
RIA Hub Toolkit OSS can be installed either as a Conda package or as a standard Python package.
|
||||||
-------------------------------------------
|
|
||||||
|
|
||||||
It is recommended to use RIA Toolkit OSS is within a `Radioconda <https://anaconda.org/ryanvolz/radioconda>`_ environment,
|
We want your experience with RIA Toolkit OSS to be as smooth and frictionless as possible. If you run into any
|
||||||
which provides pre-configured packages and libraries required for common SDR devices. Radioconda installation and setup instructions
|
issues during installation, please reach out to our support team: ``support@qoherent.ai``.
|
||||||
can be found in the project README: `radioconda-installer <https://github.com/radioconda/radioconda-installer>`_.
|
|
||||||
|
|
||||||
Install RIA Toolkit OSS into the base environment:
|
Installation with Conda (recommended)
|
||||||
|
-------------------------------------
|
||||||
|
|
||||||
.. code-block:: bash
|
Conda package for RIA Toolkit OSS are available on the RIA Hub
|
||||||
|
`here <https://riahub.ai/qoherent/-/packages/conda/ria-toolkit-oss>`_.
|
||||||
|
|
||||||
conda activate base
|
RIA Toolkit OSS can be installed into any Conda environment. However, it is recommended to install within the base environment of
|
||||||
conda install ria-toolkit-oss
|
`Radioconda <https://github.com/radioconda/radioconda-installer>`_, which includes `GNU Radio <https://www.gnuradio.org/>`_ and several pre-configured libraries for
|
||||||
|
common SDR devices. Detailed instructions for installing and setting up Radioconda are available in the project README.
|
||||||
|
|
||||||
.. note::
|
Please follow the steps below to install RIA Toolkit OSS using Conda:
|
||||||
|
|
||||||
**(Coming soon)** Install RIA Toolkit OSS from RIA Hub's Conda Package Registry.
|
1. Before installing RIA Toolkit OSS into your Conda environment, update the Conda package manager:
|
||||||
|
|
||||||
Installation from PyPI / RIA Hub
|
.. code-block:: bash
|
||||||
--------------------------------
|
|
||||||
|
|
||||||
You can also install RIA Toolkit OSS in a standard Python virtual environment using Pip.
|
conda update --force conda
|
||||||
Addition information on Python virtual environments can be found here: `W3Schools: Python Virtual Environment <https://www.w3schools.com/python/python_virtualenv.asp>`_.
|
|
||||||
|
|
||||||
|
This ensures that the Conda package manager is fully up-to-date, allowing new or updated packages to be installed into the base environment without conflicts.
|
||||||
|
|
||||||
|
2. Add RIA Hub to your Conda channel configuration:
|
||||||
|
|
||||||
|
.. code-block:: bash
|
||||||
|
|
||||||
|
conda config --add channels https://riahub.ai/api/packages/qoherent/conda
|
||||||
|
|
||||||
|
3. Activate your Conda environment and install RIA Toolkit OSS. For example, with Radioconda:
|
||||||
|
|
||||||
|
.. code-block:: bash
|
||||||
|
|
||||||
|
conda activate base
|
||||||
|
conda install ria-toolkit-oss
|
||||||
|
|
||||||
|
4. After installing RIA Toolkit OSS, verify that the installation was successful by running:
|
||||||
|
|
||||||
|
.. code-block:: bash
|
||||||
|
|
||||||
|
conda list
|
||||||
|
|
||||||
|
If installation was successful, you should see a line item for `ria-toolkit-oss`:
|
||||||
|
|
||||||
|
.. code-block:: bash
|
||||||
|
|
||||||
|
ria-toolkit-oss <version> <build> https://riahub.ai/api/packages/qoherent/conda
|
||||||
|
|
||||||
|
Installation with pip
|
||||||
|
---------------------
|
||||||
|
|
||||||
|
RIA Toolkit OSS is available as a standard Python package on both RIA Hub
|
||||||
|
`here <https://riahub.ai/qoherent/-/packages/pypi/ria-toolkit-oss>`_ and PyPI
|
||||||
|
`here <https://pypi.org/project/ria-toolkit-oss/>`_.
|
||||||
|
|
||||||
|
These packages can be installed into a standard Python virtual environment using
|
||||||
|
`pip <https://pip.pypa.io/en/stable/>`_. For help getting started with Python virtual
|
||||||
|
environments, please refer to the following tutorial:
|
||||||
|
`Python Virtual Environments <https://www.w3schools.com/python/python_virtualenv.asp>`_.
|
||||||
|
|
||||||
|
Please follow the steps below to install RIA Toolkit OSS using pip:
|
||||||
|
|
||||||
1. Create and activate a Python virtual environment:
|
1. Create and activate a Python virtual environment:
|
||||||
|
|
||||||
|
@ -42,23 +81,23 @@ Addition information on Python virtual environments can be found here: `W3School
|
||||||
python -m venv venv
|
python -m venv venv
|
||||||
venv\Scripts\activate
|
venv\Scripts\activate
|
||||||
|
|
||||||
|
2. Install RIA Toolkit OSS from PyPI with pip:
|
||||||
2. Install RIA Toolkit OSS with Pip:
|
|
||||||
|
|
||||||
.. code-block:: bash
|
.. code-block:: bash
|
||||||
|
|
||||||
pip install ria-toolkit-oss
|
pip install ria-toolkit-oss
|
||||||
|
|
||||||
RIA Toolkit OSS can also be installed from the RIA Hub Python Index. However, because RIA Hub does not yet support a proxy or cache for public packages, you need to use the `--no-deps` option with pip to skip automatic dependency installation, and then manually install each dependency afterward.
|
RIA Toolkit OSS can also be installed from RIA Hub. However, RIA Hub does not yet support a proxy or cache for public packages.
|
||||||
|
We intend to add this missing functionality soon. In the meantime, please use the ``--no-deps`` option with pip to skip automatic
|
||||||
|
dependency installation, and then manually install each dependency afterward.
|
||||||
|
|
||||||
.. code-block:: bash
|
|
||||||
|
|
||||||
pip install --index-url https://riahub.ai/api/packages/qoherent/pypi/simple/ ria-toolkit-oss --no-deps
|
|
||||||
|
|
||||||
Installation from source
|
Installation from source
|
||||||
------------------------
|
------------------------
|
||||||
|
|
||||||
You can also install RIA Toolkit OSS directly from the source code:
|
Finally, RIA Toolkit OSS can be installed directly from the source code.
|
||||||
|
This approach is only recommended if you require an unpublished or development version of the project.
|
||||||
|
Follow the steps below to install RIA Toolkit OSS from source:
|
||||||
|
|
||||||
1. Clone the repository. For example:
|
1. Clone the repository. For example:
|
||||||
|
|
||||||
|
@ -72,16 +111,8 @@ You can also install RIA Toolkit OSS directly from the source code:
|
||||||
|
|
||||||
cd ria-toolkit-oss
|
cd ria-toolkit-oss
|
||||||
|
|
||||||
3. Install the package:
|
3. Install with pip:
|
||||||
|
|
||||||
.. code-block:: bash
|
.. code-block:: bash
|
||||||
|
|
||||||
pip install .
|
pip install .
|
||||||
|
|
||||||
.. note::
|
|
||||||
|
|
||||||
If you plan to modify the project code and want changes to take effect immediately without reinstalling, you can install the project in editable mode:
|
|
||||||
|
|
||||||
.. code-block:: bash
|
|
||||||
|
|
||||||
pip install -e .
|
|
||||||
|
|
|
@ -145,100 +145,85 @@ class RadioDataset(ABC):
|
||||||
classes_to_augment: Optional[str | list[str]] = None,
|
classes_to_augment: Optional[str | list[str]] = None,
|
||||||
inplace: Optional[bool] = False,
|
inplace: Optional[bool] = False,
|
||||||
) -> RadioDataset | None:
|
) -> RadioDataset | None:
|
||||||
"""Supplement the dataset with new examples by applying various transformations to the pre-existing examples
|
"""
|
||||||
in the dataset.
|
Supplement the dataset with new examples by applying various transformations
|
||||||
|
to the pre-existing examples in the dataset.
|
||||||
|
|
||||||
.. todo::
|
.. todo::
|
||||||
|
|
||||||
This method is currently under construction, and may produce unexpected results.
|
This method is currently under construction, and may produce unexpected results.
|
||||||
|
|
||||||
The process of supplementing a dataset to artificially increases the diversity of the examples is called
|
The process of supplementing a dataset to artificially increase the diversity
|
||||||
augmentation. In many cases, training on augmented data can enhance the generalization and robustness of
|
of examples is called augmentation. Training on augmented data can enhance
|
||||||
deep machine learning models. For more information on the benefits and limitations of data
|
the generalization and robustness of deep machine learning models. For more
|
||||||
augmentation, please refer to this tutorial by Abid Ali Awan: `A Complete Guide to Data Augmentation
|
information, see `A Complete Guide to Data Augmentation
|
||||||
<https://www.datacamp.com/tutorial/complete-guide-data-augmentation>`_.
|
<https://www.datacamp.com/tutorial/complete-guide-data-augmentation>`_.
|
||||||
|
|
||||||
The metadata for each new example will be identical to the metadata of the pre-existing example from
|
Metadata for each new example will be identical to the metadata of the
|
||||||
which it was generated. However, the metadata will be extended to include a 'augmentation' column, which will
|
pre-existing example from which it was generated. The metadata will be
|
||||||
be populated for each new example with the string representation of the transform used to generate it, and left
|
extended to include an 'augmentation' column, populated with the string
|
||||||
empty for all the pre-existing examples.
|
representation of the transform used.
|
||||||
|
|
||||||
Please note that augmented data should only be utilized for model training, not for testing or validation.
|
Augmented data should only be used for model training, not for testing or
|
||||||
|
validation.
|
||||||
|
|
||||||
Unless specified, augmentations are applied equally across classes, maintaining the original class
|
Unless specified, augmentations are applied equally across classes, maintaining
|
||||||
distribution.
|
the original class distribution.
|
||||||
|
|
||||||
In the case where target_size is not equal to the sum of the original class sizes scaled by an integer
|
If target_size does not match the sum of the original class sizes scaled by
|
||||||
multiple, it is not possible to maintain the original class distribution, so the distribution gets slightly
|
an integer multiple, the class distribution is slightly adjusted to satisfy
|
||||||
skewed to satisfy target_size. To do this, each class size gets divided by the total size and then
|
target_size.
|
||||||
multiplied by target_size, then these values all get rounded to the nearest integers. If the target_size is
|
|
||||||
not equal to the sum of the rounded sizes, the sizes get sorted based on their decimal portions and then
|
|
||||||
values are adjusted one by one until the target_size is reached.
|
|
||||||
|
|
||||||
:param class_key: Class name that is used to augment from and calculate class distribution.
|
:param class_key: Class name used to augment from and calculate class distribution.
|
||||||
:type class_key: str
|
:type class_key: str
|
||||||
|
|
||||||
:param augmentations: A function or a list of functions that take as input an example from the
|
:param augmentations: A function or list of functions that take an example
|
||||||
dataset and return a transformed version of that example. If no augmentations are specified, the default
|
and return a transformed version. Defaults to ``default_augmentations()``.
|
||||||
augmentations returned by the ``default_augmentations()`` method will be applied.
|
|
||||||
:type augmentations: callable or list of callables, optional
|
:type augmentations: callable or list of callables, optional
|
||||||
|
|
||||||
:param level: The level or extent of data augmentation to apply, ranging from 0.0 (no augmentation) to
|
:param level: The extent of augmentation from 0.0 (none) to 1.0 (full). If
|
||||||
1.0 (full augmentation, where each augmentation is applied to each pre-existing example).
|
``classes_to_augment`` is specified, can be either:
|
||||||
|br| |br| If ``classes_to_augment`` is specified, this can be either:
|
|
||||||
* A single float:
|
* A single float: All classes augmented evenly to this level.
|
||||||
All classes are augmented evenly to this level, maintaining the original class distribution.
|
* A list of floats: Each element corresponds to the augmentation level
|
||||||
* A list of floats:
|
target for the corresponding class.
|
||||||
Each element corresponds to the augmentation level target for the corresponding class.
|
|
||||||
The default is 1.0.
|
|
||||||
:type level: float or list of floats, optional
|
:type level: float or list of floats, optional
|
||||||
|
|
||||||
:param target_size: Target size of the augmented dataset. If specified, ``level`` is ignored, and augmentations
|
:param target_size: Target size of the augmented dataset. Overrides ``level``
|
||||||
are applied to expand the dataset to contain the specified number of examples.
|
if specified. If ``classes_to_augment`` is specified, can be either:
|
||||||
If ``classes_to_augment`` is specified, this can be either:
|
|
||||||
* A single float:
|
* A single float: All classes are augmented proportional to their
|
||||||
All classes are augmented proportional to their relative frequency until the dataset reaches the
|
relative frequency until the dataset reaches target_size.
|
||||||
target size, maintaining the original class distribution.
|
* A list of floats: Each element corresponds to the target size for the
|
||||||
* A list of floats:
|
corresponding class.
|
||||||
Each element in the list corresponds to the target size for the corresponding class.
|
|
||||||
Defaults to None.
|
|
||||||
:type target_size: int or list of ints, optional
|
:type target_size: int or list of ints, optional
|
||||||
|
|
||||||
:param classes_to_augment: List of the metadata keys of the classes to augment. If specified, only these
|
:param classes_to_augment: List of metadata keys of classes to augment.
|
||||||
classes will be augmented. Defaults to None.
|
|
||||||
:type classes_to_augment: string or list of strings, optional
|
:type classes_to_augment: string or list of strings, optional
|
||||||
|
|
||||||
:param inplace: If True, the augmentation is performed inplace and ``None`` is returned. Defaults to False.
|
:param inplace: If True, the augmentation is performed inplace and ``None`` is returned.
|
||||||
:type inplace: bool, optional
|
:type inplace: bool, optional
|
||||||
|
|
||||||
:raises ValueError: If level has any values that are not in the range (0,1].
|
:raises ValueError: If level has any values not in the range (0,1].
|
||||||
:raises ValueError: If target_size of dataset is already sufficed.
|
:raises ValueError: If target_size of dataset is already sufficed.
|
||||||
:raises ValueError: If a class name in classes_to_augment does not exist in the specified class_key.
|
:raises ValueError: If a class in classes_to_augment does not exist in class_key.
|
||||||
|
|
||||||
:return: The augmented dataset or None if ``inplace=True``.
|
:return: The augmented dataset or None if ``inplace=True``.
|
||||||
:rtype: RadioDataset or None
|
:rtype: RadioDataset or None
|
||||||
|
|
||||||
**Examples:**
|
**Examples:**
|
||||||
|
|
||||||
>>> from ria.dataset_manager.builders import AWGN_Builder()
|
>>> from ria.dataset_manager.builders import AWGN_Builder
|
||||||
>>> builder = AWGN_Builder()
|
>>> builder = AWGN_Builder()
|
||||||
>>> builder.download_and_prepare()
|
>>> builder.download_and_prepare()
|
||||||
>>> ds = builder.as_dataset()
|
>>> ds = builder.as_dataset()
|
||||||
>>> ds.get_class_sizes(class_key='col')
|
>>> ds.get_class_sizes(class_key='col')
|
||||||
{a:100, b:500, c:300}
|
{'a': 100, 'b': 500, 'c': 300}
|
||||||
>>> new_ds = ds.augment(class_key='col', classes_to_augment=['a', 'b'], target_size=1200)
|
>>> new_ds = ds.augment(class_key='col', classes_to_augment=['a', 'b'], target_size=1200)
|
||||||
>>> new_ds.get_class_sizes(class_key='col')
|
>>> new_ds.get_class_sizes(class_key='col')
|
||||||
{a:150 b:750, c:300}
|
{'a': 150, 'b': 750, 'c': 300}
|
||||||
|
|
||||||
>>> from ria.dataset_manager.builders import AWGN_Builder()
|
|
||||||
>>> builder = AWGN_Builder()
|
|
||||||
>>> builder.download_and_prepare()
|
|
||||||
>>> ds = builder.as_dataset()
|
|
||||||
>>> ds.get_class_sizes(class_key='col')
|
|
||||||
{a:50, b:20, c:130}
|
|
||||||
>>> new_ds = ds.augment(class_key='col', level=0.5)
|
|
||||||
>>> new_ds.get_class_sizes(class_key='col')
|
|
||||||
{a:75 b:30, c:195}
|
|
||||||
"""
|
"""
|
||||||
|
|
||||||
if augmentations is None:
|
if augmentations is None:
|
||||||
|
|
|
@ -127,7 +127,7 @@ def random_split(
|
||||||
|
|
||||||
This restriction makes it unlikely that a random split will produce datasets with the exact lengths specified.
|
This restriction makes it unlikely that a random split will produce datasets with the exact lengths specified.
|
||||||
If it is important to ensure the closest possible split, consider using ria_toolkit_oss.datatypes.datasets.split
|
If it is important to ensure the closest possible split, consider using ria_toolkit_oss.datatypes.datasets.split
|
||||||
instead.
|
instead.
|
||||||
|
|
||||||
:param dataset: Dataset to be split.
|
:param dataset: Dataset to be split.
|
||||||
:type dataset: RadioDataset
|
:type dataset: RadioDataset
|
||||||
|
|
|
@ -28,7 +28,7 @@ class Recording:
|
||||||
Metadata is stored in a dictionary of key value pairs,
|
Metadata is stored in a dictionary of key value pairs,
|
||||||
to include information such as sample_rate and center_frequency.
|
to include information such as sample_rate and center_frequency.
|
||||||
|
|
||||||
Annotations are a list of :ref:`Annotation <ria_toolkit_oss.datatypes.Annotation>`,
|
Annotations are a list of :class:`~ria_toolkit_oss.datatypes.Annotation`,
|
||||||
defining bounding boxes in time and frequency with labels and metadata.
|
defining bounding boxes in time and frequency with labels and metadata.
|
||||||
|
|
||||||
Here, signal data is represented as a NumPy array. This class is then extended in the RIA Backends to provide
|
Here, signal data is represented as a NumPy array. This class is then extended in the RIA Backends to provide
|
||||||
|
@ -48,7 +48,7 @@ class Recording:
|
||||||
|
|
||||||
:param metadata: Additional information associated with the recording.
|
:param metadata: Additional information associated with the recording.
|
||||||
:type metadata: dict, optional
|
:type metadata: dict, optional
|
||||||
:param annotations: A collection of ``Annotation`` objects defining bounding boxes.
|
:param annotations: A collection of :class:`~ria_toolkit_oss.datatypes.Annotation` objects defining bounding boxes.
|
||||||
:type annotations: list of Annotations, optional
|
:type annotations: list of Annotations, optional
|
||||||
|
|
||||||
:param dtype: Explicitly specify the data-type of the complex samples. Must be a complex NumPy type, such as
|
:param dtype: Explicitly specify the data-type of the complex samples. Must be a complex NumPy type, such as
|
||||||
|
@ -444,34 +444,6 @@ class Recording:
|
||||||
else:
|
else:
|
||||||
raise ValueError(f"Key {key} is protected and cannot be modified or removed.")
|
raise ValueError(f"Key {key} is protected and cannot be modified or removed.")
|
||||||
|
|
||||||
def view(self, output_path: Optional[str] = "images/signal.png", **kwargs) -> None:
|
|
||||||
"""Create a plot of various signal visualizations as a PNG image.
|
|
||||||
|
|
||||||
:param output_path: The output image path. Defaults to "images/signal.png".
|
|
||||||
:type output_path: str, optional
|
|
||||||
:param kwargs: Keyword arguments passed on to ria_toolkit_oss.view.view_sig.
|
|
||||||
:type: dict of keyword arguments
|
|
||||||
|
|
||||||
**Examples:**
|
|
||||||
|
|
||||||
Create a recording and view it as a plot in a .png image:
|
|
||||||
|
|
||||||
>>> import numpy
|
|
||||||
>>> from ria_toolkit_oss.datatypes import Recording
|
|
||||||
|
|
||||||
>>> samples = numpy.ones(10000, dtype=numpy.complex64)
|
|
||||||
>>> metadata = {
|
|
||||||
>>> "sample_rate": 1e6,
|
|
||||||
>>> "center_frequency": 2.44e9,
|
|
||||||
>>> }
|
|
||||||
|
|
||||||
>>> recording = Recording(data=samples, metadata=metadata)
|
|
||||||
>>> recording.view()
|
|
||||||
"""
|
|
||||||
from ria_toolkit_oss.view import view_sig
|
|
||||||
|
|
||||||
view_sig(recording=self, output_path=output_path, **kwargs)
|
|
||||||
|
|
||||||
def to_sigmf(self, filename: Optional[str] = None, path: Optional[os.PathLike | str] = None) -> None:
|
def to_sigmf(self, filename: Optional[str] = None, path: Optional[os.PathLike | str] = None) -> None:
|
||||||
"""Write recording to a set of SigMF files.
|
"""Write recording to a set of SigMF files.
|
||||||
|
|
||||||
|
@ -487,22 +459,6 @@ class Recording:
|
||||||
:raises IOError: If there is an issue encountered during the file writing process.
|
:raises IOError: If there is an issue encountered during the file writing process.
|
||||||
|
|
||||||
:return: None
|
:return: None
|
||||||
|
|
||||||
**Examples:**
|
|
||||||
|
|
||||||
Create a recording and view it as a plot in a `.png` image:
|
|
||||||
|
|
||||||
>>> import numpy
|
|
||||||
>>> from ria_toolkit_oss.datatypes import Recording
|
|
||||||
|
|
||||||
>>> samples = numpy.ones(10000, dtype=numpy.complex64)
|
|
||||||
>>> metadata = {
|
|
||||||
... "sample_rate": 1e6,
|
|
||||||
... "center_frequency": 2.44e9,
|
|
||||||
... }
|
|
||||||
|
|
||||||
>>> recording = Recording(data=samples, metadata=metadata)
|
|
||||||
>>> recording.view()
|
|
||||||
"""
|
"""
|
||||||
from ria_toolkit_oss.io.recording import to_sigmf
|
from ria_toolkit_oss.io.recording import to_sigmf
|
||||||
|
|
||||||
|
|
Loading…
Reference in New Issue
Block a user