zfp-oss tools #18

Merged

benchinnery merged 5 commits from zfp-oss into main 2026-04-01 13:52:10 -04:00

Conversation 0 Commits 5 Files Changed 56 +7617 -1161

benchinnery commented 2026-03-31 13:58:03 -04:00

Owner

Copy Link

Fix: signal processing bugs, broken imports, and data handling errors

Fixes a set of correctness bugs found during code review across the transforms, datatypes, I/O, and SDR modules.

Signal/transform fixes

• generate_awgn: corrected SNR math (/ snr_linear → / sqrt(snr_linear)) and replaced magnitude/phase noise generation with proper independent Gaussian I+Q components for true complex AWGN
• time_shift(shift=0): was producing all-zeros due to data[:, :-0] evaluating to empty slice; now correctly returns data unchanged
• resample: downsampled result was returned un-padded because empty_array was never assigned back to resampled_iqdata
• quantize_tape, quantize_parts, cut_out, time_shift: replaced raise UserWarning(...) with warnings.warn(...) so invalid arguments warn-and-continue rather than crash; added fallback to default value in each case
• Removed stray print(f"noise is {noise}") debug statement from add_awgn_to_signal

Broken imports

• Recording.to_wav() and Recording.to_blue(): fixed from utils.io.recording import ... → from ria_toolkit_oss.io.recording import ... (both methods were crashing at call time)
• FrequencyUpConversion: same utils. → ria_toolkit_oss. package rename fix

Recording / datatypes fixes

• update_metadata: added missing return after add_to_metadata call, preventing a double-write and potential inconsistent state if _is_jsonable raised after the key was already inserted
• trim: annotations entirely outside the trim window were kept with negative sample_count; now filtered out before adjusting and re-indexing survivors
• normalize: silently returned all-NaN data on all-zero recordings; now raises ValueError
• IQDataset.split_examples: raise Warning(...) → warnings.warn(...)

I/O fix

• from_sigmf: SIGMF_KEY_CONVERSION lookup was done on the stripped base key (e.g. "hw") rather than the full SigMF key (e.g. "core:hw"), so mappings like hw→sdr, recorder→source_software, and meta_doi→metadata were silently skipped

SDR base class

• tx_recording: tx_time * self.tx_sample_rate was stored as float; cast to int to match the RX path and prevent indexing issues
• SDR.init: removed duplicate self._accumulated_buffer = None assignment
• copy_file (h5helpers): wrapped original_file in a with statement to prevent handle leak on exception
• overwrite_file (h5helpers): removed redundant f.close() inside with block

Fix: signal processing bugs, broken imports, and data handling errors Fixes a set of correctness bugs found during code review across the transforms, datatypes, I/O, and SDR modules. **Signal/transform fixes** - generate_awgn: corrected SNR math (/ snr_linear → / sqrt(snr_linear)) and replaced magnitude/phase noise generation with proper independent Gaussian I+Q components for true complex AWGN - time_shift(shift=0): was producing all-zeros due to data[:, :-0] evaluating to empty slice; now correctly returns data unchanged - resample: downsampled result was returned un-padded because empty_array was never assigned back to resampled_iqdata - quantize_tape, quantize_parts, cut_out, time_shift: replaced raise UserWarning(...) with warnings.warn(...) so invalid arguments warn-and-continue rather than crash; added fallback to default value in each case - Removed stray print(f"noise is {noise}") debug statement from add_awgn_to_signal **Broken imports** - Recording.to_wav() and Recording.to_blue(): fixed from utils.io.recording import ... → from ria_toolkit_oss.io.recording import ... (both methods were crashing at call time) - FrequencyUpConversion: same utils. → ria_toolkit_oss. package rename fix **Recording / datatypes fixes** - update_metadata: added missing return after add_to_metadata call, preventing a double-write and potential inconsistent state if _is_jsonable raised after the key was already inserted - trim: annotations entirely outside the trim window were kept with negative sample_count; now filtered out before adjusting and re-indexing survivors - normalize: silently returned all-NaN data on all-zero recordings; now raises ValueError - IQDataset.split_examples: raise Warning(...) → warnings.warn(...) **I/O fix** - from_sigmf: SIGMF_KEY_CONVERSION lookup was done on the stripped base key (e.g. "hw") rather than the full SigMF key (e.g. "core:hw"), so mappings like hw→sdr, recorder→source_software, and meta_doi→metadata were silently skipped **SDR base class** - tx_recording: tx_time * self.tx_sample_rate was stored as float; cast to int to match the RX path and prevent indexing issues - SDR.__init__: removed duplicate self._accumulated_buffer = None assignment - copy_file (h5helpers): wrapped original_file in a with statement to prevent handle leak on exception - overwrite_file (h5helpers): removed redundant f.close() inside with block

benchinnery added 3 commits 2026-03-31 13:58:03 -04:00

reformats and campaign additions 019b0c6f4b

server change 7335dc4c52

zfp functionality and servers 9a960e2f29

benchinnery added 1 commit 2026-04-01 11:58:03 -04:00

optimiztions and fixes c36fdcf607

benchinnery added 1 commit 2026-04-01 12:28:45 -04:00

reformats f67c995846

benchinnery requested review from gillian 2026-04-01 12:46:21 -04:00

G gillian approved these changes 2026-04-01 13:45:12 -04:00

benchinnery referenced this issue from a commit 2026-04-01 13:52:08 -04:00

Merge pull request 'zfp-oss tools' (#18) from zfp-oss into main

benchinnery merged commit 44df45160e into main 2026-04-01 13:52:10 -04:00

Sign in to join this conversation.

Reviewers

No reviewers

Ggillian

Labels

Clear labels

  

CI/CD 🛠️

This issue relates to our automated testing, building, and publishing processes

  

WIP 🚧

Someone is currently worked on this issue

  

bug 🐞

There is a problem, error, or unexpected behaviour

  

code quality 🧹

This issues pertains to code quality improvements or tech debt reduction

  

data collection 🗃️

This issue involves machine learning data collection or filing

  

dependency 📦

This issue pertains to dependency changes or deprecation warnings

  

do not merge 🚫

This pull request is not to be merged

  

documentation 📝

Improvements or additions to documentation or tutorials

  

duplicate 🔄

This issue or pull request already exists

  

enhancement ✨

This issue is important and will require attention soon, but does not pose any immediate risks

  

good first issue 🌱

A good first issue for newcomers

  

help wanted 👋

Additional development support is required

  

inference 🧠

This issue involves designing or improving inference solutions

  

invalid 🛑

Something is wrong with this issue or it not does conform to project guidelines

  

machine learning 🚀

This issue pertains to and may require expertise in machine learning

  

priority

high🔥

This issue is critical and requires immediate attention to prevent further consequence or escalation

  

priority

low 🍃

  

priority

medium 🌟

This issue is important and will require attention soon, but does not pose any immediate risks

  

quantum ⚛️

This issue pertains to and may require expertise in quantum technology

  

question ❓

Additional information is required

  

radio 📡

This issue involves radio interface components and may requires access to radio hardware

  

sprintable 🏃‍♂️

This issue could be resolved in a single sprint

  

test 🧪

This issue pertains to our testing efforts

  

wont fix ❌

We've chosen not to address this issue

No Label

CI/CD 🛠️

WIP 🚧

bug 🐞

code quality 🧹

data collection 🗃️

dependency 📦

do not merge 🚫

documentation 📝

duplicate 🔄

enhancement ✨

good first issue 🌱

help wanted 👋

inference 🧠

invalid 🛑

machine learning 🚀

priority

high🔥

priority

low 🍃

priority

medium 🌟

quantum ⚛️

question ❓

radio 📡

sprintable 🏃‍♂️

test 🧪

wont fix ❌

Milestone

Clear milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

No Assignees

2 Participants

G

Notifications

Subscribe

Due Date

The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: qoherent/ria-toolkit-oss#18

No description provided.