nuluh/thesis
1
0
Fork 0
Code Issues 47 Pull Requests Actions Packages Projects Releases 1 Wiki Activity

[EXP] Evaluate ML model performance with downsampled accelerometer data (512Hz, 256Hz,128Hz) #52

New Issue
Open
opened 2025-04-27 11:52:01 +00:00 by nuluh · 0 comments
nuluh commented 2025-04-27 11:52:01 +00:00 (Migrated from github.com)
Copy Link

Hypothesis

Downsampling the accelerometer data from 1024Hz to lower frequencies (512Hz, 128Hz) will result in acceptable model performance with minimal degradation, making the system more practical for deployment with lower-capability sensors while potentially reducing computational requirements.

Background & Motivation

Current structural health monitoring (SHM) accelerometer data is collected at 1024Hz, which may exceed the capabilities of many commercially available or lower-cost sensors. Understanding how model performance degrades with lower sampling rates would:

  1. Establish minimum sensor requirements for practical deployment
  2. Potentially reduce computational and storage requirements
  3. Broaden the applicability of the approach to scenarios where high-frequency data collection is not feasible
  4. Inform the trade-off between sensor cost/capability and detection performance

This experiment is particularly relevant for real-world applications where sensor cost, power consumption, and data transmission bandwidth may be limiting factors.

Dataset

  • Original dataset: Accelerometer time-domain data sampled at 1024Hz

  • Derived datasets (to be created):

    • 512Hz downsampled version (take every 2nd sample)
    • 128Hz downsampled version (take every 8th sample)

  • Both damaged and undamaged node data should be included

  • Same structural scenarios and damage locations across all sampling rates

Methodology

Create downsampled datasets:

  • Apply Nyquist-compliant downsampling to avoid aliasing (apply low-pass filter before downsampling)
  • Generate 512Hz and 128Hz versions from the original 1024Hz data
  • Verify signal integrity after downsampling

STFT Processing for each dataset:

  • Adjust window size and hop size proportionally for each sampling rate to maintain consistent time-frequency resolution
  • For 512Hz: window_size=512, hop_size=256
  • For 128Hz: window_size=128, hop_size=64

Train identical model architectures on each dataset:

  • Use the same model architecture across all sampling rates
  • Maintain consistent hyperparameters
  • Train separate models for each sampling rate

Comparative evaluation:

  • Compare performance metrics across sampling rates
  • Analyze differences in feature importance/activation patterns
  • Assess computational efficiency differences

Parameters & Hyperparameters

  • Sampling rates: 1024Hz (baseline), 512Hz, 128Hz

  • STFT parameters:

    • 1024Hz: window_size=1024, hop_size=512
    • 512Hz: window_size=512, hop_size=256
    • 128Hz: window_size=128, hop_size=64

  • Model architecture: [Same architecture as current model]

  • Training parameters (optional if NN were used):

    • Learning rate: [Current value]
    • Batch size: [Current value]
    • Epochs: [Current value]
    • Optimizer: [Current optimizer]

  • Cross-validation: 5-fold

Evaluation Metrics

Primary metrics:

  • Classification accuracy
  • F1-score (macro average)
  • Confusion matrix

Secondary metrics:

  • ROC-AUC
  • Precision and recall for each class
  • Classification latency (inference time)

Comparative analysis:

  • Statistical significance of performance differences
  • Frequency content analysis
  • Feature importance comparison

Notebook Location

notebooks/downsampling_experiment.ipynb

Dependencies

References

  • Eftekhar Azam, S., et al. (2017). "Vibration-based structural health monitoring using frequency domain decomposition." Advances in Mechanical Engineering.
  • Bajric, A., et al. (2018). "Requirements for minimum sensor resolution and sampling rate in vibration based structural health monitoring systems." Journal of Civil Structural Health Monitoring.

Additional Notes

This experiment will provide critical insights into the practical deployment requirements of the SHM system. If the performance remains acceptable at lower sampling rates, it would significantly enhance the practical value of the thesis by making the approach applicable to a wider range of sensor hardware.

Expected outcomes:

  • Some performance degradation is expected at lower sampling rates
  • 512Hz may maintain most of the performance characteristics of 1024Hz
  • 128Hz will likely show more significant degradation but may still be usable for detecting major damage
### Hypothesis Downsampling the accelerometer data from 1024Hz to lower frequencies (512Hz, 128Hz) will result in acceptable model performance with minimal degradation, making the system more practical for deployment with lower-capability sensors while potentially reducing computational requirements. ### Background & Motivation Current structural health monitoring (SHM) accelerometer data is collected at 1024Hz, which may exceed the capabilities of many commercially available or lower-cost sensors. Understanding how model performance degrades with lower sampling rates would: 1. Establish minimum sensor requirements for practical deployment 2. Potentially reduce computational and storage requirements 3. Broaden the applicability of the approach to scenarios where high-frequency data collection is not feasible 4. Inform the trade-off between sensor cost/capability and detection performance This experiment is particularly relevant for real-world applications where sensor cost, power consumption, and data transmission bandwidth may be limiting factors. ### Dataset - Original dataset: Accelerometer time-domain data sampled at 1024Hz - Derived datasets (to be created): - 512Hz downsampled version (take every 2nd sample) - 128Hz downsampled version (take every 8th sample) - Both damaged and undamaged node data should be included - Same structural scenarios and damage locations across all sampling rates ### Methodology Create downsampled datasets: - Apply Nyquist-compliant downsampling to avoid aliasing (apply low-pass filter before downsampling) - Generate 512Hz and 128Hz versions from the original 1024Hz data - Verify signal integrity after downsampling STFT Processing for each dataset: - Adjust window size and hop size proportionally for each sampling rate to maintain consistent time-frequency resolution - For 512Hz: window_size=512, hop_size=256 - For 128Hz: window_size=128, hop_size=64 Train identical model architectures on each dataset: - Use the same model architecture across all sampling rates - Maintain consistent hyperparameters - Train separate models for each sampling rate Comparative evaluation: - Compare performance metrics across sampling rates - Analyze differences in feature importance/activation patterns - Assess computational efficiency differences ### Parameters & Hyperparameters - Sampling rates: 1024Hz (baseline), 512Hz, 128Hz - STFT parameters: - 1024Hz: window_size=1024, hop_size=512 - 512Hz: window_size=512, hop_size=256 - 128Hz: window_size=128, hop_size=64 - Model architecture: [Same architecture as current model] - Training parameters (optional if NN were used): - Learning rate: [Current value] - Batch size: [Current value] - Epochs: [Current value] - Optimizer: [Current optimizer] - Cross-validation: 5-fold ### Evaluation Metrics Primary metrics: - Classification accuracy - F1-score (macro average) - Confusion matrix Secondary metrics: - ROC-AUC - Precision and recall for each class - Classification latency (inference time) Comparative analysis: - Statistical significance of performance differences - Frequency content analysis - Feature importance comparison ### Notebook Location notebooks/downsampling_experiment.ipynb ### Dependencies - Requires issue # (Add undamaged node classification) - Requires issue #48 ### References - Eftekhar Azam, S., et al. (2017). "Vibration-based structural health monitoring using frequency domain decomposition." Advances in Mechanical Engineering. - Bajric, A., et al. (2018). "Requirements for minimum sensor resolution and sampling rate in vibration based structural health monitoring systems." Journal of Civil Structural Health Monitoring. ### Additional Notes This experiment will provide critical insights into the practical deployment requirements of the SHM system. If the performance remains acceptable at lower sampling rates, it would significantly enhance the practical value of the thesis by making the approach applicable to a wider range of sensor hardware. Expected outcomes: - Some performance degradation is expected at lower sampling rates - 512Hz may maintain most of the performance characteristics of 1024Hz - 128Hz will likely show more significant degradation but may still be usable for detecting major damage
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
Something isn't working
 critical
Block Progress
 documentation
Improvements or additions to documentation
 documentclass
Improvements or additions to LaTeX documentclass
 duplicate
This issue or pull request already exists
 enhancement
New feature or request
 good first issue
Good for newcomers
 help wanted
Extra attention is needed
 invalid
This doesn't seem right
 latex
Related to the writings
 question
Further information is requested
 wontfix
This will not be worked on
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
nuluh
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: nuluh/thesis#52
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?