Stocklandhill HGM Monitoring

Author

Vibration Engineering Section

Published

Feb 2025

1 Sensors

Figure 1: Sensor Locations
Figure 2: Local Axes of ACC

2 Data Files

Please note that ACC4 was not configured correctly to use the TCXO. It was configured to use the internal oscillator, which is inaccurate and resulted in inaccurate timestamping. ACC4 was omitted from the merged acceleration data files in stocklandhill-accm-YYYY_mmdd_HHMM.hd5.

Files are available to download from Exeter Google Drive/20250618_Stockland-hill/integrated_data.

Figure 3: Data Availability
Table 1: First and Last Data Files
First File Last File
DataStream
acc1 2025-06-18 12:10:00+00:00 2025-07-25 12:20:00+00:00
acc3 2025-06-18 11:50:00+00:00 2025-08-27 22:10:00+00:00
acc4 2025-06-18 11:30:00+00:00 2025-07-20 23:20:00+00:00
acc5 2025-06-18 12:30:00+00:00 2025-08-08 06:30:00+00:00
wnd1 2025-06-18 11:30:00+00:00 2025-08-16 23:50:00+00:00

The plot below shows any time gaps in binary filenames of the form acc{i}-YYY-MMDD-HHMMSS. If there were missing data, the filenames should show gaps.

  • acc{i} sensors recorded continuously without apparent gaps; filename gaps were all 10 minutes.
  • wnd1 sometimes had gaps, as shown by the purple spikes in the plot above. There were two causes: 1) GPS fix failure caused new file generation to be skipped, although DAQ and saving continued in the previous filename; 2) wnd1 appears to have rebooted once around 12 July. There were no further reboots.
  • In conclusion, data files were mostly continuous.
Figure 4: Difference in timestamps of the data files

2.1 Data File Format

The original data files were in a binary format used in eNodes. However, after time-stamping and resampling, measurements are stored in Pandas DataFrame HDF5 format, which can be read with the following command.

# How to read a DataFrame HDF5 file

import pandas as pd

df = pd.read_hdf(filename, 'df')
df.plot()

2.2 MATLAB: How to Import

DataFrame files either downloaded from Google Drive or the website, can be imported into the MATLAB workspace using the following commands.

%% Example of loading a Pandas DataFrame file from MATLAB
%
% See the video at the following link
%    https://uk.mathworks.com/videos/run-python-commands-from-matlab-using-the-pyrun-function-1663925906686.html


%% Loading a DataFrame

h5file = "data/20250618/stocklandhill-accm-2025_0618_1240.hd5";

code = [
    "import pandas as pd"
    "df = pd.read_hdf(f'{filepath}', 'df')"
    ];
df = pyrun(code, 'df', filepath=h5file);
T = table(df);

%% Plotting

plot(T.index, [T.acc1x T.acc1y T.acc1z]);
xlabel('Time');
ylabel('Acceleration (g)');
title('Acceleration Data from HDF5');
legend('acc1x', 'acc1y', 'acc1z');
grid on;
Figure 5: DataFrame Imported to and Plotted in MATLAB

3 JupyterHub How-to

3.1 Database Basics

  • All timestamps are UTC-aware. When slicing by time in pandas, keep the index timezone-aware.
  • The two main tables you will use are:
    • File — where each record points to a data file (node name, timestamp, file path, etc.).
    • Summary — processed results (mean, RMSD, natural frequency, damping ratio, mode shapes, etc.). Use it to study higher-level patterns across processed values.

Typical workflow:

  1. Use File to discover and load raw data into pandas.
  2. Use Summary when you want to store processed metrics.

3.2 Loading a Raw Data File

This example queries the File table for merged accelerometer data (accm), prints a few records, and shows how to load one file into a pandas DataFrame.

from stocklandhill_tool import *  # sets up Django + database access

# 1) Query the File table for merged accelerometer files (accm)
frecs = File.objects.filter(nodeName='accm').order_by('timestamp')
print('len(frecs)=', len(frecs))

# 2) Peek at the first few records
for idx, frec in enumerate(frecs):
    print(frec)
    if idx >= 10:
        break

# 3) Get the full path of a file record
print('filepath of frecs[0]=', frecs[0].fullPath())

# 4) Load the first file into a pandas DataFrame
df = frecs[0].load()
len(frecs)= 10139
stocklandhill-accm-2025_0618_1230.hd5, Valid/Backuped=False/False
stocklandhill-accm-2025_0618_1240.hd5, Valid/Backuped=False/False
stocklandhill-accm-2025_0618_1250.hd5, Valid/Backuped=False/False
stocklandhill-accm-2025_0618_1300.hd5, Valid/Backuped=False/False
stocklandhill-accm-2025_0618_1310.hd5, Valid/Backuped=False/False
stocklandhill-accm-2025_0618_1320.hd5, Valid/Backuped=False/False
stocklandhill-accm-2025_0618_1330.hd5, Valid/Backuped=False/False
stocklandhill-accm-2025_0618_1340.hd5, Valid/Backuped=False/False
stocklandhill-accm-2025_0618_1350.hd5, Valid/Backuped=False/False
stocklandhill-accm-2025_0618_1400.hd5, Valid/Backuped=False/False
stocklandhill-accm-2025_0618_1410.hd5, Valid/Backuped=False/False
filepath of frecs[0]= /media/shm/shmcloud/stocklandhill/data/20250618/stocklandhill-accm-2025_0618_1230.hd5

3.3 Working with a Pandas DataFrame

After loading a file, you can inspect available columns and plot the time series.

# list all available columns

df.columns
Index(['acc1x', 'acc1y', 'acc1z', 'acc3x', 'acc3y', 'acc3z', 'acc5x', 'acc5y',
       'acc5z'],
      dtype='object')
df['acc1x'].plot()

df[['acc1x','acc1y','acc1z']].plot()

# You can specify options to plot()

df[['acc1x','acc1y','acc1z']].plot(
    ylabel = 'acc (g)',
    xlabel = 'time',
    grid = True,
)

3.4 Loading the Summary Table

The Summary table stores processed values (e.g., mean, RMSD, natural frequency, damping ratio, mode shapes). It is useful for identifying broader patterns and trends across processed results.

3.5 Querying a Date Range

Example: fetch merged accelerometer (accm) files for a single day, then concatenate them into one DataFrame for analysis.

from datetime import datetime, timedelta
from django.utils import timezone

# Define a UTC-aware time window
# (update the date as needed)
t0 = timezone.make_aware(datetime(2025, 6, 20))
t1 = t0 + timedelta(days=1)

# Query files in the time window
frecs = (File.objects
    .filter(nodeName='accm', timestamp__gte=t0, timestamp__lt=t1)
    .order_by('timestamp'))
print('records:', frecs.count())

# Load and concatenate the day of data
# (skip if no records)
dfs = [rec.load() for rec in frecs]
dfm = pd.concat(dfs, axis=0)
dfm.head()
records: 144
acc1x acc1y acc1z acc3x acc3y acc3z acc5x acc5y acc5z
2025-06-20 00:00:00+00:00 -0.001681 -0.003273 1.007781 -0.016480 0.009540 1.006995 0.015585 0.003031 1.005679
2025-06-20 00:00:00.020000+00:00 -0.001452 -0.003168 1.007733 -0.017716 0.009490 1.006940 0.016393 0.002917 1.005781
2025-06-20 00:00:00.040000+00:00 -0.000719 -0.003912 1.007885 -0.017169 0.009249 1.007123 0.016873 0.002836 1.005828
2025-06-20 00:00:00.060000+00:00 -0.001198 -0.003291 1.007626 -0.017505 0.010252 1.007051 0.016605 0.003060 1.005757
2025-06-20 00:00:00.080000+00:00 -0.000799 -0.003292 1.008392 -0.017853 0.010717 1.007017 0.016819 0.003094 1.005721

3.6 Filtering and resampling

Use pandas to slice by time, downsample for plotting, or save excerpts.

# Slice a two-hour window
window = dfm.loc['2025-06-19 06:00':'2025-06-19 08:00']

# Downsample to 1-minute means for quick plotting
minute_avg = dfm.resample('10min').mean()

ax = minute_avg[['acc1x','acc1y','acc1z']].plot(title='10-min mean accel')
ax.set_ylabel('g')
ax.grid(True)

# Save a CSV excerpt
window.to_csv('/tmp/accm_excerpt.csv')
print('written /tmp/accm_excerpt.csv')
written /tmp/accm_excerpt.csv

3.7 Loading Summary Table

summary records: 1584
id timestamp mean_acc1x mean_acc1y mean_acc1z mean_acc3x mean_acc3y mean_acc3z mean_acc5x mean_acc5y ... rmsd_cv1nn rmsd_cv1nh rmsd_acc1 rmsd_acc3 rmsd_acc5 rmsd_cv1d rmsd_cv1n mean_ws_avg mean_wd_avg mean_wind_temp
0 215 2025-06-20 00:00:00+00:00 -0.003651 -0.003666 1.007753 -0.014938 0.010183 1.007049 0.013486 0.002986 ... NaN None 0.002622 0.001488 0.001376 None None 6.140439 120.528121 21.094623
1 216 2025-06-20 00:10:00+00:00 -0.003673 -0.003664 1.007753 -0.014943 0.010181 1.007050 0.013488 0.002987 ... NaN None 0.003033 0.001758 0.001688 None None 5.858732 122.747664 21.029533
2 217 2025-06-20 00:20:00+00:00 -0.003689 -0.003658 1.007752 -0.014945 0.010183 1.007052 0.013472 0.002988 ... NaN None 0.003341 0.001993 0.001865 None None 6.228030 121.931447 20.975397
3 218 2025-06-20 00:30:00+00:00 -0.003691 -0.003656 1.007752 -0.014951 0.010183 1.007051 0.013464 0.002990 ... NaN None 0.003431 0.002133 0.001970 None None 6.362497 120.614540 20.969698
4 219 2025-06-20 00:40:00+00:00 -0.003692 -0.003651 1.007752 -0.014959 0.010185 1.007051 0.013468 0.002989 ... NaN None 0.003615 0.001976 0.001885 None None 6.096620 122.215577 20.997399

5 rows × 46 columns

# Plot an example metric over time
ax = summary_df.set_index('timestamp')['mean_acc1x']\
    .plot(title='Mean acc1x (June 2025)')
ax.set_ylabel('g')
ax.grid(True)