Data format¶
PDM data is acquired, triggered and time-stamped in the Zynq board. This data is then passed to the CPU. The CPU also acquires data from the other subsystems and packages this together with the PDM data during nominal night-time observations. Data is acquired every 5.24 s (128 x 128 x 128 x 1 GTU, 1 GTU = 2.5 us). The CPU generates 3 types of files, CPU_RUN_MAIN
containing the standard data acquistion, CPU_RUN_SC
for S-curve data and CPU_RUN_HV
for HV data. All data files have a matryoshka structure that is summarised below.
The CPU_RUN file¶
Files named CPU_RUN_....dat
are the output files of the Mini-EUSO software. Information is what this file contains is stored in the CpuFileHeader
which is put together at runtime. This structure has the following fields:
spacer
: a HEX ID tag 0xAA55AA55 for easy checking of the files in a hex viewerheader
: a set of bytes defining the instrument, file type and file version (they types and versions are defined inminieuso_data_format.h
and the header is put together inDataAcquisition::BuildCpuHeader()
)run_info
: a text field containing the information on the command line options and configuration at runtime (put together inDataAcquisition::BuildCpuFileInfo()
)run_size
: the number ofCPU_PACKET
in the standard acquisition run
The file is closed with a CpuFileTrailer
. This also contains the spacer
and run_size
fields as well as crc
which stores a 32 bit CRC, calcluated over the whole file excluding the trailer and appended using SynchronisedFile::Checksum()
accessed through Access::GetChecksum()
within DataAcquisition::CloseCpuRun()
.
The
CPU_RUN_MAIN
file format
At present the Zynq data format is fixed so that each ZYNQ_PACKET
contains:
N1
: the number of D1 packetsN2
: the number of D2 packetslevel1_data
:N1
x 128 GTU packet of D1 data (1 byte/pixel)level2_data
:N2
x 128 GTU_L2 packet of D2 data (2 bytes/pixel)level3_data
: 1 x 128 GTU_L3 packet of D3 data (4 bytes/pixel)
The data format holds for both triggered and non-triggered readout.
The
CPU_RUN_SC
file format
The CPU_RUN_SC
has a fixed size which represents the maximum number of threshold steps (0 - 1023). For S-curves taken over a smaller threshold ranges, the file is simply padded with the value 0xFFFFFFFF
. S-curve accumulation is calculated on-board the Zynq FPGA using the HLS scurve_adder (https://github.com/cescalara/zynq_ip_hls) allowing for S-curves to be taken with high statistics and stored in a small file size.
The
CPU_RUN_HV
file format
This file also has a fixed size and is used to store information on the HV status at the end of a run. This information is additional and complementary to that stored inside the ZYNQ_PACKET
.
The format is described in detail by the two header files minieuso_pdmdata.h
(the Zynq data format - depends on the firmware version) and minieuso_data_format.h
(the CPU data format - depends on the CPU software version). The minieuso_data_format.h
file is documented below.
A 32 bit CRC is calculated for each CPU_RUN
file prior to adding the CpuFileTrailer (the last 10 bytes). This CRC is appended to each CPU_RUN
file as part of the CpuFileTrailer.
minieuso_data_format.h¶
Defines
-
VERSION 9.7¶
-
VERSION_DATE_STRING "29/04/2022"¶
-
INSTRUMENT "Mini-EUSO"¶
-
INSTRUMENT_ME_PDM 1¶
-
ID_TAG 0xAA55AA55¶
-
ID_TAG_SUB 0xBB66BB66¶
-
RUN_SIZE 25¶
-
RUN_INFO_SIZE 512¶
-
CPU_FILE_TYPE 'C'¶
-
SC_FILE_TYPE 'S'¶
-
HV_FILE_TYPE 'H'¶
-
SC_FILE_VER 1¶
-
HV_FILE_VER 1¶
-
CPU_FILE_VER 1¶
-
THERM_PACKET_TYPE 'T'¶
-
HK_PACKET_TYPE 'H'¶
-
HV_PACKET_TYPE 'V'¶
-
SC_PACKET_TYPE 'S'¶
-
CPU_PACKET_TYPE 'P'¶
-
TRAILER_PACKET_TYPE 'Q'¶
-
THERM_PACKET_VER 1¶
-
HK_PACKET_VER 1¶
-
HV_PACKET_VER 1¶
-
SC_PACKET_VER 2¶
-
CPU_PACKET_VER 2¶
-
N_CHANNELS_PHOTODIODE 4¶
-
N_CHANNELS_SIPM 64¶
-
N_CHANNELS_THERM 10¶
-
MAX_PACKETS_L1 4¶
-
MAX_PACKETS_L2 4¶
-
MAX_PACKETS_L3 1¶
-
struct CpuFileHeader¶
- #include <minieuso_data_format.h>
cpu file header 12 bytes
-
struct CpuFileTrailer¶
- #include <minieuso_data_format.h>
cpu file trailer 12 bytes
-
struct CpuPktHeader¶
- #include <minieuso_data_format.h>
generic packet header for all cpu packets the zynq packet has its own header defined in minieuso_pdmdata.h 16 bytes
-
struct SubPktHeader¶
- #include <minieuso_data_format.h>
generic packet header for all hk sub packets the zynq packet has its own header defined in minieuso_pdmdata.h 16 bytes
-
struct CpuTimeStamp¶
- #include <minieuso_data_format.h>
timestamp 4 bytes unix time in s
Public Members
-
uint32_t cpu_time_stamp¶
-
uint32_t cpu_time_stamp¶
-
struct THERM_PACKET¶
- #include <minieuso_data_format.h>
thermistor packet for temperature data 60 bytes
Public Members
-
CpuPktHeader therm_packet_header¶
-
CpuTimeStamp therm_time¶
-
float therm_data[N_CHANNELS_THERM]¶
-
CpuPktHeader therm_packet_header¶
-
struct HK_PACKET¶
- #include <minieuso_data_format.h>
housekeeping packet for sipm and photodiode data 296 bytes
Public Members
-
SubPktHeader hk_packet_header¶
-
CpuTimeStamp hk_time¶
-
float photodiode_data[N_CHANNELS_PHOTODIODE]¶
-
float sipm_data[N_CHANNELS_SIPM]¶
-
float sipm_single¶
-
SubPktHeader hk_packet_header¶
-
struct ZYNQ_PACKET¶
- #include <minieuso_data_format.h>
zynq packet passed to the CPU every 5.24 s variable size, depending on configurable N1 and N2 NB: vector itself is not written to file, just contents which are contiguous in memory
-
struct CPU_PACKET¶
- #include <minieuso_data_format.h>
CPU packet for incoming data every 5.24 s variable size
-
struct CPU_FILE¶
- #include <minieuso_data_format.h>
CPU file to store one run shown here as demonstration only variable size
Public Members
-
CpuFileHeader cpu_file_header¶
-
CPU_PACKET cpu_run_payload[RUN_SIZE]¶
-
CpuFileTrailer cpu_file_trailer¶
-
CpuFileHeader cpu_file_header¶
-
struct SC_PACKET¶
- #include <minieuso_data_format.h>
SC packet to store S-curve from Zynq 9437220 bytes
Public Members
-
CpuPktHeader sc_packet_header¶
-
CpuTimeStamp sc_time¶
-
uint16_t sc_start¶
-
uint16_t sc_step¶
-
uint16_t sc_stop¶
-
uint16_t sc_acc¶
-
Z_DATA_TYPE_SCURVE_V1 sc_data¶
-
CpuPktHeader sc_packet_header¶
-
struct SC_FILE¶
- #include <minieuso_data_format.h>
SC file to store a single S-curve shown here as demonstration only 9437244 bytes (~9 MB)
-
struct HV_PACKET¶
- #include <minieuso_data_format.h>
HV packet for HV switching data variable file size
Public Members
-
CpuPktHeader hv_packet_header¶
-
CpuTimeStamp hv_time¶
-
uint32_t N¶
-
ZynqBoardHeader zbh¶
-
std::vector<DATA_TYPE_HVPS_LOG_V1> hvps_log¶
-
CpuPktHeader hv_packet_header¶
-
struct HV_FILE¶
- #include <minieuso_data_format.h>
HV file to store a single HVPS log shown here as demonstration only variable file size