v3.5.0

Description

New and Improved Uploader Tool

• With the new uploader tool, you can save and recall "Uploader Configs", which map to instrument time series within MIDAS
  • Accepted formats include TOA5 (.toa5 and .dat), CSV, XLSX, and DUX (specific to inclinometers)
  • Optional "instrument" column for matrix assignments where you have many instruments with the same timeseries
  • Assign custom "masked" and "validated" boolean and "comment" text columns
  • Accepts arbitrary date formats and time zone overrides

Custom Collection Group Order

• Users can now sort collection group time series for more convenient data entry

Survey123 Data API

• Exposes the API for Survey123 connections, which similarly map data from the ArcGIS Survey123 into MIDAS via Webhooks
  • NOTE: The telemetry API itself for creating these connections is pending network firewall configuration, look out for this feature in an upcoming release

Measurement Auto-Masking

• If a measurement is uploaded with a value that cannot be parsed as a number, it is considered NAN and is automatically masked. This behavior can be overridden is desired.

Reliability and performance Improvements

• Code generation to improve type safety with data layer
• Database batching to decrease round trips to the database and make uploads/batch inserts more efficient

New Endpoints

Inclinometer endpoints are now consistent with other depth-based instrument schemas:

GET /instruments/incl/:instrument_id/segments
GET /instruments/incl/:instrument_id/measurements
PUT /instruments/incl/:instrument_id/segments

Survey123 endpoints for managing AGS04 Survey123 Telemetry connections (note: Survey123 telemetry/measurement upload awaiting infrastructure updates).

GET /projects/:project_id/survey123
GET /projects/:project_id/survey123/:survey123_id/previews
POST /projects/:project_id/survey123
PUT /projects/:project_id/survey123/:survey123_id
DELETE /projects/:project_id/survey123/:survey123_id

Uploader endpoints for creating and saving upload configurations.

GET /projects/:project_id/uploader_configs
GET /projects/:project_id/uploader_configs/:uploader_config_id/mappings
POST /projects/:project_id/uploader_configs
PUT /projects/:project_id/uploader_configs/:uploader_config_id
DELETE /projects/:project_id/uploader_configs/:uploader_config_id
POST /projects/:project_id/uploader_configs/:uploader_config_id/mappings
PUT /projects/:project_id/uploader_configs/:uploader_config_id/mappings
DELETE /projects/:project_id/uploader_configs/:uploader_config_id/mappings
POST /projects/:project_id/uploader_configs/:uploader_config_id/uploads

Deprecated Endpoints

A number of API endpoint are deprecated with this release -- this would normally be a major patch revision, however these endpoints are either 1) Currently non-functional or 2) Have a paginated revision.

Legacy inclinometer endpoints were non-functioning, all inclinometer data is functionally the same as other depth-based instruments (IPI, SAA). Please see notes above on new endpoints for more information.

POST /inclinometer_explorer
POST /projects/:project_id/inclinometer_measurements

This instruments endpoint is unbounded and unused by the web client. In a future release we may need to have a paginated version.

GET /instruments

All constants endpoints have been removed, as they are no longer used by any mechanism within the API. Please query timeseries for an instrument and filter by type.

GET /projects/:project_id/instruments/:instrument_id/constants
POST /projects/:project_id/instruments/:instrument_id/constants
PUT /projects/:project_id/instruments/:instrument_id/constants/:timeseries_id
DELETE /projects/:project_id/instruments/:instrument_id/constants/:timeseries_id

This instrument_notes endpoints is unbounded and unused by the web client. In a future release we may need to have a paginated version.

GET /instruments/notes

What's Changed

• handle null/undefined/nan/empty string for measurement values by @dennisgsmith in #241
• add sqlc; performance and reliability improvements; large refactor by @dennisgsmith in #243
• Migrate Survey123 to use sqlc by @dennisgsmith in #245
• Add xlsx and instrument matrix to uploader by @dennisgsmith in #244

Full Changelog: v3.4.3...v3.5.0