002 - Ephemeris Data
Purpose
This document tackles the problem of ephemeris data in the Monitor Space Hazards service.
Context and scope
A lot of the context for this paper is provided in the Ephemeris tech spike presentation.
In summary, ephemeris data is orbit information that is shared between participants in a conjunction event. It comes in the form of a series of state vectors, which are cartesian vectors detailing position, velocity and optionally accelerations and covariance matrices.
The format of an Orbital Ephemeris Message (OEM) is specified by the Consultative Committee for Space Data standards in their orbit data messages standard. Each message is a plain text file containing a single object’s orbit, ephemeris and covariance data, along with optional comments.
The business context is that Satellite Operators generate ephemeris data for their assets and combine it with services like Space Track in order to receive more accurate Conjunction Data Messages (CDMs). Given that fuel on a launched satellite is finite, it is burnt for collision avoidance manoeuvres as a last resort.
.
Goals and non-goals
The goal of this design doc is to articulate a design for the structure of the Conjunction Event Id, not the data flow.
Goals
- Ensure that OEM data is well formed according to the orbit data messages specification.
- Validate and hygiene check the OEM to mitigate the risk of malicious payloads.
Non-goals
- Anomaly detection for outlier OEMs that are well-formed but potentially erroneous, these will be dealt with by Orbital Analysts.
The actual design
There are two starting points in the user journey for this design.
The proposal is for each ID to consist of:
- Firstly, there is the Satellite Operator wanting to upload an OEM against one of their satellites. In this scenario they will be presented with a file upload component as per the GDS Design System.
- Secondly, we will ingest CDMs based on ephemeris data from Space-Track. The data source will be shown in the data source table. Monitor Space Hazards can also make an outbound request to Space Track for the latest ephemeris data it holds against satellites monitored in Monitor Space Hazards if the operator permits us to do so.
Regardless of which route the ephemeris data comes via, the following will be enforced:
- The file extension must be “.oem”
- ensure it is the last suffix and double (or more) suffixing is rejected, e.g. “.oem.exe”
- File name will likely be used to link the oem file to a satellite. Space-Track asks operators to put the satellite name and NORAD ID in the file name, so this solution would be familiar to operators & is evidently sufficient. See file naming convention at the Space-Track user guide.
- Return an error if the file contains any binary data, it should only contain ASCII.
- Ensure the message complies with the specification outline in section 5.2 of the orbit data messages specification, reject the message for any non-compliance
“Is it safe to assume we are only accepting KVN (keyword = value notation) OEMs?”
Why no anti-virus check? If the above validations and hygiene checks have passed then the ability to upload a viral payload is mitigated, we are assured that the file is a well-formed OEM (in plain text ASCII) which will now be loaded into our service, and any downstream system will benefit from our validation and hygiene checks. The runtime environment of Monitor Space Hazards will ensure no ephemeris data is executed in anything other than a controlled manner, e.g. we will not be allowing “.oem.exe” and allowing remote shell trojan like access.
For MVP we believe that the main responsibility for Monitor Space Hazards is to act as an ephemeris message conduit, allowing the appropriate parties to upload and disclose their ephemeris data to each other. This is somewhat akin to the notion of a message broker, but with much reduced functionality. Given this, the proposition for MVP is to extract and load into structured tables the minimal data required, for this it is the OEM header part (see 5.2.2) and the object_id from the metadata. Outside this, a unique ID will be needed for each message, a pointer to the location for the message stored on a shared file-system (AWS S3 backing service) and file checksum to assure data integrity.
For MVP the minimal feature for exporting this data is to provide a download / export button for each satellite entity. This will be available to orbital analysts. When there is ephemeris data to download for a satellite/event, a green button will be clickable for orbital analysts. If the analyst chooses to click on the ephemeris data button but not download it to perform analysis, they will be prompted to explain why. whWhen there is no such data available, the button will not appear.
User access to a specific satellite’s data is controlled by the security model, e.g. satellite operators can only see their own satellites or those that a conjunction event with one of theirs is raised for. Post-MVP an API for programmatic access will be implemented, indeed there is an existing request from Aurora for this functionality. MOD would also like access to ephemeris files from operators down the line too.
Process for ingestion:
- Operator uploaded OEM file in association to a satellite
- OEM passes validation checks
- Operator user receives an automatic confirmation that file has been accepted
- OEM file saved to database in association to satellite
- OEM file listed in table on satellite page
. - OA users receive an email informing them that an OEM file has been uploaded in relation to X satellite
APIs
We offer a public API for management of ephemeris - this is documented in our OpenAPI specification under ephemeris
Data Storage
We use the following table structure to store information about ephemeris:
The definitions below were used in dbdiagram.io to generate the above diagram:
Table ephemeris as E {
id uuid [pk, unique]
creation_date timestamp
originator varchar
object_name varchar
international_designator varchar
ref_frame varchar
time_system varchar
start_time timestamp
stop_time timestamp
center_name varchar
file_name varchar
satellite varchar
uploader uuid
uploader_organization uuid
is_active boolean
deleted_by_id uuid
restored_by_id uuid
}
Table satellite as S {
id uuid [pk, unique]
norad_id varchar // NORAD_id
common_name string
}
Table organizations as O {
id uuid [pk, unique]
}
Table user as U {
id uuid
}
Ref: E.satellite - S.norad_id
Ref: E.uploader - U.id
Ref: E.uploader_organization - O.id
Code and pseudocode
There are no novel algorithms to describe in code or pseudocode for this design.
Degrees of constraint
Monitor Space Hazards is a greenfield development so is relatively free of constraint.
OEM files must not be shared outside of Government & not stored for longer than 30 days (without reason).
Alternatives considered
The main alternatives considered were:
- Loading the entire OEM into structured data tables. This was discounted because of the substantially increased complexity, with little benefit since downstream systems are already handling the variations in OEM provision already and that data analysis is also taking place downstream. This is where Monitor Space Hazards responsibilities are limited to message validation, routing and integrity, akin to a message broker, were validated.
- An API first approach was considered, but for the sake of protecting the MVP a manual download is thought to be a viable first-step. The provision of an API thereafter will be a backlog item to prioritise based on needs.
Cross-cutting concerns
It should be considered if this approach to OEMs, if accepted, should be applied to CDMs also, is it a general pattern for MVP?