Custom Data Editing

ODTK keeps a list of tracking data files (also known as measurement files) that are polled for data as needed by various processes, namely filter and least squares. Methods are provided for thinning measurements and including and/or excluding measurements over specific intervals before supplied to the requesting process. The user may specify that measurements which are being excluded either be completely ignored (skipped) or modeled but not included in the solution (force rejected).

Besides keeping bad or unwanted data out of the LS and filter processes, the editors can be used to clean unwanted data out of files or simply save off specific time spans of a file, since a method to save measurements is available.

This set of attributes makes it possible to edit tracking data based on any combination of primary object (satellite, emitter, or GNSS Satellite), tracking strand, and measurement type. Note for GNSS measurements, the primary object is always the GNSS Constellation; use the SelectedTrackingStrand to select a specific receiver attached to a satellite or facility.

When defined at the scenario level under the Measurements.ViewAndSave scope, the custom data editing schedule will be used when reporting raw measurements (in reports accessing the TrackingData database) or when saving measurements to a file. It will not be used to modify data available to filter, least squares, or any other process.

This same set of attributes is also defined for the Filter and Least Squares objects, so that each object can have its own set of data editing rules. These attributes are:

Custom Data Editing
Parameter Description
Enabled Turn on or off the entire data editing schedule.
Schedule A list of edit entries, described below.

Click the Schedule property to display a List window in which you can add, edit, copy and remove custom data editing entries. To copy an entry, select it using the Up and Down buttons and click the Copy button.

When you click the Add button or Copy button, a new line appears in the list, in which editing can be defined for selected objects and trackers and selected measurement types. To edit this line, click in each field and select or enter the appropriate information:

Editing Schedule Definition
Field Description
Enabled Set to true to apply the selected editing criteria to the measurements to be processed.
Action

Can be set to Process, Ignore, ForceReject or Thin.

A Process record allows you to specify data you want to process, and the Intervals are considered to be Inclusion Intervals: data outside those intervals will not be processed.

Ignore allows you to skip, or ignore, measurements, while ForceReject means any measurements meeting the requirements will have residuals modeled and reported but are marked for rejection by the process, and are not included in the solution. With the Ignore and ForceReject records, the Intervals specify times over which to exclude data. If no intervals are specified, then all data is excluded or rejected. Note that at the scenario level, ForceReject should not be used as no residuals are modeled when viewing or saving measurements.

If you want to Thin data, create a Thin record and specify the Thinning Time. Thinning will be applied to any measurements surviving the culling process by the Process, Ignore and ForceReject records. Thus, if you want to force thinning to begin at a certain time, you can first Ignore those measurements up to that time.

PrimaryObjects

Select among:

  • All Emitters - applies to all emitters in the scenario


  • All Satellites - applies to all satellites in the scenario


  • GNSS Constellation - applies to the satellites in the GNSS constellation OR a single, specific GNSS satellite if one exists as a GNSSSatellite object in ODTK


  • Selected Emitter - applies to a single, selected emitter


  • Selected Satellite - applies to a single, selected satellite
SelectedObject If appropriate (see above), select the emitter, satellite or GNSS satellite to which the editing is to be applied.
Trackers Select between All Trackers and Specific Tracker. Because each tracking strand contains specific tracking elements, "Specific Tracker" is an activator for the SelectedTrackingStrand option. After opting for "Specific Tracker", you can make the tracking strand listing visible by clicking below the SelectedTrackingStrand header. Then, from the list of relevant tracking strands, you can select the strand containing the specific tracking elements to which the editing would apply.
SelectedTrackingStrand Select the tracking strand to which the editing is to be applied.
MeasType Select between All and Specific. Selecting All means edit criteria may be applied to all measurements. Selecting Specific will activate the MeasTypes list, and you may click MeasTypes to enter the measurement types for which the editing would apply.
MeasTypes Select each measurement type to which the editing would apply.
ThinningTime

Only active when Action is set to Thin.

Thinning time is an exclusion period, a time following a measurement during which no other measurements will be considered. For instance, if you have measurements in a tracking file at 5 second intervals, and you set the ThinningTime to 30 seconds, you will end up having measurements processed every 30 seconds. Thinning is performed for each unique primary object - tracking strand - measurement type combination, except for GNSS measurements where it is GNSS receiver specific.

Intervals

Only active when Action is Process, Ignore or ForceReject.

The List dialog that appears will contain a line for the definition of the interval. Click the Add button to insert additional sub-intervals. On each line, set the Enabled value to true if the sub-interval is to apply; otherwise set it to false. Edit the Start and Stop times as desired. The first interval added has a default span based on the scenario's DefaultTimes.Intervals.TimeSpan attribute. To delete a line, select it and click the Remove button. To clear the list, click Remove All. To re-order intervals, select one in the list and click the Up or Down button.

NOTE: If no specific interval is set, then all measurement times are considered.

NOTE: The default start and stop times for the first new interval added are set based on these rules.

It is possible to set ambiguous directives with this Schedule. When designing a complex custom data editing schedule, you should follow the rule of starting with the most general rules first and the most specific rules last, and keep in mind a rule must include a measurement before another rule can [partially] exclude, reject or thin it*. When evaluating the schedule entries, the last entry which applies to a specific measurement will be used. We suggest starting with a default Process entry (though not necessary*) and then following with Ignore or ForceReject entries. Thinning records are only applied after the Process/Ignore/FR rules, so they can go anywhere in the list; however, if there is more than one Thinning record the later ones take precedence.

For instance, given a set of tracking data at 10-second steps, one schedule entry could say we want all Range measurements thinned to 2 minutes, and the next entry could say we want all Facility1 measurements thinned to 30 seconds. So what happens to Facility 1 Range measurements - are they thinned to 30 seconds or 2 minutes? Since the last applicable schedule entry is used, in this case the 2nd rule would take precedence and they would be thinned to 30 seconds. If the rules were reversed, they would be thinned to 2 minutes.

Also, if we add a specific Process entry, such as Emitter2-All Trackers-TDOA, and we add a second entry to Ignore Emitter3-All Trackers-FDOA over a certain time period, we will find Emitter3 measurements were ignored over all times. That is because there must have first been a record to cause them to be processed. One solution would be change the Ignore action to Process and change the Intervals from our exclusion times to the desired inclusion times. Another solution would be to create a separate Process flag for Emitter3-FDOA and move its list position to just before the Ignore record.

* If there are only Ignore, FR or Thin entries in the Schedule, we attempt to add a default Process entry when considering the Schedule. However, when using the Ignore or ForceReject records, you should add your own Process record for sanity checking.

Additional notes about Thinning Time

Starting in ODTK 6.0, measurements that are thinned may have their own independent thinning grid based on the measurement type and the full tracking strand. For instance, if range measurements are thinned to 10 seconds, and range measurements overlap for two facilities, the resulting measurement times may be something like:

  1. 00:00:32 Facility1 / Satellite1
  2. 00:00:35 Facility2 / Satellite1
  3. 00:00:42 Facility1 / Satellite1
  4. 00:00:45 Facility2 / Satellite1

The 1st and 3rd times are 10 seconds apart, as are the 2nd and 4th times, but the two time grids are 3 seconds apart.

There is an exception to this rule when processing GNSS measurements. In that case, measurements are thinned considering only the GNSS receiver ID and not the associated PRN (or PRNs for singly- or doubly-differenced measurements). Doubly-differenced GNSS measurements are thinned solely based on time and measurement type. There is no need to specify a primary object or tracking strand.

Also it is possible to move the thinning start time reference by using either an Ignore or Process record. For instance, if you have range data at 1 second starting at 11:57:47, and you want to thin it to 30 seconds in a filter run starting at 12:00:00, but don't want a measurement grid of 17, 47, 17, 47, etc., you can either create an Ignore record to skip the data up through 11:59:59 which will put the resulting range measurements at 12:00:00, 30, 00, 30, etc. Or you can create a Process record and specify the interval start at 12:00:00 to whenever.

Additional notes about Interval Times

The Intervals property allows you to define one or more subsets of measurements to be used in ODTK. For instance, if a measurement file contains 12 days of data, and you want to perform an IOD and run LS over the fifth and sixth days, you can (besides setting your LS start and stop times) set up a Process record with an Intervals entry that starts on the fifth day and ends at the start of the seventh day. Then only measurements during this time span will be considered.

For Ignore and ForceReject records, the Intervals property excludes measurements over the specified intervals. In the above example involving 12 days of data, you could achieve the same results by creating an Ignore record with two Intervals, one excluding all data up to the fifth day, and one excluding all data from the seventh day onward.

If there are no Interval times defined, then all times are considered, either to be included or excluded. For instance, if we have a Process record which specifies Satellite1, Facility2, and Range measurements, and there are no Intervals, then all available Satellite1-Facility2-Range measurements will be processed/viewed/saved. If there is an Interval defined, then only the Satellite1-Facility2-Range measurements over those times will be used; Satellite1-Facility2-Range measurements not within those intervals will be ignored. And if we want to exclude all Satellite1-Facility2-Doppler measurements, we can create an Ignore record for Satellite1-Facility2-Doppler and leave the Intervals empty.

ODTK 6.5