Detecting Events using Continuous Time

Overview

Moxie allows external tools to perform complex analysis over time naturally, instead of forcing them to use shared discrete time steps. This is especially helpful for analytical calculations that do not require time steps at all. The Moxie Engine can then ingest this analysis as time-varying values and use those values for triggering change events. In this section, you will use STK Access to compute the BooleanValue for when the sensor should take a photo. You will also control the camera in the STK scenario to point at and actually take photos of the objects of interest.

This section covers the following concepts:

• Continuous event detection
• Moxie's DataProviderExecutor

public SensorPayloadDelegate( ... ) {
     ...
    //Compute isDetectingObject on demand.
    isDetectingObjectProperty = new DerivedProperty<>(timeProvider, () -> {
        if (mStkRemainingPlaceObjects.size() > 0) {
            List<BooleanValue> results = mStkRemainingPlaceObjects.stream().map(this::canAccessObject).collect(Collectors.toList());
            return BooleanValue.any(results);
        } else {
            return BooleanValue.never();
        }
    });
     ...
}

private BooleanValue canAccessObject(IAgStkObject object) {
    StkTimeHelper stkTimeHelper = mStkToolbox.getTimeHelper();
    double currentTime = stkTimeHelper.moxieTimeToStkEpochSeconds(mTimeProvider.getCurrentTime());
    double endTime = stkTimeHelper.moxieTimeToStkEpochSeconds(mTimeProvider.getIntervalFromCurrentToStop().getStop());

    //Check for access from now until the end of the simulation.
    IAgStkAccess accessToObject = mStkSensorObject.getAccessToObject(object);
    accessToObject.setAccessTimePeriod(AgEAccessTimeType.E_USER_SPEC_ACCESS_TIME);
    accessToObject.specifyAccessTimePeriod(currentTime,endTime);
    List<Interval<MoxieTime>> access = mStkToolbox.computeAccess(accessToObject);
    return BooleanValue.fromTrueIntervals(access);
}

Figure A1: Computing a time-varying value on demand

Prerequisites

Instructions

Add the import statements and private fields

You can add all of the import statements and the field declarations and initializations that you will need in this section in advance, so that you are not slowed down by adding them in each of the later subsections.

1. In the SensorPayloadDelegate class, add the following import statements:

   import agi.stkobjects.AgEAccessTimeType;
   import agi.stkobjects.AgESTKObjectType;
   import agi.stkobjects.IAgSensor;
   import agi.stkobjects.IAgStkAccess;
   import com.agi.moxie.api.Interval;
   import com.agi.moxie.api.MoxieTime;
   import com.agi.moxie.api.SimulationLogger;
   import com.agi.moxie.api.models.DerivedProperty;
   import com.agi.moxie.stk.utilities.DataProviderExecutor;
   import com.agi.moxie.stk.utilities.DataProviderResultTable;
   import com.agi.moxie.stk.utilities.StkTimeHelper;
   import java.lang.Math;
   import java.util.ArrayList;
   import java.util.List;
   import java.util.stream.Collectors;

2. Add the following private field declarations:

   private TimeProvider mTimeProvider;
   private SimulationLogger mSimulationLogger;
   private IAgSensor mStkSensor;
   private ArrayList<IAgStkObject> mStkRemainingPlaceObjects = new ArrayList<>();
   private final String mOutputDirectory;
   private int mImageId = 0;

3. In the constructor, add the following dependency injection and field initializations:

   public SensorPayloadDelegate( ...
           @InjectByName("simulationLogger") SimulationLogger simulationLogger,
            ... ) {
       mTimeProvider = timeProvider;
       mSimulationLogger = simulationLogger;
       mOutputDirectory = stkToolbox.sendConnectCommand("GetDirectory / Scenario").getConnectCommandData().get(0);
        ...
   }

Detect the objects of interest using STK Access

Moxie requires change event triggers to either be a boolean primitive or a Moxie BooleanValue, which represents a boolean primitive that potentially varies over time. For example, to use a ScalarValue in a change event, you would first need to compare it against another value to convert it into a BooleanValue. For the when(isDetectingObject) change event in the SensorPayload State Machine, you can use STK Access to compute when the sensor detects objects of interest, and then use the STK toolbox to help convert it into a BooleanValue.

In the next subsection, you will add code that changes the sensor's field of view during the simulation, potentially invalidating the current Access results from that point on. So in this subsection, you can change the type of the sensor's isDetectingObjectProperty field to DerivedProperty to allow its value to be determined on demand instead of only being initialized at the start of the simulation.

1. In the declaration of the isDetectingObjectProperty field, replace the BasicProperty<BooleanValue> type with DerivedProperty<BooleanValue>.
2. In the constructor, remove the following parameter and initialization:

   @OptionalInjectBySlot("isDetectingObject") BooleanValue isDetectingObject,

   isDetectingObjectProperty = new BasicProperty<>(timeProvider, isDetectingObject);

3. In the initializeStkObject() method, add the following code to prepare for searching for objects of interest:

   mStkSensor = (IAgSensor) mStkSensorObject;
   
   //Get the list of all place objects in the scenario.
   for (IAgStkObject placeObject : mStkToolbox.getRoot().getCurrentScenario().getChildren().getElements(AgESTKObjectType.E_PLACE)) {
       mStkRemainingPlaceObjects.add(placeObject);
   }

4. In the startSearching() method, add the following code to start searching for objects of interest:

   mSimulationLogger.info("Searching for objects...");
   double currentTime = mStkToolbox.getTimeHelper().moxieTimeToStkEpochSeconds(mTimeProvider.getCurrentTime());
   
   //Set the sensor field of view to approximate horizontal sweeps of the camera.
   mStkToolbox.sendConnectCommand(String.format("Point %s Schedule AddAzEl \"%s\" 0 90", mStkSensorObject.getPath(), currentTime));
   mStkSensor.getCommonTasks().setPatternRectangular(stkSearchWidthAngleProperty.getValue() / 2, stkFieldOfViewAngleProperty.getValue() / 2);
   
   //Display the "Camera View" 3D window.
   mStkToolbox.sendConnectCommand("Window3D * Raise WindowID 2");
   mStkToolbox.sendConnectCommand(String.format("Window3D * ViewVolume FieldOfView %s WindowID 2", stkSearchWidthAngleProperty.getValue()));

5. Create the following private helper method to check when the sensor will be able to access a particular object:

   private BooleanValue canAccessObject(IAgStkObject object) {
       StkTimeHelper stkTimeHelper = mStkToolbox.getTimeHelper();
       double currentTime = stkTimeHelper.moxieTimeToStkEpochSeconds(mTimeProvider.getCurrentTime());
       double endTime = stkTimeHelper.moxieTimeToStkEpochSeconds(mTimeProvider.getIntervalFromCurrentToStop().getStop());
   
       //Check for access from now until the end of the simulation.
       IAgStkAccess accessToObject = mStkSensorObject.getAccessToObject(object);
       accessToObject.setAccessTimePeriod(AgEAccessTimeType.E_USER_SPEC_ACCESS_TIME);
       accessToObject.specifyAccessTimePeriod(currentTime,endTime);
       List<Interval<MoxieTime>> access = mStkToolbox.computeAccess(accessToObject);
       return BooleanValue.fromTrueIntervals(access);
   }

6. In the body of the constructor, add the following code to check on demand when the sensor will be able to access any of the remaining objects of interest:

   //Compute isDetectingObject on demand.
   isDetectingObjectProperty = new DerivedProperty<>(timeProvider, () -> {
       if (mStkRemainingPlaceObjects.size() > 0) {
           List<BooleanValue> results = mStkRemainingPlaceObjects.stream().map(this::canAccessObject).collect(Collectors.toList());
           return BooleanValue.any(results);
       } else {
           return BooleanValue.never();
       }
   });

7. Open the UAVMission.mdzip project from the previous section.
8. In the Instance Specification Diagram, double-click the uav.sensorPayload instance specification and select Slots from the left pane of the specification window.
9. In the center pane, select the isDetectingObject slot and click Remove Value. Then click Close.
10. Save your work and re-install the delegate module.
11. Run () the simulation and observe that the STK scenario now shows each time that the UAV detects an object of interest.
12. Close STK without saving.

Focus the camera using the DataProviderExecutor

Once the sensor detects an object of interest, it needs to know where to focus to be able to capture the object in a photo. You can use the DataProviderExecutor to get the azimuth and elevation values to point the camera at the object.

1. Still in the SensorPayloadDelegate class, create the following private helper method to get the object the sensor is currently detecting:

   private IAgStkObject getDetectedObject() {
       for (IAgStkObject placeObject: mStkRemainingPlaceObjects) {
           if(this.canAccessObject(placeObject).toBooleanSignal().getValueAt(mTimeProvider.getCurrentTime())) {
               return placeObject;
           }
       }
       return null;
   }

2. Create the following private helper method to get the direction from the sensor to an object:

   private double[] getAzElToObject (IAgStkObject object) {
       MoxieTime currentTime = mTimeProvider.getCurrentTime();
       double offset = calibrationDurationProperty().getValue().toTimeDurationSignal().getValueAt(currentTime).getTotalSeconds();
   
       //Get the current azimuth and elevation to the object
       DataProviderResultTable resultTable = DataProviderExecutor.getAccessAsTable(mStkSensorObject, object, "AER Data/BodyFixed",
           currentTime, currentTime.addSeconds(offset), (offset/3), "Azimuth,Elevation");
       double actualAzimuth = resultTable.getRow(0).asDouble("Azimuth");
       double actualElevation = resultTable.getRow(0).asDouble("Elevation");
   
       //Constrain the pointing of the sensor to within the search area (azimuth = -90 for left, 90 for right).
       double constrainedAzimuth = 90 * Math.signum(actualAzimuth);
       //Correct the pointing elevation for the constrained azimuth (constrainedElevation ~= actualElevation).
       double constrainedElevation = Math.toDegrees(Math.atan(
       Math.tan(Math.toRadians(actualElevation)) / Math.cos(Math.toRadians(constrainedAzimuth-actualAzimuth)) ));
   
       return new double[]{constrainedAzimuth, constrainedElevation};
   }

3. In the focusOnObject() method, add the following code to point the sensor at the detected object:

   mSimulationLogger.info("Focusing on object...");
   double currentTime = mStkToolbox.getTimeHelper().moxieTimeToStkEpochSeconds(mTimeProvider.getCurrentTime());
   
   //Get the direction to the object.
   double[] azel = this.getAzElToObject(this.getDetectedObject());
   
   //Set the sensor field of view to point at the object.
   mStkSensor.getCommonTasks().setPatternRectangular(stkFieldOfViewAngleProperty.getValue() / 2, stkFieldOfViewAngleProperty.getValue() / 2);
   mStkToolbox.sendConnectCommand(String.format("Window3D * ViewVolume FieldOfView %s WindowID 2", stkFieldOfViewAngleProperty.getValue()));
   mStkToolbox.sendConnectCommand(String.format("Point %s Schedule AddAzEl \"%s\" %s %s", mStkSensorObject.getPath(), currentTime, azel[0], azel[1]));

4. At the end of the snapPhoto() method, add the following code to resume searching for objects of interest:

   //Resume searching.
   this.startSearching();

5. Save your work and re-install the delegate module.
6. Run () the simulation again and observe that the STK scenario now shows the UAV camera pointing at each detected object of interest. Notice that the camera points at some objects multiple times. This is because it continues to detect them but has not yet taken a photo of them.
7. Close STK without saving.

Snap the photos

1. In the for loop of the initializeStkObject() method, add the following code to prepare for taking photos of objects of interest:

   //Hide the objects for now.
   mStkToolbox.sendConnectCommand(String.format("DisplayTimes %s State AlwaysOff", placeObject.getPath()));

2. In the beginning of the snapPhoto() method, add the following code to take a photo of the current sensor view:

   mSimulationLogger.info("Snapping photo...");
   
   //Check whether the camera can see the object.
   IAgStkObject detectedObject = this.getDetectedObject();
   if (detectedObject != null) {
       //Display the object and ignore it in the future.
       mStkToolbox.sendConnectCommand(String.format("DisplayTimes %s State AlwaysOn", detectedObject.getPath()));
       mStkRemainingPlaceObjects.remove(detectedObject);
   
       //Snap the photo.
       mImageId += 1;
       mStkToolbox.sendConnectCommand(String.format("VO * SnapFrame ToFile \"%sImageSnap%02d.png\" WindowID 2", mOutputDirectory, mImageId));
       mSimulationLogger.info(String.format("Image saved to \"%sImageSnap%02d.png\"", mOutputDirectory, mImageId));
   }

3. Save your work and re-install the delegate module.
4. Run () the simulation again and observe that the STK scenario now hides the objects of interest until the UAV takes photos of them. Notice that the camera occasionally takes multiple tries to capture an object in a photo, and fails entirely in one case. This is the emergent behavior that this simulation is designed to discover: the camera's ability to capture the objects of interest despite the UAV's motion changing the field of view, especially during turns.

   The photos are saved to the STK scenario folder: %UserProfile%\Documents\ 12 \UAVMission.

5. Close STK without saving.

Explore variations of the simulation

Congratulations on completing all of Moxie's tutorials! Feel free to continue to explore this simulation on your own to see more of the emergent behavior of the system. You may want to try varying some of the SysML properties, like the sensor's calibrationDuration, stkFieldOfViewAngle, and stkSearchWidthAngle. Or you could tweak the Java delegate code, for example by using the actualAzimuth and actualElevation to point the sensor instead of constraining the pointing.