Using Delete Expressions in Import Definitions for Datasets

[Q&A] General
1

(work in progress)

Deleting Data with the Import Engine

The import engine has a feature that allows you to delete data, which is especially useful when working with Datasets. For example, when copying REST endpoints and data definitions from Development to Test, Acceptance, and Production environments.

In some cases, fields in the Dataset may need to be deleted—such as when a REST parameter or node has been removed. In this case, you may want the Dataset to automatically delete these removed fields. Essentially, it will delete any elements no longer present in the dataset.

However, this functionality should be used with caution. You don’t want to delete too much data by mistake. This guide will explain how to safely delete specific fields from a dataset without removing too much.

Delete Mode

In earlier versions of the import engine, there was an option for a “Delete mode,” which provided two choices:

  1. Delete records that are not in the import file.
  2. Delete records that do not match the key values.

Based on feedback, the option to delete records not in the import file was removed in Novulo Import version 209, making the “Delete mode” obsolete. In version 208 or earlier, always select the option “Delete records that do not apply to the key values.”

Best Practices

  • Delete Expressions: Apply delete expressions only to sub-records that belong to the parent record in your dataset.
    • For example, when working with REST Data Endpoints, never apply a delete expression to the REST Data Endpoint itself. Instead, only delete the parameters or data definitions that are below the root level.

How to Safely Delete Data

A delete expression will remove all records of a certain datatype that match the delete criteria—except for records that already exist in the dataset.

Example: REST Data Endpoints and Parameters

Let’s consider a simplified version of REST Data Endpoints and their parameters:

RestDataEndpoint RestDataEndpoint RestDataEndpointParameter RestDataEndpointParameter RestDataEndpointParameter->RestDataEndpoint

When you export REST Data Endpoints, you may want to delete any parameters that have been removed from the endpoint in your dataset. However, you don’t want to delete parameters from other endpoints.

Steps to Configure the Delete Expression

  1. For the sub-datasets, add a delete expression that specifies which data should be deleted. The best practice is to set it as follows:
boolean:and([this.data_definition.guid.in(["TEMPLATEGUIDLIST"]), false])

This expression will delete all records where the data definition is in the provided list, except for those already in the import definition.

  1. You will also need an XSLT transformation to convert the TEMPLATEGUIDLIST into a list of GUIDs from the parent records.

  2. As a safeguard, the XSLT will change the false value to true as the final step. This ensures that if the XSLT is not executed properly, no data is deleted, preventing accidental data loss.