Serialization plugin (P173)

Wiki
1

Serialization plugin

The serialization plugin is used for parsing objects or data structures into a format that is understandable by the application (or API that is being communicated with). The Serialization plugin is for example used together with the REST or RestfulWebservice plugins. The NovuloRest and NovuloRestfulWebservice both use the Serialization plugin and models using those plugins should always include the Serialization plugin in their composition

The plugin has 2 actions

  • DeserializeObject - used to convert JSON, XML or FORM into a interpretable by the framework Novulo format
  • SerializeObject - used to convert a Novulo datatype object to a JSON, XML or FORM object (so in other words, the reverse of the Deserialize action)

Configuration

DeserializeObject action:

  • Output data type - the datatype of the supported by the Framework datatypes to deserialize the input value to.
  • Input data type - the datatype of the input object that has to be deserialized - supported are JSON, XML and Form

SerializeObject action:

  • Input data type - the datatype of the object that is being serialized to JSON, XML or FORM. This is an object in a Novulo datatype
  • Output content type - the required data type that the input has to be serialized to.
  • Remove underscore from _id keys - checkbox that can be set, if the input object has _id keys in it, and it is required that the _ are not part of the key. Setting it to true, will take care of removing the _ from the ID key

Parameters

Both actions have a parameter named input. In both cases it is required parameter and relates to the object that has to be serialized/deserialized.

Starting revision 96298 of the Serialization plugin:
The serialize action got a new optional parameter “skipemptynodes”. This parameter affects only when the XML serializer is used. When set to true, NULL or empty values will not be present in the resulting XML output.

Return values

Both actions have the same return values:

  • Success - true/false if the action succeeded in its purpose
  • Error - set when success is false and contains an error message explaining what went wrong
  • Output - Value, that contains the serialized/deserialized object in the defined in the plugin configuration format.

Starting revision 98718 of the Serialization plugin

With revision 98718 of the NovuloSerialization plugin there is a new parameter named “settings” for the SerializeObject and DeserializeObject actions.
This parameter is of datatype List<Tuple<String,Mixed>> and the values in this list, should be tuples with the following structure:
-Key - the option we want to set
-Value - the value for the option we want to set
Only valid option names and values should be provided, otherwise the action will throw an error, with as descriptive as possible message that explains which setting or value is wrong or not supported.

Note that currently this ONLY works for the JSON serializer - when serializing/deserializing to/from XML and Form, this parameter currently has no effect!

Example setup in the architect (further explanation of the new actions parameter and possible settings will follow down below):

1067×766 35.1 KB

JSON serializer settings
The possible settings and values can be seen at the following URL with the documentation for the newtonsoft JSON serializer (Serialization Settings - this gives reference to the newest assembly version, so some of the settings might not be present:

TODO find the documentation for the 5.0.6 version, which is used within Novulo environments).

Short explanation and the possible values will follow below.

  • removeUnderscoreFromIDs
    Optional parameter with default value “false”, when set to true “_id” will be serialized as “id”. This was already existing parameter. The new thing in this revision is that it can also be set in the new settings parameter using one of the following tuple keys:
  • removeUnderscoreFromId, Removeunderscorefromid or removeunderscorefromid. For Values the case doesn’t matter - so True,true,False,false.
    It is also possible to set this parameter from the Webconfig, by adding the key “Serialization.RemoveUnderscoreFromId”, value can be True,true,False,false (in this case as well, the case of the letters does not matter.)

For high-code developers, that want to use the Serialization plugin & more specifically the JSON serializer in other plugins:

  • The SerializationHelper class of the Serialization plugin has a new GetInstance method that has the following parameters:
  • SupportedTypes - this is a public Enum in the same class - values are JSON, XML, FORM - in the JSON serializer case, we care about only the JSON value !
  • JsonSerializerSettings - this is a class from the Newtonsoft.Json.JsonSerializerSettings namespace. Here you can set all the properties you need for the JsonSerializer (so you don’t have to use the new parameter of the SerializeObject action - instead of List<Tuple<String,Mixed>> with the settings names and values, you should set the Properties in this JsonSerializerSettings object yourself!)
    -removeUnderscoreFromIDs - optional param, with default false.

Possible values for the the tuples in the new “settings” parameter.
Note1- set only the ones you are sure you need, to avoid unexpected behaviour ! The options that are not set, use the default values defined by the Newtonsoft.JSON serializer.
Note2 - for the key values and the values of the tuples, the case of the characters does not matter for the logic of the plugin. So for ex.
key with value “culture” or “Culture” or “CuLtUrE” should be treated the same way, as long as it is spelled correctly ! Same goes for the values for these SerializerSettings - as long as it is a valid value, the casing of the letter should not matter.

Setting name and explanation Setting possible values
Culture - Sets the used culture when reading JSON. Default value is InvariantCulture - InvariantCulture
  • InstalledUICulture
  • CurrentUiCulture
  • CurrentCulture|
    |DateTimeConverter - Sets the used DateTime converter|- JavascriptDateTime - Converts a DateTime to and from a JavaScript Date constructor (e.g. new Date(52231943)).
  • ISO - Converts a DateTime to and from the ISO 8601 date format (e.g. “2008-04-12T12:53Z”).|
    |DateTimeZoneHandling - sets how DateTime time zones are handled during serialization and deserialization. The default value is RoundtripKind.|- Local - Treat as local time. If the DateTime object represents a Coordinated Universal Time (UTC), it is converted to the local time.
  • UTC - Treat as a UTC. If the DateTime object represents a local time, it is converted to a UTC.
  • Unspecified - Treat as a local time if a DateTime is being converted to a string. If a string is being converted to DateTime, convert to a local time if a time zone is specified.
  • RoundTripKind - Time zone information should be preserved when converting.|
    |DateFormatString - Gets or sets how DateTime and DateTimeOffset values are formatted when writing JSON text, and the expected date format when reading JSON text. The default value is “yyyy’-‘MM’-‘dd’T’HH’:‘mm’:'ss.FFFFFFFK”.|- For example for format"d MMMM, yyyy T HH:mm:sszzz", when serializing a DateTime object, that for example in a DateTime field has value 8/16/2022 11:00 AM to JSON (not a String DateTime value !!!) will result in : 16 augustus, 2022 T 11:00:00+02:00
  • Used settings to get the result above - the example dateTime format above, ISO DateTime converter, Unspecified DateTimezone handling.|
    |StringEscapeHandling - sets how strings are escaped when writing JSON text. The default value is Default.|-EscapeNonAscii - All non-ASCII and control characters (e.g. newline) are escaped.
    -EscapeHtml - HTML (<, >, &, ', ") and control characters (e.g. newline) are escaped.
    -Default - Only control characters (e.g. newline) are escaped|
    |Formatting - Specifies formatting options for the JsonTextWriter. Default is None.|-Indented - Causes child objects to be indented according to the Indentation and IndentChar settings.
    -None - No special formatting is applied. This is the default.|
    |||
    |FloatParseHandling - Specifies how floating point numbers, e.g. 1.0 and 9.9, are parsed when reading JSON text.(This doesn’t seem to currently have effect on Novulo Objects)|-Decimal - Floating point numbers are parsed to Decimal.
    -Double - Floating point numbers are parsed to Double.|
    |FloatFormatHandling - Specifies float format handling options when writing special floating point numbers, e.g. NaN, PositiveInfinity and NegativeInfinity with JsonWriter.(This doesn’t seem to currently have effect on Novulo Objects)|-Symbol - Write special floating point values as symbols in JSON, e.g. NaN, Infinity, -Infinity. Note that this will produce non-valid JSON.
    -DefaultValue - Write special floating point values as the property’s default value in JSON, e.g. 0.0 for a Double property
    -String - Write special floating point values as strings in JSON, e.g. “NaN”, “Infinity”, “-Infinity”.|
    |DefaultValueHandling - Specifies default value handling options for the JsonSerializer. (This doesn’t seem to currently have effect on Novulo Objects)|-IgnoreAndPopulate - Ignore members where the member value is the same as the member’s default value when serializing objects and set members to their default value when deserializing.
    -Ignore - Ignore members where the member value is the same as the member’s default value when serializing objects so that it is not written to JSON. This option will ignore all default values (e.g. null for objects and nullable types; 0 for integers, decimals and floating point numbers; and false for booleans). The default value ignored can be changed by placing the DefaultValueAttribute on the property.
    -Populate - Members with a default value but no JSON will be set to their default value when deserializing.
    -Include - Include members where the member value is the same as the member’s default value when serializing objects. Included members are written to JSON. Has no effect when deserializing.|
    |NullValueHandling - Specifies null value handling options for the JsonSerializer. (This doesn’t seem to currently have effect on Novulo Objects)|-Include - Include null values when serializing and deserializing objects.
    -Ignore - Ignore null values when serializing and deserializing objects.|
    |MaxDepth - Sets the maximum depth allowed when reading JSON. Reading past this depth will throw a JsonReaderException. A null value means there is no maximum. The default value is null. See (MaxDepth setting) for example. (This doesn’t seem to currently have effect on Novulo Objects)|- Possible values - Number or Numeric String value|
    |MissingMemberHandling - Specifies missing member handling options for the JsonSerializer. (This doesn’t seem to currently have effect on Novulo Objects)|-Error - Throw a JsonSerializationException when a missing member is encountered during deserialization.
    -Ignore - Ignore a missing member and do not attempt to deserialize it.|
    |ObjectCreationHandling - Specifies how object creation is handled by the JsonSerializer. (This doesn’t seem to currently have effect on Novulo Objects)|-Reuse - Only reuse existing objects.
    -Replace - Always create new objects.
    -Auto - Reuse existing objects, create new objects when needed.|
    |PreserveReferenceHandling - Specifies reference handling options for the JsonSerializer. Note that references cannot be preserved when a value is set via a non-default constructor such as types that implement ISerializable. (for example PreserveReferencesHandling Enumeration). (This doesn’t seem to currently have effect on Novulo Objects)|-All - Preserve references when serializing.
    -Objects - Preserve references when serializing into a JSON object structure.
    -Arrays - Preserve references when serializing into a JSON array structure.
    -None - Do not preserve references when serializing types.|
    |ReferenceLoopHandling - Specifies reference loop handling options for the JsonSerializer. (This doesn’t seem to currently have effect on Novulo Objects)|-Serialize - Serialize loop references.
    -Ignore - Ignore loop references and do not serialize.
    -Error - Throw a JsonSerializationException when a loop is encountered.|
    |TypeNameHandling - Specifies type name handling options for the JsonSerializer. (This doesn’t seem to currently have effect on Novulo Objects)|-Objects - Include the .NET type name when serializing into a JSON object structure.
    -Arrays - Include the .NET type name when serializing into a JSON array structure.
    -All - Always include the .NET type name when serializing.
    -Auto - Include the .NET type name when the type of the object being serialized is not the sa|

CSV serialization/deserialization

As of revision 111089 of the serialization plugin, it now supports converting data to CSV from Novulo objects and from CSV string to Novulo objects.

For the purpose, the plugin has introduced 2 new actions (in order to keep the backwards compatibility with previous versions of the plugin). These are

  • SerializeObjectV2

  • DeserializeObjectV2

These work in the same way for the old supported types of serialization/deserialization to JSON/XML/FORM and nothing has changed in respect to configuration in regards to these

To use the CSV serialization/deserialization, developers must use these V2 actions. Some important notes in regards to the usage

  • Since CSV data represented in Novulo datatype is List<List<Tuple<String,Mixed>>> ( type that is not directly accessible in the architect), this datatype is made the new default datatype for the input parameter (in the SerializeObjectV2 action) and output return value (in the DeserializeObjectV2 action)

  • If you want to use convert to/from CSV datatype:

SerializeObjectV2 action

  • Change output content type to CSV

  • do NOT change the Input Data Type configuration item (in the SerializeObjectV2 Action) - it is show nothing, however under the hood - it is of type List<List<Tuple<String,Mixed>>> - the correct type for CSV data.

DeserializeObjectV2 action

  • Change input content type to CSV

  • do NOT change the Output Data Type configuration item (in the DeserializeObjectV2 Action) - it is show nothing, however under the hood - it is of type List<List<Tuple<String,Mixed>>> - the correct type for CSV data.

Both actions have the “settings” parameter - List<Tuple<string,Mixed>> that represent some options which will be used to interpret the CSV data when converting to Novulo objects, or when converting Novulo objects to CSV strings. The mandatory ones always have defaults (same used as in the Export Engine, when exporting to CSV).

In a similar manner to the JSON serializer settings, the key of the tuple is the name of the option, the value of the tuple is the value that will be used for the CSV serializer setting

Among the options and possible values:

encoding UTF7, UTF8, UTF16, UTF32 (Default is UTF8)
culture supported cultures by the framework - for example: en-US, en-GB, nl-NL etc. (Default is en-US
includeheaders bool (true/false) (default is true
delimiter The delimiter that will be used/is used in the CSV (default is ")
escapedelimiter the escape delimiter (default is “”)
separator the separator for in between the csv values (default is ,)
2

Hi Joachim,

Is this also usable for deserializing an object with nested objects in them?

For instance something like this:

image

3

Hi,

Yes it is supported. You can achieve it by configuring a Deserialize action, in your case it should look something like:
Plugin config:
image
Then the input JSON string for example:
image
The result (from trace):

image
image922×38 2.45 KB

You get a List of tuple<string,mixed>. You can access the object “crewMember” by key and for value you would get another list tuple<string,mixed>

Greetings,
Miroslav

1 Like
4

Hi Miroslav,

Thank you for your answer. I will try it :slight_smile: