Novulo Export: how to export data

[Joost]

The Novulo platform has an advanced Export engine that allows for exporting all data from the application.

Components for export

To use the export, you need to install M3363 in your application.
Additional functionality can be unlocked with M5836 to structure data export to specific targets.

Furthermore, the export engine is used for document generation and used together with the Import engine for Datasets.

Export: functionalities

The Novulo export engine is able to export data into various different formats:

1. CSV
2. XLS (Excel 97-2003)
3. XLSX (from Excel 2007)
4. XML
5. JSON

When exporting to XLS or XLSX, it is possible to export to multiple tabs in Excel and to use template XLS/XLSX files to export to.

When exporting to XML or JSON, you can nest various data sources for export. For XML export, XSLT-support is embedded to offer you full flexibility about the formatting of the exported file.

The technology doesn’t impose a limit on the number of rows you can export; in the configuration of your export definition you can set a number for that specific export.

Export engine basics

In its core, the export engine has the following set-up

Export definition

The Export definition describes the export and stores:

1. Name
2. Export type (CSV, XML, XLS, XLSX, JSON)
3. Data format
4. Culture
5. Encoding type
6. Use of parameters
7. Use of template (XLSX only)
8. Using embedded import definition (with import engine, XML and XLSX)
9. Zip output
10. Remarks, for your own convenience

Export dataset

The export datasets defines the record types that are being exported. Depending on the data format of the export definition, you can add one or more datasets to an export definition.

• CSV: supports one dataset only
• XLS, XLSX: each data set is a tab in Excel
• XML, JSON: datasets can be nested

At the dataset level, you will define how to export the data. This leads to queries in the underlying database:

1. Data type: the name of the record type (table) that is being exported
2. Sequence number: determines sequence of sheets in Excel or nodes in XML, when an export definition has multiple datasets
3. Proofing record: the proofing record doesn’t alter your export, but can be useful to test expressions on your dataset fields for one specific record
4. Filter data: defines the expression to filter the data
5. Sort expression: defines the sorting of data
6. All fields: defines if always all fields must be exported, or only selected (typically: No)
7. Autogrow: allows for checking if the table has been changed in a newer revision of the application, relevant for datasets
8. Table identifier: This optional field can be used to specify the name of the tabs in Excel
9. Remarks: store remarks for your own convenience

At this screen, you can run the export by clicking the Export button.

Fix XML nodes for rows and tables

When exporting to XML, there will be XML nodes for the datasets (tables) and the rows like in the example below.

The names of the XML nodes for the dataset ands tables can change based on the users login language. Typically, you want to fix the XML structure which can be done with the two Override XML fields. Setting data like this:

Gives you this XML export (in a limited export with 1 row, 2 fields).

<NovuloExport createdon="2024-11-19T21:37:01.0336250+01:00">
 <ProductsSetXML datatype="products">
  <ProductsRowXML>
   <Code fieldname="code">1001</Code>
   <Name fieldname="name">Product 1</Name>
 </ProductsRowXML>
 </ProductsSetXML>
</NovuloExport>

Export fields

Export fields define which fields or functions need to be exported. When exporting to Excel or CSV, these are the columns of your export. In XML or JSON, these represent the single nodes.

Performance and database locking

For more information on performance and database locking, see Isolation level, Read Uncommitted, Export Engine and other settings: prevent database waits

(this post will be expanded)