# Data Collections and expressions

**URL:** <https://community.novulo.com/t/data-collections-and-expressions/201>\
**Category:** \[Q&A\] Components, Datasets and Plug-ins\
**Created:** [May 28, 2024, 12:58pm UTC](https://community.novulo.com/t/data-collections-and-expressions/201 "2024-05-28T12:58:20Z")\
**Posts on this page:** 1\
**Showing post:** 2

<div class="post-metadata">

**Author:** ![Joost](https://yyz2.discourse-cdn.com/flex008/user_avatar/community.novulo.com/joost/32/87_2.png) [@Joost](https://community.novulo.com/u/Joost)\
**Post date:** [May 28, 2024, 9:03pm UTC](https://community.novulo.com/t/data-collections-and-expressions/201/2 "2024-05-28T21:03:18Z")

</div>

Hi JesseFizor,  
You are asking about how Data Collections work and how to use it to consolidate data.

Data collections, implemented by component **M5159** , are sets of calculated, consolidated data. It is useful to make periodic snapshots of data, which can then be used to display calculation-extensive data in tables, charts or exports.

Data collections can also be integrated with Application Logging using **M7606**.

# Configuring data collections

When installed, Data Collections can be found in the Management subsystem.

Here, you can define a Data collection. It has various options, this example is about the use case you asked.

 ![image](https://canada1.discourse-cdn.com/flex008/uploads/novulo/original/1X/2858d29c6e59fb87a0c0351d03be8d45033cafe1.png)

The choses settings here are:

1. The type is “Interval” and allows you to select an interval. With this interval, you an create calendar date-based data collections with a specified interval:

2. With Cardinality (single or multiple), you identify if you want one single measurement per Data collection, or multiple. As you want data per day, per product, you choose multiple.

3. Then, you select the record type you want to check per day, and the filter expression to define the products. In this example, I filtered on the products that start with code ‘11’, to make the list smaller.

4. At value expression, you define the calculation for the value. The built-in fields ‘interval\_from’ and ‘interval\_to\_exclusive()’ are date fields that you can use for filtering.

# Running and reviewing data collections

Below the configuration, you can see the data collections. It starts generating from the ‘start date’ and from there, up until today’s date. Be careful, especially initially, not to set a start date long in the past to prevent creating large amounts of data.

 ![image](https://canada1.discourse-cdn.com/flex008/uploads/novulo/original/1X/8f846319eca638b54e5516d5506a6eae7b7dbc98.png)

When you click ‘generate’, it generates records for all Date intervals between the Start date and today.  
When you click ‘generate and process’, it also calculates all values.

When setting up and testing, we recommend to first Generate, and then Process one first to validate data. The process button is on the Data collection page.

 ![image](https://canada1.discourse-cdn.com/flex008/uploads/novulo/original/1X/5af4428bb3151bb93d155ab7a853d1752b78f714.png)

# Scheduling data collections

Now, when your dataset is Created and you want to generate Data automatically, you can use the Process `generate_sets_for_collectioncontrollercomponent_xxxxx` in the [Scheduler](https://community.novulo.com/t/how-to-add-a-new-scheduled-task-geplande-taak/196), to run the collections in the interval you need.

Typically, you use low use hours of the application, like after business hours, to calculate data.

---

_[View the full topic](https://community.novulo.com/t/data-collections-and-expressions/201)._