Introduction
The QA-scanner is an application that automatically performs several checks on a particular component-revision or application-revision. Those checks are part of a larger system of testing. A successful scan of the QA-scanner assigns a beta-status to a revision. Once the same revision has been manually tested, its status will be upgraded to stable.
The QA-scanner is available at: https://ndsp.novulo.com/QA_scanner/
You can login using your architect credentials.
Which checks are performed by the QA-scanner?
The QA-scanner checks the following information:
-
PI-information
-
Translations
-
Composability of the component-revision
-
Composability of each production application the component-revision is in
-
Substitutions
-
Architect-information
An application-revision should be considered as a set of component-revisions, so when you scan an application-revision which contains 47 component-revisions, the QA-scanner will actually scan every individual of those 47 component-revisions. The only difference is that the results will show an additional summary for each application-revision.
PI-information
Each PI that is linked to the component’s last approved revision and the one you’d like to approve, should meet the following criteria:
-
its title cannot contain an SR-number or client name (unless the client name is part of the component’s name)
-
either the PI itself has release notes or another PI with the same main PI has release notes
-
the PI has not been manually disapproved, or have QA remark.
In addition, the following must hold as well:
- every revision between the last approved revision (last beta) and the revision you’d like to approve must be linked to a PI
Translations
In order to successfully pass the check in the QA-scanner a component-revision has to retain the translations that the last approved revision (last beta) has according to the AppServer. The component-revision you’d like to approve can contain additional translations but cannot contain fewer translations.
Composability
Composability of the component-revision
In order to pass the component composability there are two scenarios:
-
the component-revision is composable with other components of beta or higher
-
the component-revision is composable with other components of which one or more have status CTP (not lower), the component-revision will only be set to beta once these other components have been set to beta
In case of scenario 1, the composability with other CTP-components will not be checked.
Composability of each production application the component-revision is in
In order to pass the production application composability there are three scenarios:
-
the production application where the current revision of the component to be scanned has been replaced by the revision you’d like to approve, is composable without the need for any additional components
-
the production application where the current revision of the component to be scanned has been replaced by the revision you’d like to approve, is composable with other components of beta or higher (baseline, stable)
-
the production application where the current revision of the component to be scanned has been replaced by the revision you’d like to approve, is composable with other components of which one or more have status CTP (not lower), the component-revision will only be set to beta once the other components have been set to beta and the scan is run again
In case of scenario 1 a call to the composer-algorithm is not required since all the consumes are already being produced by the other component-revisions in the application. In case of scenario 2, the composability with other CTP-components will not be checked.
Dependencies
In scenario 2 of the composability of the component-revision and scenario 3 of the composability of each production application, a composition can only be found with the help of one or more components with status CTP. These components with status CTP are called dependencies. Each dependency can have the following dependency-status (not to be confused with the status of the revision):
-
untested: not checked by the QA-scanner
-
rejected: checked by the QA-scanner but outcome shows errors
-
ready: checked by the QA-scanner and outcome doesn’t show any errors but the component-revision has one or more dependencies
-
waiting: checked by the QA-scanner and outcome doesn’t show any errors nor dependencies but is still waiting to be approved by running the QA-scanner with the ‘approve to beta’ option selected
Architect-information
The component-revision cannot contain errors, warnings or styling issues in the Architect. The used version of the Architect is shown in the results. Be aware that revisions saved with a particular version of the Architect can be checked against a different version of the Architect.
Substitutions
A substitution is defined as a set of component-revisions that consume and produce exactly the same as another set of component-revisions (no overlapping component-revisions between both sets) with the addition that components without any consume or produce are excluded as are components without a produce that consume only concepts.
Substitutions offer interchangeable components and in order to retain this choice and flexibility in the choice of components, this is part of the QA-scanner.
The QA-scanner currently checks for 1-on-1, 1-on-2 and 2-on-1 substitutes. This means that a list has been constructed with all the latest beta component-revisions. From that list all single component-revisions have been selected which can technically be replaced by either one or two other component-revisions so that the consumes and produces match exactly. The results hold both ways so that a 1-on-2 substitute is automatically a 2-on-1 substitute as well.
When you scan a component-revision, either standalone or as part of an application, the QA-scanner will check whether or not the substitutions still hold when the revision of the component to be checked, has been replaced by the revision to be approved.
Access
Rights management has been implemented in the QA-scanner. You can log into the QA-scanner with your Architect-account. As a result, you can scan components you normally have write permissions for. In case you scan an application, only those components will be checked. Components and applications where you do not have access, are not shown.
How to use the application?
Homepage
Once you are logged into the application, you can either scan a single component-revision or an application-revision in the following screen:
You’ll enter the model-id (including the capital M or only a number) of either the component or application you’d like to scan. When you choose to scan an application, the revision is required as well. Once you press the scan button, the scanner will start checking the application you have submitted.
In case you choose to scan a component, the revision is optional. As opposed to scanning an application, with components you can choose to receive the results of the scan by email but this is not required since the results will also be shown and saved in the application.
Important to mention is the option to ‘approve to beta’ which is available as a checkbox below the revision of a component. This enables you to automatically approve the component-revision to beta in case it meets all the requirements. When this option is not selected, you can still scan a component-revision but even in case all the requirements are met to set it to beta the status won’t be changed.
Once you’ve hit the scan button (with components only), you’ll see the following popup:
The popup shows the latest revision of the component for the following statuses: WIP, NB and CTP. You can select any revision and hit the start-button to execute the scan. It also shows whether results of this component already exist. When that’s the case, you can easily view those results by hitting the show-button.
Status page
Once you have hit the ‘scan’-button a status page will appear, which looks like this for a component-revision (when you scan an application a different screen will appear):
This page shows the progress of the scan. When the QA-scanner is already busy and your task is waiting in the queue, the position of your task in the queue will be shown so you’ll have an idea when your task is up. Important to know is that components have priority over applications in the queue. This has been done to ensure that you don’t have to wait for a long time for a component to be scanned whereas an application won’t be delayed by much when one or more components get priority in the queue.
When your task has been processed, the results will automatically be shown. In case you have scanned a component-revision, a summary of errors for each section will be followed by more detailed information below. The results of scanning a component-revision will look like this:
When an application is being scanned an overview will appear indicating the progress of each individual component-revision that will be checked:
When you select an individual component-revision on the application’s progress page, the status page of the component-revision (as shown earlier) will appear.
Results page
For most individual component-revisions it doesn’t take too long for the QA-scanner to complete the scan and you can wait while looking at the status page. However, especially the component-revisions which are part of many production applications might take quite some time and waiting might not be an option. The same is true of course for most of the applications. For those cases it is recommend to just check in with the QA-scanner after a while and visit the results page. On this page you can view all the results of scans executed either by yourself or by every one of your organization. It’s not possible to view results from scans which have been executed by people outside your organization.
In the menu you can switch between components and applications by selecting one of the tabs. Once a selection has been made you’ll see an overview. In this overview you can filter on title and user (either just you or every one of your organization, as mentioned earlier). In addition, you can also delete results of scans but only the ones executed by you. Results of scans which have been executed by other people, regardless of organization, cannot be deleted.