Enhancing Data Quality with Rules
Introduction
In the context of Building Information Modeling (BIM), maintaining data consistency and automating workflows are essential for effective project management. The Novorender platform allows users to upload their models and leverage our REST API to manage various aspects of these models, including the powerful rules system.
By utilizing rules within Novorender, users can significantly enhance how building information models are processed and refined. The rules system provides a robust framework for automating post-processing tasks, ensuring consistency, and streamlining workflows. Here are some key use cases demonstrating the capabilities of the rules system:
-
Automated Data Cleaning: Users can create rules that automatically standardize property names across different model sources, ensuring consistent data representation throughout the project. This automation can save time and minimize errors by adjusting properties across numerous elements without manual intervention.
-
Smart Filtering and Categorization: The rules system allows for intelligent filtering and categorization of model elements. Users can implement rules that automatically classify components based on specified criteria, such as tagging doors over a certain height or categorizing spaces based on their dimensions.
-
Workflow Automation: Repetitive tasks can be automated through the rules system, reducing manual effort and the risk of human error. For example, you can set up rules that generate cost estimates based on properties or flag potential clash issues before they escalate into larger problems.
-
CSV Integration for Enhanced Data Enrichment: A standout feature of the rules system is its ability to integrate external CSV data with models uploaded to Novorender. Users can match keys from CSV files to existing properties in the model, allowing for automatic enrichment with vital data such as maintenance schedules, cost data, or performance metrics.
The rules system serves as a powerful post-processing tool, enabling users to modify properties, adjust the model structure, and refine geometry after the model has been uploaded. Rulesets, consisting of one or more rules, filters, and actions, are executed sequentially, allowing complex data manipulations based on earlier results.
In this guide, we will explore how to create, add, and modify rulesets using the Novorender REST API, empowering you to leverage these features for effective management of your BIM projects.
API Operations for Managing Rulesets
Novorender provides specific API endpoints for retrieving and modifying rulesets associated with your models. Below are the available operations for working with rulesets:
-
Get Folder Rules
Use this endpoint to retrieve the rulesets applied to a specific folder or project. -
Update Folder Rules
This endpoint allows you to update the rulesets for a folder or project.
To get or update the rulesets for the entire project (root folder), use the %20 character in place of the :folderId parameter.
The %20 character represents a space in URLs, which is used to refer to the root folder when working with rulesets at the project level.
In the next section, we'll discuss the JSON structure and payload that can be passed to the PATCH endpoint to create or modify rulesets.
The PATCH endpoint accepts the following JSON payload:
{
"version": "1.1",
"regexDocumentation": "https://cplusplus.com/reference/regex/ECMAScript/",
"ruleSets": [] // all types of rules go here...
}
In this payload:
versionspecifies the version of the ruleset structure.regexDocumentationlinks to a resource explaining the ECMAScript regex syntax that can be used within rules.ruleSetsis an array that contains different types of rulesets, which we will explore in the next section.
The structure of rulesets: [] is flexible, as we do not enforce strict schema checks on it. This allows you to add arbitrary or additional properties to rulesets, such as unique IDs for each ruleset or rule, or any other custom property that may be helpful for your specific scenario.
Available Ruleset Types in Novorender
Novorender offers several ruleset types for managing and manipulating BIM models through the OPEN API. Each ruleset type allows you to automate different tasks, from filtering model elements to modifying properties. Below are the currently available ruleset types:
1. Common
Common rulesets follow a consistent pattern of object filters, rules, and result actions. This versatile framework is designed to accommodate most rule types.
Both object filters and rules share the same structure, allowing you to define property names and values that are checked against regular expressions (regex) and/or numerical ranges (min/max). These expressions can be combined using logical operators like "and (&)" or "or (|)". For example, the following filter identifies all IfcWindows that are made of Glass:
{
"property": "IfcClass",
"regex": "IfcWindow"
},
"&",
{
"property": "Material",
"regex": "Glass"
}
Filtered objects can be checked by rules. The following rule checks if the filtered objects are placed above 5 meters:
{
"property": "Location/Global Z",
"min": 5
}
Objects that pass both the filter and rules are processed by optional true and false actions, which may differ based on the rule result. The available action types include:
-
Merge
Combine an object with its properties into its parent node. This is particularly useful for.rvmfiles, which often result in deeply nested hierarchies where most properties reside higher up in the model tree.{
...
"result": {
"false": {
"action": "merge",
"copyProperties": true
}
}
...
}Example JSON payload
{
"version": "1.1",
"regexDocumentation": "https://cplusplus.com/reference/regex/ECMAScript/",
"ruleSets": [
{
"type": "common",
"outputPropertySet": "Ruleset",
"rules": [
{
"comment": "Rule 1 comment text",
"objectFilter": [
{
"objectFilterSet": [
{
"property": "IfcClass",
"regex": "IfcWindow"
},
"&",
{
"property": "Material",
"regex": "Glass"
}
]
}
],
"rule": [
{
"property": "Location/Global Z",
"regex": "",
"min": 5
}
],
"result": {
"outputProperty": "test",
"false": {
"action": "merge",
"copyProperties": true
}
}
}
]
}
]
}tipYou can use the
copyProperties: booleanoption to control whether properties are retained or discarded during the merge. Setting this tofalsecan help streamline your data by eliminating unnecessary properties that may otherwise clutter valuable information in the final output. -
Replace
Modify selected object parts based on regex matches.
Example use case: Automatically update incorrect SharePoint links by changing the root part of the URL.{
...
"result": {
"outputProperty": "IfcClass",
"true": {
"action": "replace",
"value": "Special",
"match": "Standard"
}
}
...
}Example JSON payload
{
"version": "1.1",
"regexDocumentation": "https://cplusplus.com/reference/regex/ECMAScript/",
"ruleSets": [
{
"type": "common",
"outputPropertySet": "Ruleset",
"rules": [
{
"comment": "Rule 1 comment text",
"objectFilter": [
{
"objectFilterSet": [
{
"property": "IfcClass",
"regex": "IfcWindow"
},
"&",
{
"property": "Material",
"regex": "Glass"
}
]
}
],
"rule": [
{
"property": "Location/Global Z",
"regex": "",
"min": 5
}
],
"result": {
"true": {
"action": "replace",
"value": "Standard",
"match": "Special"
}
}
}
]
}
]
} -
Generate GUID
Generate a globally unique identifier (GUID) for objects based on properties matched in the rule.{
...
"result": {
"true": {
"action": "generateGuid"
}
}
...
}Example JSON payload
{
"version": "1.1",
"regexDocumentation": "https://cplusplus.com/reference/regex/ECMAScript/",
"ruleSets": [
{
"type": "common",
"outputPropertySet": "Ruleset",
"rules": [
{
"comment": "Rule 1 comment text",
"objectFilter": [
{
"objectFilterSet": [
{
"property": "IfcClass",
"regex": "IfcWindow"
},
"&",
{
"property": "Material",
"regex": "Glass"
}
]
}
],
"rule": [
{
"property": "Location/Global Z",
"regex": "",
"min": 5
}
],
"result": {
"true": {
"action": "generateGuid"
}
}
}
]
}
]
}infoGenerating a GUID on
result.falsedoes not work, as it requires values from the match to generate the GUID. -
Regenerate BREP
Force the system to regenerate boundary representation (BREP) data. This is useful when the system can generate more accurate parametric data than the input source, such as tessellated IFC/DWG objects.{
...
"result": {
"true": {
"action": "regenBrep"
}
}
...
}Example JSON payload
{
"version": "1.1",
"regexDocumentation": "https://cplusplus.com/reference/regex/ECMAScript/",
"ruleSets": [
{
"type": "common",
"outputPropertySet": "Ruleset",
"rules": [
{
"comment": "Rule 1 comment text",
"objectFilter": [
{
"objectFilterSet": [
{
"property": "IfcClass",
"regex": "IfcWindow"
},
"&",
{
"property": "Material",
"regex": "Glass"
}
]
}
],
"rule": [
{
"property": "Location/Global Z",
"regex": "",
"min": 5
}
],
"result": {
"true": {
"action": "regenBrep"
}
}
}
]
}
]
} -
Generate Property
Create a custom property based on the rule result. You can specify both the property name and value.{
...
"result": {
"outputProperty": "Test",
"true": {
"action": "property",
"value": "Over 5 meters"
},
"false": {
"action": "property",
"value": "Below 5 meters"
}
}
...
}Example JSON payload
{
"version": "1.1",
"regexDocumentation": "https://cplusplus.com/reference/regex/ECMAScript/",
"ruleSets": [
{
"type": "common",
"outputPropertySet": "Ruleset",
"rules": [
{
"comment": "Rule 1 comment text",
"objectFilter": [
{
"objectFilterSet": [
{
"property": "IfcClass",
"regex": "IfcWindow"
},
"&",
{
"property": "Material",
"regex": "Glass"
}
]
}
],
"rule": [
{
"property": "Location/Global Z",
"regex": "",
"min": 5
}
],
"result": {
"outputProperty": "test",
"true": {
"action": "property",
"value": "Over 5 meters"
},
"false": {
"action": "property",
"value": "Below 5 meters"
}
}
}
]
}
]
} -
Generate properties based on other properties
This rule type allows you to dynamically create or edit properties by referencing other existing properties in Novorender. This is particularly useful for linking to external data sources or adding standardized codes for processes based on model attributes, without needing external edits to the model data.-
Creating and Editing URLs: In some cases, teams may need to link to external data sources directly from their Novorender models. For example, XYZ has a significant amount of maintenance data stored in their SharePoint. However, some models may still link to the client’s SharePoint instead. With property-based rules, they can correct these links to point to their own SharePoint URLs, streamlining access to maintenance data.
Use Cases:
- Case 1: Generate a SharePoint URL based on an object’s unique identifier.
- URL Format:
xyz365.sharepoint.com/... + object_id - property
- URL Format:
- Case 2: Create a more customized SharePoint URL by adding discipline, area, or other specific attributes to the link.
- URL Format:
xyz365.sharepoint.com/... + object_id - property + discipline - property + area - property
- URL Format:
- Case 1: Generate a SharePoint URL based on an object’s unique identifier.
-
Adding Process Coding for Standardization: Infrastructure projects often involve specialized codes for work processes (e.g., excavation, rock blasting, trenching, road layering). Senior team members—who may not use BIM software regularly—need access to these codes for work planning and oversight. By adding rules to enrich data based on other properties, process coding can be standardized and easily applied across projects, saving designers considerable time.
Example Setup:
For an infrastructure project, a rule can be created such that if Property A equals “material_type” and “layer_thickness,” then Property B automatically generates a process code, streamlining setup and increasing consistency across multiple projects.infoThis rule uses the same
action: "property"as theGenerate Propertyrule discussed above. However, instead of specifying a fixedvalue, it allows for a dynamic combination of prefixes, postfixes, and captured values, separated by customizable delimiters. Note that this option is only useful if at least one rule usescapture.{
...
"rule": [
{
"property": "IfcClass",
"regex": "Wall",
"capture": true // To enable `useCapture` in the `result`, some or all rules must be configured with `capture: true`
}
],
"result": {
"outputProperty": "Rule1",
"true": {
"action": "property",
"useCapture": {
"prefix": "prefix_",
"postfix": "_postfix",
"separator": "%2"
}
}
}
...
}Example JSON payload
{
"version": "1.1",
"regexDocumentation": "https://cplusplus.com/reference/regex/ECMAScript/",
"ruleSets": [
{
"type": "common",
"outputPropertySet": "Parametric data",
"rules": [
{
"comment": "Parametric data for construction elements",
"objectFilter": [],
"rule": [
{
"property": "IfcClass",
"regex": "Wall",
"capture": true
}
],
"result": {
"outputProperty": "Rule1",
"true": {
"action": "property",
"useCapture": {
"prefix": "prefix_",
"postfix": "_postfix",
"separator": "%2"
}
}
}
}
]
}
]
} -
Both result.true and result.false are optional, so utilize them whenever necessary.
2. Delete Objects
You can delete objects from the imported data before they are processed in the Novorender scene. Although filters in Novorender are often sufficient, deleting objects can be especially useful when the imported files contain large quantities of objects that will never be used. Removing unnecessary objects not only reduces loading and processing times but also enhances overall performance and visual detail when viewing the scene.
{
"outputPropertySet": "Ruleset",
"rules": [
{
"type": "deleteObjects",
"rules": [
{
"property": "IfcClass",
"regex": "IfcDoor"
}
]
}
]
}
Example JSON payload
{
"version": "1.1",
"regexDocumentation": "https://cplusplus.com/reference/regex/ECMAScript/",
"ruleSets": [
{
"comment": "Rule 1 comment text",
"type": "deleteObjects",
"rules": [
{
"property": "IfcClass",
"regex": "IfcDoor"
}
]
}
]
}
Each deleteObjects rule should be a separate entry within the ruleSets array. For example, if you have two rules, you will need to push two separate objects:
{
"version": "1.1",
"regexDocumentation": "https://cplusplus.com/reference/regex/ECMAScript/",
"ruleSets": [
{
"comment": "Rule 1 comment text",
"type": "deleteObjects",
"rules": [
{
"property": "IfcClass",
"regex": "IfcDoor"
}
]
},
{
"comment": "Rule 2 comment text",
"type": "deleteObjects",
"rules": [
{
"property": "IfcClass",
"regex": "IfcBuilding"
}
]
}
]
}
3. Double-sided Materials
This rule makes all materials double-sided. While it's generally better to work with accurate data than to disable face culling, this rule is available when necessary.
{
"outputPropertySet": "Ruleset",
"rules": [
{
"type": "doubleSidedMaterial"
}
]
}
Example JSON payload
{
"version": "1.1",
"regexDocumentation": "https://cplusplus.com/reference/regex/ECMAScript/",
"ruleSets": [
{
"comment": "Rule 1 comment text",
"type": "doubleSidedMaterial"
}
]
}
Each doubleSidedMaterial rule should be a separate entry within the ruleSets array. For example, if you have two rules, you will need to push two separate objects:
{
"version": "1.1",
"regexDocumentation": "https://cplusplus.com/reference/regex/ECMAScript/",
"ruleSets": [
{
"comment": "Rule 1 comment text",
"type": "doubleSidedMaterial"
},
{
"comment": "Rule 2 comment text",
"type": "doubleSidedMaterial"
}
]
}
4. Generate CSV Values
The rules system supports importing data from CSV files. Developers can match a key from the CSV file to an existing Novorender property, and the rule will generate the remaining CSV properties for the corresponding object.
{
"outputPropertySet": "CSV",
"rules": [
{
"rule": [
{
"csv": "Test-csv.csv",
"key": "Email", // CSV key
"property": "Test" // Novorender property name
}
]
}
]
}
The optional payload property (property) is used to specify which property in Novorender should be linked to a key (key) from the CSV file. If left empty or undefined, it is assumed that the Novorender property name matches the CSV property name (key).
Example JSON payload
{
"version": "1.1",
"regexDocumentation": "https://cplusplus.com/reference/regex/ECMAScript/",
"ruleSets": [
{
"type": "csvProperties",
"outputPropertySet": "CSV",
"rules": [
{
"rule": [
{
"property": "Email",
"csv": "Test-csv.csv"
}
]
}
]
}
]
}
Ensure that the file referenced in the csv property is already uploaded to the folder in which you want to define the rulesets.
5. Search and Replace
The search and replace rule scans property values within the model and replaces them based on user-defined search and replace strings. This is useful for bulk property updates where only two strings are needed: the search string and the replace string.
{
"outputPropertySet": "Ruleset",
"type": "searchAndReplace",
"rules": [
{
"search": "StandardWall",
"replace": "Wall"
}
]
}
Example JSON payload
{
"version": "1.1",
"regexDocumentation": "https://cplusplus.com/reference/regex/ECMAScript/",
"ruleSets": [
{
"type": "searchAndReplace",
"outputPropertySet": "Ruleset",
"rules": [
{
"comment": "Rule 1 comment text",
"search": "StandardWall",
"replace": "Wall"
}
]
}
]
}