Version: v5.0

Custom import

Custom model file import#

You can copy and modify the existing BimpkImport import class in the importModel script for your specific model package file. The following example of a Scene Graph import is a direct copy of the existing BimpkImport import class and is labelled SgpkImport. The modifications are described throughout this case study.

Navisworks export case study#

From platform version 5.0, there is an IPA plugin available for the Navisworks CAD program to export models in a package for platform consumption. Navisworks can read BIM models of various CAD formats, such as Revit, DWG, DWF, IFC, and DGN and it uses Scene Graph to store model data.

Scene Graph#

Scene Graph is node tree, with each node representing an aspect of the model, for example, for a Revit source model, it could represent a Level, Category, Family, Type, Element or Geometry part. Each CAD format has its own node structure so the logic you use to parse the Scene Graph data must accomodate this.

SGPK and BIMPK#

SGPK, the packaged model exported from the IPA plugin, reuses most of the BIMPK spec. Processing graphics and hoops node mapping remains the same, only the model element data extraction is different.

The primary change of concern is that the objects.json file is replaced with a scenegraph.json file. This means that for each occurence, such as each federated model file, there is a corresponding scenegraph.json file.

SGPK folder structure

manifest.json
thumbnail.png
occurrence
- occurrence id
  - scenegraph.json
  - graphics.scz
  - hoops_node_mapping.json

Relevant import script adjustments#

Change file path endpoint from occ.data.objects to occ.data.scenegraph
Change model file reader method from ModelFileReader.getModelBatchlet to ModelFileReader.getSgpkBatchlet

  let zipModelFile = await ModelFileReader.downloadAndUnzipModelFile(param, ctx);  const { bimFilePath, manifest } = zipModelFile;  let { files, occurrences } = manifest;    for (const model of files) {    for (const occ of occs) {      const filePath = bimFilePath + "/" + occ.data.scenegraph; // was occ.data.objects      try {        const { bimBatch, endOfFile } =          await ModelFileReader.getSgpkBatchlet( // was ModelFileReader.getModelBatchlet            filePath,            this.params.orchRunId          );        ...      }    }  }

Revit example#

This case study demonstrates how to extract Revit CAD model data stored in a scenegraph.json file.

Revit Scene Graph structure#

Revit's model data translates to the following hierarchy in the Scene Graph node-tree structure. An item's child items are nested within that item's Items property. For example, the models layer objects are in the root item's Items property, each layer object contains an array of category objects in its Items property and so on.

Revit Scene Graph nesting hierarchy

The following heirarchy exists within the parent object's RootItem property:

File
- Levels
  - Categories
    - Families
      - Types
        Elements
        Geometry

The following example drills down through the node tree, showing the object at index 0 for each nested array:

//parent object- "SourceFileName": "D:\\me\\test\\NW\\Revit\\House.rvt"- "Creator": "LcNwcLoaderPlugin:lcldrevit"- "Units": "Feet"- "Properties": []- "RootItem": //hierarchy starts here  - "Id": 1,  - "ClassName": "LcOaPartition"  - "ClassDisplayName": "File"  - "DisplayName": "House.nwd"  - "HasGeometry": "false"  - "Properties": []  - "Items": //layers start here    - [0]:      - "Id": 2,      - "ClassName": "LcRevitLayer"      - "ClassDisplayName": "Levels: Level: 1/4\" Head"      - "DisplayName": "First Floor"      - "HasGeometry": "false"      - "Properties": []      - "Items": //categories for layers[0] start here        - [0]:          - "Id": 3,          - "ClassName": "LcRevitCollection"          - "ClassDisplayName": "Category"          - "DisplayName": "Ceilings"          - "HasGeometry": "false"          - "Properties": []          - "Items": //families for layer[0].categories[0] start here            - [0]:              - "Id": 4,              - "ClassName": "LcRevitCollection"              - "ClassDisplayName": "Family"              - "DisplayName": "Compound Ceiling"              - "HasGeometry": "false"              - "Properties": []              - "Items": //types for layer[0].categories[0].family[0] start here                - [0]:                  - "Id": 5,                  - "ClassName": "LcRevitCollection"                  - "ClassDisplayName": "Type"                  - "DisplayName": "GWB on Furring"                  - "HasGeometry": "false"                  - "Properties": []                  - "Items": //elements for layer[0].categories[0].family[0].type[0] start here                    - [0]:                      - "Id": 6,                      - "ClassName": "LcRevitCollection"                      - "ClassDisplayName": "Ceilings: Compound Ceiling: GWB on Furring"                      - "DisplayName": "Compound Ceiling"                      - "HasGeometry": "false"                      - "Properties":                         - ["n"]                          - "Id": "",                          - "Name": "GUID",                          - "Value": "21f43d43-bf49-4a63-9c03-1400676a529e"                        - [1] {...}                        - [2] {...}                        //more properties                      - "Items": //geometry for element here                        - [0]:                          - "Id": 7,                          - "ClassName": "LcRevitSolid"                          - "ClassDisplayName": "Solid"                          - "DisplayName": "Metal - Stud Layer"                          - "HasGeometry": "true"                          - "Properties": []                        - [0]:                          - "Id": 8,                          - "ClassName": "LcRevitSolid"                          - "ClassDisplayName": "Solid"                          - "DisplayName": "Gypsum Wall Board"                          - "HasGeometry": "true"                          - "Properties": []                    - [1] {...}                    - [2] {...}                    //more elements                - [1] {...}                - [2] {...}                //more types            - [1] {...}            - [2] {...}            //more families        - [1] {...}        - [2] {...}        //more categories    - [1] {...}    - [2] {...}    //more layers

Constructing the RelatedItems#

To construct the RelatedItems for the elements, element props, and element types collections, the most straightforward way is to start from the end goal, ie, the RelatedItem properties that already exist for the Bimpk extraction, which the app and it's scripts expect.

The following examples show the expected object keys and how to populate that data from the Revit Scene Graph node tree:

Element Type RelatedItem object#

{  "name": "",  "_id": "",  "id": "",  "source_id": "",  "properties": {    "RevitType": {      "val": "",      "name": "",      "dname": ""    },    "Family": {      "val": "",      "name": "",      "dname": ""    },    "Category": {      "val": "",      "name": "",      "dname": ""    }  } }

Property	How to populate
`name`	Concatenate the `ClassDisplayName` values for the category, family, and type with a semicolon: `<category CDN>:<family CDN>:<type CDN>`. For example `Ceilings: Compound Ceiling: GWB on Furring`
`_id`	A generated mongodb id
`id`	The Type object's `Id` value, for example `5`.
`source_id`	Not applicable. Enter `null`
`properties.RevitType.val`	The Type's `DisplayName` value, for example `GWB on Furring`.
`properties.RevitType.name`	Enter the string `"REVIT_TYPE"`.
`properties.RevitType.dname`	Enter the string `"Revit Type"`.
`properties.Family.val`	The Family `DisplayName` value, for example `Compound Ceiling`.
`properties.Family.name`	Enter the string `"REVIT_FAMILY"`.
`properties.Family.dname`	Enter the string `"Revit Family"`.
`properties.Category.val`	The Category `DisplayName` value, for example `Ceilings`.
`properties.Category.name`	Enter the string `"REVIT_CATEGORY"`.
`properties.Category.dname`	Enter the string `"Revit Category"`.

Element RelatedItem object#

{  "type_id": "",  "source_filename": "",  "_id": "",  "package_id": "",  "source_id": ""}

Property	How to populate
`type_id`	The `id` value of the constructed Type that relates to this element.
`source_filename`	The `SourceFileName` property from the parent object.
`_id`	A generated mongodb id
`package_id`	The Element object's `Id` value, for example `6`.
`source_id`	In the Element's `Properties` array, find the object where its `Name` value is `"GUID"` and extract the `"Value"` property's value. In the example, it is `"21f43d43-bf49-4a63-9c03-1400676a529e"`.

Element props RelatedItem object#

{  "_id": "",  "properties": ""}

Property	How to populate
`_id`	The `_id` value of the constructed element that the properties relates to.
`properties`	The transformed objects in the related element's `Properties` array.

Relevant import script adjustments#

extractSgpk#

From the input bimBatch property, the function processes the layers first. extractLayersTypes extracts the types and extractTypesPropsAndElems extracts the layer elements and the properties of those elements:


  async #extractSgpk(bimBatch, modelId, IafScriptEngine) {    try {      const bimObj = bimBatch[0];      const rootItem = bimObj.RootItem;      const sourceFileName = bimObj.SourceFileName;      const layers = rootItem.Items;      let layerTypes = await this.#extractLayersTypes(layers, IafScriptEngine);
      // extract layer props and elems       const { layersProps, layersElems } = await this.#extractLayersPropsAndElems(layers, layerTypes, sourceFileName, IafScriptEngine);

After the layers are processed for types, elements, and element props, the same is done for the rest of the model:

      // extract other elems and props      const { typeObjects, props, elems } = await this.#extractTypesPropsAndElems(layers, sourceFileName, IafScriptEngine);      console.log({        typeObjects: typeObjects,        props: props,        elems: elems      });

For more information on how the types, elements, and element props are extracte, see extractTypesPropsAndElems.

The objects constructed from layers and the rest of the model are merged and then set as Script Engine variables. These are then mapped using the mapItemsAsRelated function:

      const allTypes = [ ...layerTypes, ...typeObjects ];      const allProps = [ ...layersProps, ...props ];      const allElems = [ ...layersElems, ...elems ];
      const setVars = [        IafScriptEngine.setVar(`properties_${modelId}`, allProps),        IafScriptEngine.setVar(`manage_els_${modelId}`, allElems),        IafScriptEngine.setVar(`manage_type_els_${modelId}`, allTypes)      ];      console.log(`SgpkImport extractSgpk types len ${typeObjects.length}  types: `, allTypes)            await Promise.all(setVars);      const relations = await this.#mapItemsAsRelated(allElems, allTypes, "type_id", "id")      console.log('mapItemsAsRelated relations: ', relations)
      await IafScriptEngine.setVar(        `manage_el_to_type_relations_${modelId}`,        relations      );
      return allRes;    } catch (err) {      console.log(        `RefApp Error at extractBimpk, orch_run_id: ${this.params.orchRunId}`      );      console.error(err);    }  }

extractTypesPropsAndElems#

As the data structure is different from the Bimpk data, so too is the method of extraction and how the script loops through the data. As the script loops through the node structure, it has enough information to create the types first, then the elements and their properties.

First, the function creates arrays to populate the types, elements, and properties. Use the ignoredNames array to list the names of object you want to ignore, such as lines:


  async #extractTypesPropsAndElems(layers, sourceFileName, IafScriptEngine){    let typeObjects = [];    let props = [];    let elems = [];    const ignoredNames = ['<Room Separation>', 'Center line', 'Center Line', 'Lines'];

Overall, the function loops through each level of the node-tree hierarchy through the Items parameters of each object:

  for (let i = 0; i < layers.length; i++) {    const categories = layers[i].Items;    ...    for (let j = 0; j < categories.length; j++) {      const families = categories[j].Items;      ...      for (let k = 0; k < families.length; k++) {        const types = families[k].Items;        ...        for (let l = 0; l < types.length; l++) {          const instances = types[l].Items;          ...          for (let m = 0; m < instances.length; m++) {            ...          }        }      }    }                }

Before we process anything the function handles edge cases, such as the ignored categories or Rooms categories:

    for (let i = 0; i < layers.length; i++) {      try {        if (layers[i].Items) {          const categories = layers[i].Items;          for (let j = 0; j < categories.length; j++) {            try {              //makes sure that the category is not in the ignoredNames array              if (!ignoredNames.includes(categories[j].ClassDisplayName)) {                //Rooms category is handled differently so the types, elements, and props are extracted with a different function                if (categories[j].DisplayName == 'Rooms') {                  const {resTypeObject, resProps, resElems} = await this.#extractRoomTypePropsAndElems(categories[j], sourceFileName, IafScriptEngine);                  typeObjects.push(resTypeObject);                  elems.push(...resElems);                  props.push(...resProps);                } else {
                }

Once the script reaches the types, the type object can be constructed and pushed to the types array:


  const families = categories[j].Items;  for (let k = 0; k < families.length; k++) {    if (families[k].Items) {      const types = families[k].Items;      console.log('types', types);      for (let l = 0; l < types.length; l++) {        const name = [categories[j].DisplayName, families[k].DisplayName, types[l].DisplayName].join(':');        const typeExists = typeObjects.some(obj => obj.name === name);        if (!typeExists) {          const typeObject = {            name: name.replace(/\s+/g, ''),            _id: await IafScriptEngine.newID("mongo", { format: "hex" }),            id: types[l].Id,            source_id: null,            properties: {              'Revit Type': {                val: types[l].DisplayName,                name: 'REVIT_TYPE',                dname: 'Revit Type',              },              'Revit Family': {                val: families[k].DisplayName,                name: 'REVIT_FAMILY',                dname: 'Revit Family',              },              'Revit Category': {                val: categories[j].DisplayName,                name: 'REVIT_CATEGORY',                dname: 'Revit Category',              },            }          }            typeObjects.push(typeObject);                              }

The element instances are the next level down in the node-tree. To construct the element object, first the function finds the constructed type with the same ClassDisplayName, then populates the element object properties as described in Constructing the RelatedItems.

  if (types[l].Items) {    const instances = types[l].Items;    for (let m = 0; m < instances.length; m++) {      const relatedType = typeObjects.find(t => t.name == instances[m].ClassDisplayName.replace(/\s+/g, ''));      const elemProps = instances[m].Properties; // relevant for prop object creation later      const guidProps = elemProps.filter(p => p.Name == 'GUID');      const idRegex = /^[a-zA-Z0-9]+(-[a-zA-Z0-9]+)*$/; //GUID regex      const guidProp = guidProps.filter(p => typeof p.Value === 'string' && idRegex.test(p.Value));      const elem = {        type_id: relatedType.id,        source_filename: sourceFileName,         _id: await IafScriptEngine.newID("mongo", { format: "hex" }),        package_id: instances[m].Id,        source_id: guidProp.Value,      };      elems.push(elem);

In the same loop, the element instance's properties are created as an element props object:

  const elemProps = instances[m].Properties;
  const prop = {    _id: elem._id, //the created element's mongodb id    properties: await this.#transformPropsArrayToObject(elemProps)  };
  // transformPropsArrayToObject function  async #transformPropsArrayToObject(array) {    let transformedObject;    if (!Array.isArray(array)) {       throw new Error('Expected array, but got:', typeof array);    }    try {      transformedObject = array.reduce((acc, item) => {        const rawName = item.Name;        const formattedName = this.#formatPropName(rawName);        if (item && item.Value !== undefined) {           acc[formattedName] = {            val: item.Value,            name: rawName,            dName: formattedName,            id: item.Id          };        }        return acc;      }, {}); 
    } catch (e) {      console.log('transformed object err:', e);    }    console.log('Final transformed object:', transformedObject);    return transformedObject;  }
  // formatPropName function  #formatPropName(name) {    //checks if name is already human-readable    if (/^[A-Z][a-z]+(?: [A-Z][a-z]+)*$/.test(name)) {        return name;     }    // Replaces dots with spaces    let formatted = name.replace(/\./g, ' ');    // Adds spaces before uppercase letters    formatted = formatted.replace(/([a-z])([A-Z])/g, '$1 $2');    // Trims spaces    const propName = formatted.trim().replace(/\s+/g, ' ');    return propName;  }