buildingSMART alliance January 2013 Challenge

buildingSMART alliance January 2013 Challenge

by Bill East, PhD, PE - Engineer Research and Development Center, U.S. Army Corps of Engineers

Executive Summary

Information exchange standards developed by buildingSMART chapters are intended to streamline the delivery of building information through the life of capital projects. buildingSMART alliance Challenge events allow software companies to demonstrate their ability to meet these standards. Challenge criteria requires that software companies produce and or consume the required information exchanges consistent with both the format and content required to meet the stated business objective. An independent quality control process is also conducted to verify software claims. Software vendors are required to demonstrate their products and provide sufficient configuration information to allow users to repeat this process on their own projects. This page reports on the progress of software vendors to produce and consume open-standard building information as demonstrated at the 10 January 2013 bSa Challenge.


Dr. East began the Jan 2013 bSa Challenge with an overview presentation that outlined the running order of each presenter and described an overall framework into which the information exchange projects can be understood. The presentation given at the meeting and associated youtube videos are provided in the links below.

Initial Presentations

Following the overview presentation, the morning of the Challenge event allowed project team leaders and developers to present new initiatives expected to support NBIMS-US or become balloted items in their own right. The list of presentations included:

  1. Building Information Model Service interface Exchange (BIMSie)
  2. Building Automation Management information exchange (BAMie)
  3. Specifiers’ Properties information exchange (SPie)
  4. Electrical System information exchange (Sparkie) and Water System information exchange (WSie)
  5. BIM Service interface exchange (BIMSie)


The buildingSMART alliance January 2013 Challenge focused on the definition of building requirements, the delivery of that information thru design, shop drawings, and construction, finally delivering construction handover information to the facility manager. Three open standard facility information exchange specifications where exercised in the January 2013 bSa Challenge. These specifications are described in the following paragraphs:


The Building Programming information exchange (BPie) is a refinement to the buildingSMART alliance's Life-Cycle information exchange (LCie) project at the architectural programming project stage. BPie is a joint project between buildingSMART Norway and buildingSMART alliance that specifies the minimum required set of objects and associated property sets typically found in room and equipment data sheets. The requirements analysis conducted under the BPie project reviewed the contents of all prior international and national projects in this domain: the Portfolio and Asset management—Performance Requirements (PAMPeR), International Alliance for Interoperability's AR-5 Project, the buildingSMART international Room Data Sheet "aquarium" project, the United States General Services Administration project (Concept Design BIM 2010—Spatial Program Validation), the buildingSMART alliance Spatial Compliance information exchange (SCie, pronounced "ski") project, and the Norwegian effort "IDM for Building Programming." The lessons learned from each of these projects were incorporated into BPie to create an open, international standard for architectural programming with the potential to be recognized as both national and international standards.

Mr. Rolf Jerving introduced the BPie project to the audience during the morning overview session. Click through to see his presentation.


The Construction-Operations Building information exchange (COBie) delivers building asset information. COBie provides a vendor-neutral interchange format that allows building asset information to flow without the multiple stages of rediscovery and rekeying required today. COBie is the information described in an international data exchange specification called the Industry Foundation Class (IFC) Facility Management Handover Model View Definition (MVD). Participants in this the January 2013 Challenge could submit either an IFC formatted file or the spreadsheet version 2.4 of COBie data against one of three COBie deliverables. The first test was for architectural design software at the early stages of the design process. The second test was with software that supports the delivery of a consolidated design model at the Construction Documents stage of design. The third test was with construction and commissioning software that imports data from Construction Documents stage COBie, updates that with construction data, and exports the Construction Handover Stage file.

Several overall presentations and demonstrates related to COBie were delivered during the meeting.  These presentations are listed below:

In addition to these presentations, release candidate 1 of the COBieLite format was presented. COBieLite is a National Information Exchange Model (NIEM) compliant XML specification that can support both the bulk exchange of COBie data and the exchange of specific transactional information using web services.


The third information exchange format included in the bSa January 2013 Challenge was the Heating, Ventilating and Air Conditioning information exchange (HVACie). This model view diagram identifies the information exchange requirements at the Construction Documents Stage of design for the components, assemblies and connections that describe the three major components of HVAC systems: hydronic fluid system, heat transfer equipment, and thermal distribution system. This project was jointly accomplished by the Engineer Research and Development Center (ERDC), the American Society of Heating, Refrigerating and Air-conditioning Engineers (ASHRE), and buildingSMART alliance.


The following is the approach for this Challenge published in early 2012. This information is reproduced as was originally published to demonstrate the precisions with which this Challenge was organized.

The January 2013 Challenge will demonstrate the specification and delivery of building asset information from the design team (including both Architectural and MEP) to the contractor, and from a contractor through commissioning, to facility management handover. These demonstrations will be accompanied with a series of quality control and quality assurance analysis that will (1) evaluate the success of commercially available software products to meet performance-based specifications for contracted information exchanges, (2) to quantify the areas of manual effort needed to fix the data provided when the software fails to meet contract requirements, and (3) to extend the current set of benchmark models and guides upon which practitioners can recreate the steps followed by both the software companies and the QC/QA team.

The ultimate objective is to objectively demonstrate ability of different commercially available software products to produce and/or consume the specified information outside the context of hands-on vendor demonstrations. The quality of the information is defined in this Challenge as the consistency of the building information provided in computable format with that found in the equivalent stage of related contract drawings. This will enable those writing contracts and those attempting to fulfill contract requirements that the results of the Challenge event may be repeated on real projects.

This Challenge will also require software vendors to limit IFC-based information exchanges to the precise set of information needed for each of the specified information exchanges: BPie, COBie, and HVACie. This may require software vendors who only provide an export for the IFC Coordination Model View to filter out information unneeded by the deliverables specified in this Challenge.

Software vendors shall utilize only software that is scheduled to be commercially available on or before January 2013. Vendors shall be required to provide detailed mapping information to ensure that software users are able to exactly reproduce the results demonstrated in the Challenge.


The following is the list of requirements for this Challenge published in early 2012. This information is reproduced as was originally published to demonstrate the precisions with which this Challenge was organized.

  1. To accommodate available time to complete the Challenge, this Challenge shall be broken into parallel tracks, the programming/design track and the construction/commissioning track. The Medical Clinic model (ZIP), part of a series of test files referred to as the Common BIM Files shall be used for both the design and construction challenges.
  2. Design software shall use the information currently available for the Medical Clinic Model to create IFC deliverables in Step Physical File Format conforming to the FM Handover MVD. Design software vendors that do not currently export the FM Handover MVD shall be required to either create a new Model View export or internally provide a filter of existing MVD exports to eliminate entities not required by the FM Handover MVD. BPie deliverables shall be a sub-specification of the FM Handover with the additionally required properties noted in the BPie specification.
  3. The COBie Responsibility Matrix (XLS) defines all required mappings from IFC to COBie spreadsheet formats. Vendors exporting both IFC-based and and COBie spreadsheet formatted information shall be required to ensure the information contained in both of these formats exactly matches the requirements specified in worksheet three of the responsibility matrix.
  4. The Medical Clinic model contains a complete set of redacted design drawings (ZIP). These drawings shall be the basis for the Challenge.
  5. Business rules specifically identified in the information exchange specifications, such as those found in the FM Handover MVD specification, shall be enforced by producing software. Such constraints, in the example of the FM Handover MVD specification, are shown, but are not limited to the list below:
    1. Design files shall classify Space, Type, Zone, and System records.
    2. Construction files shall have design file classification and shall classify Contact records.
    3. Spaces shall be grouped by Zone.
    4. Components shall be grouped by System.
    5. All Components shall be placed within the space they occupy or the space from which the component is operated.
    6. Attributes of products and equipment shall be accomplished at the Type level. Duplicate properties on instance objects shall not be allowed.
    7. Attribute Values of Spaces and Types shall carry through from BPie to COBie exchanges.
    8. Units shall be provided for each Attribute.
  6. Additional quality constraints shall include
    1. Allowed properties in building information models shall reflect the set of information found on the design drawing schedules and those properties found in the Specifiers' Properties information exchange (SPie) ProductGuide.
    2. Properties whose values are unknown shall be identified as "n/a."
    3. Building information model content shall identify every products and equipment found in design drawings schedules.
    4. Building information model content shall match placement of equipment within spaces identified in design drawings and/or related schedules.
  7. The construction/commissioning challenge shall have as an option the consumption an existing Clinic COBie Design file (ZIP) when this file has been updated to reflect the Building Programming information exchange content.
  8. The expected outcome of the construction/commissioning challenge shall reflect the natural progression of product selection, installation, and commissioning as shown in the Clinic COBie Facility Management Handover file (ZIP) plus those carry-thru properites identified in the Building Programming information exchange content.
  9. This challenge will not consider information on the COBie worksheets either through design or construction/commissioning: COBie.Assembly, COBie.Connection, COBie.Coordinate, COBie.Issues, COBie.Impact.

Evaluation Criteria

Based on the Approach and Requirements the following paragraphs described the criteria that were ultimately used to evaluate the deliverables submitted for each of the different exchanges presented at the 2013 bSa Challenge.


The BPie project team evaluated BPie files in an iterative fashion during the course of the BPie development project to validate the requirements for BPie and demonstrate software systems potential for compliance with a future BPie standard. BPie files provided are considered demonstrations and are provided as proof-of-concept. As this project continues, specific evaluation criteria will be developed, published, and tested.


The HVACie project team evaluated HVACie files in an iterative fashion during the course of the HVACie development project to validate the requirements for HVACie and demonstrate software systems potential for compliance with a future HVACie standard. HVACie files provided are considered demonstrations and are provided as proof-of-concept. As this project continues, specific evaluation criteria will be developed, published, and tested.


Evaluation criteria for design and construction COBie files necessarily differ. Design COBie files produced from architectural design and consulting engineering software should provide data that matches the associated design stage drawings. The design COBie file provides the "outline" of all of the equipment that is to be placed during construction. Construction COBie simply demonstrates the completion of that outline during construction, commissioning and handover.

Design COBie Challenge

The stated goal of the COBie 2013 Challenge, first presented to all vendors in February 2012 was that software companies should demonstrate the creation of a building model whose data precisely matches that of a set of sample drawings. All scheduled asses shown on drawing schedules were to have appeared in the vendors COBie models. Vendors were asked to review the Common BIM Files, Medical Clinic drawings at that time and discuss any issues that they felt would cause them problems. Despite providing eleven months notice, many vendors did not review the actual requirements for the challenge until nearing the deadline for test file submission. At that time it was determined that some vendors did not commit the resources needed to meet the Design COBie Challenge.

As a result of the vendors' failure to commit needed modeling resources, two additional levels for the Design COBie Challenge were added to accommodate those who wished to participate without meeting the COBie Challenge. The first of these Design Challenge tests was "Could the software create a file whose data was in the COBie format?" This test did not assess the content of the COBie data but only the format of the file submitted. The second Design Challenge test was "Could the software produce a COBie file that contained part of the information found in the Medical Clinic drawings?" This test evaluated both the format and that content that was found against the Medical Clinic drawings.

Construction COBie Challenge

The requirement for evaluating a COBie deliverable at Construction Handover is to determine if the data in the COBie file, and associated attachments, is equivalent to that which would be provided in traditional paper construction documents. The Construction Challenge demonstrates the importation of COBie at Construction Documents Design stage and updating that information to match the information found in the COBie Handover data set. Since construction software products will import whatever design data is provided, this test only evaluated the quality of the output file based on the Common BIM Files Construction Handover stage Clinic Model. As design software vendors develop better tools and procedures to produce high-quality COBie design deliverables, contractors will have less hand-work to ensure the COBie data matches design drawings.

Challenge Contributions

Each Challenge event, since the first in 2008, has been increasingly precise and demonstrated new features and concepts in the process of evaluating contracted information exchanges. In 2008 we began with demonstrations whose resulting data had little to do with actual contract deliverables. During this 2013 Challenge process there were several contributions that allowed the community to see that there is a real potential to have data that 100% matches the information currently locked away on design drawings and paper construction contract deliverables.

An essential part of the development of this Design COBie Challenge through the spring of 2012 was the request by vendors to specify how models from multiple design disciplines should be merged at the Construction Documents Stage COBie submission. This is required since few vendors have products oriented toward proper merging of building information; the focus of most products during design has been to merge the geometry only. With detailed review by several vendors, the COBie Challenge team provided a new model merging specification, for Construction Documents Design Stage models that identified the order of precedence of potentially conflicting data in multiple model files. The proposed merging specification was founded on basic contract interpretation procedures that detailed information from engineering models takes precedence over less detailed information in architectural models. For example, the list of fixtures on the plumbing schedules takes precedence over the architectural layout of those fixtures in the floor plans if there are differences in these data sets.

This new merge model specification references IFC 2x3 and 2x4 objects and identifies which discipline takes priority at the Construction Documents Design Stage COBie deliverable. During the course of this challenge several software companies implemented part or all of these rules. The development of this model merging rule set is considered to be a significant achievement for implementers' model merging algorithms.

Another contribution of this COBie Challenge was that, for the first time, the challenge could be based on actual design drawings from a real facility. Past challenge events utilized only "toy" projects, such as the Common BIM File Duplex Apartment, that had never been built. The requirement that design software vendors use the same exact drawings as the basis for their models is the first attempt to provide a truly objective evaluation of information exchange products produced by each vendor. The requirement that all vendors use exactly the same "BIM Petri Dish" to conduct their experiments is crucial for the continued success of future challenges since vendor results can now be tested head-to-head. Advanced practitioners and researchers should both find the differences in vendor implementations of the same information exchange standard helpful to understanding the low-level formatting of vendors' information exchange files.

Another contribution of this challenge was the development, and mandatory use, of a robust, open-source COBie Tool Kit. Effort on the toolkit began with the explicit declarations of the mappings between IFC and COBie spreadsheet format in the COBie Responsibility Matrix. Utilizing the published COBie Responsibility Matrix Mappings the COBie Tool Kit provides 100% reversible transformations between COBie in spreadsheet and IFC FM Handover MVD file formats. The COBie Tool Kit also includes specific COBie Design and COBie Construction Quality Control Reports. These reports were required by all vendors participating in this Challenge. The COBie Tool Kit, based on the open source code allows all users to to automatically determine if a deliverable file meets COBie format specification.

The final contribution of the 2013 bSa Challenge was the first presentation of a unified framework for current and proposed information exchange standards. The framework, described as part of a Unified Theory of BIM, ties together components, assemblies and connections to represent building systems with the expected utilization of resources may allow the AECO industry to begin to systematically address issues related to the efficient use of natural resources within our built environment.


Participants in the COBie Challenge will be judged by the estimated time required to repair a file submitted against published models and quality standards.

For design COBie deliverables at the Schematic or Construction Documents stages, the information provided must match that provided at the Clinic Model drawings for that stage of design. The specific criteria used to evaluate the first of the two rounds of internal testing may be found here. The criteria for the second round of testing will be published here when that documentation has been completed.

Those construction software products consuming COBie files will also be evaluated to determine the extent to which data must be re-entered by hand from the design model. Construction handover submissions will be compared to the Clinic example model at FM Handover stage. As with the design software the "estimated time to repair" the resulting data will be reported in minutes.

The evaluation of the results from the two new information exchange formats presented during this meeting, BPie and HVACie, will be evaluated based on the tested software's ability to meet the data exchange format requirements. A data quality assessment will be completed for these formats in future meetings.

The Results

BPie Demonstration

Two companies demonstrated their ability to produce architectural programming information in BPie format. Now that there has been an initial demonstration of interest in this contracted information exchange format, future Challenges will provide quality assessments.  For this event, you may click through the links below to view the results of these demonstrations.

Company Presentation Example File
Distributed Room and Function System (DRofus) Presentation Programming File (IFC Format)
Onuma System Presentation Early Programming (BPie Format)
Design Programming (BPie Format)

HVACie Demonstration

One company demonstrated their ability to export ready-to-fabricate HVAC models in HVACie format. The model produced corresponds to shop drawings that would be submitted to an owner during the construction phase of the project. Since the resulting shop drawings are used directly to prefabricate duct runs and piping spools this effort is a major step forward in the production of near as-built models for those aspects of facilities for which as-builts are most critical, the HVAC system. Click through the links below to view the results of this demonstration.

Company Presentation Example File
East Coast CADD/CAM Presentation HVACie (IFC Format)
Extracted COBie output

COBie Challenge Results

The results of the COBie Design Challenge for Architectural Design and Coordinated Design are shown in the table below.

Company Presentation COBie File Result/Details
AutoDesk Coordinated Design COBie Coordinated Design 9 minutes to correct
Bentley Systems Coordinated Design COBie (IFC) Coordinated Design 3.6 hours to correct
GraphiSoft Architectural Design COBie (IFC) Architecture 8 hours to correct

The results of the COBie Construction Challenge at Handover are shown in the table below.

Company Presentation COBie File Result/Details
4Projects Construction Handover COBie Handover 0 hours to correct
Asite Construction Handover COBie Handover 0 hours to correct
CxAlloy Construction Handover COBie Handover 0 hours to correct
EcoDomus Construction Handover COBie Handover 0 hours to correct
Onuma Construction Handover COBie Handover 0 hours to correct

Back to bSa Proceedings

Innovative Solutions for the Built Environment

Established by the United States Congress, the Institute’s mission is to unite the entire building community in advancing building science and technology.

Address 1090 Vermont Avenue, NW, Suite 700, Washington, DC 20005-4950

© 2020 National Institute of Building Sciences. All rights reserved.

Contact Us:
Phone: (202) 289-7800
Fax: (202) 289-1092

Follow us on: