Eclipse Buckminster Component Assembly

iDempiere and Buckminster

This page looks at the use of Buckminster in the iDempiere project so that you can get a better understanding of the build process, and how Buckminster manages the components.

It was originally written by Thomas Bayen after his insight that he did not know anything about the iDempiere build process and after some research to change this fact. It has been updated following his invitation to "feel free to change all this here".

What is Buckminster?

Buckminster Component Assembly is a set of frameworks and tools for automating build, assemble and deploy processes. It is an Eclipse project that was named after the architect R. Buckminster Fuller.

Buckminster has been been designed to build and assemble multiple software components from multiple sources and formats, in combinations that are specific to a given target platform. It is in the same family of products as Maven, Apache Ivy, Apache Ant, and Gradle.

Where Buckminster is different is that its architecture allows for different types of component and its connectors are written so they use the native meta-data format of those components, so if your component is an Eclipse project, Buckminster uses the Eclipse Project information to get information about dependencies, build options, classpaths, etc.

How does it work?

Buckminster uses a Component Query (or CQUERY) to identify the components and their associated component information and create a bill of materials (BOM). It then locates all the components in the BOM using a Resource Map (or RMAP) and materializes the project by copying them into the target platform, resolving the dependencies according to the work context, system setup or operating system.

In the context of iDempiere, a component is a collection of files that make up an OSGi bundle, or plug-in. They are coarse-grained, self-contained, deployable units of code. Components can exist outside the workspace, and they may be dependent on other components.

Buckminster can store information about a component in the form of a Component Specification (CSPEC) in an XML file inside the component, but it is also able to read meta-data from pre-existing sources such as plugin.xml, feature.xml, POM files, etc, and translate such information on-the-fly into CSPEC, which it does in the case of iDempiere.

Buckminster allows you to retrieve components from existing repositories and component formats, such as CVS, SVN, git, Maven, or p2 repositories. This enables the iDempiere team to integrate existing Adempiere code with bespoke iDempiere plug-ins and other third-party modules, without reconfiguring them.

Buckminster can get the same component from different repositories, depending on the circumstances (different geographic locations, for example, or different software development environments like DEV, TEST, QA, PROD, and so on), making it easy to build your own on-site repositories for iDempiere.

Why Buckminster?

iDempiere is a collection of plugins that grows all the time. The conversion of ADempiere to the OSGi architecture to create iDempiere meant that there was a need for a build tool to manage components in a flexible way that could use the existing code repositories. The plug-in architecture of the project includes OSGi plugins from various external sources that should be managed in the same way as iDempiere's own plugins. Because the whole project was created as a collection of Eclipse projects, their meta-data and classpaths already there in the Eclipse project files, so Buckminster was the best tool to do this job.

Essential concepts

The key essential concepts used in Buckminster are:

• A Component Query (CQUERY). A CQUERY is a (usually quite small) XML file which names some component that you want to download and materialise. Included within the CQUERY is a reference (usually a URL) to a resource map (RMAP).
• A Resource Map (RMAP). An RMAP (again, an XML file) identifies possible locations - e.g. internet repositories - where a component - given in a CQUERY - can be found. The component in the CQUERY may depend on having further components (which in turn may need further ones, forming a transitive closure): the RMAP lists all the places where all of the components may be found.
• A publisher of components normally provides his/her audience of consumers with a CQUERY and RMAP. Armed with these, consumers can give the CQUERY (which in turn points to the given RMAP) to the Buckminster plugin to the Eclipse IDE and the appropriate components will be materialised into Eclipse. Materialisation into non-Eclipse environments, including stand-alone on a local or remote machine, is also possible.
• Bills of Materials (BOM). A BOM is (an XML file) generated automatically by Buckminster, given a CQUERY and accompanying RMAP: it gives a detailed list of all components, and their respective locations, in order to fulfill that particular CQUERY. A BOM file can also be published and used to materialise, in situations where the publisher wants to ensure that all of his/her audience of consumers get precisely the same set of specific component instances from the same set of repositories.
• Component Specification (CSPEC). A CSPEC (is an XML file and) describes the metadata, including dependencies on other components, for a particular component. CSPECs are automatically generated by Buckminster for Eclipse plugins and features, and thus Eclipse-based publishers in most cases do not need to manually write CSPECs.
• Component Specification Extension (CSPEX). A CSPEX (is an XML file and) is manually written to extend an automatically generated CSPEC in cases where Buckminster cannot find the entire metadata needed automatically.
• Materialisation Specification (MSPEC). An MSPEC (is an XML file which) can be provided by a publisher to ensure that downloaded components materialise to particular folders/directories on target machines, and with appropriate replace/update/discard policies if some of the components are in fact already downloaded on a specific machine.

Materializing iDempiere

iDempiere can be materialized in Eclipse in a development environment, or in a headless mode without Eclipse to build and deploy iDempiere on another server as part of a continuous integration process, for example.

Materializing iDempiere in Eclipse

Materialization Specification (MSPEC)

When you install iDempiere in Eclipse, you must first materialize the project by using a wizard to populate the workspace from the Buckminster Materialization Specification (MSPEC):

  File > Import... > Buckminster > Materialize from Buckminster MSPEC, CQUERY, or BOM > 
     Enter a URL or either MSPEC, BOM, or CQUERY: $IDEMPIERE_HOME/org.adempiere.sdk-feature/adempiere.mspec

where $IDEMPIERE_HOME is the root directory of your iDempiere download.

The iDempiere MSPEC instructs Buckminster to install the materialized components in ${workspace.root}/targetPlatform/, using the Component Query in adempiere.cquery, and gives the MSPEC a human-readable name: org.adempiere.sdk:eclipse.feature:1.0.0.qualifier

Component Query (CQUERY)

Buckminster then reads the Component Query from the CQUERY file named in the MSPEC, triggering its resolution and provisioning process. The CQUERY tells Buckminster where to find the Resource Map, adempiere.rmap, and also defines a number of advisorNodes which specify how groups of components should be handled.

The CQUERY resolution runs for a few minutes to build the Bill of Materials. The wizard gives you the option of saving the BOM to a file if you are interested in reading 20,097 lines of XML.

Resource map (RMAP)

Finally, Buckminster materializes the BOM by looking in the Resource Map (RMAP) for the location of the components and copying the JARs to the targetPlatform.

Note: materialization takes a very looooooong time, so read the BuckyBook while you wait.

Materializing iDempiere without Eclipse

It is possible to build iDempiere without Eclipse using the Buckminster headless functionality.

Documentation

• Key concepts
• Quick start to Buckminster
• The BuckyBook is the definitive guide
• Buckminster Project Wiki
• Reduan's Buckminster video