explore paths ahead

Symbiosis

Labs

Source

license: MIT
README, browse source
snapshot: tar.gz, zip
git clone git://git.dvide.com/pub/symbiosis

Tutorial

Overview

Symbiosis is a high-level build engine to drive component builds in dependency order.

Symbiosis contains agents to cooperate with source control tools, build tools, and any other tools one could think of applying to a source tree. Adding new agents or improving existing agents are obvious areas of improvement.

Conceptually, Symbiosis provide a shared infrastructure in which components can be registered, located and fetched, so when a project needs another little piece, it can tap into the Symbiosis directory and have it made available from a compatible development branch.

There are no projects constructed from components. All components are projects and vice-versa.

Some may view this as a package manager, but the purpose is a bit different since it is being used during active development across different platforms and with no specific programming language in mind.

To deal with the "component" problem, many projects tend to copy source code from many places into one large source tree. This results in various maintenance headaches, and in some cases very long checkout times and/or complex local configurations before a build can run.

After a version release, platform specific package maintainers then take all this apart again and replace the assimilated code with platform specific modular packages.

Clearly, there is room for improvement, and Symbiosis attempts to address some of these issues.

Symbiosis does not aim to replace platform package managers, but hopefully a symbiosis component can easily generate the necessary platform specific packages as an option. No attempt is made to control this inside Symbiosis, but the hooks are in place to do such things.

Symbiosis is also not the perfect tool. Various command line and web wrappers would help, but there you are ...

Current State

For source control there is a basic git agent, a monotone agent and a wget agent. Git might fail in some cases as it seems to very repository specific what operations generate error messages. Input from git experts is welcome.

For building, any tool that works like Make with command-line targets and argument assignments can be used. There is also support for generating name/value pairs of build variables to be included by Make and Ruby build scripts, for example.

Description

The basic idea is to make it easy to configure large software projects out of available component building blocks.

We want to avoid having to copy source code into one single large source controlled tree with a forced consistent build system, or to depend on platform or programming language specific package managers.

This should make it simpler to maintain large multi-versioned projects, make it easer to reuse own software, and better manage third-party software.

For individual developers, all that is needed is a single pre-compiled symbiosis binary, a system configuration file, and a few compiler and source control tools. Symbiosis will then take care of the rest, assuming configuration work has been done properly.

It is easy to change component branches, or work with multiple component versions simultaneously. This includes having different branches for test and development, as an example.

Component source code is available in separate source repositories and branches. According to dependencies these are checked out as needed during an ongoing build process.

Developers need not download all potential components, only those that are actually needed.

JSON configuration files guide the checkout and build process, most notably the proxy files that identify available components. The configuration is deliberately distributed in many files to avoid merge conflicts and to support modularity.

Individual components have their own build system with minimal interference from symbiosis.

Location of build products, configuration settings, and dependencies are communicated through the symbiosis infrastructure.

The build is guided through customizable actions. There is no hardcoded sequence of actions such as fetch, patch, build, ..., although any action that requires source code will depend on a checkout action. New actions can be introduced and included as a dependency.

The compiled tool is a self-contained binary. It requires ocamlbuild from the OCaml 3.10.1+ to compile, but individual developers are not required to have OCaml installed.

Small tool wrappers known as agents interact with build systems and source control systems. New agents can be added and compiled into the symbiosis binary by modifying the source file ocamlbuild.ml.

Build Systems

To build the individual components, an actual build system such as Make is required. Existing components will already have made their choice.

For new components, ocamlbuild (part of the OCaml distribution) is a powerful alternative to Make and requires minimal configuration to build known target types. The ocamlbuild extension ocamlbuild-ctools is available at the Labs, and supports cross-platform generic C/C++ build targets, mixed ocaml / C compilation, and multi-variant build configurations. It is not required for running symbiosis, although ocamlbuild is also used by symbiosis internally.