OSR » Ports-like management system

Summary

(originally by Berke DURAK)

First, the PMSs of Linux distributions are perfectly adequate for end users and administrators having to deploy and manage identical packages accross multiple machines. These are very complex systems including a substantial social part.

But they are not very suitable for developers.

As Markus pointed out, developers cannot go thru a packaging step to test the result of a change, much less wait for the fine Debian team.

One reason is that most developers, including myself, cannot be bothered to package software for a Linux distribution, because correct packaging is complex and requires adherence to a set of rules must be remembered and which cannot all be checked by software. As packaging is not done very often, you tend to forget those rules, and that is why we have people who package often to not forget the rules: maintainers. Without maintainers, we would be in a world of pain to install any piece of non-trivial software and we are thankful to them.

But developers absolutely need to be able to work on multiple versions of the same software component at once, patch those versions or compile them with unusual flags, and often use the absolutely latest unpackaged version. And that is the second reason why the Debian or Red Hat PMSs are not adequate. They have a single global state per system, which includes the installed files, and the package database, and cannot handle multiple versions of the same package, nor multiple compilations of the same version. (This also applies to Gentoo.)

We thus need versions, and lots of them! We need to base our developer packages on a version control system, in the style of BSD ports. BSD ports are usually based on CVS, sometimes on Subversion. As we are looking to increase collaboration, having a single point of contention is a serious limitations of these centralized systems; we'll prefer more recent "distributed" version control system.

Of available distributed VCSs with a serious user base, we have Darcs, Mercurial and Git.

Darcs is known to have some performance problems.

However Darcs2 is moving really fast. We're hoping that this will no longer be true, and that more serious comparisons will arise. -- Nicolas Pouillard

Mercurial (Hg) is written in Python extended in C for performance. It is quite friendly and works well under Windows. However, its developers are not as elite as Git's, its merging features are less advanced and Python sux0rz.

I have been using Hg for the past few months and been quite happy with it, but then I was mostly working alone. Git is certainly as good as Mercurial for that kind of usage and, as it is written in pure C, I advocate its use. Until someone writes a VCS in Ocaml, that is.

Let's get back to the subject. BSD ports are also based on make, whose main limitation, the static dependency graph, has been addressed in ocamlbuild. I know there is Omake, but I think it suffers from the ``Yet Another Turing-Complete Language'' syndrome.

So I am calling for a solution based on a ports-like system but based on a distributed VCS and on an improved ocamlbuild.

Assume you are writing a program FOO and want to use a package BAR available from bar.org. You tell ocamlbuild by adding some tag such as

  <mytarget.native>: require(http://bar.org/repository/)

And when you run ocamlbuild, it automatically checks out a copy of BAR, compiles and loads its myocamlbuild.ml module which adds the required flags. Of course it should be possible to specify a particular revision... And if BAR has itself dependencies, those too would be checked out.

Note that Git has a nice option for cloning checked out repositories using hard links; that could be used to maintain a cache of checkouts, for instance in the user's ~/.ocamlbuild/checkouts/ directory.

So basically I propose that we improve ocamlbuild to allow for multiple plugin files (using dynamic loading) and use that to define a BSD ports-like system targeted at developers.

The questions

(by Paolo Donadeo)

Where can I find a document with a general description of Ports?

(by Berke Durak)

http://www.lpthe.jussieu.fr/~talon/freebsdports.pdf

(suggested by Yaron Minsky on the Caml list)

Inspiration can be drawn from Cabal. See also the Haskell wiki.

Comparison of OMake and OCamlbuild desired

Omake

ocamlbuild

VCS?

Express yourselves here

Ideally we would abstract the VCS through a module interface anyway but here are some thoughts shared on the ocaml list: