vesta-intro - Introduction to Vesta

vesta-intro(1)

Name

vesta-intro - Introduction to the Vesta configuration management system

Description
Terminology
Configuration Files
Name Interpretation
See Also
Limitations
Author

Description

Vesta is a system for software configuration management. It consists of a source code storage and control component, comprising the repository and repository tools, and a building component, comprising the evaluator, the function cache, and the weeder. For an overview, see the paper The Vesta Approach to Software Configuration Management, available as Research Report 168, Compaq Systems Research Center. Additional documentation is available in the references listed under See Also below.

Terminology

Here is a brief introduction to some basic Vesta terminology. The terms are ordered to avoid forward references as much as possible.
Alphabetical list: advance, agreement, appendable directory, base, branch, checkin, checkout, configuration file, evaluator, function cache, ghost, immutable, immutable directory, immutable file, import, master, master stub, model, mutable attributes, mutable directory, mutable file, nonmaster stub, package, replication, repository, repository tools, reservation stub, runtool server, SDL, session, snapshot, standard construction environment, stub, symlink-to, version, vesta.cfg, volatile directory, weeder, working directory

repository
The Vesta repository is responsible for long-term data storage in files and directories. It acts as an NFS file server, making all source code stored in Vesta visible in the ordinary file name space. See repository(8).

repository tools
The repository has several special features not shared by ordinary NFS servers, which are accessible through the command line tools referenced below. Many of the standard Vesta conventions and policies are implemented in the tools, not in the repository proper, making them relatively easy to change. The documentation for each tool is referenced under See Also below.

immutable
All source code permanently stored by the repository is immutable; that is, it cannot be modified. Once a filename is bound to a particular file, that file cannot be edited or replaced with a different file. This feature supports reproducible building and simplifies both source replication and build caching.

version
By convention, every file permanently stored in the repository has a version number somewhere in its name. Thus, although each file is immutable, the repository can represent an evolving program as a sequence of files with different version numbers.

package
We have found it convenient to organize sources into packages. A package is a tree of files and directories that typically change together. The Vesta repository tools do versioning at the package level, not the level of individual files: each package is a sequence of versions. For example, /vesta/src.dec.com/vesta/repos/74/src/RepositoryMain.C is one of the files in version 74 of the package named /vesta/src.dec.com/vesta/repos. The vcreate(1) tool creates a package.

branch
The versions of a package are numbered in a linear sequence. To represent forked or parallel development, the repository tools support the concept of a branch. A branch is a new package that is placed as a subdirectory of the parent package, with a name based on one of the versions of the parent package (the branch base), and with a version 0 that is identical to the branch base. The vbranch(1) tool creates a branch.

appendable directory
An appendable directory is a directory in which new names can be inserted, but existing names cannot be deleted and reused. Packages are stored in a tree of appendable directories that is conventionally mounted as /vesta. Each package is itself an appendable directory so that new versions can be added.

immutable directory
An immutable directory is a directory that cannot be changed at all; names can neither be created nor deleted. A package version is stored as a tree of immutable directories.

immutable file
An immutable file is a file whose contents cannot be changed. The files in a package version are immutable.

mutable attributes
A source object's mutable attributes are a set of (name, value) pairs that the repository attaches to the object. The repository tools use mutable attributes to store many kinds of information; for example, when and by whom a package was created, and on what previous versions a new version was based. The vattrib(1) tool can be used to view or modify mutable attributes.

mutable directory
A mutable directory is a directory implemented by the repository that looks much like an ordinary filesystem directory; files and subdirectories in such a directory can be freely created, deleted, or edited. Mutable directories are stored in a tree that is conventionally mounted as /vesta-work.

mutable file
A mutable file is a file in a mutable directory. Internally, a mutable file may be implemented as a link to an immutable file, with copy-on-write if a user attempts to edit the file, but this detail is not visible through the NFS interface.

base
The repository can quickly create a mutable directory that is based on (initialized from) any immutable directory.

snapshot
The repository can quickly create an immutable directory that is a snapshot (immutable copy) of any mutable directory.

working directory
A working directory is a mutable directory created for the purpose of editing sources and developing new package versions. It may either be initially empty or based on a previous package version.

checkout
To check out a package means to initiate development of the package's next version. The repository tools use a locking (pessimistic) concurrency control model, so the next version number is reserved at checkout time. The vcheckout(1) tool is used to check out a package. By default, it creates a working directory based on the most recent previous package version. It also creates a session directory and a reservation stub, described next.

session
Because the Vesta evaluator (builder) can operate only on immutable sources, Vesta users end up creating a large number of versions, most of which are of little long-term interest. For example, each attempt to modify and recompile a program will generate a new version. Typically many of these versions will have syntax errors or other simple bugs. We have therefore found it convenient to distinguish between two kinds of package versions: public versions, which a developer typically creates when he is ready to release code for use by others, and session versions, in-between versions created during development and debugging. A session directory holds session versions. For example, the versions in the development session that led up to the creation of version /vesta/src.dec.com/vesta/repos/74 appear in the session directory /vesta/src.dec.com/vesta/repos/checkout/74, under the names 0, 1, 2, etc.

advance
To advance a session means to create the next session version, by making a snapshot of the working directory and placing it in the session directory. The vadvance(1) tool handles this task.

stub
A stub is an object that can appear in the repository name space as a placeholder. A stub is manifested through the NFS interface as an empty file with unusual access permissions (details below).

reservation stub
A reservation stub is a stub created by the checkout tool as a placeholder to reserve the next package version number. On a reservation stub, all the read and execute permissions are turned off, but the setuid bit is turned on. The write permissions are meaningful, so they are allowed to show through. Thus a reservation stub will typically appear in an "ls -l" listing with mode --wS------ or --wS-w----.

checkin
To check in a package means to create a new public version. The vcheckin(1) tool handles this task. It works by replacing the reservation stub created at checkout time with a copy of the last version in the session directory. Checkin also ends the session by deleting the working directory.

ghost
Although names in appendable directories can never be deleted and reused for different objects, the repository does support a limited form of deletion. A ghost is a placeholder for an object that has been deleted. The ghost preserves the knowledge that its name was used for a real (non-ghost) object in the past and must not be used for a different object in the future. A ghost can safely be replaced with an exact copy of the deleted object if one can be found, but it is otherwise unchangeable and cannot be deleted. A ghost is manifested through the NFS interface as an empty file with all access permissions turned off and the sticky bit turned on, so "ls -l" will show ---------T.

replication
Conceptually, all objects in all Vesta repositories are named in one large hierarchical namespace rooted at /vesta. (We exclude mutable and volatile directories here.) Each repository may contain some subtree of the global namespace. Replication exists when the subtrees stored at two or more repositories overlap. The Vesta Replication section in the vrepl(1) man page explains replication in more detail.

agreement
Roughly speaking, in the presence of replication, one or more repositories agree (are consistent) if (1) there are no cases where the same name means something different in two different repositories, and (2) each name is marked as master in at most one repository. Agreement is a somewhat subtle concept that is explained more fully in the Vesta Replication section of the vrepl(1) man page, and defined precisely in Vesta: A System for Software Configuration Management.

master
Each object under /vesta has a boolean master flag. Mastership is used for several purposes, explained more fully in the Vesta Replication section of the vrepl(1) man page.

master stub
A stub whose master flag is true. A reservation stub is always a master stub.

symlink-to
The repository stores symbolic links in a somewhat odd way, as stubs or ghosts with the mutable attribute symlink-to. The value of this attribute is the link value. This detail is not visible through the NFS interface, but is mentioned in the man pages for some of the repository tools.

nonmaster stub
A stub whose master flag is false. A nonmaster stub is used as a placeholder for an object that may exist in some repository, but is not replicated locally. On a non-master stub, all permissions are turned off, but the setgid bit is turned on, so "ls -l" will show ------S---.

SDL
The Vesta software description language is a specialized language used to describe how to build a software system from its components. It includes native data structures suited for representing files and directories, a primitive for invoking external tools like compilers and linkers, and general-purpose language features such as functions, conditionals, arithmetic, lists, and string manipulation. See The Vesta-2 Software Description Language, also available as an appendix in Vesta: A System for Software Configuration Management.

model
A model or system model is a program written in the Vesta SDL.

import
The Vesta SDL is modular; the description of a large software system can be broken up into many different models. One model uses another by importing it.

standard construction environment
The standard construction environment is a standard set of build tools (such as compilers and linkers), libraries, and header files. User-written models gain access to this environment by importing a Vesta-provided model that describes it. For example, the standard construction environment models that are used to build Vesta on Red Hat i686 Linux come from the package /vesta/vestasys.org/platforms/linux/redhat/i686/std_env.

evaluator
The evaluator is an interpreter for the Vesta SDL. See vesta(1).

runtool server
The runtool server is a server process that runs on each Vesta client machine. It assists the evaluator in invoking external tools like compilers and linkers. See RunToolServer(1).

function cache
The function cache is a central server that stably caches the results of past Vesta evaluations in order to speed up future evaluations. See VCache(1).

weeder
The weeder deletes material from the function cache that is no longer wanted, in order to free up disk space. See VestaWeed(1).

volatile directory
Volatile directories allow tools started by the evaluator to view Vesta bindings as filesystem directories, with the changes the tools make recorded by the repository and available to the evaluator. The volatile directory tree is conventionally mounted under /vesta-work/.volatile. It should normally never be accessed directly by users, to avoid creating false dependencies in ongoing evaluations.

Configuration Files

Most Vesta tools use a standard mechanism for determining configuration information about a Vesta installation. This mechanism attempts to locate a configuration file named vesta.cfg by searching in a number of locations. The vgetconfig(1) man page describes a command-line interface for determining the location of the configuration file and for printing the value of a particular configuration variable. The meanings of the configuration variables recognized by the various tools are documented in their individual man pages.
For information on configuration file location and syntax, see the vesta.cfg(5) man page.

Name Interpretation

Although the names that you can give on the command line to a Vesta repository tool look like filenames, they have somewhat different properties, because they must be looked up directly in the repository, not in the local filesystem.
First, if a name is unqualified (does not have "/", ".", or ".." as its first arc), it is usually qualified by prepending an appropriate default directory name, as explained in the individual documentation for each tool.
Next, a name that does not begin with "/" is handled by asking the system for the absolute name of the current working directory (using the getcwd(3) library routine) and prepending the string that is returned. This is the only case where the state of the local filesystem matters.
Next, the name is textually processed to remove ".", empty, and ".." arcs. Note that we do not check whether an arc cancelled out by ".." actually exists in the repository.
Finally, a name that begins with /vesta or [UserInterface]AppendableRootName is looked up in the appendable part of the repository. Usually the default local repository is used, but some of the tools have command line options to let you specify a different repository, and some will automatically find the master copy or another copy of the specified source, as explained in their individual documentation. Similarly, a name that begins with /vesta-work or [UserInterface]MutableRootName is always looked up in the mutable part of the default local repository. It makes no difference whether the default repository (or even some other repository) is NFS mounted on the machine where you run the command.
Any name that does not begin with one of the recognized prefixes is rejected with the error message "not in the Vesta repository".
Note that symbolic links are not expanded during this name processing and are not understood by the repository. Thus, for example, the command "vcheckout -o latest foo/bar" will not use the latest version of foo/bar as the old version; it will attempt to use the "latest" link itself as the old version, which will not work.

Limitations

This man page should probably cover the SDL, evaluator, cache, and weeder in more detail. Maybe the glossary should be divided into sections.

Author (of this document)

Tim Mann

Last modified on Fri Nov  9 12:14:21 EST 2001 by ken@xorian.net  
     modified on Fri Mar  9 16:56:30 PST 2001 by mann  
     modified on Tue Jul 20 14:59:25 PDT 1999 by heydon

This page was generated automatically by mtex software.