path: root/README.md

# MISC
## Manually Installed and Source Compiled

*an easier way to source-compile programs and stuff*

## why?

I run Debian Stable, which is great for the most part.  However, there are some
programs that I want to have a newer version of than what's in the Debian repos
--- and the easiest way to install those is by compiling them myself from
source.

However, compiling from source can be complex and fiddly, which makes it ripe
for automation!  `misc` is my submission into the wide and varied world of
software building software.  I use it and not other solutions for the following
reasons:

- It's written in bash and extended in bash, the _lingua franca_ of Unix-ish
  operating systems.  I could've done it in POSIX sh, but I really wanted to
  have arrays.
- Because it's written in bash, it's usable anywhere bash is installed.  So most
  places.
- It doesn't make too many assumptions.  It only _barely_ installs dependencies,
  and really is just a wrapper around the ol' `./configure`, `make`, `make
  install` workflow we all know and love.
- It's (_going to be_) extensible to cover uses I haven't thought of yet.

## usage

Basic usage is simply running `misc PLAN`, where `PLAN` is a file in
`$MISC_PLAN_ROOT` (default: `$XDG_CONFIG_HOME/misc`) that details how to install
a program.  There's no differentiation between `install` and `update`, and
there's currently no method of removing a program (in my experience,
_uninstalling_ is more bespoke than installing and would require greater
complexity).

Programs are installed under `$MISC_INSTALL_PREFIX`, which is `$HOME/.local` by
default.  I like putting programs under the `$HOME/.local` prefix for a few
reasons:

- They don't require `sudo` to install
- `$HOME/.local` already includes `$HOME/.local/share`, which is
  `$XDG_DATA_HOME` and already includes a bunch of stuff
- It's pretty easy to add `$HOME/.local/bin` and the like to `$PATH`.

Of course, you can change `$MISC_INSTALL_PREFIX` to whatever you like, either in
your shell's initiation file or in `$MISC_CONFIG`, or
`$XDG_CONFIG_HOME/misc/misc.sh` by default.

### `misc.sh`

In this section, I'll discuss all the environment variables that `misc` itself
uses.  Plans use a lot of environment variables too, but those are in the
following section (_Plans_).

- `$MISC_CONFIG` defines the location of `misc`'s configuration file.  This is a
  shell script that's sourced before building anything.  Default:
  `$XDG_CONFIG_HOME/misc/misc.sh`.
- `$MISC_REPO_ROOT` is the directory that all programs are downloaded or cloned
  to.  Default: `$XDG_DATA_HOME/misc`.
- `$MISC_PLAN_ROOT` is the directory where `misc` will search for plan files.
  Default: `$XDG_CONFIG_HOME/misc`.  *There might be a better place for these
  but until I can think of one this is where it'll be.*

The following variables are used when processing plans, but they're
`misc`-specific.  You might want to reference these in your plans.

- `$MISC_PLAN`: The name of the program to build and install or upgrade.
- `$MISC_FULL_PLAN`: The full path of the plan file; formed from
  `$MISC_PLAN_ROOT/$MISC_PLAN`.
- `$MISC_REPO`: The local directory holding the source of the program being
  built.  Most tasks should be performed within this directory.
- `$MISC_INSTALL_PREFIX`: The installation `PREFIX` variable that's invoked when
  `make`ing the program.  It can also be used in `$CONFIGURE_ARGS` and the like,
  which see.
- `$MISC_DONE`: A sigil variable indicating when building is done.  If this
  variable is non-empty, `misc` will detect the build/install as
  done---otherwise, it will run the default build/install flow.  The following
  functions automatically set `$MISC_DONE`; of course you can set it yourself in
  a plan.
  - `repo_pull`
  - `repo_ready`
  - `make`
  - `make_install`
  - `configure`

### `misc` invocation

`misc` takes a number of command-line arguments.  What follows is the output
you'll get when you run `misc -h`.

```
misc: install programs from source
--- Usage:
    misc -h
    misc [-d] [-n] [-r DIR] [-p DIR] REPO ...
--- Parameters:
    REPO    A program to pull from upstream, build, and install locally.
--- Flags:
    -h      Show this help and exit.
    -d      DRY RUN: Don't do anything, just print what would happen.
    -n      NUKE: Delete REPO, then re-download and build.
--- Options:
    -r DIR  Change the misc repo directory to DIR
              (default: $MISC_REPO_ROOT).
    -p DIR  Change the directory to search for install plans
              (default: $MISC_PLAN_ROOT).
    -i DIR  Change the PREFIX to use when installing programs
              (default: $MISC_INSTALL_PREFIX).
```

### Plans

"Plans" are what `misc` calls the instruction files to install programs.  There
are some examples in the `examples/` subfolder of this repo.

Plans are simply bash source files defining some variables and optionally
specifying build/install steps.  There is a default build process that `misc`
will use if it does not detect any build steps being defined in the plan file.
Below are the variables and functions in the `misc` environment at build-time;
of course since plans are just bash files, you can define your own or really do
anything you want.

### Plan variables

- `$SOURCE`: The source URL of the program's file or repository.
- `$CHECKOUT`: The Git checkout to clone.  Default: `master`.
- `$CONFIGURE_ARGS`: Arguments to pass to the `./configure` script.
- `$MISC_DEPENDENCIES`: Dependencies to install via `misc` before installing the
  planned program.
- `$APT_DEPENDENCIES`: Depenedencies to install via `apt` before installing the
  planned program.

### Plan functions

There are a number of functions defined in `misc` that plans inherit into their
environments.  You can use all of these in plan files, though this is not an
exhaustive list.  This is, however, all the functions that I think you might
find useful in a plan file.

#### Utility functions

- `die CODE MESSAGE...`: output MESSAGE to standard error and exit with CODE.
- `run COMMAND...`: log COMMAND, then execute it if not in dry-run mode.
- `cmd COMMAND...`: the same as `run`, but uses `command` instead of `eval`.
- `with_repo COMMAND...`: `cd` to `$MISC_REPO`, then execute COMMAND.

#### Building and installation functions

- `install_deps`: install the dependencies in `$MISC_DEPENDENCIES` and `$APT_DEPENDENCIES`.
- `repo_pull [CHECKOUT]`: pull the repo from `$SOURCE`, a git repository.  If
  CHECKOUT is given, use that branch; otherwise use `master`.
- `repo_download_extract`: download `$SOURCE` using `curl`, then untar it.  This
  expects the source URL to be a `.tar.gz` file.
- `repo_ready [TARGET...]`: ready the local source directory by running `git
  -dxf` (if applicable) and `make clean`.  If TARGETs are given, use those as the
  make targets instead of `clean`.
- `repo_nuke`: Delete the repository altogether before re-downloading or cloning it.

#### Plan convenience aliases

Since `misc` and plan files are simply shell scripts, you could just write the
direct commands to download, extract, and install a given program.  However,
then there's little reason to use `misc` --- so I've included a few thin wrapper
functions around common commands that operate in the `$MISC_REPO` directory and
specify a prefix of `$MISC_INSTALL_PREFIX`.

- `make [ARG...]`, `git [ARG...]`, `tar [ARG...]`: These run their commands with
  a `-C $MISC_REPO` argument for ease of use.  If you want to use the raw
  commands, `_make` `_git`, and `_tar` are defined for those.
- `make_and_install [TARGET...]`: First runs `make` with the supplied `TARGET`s,
  then runs `make install`.  Note that `make` is the `make` defined above, that
  is, the `misc`-ified one.
- `configure [ARG...]`: This runs `./configure` in the `$MISC_REPO` directory.
  If called with `ARG`s, it uses those; otherwise it uses the arguments defined
  in `$CONFIGURE_ARGS`.

## caveats & todos

As you can see from these variables and functions, `misc` is currently optimized
for a `./configure; make; make install`-style build system on a Debian-derived
host.  Again, since a plan file is just a shell script, you can run any commands
you like in there---and it'll still take advantage of `misc`'s standardized
prefix and build root.  See the `keepassxc` example for an example using the
`cmake` system, and `lieer` for a `python`-based system.

Of course, it'd be ideal to have these vagaries somewhat abstracted-away by
`misc`, so at some point in the future I'll include some sort of "add-on" system
where a user will be able to define further functions to fit alternative build
systems.  I'll also want to figure out a way to generalize dependency
installation across distributions, since the current solution (`apt install`
hardcoded into the script) is brittle.

If you have any ideas about these todos or how to solve the problems I'm
currently facing, feel free to [email me][mailto:git@acdw.net?subject=misc] with
your ideas.

## license

`misc` and all other code in this repository is licensed under the terms of the
ISC license.  See COPYING for details.