NAME

public-inbox-clone - "git clone --mirror" wrapper

SYNOPSIS

public-inbox-clone [OPTIONS] INBOX_URL [INBOX_DIR]

public-inbox-clone [OPTIONS] ROOT_URL [DESTINATION] # public-inbox 2.0+

DESCRIPTION

public-inbox-clone is a wrapper around "git clone --mirror" for making the initial clone of a remote HTTP(S) public-inbox. It allows cloning multi-epoch v2 inboxes with a single command and zero configuration.

In public-inbox 2.0+, public-inbox-clone can create and maintain a mirror of multiple inboxes or code repositories using manifest.js.gz files like grok-pull(1) from grokmirror. public-inbox-fetch(1) is NOT required when using this mode.

It does not run public-inbox-init(1) nor public-inbox-index(1). Those commands must be run separately if serving/searching the mirror is required. As-is, public-inbox-clone is suitable for creating a git-only backup without Xapian and SQLite indices.

When cloning a single inbox, public-inbox-clone creates a Makefile with handy targets to update the inbox once indexed. This Makefile may be edited by the user; it will not be rewritten by public-inbox-fetch(1) unless it is removed completely.

public-inbox-clone does not use nor require any extra configuration files (not even "~/.public-inbox/config"), but it can download snippets suitable for adding to any public-inbox-config(5) file.

public-inbox-fetch(1) may be used to keep a single "INBOX_DIR" up-to-date.

For v2 inboxes, it will create a "$INBOX_DIR/manifest.js.gz" file to speed up subsequent public-inbox-fetch(1).

OPTIONS

--epoch=RANGE

Restrict clones of public-inbox-v2-format(5) inboxes to the given range of epochs. The range may be a single non-negative integer or a (possibly open-ended) "LOW..HIGH" range of non-negative integers. "~" may be prefixed to either (or both) integer values to represent the offset from the maximum possible value.

For example, "--epoch=~0" alone clones only the latest epoch, "--epoch=~2.." clones the three latest epochs.

Default: "0..~0" or 0.. or "..~0" (all epochs, all three examples are equivalent)

-I PATTERN

--include=PATTERN

When cloning a top-level with multiple inboxes via manifest, only clone inboxes and repositories matching a given wildcard pattern (using "*?" and "[]" is supported).

This is a new option in public-inbox 2.0+

--exclude=PATTERN

When cloning a top-level with multiple inboxes via manifest, ignore inboxes and repositories matching the given wildcard pattern. Supports the same wildcards as "--include"

This is a new option in public-inbox 2.0+

--inbox-config=always|v2|v1|never

Whether or not to retrieve the "$INBOX/_/text/config/raw" HTTP(S) endpoint when cloning.

Since we can't deduce v1 inboxes from code repositories, setting this to "v2" or "never" can allow faster clones of code repositories if no v1 inboxes are present.

Default: "always"

This is a new option in public-inbox 2.0+

--inbox-version=NUM

Force a remote public-inbox version (must be 1 or 2). This is auto-detected by default, and this option exists mainly for testing.

This is a new option in public-inbox 2.0+

--objstore=DIR

Enables space savings when the remote "manifest.js.gz" includes "forkgroup" entries as generated by grokmirror 2.x.

If "DIR" does not start with "/", "./", or "../", it is treated as relative to the "DESTINATION" directory. If only "--objstore=" is specified where "DIR" is an empty string (""), then "objstore" ("$DESTINATION/objstore") is the implied value of "DIR".

This is a new option in public-inbox 2.0+

--manifest=FILE

When incrementally updating an existing mirror, load the given manifest (typically "manifest.js.gz") to speed up updates.

By default, public-inbox writes the retrieved manifest to "$DESTINATION/manifest.js.gz", this directive also changes the destination to the specified "FILE"

If "FILE" does not start with "/", "./", or "../", it is treated as relative to the "DESTINATION" directory. If only "--manifest=" is specified where "FILE" is an empty string (""), then "manifest.js.gz" ("$DESTINATION/manifest.js.gz") is the implied value of "FILE".

When updating manifests with many forks using the same objstore, git 2.41+ is highly recommended for performance as we automatically use the "fetch.hideRefs" feature to speed up negotiation.

"--manifest=" is a new option in public-inbox 2.0+

--remote-manifest=URL|RELATIVE_PATH

Use an alternate location for the remote manifest.js.gz file. This may be specified as a full absolute URL (e.g "--remote-manifest=https://80x24.org/lore/pub/manifest.js.gz"), or a pathname relative to the ROOT_URL (e.g "--remote-manifest=pub/manifest.js.gz" when ROOT_URL is "https://80x24.org/lore/"

By default, "ROOT_URL/manifest.js.gz" is used.

This is a new option in public-inbox 2.0+

--project-list=FILE

When cloning code repos from a manifest, generate a cgit-compatible project list.

If "FILE" does not start with "/", "./", or "../", it is treated as relative to the "DESTINATION" directory. If only "--project-list=" is specified where "FILE" is an empty string (""), then "projects.list" ("$DESTINATION/projects.list") is the implied value of "FILE".

This is a new option in public-inbox 2.0+

--post-update-hook=COMMAND

Hooks to run after a repository is cloned or updated, "COMMAND" will have the bare git repository destination given as its first and only argument.

For v2 inboxes, this operates on a per-epoch basis.

May be specified multiple times to run multiple commands in the order specified on the command-line.

This is a new option in public-inbox 2.0+

-p

--prune

Pass the "--prune" and "--prune-tags" flags to git-fetch(1) calls on incremental clones.

This is a new option in public-inbox 2.0+

--purge

Deletes entire repos which no longer exist in the remote manifest, or are filtered out by "--include=" or "--exclude=".

This is only useful when using "--manifest"

This is a new option in public-inbox 2.0+

--exit-code

Exit with 127 if no updates are done when relying on a manifest. Updates include fingerprint mismatches in the manifest, new symlinks, new repositories, and removed repositories from the --project-list

This is a new option in public-inbox 2.0+

-k

--keep-going

Continue as much as possible after an error.

This is a new option in public-inbox 2.0+

-n

--dry-run

Show what would be done, without making any changes.

This is a new option in public-inbox 2.0+

-q

--quiet

Quiets down progress messages, also passed to git-fetch(1).

-v

--verbose

Increases verbosity, also passed to git-fetch(1).

--torsocks=auto|no|yes

--no-torsocks

Whether to wrap git(1) and curl(1) commands with torsocks(1).

Default: "auto"

-j JOBS =item --jobs=JOBS

The number of parallel processes to spawn at once for various network operations using git(1) and/or curl(1).

EXAMPLES

To mirror the most recent epochs of dwarves and LTTng inboxes:

  public-inbox-clone --epoch=~0 \
        --include='*lttng*' --include='*dwarves' \
        https://80x24.org/lore/ /path/to/inbox-mirror

"https://lore.kernel.org/" may be used instead of "https://80x24.org/lore/"

To mirror all code repos of the sparse project:

  public-inbox-clone --objstore= --project-list= --prune \
        --include='*sparse*' --inbox-config=never \
        --remote-manifest=https://80x24.org/lore/pub/manifest.js.gz \
        https://80x24.org/lore/ /path/to/code-mirror

"https://git.kernel.org/" may be used instead of "https://80x24.org/lore/" and the "--remote-manifest" option can be omitted.

CONTACT

Feedback welcome via plain-text mail to <mailto:meta@public-inbox.org>

The mail archives are hosted at <https://public-inbox.org/meta/> and <http://4uok3hntl7oi7b4uf4rtfwefqeexfzil2w6kgk2jn5z2f764irre7byd.onion/meta/>

COPYRIGHT

Copyright all contributors <mailto:meta@public-inbox.org>

License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>

public-inbox-clone(1)

NAME

SYNOPSIS

DESCRIPTION

OPTIONS

EXAMPLES

CONTACT

COPYRIGHT

SEE ALSO