How to Create a Custom Transport Script for Backups

This article provides a comprehensive guide on creating a custom transport script for backup operations, a method that allows for highly tailored backup destinations. This functionality is part of the Backup Configuration feature.

Understanding Custom Backup Transport Scripts

The custom transport script feature offers advanced users the flexibility to define unique backup destinations. It is important to note that creating these scripts requires a strong understanding of system administration and scripting.

Warning: Custom transport scripts are intended for advanced users only. For most users, WHM offers several convenient, pre-configured backup options:

A local directory

Amazon S3™

Backblaze B2

FTP

Google Drive™

Rsync

S3 Compatible

SFTP

WebDAV

The Backup Configuration feature enables the creation of custom destinations for your system backups.

Developing a Custom Transport Script

For each custom backup destination you configure in WHM’s Backup Configuration interface (WHM » Home » Backups » Backup Configuration), you must provide a custom transport script. The absolute path to this script is specified via the Script setting for the Custom destination type within the Additional Destinations section.

Script Execution Principles

The following rules govern how your custom transport script interacts with the system:

The script executes once for each command.
The script cannot retain state information between commands.
The system does not reuse connections between commands; instead, a new connection to the remote custom destination is established each time the script runs and is subsequently closed.

Information is passed to the script via the command line in the following sequence:

Command name.
Current directory.
Command-specific parameters.
Host.
Username.

Password information is securely passed to the script through the environment variables.

Required Script Commands

Your custom transport script must be capable of implementing the following commands:

Command	Description	Parameters
`chdir`	Changes the current directory on the remote destination. This is functionally equivalent to the `cd` command.	`$path` — The target file path.
`delete`	Removes an individual file from the remote destination.	`$path` — The target file path.
`get`	Copies a file from the remote destination to a specified local directory.	`$dest_root_dir` — The remote directory. `$dest_file` — The remote file name. `$local_file` — The full path to the local file.
`ls`	Produces output identical to the `ls -l` command, listing files and directories.	`$path` — The target file path.
`mkdir`	Creates a new directory on the remote destination.	`$path` — The target file path.
`put`	Copies a local file to a specified remote destination.	`$dest_root_dir` — The remote directory. `$dest_file` — The remote file name. `$local_file` — The full path to the local file.
`rmdir`	Deletes a directory on the remote destination.	`$path` — The target file path.

Important: When using the rmdir command, we strongly recommend that you carefully verify the path you intend to delete. Passing the root directory (/) as the path to delete will result in severe system issues.

Each of these commands is executed individually by the system as it transports backup files and validates the destination.

Any data intended for the user should be returned by your script to STDOUT.

Note: In the event of a script failure, output is directed to STDERR. Any data returned to STDERR will be logged by the system as part of the failure report.

Script Templates

You can leverage the /usr/local/cpanel/scripts/custom_backup_destination.pl.skeleton script provided in cPanel & WHM as a foundational template for developing your own custom_backup_destination.pl script. For a practical example of a backup transport script, refer to the /usr/local/cpanel/scripts/custom_backup_destination.pl.sample script.

Sample Skeleton Script

#!/usr/local/cpanel/3rdparty/bin/perl

# cpanel - scripts/custom_backup_destination.pl.skeleton      Copyright 2021 cPanel, L.L.C
#                                                           All rights Reserved.
# [email protected]                                         http://cpanel.net
# This code is subject to the cPanel license. Unauthorized copying is prohibited

use strict;
use warnings;

# These are the commands that a custom destination script must process
my %commands = (
    put    => \&my_put,
    get    => \&my_get,
    ls     => \&my_ls,
    mkdir  => \&my_mkdir,
    chdir  => \&my_chdir,
    rmdir  => \&my_rmdir,
    delete => \&my_delete,
);

# There must be at least the command and the local directory
usage() if ( @ARGV < 2 );

#
# The command line arguments passed to the script will be in the following order:
# command, local_directory, command arguments, and optionally, host, user password
# The local directory is passed in so we know from which directory to run the command
# We need to pass this in each time since we start the script fresh for each command
#
my ( $cmd, $local_dir, @args ) = @ARGV;

# complain if the command does not exist
usage() unless exists $commands{$cmd};

# Run our command
$commands{$cmd}->(@args);

#
# This script should only really be executed by the custom backup destination type
# If someone executes it directly out of curiosity, give them usage info
#
sub usage {
    my @cmds = sort keys %commands;
    print STDERR "This script is for implementing a custom backup destination\n";
    print STDERR "It requires the following arguments:  cmd, local_dir, cmd_args\n";
    print STDERR "These are the valid commands:  @cmds\n";
    exit 1;
}

#
# This portion contains the implementations for the various commands
# that the script needs to support in order to implement a custom destination
#

#
# Copy a local file to a remote destination
#
sub my_put {
    my ( $local, $remote, $host, $user ) = @_;
    my $password = $ENV{'PASSWORD'};
    return;
}

#
# Copy a remote file to a local destination
#
sub my_get {
    my ( $remote, $local, $host, $user ) = @_;
    my $password = $ENV{'PASSWORD'};
    return;
}

#
# Print out the results of doing an ls operation
# The calling program will expect the data to be
# in the format supplied by 'ls -l' and have it
# printed to STDOUT
#
sub my_ls {
    my ( $path, $host, $user ) = @_;
    my $password = $ENV{'PASSWORD'};
    return;
}

#
# Create a directory on the remote destination
#
sub my_mkdir {
    my ( $path, $recurse, $host, $user ) = @_;
    my $password = $ENV{'PASSWORD'};
    return;
}

#
# Change into a directory on the remote destination
# This does not have the same meaning as it normally would since the script
# is run anew for each command call.
# This needs to do the operation to ensure it doesn't fail
# then print the new resulting directory that the calling program
# will pass in as the local directory for subsequent calls
#
sub my_chdir {
    my ( $path, $host, $user ) = @_;
    my $password = $ENV{'PASSWORD'};
    return;
}

#
# Recursively delete a directory on the remote destination
#
sub my_rmdir {
    my ( $path, $host, $user ) = @_;
    my $password = $ENV{'PASSWORD'};
    return;
}

#
# Delete an individual file on the remote destination
#
sub my_delete {
    my ( $path, $host, $user ) = @_;
    my $password = $ENV{'PASSWORD'};
    return;
}

Sample Implementation Script

#!/usr/local/cpanel/3rdparty/bin/perl

# cpanel - scripts/custom_backup_destination.pl.sample      Copyright 2013 cPanel, L.L.C
#                                                           All rights Reserved.
# [email protected]                                         http://cpanel.net
# This code is subject to the cPanel license. Unauthorized copying is prohibited

use strict;
use warnings;
use Cwd qw(getcwd abs_path);
use File::Spec;
use File::Copy;
use File::Path qw(make_path remove_tree);
use autodie qw(:all copy);

# These are the commands that a custom destination script must process
my %commands = (
    put    => \&my_put,
    get    => \&my_get,
    ls     => \&my_ls,
    mkdir  => \&my_mkdir,
    chdir  => \&my_chdir,
    rmdir  => \&my_rmdir,
    delete => \&my_delete,
);

# There must be at least the command and the local directory
usage() if ( @ARGV < 2 );

#
# The command line arguments passed to the script will be in the following order:
# command, local_directory, command arguments, and optionally, host and user
# The local directory is passed in so we know from which directory to run the command
# we need to pass this in each time since we start the script fresh for each command
#
my ( $cmd, $local_dir, @args ) = @ARGV;

# complain if the command does not exist
usage() unless exists $commands{$cmd};

# For this example transport, we are going to simply copy everything under this directory
my $dest_root_dir = '/custom_transport_demo';
mkdir $dest_root_dir unless -d $dest_root_dir;

# Step into the local directory
# This will be under the directory that we have as the file destination
$local_dir = File::Spec->catdir( $dest_root_dir, $local_dir );
make_path($local_dir) unless -d $local_dir;
chdir $local_dir;

# Run our command
$commands{$cmd}->(@args);

#
# This script should only really be executed by the custom backup destination type
# If someone executes it directly out of curiosity, give them usage info
#
sub usage {
    my @cmds = sort keys %commands;
    print STDERR "This script is for implementing a custom backup destination\n";
    print STDERR "It requires the following arguments:  cmd, local_dir, cmd_args\n";
    print STDERR "These are the valid commands:  @cmds\n";
    exit 1;
}

#
# Convert a path to be under our destination directory
# Absolute paths will be directly under it,
# relative paths will be relative to the local directory
#
sub convert_path {
    my ($path) = @_;

    if ( $path =~ m\|^/\| ) {
        $path = File::Spec->catdir( $dest_root_dir, $path );
    }
    else {
        $path = File::Spec->catdir( $local_dir, $path );
    }

    return $path;
}

#
# Convert a full path to the path under the the directory
# where we copy all the files
#
sub get_sub_directory {
    my ($path) = @_;

    # The first part will be the destination root directory,
    # Remove that part of the path and we will have the subdirectory
    $path =~ s\|^$dest_root_dir\|;

    return $path;
}

#
# This portion contains the implementations for the various commands
# that the script needs to support in order to implement a custom destination
#

#
# Copy a local file to a remote destination
#
sub my_put {
    my ( $local, $remote, $host, $user ) = @_;
    my $password = $ENV{'PASSWORD'};

    $remote = convert_path($remote);

    # Make sure the full destination directory exists
    my ( undef, $dir, undef ) = File::Spec->splitpath($remote);
    make_path($dir) unless ( $dir and -d $dir );
    copy( $local, $remote );
    return;
}

#
# Copy a remote file to a local destination
#
sub my_get {
    my ( $remote, $local, $host, $user ) = @_;
    my $password = $ENV{'PASSWORD'};

    $remote = convert_path($remote);

    copy( $remote, $local );
    return;
}

#
# Print out the results of doing an ls operation
# The calling program will expect the data to be
# in the format supplied by 'ls -l' and have it
# printed to STDOUT
#
sub my_ls {
    my ( $path, $host, $user ) = @_;
    my $password = $ENV{'PASSWORD'};

    $path = convert_path($path);

    # Cheesy, but this is a demo
    my $ls = `ls -al $path`;

    # Remove the annoying 'total' line
    $ls =~ s\|^total[^\n]*\n\|;

    print $ls;
    return;
}

#
# Create a directory on the remote destination
#
sub my_mkdir {
    my ( $path, $recurse, $host, $user ) = @_;
    my $password = $ENV{'PASSWORD'};

    $path = convert_path($path);

    make_path($path);

    die "Failed to create $path" unless -d $path;
    return;
}

#
# Change into a directory on the remote destination
# This does not have the same meaning as it normally would since the script
# is run anew for each command call.
# This needs to do the operation to ensure it doesn't fail
# then print the new resulting directory that the calling program
# will pass in as the local directory for subsequent calls
#
sub my_chdir {
    my ( $path, $host, $user ) = @_;
    my $password = $ENV{'PASSWORD'};

    $path = convert_path($path);
    chdir $path;

    print get_sub_directory( getcwd() ) . "\n";
    return;
}

#
# Recursively delete a directory on the remote destination
#
sub my_rmdir {
    my ( $path, $host, $user ) = @_;
    my $password = $ENV{'PASSWORD'};

    $path = convert_path($path);

    remove_tree($path);

    die "$path still exists" if -d $path;    return;
}

#
# Delete an individual file on the remote destination
#
sub my_delete {
    my ( $path, $host, $user ) = @_;
    my $password = $ENV{'PASSWORD'};

    $path = convert_path($path);

    unlink $path;
    return;
}

Code Example Components

Most variables are passed to the script as command-line arguments. If your script does not pass one of the hardcoded arguments to the core functions, the system will display all valid arguments in the global %commands hash.

Use Statements

Start your script with the necessary use statements, including any modules required for your specific transport implementation:

use strict;
use warnings;
use Cwd qw(getcwd abs_path);
use File::Spec;
use File::Copy;
use File::Path qw(make_path remove_tree);
use autodie qw(:all copy);

The %commands Hash

Note: Your script is restricted to processing only the commands defined in the %commands hash.

my %commands = (
    put    => \&my_put,
    get    => \&my_get,
    ls     => \&my_ls,
    mkdir  => \&my_mkdir,
    chdir  => \&my_chdir,
    rmdir  => \&my_rmdir,
    delete => \&my_delete,
);

Command Subroutines

Every script invocation commences with a command and a local directory. Command-line arguments must be provided in the following strict order:

$cmd — The command to be executed.
$local_dir — The local working directory.
@args — The arguments specific to the command.
$host — (Optional) The hostname or IP address of the remote destination.
$user — (Optional) The username for the remote destination account.
$password — (Optional) The password for the remote destination.

Ensure that you use the arguments appropriate for each command and variable.

Important Considerations:

The optional $host, $user, and $password values should only be included if they have been configured within the transport setup.

If the $password value is included, it **must** be accessed through the ENV hash.

The $local_dir variable **must** be present in every command subroutine you create, as each command is called individually by the script.

my ( $cmd, $local_dir, @args, $host, $user ) = @ARGV;
my $password = $ENV{'PASSWORD'};
usage() unless exists $commands{$cmd};

The `put` Function

The put function instructs the script to upload or copy a local file to a remote destination, operating similarly to the FTP put command.

Recommendation: For more robust and reliable transports, we strongly advise implementing comprehensive error checks at each step to ensure all errors are accurately reported.

sub my_put {
    my ( $local, $remote, $host, $user ) = @_;   # Required argument order

    my $password = $ENV{'PASSWORD'};     # Enclose the password in the ENV hash

    $remote = convert_path($remote);     # the remote file's variable

    my ( undef, $dir, undef ) = File::Spec->splitpath($remote); # Make sure the full destination directory exists
    make_path($dir) unless ( $dir and -d $dir );

    copy( $local, $remote ); # copy the local file to the remote file
    return;
}

The `get` Function

The get function directs the script to download or retrieve a file from a remote destination to a local one, functionally mirroring the FTP get command.

sub my_get {
    my ( $remote, $local, $host, $user ) = @_;
    my $password = $ENV{'PASSWORD'};
    $remote = convert_path($remote);

    copy( $remote, $local );
    return;
}

The `ls` Function

The ls function retrieves and displays a listing of a remote file or directory, comparable to the output of the ls -l command in FTP or a local command line environment.

sub my_ls {
    my ( $path, $host, $user ) = @_;

    my $password = $ENV{'PASSWORD'};

    $path = convert_path($path);

    my $ls = `ls -al $path`;

    # Remove the annoying 'total' line
    $ls =~ s\|^total[^\n]*\n\|;

    print $ls;
    return;
}

The `mkdir` Function

The mkdir function is responsible for ensuring that a specified directory exists on the remote machine, thereby allowing the backup system to upload files to a valid and accessible path.

Note: Although not all transports may utilize a feature equivalent to the mkdir function, it is mandatory to include this function within your script.

sub my_mkdir {
    my ( $path, $recurse, $host, $user ) = @_;

    my $password = $ENV{'PASSWORD'};

    $path = convert_path($path);

    make_path($path);

    die "Failed to create $path" unless -d $path;
    return;
}

The `chdir` Function

The chdir function facilitates storing the current working directory, which is useful for contextual operations. However, it's important to understand that for custom transport scripts, the system does not persist session information across individual operations performed by a single active process.

sub my_chdir {
    my ( $path, $host, $user ) = @_;
    my $password = $ENV{'PASSWORD'};

    $path = convert_path($path);
    chdir $path;

    print get_sub_directory( getcwd() ) . "\n";
    return;
}

The `rmdir` Function

The rmdir function is designed to remove a directory, including all its contents recursively. Depending on the specific transport method employed, it may be necessary to delete all files and subdirectories within the target directory before this function can execute successfully.

Critical Warning: We strongly advise exercising extreme caution when using the rmdir function. Verifying the path you intend to delete recursively is absolutely crucial. Deleting the root directory (/) will lead to severe and potentially unrecoverable system issues.

sub my_rmdir {
    my ( $path, $host, $user ) = @_;

    my $password = $ENV{'PASSWORD'};

    $path = convert_path($path);

    remove_tree($path);

    die "$path still exists" if -d $path;    return;
}

The `delete` Function

The delete function is used to remove a single file from the remote destination.

Recommendation: It is strongly advised to ensure that the path used, whether relative or full, is appropriate and correctly specified for your transport method. If your transport does not offer a direct error status check, you can use the ls function on the file to confirm its successful deletion.

sub my_delete {
    my ( $path, $host, $user ) = @_;

    my $password = $ENV{'PASSWORD'};

    $path = convert_path($path);

    unlink $path;
    return;
}

Basic Error Handling

Best Practices:

Implement a basic error check to verify that the script receives the correct arguments when it is called by the system. At a minimum, the usage() if ( @ARGV < 2 ) command should be included.

Incorporate a built-in description within the script itself, detailing its purpose, functionality, and any other relevant information to facilitate easy identification and understanding.

sub my_delete {
usage() if ( @ARGV < 2 );

sub usage {
    my @cmds = sort keys %commands;
    print STDERR "This script is for implementing a custom backup destination\n";
    print STDERR "It requires the following arguments:  cmd, local_dir, cmd_args\n";
    print STDERR "These are the valid commands:  @cmds\n";
    exit 1;
}

The `$cmd` Command Execution

Utilize the %commands hash to call the specific code block associated with each command that needs to be executed.

$commands{$cmd}->(@args);

For further assistance or inquiries, please visit our support page.

How to Create a Custom Transport Script for Backups

Support

Understanding Custom Backup Transport Scripts

Developing a Custom Transport Script

Script Execution Principles

Required Script Commands

Script Templates

Sample Skeleton Script

Sample Implementation Script

Code Example Components

Use Statements

The %commands Hash

Command Subroutines

The `put` Function

The `get` Function

The `ls` Function

The `mkdir` Function

The `chdir` Function

The `rmdir` Function

The `delete` Function

Basic Error Handling

The `$cmd` Command Execution

Related Articles

Most Popular Articles

Company

How to Create a Custom Transport Script for Backups

Support

Understanding Custom Backup Transport Scripts

Developing a Custom Transport Script

Script Execution Principles

Required Script Commands

Script Templates

Sample Skeleton Script

Sample Implementation Script

Code Example Components

Use Statements

The %commands Hash

Command Subroutines

The put Function

The get Function

The ls Function

The mkdir Function

The chdir Function

The rmdir Function

The delete Function

Basic Error Handling

The $cmd Command Execution

Related Articles

Most Popular Articles

Company

Generate Password

The `put` Function

The `get` Function

The `ls` Function

The `mkdir` Function

The `chdir` Function

The `rmdir` Function

The `delete` Function

The `$cmd` Command Execution