Docker
Minimal, production-ready container image to run FastTransfer, a high-performance data transfer utility designed for data integration and automation workflows.
Image Overview
| Item | Value |
|---|---|
| Base image | debian:trixie-slim |
| Entrypoint | /usr/local/bin/FastTransfer |
| Updates | Published automatically via GitHub Actions for each new release and weekly security updates |
- arpeio/fasttransfer on DockerHubthe prebuilt image
- FastTransfer-Image on GitHubrepository of the image
Prerequisites
- Docker 24+ (or Podman)
A container engine to pull and run the image.
- Valid FastTransfer license
Request a trial license.
- FastTransfer_Settings.jsonOptional
For custom logging configuration.
Using the Prebuilt Image from DockerHub
Available Tags
- Version-specific tagsAligned with FastTransfer releases (e.g.,
v0.15.0) latesttagAlways points to the most recent FastTransfer version
Automatic Updates
- New releasesImages are automatically built when new FastTransfer versions are released
- Security updatesThe latest version of each minor branch is automatically rebuilt weekly (every Monday) with the latest base image and security patches
Pull and Run
- Pull the image
- Basic commands
# Latest version
docker pull arpeio/fasttransfer:latest
# Specific version
docker pull arpeio/fasttransfer:v0.15.0
# Get command line help
docker run --rm arpeio/fasttransfer:latest -?
# Check version
docker run --rm arpeio/fasttransfer:latest --version
License Requirement
Since version 0.14.0, pass the license content directly via --license "…":
export licenseContent=$(cat ./FastTransfer.lic)
# Use $licenseContent in your docker run commands
docker run --rm arpeio/fasttransfer:latest \
--license "$licenseContent" \
[other parameters...]
Prefer --env-file, Docker/Compose/Kubernetes secrets, or managed identities for cloud credentials. Avoid leaving the license content in shell history.
Examples
- SQL Server → SQL Server
- CSV files → SQL Server
Parallel database-to-database transfer:
export licenseContent=$(cat ./FastTransfer.lic)
docker run --rm \
arpeio/fasttransfer:latest \
--sourceconnectiontype "mssql" \
--sourceserver "host.docker.internal,1433" \
--sourceuser "SrcUser" \
--sourcepassword "SrcPass" \
--sourcedatabase "source_db" \
--sourceschema "dbo" \
--sourcetable "orders" \
--targetconnectiontype "msbulk" \
--targetserver "host.docker.internal,1433" \
--targetuser "DestUser" \
--targetpassword "DestPass" \
--targetdatabase "dest_db" \
--targetschema "dbo" \
--targettable "orders" \
--parallelmethod "Ntile" \
--distributekeycolumn "id" \
--loadmode "Truncate" \
--degree 8 \
--license "$licenseContent"
Import CSV files to SQL Server in parallel:
export licenseContent=$(cat ./FastTransfer.lic)
docker run --rm \
-v /local/data:/data \
arpeio/fasttransfer:latest \
--sourceconnectiontype "duckdbstream" \
--sourceserver ":memory:" \
--query "SELECT * FROM read_csv('/data/lineitem*.CSV')" \
--targetconnectiontype "msbulk" \
--targetserver "host.docker.internal,1433" \
--targetuser "DestUser" \
--targetpassword "DestPass" \
--targetdatabase "tpch_test" \
--targetschema "dbo" \
--targettable "lineitem" \
--parallelmethod "Random" \
--distributekeycolumn "l_orderkey" \
--mapmethod "Name" \
--loadmode "Truncate" \
--degree 16 \
--license "$licenseContent"
Configuration
Volumes
The Docker image declares several volumes to organize data and configuration:
VOLUME ["/config", "/data", "/work", "/logs"]
/configRead-only / Read-manyUser-provided configuration files (e.g., Serilog settings). Shared across multiple containers; not modified.
/dataRead-many / Write-manyInput/output data directory for file-based operations. Stores imported or exported data files.
/workRead-many / Write-manyTemporary working directory (container WORKDIR). Used internally for temporary processing.
/logsRead-many / Write-manyLog output directory (per-run or aggregated logs). Stores runtime and execution logs.
Configuring FastTransfer Logging
FastTransfer supports custom logging configuration through an external Serilog settings file in JSON format.
Example: Logging to Console, Airflow, and Dynamic Log Files
- ConsoleLogs to the console for real-time visibility
- AirflowRun summary to
/airflow/xcom/return.json, compatible with Airflow's XCom mechanism - Per-run filesJSON logs under
/logs, named from the date, source database, file name, timestamp and trace ID
FastTransfer_Settings.json
{
"Serilog": {
"Using": [
"Serilog.Sinks.Console",
"Serilog.Sinks.File",
"Serilog.Enrichers.Environment",
"Serilog.Enrichers.Thread",
"Serilog.Enrichers.Process",
"Serilog.Enrichers.Context",
"Serilog.Formatting.Compact"
],
"WriteTo": [
{
"Name": "Console",
"Args": {
"outputTemplate": "{Timestamp:yyyy-MM-ddTHH:mm:ss.fff zzz} -|- {Application} -|- {runid} -|- {Level:u12} -|- {fulltargetname} -|- {Message}{NewLine}{Exception}",
"theme": "Serilog.Sinks.SystemConsole.Themes.ConsoleTheme::None, Serilog.Sinks.Console",
"applyThemeToRedirectedOutput": false
}
},
{
"Name": "File",
"Args": {
"path": "/airflow/xcom/return.json",
"formatter": "Serilog.Formatting.Compact.CompactJsonFormatter, Serilog.Formatting.Compact"
}
},
{
"Name": "Map",
"Args": {
"to": [
{
"Name": "File",
"Args": {
"path": "/logs/{logdate}/{sourcedatabase}/log-{filename}-{LogTimestamp}-{TraceId}.json",
"formatter": "Serilog.Formatting.Compact.CompactJsonFormatter, Serilog.Formatting.Compact",
"rollingInterval": "Infinite",
"shared": false,
"encoding": "utf-8"
}
}
]
}
}
],
"Enrich": [
"FromLogContext",
"WithMachineName",
"WithProcessId",
"WithThreadId"
],
"Properties": {
"Application": "FastTransfer"
}
}
}
- If a target directory (such as
/logsor/airflow/xcom) does not exist, FastTransfer automatically creates it. - The file
/airflow/xcom/return.jsonis designed to provide run summaries compatible with Airflow's XCom mechanism.
Available Tokens for Path or Filename Formatting
{logdate}{logtimestamp}{sourcedatabase}{sourceschema}{sourcetable}{filename}{runid}{traceid}Mounting a Custom Settings File
# First, copy your config file to a volume location
cp ~/FastTransfer_Settings.json /volumes/fasttransfer-config/
# Then run FastTransfer with mounted volumes
docker run --rm \
-v fasttransfer-config:/config \
-v fasttransfer-logs:/logs \
arpeio/fasttransfer:latest \
--settingsfile "/config/FastTransfer_Settings.json" \
--sourceconnectiontype "mssql" \
--sourceserver "host.docker.internal,1433" \
--sourceuser "SrcUser" \
--sourcepassword "SrcPass" \
--sourcedatabase "source_db" \
--sourceschema "dbo" \
--sourcetable "orders" \
--targetconnectiontype "pgcopy" \
--targetserver "host.docker.internal:5432" \
--targetuser "PgUser" \
--targetpassword "PgPass" \
--targetdatabase "pg_db" \
--targetschema "dbo" \
--targettable "orders" \
--degree 12 \
--parallelmethod "Ntile" \
--distributekeycolumn "o_orderkey" \
--license "$licenseContent"
Related pages
- Apache Airflowrun the image from a DAG with the DockerOperator
- Kestrarun the image from a Kestra flow
- Logging overviewlogging settings and sinks