Transfer Parameters
FastTransfer requires parameters to define which data to transfer and how to load it into the target database. This page covers all transfer-related parameters for source selection, target configuration, and load behavior.
- Source ConfigurationWhat data to extract: a table, a query or a SQL file
- Target ConfigurationWhere to load the data: target schema and table
- Load BehaviorHow the data is loaded: load mode, batch size, work tables, column mapping
Source Configuration
These parameters define what data to extract from the source database.
Source Schema
Define the source schema name. This parameter is required by the Physical and DBPartition parallel methods.
Example:
./FastTransfer \
...
--sourceschema "public" \
...
--sourceschema--sourceschema "schema_name"
This parameter must be set if the Physical (except on Netezza) or DBPartition parallel method is used.
Source Table
Define the source table name. This parameter is required by the Physical and DBPartition parallel methods.
Example:
./FastTransfer \
...
--sourcetable "customers" \
...
--sourcetable--sourcetable "table_name"
This parameter must be set if the Physical (except on Netezza) or DBPartition parallel method is used.
Query
Plain text SQL query. Will be used instead of source table if provided.
Example:
./FastTransfer \
...
--query "SELECT * FROM customers WHERE country = 'USA'" \
...
--query--query "sql_query"
When using a custom query, FastTransfer will execute this query on the source database instead of reading from a table directly.
File Input
Input file storing SQL query. The file must exist. This allows you to store complex queries in a separate file.
Example:
./FastTransfer \
...
--fileinput "query.sql" \
...
--fileinput--fileinput "file_path"
The specified file must exist and contain a valid SQL query.
Target Configuration
These parameters define where to load the data in the target database.
- Target Schema
- Target Table
Load Behavior
These parameters control how data is loaded into the target table.
Load Mode
Load mode of the data into the target table.
Example:
./FastTransfer \
...
--loadmode Truncate \
...
--loadmodedefault Append--loadmode "mode"
Allowed Values:
AppendDefaultAppend data to the target table.
TruncateTruncate the target table before loading.
Batch Size
Batch Size for BulkCopy operations.
Example:
./FastTransfer \
...
--batchsize 500000 \
...
--batchsizedefault 1048576--batchsize "size"
Adjust the batch size based on your network bandwidth and target database capabilities. Smaller batches may be better for slower networks, while larger batches can improve throughput on fast connections.
Ignored with the adbc_* connection types: the drivers size their own batches.
Use Work Tables
Switch that will activate the usage of intermediate work tables. Useful in some rare cases.
Example:
./FastTransfer \
...
--useworktables \
...
--useworktablesSwitch, no value.
This option is useful in rare scenarios where direct loading causes issues. Work tables act as an intermediate staging area.
Not available with an adbc_* target.
Map Method
Mapping method for the columns between source and target.
Example:
./FastTransfer \
...
--mapmethod Name \
...
--mapmethoddefault Position--mapmethod "method"
Allowed Values:
PositionDefaultMap columns by their position in the source and target tables.
NameMap columns by their name (case insensitive) and ignore missing columns.
Use Position when source and target tables have identical column order. Use Name when column order differs or when some columns may be missing from either source or target.
- Parallel Parameterssplit the transfer over several threads
- Target Connection Parametersthe target database
- ADBC Connectionsbatch size and work tables with
adbc_*types