Loading and Exporting DataCommand Line

Exporting Data

View as Markdown

COPY TO

COPY ( <SELECT statement> ) TO '<file path>' [WITH (<property> = value, ...)];

<file path> must be a path on the server. This command exports the results of any SELECT statement to the file. There is a special mode when <file path> is empty. In that case, the server automatically generates a file in <HEAVY.AI Directory>/export that is the client session id with the suffix .txt.

Available properties in the optional WITH clause are described in the following table.

ParameterDescriptionDefault Value
array_null_handling

Define how to export with arrays that have null elements:

  • ‘abort’ - Abort the export. Default.
  • ‘raw’ - Export null elements as raw values.
  • ‘zero’ - Export null elements as zero (or an empty string).
  • ‘nullfield’ - Set the entire array column field to null for that row.

Applies only to GeoJSON and GeoJSONL files.

'abort'
delimiter

A single-character string for the delimiter between column values; most commonly:

  • , for CSV files
  • for tab-delimited files

Other delimiters include

,~, ^, and;.

Applies to only CSV and tab-delimited files.

Note: HEAVY.AI does not use file extensions to determine the delimiter.

escapeA single-character string for escaping quotes. Applies to only CSV and tab-delimited files.' (quote)
file_compression

File compression; can be one of the following:

  • ‘none'
  • 'gzip'
  • 'zip’

For GeoJSON and GeoJSONL files, using GZip results in a compressed single file with a .gz extension. No other compression options are currently available.

'none'
file_type

Type of file to export; can be one of the following:

  • ‘csv’ - Comma-separated values file.
  • ‘geojson’ - FeatureCollection GeoJSON file.
  • ‘geojsonl’ - Multiline GeoJSONL file.
  • ‘shapefile’ - Geospatial shapefile.

For all file types except CSV, exactly one geo column (POINT, LINESTRING, POLYGON or MULTIPOLYGON) must be projected in the query. CSV exports can contain zero or any number of geo columns, exported as WKT strings.

Export of array columns to shapefiles is not supported.

'csv'
headerEither 'true' or 'false', indicating whether to output a header line for all the column names. Applies to only CSV and tab-delimited files.'true'
layer_name

A layer name for the geo layer in the file. If unspecified, the stem of the given filename is used, without path or extension.

Applies to all file types except CSV.

Stem of the filename, if unspecified
line_delimiterA single-character string for terminating each line. Applies to only CSV and tab-delimited files.'\n'
nullsA string pattern indicating that a field is NULL. Applies to only CSV and tab-delimited files.An empty string, 'NA', or
quoteA single-character string for quoting a column value. Applies to only CSV and tab-delimited files." (double quote)
quotedEither 'true' or 'false', indicating whether all the column values should be output in quotes. Applies to only CSV and tab-delimited files.'true'

When using the COPY TO command, you might encounter the following error:

Query couldn’t keep the entire working set of columns in GPU Memory.

To avoid this error, use the heavysql command \cpu to put your HEAVY.AI server in CPU mode before using the COPY TO command. See Configuration.

Example

1COPY (SELECT * FROM tweets) TO '/tmp/tweets.csv';
2COPY (SELECT * tweets ORDER BY tweet_time LIMIT 10000) TO
3  '/tmp/tweets.tsv' WITH (delimiter = '\t', quoted = 'true', header = 'false');