Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

A step by step guide to setup Rclone can be found at Step by step Rclone Setup for CloudStor file upload.
Advanced users can follow the compacted Advanced Rclone for CloudStor file upload guide

Upload files to Cloudstor

The following is an example of a command you can run in the terminal / command line to upload data to Cloudstor:

...

A complete list of additional options that can be passed to Rclone can be found at https://rclone.org/flags/

Example Initial Upload

For this example, the Cloudstor Access level given was edit for Folder_1 and read for Folder_2. Additionally, edit access was given to Folder_2a, which is a subfolder of Folder_2 as per below

...

Rclone will start the copy of the files and will provide you real time feedback about the processes running. By default, each transaction is retried 10 times in case there is an issue with the transfer.

Rclone in progress

The following is an example what will display during the Rclone process.

Code Block
breakoutModefull-width
2020-10-09 14:14:49 ERROR : lai/core1ha/20180706/DSC00165.JPG: Failed to copy: Update mkParentDir failed: Could not create directory /Shared/Folder_1/lai: Sabre\DAV\Exception\Forbidden: 403 Forbidden
2020-10-09 14:14:49 ERROR : lai/core1ha/20180706/DSC00178_C3.JPG: Failed to copy: Update mkParentDir failed: Could not create directory /Shared/Folder_1/lai: Sabre\DAV\Exception\Forbidden: 403 Forbidden
2020-10-09 14:14:50 ERROR : lai/core1ha/20180706/DSC00187_B2.JPG: Failed to copy: Update mkParentDir failed: Could not create directory /Shared/Folder_1/lai/core1ha: Sabre\DAV\Exception\Forbidden: 403 Forbidden
2020-10-09 14:14:52 ERROR : lai/core1ha/20180706/DSC00183_E2.JPG: Failed to copy: Update mkParentDir failed: Could not create directory /Shared/Folder_1/lai/core1ha: Sabre\DAV\Exception\Forbidden: 403 Forbidden
2020-10-09 14:14:54 ERROR : lai/core1ha/20180706/DSC00168_D5.JPG: Failed to copy: Update mkParentDir failed: Parent node does not exist: Sabre\DAV\Exception\Conflict: 409 Conflict
2020-10-09 14:14:54 ERROR : lai/core1ha/20180706/DSC00167_C5.JPG: Failed to copy: Update mkParentDir failed: Parent node does not exist: Sabre\DAV\Exception\Conflict: 409 Conflict
2020-10-09 14:14:55 ERROR : lai/core1ha/20180706/DSC00193_G1.JPG: Failed to copy: Update mkParentDir failed: Could not create directory /Shared/Folder_1/lai/core1ha: Sabre\DAV\Exception\Forbidden: 403 Forbidden
2020-10-09 14:14:56 ERROR : lai/core1ha/20180706/DSC00182_F2.JPG: Failed to copy: Update mkParentDir failed: Could not create directory /Shared/Folder_1/lai/core1ha: Sabre\DAV\Exception\Forbidden: 403 Forbidden
2020-10-09 14:15:46 ERROR : lai/core1ha/20190807/DHP LAI Boyagin 07082019.docx: Failed to copy: Update mkParentDir failed: Could not create directory /Shared/Folder_1/lai/core1ha/20190807: Sabre\DAV\Exception\Forbidden: 403 Forbidden
Transferred:   	    1.941G / 4.358 GBytes, 45%, 6.073 MBytes/s, ETA 6m47s
Errors:                 9 (retrying may help)
Transferred:          231 / 637, 36%
Elapsed time:      5m27.2s
Transferring:
 *              lai/core1ha/20180222/index 2.JPG:100% /7.906M, 1008.665k/s, 0s
 *             lai/core1ha/20180706/DSC00195.JPG:100% /5.250M, 0/s, 0s
 <...and some more lines that have been taken out...>
 *       lai/core1ha/20180222/DSC00153_R6_T9.JPG:100% /8.281M, 112.646k/s, 0s
 * lai/core1ha/20180222/D…Boyagin_22_2_2018.docx:100% /228.990k, 28.684k/s, 0s
 * lai/core1ha/20190510/D… Boyagin 10052019.docx:100% /195.322k, 48.850k

Rclone in finished

Once the cloning is finished, the final result of the cloning will be shown.
In this case, there were some initial errors with creating directories and files, which were resolved after retrying the command. Rclone will retry the attempt 3 times to progress the copy.

...

Code Block
breakoutModefull-width
2020-10-09 14:31:33 ERROR : lai/core1ha/20190807/DSC00569.JPG: Failed to copy: Update mkParentDir failed: Parent node does not exist: Sabre\DAV\Exception\Conflict: 409 Conflict
2020-10-09 14:31:33 ERROR : lai/core1ha/20190807/DSC00547.JPG: Failed to copy: Update mkParentDir failed: Parent node does not exist: Sabre\DAV\Exception\Conflict: 409 Conflict
2020-10-09 14:31:33 ERROR : lai/core1ha/20190807/DSC00552.JPG: Failed to copy: Update mkParentDir failed: Parent node does not exist: Sabre\DAV\Exception\Conflict: 409 Conflict
2020-10-09 14:31:33 ERROR : lai/core1ha/20190807/DSC00577.JPG: Failed to copy: Update mkParentDir failed: Parent node does not exist: Sabre\DAV\Exception\Conflict: 409 Conflict
2020-10-09 14:31:33 ERROR : lai/core1ha/20190807/DSC00546.JPG: Failed to copy: Update mkParentDir failed: Parent node does not exist: Sabre\DAV\Exception\Conflict: 409 Conflict
2020-10-09 14:31:33 ERROR : lai/core1ha/20190807/DSC00573.JPG: Failed to copy: Update mkParentDir failed: Parent node does not exist: Sabre\DAV\Exception\Conflict: 409 Conflict
2020-10-09 14:31:33 ERROR : lai/core1ha/20190807/DSC00571.JPG: Failed to copy: Update mkParentDir failed: Parent node does not exist: Sabre\DAV\Exception\Conflict: 409 Conflict
2020-10-09 14:31:34 ERROR : lai/core1ha/20190807/DSC00554.JPG: Failed to copy: Update mkParentDir failed: Parent node does not exist: Sabre\DAV\Exception\Conflict: 409 Conflict
2020-10-09 14:31:34 ERROR : lai/core1ha/20190807/DSC00570.JPG: Failed to copy: Update mkParentDir failed: Parent node does not exist: Sabre\DAV\Exception\Conflict: 409 Conflict
2020-10-09 14:31:34 ERROR : lai/core1ha/20190807/DSC00568.JPG: Failed to copy: Update mkParentDir failed: Could not create directory /Shared/Folder_1/lai/core1ha: Sabre\DAV\Exception\Forbidden: 403 Forbidden
2020-10-09 14:31:34 ERROR : lai/core1ha/20190807/DSC00557.JPG: Failed to copy: Update mkParentDir failed: Could not create directory /Shared/Folder_1/lai/core1ha/20190807: Sabre\DAV\Exception\Forbidden: 403 Forbidden
2020-10-09 14:44:46 ERROR : Attempt 1/3 failed with 11 errors and: Update mkParentDir failed: Could not create directory /Shared/Folder_1/lai/core1ha/20190807: Sabre\DAV\Exception\Forbidden: 403 Forbidden
2020-10-09 14:44:52 NOTICE: webdav root 'Shared/Folder_1': --checksum is in use but the source and destination have no hashes in common; falling back to --size-only
2020-10-09 14:49:59 ERROR : Attempt 2/3 succeeded
Transferred:   	    4.477G / 4.477 GBytes, 100%, 4.153 MBytes/s, ETA 0s
Checks:               635 / 635, 100%
Transferred:          646 / 646, 100%
Elapsed time:     18m24.0s

Rclone synchronisation

Once the initial upload is executed, every sequential Rclone call will only copy the files that were added or modified after the last Rclone run. With no changes to the directory, the outcome will be:

...