To sync a SharePoint Document Library safely using rsync, you must first bridge the gap between SharePoint’s cloud API and your local file system because rsync cannot natively communicate with Microsoft 365 cloud protocols. Running a direct sync blindly can result in catastrophic data loss, accidental file deletions, or API rate-limiting blocks.
Here is the comprehensive guide to setting up and executing a safe sync workflow. Step 1: Establish a Safe Local Mount (The Bridge)
Do not try to pass raw SharePoint web URLs into rsync. Instead, map the SharePoint Document Library locally using one of these secure methods:
For Windows/macOS: Use the official Microsoft OneDrive Sync Client. Navigate to your SharePoint site, click Sync, and let it map to your local File Explorer or Mac Finder. Crucial: Right-click the folder locally and select Always keep on this device to download the physical files. If you leave “Files On-Demand” active (cloud-only icons), rsync will force a massive, broken download loop or skip files entirely.
For Linux Servers: Use a production-grade open-source client like onedrive (the abraunegg client) or rclone. Map your SharePoint site using secure Microsoft OAuth2 authentication. Step 2: Formulate the Safe Rsync Command
When pushing local files into the SharePoint mount (or pulling backups from it), your rsync command flags must be explicitly tailored to protect cloud metadata. Use this foundational syntax for safety:
rsync -rtv –size-only –inplace –dry-run /path/to/source/ /path/to/sharepoint_mount/ Use code with caution. Breakdown of Critical Safety Flags:
-r (Recursive): Copies directories and subdirectories safely. Avoid using -a (Archive mode). -a attempts to sync Linux permissions, groups, and owners, which SharePoint’s NTFS/cloud file system rejects, throwing continuous errors.
-t (Times): Preserves modification times. This is vital so SharePoint knows which file is truly newer.
–size-only: Forces rsync to compare files strictly by file size. SharePoint constantly updates minor cloud-side timestamps and metadata on files; relying on standard timestamp checks will cause rsync to unnecessarily re-upload unmodified files.
–inplace (Optional but recommended): Updates destination files directly rather than creating a temporary file and renaming it. This prevents the SharePoint sync engine from treating updates as entirely “new” file creations. Step 3: Implement Core Safety Protections 1. Always Run a Dry Run First
Before committing to any transfer, append -n or –dry-run to your command. rsync -rtv –size-only –dry-run /source/ /destination/ Use code with caution.
Review the terminal output carefully. Ensure it is only modifying the files you expect and is not flagging thousands of items for accidental deletion. 2. Handle the –delete Flag with Extreme Caution
If you need the destination to exactly mirror the source by removing old files, you may add –delete. However: Never use –delete without a preceding –dry-run.
If a local folder accidentally mounts empty due to a network glitch, rsync –delete will view the source as empty and wipe out your entire remote SharePoint library. 3. Throttle Your Transfer Speed
SharePoint Online enforces strict API rate limits. If you upload thousands of files at maximum speed, Microsoft will throttle your IP address.
Use the –bwlimit=KBYTES flag (e.g., –bwlimit=5000 to cap speeds at 5MB/s) to keep a steady, low-profile upload that won’t trigger security blocks. Alternative Production Option: Rclone Sync
If you are operating in a command-line or Linux server environment, cloud engineers heavily favor rclone over standard rsync for Microsoft 365. rclone natively speaks the SharePoint Graph API, handles multi-thread tokens natively, and bypasses local caching entirely. A safe rclone sequence mirrors rsync: rclone copy /local/dir sharepoint:remote_dir –dry-run -vv Use code with caution.
To help narrow down the safest implementation for your environment, please let me know:
What Operating System is running your rsync environment (Linux, Windows/WSL, or macOS)?
Are you using rsync to back up data from SharePoint onto local storage, or to upload local files into SharePoint?
Roughly how many files or how much data (GB/TB) are in this sync list? Sync SharePoint files and folders – Microsoft Support
Leave a Reply