Rsync with the -a option is meant to preserve as much as possible.
this post was submitted on 22 Jul 2024
10 points (91.7% liked)
Linux
48323 readers
838 users here now
From Wikipedia, the free encyclopedia
Linux is a family of open source Unix-like operating systems based on the Linux kernel, an operating system kernel first released on September 17, 1991 by Linus Torvalds. Linux is typically packaged in a Linux distribution (or distro for short).
Distributions include the Linux kernel and supporting system software and libraries, many of which are provided by the GNU Project. Many Linux distributions use the word "Linux" in their name, but the Free Software Foundation uses the name GNU/Linux to emphasize the importance of GNU software, causing some controversy.
Rules
- Posts must be relevant to operating systems running the Linux kernel. GNU/Linux or otherwise.
- No misinformation
- No NSFW content
- No hate speech, bigotry, etc
Related Communities
Community icon by Alpár-Etele Méder, licensed under CC BY 3.0
founded 5 years ago
MODERATORS
Thanks for the suggestion. In fact I tried rsync and it works. But is it possible to integrate in my current workflow? Maybe copying/moving files using a file manager?
I'm asking because with the 3 options I mentioned I may, for example, create mount points in fstab and from this there on everything would be transparent to the user. Would it be possible using rsync?
Secure file transfers frequently trade off some performance for their crypto. You can't have it both ways. (Well, you can but you'd need hardware crypto offload or end to end MACSEC, where both are more exotic use cases)
rsync is basically a copy command with a lot of knobs and stream optimization. It also happens to be able to invoke SSH to pipeline encrypted data over the network at the cost of using ssh for encrypting the stream.
Your other two options are faster because of write-behind caching in to protocol and transfer in the clear-- you don't bog down the stream with crypto overhead, but you're also exposing your payload
File managers are probably the slowest of your options because they're a feature of the DE, and there are more layers of calls between your client and the data stream. Plus, it's probably leveraging one of NFS, Samba or SSHFS anyway.
I believe "rsync -e ssh" is going to be your best over all case for secure, fast, and xattrs. SCP might be a close second. SSHFS is a userland application, and might suffer some penalties for it
I'll take a closer look into rsync possibilities and see if it applies to my situation. I appreciate your input.
How much delay could you live with between syncs? If it's not important to be immidiate, just an end-of-the-day thing you could cronjob the rync with the update flag every so often.
You didn't mention rsync, which I think is usually considered standard. I'd look into that.
I assume you don't intend to copy the files but use them from a remote host? As security is a concern I suppose we're talking about traffic over the public network where (if I'm not mistaken) kerberos with NFS doesn't provide encryption, only authentication. You obviously can tunnel NFS with SSH or VPN and I'm pretty sure you can create a kerberos ticket which stores credentials locally for longer periods of time and/or read them from a file.
SSH/VPN obviously causes some overhead, but they also provide encryption over the public network. If this is something ran in a LAN I wouldn't worry too much about encrypting the traffic and in my own network I wouldn't worry about authentication either too much. Maybe separate the NFS server to it's own VLAN or firewall it heavily.
"rsync -X"
forces me to reenter the credentials frequently
Can you explain what your need is for copying files this frequently? Is this for backups? Do you always want the two sides to stay in sync? If so, something like a distributed filesystem such as gluster/ceph/etc. might work better for you.
I appreciate your help, but notice that the article just tell some basics of xattrs usage (I already know how to use it), but has no reference of file transfering files, which is what I need.