this post was submitted on 31 Jan 2024

47 points (92.7% liked)

Linux

47940 readers

1551 users here now

From Wikipedia, the free encyclopedia

Linux is a family of open source Unix-like operating systems based on the Linux kernel, an operating system kernel first released on September 17, 1991 by Linus Torvalds. Linux is typically packaged in a Linux distribution (or distro for short).

Distributions include the Linux kernel and supporting system software and libraries, many of which are provided by the GNU Project. Many Linux distributions use the word "Linux" in their name, but the Free Software Foundation uses the name GNU/Linux to emphasize the importance of GNU software, causing some controversy.

Rules

Posts must be relevant to operating systems running the Linux kernel. GNU/Linux or otherwise.
No misinformation
No NSFW content
No hate speech, bigotry, etc

Related Communities

Community icon by Alpár-Etele Méder, licensed under CC BY 3.0

founded 5 years ago

MODERATORS

[email protected]

Best way to make an external usb HDD that is compatible with windows and linux? (discuss.tchncs.de)

submitted 9 months ago* (last edited 9 months ago) by [email protected] to c/[email protected]

33 comments fedilink hide all child comments

Ok i have a 5TB usb hard drive that is mostly used for weekly backups of my system. It is hooked to a raspberry pi and has an exfat file system and used as a samba share.

But i need to occasionally unplug it and connect it directly to some other machines for 2 reasons.

For use with steam on linux (i store my games on it)
For use with obs on windows (i do some screen recording and need to be able to save the output to the drive)

Here's the dilemma. Games that require proton will not launch from an exfat drive (trust me i've tried all the "hacks" and cant make it happen). So i plan to reformat the drive as an ext4 file system.

However, windows does not support ext4 natively. I do have WSL and could probably mount it there but here are the issues.

Another user in my house will be primarily the one using obs and they are not technically inclined (or at least not linux inclined) i want them to be able to plug this device in and just press record and have it output to the drive, this was working fine with exfat but obviously once i reformat it could be an issue.

I have a few options and wanted to run it by here first before committing to the reformat.

Option 1) reformat drive to ext4. Keep it as a samba drive at all times, except when gaming, and have obs output the video to the drive over the network

-is obs capable of doing this reliably?

Option 2) reformat to ext4 and unplug drive when gaming or needing to use obs. Rig up a script to automount the drive with wsl when detected.

-will the mounted drive show up in the windows file explorer natively? Or could i at least pin its location to quick access?
-can i actually create some way for it to automount the drive with no user input required?
-ooh but the distro is on the drive to conserve the laptop's space, could present an issue

Option 3) reformat the drive as part ntfs part ext4 and then plug it in for gaming and obs?

-i believe in linux i could just add the part-uuid to fstab so it only mounts the ext4 portion or maybe i could mount both?
-would windows automatically just mount the ntfs portion?

Imo, option 1 is the best but i would like to have a fail safe in case i ever need to plug it directly to the windows machine. So realistically a hybrid of option 1 and 2 is best. Is there a better way to do this? Will any of these methods work?

all 35 comments

sorted by: hot top controversial new old

[–] [email protected] 54 points 9 months ago (3 children)

Just use NTFS for the whole drive. The new NTFS3 kernel driver in Linux has fairly decent performance (waay better than the old ntfs-3g driver), and has been pretty stable since kernel 6.2.

The key thing you'd want to do though is to use the mount options nocase and windows_names in your fstab.

nocase enables case-insensitive file/folder support similar to the default behavior under Windows with NTFS volumes.
windows_names prevents the creation of files or directories with names not allowed under Windows. This checks for forbidden characters in the name like /, , :, *, ?, <, >, |, " or ending with a space or a period. There are also other checks for matching the behavior of Windows with this mount option for rejecting file/folder names that may be valid on Linux systems but not under Windows.

These two mount options should solve most of the issues Linux NTFS users may face. Do note that to use the above mount options, you'll need to be on at least kernel 6.2.

[–] [email protected] 7 points 9 months ago (1 children)

I got 2 questions. How do permissions work with ntfs on linux. I use rsync a lot to backup to the drive and i've read how exfat doesnt really save all the linux permissions. Likewise from my research in order to run a proton game on a ntfs drive would require symlinking rhe compat directory to my home folder. And i use the home folder for smaller games. I have heard that it may or may not work with ntfs

[–] [email protected] 6 points 9 months ago

The permissions should be mapped to your Linux user account (whilst maintaining any existing NTFS permissions). So by default the drive would be mounted with uid=1000, gid=1000 and umask=0022 - which should result in 0755 for directories and 0644 for files. In plain English, that means you shouldn't have any permission issues.

But yeah I believe the compatdata symlink is still required as wine/proton creates files with :, which won't be allowed if you mount the drive with windows_names.

[–] [email protected] 5 points 9 months ago

Yeah, I'd probably go with NTFS in OP's situation, provided the steam games work with that setup. Back when I needed compatibility between windows and Linux I did it that way, but performance wasn't really a consideration then.

[–] [email protected] 4 points 9 months ago

I second the recommendation to use NTFS. I don't have the same use cases as OP, but in my experience it works really well. Back in the days when I was using Windows, I had a system and a data partition (i.e. personal files, pictures, videos... you get it). When I switched to Linux, I kept my data partition and just mounted it on my Linux system. I started with dual boot and didn't have any issues. No need to manually install a NTFS driver these days.

That's a couple of years ago and my secondary SSD's still that same old NTFS partition. Thought about moving to a Linux native filesystem, since I don't use Windows anymore, but never had an actual reason to do it.

[–] [email protected] 8 points 9 months ago* (last edited 9 months ago) (2 children)

NTFS will give you issues just like exfat. You can partition it as multiple partitions. 1 ntfs, 1 ext4. Or I personally for Linux have been using btrfs.

There is a btrfs driver for windows, but I don't know much about ita use cases.

[–] [email protected] 2 points 9 months ago

Hmm i dont have any experience with btrfs but im open to it.

Is speed comparable to ext4? And should i install the driver on windows i could just plug the drive in an it would become accessible without user input?

[–] [email protected] 2 points 9 months ago (1 children)

The windows btrfs driver can be a mixed bag. Don't unplug it suddenly and make backups.

[–] [email protected] 2 points 9 months ago (1 children)

That's disappointing, but mostly i would be plugging it in to windows every once in a while. Whereas i would be running nightly backups when its hooked as a samba drive. Would i need to eject it every time i unplugged? I wouldnt mind that

[–] [email protected] 3 points 9 months ago

I would eject or else you likely will get a blue screen. Even if you do eject there is likely a bunch of bugs.

If it does work for you make sure you donate to the Dev. He deals will a lot of harassment on Github and does the good work.

[–] [email protected] 5 points 9 months ago* (last edited 9 months ago) (1 children)

Damn, I just formatted a new SSD with exfat for a shared Steam library across Windows and Linux PCs. I haven't noticed any issues yet, but thanks for the topic. I might just end up switching to BTRFS.

[–] [email protected] 5 points 9 months ago* (last edited 9 months ago) (3 children)

For use with steam on linux (i store my games on it)

Have you actually played a game from a mechanical HDD yet? I’m asking because 99% of “freezing” troubleshooting/help posts in gaming subs that I follow on that other site that shall not be named are because the OP stores their games on a mechanical HDD instead of a SSD, so if you’ve been doing this, I’m curious how your experience has been so far.

If you have a little extra cash to burn, I’d recommend springing for an extra 256 GB SSD or something, which can be had for $30 on Amazon in the US (not sure about your country).

EDIT:

For use with obs on windows (i do some screen recording and need to be able to save the output to the drive)

Now that I think about it, I’d be shocked if recording video to a mechanical HDD went smoothly as well. If my prediction is correct, then you might consider first recording the session to the OS drive (assuming it’s a SSD), and then copying the file to the HDD for long term storage after you finish recording.

[–] [email protected] 3 points 9 months ago

I used and use for gaming and OBS mechanical hard drives - no issues with them.

I won't run OS from them anymore, but gaming and recording full hd live is not a problem.

maybe the OS doing some buffering to SSD, idk, but as NOT the only drive, they are still pretty nice.

[–] [email protected] 2 points 9 months ago* (last edited 9 months ago)

I played many games that didn't fit onto either of my SSDs, on an HDD, including Destiny 2 and for a while Baldur's Gate 3 - they had ~70% longer loading times, but they didn't freeze outside of loading screens.

As for recording, you'd be amazed at how filesystem caches are good at what they do

[–] [email protected] 2 points 9 months ago

So i mostly play older games (half life, skyrim, dishonored, etc) i have no issues with the mechanical drive for these. Now admittedly the larger games like skyrim i havent tried on linux only on windows but i dont see why it would be an issue.

I will probably eventually get a ssd for gaming and make the hdd purely for storage.

I have already recorded to this drive from the windows device with no issues. It is not intended to be super high quality tho. I just dont know if running over the network will go as smooth.

Rn, im leaning towards using btrfs bc it seems the performance speed between it and ext4 will be negligible and i will be testing out the driver for windows and obs over the network.

[–] [email protected] 4 points 9 months ago (1 children)

I just use BTRFS. There's an open source driver for Windows you can install.

[–] [email protected] 1 points 9 months ago (1 children)

I'm leaning toward btrfs. Gonna set it up tonight and test. Is there any major differences from ext4 i should know? I know it has snapshots but i read about if you dont use the right kind of drives it can corrupt data, is thus true or not really something i should worry about?

[–] [email protected] 2 points 9 months ago (1 children)

It has a lot of differences to ext4 but it's still just a file system. You're not gonna notice any difference just normally using the drive (aside from copying being instant). I've never heard about it causing corruptions on different kinds of drives. I use it on 3 SSDs and one HDD and have no issues.

[–] [email protected] 1 points 9 months ago (1 children)

Ok i'm running into some issues. I have formatted with btrfs but immediately it does not work when attempting to mount.

I had it mounted and then set up with fstab, test installing some games to it but the second i ejected the device and then reattached it it cannot be mounted.

I get the error

An error occurred while accessing 'wd_black5tb', the system responded: The requested operation has failed: Error mounting system-managed device /dev/sdb1: wrong fs type, bad option, bad superblock on /dev/sdb1, missing codepage or helper program, or other error

I also installed winbtrfs with chocolatey but the drive fails to show up once plugged in and i dont see the drive anywhere

[–] [email protected] 1 points 9 months ago (1 children)

It looks like it either didn't get ejected properly and something is corrupt or you're mounting it wrong. It's probably the first tho. BTRFS has very good commandline tools that can help you fix this.

[–] [email protected] 1 points 9 months ago

Yeah it seems like it was a permissions error. I set the tight permissions and now i have it working

[–] [email protected] 4 points 9 months ago (1 children)

the UDF filesystem is supported by both windows and linux, and will have acceptable performance on both unlike the windows BTRFS driver. Unlike exfat, it supports all the funky stuff needed to make steam games work.

[–] [email protected] 1 points 9 months ago (1 children)

I'm not really worried about performance on windows since i will only very occasionally be plugging it in there. Ive never heard of udf but will check it out too

[–] [email protected] 1 points 9 months ago (1 children)

Well Windows just doesnt really support any performant Linux partition formats, afaik

[–] [email protected] 4 points 9 months ago

Except for UDF, as I pointed out in my comment. UDF is easily forgotten because it is kinda weird, but it fits the bill here and has native drivers in Windows, Linux, and MacOS.

[–] [email protected] 3 points 9 months ago* (last edited 9 months ago)

Not sure if you're looking to keep only to foss solutions, but if you're OK with a proprietary driver (and as you're on windows anyway), the ext4 driver for windows from paragon sounds like it would work well in your situation. You can install it on the windows machine and it'll mount ext4 read/write like any other windows-compatible drive, complete with automount. They give you a ten day trial to try it out, so you could test it a bit first to see if it'll work ok with OBS, then it's twenty dollars USD. Not advertising for them, but I just went through this process myself for an ext4 external hdd earlier today and wanted to share. :)

https://www.paragon-software.com/home/linuxfs-windows/#

[–] [email protected] 3 points 9 months ago

Share it over network is the least painful way, imo.

[–] [email protected] 1 points 9 months ago* (last edited 9 months ago)

‎

[–] [email protected] 1 points 9 months ago

Def try 1, obs will write to the file as usual and windows will be responsible for moving data over samba. Really depends on your network and recording settings.

[–] [email protected] -5 points 9 months ago (1 children)

Mount it on a spare PC and share it on the network with samba.

[–] [email protected] 4 points 9 months ago (1 children)

Did you read it at all?

[–] [email protected] 1 points 9 months ago (1 children)

Yes and the answer is you can't do it in anyway that's fool proof that won't eventually result in data loss. Just buy another one and be done with it.

[–] [email protected] 1 points 9 months ago

I probably will eventually. I put btrfs on it and it works pretty well. Im a little iffy on the windows driver but i tested obs and it can write over the network just fine.