Let's talk Music Video file storage ??

[KevWind]

Some history::: With the advent of the Music Video almost becoming ubiquitous as a release format particularly YouTube

While I did learn about and start doing some few music video's about 8 to 10 years ago I did so infrequently such that storage was never really and issue.
Particularly because my old Mac Pro Tower had 2 TB for video files and 2 TB for audio only files

Having gone to a new iMac in late 2020 I thought I had "splurged" and ordered it 2 TB SDD drive

But since doing more video's and more multi camera etc. I got a warning last week I was running short on internal drive storage ......
I assume from having kept all the raw footage for every project and the edited files as well I google and realized I had to purge something so I did ::: BUT

Because of having Thunderbolt 3 connection ability and the intention of also doing more music videos I have also considered looking into external SSD storage


I have been looking at one OWC solution with a OWC Express 4M2 4-Slot NVMe M.2 SSD Enclosure with SoftRAID. and 4 2.0TB OWC Aura P12 Pro NVMe M.2 2280 SSD.

But I am curious what those of doing a lot of video are doing for extra storage , and how ?

[keith.rogers]

I have a pair of 3TB spinners mirrored at the end of a 5m USB cable stuck in a closet. (SSDs in that size are still too expensive [for me] and you might still want a fan on a couple in a small case if you spend that kind of money.)

I’m using Final Cut Pro and it’s still necessary to regularly clean up project libraries to delete optimized/proxy media generated during editing on projects that aren’t being worked on. Video goes through memory like, I don’t know, pick your favorite metaphor….

[KevWind]

Originally Posted by keith.rogers

I have a pair of 3TB spinners mirrored at the end of a 5m USB cable stuck in a closet. (SSDs in that size are still too expensive [for me] and you might still want a fan on a couple in a small case if you spend that kind of money.)

I’m using Final Cut Pro and it’s still necessary to regularly clean up project libraries to delete optimized/proxy media generated during editing on projects that aren’t being worked on. Video goes through memory like, I don’t know, pick your favorite metaphor….

Like Hot Butter through a knife

Interesting so the optimized/proxy media ? I don't think I have been doing that (isn't that selection you have to make ?) and I am not clear on why I might want to do that , other than it is smaller files ? But does that effect (compress) the resolution of the final FCPX file to upload to Youtube ?
Also when I went to purge I did notice many projects had multiple "library" icons ? But not sure why .... but I went through and deleted all but the last one (Is that good practice ?)

[Chipotle]

I'm on Windows so can't speak to Mac-specific drive options. I'm not to that point yet, but I would just add a big external drive to archive completed projects. Keep the current stuff on the fast internal drives, and move finished stuff to the external. Then the speed of the external drives isn't as important and you can save a bit of money or buy more storage for the same dollars (or go with a RAID setup in case of a failure).

[keith.rogers]

Originally Posted by KevWind

Like Hot Butter through a knife

Interesting so the optimized/proxy media ? I don't think I have been doing that (isn't that selection you have to make ?) and I am not clear on why I might want to do that , other than it is smaller files ? But does that effect (compress) the resolution of the final FCPX file to upload to Youtube ?
Also when I went to purge I did notice many projects had multiple "library" icons ? But not sure why .... but I went through and deleted all but the last one (Is that good practice ?)

For editing in FCP, you're not actually editing your 1080 or greater footage - chances are the CPU, GPU and/or storage would
choke, so they create an optimized, version of the video. And, since it's likely your originals are actually compressed, the "optimized" stuff is huge, i.e., its not something that's de-compressed/compressed as it's being worked on (though it might be a lower res that's upscaled in the GPU or hardware - dunno).

Deleting the proxy/optimized stuff once you are done doesn't matter, but if you go back in to edit the project, it will be regenerated, which may slow down editing/sharing for a little right at the start.

The resolution uploaded to YouTube is based on the project settings, or something you choose in that Share option, I suppose. Anymore, I always "Share to Apple Devices ..." and inspect/view the file itself, and then upload. That lets me see/hear what happens in each of the compressions, first by FCP, then by YT, that occur.

Sometime later I'll have to walk upstairs and open FCP to see if I have multiple library icons. It may have something to do with sharing/copying bits from one to another, like I might have done with some title or credits templates, solids, etc. (I don't have but one library on the install on the old Mini I'm on now.

I think this is a link I tried to puzzle through when I started seeing what I thought was "unlimited" storage with those 3TB drives disappear.

https://support.apple.com/guide/fina...ver4e2e1bd/mac

p.s. I have not seen a problem running projects off the external drives, though they are not super fast (likely cable-length-limited), but I don't have anything over 1080p. Another reason I am not rushing to 4k...

[hubcapsc]

My last mechanical disk drive finally froze up, I'm sorry I
kept those backups on it for so long.

Nothing lasts forever, but I'd only use SSD external disks
now, until the next better thing comes along...

-Mike

[jim1960]

I don't do video but I do have a pretty large library of virtual instruments so I needed a storage solution early on as I didn't want to put all that (and it wouldn't fit anyway) on my main drive. I bought an Akitio Thunder3 Quad Mini. It has four bays for holding SSDs and connects to the computer via TB3. Very fast and works flawlessly.

[Doug Young]

Yeah, in Final Cut, the project file gets huge, and editing also seems to make the project grow. I'm not sure what all they're storing, but it helps to periodically clean up with the Delete Generated Files command. I have been working in 4K and I've seen projects grow to 1TB, only to go back down to a few hundred G after cleanup.

I work with an OWC 2TB Accelsior SSD to provide fast editing and rendering, but I can't have more than 2 projects typically at a time on that. Once I'm done and have released a video, I do the clean up, get rid of everything that isn't needed to recreate it, and move everything to a 8TB external - I have a bunch of traditional drives in a thunderbolt cage. Now and then I archive older stuff off to a removable archive drive and store it in the closet. I also sometimes decide I'd never need those projects again and just get rid of them. Ideally, I'd keep everything - I just had AG hit me up for video from interviews I did 10 years ago, but both the need for disk space and all the overhead of keeping track of old files makes me be a bit more blase about it.

I'm not sure if what I have is the same as what you're looking at Kev, since I have the Mac Pro, mine's an internal PCI gizmo, but it seems like the same idea, just via thunderbolt. But I will say that mine's really fast!. I forget exact numbers, but I seem to recall it was 3-4X the read and write speed of the Mac's own SSD, and the difference between it and a regular drive was astonishing. The SoftRaid can be a bit annoying (updates, sometimes paid, occasionally requires reboots..) but mostly it just works.

[KevWind]

Originally Posted by jim1960

I don't do video but I do have a pretty large library of virtual instruments so I needed a storage solution early on as I didn't want to put all that (and it wouldn't fit anyway) on my main drive. I bought an Akitio Thunder3 Quad Mini. It has four bays for holding SSDs and connects to the computer via TB3. Very fast and works flawlessly.

Thanks for the heads up .Just checked looks like that one is for SATA drives
Since I already have two M.2 NVMe PCIe 1 TB SSD drives from my old Mac Pro. I am looking for a M.2 NVMe SSD enclosure and why I was looking at the one I mentioned in the op with 4 slots so I will get 2 more 2 TB drives

[KevWind]

Originally Posted by keith.rogers

For editing in FCP, you're not actually editing your 1080 or greater footage - chances are the CPU, GPU and/or storage would
choke, so they create an optimized, version of the video. And, since it's likely your originals are actually compressed, the "optimized" stuff is huge, i.e., its not something that's de-compressed/compressed as it's being worked on (though it might be a lower res that's upscaled in the GPU or hardware - dunno).

Deleting the proxy/optimized stuff once you are done doesn't matter, but if you go back in to edit the project, it will be regenerated, which may slow down editing/sharing for a little right at the start.

The resolution uploaded to YouTube is based on the project settings, or something you choose in that Share option, I suppose. Anymore, I always "Share to Apple Devices ..." and inspect/view the file itself, and then upload. That lets me see/hear what happens in each of the compressions, first by FCP, then by YT, that occur.

Sometime later I'll have to walk upstairs and open FCP to see if I have multiple library icons. It may have something to do with sharing/copying bits from one to another, like I might have done with some title or credits templates, solids, etc. (I don't have but one library on the install on the old Mini I'm on now.

I think this is a link I tried to puzzle through when I started seeing what I thought was "unlimited" storage with those 3TB drives disappear.

https://support.apple.com/guide/fina...ver4e2e1bd/mac

p.s. I have not seen a problem running projects off the external drives, though they are not super fast (likely cable-length-limited), but I don't have anything over 1080p. Another reason I am not rushing to 4k...

Thanks. So What I have been doing lately is using my new iMac camera at 1080 in the Photo Boot app (video) as well for multi cam my iPhone The iMac footage from Photo booth I export to my desktop via Quicktime and then import into FCPX. I do notice when I do put any of those clips into the time line there is some kind of conversion.

[KevWind]

Originally Posted by Doug Young

Yeah, in Final Cut, the project file gets huge, and editing also seems to make the project grow. I'm not sure what all they're storing, but it helps to periodically clean up with the Delete Generated Files command. I have been working in 4K and I've seen projects grow to 1TB, only to go back down to a few hundred G after cleanup.

I work with an OWC 2TB Accelsior SSD to provide fast editing and rendering, but I can't have more than 2 projects typically at a time on that. Once I'm done and have released a video, I do the clean up, get rid of everything that isn't needed to recreate it, and move everything to a 8TB external - I have a bunch of traditional drives in a thunderbolt cage. Now and then I archive older stuff off to a removable archive drive and store it in the closet. I also sometimes decide I'd never need those projects again and just get rid of them. Ideally, I'd keep everything - I just had AG hit me up for video from interviews I did 10 years ago, but both the need for disk space and all the overhead of keeping track of old files makes me be a bit more blase about it.

I'm not sure if what I have is the same as what you're looking at Kev, since I have the Mac Pro, mine's an internal PCI gizmo, but it seems like the same idea, just via thunderbolt. But I will say that mine's really fast!. I forget exact numbers, but I seem to recall it was 3-4X the read and write speed of the Mac's own SSD, and the difference between it and a regular drive was astonishing. The SoftRaid can be a bit annoying (updates, sometimes paid, occasionally requires reboots..) but mostly it just works.

"Delete Generated Files" is that a command in FCPX ?

Yes basically the same but external ,,,as I told Jim I have two left over 1 TB M.2 NVMe PCIe 1 TB SSD and why I am looking at this 4 slot enclosure (PCIe type) https://eshop.macsales.com/item/OWC/TB3EX4M2SL/
And I will also get 2 more 2-TB Drives for a total of 6TB in the enclosure plus about 1.4 usable now on my iMac and about 700+ GB on my new Macbook Pro M1 Pro / with a 1 TB drive
My plan is to use the new MBP for FCPX editing assuming it will be faster than my intel i7 iMac

What I am uncertain about is how to use the RAID configuration, but will call OWC on Monday
What I don't know is if the computer will see the 4 drives in the enclosure as 4 different drives, or as on big one ,
On my old Mac Pro the two SSD drives (even though on one Dual PCIe drive adaptor sled) appeared as two different drives.

[endpin]

I have been using the Samsung T7 External SSDs for FCPX projects connected to a MacBookAir M1 via USB-C to USB-C connections and ultimately plan to junk all my big disk arrays as video storage.

I don't use the internal SSD for ANYTHING except the applications and system stuff.
I delete the generated render files to archive the projects.

[Doug Young]

Originally Posted by KevWind

"Delete Generated Files" is that a command in FCPX ?

Yes:

Screen Shot 2022-01-22 at 11.53.22 AM.jpg

This discussion reminded me I hadn't done this stuff for the last video I posted, so I just did the cleanup. Project was 781G, after removing a few clips I didn't use and issuing the "delete" command, it's 125G, now moved to my archive.

What I am uncertain about is how to use the RAID configuration, but will call OWC on Monday
What I don't know is if the computer will see the 4 drives in the enclosure as 4 different drives, or as on big one ,
On my old Mac Pro the two SSD drives (even though on one Dual PCIe drive adaptor sled) appeared as two different drives.

For speed, I'm using Raid 0, which is what the OWC enclosure expects - to meet their claimed speed specs. So it appears as one drive, but data can be written to multiple SSDs in parallel. You could use other raid configs if you want redundancy.

It was pretty easy to install and setup if I recall.

[KevWind]

Thanks all very informative

[jim1960]

Originally Posted by KevWind

Thanks for the heads up .Just checked looks like that one is for SATA drives
Since I already have two M.2 NVMe PCIe 1 TB SSD drives from my old Mac Pro. I am looking for a M.2 NVMe SSD enclosure and why I was looking at the one I mentioned in the op with 4 slots so I will get 2 more 2 TB drives

I was just making the point that these things work well with no noticeable loss in performance. Grab that OWC enclosure. They're a good company and the stuff they make is well regarded.