The Acoustic Guitar Forum

Go Back   The Acoustic Guitar Forum > General Acoustic Guitar and Amplification Discussion > RECORD

Reply
 
Thread Tools
  #1  
Old 01-22-2022, 08:36 AM
KevWind's Avatar
KevWind KevWind is offline
Charter Member
 
Join Date: Apr 2008
Location: Edge of Wilderness Wyoming
Posts: 16,863
Default Let's talk Music Video file storage ??

Some history::: With the advent of the Music Video almost becoming ubiquitous as a release format particularly YouTube

While I did learn about and start doing some few music video's about 8 to 10 years ago I did so infrequently such that storage was never really and issue.
Particularly because my old Mac Pro Tower had 2 TB for video files and 2 TB for audio only files

Having gone to a new iMac in late 2020 I thought I had "splurged" and ordered it 2 TB SDD drive

But since doing more video's and more multi camera etc. I got a warning last week I was running short on internal drive storage ......
I assume from having kept all the raw footage for every project and the edited files as well I google and realized I had to purge something so I did ::: BUT

Because of having Thunderbolt 3 connection ability and the intention of also doing more music videos I have also considered looking into external SSD storage


I have been looking at one OWC solution with a OWC Express 4M2 4-Slot NVMe M.2 SSD Enclosure with SoftRAID. and 4 2.0TB OWC Aura P12 Pro NVMe M.2 2280 SSD.

But I am curious what those of doing a lot of video are doing for extra storage , and how ?
__________________
Enjoy the Journey.... Kev...

KevWind at Soundcloud

KevWind at YouYube
https://www.youtube.com/user/KevWind1/videos

System :
Carbon interface , PT Ultimate 2021.12 -Mid 2020 iMac 27" 3.8GHz 8-core i7 10th Gen ,,128GB 2666MHz DDR4 RAM,,2TB SSD storage,Radeon Pro 5700 XT16GB Catalina 10.15.7
Reply With Quote
  #2  
Old 01-22-2022, 09:50 AM
keith.rogers's Avatar
keith.rogers keith.rogers is offline
Charter Member
 
Join Date: May 2019
Location: Texas
Posts: 1,241
Default

I have a pair of 3TB spinners mirrored at the end of a 5m USB cable stuck in a closet. (SSDs in that size are still too expensive [for me] and you might still want a fan on a couple in a small case if you spend that kind of money.)

Iím using Final Cut Pro and itís still necessary to regularly clean up project libraries to delete optimized/proxy media generated during editing on projects that arenít being worked on. Video goes through memory like, I donít know, pick your favorite metaphorÖ.
__________________
"I know in the morning that it's gonna be good, when I stick out my elbows and they don't bump wood." - Bill Kirchen
Reply With Quote
  #3  
Old 01-22-2022, 10:30 AM
KevWind's Avatar
KevWind KevWind is offline
Charter Member
 
Join Date: Apr 2008
Location: Edge of Wilderness Wyoming
Posts: 16,863
Default

Quote:
Originally Posted by keith.rogers View Post
I have a pair of 3TB spinners mirrored at the end of a 5m USB cable stuck in a closet. (SSDs in that size are still too expensive [for me] and you might still want a fan on a couple in a small case if you spend that kind of money.)

I’m using Final Cut Pro and it’s still necessary to regularly clean up project libraries to delete optimized/proxy media generated during editing on projects that aren’t being worked on. Video goes through memory like, I don’t know, pick your favorite metaphor….
Like Hot Butter through a knife

Interesting so the optimized/proxy media ? I don't think I have been doing that (isn't that selection you have to make ?) and I am not clear on why I might want to do that , other than it is smaller files ? But does that effect (compress) the resolution of the final FCPX file to upload to Youtube ?
Also when I went to purge I did notice many projects had multiple "library" icons ? But not sure why .... but I went through and deleted all but the last one (Is that good practice ?)
__________________
Enjoy the Journey.... Kev...

KevWind at Soundcloud

KevWind at YouYube
https://www.youtube.com/user/KevWind1/videos

System :
Carbon interface , PT Ultimate 2021.12 -Mid 2020 iMac 27" 3.8GHz 8-core i7 10th Gen ,,128GB 2666MHz DDR4 RAM,,2TB SSD storage,Radeon Pro 5700 XT16GB Catalina 10.15.7
Reply With Quote
  #4  
Old 01-22-2022, 11:04 AM
Chipotle Chipotle is offline
Registered User
 
Join Date: Aug 2016
Posts: 1,559
Default

I'm on Windows so can't speak to Mac-specific drive options. I'm not to that point yet, but I would just add a big external drive to archive completed projects. Keep the current stuff on the fast internal drives, and move finished stuff to the external. Then the speed of the external drives isn't as important and you can save a bit of money or buy more storage for the same dollars (or go with a RAID setup in case of a failure).
Reply With Quote
  #5  
Old 01-22-2022, 11:49 AM
keith.rogers's Avatar
keith.rogers keith.rogers is offline
Charter Member
 
Join Date: May 2019
Location: Texas
Posts: 1,241
Default

Quote:
Originally Posted by KevWind View Post
Like Hot Butter through a knife

Interesting so the optimized/proxy media ? I don't think I have been doing that (isn't that selection you have to make ?) and I am not clear on why I might want to do that , other than it is smaller files ? But does that effect (compress) the resolution of the final FCPX file to upload to Youtube ?
Also when I went to purge I did notice many projects had multiple "library" icons ? But not sure why .... but I went through and deleted all but the last one (Is that good practice ?)
For editing in FCP, you're not actually editing your 1080 or greater footage - chances are the CPU, GPU and/or storage would
choke, so they create an optimized, version of the video. And, since it's likely your originals are actually compressed, the "optimized" stuff is huge, i.e., its not something that's de-compressed/compressed as it's being worked on (though it might be a lower res that's upscaled in the GPU or hardware - dunno).

Deleting the proxy/optimized stuff once you are done doesn't matter, but if you go back in to edit the project, it will be regenerated, which may slow down editing/sharing for a little right at the start.

The resolution uploaded to YouTube is based on the project settings, or something you choose in that Share option, I suppose. Anymore, I always "Share to Apple Devices ..." and inspect/view the file itself, and then upload. That lets me see/hear what happens in each of the compressions, first by FCP, then by YT, that occur.

Sometime later I'll have to walk upstairs and open FCP to see if I have multiple library icons. It may have something to do with sharing/copying bits from one to another, like I might have done with some title or credits templates, solids, etc. (I don't have but one library on the install on the old Mini I'm on now.

I think this is a link I tried to puzzle through when I started seeing what I thought was "unlimited" storage with those 3TB drives disappear.

https://support.apple.com/guide/fina...ver4e2e1bd/mac

p.s. I have not seen a problem running projects off the external drives, though they are not super fast (likely cable-length-limited), but I don't have anything over 1080p. Another reason I am not rushing to 4k...
__________________
"I know in the morning that it's gonna be good, when I stick out my elbows and they don't bump wood." - Bill Kirchen
Reply With Quote
  #6  
Old 01-22-2022, 12:27 PM
hubcapsc's Avatar
hubcapsc hubcapsc is offline
Charter Member
 
Join Date: Jan 2020
Location: upstate SC
Posts: 1,860
Default

My last mechanical disk drive finally froze up, I'm sorry I
kept those backups on it for so long.

Nothing lasts forever, but I'd only use SSD external disks
now, until the next better thing comes along...

-Mike
Reply With Quote
  #7  
Old 01-22-2022, 12:37 PM
jim1960 jim1960 is offline
Charter Member
 
Join Date: Sep 2015
Posts: 4,570
Default

I don't do video but I do have a pretty large library of virtual instruments so I needed a storage solution early on as I didn't want to put all that (and it wouldn't fit anyway) on my main drive. I bought an Akitio Thunder3 Quad Mini. It has four bays for holding SSDs and connects to the computer via TB3. Very fast and works flawlessly.

__________________
Jim
2017 Circle Strings 00 bastogne walnut/sinker redwood
2015 Circle Strings Parlor shedua/western red cedar
2009 Bamburg JSB Signature Baritone macassar ebony/carpathian spruce
2004 Taylor XXX-RS indian rosewood/sitka spruce
1988 Martin D-16 mahogany/sitka spruce

along with some electrics, zouks, dulcimers, and banjos.

SoundCloud link
Spotify
YouTube
Reply With Quote
  #8  
Old 01-22-2022, 12:41 PM
Doug Young's Avatar
Doug Young Doug Young is offline
AGF Sponsor
 
Join Date: Apr 2005
Location: Mountain View, CA
Posts: 8,879
Default

Yeah, in Final Cut, the project file gets huge, and editing also seems to make the project grow. I'm not sure what all they're storing, but it helps to periodically clean up with the Delete Generated Files command. I have been working in 4K and I've seen projects grow to 1TB, only to go back down to a few hundred G after cleanup.

I work with an OWC 2TB Accelsior SSD to provide fast editing and rendering, but I can't have more than 2 projects typically at a time on that. Once I'm done and have released a video, I do the clean up, get rid of everything that isn't needed to recreate it, and move everything to a 8TB external - I have a bunch of traditional drives in a thunderbolt cage. Now and then I archive older stuff off to a removable archive drive and store it in the closet. I also sometimes decide I'd never need those projects again and just get rid of them. Ideally, I'd keep everything - I just had AG hit me up for video from interviews I did 10 years ago, but both the need for disk space and all the overhead of keeping track of old files makes me be a bit more blase about it.

I'm not sure if what I have is the same as what you're looking at Kev, since I have the Mac Pro, mine's an internal PCI gizmo, but it seems like the same idea, just via thunderbolt. But I will say that mine's really fast!. I forget exact numbers, but I seem to recall it was 3-4X the read and write speed of the Mac's own SSD, and the difference between it and a regular drive was astonishing. The SoftRaid can be a bit annoying (updates, sometimes paid, occasionally requires reboots..) but mostly it just works.
Reply With Quote
  #9  
Old 01-22-2022, 01:05 PM
KevWind's Avatar
KevWind KevWind is offline
Charter Member
 
Join Date: Apr 2008
Location: Edge of Wilderness Wyoming
Posts: 16,863
Default

Quote:
Originally Posted by jim1960 View Post
I don't do video but I do have a pretty large library of virtual instruments so I needed a storage solution early on as I didn't want to put all that (and it wouldn't fit anyway) on my main drive. I bought an Akitio Thunder3 Quad Mini. It has four bays for holding SSDs and connects to the computer via TB3. Very fast and works flawlessly.
Thanks for the heads up .Just checked looks like that one is for SATA drives
Since I already have two M.2 NVMe PCIe 1 TB SSD drives from my old Mac Pro. I am looking for a M.2 NVMe SSD enclosure and why I was looking at the one I mentioned in the op with 4 slots so I will get 2 more 2 TB drives
__________________
Enjoy the Journey.... Kev...

KevWind at Soundcloud

KevWind at YouYube
https://www.youtube.com/user/KevWind1/videos

System :
Carbon interface , PT Ultimate 2021.12 -Mid 2020 iMac 27" 3.8GHz 8-core i7 10th Gen ,,128GB 2666MHz DDR4 RAM,,2TB SSD storage,Radeon Pro 5700 XT16GB Catalina 10.15.7
Reply With Quote
  #10  
Old 01-22-2022, 01:16 PM
KevWind's Avatar
KevWind KevWind is offline
Charter Member
 
Join Date: Apr 2008
Location: Edge of Wilderness Wyoming
Posts: 16,863
Default

Quote:
Originally Posted by keith.rogers View Post
For editing in FCP, you're not actually editing your 1080 or greater footage - chances are the CPU, GPU and/or storage would
choke, so they create an optimized, version of the video. And, since it's likely your originals are actually compressed, the "optimized" stuff is huge, i.e., its not something that's de-compressed/compressed as it's being worked on (though it might be a lower res that's upscaled in the GPU or hardware - dunno).

Deleting the proxy/optimized stuff once you are done doesn't matter, but if you go back in to edit the project, it will be regenerated, which may slow down editing/sharing for a little right at the start.

The resolution uploaded to YouTube is based on the project settings, or something you choose in that Share option, I suppose. Anymore, I always "Share to Apple Devices ..." and inspect/view the file itself, and then upload. That lets me see/hear what happens in each of the compressions, first by FCP, then by YT, that occur.

Sometime later I'll have to walk upstairs and open FCP to see if I have multiple library icons. It may have something to do with sharing/copying bits from one to another, like I might have done with some title or credits templates, solids, etc. (I don't have but one library on the install on the old Mini I'm on now.

I think this is a link I tried to puzzle through when I started seeing what I thought was "unlimited" storage with those 3TB drives disappear.

https://support.apple.com/guide/fina...ver4e2e1bd/mac

p.s. I have not seen a problem running projects off the external drives, though they are not super fast (likely cable-length-limited), but I don't have anything over 1080p. Another reason I am not rushing to 4k...
Thanks. So What I have been doing lately is using my new iMac camera at 1080 in the Photo Boot app (video) as well for multi cam my iPhone The iMac footage from Photo booth I export to my desktop via Quicktime and then import into FCPX. I do notice when I do put any of those clips into the time line there is some kind of conversion.
__________________
Enjoy the Journey.... Kev...

KevWind at Soundcloud

KevWind at YouYube
https://www.youtube.com/user/KevWind1/videos

System :
Carbon interface , PT Ultimate 2021.12 -Mid 2020 iMac 27" 3.8GHz 8-core i7 10th Gen ,,128GB 2666MHz DDR4 RAM,,2TB SSD storage,Radeon Pro 5700 XT16GB Catalina 10.15.7
Reply With Quote
  #11  
Old 01-22-2022, 01:30 PM
KevWind's Avatar
KevWind KevWind is offline
Charter Member
 
Join Date: Apr 2008
Location: Edge of Wilderness Wyoming
Posts: 16,863
Default

Quote:
Originally Posted by Doug Young View Post
Yeah, in Final Cut, the project file gets huge, and editing also seems to make the project grow. I'm not sure what all they're storing, but it helps to periodically clean up with the Delete Generated Files command. I have been working in 4K and I've seen projects grow to 1TB, only to go back down to a few hundred G after cleanup.

I work with an OWC 2TB Accelsior SSD to provide fast editing and rendering, but I can't have more than 2 projects typically at a time on that. Once I'm done and have released a video, I do the clean up, get rid of everything that isn't needed to recreate it, and move everything to a 8TB external - I have a bunch of traditional drives in a thunderbolt cage. Now and then I archive older stuff off to a removable archive drive and store it in the closet. I also sometimes decide I'd never need those projects again and just get rid of them. Ideally, I'd keep everything - I just had AG hit me up for video from interviews I did 10 years ago, but both the need for disk space and all the overhead of keeping track of old files makes me be a bit more blase about it.

I'm not sure if what I have is the same as what you're looking at Kev, since I have the Mac Pro, mine's an internal PCI gizmo, but it seems like the same idea, just via thunderbolt. But I will say that mine's really fast!. I forget exact numbers, but I seem to recall it was 3-4X the read and write speed of the Mac's own SSD, and the difference between it and a regular drive was astonishing. The SoftRaid can be a bit annoying (updates, sometimes paid, occasionally requires reboots..) but mostly it just works.
"Delete Generated Files" is that a command in FCPX ?

Yes basically the same but external ,,,as I told Jim I have two left over 1 TB M.2 NVMe PCIe 1 TB SSD and why I am looking at this 4 slot enclosure (PCIe type) https://eshop.macsales.com/item/OWC/TB3EX4M2SL/
And I will also get 2 more 2-TB Drives for a total of 6TB in the enclosure plus about 1.4 usable now on my iMac and about 700+ GB on my new Macbook Pro M1 Pro / with a 1 TB drive
My plan is to use the new MBP for FCPX editing assuming it will be faster than my intel i7 iMac

What I am uncertain about is how to use the RAID configuration, but will call OWC on Monday
What I don't know is if the computer will see the 4 drives in the enclosure as 4 different drives, or as on big one ,
On my old Mac Pro the two SSD drives (even though on one Dual PCIe drive adaptor sled) appeared as two different drives.
__________________
Enjoy the Journey.... Kev...

KevWind at Soundcloud

KevWind at YouYube
https://www.youtube.com/user/KevWind1/videos

System :
Carbon interface , PT Ultimate 2021.12 -Mid 2020 iMac 27" 3.8GHz 8-core i7 10th Gen ,,128GB 2666MHz DDR4 RAM,,2TB SSD storage,Radeon Pro 5700 XT16GB Catalina 10.15.7

Last edited by KevWind; 01-22-2022 at 01:41 PM.
Reply With Quote
  #12  
Old 01-22-2022, 01:37 PM
endpin endpin is offline
Registered User
 
Join Date: Oct 2008
Location: Fort Worth, Texas
Posts: 411
Default

I have been using the Samsung T7 External SSDs for FCPX projects connected to a MacBookAir M1 via USB-C to USB-C connections and ultimately plan to junk all my big disk arrays as video storage.

I don't use the internal SSD for ANYTHING except the applications and system stuff.
I delete the generated render files to archive the projects.


Last edited by endpin; 01-22-2022 at 03:55 PM.
Reply With Quote
  #13  
Old 01-22-2022, 01:57 PM
Doug Young's Avatar
Doug Young Doug Young is offline
AGF Sponsor
 
Join Date: Apr 2005
Location: Mountain View, CA
Posts: 8,879
Default

Quote:
Originally Posted by KevWind View Post
"Delete Generated Files" is that a command in FCPX ?
Yes:

Screen Shot 2022-01-22 at 11.53.22 AM.jpg

This discussion reminded me I hadn't done this stuff for the last video I posted, so I just did the cleanup. Project was 781G, after removing a few clips I didn't use and issuing the "delete" command, it's 125G, now moved to my archive.

Quote:
What I am uncertain about is how to use the RAID configuration, but will call OWC on Monday
What I don't know is if the computer will see the 4 drives in the enclosure as 4 different drives, or as on big one ,
On my old Mac Pro the two SSD drives (even though on one Dual PCIe drive adaptor sled) appeared as two different drives.

For speed, I'm using Raid 0, which is what the OWC enclosure expects - to meet their claimed speed specs. So it appears as one drive, but data can be written to multiple SSDs in parallel. You could use other raid configs if you want redundancy.

It was pretty easy to install and setup if I recall.
Reply With Quote
  #14  
Old 01-22-2022, 02:07 PM
KevWind's Avatar
KevWind KevWind is offline
Charter Member
 
Join Date: Apr 2008
Location: Edge of Wilderness Wyoming
Posts: 16,863
Default

Thanks all very informative
__________________
Enjoy the Journey.... Kev...

KevWind at Soundcloud

KevWind at YouYube
https://www.youtube.com/user/KevWind1/videos

System :
Carbon interface , PT Ultimate 2021.12 -Mid 2020 iMac 27" 3.8GHz 8-core i7 10th Gen ,,128GB 2666MHz DDR4 RAM,,2TB SSD storage,Radeon Pro 5700 XT16GB Catalina 10.15.7
Reply With Quote
  #15  
Old 01-22-2022, 02:28 PM
jim1960 jim1960 is offline
Charter Member
 
Join Date: Sep 2015
Posts: 4,570
Default

Quote:
Originally Posted by KevWind View Post
Thanks for the heads up .Just checked looks like that one is for SATA drives
Since I already have two M.2 NVMe PCIe 1 TB SSD drives from my old Mac Pro. I am looking for a M.2 NVMe SSD enclosure and why I was looking at the one I mentioned in the op with 4 slots so I will get 2 more 2 TB drives
I was just making the point that these things work well with no noticeable loss in performance. Grab that OWC enclosure. They're a good company and the stuff they make is well regarded.
__________________
Jim
2017 Circle Strings 00 bastogne walnut/sinker redwood
2015 Circle Strings Parlor shedua/western red cedar
2009 Bamburg JSB Signature Baritone macassar ebony/carpathian spruce
2004 Taylor XXX-RS indian rosewood/sitka spruce
1988 Martin D-16 mahogany/sitka spruce

along with some electrics, zouks, dulcimers, and banjos.

SoundCloud link
Spotify
YouTube
Reply With Quote
Reply

  The Acoustic Guitar Forum > General Acoustic Guitar and Amplification Discussion > RECORD

Thread Tools





All times are GMT -6. The time now is 01:50 PM.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2022, vBulletin Solutions Inc.
Copyright ©2000 - 2022, The Acoustic Guitar Forum
vB Ad Management by =RedTyger=