Data Workflow

Home / News / Data Workflow

Data Workflow

Data workflow in the field is an important part of the filmmaking process. Its often overlooked as the techie thing and the responsibility pushed off to “I’ll get around to it”. The thing is, hard drives FAIL. Life, death, taxes, and failing hard drives…it is a fact of life. Sadly data workflow and backup considerations typically happen AFTER your hard drive fails and you go into panic mode. Once it happens to you, you’ll never go back. On “The Southwest” data management is crucial. We are often out in the middle of nowhere, down bumpy roads far from home. We are at a higher risk of losing a hard drive which makes our data workflow very important. Our shoots cost thousands of dollars and years of planning. Most of them can’t be recreated, so getting the footage and keeping it safe is key.

Speed, Reliability, Safe Guards

Speed – When shooting 6k, speed is paramount. We can’t spend a down day just copying data. Thunderbolt 2 not only offers high speed but ease of use, Its simply plug and play.

thunderbolt-connector-100054566-large

Reliability – Our AKiTiO drives offer dual TB 2 ports for daisy chaining which helps reduce cables and therefore reliability. The Thunder 2 Quad Mini is what we use in the field and its well build to withstand being bounced around on rough roads.

akitio-thunder2-quad-mini-rear-highres

Safe Guards – Making sure your data does not become corrupted or destroyed is key. Create safe guards to protect your information from the point its being shot all the way thru the end of production. Remember to always have MULTIPLE copies of your data in MULTIPLE locations. If you have all of your data at your house and it goes up in flames, your screwed.

So here is how we manage footage from the field back to the office

We use several types of cameras including Red Epic Dragon, Nikon DSLR’s, & Vision Research Phantom Flex 4k. After a shoot the media card is connected to our Macbook Pro via its appropriate card reader via USB3 or Thunderbolt.

A026_C029_0808QO

 

For Nikon astro-timelapse footage, we use the mac app called CopyVerified which makes verified checksum copies of your data. It insures that the data matches from its source to its destination.

copyverified

For Red footage, we use the app called DoubleData. The app is the same as CopyVerified but designed for Red and Arri file types.

Unknown-3

The files are copied to our AKiTiO Thunder 2 Quad Mini. This is a 4 bay enclosure that holds 2.5″ inch drives. We have 1TB HGST 7200RPM drives. Ideally we would have SSD drives but at present time they are still a bit expensive.

A026_C015_0808GR

 

The drive array was setup using an app called SoftRaid, which creates a RAID-5 array for data safety.

The AKiTiO Thunder 2 Quad Mini is housed in its own HPRC case with the cables to ensure while driving the hard drive is much more safe. Once copied the data is now safe for in the field focus and quality checks. DSLR timelapse’s are rendered in After Effects CC and Red footage is checked in Redcine-x.

A026_C014_0808WW

Once we return to the office the footage is backed up for longer term storage. Again using CopyVerified all the folders are copied from the AKiTiO Thunder 2 Quad Mini to two AKiTiO Thunder 2 Quads. These larger units use 3.5 inch drives in a RAID-5 configuration. The Data is copied to two physical units. One unit stays in the office and another is sent to an off-site location for safety. If the house were to burst in flames we would still have all the footage at the other location. Ideally we would also make another copy to the cloud but the internet here is too slow to copy several terabytes of data.

workflowchart

Conclusion

This may all sound a bit extreme, but for a film its what we have to do to protect the footage. Remember that this also applies to your personal data. Don’t put all your eggs in one basket. Again make sure you have multiple copies in multiple locations. Ask a friend to join together and give each other hard drive copies so that you both are backed up. Cloud storage is inexpensive and usually unlimited data.

AKiTiO is the official storage solution of “The Southwest”. Their support is excellent and they are based out of southern California.

Hardware

Apple Macbook Pro

apple_15inch_macbook_pro23ghz_core_i7_mid_2012_1220660_g1

AKiTiO Thunder 2 Quad

akitio-thunder2-quad-angle-highres

AKiTiO Thunder 2 Quad Mini

akitio-thunder2-quad-mini-angle-highresDouble Data

Unknown-3

SoftRAID

Unknown-2

Copy Verified

copyverified