Thursday, January 5, 2023

Understanding Where to Host Your Autodesk Files For EDITING...

 I really wanted to title this, DON'T DO THIS...since I wind up spending an inordinate amount of time trying to explain this to the general design and construction public. Be aware that I'm presenting this from a layman's point of view, to help make it easier to understand why these issues occur.

The internet continues to change the way we work, but alas, we do not. We continue to try to do the same thing we did twenty years ago, thinking that everything happens in a static place where nothing changes. It's almost like trying to argue with the 30 year AutoCAD user, who bought a "perpetual license" and really thinks it's going to work just peachy for the next thirty years. Do yourself a favor - take the time to understand how the technology has changed, and avoid issues with file corruption, lost data and more.

Here's the real part of the story. While there are a large number of resources online that allow you to share files - from Facebook, Instagram and other sites where we share our pictures of our dogs, kids and videos of us trying to look cool while dancing in a street and not get run over by another car - to legitimate file sharing services such as OneDrive, Dropbox, WeTransfer, Google Drive, Autodesk Drive, iCloud, Box, Sharepoint...you get the idea. It's important to understand the differences of each of these services and the intent for how they are supposed to be used.

All of these offer a service of providing a backup space as well as the ability to securely share and transfer data and content between locations. But here's the rub - NONE of them are supported as a platform for EDITING Autodesk files. 

Let's get down in the weeds. In order for Revit, AutCAD, Civil 3D and other file formats to be managed correctly - when it comes to file locking, access, permissions and more - the files must live somewhere within a supporting operating system environment. AutoCAD hasn't locked its own files since 2007, and with Revit central models allowing multiple users in a single file, the permissions and access for the temporary files that constantly communicate with the central model are critical in order to prevent file corruption and lost data.

To do this, Autodesk has been publicly addressing this through their system requirements - for example, here is AutoCAD:

https://knowledge.autodesk.com/support/autocad/troubleshooting/caas/sfdcarticles/sfdcarticles/System-requirements-for-AutoCAD.html

Each item listed shows the support operating system, from Windows, to Mac and Linux environments. Revit also has similar system requirements:

https://knowledge.autodesk.com/support/revit/learn-explore/caas/sfdcarticles/sfdcarticles/System-requirements-for-Autodesk-Revit-products.html

Here's the key - communications between the location and file must be in an environment that is supported by SMB protocol - in short, this stands for Server Message Block. The versions of SMB, also known as dialects, align with the versions of Windows. With SMB 1.0 came support for the original DOS-based systems. This originated in 1984, and with each new release of the Windows OS ecosystem, a newer version of SMB was released. For example, Windows Vista and Server 2008 included SMB 2.0, while version 3.0 was associated with Windows 8 in 2012. SMB 3.1.1 was released with Windows 10 in 2015, and has been the baseline for both versions 10 and 11. 

If a program such as AutoCAD or Revit is supported on one of these operating systems, then it's supported by the corresponding SMB dialect.

Wait...Explain what SMB does!

So what does SMB do? TechTarget (techtarget.com) includes a great description about the definition of SMB here. The key description for how it works is:

"The SMB protocol enables applications and their users to access files on remote servers, as well as connect to other resources, including printers, mailslots and named pipes. SMB provides client applications with a secure and controlled method for opening, reading, moving, creating and updating files on remote servers. The protocol can also communicate with server programs configured to receive SMB client requests.

Known as a response-request protocol, the SMB protocol is one of the most common methods used for network communications. In this model, the client sends an SMB request to the server to initiate the connection. When the server receives the request, it replies by sending an SMB response back to the client, establishing the communication channel necessary for a two-way conversation."

Autodesk specifically references it's requirement to operate in this link:

https://knowledge.autodesk.com/support/autocad/learn-explore/caas/sfdcarticles/sfdcarticles/Support-for-non-Microsoft-networks.html

So Where Can I Work?

So here's the summary - NONE of these examples of file sharing services are properly defined/managed to serve as locations where files can be EDITED (that's the key phrase)

  • Sharepoint/OneDrive
  • Google Drive
  • Autodesk Drive (Yea, it's an Autodesk product, but not the right one for editing...keep reading)
  • DropBox
  • Box

Let's provide a little context. A user named Bob places an AutoCAD Drawing (.DWG) in a Sharepoint file location, such as \\myoffice.sharepoint.com\sites\myproject\files. He proceeds to use the Open command to browse to the Onedrive link to the file at this site through File Explorer, and opens the drawing in AutoCAD.

Any user that is a member or guest of the site, if they have access permissions, can see and open this file. But what they are opening is not this file, but a local, cached copy of the file where Sharepoint is the host location. Since OneDrive is basically the "desktop" version of Sharepoint, it will also be working on a cached copy of the file. The confusion comes from users that were using a traditional networked server that users accessed via a mapped drive or UNC path before moving to the cloud. In those cases you were opening the actual file.

The file on the site does not know that it needs to be locked - so on another computer a user named Pat also opens the file - but again, not the original but a cached copy. Pat's fast, and makes a few changes, and saves the file while Bob still is working on his file, because...well, he's slow. Bob's done and saves his changes - which don't include Pat's changes since they were not in the cached copy he downloaded. Now the files are out of sync, and Bob just wiped out Pat's work, and now everyone's mad.

What about using a DFS configured network?

Similar issues will occur when working in a DFS (Distributed File Service) Environment. In this case, the first "available" copy of a file is what is opened, as the files are "replicated" on other related systems within the network. Again, TechTarget has a great explanation of this type of system and how it works here.

Autodesk addresses DFS issues here:

https://knowledge.autodesk.com/support/autocad/learn-explore/caas/sfdcarticles/sfdcarticles/Distributed-File-System-DFS-support.html

How about using NAS/SAN appliances?

Network Attached Storage (NAS) devices are very similar to the external hard drives and USB drives we use now as a primary backup for many user's home or small office systems - and in some cases, can be used in larger enterprise solutions. They contain a built in hard drive that can allow users to have a shared location to store data, but can experience performance issues if too many users try to access the data simultaneously. Since their primary purpose is to provide basic data storage and backup, they typically cannot handle the constant traffic between Revit local and central files - and the more users accessing the files, the slower the system. TechTarget's full description can be found here.

Storage Area Networks (SAN) appliances are typically associated with high speed networks, and have improved performance over the traditional NAS device. TechTarget's description of a SAN explains how SAN is differentiated from a NAS appliance/system:

"The SAN interconnects all the disks into a dedicated storage area network. That dedicated network exists separate and apart from the common LAN. This approach enables any of the servers connected to the SAN to access any of the disks attached to the SAN, effectively treating storage as a single collective resource. None of the SAN storage data needs to pass across the LAN -- mitigating LAN bandwidth needs and preserving LAN performance."

So while skipping a LAN connection and behaving similar to a traditional WAN, it can provide users access from multiple locations without the pass-through throttling that occurs on a traditional WAN. There also some SAN configurations that do support SMB protocols, but it would be up to the vendor to validate this as well as support any issues that arise. Techtalk's full description of SAN systems can be found here.

The Autodesk article on the use of NAS and SAN appliances can be found here:

https://knowledge.autodesk.com/support/revit/troubleshooting/caas/sfdcarticles/sfdcarticles/Revit-support-with-network-file-servers-or-cloud-storage-solutions.html

What does this leave us:

- Windows 10/11 OS on your local hard drive - with the current supported versions of 2021-2024, all of these products are supported on Windows 10 and 11. Be aware that none of them will work on Windows 8.1 - and you shouldn't be using this version either, as it's no longer being updated by Microsoft, including security fixes. And yes, a MAC OS is supported, primarily in a Windows emulation environment.

- Window Server 2012/2016/2019 - check the specific version system requirements for Revit Server from the previous links above. Depending on the Autodesk software version, the server OS must be one of these versions. 

- Autodesk Docs/BIM 360 - Purpose-built to provide a cloud storage AND editing solution, this is best location to edit Autodesk documents. Newer releases such as AutoCAD and Civil 3D 2024 utilize updated Sheet Set Manager tools that either can bypass or work correctly with the Autodesk Desktop Connector application that normally manages the caching and file permissions between a local hard drive and the AWS-based "server" location. The rest of the time, it's the Desktop Connector tool that "manages" file access.

Be aware that programs like Revit are in constant contact with central files, regardless of where they are stored. Simple things, such as being aware that Bob has started editing an element (that's not in a checked out workset - better known as "borrowing"). If Pat tries to edit the same object - or edit anything in an MEP system or hosted object that is associated with and impacted by changes to the object - then they know right away, since "ownership" of objects is communicated via the temporary files associated with the project.

There's a couple of caveats here as well - if you are a single user, you can get away with using Autodesk Docs to edit drawings and work on non-central Revit models. If you are collaborating in Civil 3D, Plant 3D and Revit, and working with Revit central models, you need to have the BIM Collaborate Pro entitlement. This is over and above your software license subscription, but can provide an ROI very quickly when you have more than one user on a project.

I know we're going to have some folks disagree with this. You might hear about issues with Desktop Connector - yes, there have been some that Autodesk has struggled to fix, but at the same time, the application has improved dramatically over the last few years. For example, it works great with Revit, but you also need to make sure all users on the projects have the same build of Desktop Connector as well as the same build (not just version) of Revit as well. We're using it on hundreds of projects and rarely encounter a problem - but these only occur when users try to add links to content that's not in the project hub, or connected to one of these other sites. 

What about everything else?

That's easy - you can take the risk to use any of these platforms  (i.e. Panzura, Egnyte, etc. and the other file sharing locations listed above as examples). But when you lose changes...don't have access to the files...get file corruption...you're going to have to reach out to these providers to figure out the solution. The cost, time and effort invested in a platform or service that isn't supported by Autodesk is wasted when you don't start by taking a full look at how the design platforms developed today work, and how they improve the design and operational environment.

You also have Bentley's Projectwise platform but be aware that vertical applications like Revit, AutoCAD Plant 3D and Civil 3D also experience issues here as well. Autodesk Vault is also still available as a "behind the firewall" solution, but again, Revit does not play well in this environment. Both of these platforms utilize effective single file check-in/check-out features that prevent files from being edited by multiple users, but lose effectiveness when external files are required to be accessible at the same time.

In summary, do your homework BEFORE implementing your primary work platform, and understand the limitations/benefits for each first. And a great thanks to Techtarget for the easy to read and understand descriptions of each - hopefully they will help you as much as they helped an "old dog" like me.

4-12-2023 UPDATE - with the release of 2024 products, the Autodesk support versions are now back to 2021 - but you can still use older versions back to 2018.3 to work on existing Autodesk cloud projects as long as you have a continually maintained subscription. Get these older projects upgraded to a supported version as fast as possible!



1 comment:

Rob said...

Nice job David!!!