Note: This applies to DocuShare Flex 2.11 and further.
Best practices for configuring CIM and sending files to the watch folder:
Note: The example listed below uses csv. However, the process is the same for xml files.
- Create a Report and Error folder somewhere in Flex before configuring the Content Intake Queue. These additions to CIM settings in Flex 2.11 will be used to store completed uploads (report) and metadata files that have failed a screening for correctness. Set permissions appropriately on these folders.
- Set an alert on the Error folder to notify an Coordinator user when an upload fails. The file in the Error folder is the metadata file that failed, and the Summary field will contain the reason(s) for that failure. Fix the file and re-upload to the watch folder location to start the CIM anew.
- Optionally set an alert on the Report folder if a Coordinator wants notification of successful uploads.
- Create a Content Rule or process to delete Report and Error files that are no longer needed.
- Upload the transform file in the watch folder before configuring the Content Intake Queue.
- Create and configure the Content Intake Queue.
- Copy the files (PDFs) to the watch folder.
Copy the csv metadata last. Here is an example of a best practice when copying the .csv to the watch folder:
- Copy csv to the watch folder by temporarily giving it a non-csv extension - eg. metadata.csv.tmp. This prevents the CIM from processing the csv while it is still in transit to the watch folder.
- Rename metadata.csv.tmp to metadata.csv to start triggering the ingestion.
Other things to consider:
- Files uploaded in different batches should not have same file name to avoid overwriting in the watch folder. Always have unique names for files you ingest so that you know which batch they are from. A good strategy would be to append the batch name to the files, also ensuring that the batch name is unique. . Alternatively, add a time stamp to the metadata file name to ensure uniqueness.
- Keep batches to a "manageable" size - eg. 500-1000 files per csv (batch). The bigger the batch, the more things to debug if something goes wrong.
- Inspect the batch status file (*.status.xml) in the watch folder .status directory to spot check ingestions.
- Look at the .error to directory to find more info about ingestion errors.
- Clean the contents of the .status and .error directories from time to time to free up disk space. Note: This cleanup would be the customer's responsibility to do this as they will if the files here need to be kept. (example: auditing purposes).
Keywords:
CIM Best Practices
Content Intake Queue
Transformation
Transform
CIM Config