HCP Upload Daemon (0.2.5)

hcpupd is a daemon that automatically uploads files to HCP.

It’s using the Linux inotify kernel subsystem to monitor a folder (aka watchdir), sending every file that is moved or written to it to HCP, immediately. Folders created in the watchdir will be watched as well.

Features:

  • Two different upload modes:

    1. Transfer the folder structure created in watchdir to HCP as it is
      • human-readable
      • performance-wise not the best possible solution
      • not tolerant against filename duplicates (except if the Namespace has Versioning enabled)

    or

    1. Obfuscate the folder structure on HCP by creating an UUID per file, used as filename as well as to construct a path to it
      • best possible ingest performance
      • 64k folders created at max. (to be precise: 256**2 + 256)
      • an unlimited number of hcpupd’s can write into the same Namespace without the risk of filename conflicts
      • supports search for the original filename by adding an annotation to each file, which can be used by HCP’s Metadata Query Engine, Hitachi Content Intelligence or any other indexer that is able to crawl a HCP Namespace
  • Optionally:

    • on start-up, upload existing files already stored in the watchdir
    • auto-delete files from watchdir after successful upload
    • auto-delete folders after the last file has been uploaded
  • Made to run as a Linux daemon (but can run in an interactive session as well)

  • Extended logging available, incl. log rotation

Tip

Please note the Limitations.