FileData API¶

FileData Overview¶

The FileData store on the device cloud provides a hierarchical mechanism for temporarily storing information in files sent from devices. With the APIs provided by the device cloud, it is possible to use the FileData store in a number of different ways to implement various use cases.

There are two main ways of thinking about the FileData store:

As hierarchical data store for temporarily storing data pushes from devices
As a message queue

The usage for both scenarios is similar. In the first case, it is likely that a web service will poll the FileData store for new files matching some criterion on a periodic basis. If using the FileData store as a queue, one will likely want to setup monitors on FileData paths matching certain criterion. The set of files matching some condition can then be thought of as a channel.

This library seeks to make using the device cloud for both of these use cases simple and robust.

Navigating the FileData Store¶

There are a few main ways to navigate the FileData store.

Method 1: Iterate through the file tree by getting the filedata objects associated with paths or other conditions:

condtion = (fd_path == '~/')
for filedata in dc.filedata.get_filedata(condition):
    print filedata

The FileDataAPI.get_filedata() method will return a generator over the set of FileData directories and files matching the provided conditions. The two types availble (which are both instances of FileDataObject are FileDataDirectory and FileDataFile.

Other methods of navigating the store are built upon the functionality provided by FileDataAPI.get_filedata().

Method 2: The FileDataAPI.walk() method provides a convenient way to iterate over all files and directories starting with some root within the filedata store. This method mimicks the interface and behavior provided by the python standard library os.walk() which is used to iterate over one’s local filesystem.

Here is a basic example:

for dirname, directories, files in dc.filedata.walk():
    for file in files:
       print file.get_full_path()

It is also possible to perform a walk from a given FileDataDirectory as follows:

d = dc.filedata.get_filedata("~/mydevice/mydir")
for dirname, directories, files in d.walk():
   for file in files:
      print file.get_full_path()

Reading Files and File Metadata¶

After finding a FileDataFile using either FileDataAPI.get_filedata() or FileDataAPI.walk(), access to file contents and metadata is provided by a set of straightforward accessors. Many of these are also available on FileDataDirectory objects as well. Here we show the basic methods:

# f is a FileDataFile object
f.get_data()  # the data contained in the file
f.get_last_modified_date()  # returns datetime object
f.get_type()  # returns 'file'.  'directory' for directories
f.get_content_type()  # returns content type (e.g. 'application/xml')
f.get_customer_id()  # returns customer id (string)
f.get_created_date()  # returns datetime object
f.get_name()  # returns name of file (e.g. test.txt)
f.get_path()  # returns path leading up to the file (e.g. /a/b/c)
f.get_full_path()  # returns full path including name (e.g. /a/b/c/test.txt)
f.get_size()  # returns the size of the file in bytes as int

Creating a Directory¶

TODO: This functionality is not current active.

There are three ways to create a new directory:

Create full path with FileDataAPI.make_dirs(). This will recursively create the full path specified.
Write a file to a directory that does not yet exist. If the path is valid, all directories should be created recursively.
By calling FileDataDirectory.add_subdirectory()

Different methods may suit your needs depending on your use cases.

Writing a File¶

The following methods may be used to write a file:

Use FileDataAPI.write_file(). This requires a full path to be specified.
Use FileDataDirectory.write_file(). As you already have a directory path in that case, you do not need to specify the path leading to the file.

Here’s a basic example:

dc.filedata.write_file(
    path="~/test",
    name="test.json",
    content_type="application/json",
    archive=False
)

Viewing File History¶

Note

Support for programmatically getting history for files is not supported at this time.

API Documentation¶

The filedata module provides function for reading, writing, and deleting “files” from the device cloud FileData store.

Provide access to the device cloud filedata API

class devicecloud.filedata.FileDataAPI(conn)¶

Encapsulate data and logic required to interact with the device cloud file data store

get_filedata(condition=None, page_size=1000)¶

Return a generator over all results matching the provided condition

Parameters:

condition (Expression or None) – An Expression which defines the condition which must be matched on the filedata that will be retrieved from file data store. If a condition is unspecified, the following condition will be used fd_path == '~/'. This condition will match all file data in this accounts “home” directory (a sensible root).
page_size (int) – The number of results to fetch in a single page. Regardless of the size specified, get_filedata() will continue to fetch pages and yield results until all items have been fetched.

Returns:

Generator yielding FileDataObject instances matching the provided conditions.

write_file(path, name, data, content_type=None, archive=False, raw=False)¶

Write a file to the file data store at the given path

Parameters:

path (str) – The path (directory) into which the file should be written.
name (str) – The name of the file to be written.
data (str (Python2) or bytes (Python3)) – The binary data that should be written into the file.
content_type (str or None) – The content type for the data being written to the file. May be left unspecified.
archive (bool) – If true, history will be retained for various revisions of this file. If this is not required, leave as false.
raw (bool) – If true, skip the FileData XML headers (necessary for binary files)

delete_file(path)¶

Delete a file or directory from the filedata store

This method removes a file or directory (recursively) from the filedata store.

Parameters:	path – The path of the file or directory to remove from the file data store.

walk(root='~/')¶

Emulation of os.walk behavior against the device cloud filedata store

This method will yield tuples in the form (dirpath, FileDataDirectory's, FileData's) recursively in pre-order (depth first from top down).

Parameters:	root (str) – The root path from which the search should commence. By default, this is the root directory for this device cloud account (~).
Returns:	Generator yielding 3-tuples of dirpath, directories, and files
Return type:	3-tuple in form (dirpath, list of `FileDataDirectory`, list of `FileDataFile`)

class devicecloud.filedata.FileDataObject(fdapi, json_data)¶

Encapsulate state and logic surrounding a “filedata” element

delete()¶: Delete this file or directory

get_data()¶

Get the data associated with this filedata object

Returns:	Data associated with this object or None if none exists
Return type:	str (Python2)/bytes (Python3) or None

get_type()¶: Get the type (file/directory) of this object

get_last_modified_date()¶: Get the last modified datetime of this object

get_content_type()¶: Get the content type of this object (or None)

get_customer_id()¶: Get the customer ID associated with this object

get_created_date()¶: Get the datetime this object was created

get_name()¶: Get the name of this object

get_path()¶: Get the path of this object

get_full_path()¶: Get the full path (path and name) of this object

get_size()¶: Get this size of this object (will be 0 for directories)

class devicecloud.filedata.FileDataDirectory(fdapi, data)¶

Provide access to a directory and its metadata in the filedata store

walk()¶

Walk the directories and files rooted with this directory

This method will yield tuples in the form (dirpath, FileDataDirectory's, FileData's) recursively in pre-order (depth first from top down).

Returns:	Generator yielding 3-tuples of dirpath, directories, and files
Return type:	3-tuple in form (dirpath, list of `FileDataDirectory`, list of `FileDataFile`)

write_file(*args, **kwargs)¶

Write a file into this directory

This method takes the same arguments as FileDataAPI.write_file() with the exception of the path argument which is not needed here.

class devicecloud.filedata.FileDataFile(fdapi, json_data)¶: Provide access to a file and its metadata in the filedata store