Process Task Base

For more information on how to use this class, see here.

You probably want to process some data, get a result and cache it. There is another class TaskWithState that helps you do this, and so may be more useful for you. For more information on this helper class, see here.

class libfjordweb.process.base.ProcessDataTask(supplied_data: SuppliedData, supplied_data_files: List[SuppliedDataFile])

Base class for a task to apply to some user supplied data.

get_context()

Load all relevant data for this task and return it in a dict. This will be passed to the explore template when the user looks at some data.

This is called on a user request on the web, so no long processing work should be done here. Instead, do long processing work in process() and cache the results.

is_processing_applicable() → bool

Should return True if this task may ever need to do anything far this supplied data.

eg. A task to convert a spreadsheet to JSON will never be applicable

if JSON is uploaded in the first place.

eg. A task to check the data against JSON Schema will always be applicable.

This method is called on a user request on the web; so it must not take a long time to finish.

is_processing_needed() → bool

Should return True if this task needs to do any processing.

This method is called on a user request on the web; so it must not take a long time to finish.

process(process_data: dict) → dict

Called to process data.

This takes in a dict, process_data, and it should always return this dict. This dict starts as an empty dict at the start of the pipeline. Tasks at the start of the pipeline may add useful information that tasks at the end of the pipeline can use.

Note this is always called when processing data even if is_processing_needed returns False. You should do your own checks to make sure you are not doing unneeded work. This is so you can still add relevant info to process_data dict.

But it’s not called if is_processing_applicable() is false.

This is only called by the background worker, so it can take as long as it needs to do it’s work.