All, I need some advice please how to build a flow to do this:
I have a Calgary database which I update daily using an Alteryx flow. This flow is the most basic imaginable. An Input tool runs an SQL query against a remote Teradata database. The results are fed to a Calgary loader which re-builds the Calgary database. The data is around 46 million rows, so it takes a while to run, and pulls a lot of data over a transatlantic link. There is no point running this flow more than once per day, the data only changes overnight.
I have a second, more complex, flow which uses the Calgary database as one of its data sources, by using the Calgary Input tool. This flow gets run several times per day.
I would like to make life simpler by combining these two flows. However, I want to avoid re-running the Teradata query each time the combined flow is run. So on the first run of the day, I want the flow to run the Teradata query, and use the results, but also update the Calgary database. On second and subsequent runs, I want to avoid running the Teradata query and use the data from the Calgary database that was updated on the first run.
So in effect I am using the Calgary database as a local cache for the data from the remote Teradata system.
I have been trying to build something using the Directory tool to find the last modified date on the Calgary database file. I can compare this to today's date to test if the Calgary database has been updated yet today. I can put parts of my flow into containers and enable/disable them with the Action tool. I guess I may need to use the Detour and Detour End tools. But I can't figure out how to glue this all together.
I have no ability to schedule flows to run, or make dependancies between flows. All flows are run by hand and I am looking for a solution usingthe existing tools in Designer. I cannot install extra packages because of corporate security restrictions.
Any suggestions please?